Inference Platform & Optimization: Build and optimize enterprise LLM serving platforms (e.g., vLLM, TensorRT-LLM) using techniques like PagedAttention, continuous batching, and quantization (AWQ/FP8) for high throughput and low latency.
GPU Pooling & AI Infra: Design GPU pooling, virtualization, and scheduling solutions on Kubernetes to maximize hardware utilization. Manage distributed training clusters and high-performance networking (RDMA/NCCL).
Model Deployment & MLOps: Streamline the CI/CD pipeline for AI models. Implement automated benchmarking, zero-downtime deployment, and comprehensive observability (TTFT, TPS, GPU metrics).
GPU Pooling & AI Infra: Design GPU pooling, virtualization, and scheduling solutions on Kubernetes to maximize hardware utilization. Manage distributed training clusters and high-performance networking (RDMA/NCCL).
Model Deployment & MLOps: Streamline the CI/CD pipeline for AI models. Implement automated benchmarking, zero-downtime deployment, and comprehensive observability (TTFT, TPS, GPU metrics).
Qualifications
Bachelor’s, Master’s, or Ph.D. in Computer Science, Computer Engineering, or a related field.
Experience writing custom CUDA kernels or using Triton for operator fusion.
3+ years of experience in Backend Systems, Distributed Systems, or AI Infrastructure/MLOps, with at least 1-2 years specifically focused on LLM serving, GPU optimization, or ML Systems.
Deep understanding of Linux internals, networking, and distributed systems architecture.
Hands-on experience with container orchestration (Kubernetes, Docker) and building custom K8s operators or controllers.
Deep familiarity with LLM inference engines (vLLM, TensorRT-LLM, TGI) and understanding of their underlying architectural designs.
Qualification wording
Bachelor’s, Master’s, or Ph.D. in Computer Science, Computer Engineering, or a related field.
Experience writing custom CUDA kernels or using Triton for operator fusion.
3+ years of experience in Backend Systems, Distributed Systems, or AI Infrastructure/MLOps, with at least 1-2 years specifically focused on LLM serving, GPU optimization, or ML Systems.
Deep understanding of Linux internals, networking, and distributed systems architecture.
Hands-on experience with container orchestration (Kubernetes, Docker) and building custom K8s operators or controllers.
Deep familiarity with LLM inference engines (vLLM, TensorRT-LLM, TGI) and understanding of their underlying architectural designs. Or
Education & alternatives
1. Education & Experience:
- Bachelor’s, Master’s, or Ph.D. in Computer Science, Computer Engineering, or a related field.
- 3+ years of experience in Backend Systems, Distributed Systems, or AI Infrastructure/MLOps, with at least 1-2 years specifically focused on LLM serving, GPU optimization, or ML Systems.
Open 4 monthsWe first saw this posting 143 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
Status in our records
Live
First seen by us
Jul 1, 2026143 days ago
Recorded sightings
276
Last seen by us
Nov 21, 2026
Employer says posted
Jun 3, 2026
Jul 1, 2026today
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
How this job compares
Time since we first saw it
Day 143
776
Up longer than most. Similar jobs usually come down between day 7 and day 76.
Well past JPMorgan Chase’s usual: its jobs come down by about day 17.
From 6,398 similar postings that came down, and 1,118 of JPMorgan Chase’s. Coming down is not always a hire.
JPMorgan Chase is hiring
371 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
Docker
Kubernetes
PyTorch
Python
Java
Where the posting names them
- Deep understanding of Linux internals, networking, and distributed systems architecture.
- Hands-on experience with container orchestration (Kubernetes, Docker) and building custom K8s operators or controllers.
3. AI Infrastructure & Optimization Skills:
- Inference Platform & Optimization: Build and optimize enterprise LLM serving platforms (e.g., vLLM, TensorRT-LLM) using techniques like PagedAttention, continuous batching, and quantization (AWQ/FP8) for high throughput and low latency.
- GPU Pooling & AI Infra: Design GPU pooling, virtualization, and scheduling solutions on Kubernetes to maximize hardware utilization. Manage distributed training clusters and high-performance networking (RDMA/NCCL).
- Model Deployment & MLOps: Streamline the CI/CD pipeline for AI models. Implement automated benchmarking, zero-downtime deployment, and comprehensive observability (TTFT, TPS, GPU metrics).
Preferred
- Contributions to open-source AI Infra projects (e.g., vLLM, Ray, PyTorch).
- Experience writing custom CUDA kernels or using Triton for operator fusion.
2. Core Engineering & Systems Skills:
- Expert-level proficiency in Python and strong proficiency in Java (essential for inference engines and CUDA integration).
- Deep understanding of Linux internals, networking, and distributed systems architecture.
Inference Platform & Optimization: Build and optimize enterprise LLM serving platforms (e.g., vLLM, TensorRT-LLM) using techniques like PagedAttention, continuous batching, and quantization (AWQ/FP8) for high throughput and low latency.
GPU Pooling & AI Infra: Design GPU pooling, virtualization, and scheduling solutions on Kubernetes to maximize hardware utilization. Manage distributed training clusters and high-performance networking (RDMA/NCCL).
Model Deployment & MLOps: Streamline the CI/CD pipeline for AI models. Implement automated benchmarking, zero-downtime deployment, and comprehensive observability (TTFT, TPS, GPU metrics).
Qualifications
1. Education & Experience:
Bachelor’s, Master’s, or Ph.D. in Computer Science, Computer Engineering, or a related field.
3+ years of experience in Backend Systems, Distributed Systems, or AI Infrastructure/MLOps, with at least 1-2 years specifically focused on LLM serving, GPU optimization, or ML Systems.
2. Core Engineering & Systems Skills:
Expert-level proficiency in Python and strong proficiency in Java (essential for inference engines and CUDA integration).
Deep understanding of Linux internals, networking, and distributed systems architecture.
Hands-on experience with container orchestration (Kubernetes, Docker) and building custom K8s operators or controllers.
3. AI Infrastructure & Optimization Skills:
Deep familiarity with LLM inference engines (vLLM, TensorRT-LLM, TGI) and understanding of their underlying architectural designs. Or
Solid understanding of GPU architecture (NVIDIA Ampere/Hopper), CUDA programming, and GPU memory management. Or
Experience with distributed training frameworks (DeepSpeed, Megatron-LM, Ray) and high-performance networking (RDMA, RoCE, InfiniBand).
4. Mindset & Soft Skills:
A "hacker" mindset with a passion for squeezing every drop of performance out of hardware.
Ability to collaborate effectively with AI Researchers (to understand their models) and Backend Engineers (to integrate AI into business systems).
Preferred
Contributions to open-source AI Infra projects (e.g., vLLM, Ray, PyTorch).
Experience writing custom CUDA kernels or using Triton for operator fusion.
Financial industry (Asset Management/Quant) experience is a plus.
Language: Professional working proficiency in English to collaborate with global teams.
About JPMorgan Chase
People
About 318,512 in 2025.Annual report (SEC Form 10-K)
Company
Listed as JPM.
Layoff notices
Notices this year and last cover 396 people. The latest was filed June 23, 2026.State WARN notices
Demand for ML Engineer roles
New ML Engineer postings at the employers we follow are down 4% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under ML Engineer, by the work the posting describes.
ML engineers put machine learning models into products. They write the code that trains, ships, serves and monitors models so a recommendation or prediction works for real users.
Postings ask for strong Python and software engineering, a framework such as PyTorch, cloud services, and more and more often experience with large language models.
A data scientist often works out which model to build; an ML engineer makes it run reliably in production.
US government figures for computer and information research scientists
Median yearly pay, May 2025
$140,300
Expected change, 2025 to 2035
21.8% more jobsagainst 3.5% for all jobs
The government has no code for ML engineers, so these figures are for the closest group, computer and information research scientists.