Time since we first saw it
Day 37
Similar jobs usually come down between day 7 and day 76.
From 6,398 similar postings that came down. Coming down is not always a hire.
Austria; France; Germany; Italy; Netherlands; Poland; Spain; United Kingdom
Live, first seen by us 37 days agoMove those four numbers in the right direction and two things happen: customers get faster, cheaper inference, and our margins improve.
The problem: the checkpoints run 0.3-1.5 TB, and the "right" serving configuration depends on traffic shape, sequence-length distribution, batch dynamics, GPU SKU, memory bandwidth, and quantization tolerance, all variables that shift week to week.
Raise throughput on each GPU SKU via batching, speculative decoding and scheduler tuning
Cut TTFT and inter-token latency by profiling for the real bottleneck
Improve KV cache use with paged attention, prefix caching and quantized KV
Speed up cold starts for terabyte-scale checkpoints with faster weight loaders
Quantize models to INT8/INT4 or FP8/FP4 without hurting quality on real workloads
Move those four numbers in the right direction and two things happen: customers get faster, cheaper inference, and our margins improve. That's the entire thesis of this role. Every loader you tune, every scheduler decision you land, every kernel you stare at shows up in a customer's p99 and on our P&L. Your mandate is the inference stack's optimization, whatever it takes, and the technical direction of how we optimize it is yours to argue with data.
The problem: the checkpoints run 0.3-1.5 TB, and the "right" serving configuration depends on traffic shape, sequence-length distribution, batch dynamics, GPU SKU, memory bandwidth, and quantization tolerance, all variables that shift week to week. Most teams pick a configuration once, over-provision GPUs, and absorb the cost. We don't: we took Kimi-K3 (1.5 TB) from over an hour cold-to-serving to under twelve minutes with a shared filesystem, O_DIRECT weight loaders, and node-local JIT caches, and we're nowhere near done. Prefill/decode disaggregation and tiered KV caches are next.
Continuous batching, chunked prefill, speculative decoding with draft models, CUDA-graph capture, and scheduler tuning across the engine fleet
Attack TTFT and inter-token latency separately; profile to find the actual bottleneck
Hierarchical and paged attention, prefix caching on heavy-reuse coding traffic, eviction policies, quantized KV
Weight loaders (O_DIRECT, fastsafetensors-class), JIT and compile caches, shared-filesystem read paths
Quantize without regressing quality. INT8/INT4, FP8/FP4 across weights, activations, and KV, measured on real workloads
As part of our standard hiring process, we would like to inform you that a background check may be conducted at the final stage of recruitment through our third-party provider, Checkr. Please note that Cast AI does not provide any form of visa sponsorship/work permit.Read the full posting
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 37
Similar jobs usually come down between day 7 and day 76.
From 6,398 similar postings that came down. Coming down is not always a hire.
3 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Hiring recordAdd your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
The problem: the checkpoints run 0.3-1.5 TB, and the "right" serving configuration depends on traffic shape, sequence-length distribution, batch dynamics, GPU SKU, memory bandwidth, and quantization tolerance, all variables that shift week to week. Most teams pick a configuration once, over-provision GPUs, and absorb the cost. We don't: we took Kimi-K3 (1.5 TB) from over an hour cold-to-serving to under twelve minutes with a shared filesystem, O_DIRECT weight loaders, and node-local JIT caches, and we're nowhere near done. Prefill/decode disaggregation and tiered KV caches are next. Stack: Python; SGLang; vLLM; TensorRT-LLM; PyTorch; fastsafetensors; CUDA-adjacent tooling; Kubernetes; virtiofs shared filesystems; ClickHouse; OpenTelemetry; Prometheus; Grafana; GitLab CI. Requirements
- You've built real ML systems for 5+ years, and your portfolio shows depth in inference or training infrastructure, not just model training notebooks. - Your Python is production-grade: services, not scripts. - You've run vLLM, SGLang, or TensorRT-LLM for real, and you have a working mental model of why an engine performs the way it does on a given GPU.
Why Kimchi? Great models are raw ingredients. Kimchi is what happens when you ferment them with good engineering. We started inside CAST AI helping companies run LLMs on their own Kubernetes clusters; today we run our own coding harness on any model you pick, and we run the models underneath it ourselves. Kimchi Inference
- Enjoy a flexible, remote-first global environment. - Collaborate with a global team of cloud experts and innovators, passionate about pushing the boundaries of Kubernetes technology - Equity options.
Complete your application on cast.ai. The employer’s form will show what is required.
Already applied? Track this application
Great models are raw ingredients. Kimchi is what happens when you ferment them with good engineering. We started inside CAST AI helping companies run LLMs on their own Kubernetes clusters; today we run our own coding harness on any model you pick, and we run the models underneath it ourselves.
Kimchi Inference
One API line and a customer's existing client just works: DeepSeek, GLM, Kimi, MiniMax, Nemotron served serverless from our own GPU fleet (OpenAI- and Anthropic-compatible, pay per token, spend visible per user, team, and key). When control matters, the same models run on the customer's own cluster instead. Behind that one line sits the machinery you'll own: SGLang and vLLM engines, B300 nodes, a shared cluster filesystem that holds every checkpoint exactly once, KV caches riding coding traffic with heavy prefix reuse, and the benchmark harness that gates every change we ship.
Made by Cast AI — a Miami-headquartered, globally distributed team across 34 countries that hit unicorn status in January 2026 with a $50B+ backed investment pushing its valuation past $1 billion.
Throughput. Latency. Cold starts. Cost per token.
Move those four numbers in the right direction and two things happen: customers get faster, cheaper inference, and our margins improve. That's the entire thesis of this role. Every loader you tune, every scheduler decision you land, every kernel you stare at shows up in a customer's p99 and on our P&L. Your mandate is the inference stack's optimization, whatever it takes, and the technical direction of how we optimize it is yours to argue with data.
The problem: the checkpoints run 0.3-1.5 TB, and the "right" serving configuration depends on traffic shape, sequence-length distribution, batch dynamics, GPU SKU, memory bandwidth, and quantization tolerance, all variables that shift week to week. Most teams pick a configuration once, over-provision GPUs, and absorb the cost. We don't: we took Kimi-K3 (1.5 TB) from over an hour cold-to-serving to under twelve minutes with a shared filesystem, O_DIRECT weight loaders, and node-local JIT caches, and we're nowhere near done. Prefill/decode disaggregation and tiered KV caches are next.
Stack: Python; SGLang; vLLM; TensorRT-LLM; PyTorch; fastsafetensors; CUDA-adjacent tooling; Kubernetes; virtiofs shared filesystems; ClickHouse; OpenTelemetry; Prometheus; Grafana; GitLab CI.
As part of our standard hiring process, we would like to inform you that a background check may be conducted at the final stage of recruitment through our third-party provider, Checkr.
Please note that Cast AI does not provide any form of visa sponsorship/work permit.
#LI-Remote
New ML Engineer postings at the employers we follow are down 4% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under ML Engineer, by the work the posting describes.
ML engineers put machine learning models into products. They write the code that trains, ships, serves and monitors models so a recommendation or prediction works for real users.
Postings ask for strong Python and software engineering, a framework such as PyTorch, cloud services, and more and more often experience with large language models.
A data scientist often works out which model to build; an ML engineer makes it run reliably in production.
The government has no code for ML engineers, so these figures are for the closest group, computer and information research scientists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1221.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
Austria; France; Germany; Italy; Netherlands; Poland; Spain; United Kingdom
- Competitive salary (depending on the level of experience). - Enjoy a flexible, remote-first global environment. - Collaborate with a global team of cloud experts and innovators, passionate about pushing the boundaries of Kubernetes technology
As part of our standard hiring process, we would like to inform you that a background check may be conducted at the final stage of recruitment through our third-party provider, Checkr. Please note that Cast AI does not provide any form of visa sponsorship/work permit. #LI-Remote
- Culture Check interview with an executive As part of our standard hiring process, we would like to inform you that a background check may be conducted at the final stage of recruitment through our third-party provider, Checkr. Please note that Cast AI does not provide any form of visa sponsorship/work permit. #LI-Remote