Machine Learning Engineer — Inference Optimization
Remote (world)
- Pay
- Pay not listed
- Work setting
Remote stated — work setting source
Listed location: Remote (world)
Read the full posting- Employment
Full-time — employment source
Employment type: Full-time
From the employer’s posting- Team
Research — team source
Department: Research
From the employer’s posting
What you’ll work on
Full postingWe’re looking for a Machine Learning Engineer to own and push the limits of model inference performance at scale.
This role is ideal for someone who enjoys deep technical work, profiling systems down to the kernel/GPU level, and translating research ideas into production-grade performance gains.
Optimize inference latency, throughput, and cost for large-scale models in production
Profile and bottleneck GPU/CPU inference pipelines (memory, kernels, batching, IO)
Implement quantization, KV-cache reuse, speculative decoding, and batching
Productionize new model architectures with research engineers
Build and maintain inference-serving systems such as Triton or custom runtimes
From the employer’s posting
We’re looking for a Machine Learning Engineer to own and push the limits of model inference performance at scale. You’ll work at the intersection of research and production—turning cutting-edge models into fast, reliable, and cost-efficient systems that serve real users.
This role is ideal for someone who enjoys deep technical work, profiling systems down to the kernel/GPU level, and translating research ideas into production-grade performance gains.
Optimize inference latency, throughput, and cost for large-scale ML models in production
Profile and bottleneck GPU/CPU inference pipelines (memory, kernels, batching, IO)
Implement and tune techniques such as: Quantization (fp16, bf16, int8, fp8) KV-cache optimization & reuse Speculative decoding, batching, and streaming
Collaborate with research engineers to productionize new model architectures
Build and maintain inference-serving systems (e.g. Triton, custom runtimes, or bespoke stacks)
Tools in this posting
- PyTorch
Source — Tool mentions in context
- Solid understanding of deep learning internals (attention, memory layout, compute graphs) - Hands-on experience with PyTorch (or similar) and model deployment - Familiarity with GPU performance tuning (CUDA, ROCm, Triton, or kernel-level optimizations)
Posting record
Report an errorOpen 4 monthsWe first saw this listing 143 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
- Status in our records
- Live
- First seen by us
- Jun 2, 2026 · 143 days ago
- Recorded sightings
- 82
- Last seen by us
- Oct 23, 2026
- Employer says posted
- Jan 22, 2026
These dates show when we found the listing. Check the employer’s website to confirm it is still accepting applications.
Your next step
- Have your resume and examples of relevant work ready.
- Check the listed location, eligibility and core experience before starting.
- Ask the employer about the pay range before committing time to the process.
Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.
Already applied? Track this application
Job description
About the Role
We’re looking for a Machine Learning Engineer to own and push the limits of model inference performance at scale. You’ll work at the intersection of research and production—turning cutting-edge models into fast, reliable, and cost-efficient systems that serve real users.
This role is ideal for someone who enjoys deep technical work, profiling systems down to the kernel/GPU level, and translating research ideas into production-grade performance gains.
What You’ll Do
Optimize inference latency, throughput, and cost for large-scale ML models in production
Profile and bottleneck GPU/CPU inference pipelines (memory, kernels, batching, IO)
Implement and tune techniques such as:
Quantization (fp16, bf16, int8, fp8)
KV-cache optimization & reuse
Speculative decoding, batching, and streaming
Model pruning or architectural simplifications for inference
Collaborate with research engineers to productionize new model architectures
Build and maintain inference-serving systems (e.g. Triton, custom runtimes, or bespoke stacks)
Benchmark performance across hardware (NVIDIA / AMD GPUs, CPUs) and cloud setups
Improve system reliability, observability, and cost efficiency under real workloads
What We’re Looking For
Strong experience in ML inference optimization or high-performance ML systems
Solid understanding of deep learning internals (attention, memory layout, compute graphs)
Hands-on experience with PyTorch (or similar) and model deployment
Familiarity with GPU performance tuning (CUDA, ROCm, Triton, or kernel-level optimizations)
Experience scaling inference for real users (not just research benchmarks)
Comfortable working in fast-moving startup environments with ownership and ambiguity
Nice to Have
Experience with LLM or long-context model inference
Knowledge of inference frameworks (TensorRT, ONNX Runtime, vLLM, Triton)
Experience optimizing across different hardware vendors
Open-source contributions in ML systems or inference tooling
Background in distributed systems or low-latency services
Why Join Us
Real ownership over performance-critical systems
Direct impact on product reliability and unit economics
Close collaboration with research, infra, and product
Competitive compensation + meaningful equity at Series A
A team that cares about engineering quality, not hype
Source details
Source notes
Source excerptsSelected passages from the saved posting. Check the full description for conditions and exceptions.
- Pay
No pay amount identified in the saved description.
- Location & working pattern
Remote (world)
Working pattern and location restrictions need checking in the full posting.
- Work authorization
No clear work-authorization passage found. Eligibility is unconfirmed.
See how this role fits your experience
Add your resume to compare the role’s scope, tools and requirements with your experience.