Back to jobs

Senior AI Platform Engineer

Boston Hub · Remote · San Francisco Hub · Washington DC Office · Boston, Massachusetts, United States

Live, first seen by us 43 days ago
Pay
Pay not listed
Work setting
Hybrid — work setting source
Location type: Hybrid
From the employer’s posting
Employment
Full-time — employment source
Employment type: Full-time
From the employer’s posting
Team
Engineering — team source
Department: Engineering
From the employer’s posting

What you’ll work on

Full posting
  • Build evaluation harnesses and the experiment-tracking and artifact layer that lets engineers and researchers reproduce and compare results.

  • Own 1 or more platform components end to end, from design doc to production operation.

  • Build and operate the services in your focus area, and contribute across the rest of the stack.

From the employer’s posting
Agentic infrastructure: Build agent harnesses, orchestration primitives, and context-engineering services (memory, retrieval, and data discovery) that product teams compose into reliable, verifiable workflows. Observability and experimentation: Instrument the stack end to end, for example, with OpenTelemetry traces and service metrics. Build evaluation harnesses and the experiment-tracking and artifact layer that lets engineers and researchers reproduce and compare results. Core Responsibilities
Core Responsibilities Own 1 or more platform components end to end, from design doc to production operation. Build and operate the services in your focus area, and contribute across the rest of the stack.
Own 1 or more platform components end to end, from design doc to production operation. Build and operate the services in your focus area, and contribute across the rest of the stack. Write clean, well-tested code, and hold AI-generated code to the same bar: correct, modular, and fully tested.

Before you apply

Work authorization
No visa sponsorship

What you’ll bring

All qualifications

Core experience

  • Experience owning a service or component in production, including debugging and improving its reliability.
  • Experience writing design docs for focused projects, reviewing code, and mentoring or onboarding teammates.

Preferred experience

  • Experience in more than 1 focus area.
  • Familiarity with inference optimization: speculative decoding, prefix caching, tensor/pipeline/expert parallelism, GPU profiling.
  • Experience with durable workflows, agent frameworks, or tool protocols such as MCP.
  • Experience fine-tuning language models, including distributed training across multiple GPU nodes.
Qualification wording
Experience owning a service or component in production, including debugging and improving its reliability.
Experience writing design docs for focused projects, reviewing code, and mentoring or onboarding teammates.
Experience in more than 1 focus area.
Familiarity with inference optimization: speculative decoding, prefix caching, tensor/pipeline/expert parallelism, GPU profiling.
Experience with durable workflows, agent frameworks, or tool protocols such as MCP.
Experience fine-tuning language models, including distributed training across multiple GPU nodes.

Benefits in the posting

Full benefits wording
  • Health care plan with 100% premium coverage, including medical, dental, and vision
  • 401k with 5% matching
  • Paid Time Off (uncapped vacation, plus sick and public holidays)
  • Flexible hybrid or remote work arrangement
  • Relocation assistance for qualifying employees
  • Wage Transparency - The salary range for this role is not a guarantee of compensation or salary, as the final offer amount may vary based on factors including, but not limited to, individual proficiency, skills, experience, and location.

From the employer’s posting.

About Code Metal

Code Metal is the leader in automated software engineering you can trust.

In the employer’s words · Read in context · Company website ↗

Apply at Code Metal

Posting record

Report an error
Status in our records
Live
First seen by us
Oct 9, 202643 days ago
Recorded sightings
25
Last seen by us
Nov 22, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 43

Up longer than most. Similar jobs usually come down between day 1 and day 15.

From 1,371 similar postings that came down. Coming down is not always a hire.

Code Metal is hiring

5 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • Kubernetes
  • MLflow
  • Hugging Face
  • PyTorch
Where the posting names them
Required Qualifications - Production-grade Python and solid platform engineering fundamentals: API and service design, distributed systems, containers and orchestration (such as Kubernetes), CI/CD, and testing. - Production experience in at least 1 focus area:
- Experience fine-tuning language models, including distributed training across multiple GPU nodes. - Experience with experiment-tracking or artifact management systems such as MLflow or Weights & Biases. - Experience deploying AI systems on-prem or in regulated domains such as defense or aerospace.
- Observability and experimentation: instrumenting and operating services against SLOs, or building evaluation and experiment-tracking systems. - Solid data science and AI research fundamentals: how transformers and LLM inference work, experiment design, benchmarking, and model evaluation. Working familiarity with PyTorch and Hugging Face. - Experience owning a service or component in production, including debugging and improving its reliability.

Your next step

Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.

Applying through Ashby

  • No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.

Already applied? Track this application

Job description

View original posting ↗

About Code Metal

Code Metal is the leader in automated software engineering you can trust. As AI writes more of the world's code, the bottleneck in software has shifted from writing code to verifying it works, and AI cannot verify its own work with certainty. Code Metal takes a fundamentally different approach: constrain AI to what it does reliably, verify every step independently of the model using formal methods, and keep engineers in the loop on the decisions that matter. The result isn't code that probably works — it's code that is provably correct, with auditable proof. Customers including the U.S. Air Force, L3Harris, RTX, and Toshiba use Code Metal to modernize legacy code, optimize performance on real hardware, and move prototypes to production, fast. Founded in 2023 with offices in Boston and San Francisco, Code Metal is funded by Accel, Salesforce Ventures, B Capital, Smith Point Capital, J2 Ventures, Shield Capital, Overmatch, RTX, and others.
Learn more at codemetal.ai.

The Role

Code Metal's engineering teams are building AI-driven code transpilation, modernization, optimization, modeling and simulation solutions. They all need the same foundations: models to serve, agents to run, context to manage, and results to measure. Our AI Platform team builds those foundations.

As a Senior Software Engineer on the AI Platform team, you'll design, build, and operate parts of the AI enablement stack our engineers depend on: GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. It is an internal platform, but we're building it to product standard.

Over time you will own 1 area, such as model serving, agentic infrastructure, or experimentation and telemetry, while you also contribute across the rest of the stack. We don't expect you to arrive with every skill on this page.

This is an engineering role first. Most of your time goes to designing, building, and operating production systems. You'll also need solid data science and AI research fundamentals: you'll work closely with our Applied AI Research team, and you'll sometimes run experiments when a platform decision needs evidence.

Focus Areas

Each engineer on the team brings depth in at least 1 of these areas:

  • Model serving: Deploy, benchmark, and tune elastic, production inference for open-weight models on runtimes such as vLLM, SGLang, and TensorRT-LLM. Run the model gateways that teams use to reach self-hosted and commercial models.

  • Agentic infrastructure: Build agent harnesses, orchestration primitives, and context-engineering services (memory, retrieval, and data discovery) that product teams compose into reliable, verifiable workflows.

  • Observability and experimentation: Instrument the stack end to end, for example, with OpenTelemetry traces and service metrics. Build evaluation harnesses and the experiment-tracking and artifact layer that lets engineers and researchers reproduce and compare results.

Core Responsibilities

  • Own 1 or more platform components end to end, from design doc to production operation.

  • Build and operate the services in your focus area, and contribute across the rest of the stack.

  • Write clean, well-tested code, and hold AI-generated code to the same bar: correct, modular, and fully tested.

  • Debug hard production issues in the components you own, such as tail latency, GPU memory pressure, or agent runs that fail or loop.

  • Scope and estimate efforts with internal customers, and raise risks early.

  • Review teammates' code and designs, and help onboard new engineers.

  • Run focused benchmarks and experiments when a platform decision needs evidence.

Required Qualifications

  • Production-grade Python and solid platform engineering fundamentals: API and service design, distributed systems, containers and orchestration (such as Kubernetes), CI/CD, and testing.

  • Production experience in at least 1 focus area:

    • Model serving: deploying and tuning LLM inference, or running an LLM gateway or API gateway.

    • Agentic infrastructure: shipping agentic systems, or building context-engineering services such as retrieval-augmented generation, embeddings, and vector or hybrid search.

    • Observability and experimentation: instrumenting and operating services against SLOs, or building evaluation and experiment-tracking systems.

  • Solid data science and AI research fundamentals: how transformers and LLM inference work, experiment design, benchmarking, and model evaluation. Working familiarity with PyTorch and Hugging Face.

  • Experience owning a service or component in production, including debugging and improving its reliability.

  • Experience writing design docs for focused projects, reviewing code, and mentoring or onboarding teammates.

Preferred Qualifications

  • Experience in more than 1 focus area.

  • Production experience running an LLM gateway or proxy such as SMG or Bifrost, or building an API gateway, including routing, auth, rate limiting, quotas, failover, and cost attribution.

  • Familiarity with inference optimization: speculative decoding, prefix caching, tensor/pipeline/expert parallelism, GPU profiling.

  • Experience with durable workflows, agent frameworks, or tool protocols such as MCP.

  • Experience fine-tuning language models, including distributed training across multiple GPU nodes.

  • Experience with experiment-tracking or artifact management systems such as MLflow or Weights & Biases.

  • Experience deploying AI systems on-prem or in regulated domains such as defense or aerospace.

Experience Level

Typically 4+ years of software engineering experience, including 2+ years building and operating ML/AI or LLM systems in production.

Benefits
  • Pay depends on experience, but we strive to be at the upper end of the salary range

  • Health care plan with 100% premium coverage, including medical, dental, and vision

  • 401k with 5% matching

  • Paid Time Off (uncapped vacation, plus sick and public holidays)

  • Flexible hybrid or remote work arrangement

  • Relocation assistance for qualifying employees

Wage Transparency - The salary range for this role is not a guarantee of compensation or salary, as the final offer amount may vary based on factors including, but not limited to, individual proficiency, skills, experience, and location.


We are an equal opportunity employer. US Citizenship may be required for certain project assignments involving security clearance.

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Demand for AI Engineer roles

New AI Engineer postings at the employers we follow are up 340% over the last 6 weeks compared with the 6 before.

Weekly new AI Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 78, the most 803, and the latest week had 438.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under AI Engineer, by the work the posting describes.

AI engineers build products on top of AI models that already exist, most often large language models: they connect a model to a company's data and tools and ship the assistants, agents and search features people use.

Postings ask for strong Python or TypeScript, experience with model APIs, retrieval and agent frameworks, and ways to test whether the model's answers are good enough.

An ML engineer trains and serves models; an AI engineer usually starts from a model someone else trained and builds the product around it, though titles often mix the two.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Boston, Massachusetts, United States

- Model serving: deploying and tuning LLM inference, or running an LLM gateway or API gateway. - Agentic infrastructure: shipping agentic systems, or building context-engineering services such as retrieval-augmented generation, embeddings, and vector or hybrid search. - Observability and experimentation: instrumenting and operating services against SLOs, or building evaluation and experiment-tracking systems.
More source context
- Paid Time Off (uncapped vacation, plus sick and public holidays) - Flexible hybrid or remote work arrangement - Relocation assistance for qualifying employees
Work authorization
Wage Transparency - The salary range for this role is not a guarantee of compensation or salary, as the final offer amount may vary based on factors including, but not limited to, individual proficiency, skills, experience, and location. We are an equal opportunity employer. US Citizenship may be required for certain project assignments involving security clearance. Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.