Back to jobs

Senior ML Infra Engineer

New York, New York, United States, San Francisco, California, United States

Actively hiring, first seen by us 1 day ago
Pay
$200,000–275,000/yearAnnual period assumed — pay source
Compensation & Benefits $200,000 - $275,000 base, calibrated to experience and location Health, dental, vision; unlimited PTO; wellness stipend & more…
Read the full posting
Work setting
Not stated
Employment
Not stated

What you’ll work on

Full posting
  • Build and own infrastructure for training, evaluating, and serving AI models

  • Develop systems for running large-scale experiments and evaluations quickly and reproducibly

  • Build pipelines for generating, processing, versioning, and managing training and evaluation data

  • Improve reliability, latency, throughput, and cost efficiency of model inference

  • Build observability and monitoring for AI systems in production

From the employer’s posting
Build and own infrastructure for training, evaluating, and serving AI models
Develop systems for running large-scale experiments and evaluations quickly and reproducibly
Build pipelines for generating, processing, versioning, and managing training and evaluation data
Improve the reliability, latency, throughput, and cost efficiency of model inference
Build observability and monitoring for AI systems in production

Before you apply

Work authorization
Must already have the right to work in the USEligible to work in the US, and able to work in-office a minimum of 3 days a week in our SF or NY office locations

What you’ll bring

All qualifications

Core experience

  • 5+ years of software engineering, machine learning engineering, or infrastructure experience
  • Experience building production infrastructure for machine learning systems
  • Experience training, fine-tuning, or serving large language models
  • Experience with cloud infrastructure, distributed systems, and containerized workloads
  • Experience with GPU infrastructure and distributed training or inference
  • Ability to independently design, build, and operate complex technical systems
Qualification wording
5+ years of software engineering, machine learning engineering, or infrastructure experience
Experience building production infrastructure for machine learning systems
Experience training, fine-tuning, or serving large language models
Experience with cloud infrastructure, distributed systems, and containerized workloads
Experience with GPU infrastructure and distributed training or inference
Ability to independently design, build, and operate complex technical systems

Benefits in the posting

Full benefits wording
  • Health, dental, vision; unlimited PTO; wellness stipend & more…

From the employer’s posting.

Apply at General Legal

Posting record

Report an error

Actively hiringFirst seen by us 1 day ago, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 18, 20261 day ago
Recorded sightings
3
Last seen by us
Nov 20, 2026
Employer says posted
Aug 27, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Pay

Higher than most similar jobs

$200K–$275K against $166K–$237K, the middle half of what similar US postings state.

From 1,507 postings in the last year. Stated ranges, not salaries paid.

Time since we first saw it

Day 1

Early. Similar jobs usually come down between day 7 and day 76.

From 6,269 similar postings that came down. Coming down is not always a hire.

General Legal is hiring

1 open job

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
Where the posting names them
- 5+ years of software engineering, machine learning engineering, or infrastructure experience - Strong software engineering fundamentals and proficiency in Python - Experience building production infrastructure for machine learning systems

Your next step

Complete your application on job-boards.greenhouse.io. The employer’s form will show what is required.

Applying through Greenhouse

  • No account needed: the form is one page, with an “Autofill my application” option that reads your resume.
  • Greenhouse doesn’t read resumes over 2.5 MB, or details placed in a header, footer, text box, table or second column. The file still attaches, but a recruiter has to type you in by hand.

Already applied? Track this application

Job description

View original posting ↗

The Opportunity

The legal industry is broken — lawyers bill by the hour, charge enormous sums, and take forever to respond. This means that a lot of people who need legal advice don't get it. We're using AI to solve this problem. AI takes the first pass at everything our lawyers do, so that the lawyers can focus on human interaction. We're starting with negotiating commercial contracts, and will expand quickly from there to other use cases within legal. We're reimagining what a contract can be in the AI age. We're a veteran team, with one successful exit already, who've been doing deep learning in the legal space since long before ChatGPT.

 

Description

General Legal is seeking a Senior ML Infra Engineer to build the systems that allow us to rapidly experiment with, evaluate, train, and deploy increasingly capable AI systems.

You'll sit at the intersection of research and production engineering. Your job will be to make our researchers and engineers dramatically more effective: building reliable infrastructure for model experimentation, evaluation, inference, data generation, training, and observability while ensuring that promising ideas can move quickly from an experiment into production.

This is a unique opportunity to join us at the ground level and define the AI infrastructure behind a platform that will set new standards for the legal industry. You'll have substantial autonomy over architecture and tooling and will help determine how our ML stack evolves as we scale.

 

Responsibilities

  • Build and own infrastructure for training, evaluating, and serving AI models
  • Develop systems for running large-scale experiments and evaluations quickly and reproducibly
  • Build pipelines for generating, processing, versioning, and managing training and evaluation data
  • Improve the reliability, latency, throughput, and cost efficiency of model inference
  • Build observability and monitoring for AI systems in production
  • Develop infrastructure for agentic workloads, including long-running and asynchronous model execution
  • Work closely with research scientists and product engineers to turn new AI capabilities into reliable production systems
  • Evaluate and integrate new models, inference systems, training frameworks, and infrastructure as the field evolves
  • Be proactive and come up with ideas on how to build our AI systems better

 

Requirements

  • 5+ years of software engineering, machine learning engineering, or infrastructure experience
  • Strong software engineering fundamentals and proficiency in Python
  • Experience building production infrastructure for machine learning systems
  • Experience with cloud infrastructure, distributed systems, and containerized workloads
  • Ability to independently design, build, and operate complex technical systems
  • Computer Science degree or equivalent experience
  • Eligible to work in the US, and able to work in-office a minimum of 3 days a week in our SF or NY office locations.

 

Nice-to-Haves

  • Experience training, fine-tuning, or serving large language models
  • Experience with GPU infrastructure and distributed training or inference
  • Experience with reinforcement learning, post-training, or large-scale evaluation systems
  • Experience building infrastructure for AI agents or long-running model workloads
  • Experience optimizing inference latency and cost at scale
  • Experience at an early-stage startup

 

Compensation & Benefits

$200,000 - $275,000 base, calibrated to experience and location

Health, dental, vision; unlimited PTO; wellness stipend & more…

Demand for ML Engineer roles

New ML Engineer postings at the employers we follow are up 12% over the last 6 weeks compared with the 6 before.

Weekly new ML Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 556, the most 1,080, and the latest week had 556.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under ML Engineer, by the work the posting describes.

ML engineers put machine learning models into products. They write the code that trains, ships, serves and monitors models so a recommendation or prediction works for real users.

Postings ask for strong Python and software engineering, a framework such as PyTorch, cloud services, and more and more often experience with large language models.

A data scientist often works out which model to build; an ML engineer makes it run reliably in production.

US government figures for computer and information research scientists

Median yearly pay, May 2025
$140,300
Expected change, 2025 to 2035
21.8% more jobsagainst 3.5% for all jobs

The government has no code for ML engineers, so these figures are for the closest group, computer and information research scientists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1221.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay
Compensation & Benefits $200,000 - $275,000 base, calibrated to experience and location Health, dental, vision; unlimited PTO; wellness stipend & more…
Location & working pattern

New York, New York, United States, San Francisco, California, United States

Working pattern and location restrictions need checking in the full posting.

Work authorization
- Computer Science degree or equivalent experience - Eligible to work in the US, and able to work in-office a minimum of 3 days a week in our SF or NY office locations. Nice-to-Haves

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.