Back to jobs

Data Infra - Signals and Evaluations (IC)

San Francisco · San Francisco, California, United States

Live, first seen by us 67 days ago
Pay
Pay not listedSimilar roles state $100K–$169K
Work setting
On-site — work setting source
Location type: On-site
From the employer’s posting
Employment
Full-time — employment source
Employment type: Full-time
From the employer’s posting
Team
Software Engineering › Infrastructure — team source
Department: Software Engineering › Infrastructure
From the employer’s posting

What you’ll work on

Full posting

Matter is hiring a Signal, Data, and Evaluation Engineer to build the datasets, benchmarks, environments, and evidence that make our models and AI systems scientifically defensible.

  • Build versioned dataset workflows for curation, labeling, ground truth, and train-evaluation isolation

  • Design benchmark suites across perception, multimodal reasoning, planning, and customer tasks

  • Create simulation and environment infrastructure for reinforcement and imitation learning

  • Turn field, mission, and customer failures into datasets, benchmark cases, and environment scenarios

  • Build automated promotion gates and state what has been tested and what evidence supports release

From the employer’s posting
Matter is hiring a Signal, Data, and Evaluation Engineer to build the datasets, benchmarks, environments, and evidence that make our models and AI systems scientifically defensible. Reporting to Ignacio Cases Martin, this individual contributor will work across data curation, simulation and learning environments, model and agent evaluation, and release-quality evidence.
Build versioned dataset workflows for collection, curation, filtering, labeling, ground truth, synthetic data, hard-negative mining, contamination controls, and train-evaluation isolation.
Design benchmark suites across perception, multimodal reasoning, physics-informed prediction, retrieval, planning, tool use, world modeling, and customer tasks.
Create simulation and environment infrastructure for reinforcement learning, imitation learning, offline learning, model-based learning, and agent training.
Turn field, mission, and customer failures into versioned datasets, benchmark cases, adversarial tests, and environment scenarios.
Build automated promotion gates and communicate clearly what has been tested, what remains unproven, and what evidence supports release.

Before you apply

Work authorization
Citizens or permanent residents onlyA lawful permanent resident (green card holder)

What you’ll bring

All qualifications

Core experience

  • Experience in ML evaluation, dataset engineering, reinforcement-learning environments, simulation, scientific ML, test infrastructure, or production AI systems.
  • Experience with dataset curation, benchmark platforms, deep reinforcement learning, world models, synthetic data, multimodal evaluation, or scientific validation.
  • 100% employer-paid health, dental, and vision coverage
  • Experience converting qualitative model or agent failures into reproducible datasets, tests, or environment scenarios.
  • Experience with PyTorch or JAX, distributed evaluation, experiment tracking, environment frameworks, or model and agent observability.
  • Ability to reason across the complete data path and identify how acquisition, transformation, indexing, orchestration, and presentation affect correctness.
Qualification wording
Experience in ML evaluation, dataset engineering, reinforcement-learning environments, simulation, scientific ML, test infrastructure, or production AI systems.
Experience with dataset curation, benchmark platforms, deep reinforcement learning, world models, synthetic data, multimodal evaluation, or scientific validation.
100% employer-paid health, dental, and vision coverage
Experience converting qualitative model or agent failures into reproducible datasets, tests, or environment scenarios.
Experience with PyTorch or JAX, distributed evaluation, experiment tracking, environment frameworks, or model and agent observability.
Ability to reason across the complete data path and identify how acquisition, transformation, indexing, orchestration, and presentation affect correctness.

About Matter Intelligence

Welcome to Matter, where we are building the future of vision AI: pairing a world-first sensor that sees molecular chemistry, temperature, and 3D shape with a Large World Model that will be the most powerful intelligence engine for the physical world.

In the employer’s words · Read in context · Company website ↗

Apply at Matter Intelligence

Posting record

Report an error
Status in our records
Live
First seen by us
Sep 15, 202667 days ago
Recorded sightings
46
Last seen by us
Nov 22, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Pay

$100K–$169K is typical

The middle half of what similar US postings state. This one doesn’t list pay.

From 163 postings in the last year. Stated ranges, not salaries paid.

Time since we first saw it

Day 67

Up longer than most. Similar jobs usually come down between day 6 and day 52.

From 2,081 similar postings that came down. Coming down is not always a hire.

Matter Intelligence is hiring

4 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • PyTorch
Where the posting names them
- Strong foundations in statistics, experimental design, benchmark validity, distribution shift, calibration, reward design, and evidence-based acceptance. - Fluency in Python and modern machine-learning frameworks, with the software engineering discipline to build tested modules, data contracts, or services. - Experience converting qualitative model or agent failures into reproducible datasets, tests, or environment scenarios.
- Experience with dataset curation, benchmark platforms, deep reinforcement learning, world models, synthetic data, multimodal evaluation, or scientific validation. - Experience with PyTorch or JAX, distributed evaluation, experiment tracking, environment frameworks, or model and agent observability. - Experience with uncertainty quantification, calibration, out-of-distribution detection, conformal methods, or selective prediction.

Your next step

Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.

Applying through Ashby

  • No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.

Already applied? Track this application

Job description

View original posting ↗

About Matter Intelligence

Welcome to Matter, where we are building the future of vision AI: pairing a world-first sensor that sees molecular chemistry, temperature, and 3D shape with a Large World Model that will be the most powerful intelligence engine for the physical world. This system doesn't just see what something looks like; it understands everything from a single pixel. We call this Superintelligent Vision.

Our team has delivered technologies to Mars for NASA/JPL, designed advanced sensors for U.S. Defense, and frontier artificial intelligence systems. We are now building the next generation of space- and airborne-based sensing systems.

About the Role

Matter is hiring a Signal, Data, and Evaluation Engineer to build the datasets, benchmarks, environments, and evidence that make our models and AI systems scientifically defensible. Reporting to Ignacio Cases Martin, this individual contributor will work across data curation, simulation and learning environments, model and agent evaluation, and release-quality evidence.

Key Responsibilities

  • Build versioned dataset workflows for collection, curation, filtering, labeling, ground truth, synthetic data, hard-negative mining, contamination controls, and train-evaluation isolation.

  • Design benchmark suites across perception, multimodal reasoning, physics-informed prediction, retrieval, planning, tool use, world modeling, and customer tasks.

  • Create simulation and environment infrastructure for reinforcement learning, imitation learning, offline learning, model-based learning, and agent training.

  • Define repeatable evaluation methods across accuracy, calibration, generalization, robustness, safety, latency, cost, and operational outcomes.

  • Turn field, mission, and customer failures into versioned datasets, benchmark cases, adversarial tests, and environment scenarios.

  • Build automated promotion gates and communicate clearly what has been tested, what remains unproven, and what evidence supports release.

Qualifications

Required

  • Experience in ML evaluation, dataset engineering, reinforcement-learning environments, simulation, scientific ML, test infrastructure, or production AI systems.

  • Strong foundations in statistics, experimental design, benchmark validity, distribution shift, calibration, reward design, and evidence-based acceptance.

  • Fluency in Python and modern machine-learning frameworks, with the software engineering discipline to build tested modules, data contracts, or services.

  • Experience converting qualitative model or agent failures into reproducible datasets, tests, or environment scenarios.

  • Ability to reason across the complete data path and identify how acquisition, transformation, indexing, orchestration, and presentation affect correctness.

Preferred

  • Experience with dataset curation, benchmark platforms, deep reinforcement learning, world models, synthetic data, multimodal evaluation, or scientific validation.

  • Experience with PyTorch or JAX, distributed evaluation, experiment tracking, environment frameworks, or model and agent observability.

  • Experience with uncertainty quantification, calibration, out-of-distribution detection, conformal methods, or selective prediction.

  • Experience evaluating models deployed to aircraft, satellites, robotics, industrial systems, or other constrained environments.

What Success Looks Like

  • Model and agent releases are supported by reproducible datasets, benchmarks, and clearly stated evidence.

  • Failures from experiments and deployments become durable test cases that improve future systems.

  • Evaluation results preserve scientific meaning and make capability, uncertainty, and residual risk legible to the team.

Location

This role is based in San Francisco, CA, and requires onsite work.

ITAR Requirements

To comply with U.S. export regulations, applicants must be one of the following:

  • A U.S. citizen or national

  • A lawful permanent resident (green card holder)

  • Eligible to obtain required authorizations from the U.S. Department of State

Employee Offerings and Benefits

At Matter, we believe in rewarding high performance and providing the support you need to thrive. Our compensation and benefits package includes:

  • Competitive compensation based on experience

  • Early-stage equity package

  • 100% employer-paid health, dental, and vision coverage

  • Opportunity to work on novel sensing, data, and AI systems with real-world deployment paths to the largest industries in the world

Matter Intelligence is an equal opportunity employer. We welcome candidates from all backgrounds who can raise the ambition and performance of the team.

Demand for Data Governance roles

New Data Governance postings at the employers we follow are up 164% over the last 6 weeks compared with the 6 before.

Weekly new Data Governance postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 75, the most 445, and the latest week had 321.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Governance, by the work the posting describes.

Data governance staff keep a company's data accurate, documented and used within the rules. They agree what key fields mean, set up quality checks, track who owns each dataset and look after master data.

Postings ask for SQL, a governance tool such as Collibra or Informatica, knowledge of privacy and data rules, and a lot of work with other teams.

A data engineer builds the systems that move data; governance staff set and check the standards that data has to meet.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

San Francisco, California, United States

- A lawful permanent resident (green card holder) - Eligible to obtain required authorizations from the U.S. Department of State Employee Offerings and Benefits
Work authorization
To comply with U.S. export regulations, applicants must be one of the following: - A U.S. citizen or national - A lawful permanent resident (green card holder) - Eligible to obtain required authorizations from the U.S. Department of State

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.