Back to jobs

Human Data Evals Lead (Remote/US/LATAM)

Argentina - Fully Remote · Uruguay · Chile · Ecuador - Fully Remote · Mexico - Fully Remote · Peru - Fully Remote · Buenos Aires, Distrito Federal, Argentina

Actively hiring, first seen by us 1 day ago
Pay
Pay not listed
Work setting
Remote — work setting source
Owns: data proposals, sample development, quality, and pilot delivery Location: Remote / Latam / US The role
Location type: Remote
From the employer’s posting
Employment
Temporary — employment source
Employment type: Temporary
From the employer’s posting
Team
Anyone AI Internal Team — team source
Department: Anyone AI Internal Team
From the employer’s posting

What you’ll work on

Full posting
  • Turn public benchmarks and eval targets into proposals and sample packages

  • Design sample packages with expert-verified ground truth and multi-model headroom checks

  • Recruit, brief, calibrate and review a pool of subject-matter experts

  • Own pilots end to end, from scoping and SOW through QC and delivery

  • Serve as direct point of contact for lab partners on Slack and calls

From the employer’s posting
Study public benchmarks and eval targets, and turn them into proposals and sample packages that demonstrate capability and win the work.
Expert-verified, exact-match-checkable ground truth and gold reasoning trajectories. Multi-model evaluation showing real headroom
Recruit, brief, calibrate, and review a pool of experts across coding, agentic/tool-use, and STEM/reasoning.
Own pilots end to end: scoping, SOW, staffing, production, QC, and delivery.
Be a direct point of contact for lab partners on Slack and calls

What you’ll bring

All qualifications

Core experience

  • 5+ years in technical delivery, quality, or program management, with recent experience in AI/ML data, model evaluation, or benchmarking.
  • Hands-on experience delivering data or evaluation work to AI labs or enterprise ML teams, scoping through delivery.
Qualification wording
5+ years in technical delivery, quality, or program management, with recent experience in AI/ML data, model evaluation, or benchmarking.
Hands-on experience delivering data or evaluation work to AI labs or enterprise ML teams, scoping through delivery.
Apply at Anyone AI

Posting record

Report an error

Actively hiringFirst seen by us 1 day ago, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 20, 20261 day ago
Recorded sightings
2
Last seen by us
Nov 21, 2026
Employer says posted
Jun 19, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 1

Early. Similar jobs usually come down between day 8 and day 57.

From 9,984 similar postings that came down. Coming down is not always a hire.

Anyone AI is hiring

4 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume to compare the role’s scope, tools and requirements with your experience.

Your next step

Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.

Applying through Ashby

  • No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.

Already applied? Track this application

Job description

View original posting ↗

Reports to: CEO

Owns: data proposals, sample development, quality, and pilot delivery

Location: Remote / Latam / US


The role

You will own Anyone AI’s data initiatives and proposals to AI labs, from the data proposal or responding to requests, through pilot delivery. You own how we build proposals and develop the sample packages and benchmarks: frontier-grade packages across reasoning, coding, agents, and tool use, multi-modal and others, produced in collaboration with subject-matter experts, with expert-verified ground truth, multi-model headroom results, and QC that survives buyer-side scrutiny. You are the person who designs the sample that demonstrates our quality, converts pilots into production engagements. On a small team, this is the operational center of the Human Data Division.

Responsibilities

  • Proposals & requests. Study public benchmarks and eval targets, and turn them into proposals and sample packages that demonstrate capability and win the work. Respond to lab data requests and pilots.

  • Sample & benchmark development. Design and build the sample packages, working with subject-matter experts. Every package meets the bar of our current sample set:

    • Expert-verified, exact-match-checkable ground truth and gold reasoning trajectories.

    • Multi-model evaluation showing real headroom, and proof the task discriminates the model, not just that it's hard.

    • Rigorous QC structure: calibration layers, severity-weighted rubrics, deterministic verifiers, evidence maps, etc.

  • Subject-matter experts. Recruit, brief, calibrate, and review a pool of experts across coding, agentic/tool-use, and STEM/reasoning. Raise their output to our standard and keep it there; be the arbiter of what "correct" and "frontier-difficulty" mean.

  • Lab relationships. Be a direct point of contact for lab partners on Slack and calls, with support from the CEO and the wider team. Keep senior lab contacts informed, surface what they actually need, and pull in the CEO and subject-matter experts when the conversation calls for it.

  • Pilot delivery. Own pilots end to end: scoping, SOW, staffing, production, QC, and delivery. Nothing ships before it's lab-ready, and nothing comes back rejected as "not frontier-level" without us already knowing why.

Experience

  • Originated data or benchmark proposals for AI labs, translated eval targets into sample tasks that demonstrate capability, and owned the engagement through delivery.

  • Deep evaluation and quality expertise: LLM benchmarking, with real strength in code-model evaluation.

  • Built QC processes and artifact standards that met enterprise or lab requirements, and set a quality bar a team of experts was held to.

  • Thrives in ambiguous, fast-moving environments where the rules are still being written, and delivers under pressure.

Qualifications

  • 5+ years in technical delivery, quality, or program management, with recent experience in AI/ML data, model evaluation, or benchmarking.

  • Hands-on experience delivering data or evaluation work to AI labs or enterprise ML teams, scoping through delivery.

  • Working fluency with how frontier models are evaluated: benchmarks, rubrics, pass rates, headroom, and what makes a task discriminate a model.

  • Proven people/vendor leadership, you've recruited, calibrated, and held a team or expert pool to a quality standard.

  • Fluent English. Spanish is a nice to have.

Demand for Data Leadership roles

New Data Leadership postings at the employers we follow are up 8% over the last 6 weeks compared with the 6 before.

Weekly new Data Leadership postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 751, the most 1,165, and the latest week had 775.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Leadership, by the work the posting describes.

Data leaders run data teams or design how a company's data fits together. Managers hire, set priorities and answer for whether the data work pays off; architects choose the systems and the shape of the data.

Expect years of hands-on data work, experience leading people or projects, and the ability to explain tradeoffs to executives.

Unlike a senior engineer or analyst, most of a data leader's week goes to people, planning and decisions rather than building.

US government figures for computer and information systems managers

Median yearly pay, May 2025
$175,140
Expected change, 2025 to 2035
15.8% more jobsagainst 3.5% for all jobs

The government has no code for data leaders, so these figures are for the closest group, computer and information systems managers, which includes data processing and data operations managers.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 11-3021.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Buenos Aires, Distrito Federal, Argentina

Owns: data proposals, sample development, quality, and pilot delivery Location: Remote / Latam / US The role
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.