Back to jobs

Human Data Quality Engineer (Founding Team)

Pay
Pay not listed
Work setting
Not stated
Employment
Not stated

What you’ll work on

Full posting
  • Design quality frameworks and evaluation rubrics for complex human data programmes

  • Challenge annotation schemas and data requirements with clients when they won't yield the needed signal

  • Build quality checks into recruitment, screening, training, guidelines and calibration

  • Build scalable quality systems, measurement frameworks and automated checks in Python and SQL

  • Investigate data quality issues and find root causes

From the employer’s posting
Design the quality frameworks that underpin complex human data programmes, from evaluation rubrics through to launch readiness.
challenging annotation schemas and data requirements when they won't produce the signal the model needs
Build quality upstream across the operational workflow, from recruitment, screening, and training through to writing guidelines and running calibration sessions.
Build scalable quality systems, measurement frameworks and automated checks using Python and SQL.
Investigate data quality and integrity issues, identifying root causes and turning insights into scalable improvements.
Apply at Prolific

Posting record

Report an error

Open 7 monthsWe first saw this posting 221 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.

Status in our records
Live
First seen by us
Apr 14, 2026221 days ago
Recorded sightings
178
Last seen by us
Nov 22, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 221

Up longer than most. Similar jobs usually come down between day 6 and day 52.

Well past Prolific’s usual: its jobs come down by about day 39.

From 2,085 similar postings that came down, and 10 of Prolific’s. Coming down is not always a hire.

Prolific is hiring

7 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Hiring at a steady pace
  • Takes jobs down at a usual pace
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
Where the posting names them
- Build quality upstream across the operational workflow, from recruitment, screening, and training through to writing guidelines and running calibration sessions. - Build scalable quality systems, measurement frameworks and automated checks using Python and SQL. - Partner with product and engineering to build the quality infrastructure that delivers high-quality human data at scale.
- 5+ years of experience in building quality, evaluation or annotation systems within AI, machine learning, LLMs or human data environments. - Strong Python and SQL skills, with a passion for using data to solve complex quality problems. - A solid understanding of machine learning pipelines and how human data impacts model performance.

Your next step

Complete your application on job-boards.eu.greenhouse.io. The employer’s form will show what is required.

Applying through Greenhouse

  • No account needed: the form is one page, with an “Autofill my application” option that reads your resume.
  • Greenhouse doesn’t read resumes over 2.5 MB, or details placed in a header, footer, text box, table or second column. The file still attaches, but a recruiter has to type you in by hand.

Already applied? Track this application

Job description

View original posting ↗

Human Data Quality Engineer (Founding Team) 

 

Prolific

Prolific isn’t just enabling AI innovation – we’re redefining it. While foundational AI technologies are becoming commoditized, Prolific’s human data infrastructure provides the high-quality, diverse data required to train the next generation of AI models. Through our platform, we empower researchers and companies to access a global, ethically curated participant base, ensuring cutting-edge AI research and training grounded in inclusivity and precision.

 

The Role

As one of the founding members of Prolific's newly formed AI Data Services team, you'll help build the quality systems behind some of the world's most advanced AI models. Data quality is a strategic priority for Prolific, so this is a high-visibility role with direct exposure to senior stakeholders. This isn't a traditional QA role. We are not looking for someone to review data against a predefined checklist. We are looking for an innovative thinker that can leverage their expertise to define what good means where no definition exists yet.

Acting as a strategic thought partner, you’ll work at the intersection of human data, machine learning, evaluation across frontier use-cases that define what high-quality human data looks like for the next generation of advanced AI. This means that much of the work involves novel problems with no established answer, so you’ll be comfortable working through ambiguity.. . 

Your primary focus is working directly with clients and alongside frontier AI labs, translating what their models need into robust human data and evaluation strategies. Rather than checking quality at the end of a project, you'll engineer quality into every stage of the lifecycle, from study design and participant strategy through to evaluation, launch readiness and client delivery.You will also work alongside our product engineering, and supply teams to define and build the quality infrastructure that will enable us to deliver high quality human data at scale.

Much of the work you'll tackle won't have an existing playbook. You'll help create it.

 

What You’ll Be Doing

  • Design the quality frameworks that underpin complex human data programmes, from evaluation rubrics through to launch readiness.
  • Work with clients as a strategic thought partner, challenging annotation schemas and data requirements when they won't produce the signal the model needs. 
  • Advise on project design and how the choice of schema can impact data quality.
  • Build quality upstream across the operational workflow, from recruitment, screening, and training through to writing guidelines and running calibration sessions. 
  • Build scalable quality systems, measurement frameworks and automated checks using Python and SQL.
  • Partner with product and engineering to build the quality infrastructure that delivers high-quality human data at scale.
  • Investigate data quality and integrity issues, identifying root causes and turning insights into scalable improvements.
  • Architect and build dashboards, monitoring and reporting that provide clear visibility into quality and operational performance.
  • Raise the quality capability across the company, upskilling operations and acting as a thought mentor to junior analysts.
  • Help define how Prolific approaches quality across new AI domains, shaping best practice as the team grows.

What You’ll Bring to the Role

  • 5+ years of experience in building quality, evaluation or annotation systems within AI, machine learning, LLMs or human data environments.
  • Strong Python and SQL skills, with a passion for using data to solve complex quality problems.
  • A solid understanding of machine learning pipelines and how human data impacts model performance.
  • Strong analytical and statistical thinking, with experience designing scalable quality frameworks.
  • The confidence and credibility to interact with stakeholders at frontier labs and act as a partner.
  • The ability to leverage your experience and expertise to influence and guide stakeholders at every level, both client side and internally. in
  • The ability to turn your own data analysis and quality methodology into requirements that product and engineering can build into systems.
  • The ability to explain yout data analysis and findings clearly to non-technical stakeholders, so they can act on them. 
  • A proactive, builder's mindset -you enjoy creating new systems, navigating ambiguity and improving how things work.


    Even Better if you have:
  • Experience with LLM evaluation, RLHF, AI safety or red teaming.
  • Experience translating vague model or evaluation goals into clear annotation specifications. 
  • Experience working with human annotation programmes or human data operations.
  • Familiarity with calibration, inter-rater agreement, drift detection or other evaluation methodologies.

Why Prolific is a great place to work

We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.

We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breath and the best of humanity.

Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research.

Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture. At Prolific, our compensation packages for eligible roles include base salary, equity, and benefits. Many roles also include the opportunity to earn a cash variable element, such as a bonus or commission. Each job posting shows a salary range that reflects the minimum and maximum target for new hires, based on the role’s location as well as your skills, experience, and relevant education or training. Your recruiter will also be happy to share the specific salary range for your preferred location during the hiring process.

 

Links to More Information

Website 

Youtube 

Benefits 

External Handbook 

 

Privacy Statement

By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal information.

Demand for Data Governance roles

New Data Governance postings at the employers we follow are up 164% over the last 6 weeks compared with the 6 before.

Weekly new Data Governance postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 75, the most 445, and the latest week had 321.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Governance, by the work the posting describes.

Data governance staff keep a company's data accurate, documented and used within the rules. They agree what key fields mean, set up quality checks, track who owns each dataset and look after master data.

Postings ask for SQL, a governance tool such as Collibra or Informatica, knowledge of privacy and data rules, and a lot of work with other teams.

A data engineer builds the systems that move data; governance staff set and check the standards that data has to meet.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

North America

Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research. Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture. At Prolific, our compensation packages for eligible roles include base salary, equity, and benefits. Many roles also include the opportunity to earn a cash variable element, such as a bonus or commission. Each job posting shows a salary range that reflects the minimum and maximum target for new hires, based on the role’s location as well as your skills, experience, and relevant education or training. Your recruiter will also be happy to share the specific salary range for your preferred location during the hiring process. Links to More Information
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.