Back to jobs

AI Quality Engineer

Pay
Pay not listed
Work setting
Not stated
Employment
Not stated

What you’ll work on

Full posting

Frontier models are now limited less by compute but more by high quality human data and evaluations behind them.

At Deccan AI, our annotators, raters and QCs produce evaluation and training data for some of the world's leading AI labs - multi-turn assistant evals, agentic tool-use tasks, A/B conversation evals, rubric-based rating and robotics video labeling.

  • Design quality systems, mixing agents and code, from project kickoff to final delivery

  • Interpret client guidelines and design training programs for annotators

  • Analyse datasets and write insight reports for client research teams

  • Turn quality findings into feedback and example-driven LMS training content

From the employer’s posting
Frontier models are now limited less by compute but more by high quality human data and evaluations behind them. This role sits exactly at that intersection.
At Deccan AI, our annotators, raters and QCs produce evaluation and training data for some of the world's leading AI labs - multi-turn assistant evals, agentic tool-use tasks, A/B conversation evals, rubric-based rating and robotics video labeling. You will be the person who makes that data trustworthy at scale.
design and build quality systems, using a combination of agents and deterministic code, right from project’s kickoff to final delivery
Understand client guidelines, interpret nuanced requirements and design training programs
Analyse datasets and findings to build insight reports for client research teams.
Turn quality findings into targeted feedback into example-driven training content on the LMS.

About Deccan AI Automation

Deccan AI is a GenAI data company that runs the post-training and production layer for the world's leading AI labs and enterprises — from expert feedback and RL environments to managed AI workforces and evaluation infrastructure.

In the employer’s words · Read in context

Apply at Deccan AI Automation

Posting record

Report an error

Actively hiringFirst seen by us 3 days ago, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 18, 20263 days ago
Recorded sightings
4
Last seen by us
Nov 21, 2026
Employer says posted
Nov 11, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 3

Early. Similar jobs usually come down between day 6 and day 52.

From 2,081 similar postings that came down. Coming down is not always a hire.

Deccan AI Automation is hiring

4 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
  • Looker
  • Metabase
  • Superset
Where the posting names them
- Hands-on quality systems experience - you have designed or run QC workflows, sampling plans, rubrics or audit programmes. - Strong Python - you write scripts and small pipelines to process, score and analyse task data without using automations. - SQL fluency - comfortable joining messy production tables and building dashboards (Metabase, Superset, Looker or similar).
- Strong Python - you write scripts and small pipelines to process, score and analyse task data without using automations. - SQL fluency - comfortable joining messy production tables and building dashboards (Metabase, Superset, Looker or similar). - Working ML and LLM knowledge - how models are trained (SFT, RLHF, preference data), how evals work, and what inter-rater agreement, precision/recall and calibration mean in practice.

Your next step

Complete your application on deccanaiauto.keka.com. The employer’s form will show what is required.

Already applied? Track this application

Job description

View original posting ↗

AI Quality Engineer (LLM Evals & Human Data)

About Deccan AI
Deccan AI is a GenAI data company that runs the post-training and production layer for the world's leading AI labs and enterprises — from expert feedback and RL environments to managed AI workforces and evaluation infrastructure. Backed by A91 Partners, SIG (Susquehanna International Group) and Prosus Ventures, and having grown 10x last year, Deccan works with a majority of the Magnificent 7 and the top frontier labs globally. Its global network of 1 million+ domain experts powers Helix, its hybrid eval suite, and EnterpriseOS, its scaled ops automation platform. Headquartered in Mountain View, CA.

About the role
Frontier models are now limited less by compute but more by  high quality human data and evaluations behind them. This role sits exactly at that intersection.
At Deccan AI, our annotators, raters and QCs produce evaluation and training data for some of the world's leading AI labs - multi-turn assistant evals, agentic tool-use tasks, A/B conversation evals, rubric-based rating and robotics video labeling. You will be the person who makes that data trustworthy at scale.
We are looking for an analytical thinker with high attention to detail and an uncompromising approach when it comes to data quality - along with the drive to deliver the best outcome for our customers (clients). You will need to have a strong backbone, often requiring pushing back on defective data outcomes.

What you'll own

  • Work with project teams to design and build quality systems, using a combination of agents and deterministic code, right from project’s kickoff to final delivery
  • Understand client guidelines, interpret nuanced requirements and design training programs to ensure effective talent deployment on projects.
  • Analyse datasets and findings to build insight reports for client research teams.
  • Turn quality findings into targeted feedback into example-driven training content on the LMS.

What you must bring

  • 2–3 years in the annotation industry - at a data vendor, an AI lab, or an in-house labelling team - working on evals, SFT, multimodal or agentic data.
  • Hands-on quality systems experience - you have designed or run QC workflows, sampling plans, rubrics or audit programmes.
  • Strong Python - you write scripts and small pipelines to process, score and analyse task data without using automations.
  • SQL fluency - comfortable joining messy production tables and building dashboards (Metabase, Superset, Looker or similar).
  • Working ML and LLM knowledge - how models are trained (SFT, RLHF, preference data), how evals work, and what inter-rater agreement, precision/recall and calibration mean in practice.
  • Strong analytical skills - you can break down large, messy datasets, spot patterns and root causes, and turn them into clear, defensible insights.
  • Ownership under pressure - you can run several projects at once on tight timelines and still hold the highest quality bar.

Great if you also have

  • Built agents or pipelines with LLM APIs, LangChain/LangGraph, or similar tooling.
  • Prepared golden-sets, calibrated LLM judges and hillclimbed judge performance.
  • Worked on agentic or tool-use evals, coding / SWE-bench-style tasks, or robotics and video labeling.
  • Experience with statistics for audits: confidence intervals, stratified sampling, Cohen's kappa / Krippendorff's alpha.

Why this role

  • Frontier-lab exposure from day one - your work goes straight into how leading AI models are trained and evaluated.
  • Build, don't maintain - many of our quality systems and auto-QC pipelines are being built now; you'll shape them, not inherit them.
  • Real autonomy - you own projects, setting quality across accounts.
  • The skill set the market is short of - ML + code + evals + human-data judgement is one of the hardest profiles to hire in AI; this role builds all four at once.

Demand for Data Governance roles

New Data Governance postings at the employers we follow are up 164% over the last 6 weeks compared with the 6 before.

Weekly new Data Governance postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 75, the most 445, and the latest week had 321.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Governance, by the work the posting describes.

Data governance staff keep a company's data accurate, documented and used within the rules. They agree what key fields mean, set up quality checks, track who owns each dataset and look after master data.

Postings ask for SQL, a governance tool such as Collibra or Informatica, knowledge of privacy and data rules, and a lot of work with other teams.

A data engineer builds the systems that move data; governance staff set and check the standards that data has to meet.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Hyderabad, TG, India

About Deccan AI Deccan AI is a GenAI data company that runs the post-training and production layer for the world's leading AI labs and enterprises — from expert feedback and RL environments to managed AI workforces and evaluation infrastructure. Backed by A91 Partners, SIG (Susquehanna International Group) and Prosus Ventures, and having grown 10x last year, Deccan works with a majority of the Magnificent 7 and the top frontier labs globally. Its global network of 1 million+ domain experts powers Helix, its hybrid eval suite, and EnterpriseOS, its scaled ops automation platform. Headquartered in Mountain View, CA. About the role
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.