Pay
$166K–$240K is typical
The middle half of what similar US postings state. This one doesn’t list pay.
From 1,279 postings in the last year. Stated ranges, not salaries paid.
Data Scientist (AI Quality & Evaluation)
Reads like a ML Engineer role: putting models into production.
Boston, MA, United States
Open 5 months, first seen by us 155 days ago- Pay
- Pay not listedSimilar roles state $166K–$240K
- Work setting
Hybrid — work setting source
Work setup in the employer’s posting: Hybrid
From the employer’s posting- Employment
Full-time — employment source
Employment type in the employer’s posting: FullTime
From the employer’s posting
What you’ll work on
Full postingWe're looking for a Data Scientist to own the quality, reliability, and trustworthiness of our clinical AI outputs.
Design automated pipelines that assess AI output quality, accuracy, and safety
Develop uncertainty quantification so confidence scores track actual accuracy
Build evaluation frameworks combining automated checks with clinician-validated test cases
Implement feedback loops that improve model outputs from validation signals
Establish quality gates that catch errors before they reach end users
From the employer’s posting
We're looking for a Data Scientist to own the quality, reliability, and trustworthiness of our clinical AI outputs. You'll build the systems that ensure our AI "knows what it doesn't know" — developing evaluation frameworks, calibrated confidence scoring, and automated quality assurance that physicians can actually trust.
Design and implement automated evaluation pipelines that assess AI output quality, accuracy, and safety at scale
Develop uncertainty quantification systems where confidence scores meaningfully correlate with accuracy
Build comprehensive evaluation frameworks combining automated assessment with clinician-validated test cases
Implement feedback loops that continuously improve model outputs based on validation signals
Establish scalable quality gates that catch errors before they reach end users
What you’ll bring
All qualificationsCore experience
- Strong foundation in PyTorch and LLM architectures
- Experience with model evaluation, benchmarking, and quality metrics
- Proficiency in Python and modern ML development tools
- Strong statistical foundations
- Ability to read, implement, and extend research papers
Preferred experience
- PhD preferred
Qualification wording
Strong foundation in deep learning frameworks (PyTorch) and LLM architectures
Experience with model evaluation, benchmarking, and quality metrics
Proficiency in Python and modern ML development tools
Strong statistical foundations
Ability to read, implement, and extend research papers
Master's degree in Computer Science, Machine Learning, Statistics, or related quantitative field (PhD preferred)
Education & alternatives
Preferred - Master's degree in Computer Science, Machine Learning, Statistics, or related quantitative field (PhD preferred) - Publications in top ML/AI venues (NeurIPS, ICML, ICLR, ACL)
Posting record
Report an errorOpen 5 monthsWe first saw this posting 155 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
- Status in our records
- Live
- First seen by us
- Jun 2, 2026155 days ago
- Recorded sightings
- 73
- Last seen by us
- Nov 4, 2026
- Employer says posted
- Jan 14, 2026
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
How this job compares
Time since we first saw it
Day 155
Up longer than most. Similar jobs usually come down between day 6 and day 69.
From 4,362 similar postings that came down. Coming down is not always a hire.
bioscope%20ai is hiring
2 open jobs
Hiring recordSee how this role fits your experience
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
- Python
- PyTorch
Where the posting names them
- Experience with model evaluation, benchmarking, and quality metrics - Proficiency in Python and modern ML development tools - Strong statistical foundations
Required - Strong foundation in deep learning frameworks (PyTorch) and LLM architectures - Experience with model evaluation, benchmarking, and quality metrics
Your next step
Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.
Applying through Ashby
- No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.
Already applied? Track this application
About the role
We file this job under ML Engineer, by the work the posting describes.
ML engineers put machine learning models into products. They write the code that trains, ships, serves and monitors models so a recommendation or prediction works for real users.
Postings ask for strong Python and software engineering, a framework such as PyTorch, cloud services, and more and more often experience with large language models.
A data scientist often works out which model to build; an ML engineer makes it run reliably in production.
US government figures for computer and information research scientists
- Median yearly pay, May 2025
- $140,300
- Expected change, 2025 to 2035
- 21.8% more jobsagainst 3.5% for all jobs
The government has no code for ML engineers, so these figures are for the closest group, computer and information research scientists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1221.
Job description
About the Role
We're looking for a Data Scientist to own the quality, reliability, and trustworthiness of our clinical AI outputs. You'll build the systems that ensure our AI "knows what it doesn't know" — developing evaluation frameworks, calibrated confidence scoring, and automated quality assurance that physicians can actually trust.
What You'll Do
Design and implement automated evaluation pipelines that assess AI output quality, accuracy, and safety at scale
Develop uncertainty quantification systems where confidence scores meaningfully correlate with accuracy
Build comprehensive evaluation frameworks combining automated assessment with clinician-validated test cases
Implement feedback loops that continuously improve model outputs based on validation signals
Establish scalable quality gates that catch errors before they reach end users
Contribute to model alignment and fine-tuning efforts
Qualifications
Required
Strong foundation in deep learning frameworks (PyTorch) and LLM architectures
Experience with model evaluation, benchmarking, and quality metrics
Proficiency in Python and modern ML development tools
Strong statistical foundations
Ability to read, implement, and extend research papers
Excellent communication skills
Preferred
Master's degree in Computer Science, Machine Learning, Statistics, or related quantitative field (PhD preferred)
Publications in top ML/AI venues (NeurIPS, ICML, ICLR, ACL)
Experience with RLHF, DPO, or preference optimization techniques
Background in healthcare AI or regulated industries
Experience building evaluation systems for production LLM applications
Source details
Source notes
Source excerptsSelected passages from the saved posting. Check the full description for conditions and exceptions.
- Pay
No pay amount identified in the saved description.
- Location & working pattern
Boston, MA, United States
Working pattern and location restrictions need checking in the full posting.
- Work authorization
No clear work-authorization passage found. Eligibility is unconfirmed.