Time since we first saw it
Day 3
Early. Similar jobs usually come down between day 7 and day 76.
From 6,398 similar postings that came down. Coming down is not always a hire.
Porto, Portugal
Actively hiring, first seen by us 3 days agoEmployment type in the employer’s posting: FullTimeFrom the employer’s posting
Design and curate golden sets, labeled data and synthetic data for agent behaviors and search ranking
Build scalable, reproducible pipelines for offline evals, regression testing and benchmarking
Define metrics for real user value, including LLM-as-judge and human-eval approaches
Develop and deploy ML models for search ranking and proactive suggestions
Connect offline evals to online experiments and real-world signals to catch regressions
Design, source, and curate golden sets, labeled data, and synthetic data for agent behaviors, search ranking, and proactive suggestions.
Create scalable, reproducible pipelines for running offline evaluations, regression testing, and benchmarking across models, prompts, and agent configurations.
Define metrics that reflect real user value, from classic ML metrics (precision/recall, NDCG, calibration) to LLM-as-judge and human-eval approaches.
Develop and deploy ML models for core product experiences, including search ranking and proactive suggestions that anticipate user needs
Connect offline evals to online experiments and real-world signals, so we catch regressions early
5+ years of industry experience as a machine learning engineer or applied research scientist.
Hands-on experience building evaluation datasets and the tooling and infrastructure to run evals at scale (data pipelines, labeling workflows, benchmarking frameworks).
Experience using evaluation results to measure and improve model or product quality, whether for ranking, classification, or LLM-based systems.
Strong Python and data engineering skills, with experience in reproducible, production-grade ML pipelines.
Ability to work independently with minimal guidance, manage priorities across multiple projects, and collaborate effectively with cross-functional teams in a fast-paced, results-driven environment.
- Ability to work independently with minimal guidance, manage priorities across multiple projects, and collaborate effectively with cross-functional teams in a fast-paced, results-driven environment. - Nice to have: experience with LLM-as-judge, agent or workflow evaluation, or search/ranking systems; MS/PhD in Computer Science or a related field. Compensation and Benefits
From the employer’s posting.
Superhuman is an AI productivity suite that makes AI work for everyone.
In the employer’s words · Read in context
Actively hiringFirst seen by us 3 days ago, still listed, and not a repost of an earlier posting.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 3
Early. Similar jobs usually come down between day 7 and day 76.
From 6,398 similar postings that came down. Coming down is not always a hire.
13 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Hiring recordAdd your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
- Experience using evaluation results to measure and improve model or product quality, whether for ranking, classification, or LLM-based systems. - Strong Python and data engineering skills, with experience in reproducible, production-grade ML pipelines. - Deep care about the end-user experience and a drive to ensure high quality.
Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.
Already applied? Track this application
Yes. We’ve followed one other Machine Learning Engineer job there since November 2026. It came down after 2 days.
Superhuman offers a dynamic hybrid working model for this role. This flexible approach gives team members the best of both worlds: plenty of focus time along with in-person collaboration that helps foster trust, innovation, and a strong team culture.
Superhuman is an AI productivity suite that makes AI work for everyone. Over 40 million people rely on it daily to get work done. A recognized leader in productivity tools, Superhuman is the maker of popular products including Docs, Fathom, Go, GPTZero, Grammarly, and Mail. The newest product, Superhuman Go, builds on Grammarly’s ability to deliver help everywhere people work. It lets anyone build proactive agents that act as teammates across more than 1 million apps and websites, without changing how they work. Learn more at superhuman.com.
We’re looking for a foundational Machine Learning Engineer to join our team in Porto and build the evaluation foundation behind the next generation of AI-native productivity at Superhuman, specifically our agent platform, where multiple AI agents collaborate to solve complex user problems. You’ll own how we define, measure, and improve quality. You’ll build the eval datasets, the infrastructure around them, and the metrics that tell us whether our models and agents are actually getting better for users.
This is a high-autonomy, high-impact role on a foundational team that moves fast from idea to production. As Superhuman Docs’ collaborative workspaces evolve so users can solve arbitrarily complex tasks with agents, rigorous measurement will be what lets us ship with confidence.
In this role, you will:
Build high-quality evaluation datasets: Design, source, and curate golden sets, labeled data, and synthetic data for agent behaviors, search ranking, and proactive suggestions. Define annotation guidelines, manage labeling workflows, and ensure datasets are representative, versioned, and trustworthy.
Build the eval infrastructure: Create scalable, reproducible pipelines for running offline evaluations, regression testing, and benchmarking across models, prompts, and agent configurations. Make it easy for every team to run evals and trust the results.
Measure and improve quality: Define metrics that reflect real user value, from classic ML metrics (precision/recall, NDCG, calibration) to LLM-as-judge and human-eval approaches. Validate that automated metrics correlate with user outcomes, and use the results to drive model and product decisions.
Own classic ML systems: Develop and deploy ML models for core product experiences, including search ranking and proactive suggestions that anticipate user needs, with evaluation built in from day one.
Support the agent platform: Partner with the team building orchestration, routing, and planning to define success criteria and quality gates for multi-agent workflows.
Close the loop with production: Connect offline evals to online experiments and real-world signals, so we catch regressions early and learn from what users actually experience.
Ship fast: Move quickly from idea to impact, with no red tape or endless approvals, on a platform used by millions.
Qualifications
5+ years of industry experience as a machine learning engineer or applied research scientist.
Strong foundation in classic ML and NLP, including experimental design, statistics, and metric design, plus working knowledge of modern LLM approaches.
Hands-on experience building evaluation datasets and the tooling and infrastructure to run evals at scale (data pipelines, labeling workflows, benchmarking frameworks).
Experience using evaluation results to measure and improve model or product quality, whether for ranking, classification, or LLM-based systems.
Strong Python and data engineering skills, with experience in reproducible, production-grade ML pipelines.
Deep care about the end-user experience and a drive to ensure high quality.
Ability to work independently with minimal guidance, manage priorities across multiple projects, and collaborate effectively with cross-functional teams in a fast-paced, results-driven environment.
Nice to have: experience with LLM-as-judge, agent or workflow evaluation, or search/ranking systems; MS/PhD in Computer Science or a related field.
Professional Growth: We believe that autonomy and trust are key to empowering our team members to do their best, most innovative work in a way that aligns with their interests, talents, and well-being
A connected team: Superhuman builds a product that helps people connect, and we apply this mindset to our own team. Our remote-first, hybrid model enables a highly collaborative culture rooted in our values. We work to foster belonging among team members in a variety of ways. This includes our team member resource groups, Superhuman Circles, which promote connection among those with shared identities including BIPOC and LGBTQIA+ team members, women, and parents. We also celebrate our colleagues and accomplishments with global, local, and team-specific programs.
Comprehensive benefits for candidates based in Portugal: Superhuman offers all team members competitive pay along with a benefits package encompassing life care (including mental health care and risk benefits) and ample and defined time off. We also offer support to set up a home office, wellness and pet care stipends, learning and development opportunities, and more.
Relocation Support: Superhuman provides relocation support to make your move to Porto seamless. Our package includes visa assistance, destination services to help you and your family settle in comfortably, and a relocation bonus to cover additional expenses, such as temporary housing.
At Superhuman, we value our differences, and we encourage all—especially those whose identities are traditionally underrepresented in tech organizations—to apply. Superhuman is an equal opportunity company. We do not discriminate on the basis of race or ethnic origin, religion or belief, color, gender expression or identity, sexual orientation, sexual identity, national origin, citizenship, age, marital status, veteran status, disability status, criminal prosecution, judgment in a criminal case, or any other characteristic protected by law.
For more details about the personal data Superhuman collects during the recruitment process, for what purposes, and how you can address your rights, please see the Superhuman Data Privacy Notice for Candidates here.
New ML Engineer postings at the employers we follow are down 4% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under ML Engineer, by the work the posting describes.
ML engineers put machine learning models into products. They write the code that trains, ships, serves and monitors models so a recommendation or prediction works for real users.
Postings ask for strong Python and software engineering, a framework such as PyTorch, cloud services, and more and more often experience with large language models.
A data scientist often works out which model to build; an ML engineer makes it run reliably in production.
The government has no code for ML engineers, so these figures are for the closest group, computer and information research scientists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1221.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
Porto, Portugal
Superhuman offers a dynamic hybrid working model for this role. This flexible approach gives team members the best of both worlds: plenty of focus time along with in-person collaboration that helps foster trust, innovation, and a strong team culture. About Superhuman
- Professional Growth: We believe that autonomy and trust are key to empowering our team members to do their best, most innovative work in a way that aligns with their interests, talents, and well-being - A connected team: Superhuman builds a product that helps people connect, and we apply this mindset to our own team. Our remote-first, hybrid model enables a highly collaborative culture rooted in our values. We work to foster belonging among team members in a variety of ways. This includes our team member resource groups, Superhuman Circles, which promote connection among those with shared identities including BIPOC and LGBTQIA+ team members, women, and parents. We also celebrate our colleagues and accomplishments with global, local, and team-specific programs. - Comprehensive benefits for candidates based in Portugal: Superhuman offers all team members competitive pay along with a benefits package encompassing life care (including mental health care and risk benefits) and ample and defined time off. We also offer support to set up a home office, wellness and pet care stipends, learning and development opportunities, and more.
This passage needs a closer read in the full description.