Pay
$166K–$237K is typical
The middle half of what similar US postings state. This one doesn’t list pay.
From 1,537 postings in the last year. Stated ranges, not salaries paid.
Reads like an ML Engineer role: training models and running them in production.
United States
Open 4 months, first seen by us 143 days agoLocation type: RemoteFrom the employer’s posting
Employment type: Full-timeFrom the employer’s posting
Department: EngineeringFrom the employer’s posting
You’ll build benchmarks, datasets, automated evaluations, regression testing pipelines, and observability systems that help us continuously improve agent performance.
How do we measure reliability, trustworthiness, and business impact? You’ll build benchmarks, datasets, automated evaluations, regression testing pipelines, and observability systems that help us continuously improve agent performance. What you’ll bring to the team:
Open 4 monthsWe first saw this posting 143 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
$166K–$237K is typical
The middle half of what similar US postings state. This one doesn’t list pay.
From 1,537 postings in the last year. Stated ranges, not salaries paid.
Day 143
Up longer than most. Similar jobs usually come down between day 7 and day 76.
From 6,389 similar postings that came down. Coming down is not always a hire.
2 open jobs
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
What you’ll bring to the team: - Strong programming skills, preferably in Python. - Solid software engineering fundamentals, including testing, debugging, and system design.
Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.
Already applied? Track this application
Crogl
Fully Autonomous AI Team Member for Your SOC
Crogl is building AI-powered systems that help security teams investigate, understand, and respond to threats faster. We combine advances in large language models, agent architectures, and security expertise to create intelligent systems that solve real-world problems for security practitioners.
We’re looking for AI Engineers who are excited about building practical AI products. You’ll help design, evaluate, and improve agentic systems that operate in production, working closely with customers, engineers, and product teams to push the boundaries of what’s possible with modern AI.
What you’ll be doing:
Build LLM-powered features, workflows, and agentic systems that solve real customer problems.
Design and implement evaluation frameworks to measure agent quality, reliability, and business impact.
Create automated benchmarks, regression tests, and datasets for evaluating AI behavior.
Investigate agent failures and develop systematic approaches to improve performance.
Experiment with prompting, tool use, retrieval, memory, planning, and reasoning strategies.
Build infrastructure that supports rapid experimentation, evaluation, deployment, and monitoring.
Work closely with customers and internal teams to understand workflows and identify opportunities for AI automation.
Contribute to engineering best practices for testing, observability, and production reliability.
Stay current with advances in LLMs, agents, evaluation methodologies, and AI engineering.
A core part of this role: evaluating AI systems
Building agents is only half the challenge. Understanding whether they are actually improving is equally important. A significant portion of this role involves designing and maintaining evaluation systems for AI agents operating in real-world security workflows.
You’ll help answer questions like:
Is the agent producing accurate investigations?
Are changes to prompts, tools, or models actually improving outcomes?
How do we detect regressions before customers experience them?
Which failure modes matter most?
How do we measure reliability, trustworthiness, and business impact?
You’ll build benchmarks, datasets, automated evaluations, regression testing pipelines, and observability systems that help us continuously improve agent performance.
What you’ll bring to the team:
Strong programming skills, preferably in Python.
Solid software engineering fundamentals, including testing, debugging, and system design.
Experience building applications, projects, or products using LLMs and modern AI tools.
Ability to design experiments, interpret results, and make data-driven decisions.
Strong communication skills and willingness to collaborate across disciplines.
Curiosity, ownership, and a desire to learn quickly.
What makes you stand out from others:
Experience building AI agents, copilots, or workflow automation systems.
Experience designing evaluations, benchmarks, or testing frameworks for AI systems.
Familiarity with OpenAI, Anthropic, Gemini, or open-source LLM ecosystems.
Experience with retrieval systems, vector databases, and RAG architectures.
Familiarity with LangGraph, OpenAI Agents SDK, MCP, or similar agent frameworks.
Experience with observability, tracing, and production monitoring for AI systems.
Exposure to cybersecurity, security operations, or developer tooling.
Open-source contributions, research projects, or personal AI products.
Example projects:
Building evaluation suites that measure the quality of AI-driven security investigations.
Creating automated regression tests to detect model and prompt regressions.
Improving agent reasoning, planning, and tool-use capabilities.
Developing datasets and benchmarks that reflect real customer workflows.
Building systems that automatically identify and categorize agent failures.
Designing feedback loops that continuously improve production agent performance.
Who thrives at Crogl:
The strongest candidates for this role are usually builders.
You may have experience as a software engineer, ML engineer, researcher, AI engineer, or founder. What matters most is your ability to learn quickly, work independently, and ship impactful systems. We’re particularly excited about candidates who have:
Built and deployed AI agents.
Created evaluation frameworks for LLM applications.
Published AI projects, demos, or open-source contributions.
Developed tools that other people actively use.
Strong opinions about what makes AI systems reliable and useful.
If you’ve spent nights and weekends building AI products because you genuinely enjoy it, you’ll probably fit in well here.
What we value:
We care more about demonstrated ability than specific credentials.
You might be a great fit if you’ve:
Built AI products, agents, or developer tools that people actually use.
Created evaluation frameworks, benchmarks, or testing systems for LLM applications.
Contributed to open-source AI projects.
Conducted independent research or published technical writing.
Built ambitious side projects and iterated on them based on real-world feedback.
Shown exceptional curiosity and a track record of learning quickly.
We’re especially interested in candidates who can show us what they’ve built. GitHub repositories, demos, blog posts, open-source contributions, benchmarks, evaluation frameworks, AI products, and side projects are all valuable signals.
We don’t expect candidates to check every box. If you’re excited about the role and believe you can contribute, we’d encourage you to apply.
What success looks like:
Within your first six months, you will:
Ship improvements to production AI systems used by customers.
Help establish rigorous evaluation practices across the company.
Contribute new ideas that improve agent performance and reliability.
Develop a strong understanding of customer workflows and security investigations.
Become a trusted contributor across engineering, product, and AI initiatives.
Why Crogl?
Work on some of the most challenging problems in applied AI.
Help define how agentic systems are evaluated and deployed in production.
Join a small, highly collaborative team with significant ownership and impact.
Learn quickly while working alongside experienced engineers, researchers, and security experts.
Shape the future of AI-powered security operations.
New ML Engineer postings at the employers we follow are down 4% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under ML Engineer, by the work the posting describes.
ML engineers put machine learning models into products. They write the code that trains, ships, serves and monitors models so a recommendation or prediction works for real users.
Postings ask for strong Python and software engineering, a framework such as PyTorch, cloud services, and more and more often experience with large language models.
A data scientist often works out which model to build; an ML engineer makes it run reliably in production.
The government has no code for ML engineers, so these figures are for the closest group, computer and information research scientists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1221.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
United States
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.