Back to jobs

Staff Applied AI Engineer, Product & Agent Performance

Pay
USD 175,000–200,000/year — pay source
Salary 175,000 – 200,000 USD per year About Arcadia
Read the full posting
Work setting
Remote — work setting source
Listed location: Remote (USA)
Work setup in the employer’s posting: remote
From the employer’s posting
Employment
Full-time — employment source
Employment type Full-time
Employment type in the employer’s posting: Full-time
From the employer’s posting
Education & alternatives
What You'll Bring - Bachelors degree in Engineering or related field and/or masters of Data Science or related field - 4+ years of production software engineering experience, including 3+ years of hands-on ownership of ML, LLM, or agentic systems in production, with direct experience in healthcare, finance, or another regulated industry

About Arcadia

Arcadia.io helps innovative providers and payers across the country transform healthcare to reduce cost while improving patient health.

In the employer’s words · Read in context

Apply at Arcadia

Posting record

Report an error

Open 3 monthsWe first saw this posting 91 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.

Status in our records
Live
First seen by us
Aug 22, 202691 days ago
Recorded sightings
49
Last seen by us
Nov 21, 2026
Employer says posted
Aug 20, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 91

Up longer than most. Similar jobs usually come down between day 1 and day 15.

From 1,368 similar postings that came down. Coming down is not always a hire.

Arcadia is hiring

4 open jobs

  • No new data jobs since August
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • AWS
  • SageMaker
Where the posting names them
- Hands-on experience with RAG architecture, production-grounded evaluation frameworks, and fallback or human-in-the-loop logic for automated systems - Working familiarity with AWS AI/ML services, including Bedrock and SageMaker, sufficient to build and evaluate effectively in Arcadia’s environment - Evidence-led judgment and the credibility to push back on launch decisions, paired with a builder’s instinct to run the experiment and move from a production failure to a fix

Your next step

Complete your application on jobs.lever.co. The employer’s form will show what is required.

Applying through Lever

  • No account needed: one form, and Lever reads your resume into the profile recruiters search.

Already applied? Track this application

Job description

View original posting ↗

Arcadia is the most trusted healthcare platform powering outcomes. We transform complex healthcare data into trusted intelligence, helping providers, payers, and life sciences organizations act with clarity, make confident decisions, and achieve measurable clinical, operational, and financial outcomes. 

Built on a comprehensive data foundation spanning tens of millions of patient lives, Arcadia combines advanced analytics and responsible AI to surface meaningful insights, coordinate action, and improve performance at scale. Our approach to AI and automation is governed and transparent — designed to strengthen human expertise, not replace judgment or obscure responsibility. 

Hundreds of organizations rely on Arcadia to improve cost, quality, and outcomes. Backed by Nordic Capital, we continue to invest in our platform, AI capabilities, and people as we pursue our purpose: helping healthcare deliver better outcomes for every person, every community, and every generation. 

 

Why This Role Is Important to Arcadia

Arcadia’s data and analytics platform is used by hundreds of health systems, ACOs, payers, and life sciences organizations, touching tens of millions of patient lives. This role owns how our agentic capabilities perform at that same scale: accurate, transparent about their own confidence, and safe for the clinicians, care teams, and patients who depend on them. 

As a staff-level individual contributor, you will own the product-layer decisions that shape agent behavior, including prompting, retrieval and context, memory and state, evaluation, and escalation, while partnering with Product and Engineering on the systems that support them. Your work will help Arcadia make evidence-based launch decisions and scale responsible AI that is steerable, trustworthy, and ready for real healthcare workflows. 

 
What Success Looks Like
In 3 months
  • You have established a production-grounded baseline for priority agentic workflows, with documented failure modes, severity-weighted evaluation rubrics, and a clear measurement plan
  • You have mapped the current retrieval, context, memory, and escalation patterns and identified the highest-value opportunities to improve reliability, calibration, and cost
  • You have earned trust across Product and Engineering by turning production evidence into clear, actionable recommendations

In 6 months

  • Production-representative evaluation suites and regression checks inform model-change decisions for priority agentic workflows
  • You have delivered measurable improvements in accuracy, reliability, steerability, latency, or cost for one or more priority workflows
  • Human-review and escalation behavior has been validated under adversarial and edge-case conditions, with decision criteria and ownership boundaries clearly documented

In 12 months 

  • Arcadia has a repeatable product-layer AI performance practice that moves from production failure to diagnosis, experiment, evaluation, and release decision
  • High-severity regressions are caught earlier, and agent behavior is more transparent, calibrated, and trustworthy at scale
  • Model cards, intended-use guidance, limitations, and performance documentation are current and useful to product and customer-facing teams

What You'll Be Doing

  • Design and iterate on agent behavior across real, live workflows, including long-horizon, multi-turn agentic tasks
  • Design retrieval and context architecture so the right source data reaches a model in the right structure and agents remain grounded in real data rather than filling gaps with assumptions
  • Design memory and state handling across multi-turn and multi-agent flows, determining what is carried forward, summarized, or dropped and why
  • Create context and prompt templates that combine few-shot examples, structured formatting, and reasoning scaffolding for consistent agent behavior
  • Improve performance through prompting, tool-use strategy, and context construction, validated through direct experimentation rather than guesswork
  • Build and run evaluations against real production conditions to measure performance, regressions, failure modes, and edge cases
  • Author evaluation rubrics, quality heuristics, and thresholds that weight failures by severity and cost, not just frequency, and monitor those measures against production behavior
  • Design and validate escalation paths that route agents to human review based on confidence and uncertainty while preserving safety and consistency under adversarial and edge-case conditions
  • Design for cost-aware performance alongside latency, reliability, and accuracy through efficient context construction and tool-call economy
  • Evaluate and sign off on model changes by baselining current behavior, running comparative evaluations, and making the go/no-go call before a change reaches a customer
  • Maintain product-level AI documentation, including model cards, intended use, limitations, and known failure modes, so customer-facing teams work from actual agent behavior
  • Partner closely with Product and product managers to ensure agents are not just capable, but steerable, trustworthy, and ready to scale 

What You'll Bring

  • Bachelors degree in Engineering or related field and/or masters of Data Science or related field
  • 4+ years of production software engineering experience, including 3+ years of hands-on ownership of ML, LLM, or agentic systems in production, with direct experience in healthcare, finance, or another regulated industry
  • Demonstrated ability to diagnose why an agent failed, correctly attribute the fix to instruction, retrieval, context, or memory design, and weigh failures by severity and cost rather than frequency alone
  • Hands-on experience with RAG architecture, production-grounded evaluation frameworks, and fallback or human-in-the-loop logic for automated systems
  • Working familiarity with AWS AI/ML services, including Bedrock and SageMaker, sufficient to build and evaluate effectively in Arcadia’s environment
  • Evidence-led judgment and the credibility to push back on launch decisions, paired with a builder’s instinct to run the experiment and move from a production failure to a fix

Would Love for You to Have

  • Experience applying AI to healthcare data or workflows where safety, transparency, and calibrated uncertainty directly affect care teams or patients
  • Experience with long-horizon, multi-turn or multi-agent workflows and product-level AI documentation such as model cards 

What You'll Get

  • The opportunity to define how agent performance, safety, and readiness are measured for production healthcare workflows
  • Meaningful ownership across prompts, context, memory, evaluations, and escalation patterns at product scale
  • A cross-functional role translating production evidence into AI improvements used across Arcadia’s platform
  • A mission-driven company working to improve how patients receive care
  • A flexible, remote-friendly culture with personality and heart
  • Employee-driven programs and initiatives for personal and professional development
  • Membership in the talented, energized, diverse, and purpose-driven Arcadian community 

Salary

175,000 – 200,000 USD per year

About Arcadia
Arcadia.io helps innovative providers and payers across the country transform healthcare to reduce cost while improving patient health. We do this by aggregating large amounts of disparate data, applying algorithms to identify opportunities to provide better patient care, and making those opportunities actionable by physicians at the point of care in near-real time. We are passionate about helping our customers drive meaningful outcomes. We are growing fast and have emerged as a market leader in the highly competitive population health management software market and have been recognized by industry analysts KLAS, IDC, Forrester, and Chilmark for our leadership. For a better sense of our brand and products, please explore our website.

Protect Yourself
If you have concerns about the authenticity of a job offer or recruitment-related communication claiming to be from Arcadia, we encourage you to verify by contacting us directly at (781) 202-3600 and select option 3. For more information, visit our website.

This position is responsible for following all Security policies and procedures in order to protect all PHI under Arcadia's custodianship as well as Arcadia Intellectual Properties.  For any security-specific roles, the responsibilities would be further defined by the hiring manager.

Employment type

Full-time

Demand for AI Engineer roles

New AI Engineer postings at the employers we follow are up 340% over the last 6 weeks compared with the 6 before.

Weekly new AI Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 78, the most 803, and the latest week had 438.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under AI Engineer, by the work the posting describes.

AI engineers build products on top of AI models that already exist, most often large language models: they connect a model to a company's data and tools and ship the assistants, agents and search features people use.

Postings ask for strong Python or TypeScript, experience with model APIs, retrieval and agent frameworks, and ways to test whether the model's answers are good enough.

An ML engineer trains and serves models; an AI engineer usually starts from a model someone else trained and builds the product around it, though titles often mix the two.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay
Salary 175,000 – 200,000 USD per year About Arcadia
Location & working pattern

Remote (USA)

- A mission-driven company working to improve how patients receive care - A flexible, remote-friendly culture with personality and heart - Employee-driven programs and initiatives for personal and professional development
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.