Back to jobs

Senior AI Engineer (Agentic Systems)

Reads like an ML Engineer role: training models and running them in production.

Pay
$140,000–180,000/yearAnnual period assumed. Location-specific pay — pay source
Compensation The salary range for this position is $140,000 - $180,000. This is not, however, a guarantee of compensation or salary. Rather, salary will be set based on experience, geographic location and possibly contractual requirements and could fall outside of this range. Benefits Overview
Read the full posting
Work setting
Working pattern needs review — work setting source
Applicants selected will be subject to a security investigation and may need to meet eligibility requirements. Candidates must have the ability to obtain and maintain a Secret clearance; an active Secret clearance is preferred. Work Location: Reston, VA - Remote Our company prioritizes the benefits of flexibility and collaboration, whether that happens in person or remotely.
Our company prioritizes the benefits of flexibility and collaboration, whether that happens in person or remotely. If the position is remote or hybrid, you may periodically work from a Pantheon Data office location or client site. If this position is assigned to a Pantheon Data office location or client site, you'll work with colleagues and clients in person, as needed for specific client requirements.
Read the full posting
Employment
Full-time — employment source
Employment type in the employer’s posting: FULL_TIME
From the employer’s posting

What you’ll work on

Full posting
  • Work Location: Reston, VA - Remote

From the employer’s posting
Applicants selected will be subject to a security investigation and may need to meet eligibility requirements. Candidates must have the ability to obtain and maintain a Secret clearance; an active Secret clearance is preferred. Work Location: Reston, VA - Remote Our company prioritizes the benefits of flexibility and collaboration, whether that happens in person or remotely.

Before you apply

Work authorization
Citizens onlyCandidates must have the ability to obtain and maintain a Secret clearance
Education & alternatives
Required Skills and Experience - Bachelor's degree in Computer Science, Engineering, or a related technical field from an accredited university, or equivalent professional experience. - 8+ years of professional software engineering experience, including 2+ years building production LLM-backed systems.
Apply at Pantheon Data

Posting record

Report an error
Status in our records
Live
First seen by us
Nov 3, 202614 days ago
Recorded sightings
15
Last seen by us
Nov 17, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Pay

Lower than most similar jobs

$140K–$180K against $166K–$237K, the middle half of what similar US postings state.

From 1,505 postings in the last year. Stated ranges, not salaries paid.

Time since we first saw it

Day 14

Similar jobs usually come down between day 7 and day 76.

At Pantheon Data, jobs usually come down by about day 81.

From 5,756 similar postings that came down, and 13 of Pantheon Data’s. Coming down is not always a hire.

Pantheon Data is hiring

3 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Takes jobs down at a usual pace
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • AWS
  • DynamoDB
  • Excel
  • Terraform
Where the posting names them
- Experience evaluating nondeterministic systems, including building test sets, automated evaluations, and regression checks for LLM behavior. - Strong Python engineering skills: readable, maintainable, tested code; debugging; packaging; and API and service design. - Working knowledge of AI security risks such as prompt injection, excessive tool permissions, and data exfiltration, and of controls that mitigate them.
Position Overview Pantheon Data is seeking a Senior AI Engineer (Agentic Systems) to design, build, and operate the agentic framework at the core of our AI platform in AWS GovCloud, serving workloads that process CUI, PII, and PHI. You will own the framework that routes requests to the right agents, composes new agents at runtime from a registry of capabilities, and coordinates their tools, state, and handoffs, so that these systems behave predictably, securely, and auditably under FedRAMP High and DoD IL4/IL5. This role owns the agent runtime: orchestration, routing, the capability registry, and the contracts that capabilities plug into. Capabilities such as retrieval, document extraction, and application integrations are built by machine learning and software engineers and exposed through the registry; you define how they are described, discovered, composed, and governed. You will partner closely with the Data Engineer, who owns the pipelines and data stores agents draw on; the Cloud Security Engineer, who owns cloud security controls and ATO evidence; and the DevSecOps and Cloud Operations Engineers, who own delivery and infrastructure. This is a systems engineering role, not a prompt-only or prototype role: successful candidates can walk through production agentic systems they have personally built, including the orchestration design, how routing decisions were made and evaluated, how failures were contained, and what they would change.
- Working knowledge of AI security risks such as prompt injection, excessive tool permissions, and data exfiltration, and of controls that mitigate them. - Experience with AWS services relevant to this work, such as Amazon Bedrock, Lambda, Step Functions, ECS or EKS, SQS, DynamoDB, and CloudWatch. - Experience with Git-based workflows, code review, CI/CD, and team-based software delivery.
Preferred Skills and Experience - Experience building or operating agentic systems in AWS GovCloud or other environments supporting FedRAMP High or DoD IL4/IL5 workloads. - Experience implementing or integrating MCP servers and clients, or agent-to-agent protocols, in production.
- Ability to meet deadlines and produce quality work. - Proficient in Microsoft Suite software including Outlook, Word, Excel, SharePoint, and PowerPoint. Preferred Skills and Experience
- Familiarity with federal compliance frameworks and their engineering implications: NIST SP 800-53, CUI handling, audit logging, and least-privilege access in regulated environments. - Experience with infrastructure as code (Terraform), containerized deployment, and GitLab CI or GitHub Actions in regulated environments. - Advanced degree in a technical discipline, or demonstrated contribution to team technical direction and mentorship of other engineers.

Your next step

Complete your application on pantheondata.isolvedhire.com. The employer’s form will show what is required.

Already applied? Track this application

Job description

View original posting ↗

Company Overview

Pantheon Data (a Kenific Holding company) is a private, small business based in the Washington, DC, area. Pantheon Data was founded in 2011, initially providing acquisition and supply chain management services to the US Coast Guard. Our service offerings have grown in the past ten years, including infrastructure resiliency, contact center operations, information technology, software engineering, program management, strategic communications, engineering, and cybersecurity. We have also grown our customer base to include commercial clients. The company has used this experience to expand our service offerings to other agencies within the Department of Homeland Security (DHS), the Department of Defense (DoD), and other Federal Civilian Agencies.

Position Overview

Pantheon Data is seeking a Senior AI Engineer (Agentic Systems) to design, build, and operate the agentic framework at the core of our AI platform in AWS GovCloud, serving workloads that process CUI, PII, and PHI. You will own the framework that routes requests to the right agents, composes new agents at runtime from a registry of capabilities, and coordinates their tools, state, and handoffs, so that these systems behave predictably, securely, and auditably under FedRAMP High and DoD IL4/IL5.

This role owns the agent runtime: orchestration, routing, the capability registry, and the contracts that capabilities plug into. Capabilities such as retrieval, document extraction, and application integrations are built by machine learning and software engineers and exposed through the registry; you define how they are described, discovered, composed, and governed. You will partner closely with the Data Engineer, who owns the pipelines and data stores agents draw on; the Cloud Security Engineer, who owns cloud security controls and ATO evidence; and the DevSecOps and Cloud Operations Engineers, who own delivery and infrastructure. This is a systems engineering role, not a prompt-only or prototype role: successful candidates can walk through production agentic systems they have personally built, including the orchestration design, how routing decisions were made and evaluated, how failures were contained, and what they would change.

Responsibilities

  • Agent Orchestration: Design and build the orchestration layer that plans, dispatches, and coordinates work across agents, including multi-step and multi-agent workflows, handoffs, parallel execution, cancellation, and human-in-the-loop checkpoints.
  • Routing: Build the routing that maps requests and intermediate steps to the right agents, tools, and models, combining deterministic rules, classifiers, and LLM-based routing where each is appropriate; measure routing accuracy and cost, and make routing decisions observable and explainable.
  • Capability Registry and Dynamic Agents: Own the capability registry, including schemas for describing capabilities, versioning, discovery, and access policy, and the mechanisms that assemble agents at runtime from registered capabilities, tools, and instructions.
  • Tool and Integration Contracts: Define the interfaces capabilities plug into, including tool definitions, Model Context Protocol (MCP) servers, input and output schemas, and error semantics, and help capability owners onboard to them.
  • State, Memory, and Reliability: Design agent state, session memory, and context management; build retries, timeouts, idempotency, loop and budget limits, and graceful degradation so agent failures are contained and recoverable.
  • Evaluation and Observability: Build evaluation harnesses for agent behavior, including routing accuracy, task completion, tool-call correctness, and regression across model and prompt versions, and trace every agent run end to end for debugging, cost, and latency.
  • Agent Security and Auditability: Enforce least-privilege tool access per agent and per request, contain prompt injection across tool and agent boundaries, and record a complete audit trail of agent decisions and actions; work with the Cloud Security Engineer to map these controls to NIST SP 800-53 and support ATO.
  • Technical Leadership: Set the architecture and standards for agentic development across the team, review designs and code, and mentor engineers building capabilities on the framework.

Required Skills and Experience

  • Bachelor's degree in Computer Science, Engineering, or a related technical field from an accredited university, or equivalent professional experience.
  • 8+ years of professional software engineering experience, including 2+ years building production LLM-backed systems.
  • Demonstrated experience designing and shipping a production agentic system: multi-step or multi-agent orchestration with tool use, beyond single-prompt chat applications or demos.
  • Hands-on experience with LLM tool calling, structured outputs, and at least one agent framework or SDK, such as LangGraph, Strands Agents, the Claude Agent SDK, Semantic Kernel, or a custom framework you built.
  • Strong software architecture skills: plugin or registry-based designs, interface and schema design, versioning, and extensibility.
  • Experience with distributed and asynchronous systems: queues, event-driven workflows, concurrency, retries, idempotency, and failure handling.
  • Experience evaluating nondeterministic systems, including building test sets, automated evaluations, and regression checks for LLM behavior.
  • Strong Python engineering skills: readable, maintainable, tested code; debugging; packaging; and API and service design.
  • Working knowledge of AI security risks such as prompt injection, excessive tool permissions, and data exfiltration, and of controls that mitigate them.
  • Experience with AWS services relevant to this work, such as Amazon Bedrock, Lambda, Step Functions, ECS or EKS, SQS, DynamoDB, and CloudWatch.
  • Experience with Git-based workflows, code review, CI/CD, and team-based software delivery.
  • Clear written and verbal communication, including explaining agent behavior, tradeoffs, and failure modes to technical and non-technical stakeholders.
  • Ability to work effectively remotely in cross-functional teams.
  • Ability to meet deadlines and produce quality work.
  • Proficient in Microsoft Suite software including Outlook, Word, Excel, SharePoint, and PowerPoint.

Preferred Skills and Experience

  • Experience building or operating agentic systems in AWS GovCloud or other environments supporting FedRAMP High or DoD IL4/IL5 workloads.
  • Experience implementing or integrating MCP servers and clients, or agent-to-agent protocols, in production.
  • Experience with Amazon Bedrock beyond basic inference: Agents, Guardrails, Knowledge Bases, AgentCore, provisioned throughput, and cross-model routing.
  • Experience building model routing or cascading strategies that balance quality, latency, and cost across multiple models.
  • Experience with agent observability and tracing tools such as OpenTelemetry, Langfuse, LangSmith, or Arize Phoenix.
  • Experience integrating retrieval, document processing, or other ML capabilities as tools within an agentic framework.
  • Familiarity with federal compliance frameworks and their engineering implications: NIST SP 800-53, CUI handling, audit logging, and least-privilege access in regulated environments.
  • Experience with infrastructure as code (Terraform), containerized deployment, and GitLab CI or GitHub Actions in regulated environments.
  • Advanced degree in a technical discipline, or demonstrated contribution to team technical direction and mentorship of other engineers.

Clearance Requirements

Applicants selected will be subject to a security investigation and may need to meet eligibility requirements. Candidates must have the ability to obtain and maintain a Secret clearance; an active Secret clearance is preferred.

Work Location: Reston, VA - Remote

  • Our company prioritizes the benefits of flexibility and collaboration, whether that happens in person or remotely.
  • If the position is remote or hybrid, you may periodically work from a Pantheon Data office location or client site.
  • If this position is assigned to a Pantheon Data office location or client site, you'll work with colleagues and clients in person, as needed for specific client requirements.

Interview Requirement: Candidates who are local to the area should be prepared to participate in an in-person interview as part of the selection process. Candidates outside the local area may be considered for a virtual interview.

Compensation

The salary range for this position is $140,000 - $180,000. This is not, however, a guarantee of compensation or salary. Rather, salary will be set based on experience, geographic location and possibly contractual requirements and could fall outside of this range.

Benefits Overview

We are always looking for good people! Pantheon Data is committed to providing its employees with competitive salaries and benefits in order to increase employee satisfaction and productivity. In addition to our benefits, we also offer SmartBenefits through the Washington Metro Area Transportation Authority, where you specify an amount of your pre-tax wages be paid directly to your SmarTrip account. In some cases, tuition assistance may be available for continuing education expenses and certifications related to their position. Additional details may be found at https://pantheon-data.com/careers/

Pantheon Data Important Information

All qualified applicants will be considered for employment without regard to disability, status as a protected veteran, or any other status protected by applicable federal, state, local, or international law.

As part of the application process, you are expected to be on camera during interviews and assessments. We reserve the right to take your picture to verify your identity and prevent fraud.

If you require reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employee selection process, please direct your inquiries to our Talent Team at Recruiting@pantheon-data.com or by phone (571) 363-4020.

This company uses E-Verify to confirm each employee's work authorization. For more information, click here E-Verify Participation Poster



Demand for ML Engineer roles

New ML Engineer postings at the employers we follow are up 12% over the last 6 weeks compared with the 6 before.

Weekly new ML Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 556, the most 1,080, and the latest week had 556.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under ML Engineer, by the work the posting describes.

ML engineers put machine learning models into products. They write the code that trains, ships, serves and monitors models so a recommendation or prediction works for real users.

Postings ask for strong Python and software engineering, a framework such as PyTorch, cloud services, and more and more often experience with large language models.

A data scientist often works out which model to build; an ML engineer makes it run reliably in production.

US government figures for computer and information research scientists

Median yearly pay, May 2025
$140,300
Expected change, 2025 to 2035
21.8% more jobsagainst 3.5% for all jobs

The government has no code for ML engineers, so these figures are for the closest group, computer and information research scientists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1221.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay
Compensation The salary range for this position is $140,000 - $180,000. This is not, however, a guarantee of compensation or salary. Rather, salary will be set based on experience, geographic location and possibly contractual requirements and could fall outside of this range. Benefits Overview
Location & working pattern

Reston, VA, US

Applicants selected will be subject to a security investigation and may need to meet eligibility requirements. Candidates must have the ability to obtain and maintain a Secret clearance; an active Secret clearance is preferred. Work Location: Reston, VA - Remote - Our company prioritizes the benefits of flexibility and collaboration, whether that happens in person or remotely. - If the position is remote or hybrid, you may periodically work from a Pantheon Data office location or client site. - If this position is assigned to a Pantheon Data office location or client site, you'll work with colleagues and clients in person, as needed for specific client requirements.
Work authorization
If you require reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employee selection process, please direct your inquiries to our Talent Team at Recruiting@pantheon-data.com or by phone (571) 363-4020. This company uses E-Verify to confirm each employee's work authorization. For more information, click here E-Verify Participation Poster

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.