Back to jobs

Senior AI Reliability Engineer

Pay
Pay not listed
Work setting
Remote — work setting source
Work setup in the employer’s posting: TELECOMMUTE
From the employer’s posting
Employment
Not stated

Tools in this posting

  • Python
  • AWS
  • Azure
  • Databricks
  • Datadog
  • GCP
  • Grafana
  • Kubernetes
  • MLflow
  • Terraform
Source — Tool mentions in context
What You'll Get A brand-new discipline within EPAM where you shape the approach rather than follow an existing playbook Focus on engineering work without 24/7 on-call responsibilities Sponsored certification and training programs (Anthropic/Claude, Databricks, AI & Data Observability learning paths) Exposure to multiple clients and a direct route into presales and solution engineering Responsibilities Add AI telemetry to production LLM, RAG, and agentic applications using OpenTelemetry and APM-native AI monitoring tools Deploy distributed tracing across multi-model chains, agent workflows, and retrieval-augmented generation pipelines to identify systemic latency and failure points Establish and track AI-native SLIs and SLOs, including time to first token (TTFT), throughput, error and refusal rates, cost per request, semantic drift, hallucination boundaries, and contextual accuracy Implement structured semantic logging and prompt/response monitoring to support quality analysis Create and maintain evaluation loops for output quality and safety using golden sets, LLM-as-a-judge methods, and Ragas/DeepEval-style frameworks, integrating them into CI/CD and runtime environments Monitor token-based cloud spend, model API rate limits, and quota usage while driving AI cost optimization efforts Set up AI gateways to manage API load balancing, failover, and fallback models across multiple LLM providers Build guardrails covering prompt injection and jailbreak filtering, output compliance, and bias and safety constraints Develop detection, triage, restoration, and problem management workflows for AI incidents, incorporating autonomous AI agents into root cause analysis to parse logs, generate hypotheses, and correlate state changes Enable rollback, canary, and fail-safe patterns for model, prompt, and configuration releases, while maintaining reproducibility through versioning of data, code, prompts, and models Develop practice accelerators, reference architectures, and internal training materials, and contribute to presales activities and client assessments Requirements 4+ years of experience in SRE, DevOps, platform, or observability engineering, with hands-on exposure to production AI/ML or LLM workloads Strong grasp of SRE fundamentals, including Golden Signals, SLI/SLO definition, error budgets and burn rate, incident lifecycle, and ITIL basics Strong Python skills for building instrumentation, automation, and evaluation tools Hands-on experience with OpenTelemetry and at least one APM/observability platform such as New Relic, Datadog, Grafana LGTM stack, Splunk, or Elastic Production experience with at least one cloud platform (Azure preferred, AWS or GCP also acceptable) and Kubernetes Solid understanding of LLM application architecture, including prompts, embeddings and vector stores, RAG, and agent orchestration tools like LangChain/LangGraph or similar Awareness of MLOps concepts, including model lifecycle (training vs.
inference), model endpoints, containerization, and deployment/rollback patterns Experience with Infrastructure as Code using Terraform, along with CI/CD tools such as Azure DevOps, GitLab CI, or GitHub Actions B2+ English proficiency, since the role involves direct client interaction and requires clear technical communication in both writing and speech Nice to have Familiarity with AI-specific observability and evaluation tools such as Traceloop/OpenLLMetry, Langfuse, Arize Phoenix, Ragas, DeepEval, or MLflow Experience with distributed inference serving at scale, including vLLM, KServe, Ray Serve, Kubernetes-native LLM orchestration, and GPU capacity planning Knowledge of AI security practices, including the OWASP LLM Top 10, prompt injection defense, and guardrail frameworks like NeMo Guardrails or Llama Guard Experience with Databricks (including Mosaic AI/MLflow) or Azure AI Foundry Relevant certifications such as Anthropic Claude, Azure AI Engineer, AWS ML Specialty, or Databricks GenAI Background in FinOps for AI workloads, particularly token and GPU cost modeling Experience in Data Reliability Engineering, covering data quality and pipeline SLOs, since AI reliability depends on data reliability Prior mentoring or team lead experience
Check the employer’s page ↗

Posting record

Report an error
Status in our records
Live
First seen by us
Nov 1, 202620 days ago
Recorded sightings
20
Last seen by us
Nov 21, 2026
Employer says posted
Sep 29, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 20

Up longer than most. Similar jobs usually come down between day 1 and day 15.

From 1,376 similar postings that came down. Coming down is not always a hire.

EPAM Systems is hiring

892 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

Your next step

Check the employer’s posting for the current role and application details.

Already applied? Track this application

Has EPAM Systems posted this job before?

Yes. We’ve followed 2 other Senior AI Reliability Engineer jobs there since November 2026. All 2 are still up.

  1. Place not statedStill up after 20 days, first seen Nov 1
  2. Place not statedStill up after 20 days, first seen Nov 1
  3. This jobUp 20 days so far, first seen Nov 1

Job description

View original posting ↗

EPAM's Operational Intelligence practice is developing a new capability called AI Reliability Engineering (AIRE) , which applies SRE principles and cloud-native practices throughout the lifecycle of production AI/ML and LLM systems. As clients transition their GenAI applications and agentic systems from pilot to production, they find that traditional APM provides no visibility into token latency, cost per request, semantic drift, or hallucinations. This role exists to close that gap. Your work will involve instrumenting, monitoring, and hardening production AI systems, establishing AI-native service level objectives, and creating accelerators and reference implementations that the practice can reuse across accounts. Note that this is an engineering position rather than an L1/L2 support role, and it does not require a 24/7 on-call rotation. What You'll Get A brand-new discipline within EPAM where you shape the approach rather than follow an existing playbook Focus on engineering work without 24/7 on-call responsibilities Sponsored certification and training programs (Anthropic/Claude, Databricks, AI & Data Observability learning paths) Exposure to multiple clients and a direct route into presales and solution engineering Responsibilities Add AI telemetry to production LLM, RAG, and agentic applications using OpenTelemetry and APM-native AI monitoring tools Deploy distributed tracing across multi-model chains, agent workflows, and retrieval-augmented generation pipelines to identify systemic latency and failure points Establish and track AI-native SLIs and SLOs, including time to first token (TTFT), throughput, error and refusal rates, cost per request, semantic drift, hallucination boundaries, and contextual accuracy Implement structured semantic logging and prompt/response monitoring to support quality analysis Create and maintain evaluation loops for output quality and safety using golden sets, LLM-as-a-judge methods, and Ragas/DeepEval-style frameworks, integrating them into CI/CD and runtime environments Monitor token-based cloud spend, model API rate limits, and quota usage while driving AI cost optimization efforts Set up AI gateways to manage API load balancing, failover, and fallback models across multiple LLM providers Build guardrails covering prompt injection and jailbreak filtering, output compliance, and bias and safety constraints Develop detection, triage, restoration, and problem management workflows for AI incidents, incorporating autonomous AI agents into root cause analysis to parse logs, generate hypotheses, and correlate state changes Enable rollback, canary, and fail-safe patterns for model, prompt, and configuration releases, while maintaining reproducibility through versioning of data, code, prompts, and models Develop practice accelerators, reference architectures, and internal training materials, and contribute to presales activities and client assessments Requirements 4+ years of experience in SRE, DevOps, platform, or observability engineering, with hands-on exposure to production AI/ML or LLM workloads Strong grasp of SRE fundamentals, including Golden Signals, SLI/SLO definition, error budgets and burn rate, incident lifecycle, and ITIL basics Strong Python skills for building instrumentation, automation, and evaluation tools Hands-on experience with OpenTelemetry and at least one APM/observability platform such as New Relic, Datadog, Grafana LGTM stack, Splunk, or Elastic Production experience with at least one cloud platform (Azure preferred, AWS or GCP also acceptable) and Kubernetes Solid understanding of LLM application architecture, including prompts, embeddings and vector stores, RAG, and agent orchestration tools like LangChain/LangGraph or similar Awareness of MLOps concepts, including model lifecycle (training vs. inference), model endpoints, containerization, and deployment/rollback patterns Experience with Infrastructure as Code using Terraform, along with CI/CD tools such as Azure DevOps, GitLab CI, or GitHub Actions B2+ English proficiency, since the role involves direct client interaction and requires clear technical communication in both writing and speech Nice to have Familiarity with AI-specific observability and evaluation tools such as Traceloop/OpenLLMetry, Langfuse, Arize Phoenix, Ragas, DeepEval, or MLflow Experience with distributed inference serving at scale, including vLLM, KServe, Ray Serve, Kubernetes-native LLM orchestration, and GPU capacity planning Knowledge of AI security practices, including the OWASP LLM Top 10, prompt injection defense, and guardrail frameworks like NeMo Guardrails or Llama Guard Experience with Databricks (including Mosaic AI/MLflow) or Azure AI Foundry Relevant certifications such as Anthropic Claude, Azure AI Engineer, AWS ML Specialty, or Databricks GenAI Background in FinOps for AI workloads, particularly token and GPU cost modeling Experience in Data Reliability Engineering, covering data quality and pipeline SLOs, since AI reliability depends on data reliability Prior mentoring or team lead experience

About EPAM Systems

People
About 3,012 in 2024.Benefit plan filings (Form 5500)
Company
Listed as EPAM.
H-1B visas
21 approved for new hires in fiscal 2025.US Citizenship and Immigration Services

Demand for AI Engineer roles

New AI Engineer postings at the employers we follow are up 340% over the last 6 weeks compared with the 6 before.

Weekly new AI Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 78, the most 803, and the latest week had 438.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under AI Engineer, by the work the posting describes.

AI engineers build products on top of AI models that already exist, most often large language models: they connect a model to a company's data and tools and ship the assistants, agents and search features people use.

Postings ask for strong Python or TypeScript, experience with model APIs, retrieval and agent frameworks, and ways to test whether the model's answers are good enough.

An ML engineer trains and serves models; an AI engineer usually starts from a model someone else trained and builds the product around it, though titles often mix the two.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Location not supplied.

Working pattern and location restrictions need checking in the full posting.

Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

AI answers unavailable

We couldn’t identify enough role detail in this saved description to support an AI answer. Read the full posting