Back to jobs

AI/LLM Safety Engineer

Pay
Pay not listed
Work setting
Working pattern needs review — work setting source
Listed location: Remote
Work setup in the employer’s posting: Hybrid
From the employer’s posting
Employment
Not stated

What you’ll work on

Full posting
  • You will design the evaluations that catch unsafe behavior, build the guardrails that stop it, and lead the red-teaming that finds the gaps before our users—or attackers—do.

From the employer’s posting
What You'll Own The AI/LLM Safety Engineer will join our AI team and take ownership of how safely our models and agents behave in production; with a focus on AI Safety, Trust & Safety, and Responsible AI. You will design the evaluations that catch unsafe behavior, build the guardrails that stop it, and lead the red-teaming that finds the gaps before our users—or attackers—do. Agent safety is the primary focus of this role: you will help ensure that as our systems gain the ability to call tools and take actions, they do so within well-defined, well-tested boundaries. LLM Safety Evaluation & Red Teaming

What you’ll bring

All qualifications

Core experience

  • Bachelor's or Master's degree in Computer Science, Software Engineering, Cybersecurity, or a related technical field—or equivalent practical experience.
  • 4+ years building production software, with direct experience working on—or securing—ML/LLM systems.
  • Solid understanding of LLMs and ML: how models work, prompt engineering, and the safety implications of fine-tuning and RAG (e.g., unsafe retrieval, tool misuse, and data exfiltration).
  • Familiarity with the LLM attack surface—prompt injection, jailbreaks, data poisoning, and supply-chain risk—and working knowledge of the OWASP LLM Top 10.
  • Hands-on experience with at least one of safety evaluation or red teaming, with the ability to walk through a real finding and how it was remediated.

Preferred experience

  • Hands-on experience with industry safety tooling such as garak, PyRIT, promptfoo, Giskard, and NeMo Guardrails, and the ability to articulate the trade-offs between them.
  • Familiarity with AI governance and compliance frameworks (NIST AI RMF, ISO/IEC 42001, EU AI Act) and the ability to translate compliance requirements into concrete engineering tasks.
  • Practical experience with threat-modeling methodologies such as MITRE ATLAS and STRIDE/PASTA.
Qualification wording
Bachelor's or Master's degree in Computer Science, Software Engineering, Cybersecurity, or a related technical field—or equivalent practical experience.
4+ years building production software, with direct experience working on—or securing—ML/LLM systems.
Solid understanding of LLMs and ML: how models work, prompt engineering, and the safety implications of fine-tuning and RAG (e.g., unsafe retrieval, tool misuse, and data exfiltration).
Familiarity with the LLM attack surface—prompt injection, jailbreaks, data poisoning, and supply-chain risk—and working knowledge of the OWASP LLM Top 10.
Hands-on experience with at least one of safety evaluation or red teaming, with the ability to walk through a real finding and how it was remediated.
Hands-on experience with industry safety tooling such as garak, PyRIT, promptfoo, Giskard, and NeMo Guardrails, and the ability to articulate the trade-offs between them.
Familiarity with AI governance and compliance frameworks (NIST AI RMF, ISO/IEC 42001, EU AI Act) and the ability to translate compliance requirements into concrete engineering tasks.
Practical experience with threat-modeling methodologies such as MITRE ATLAS and STRIDE/PASTA.
Education & alternatives
Required Qualifications - Bachelor's or Master's degree in Computer Science, Software Engineering, Cybersecurity, or a related technical field—or equivalent practical experience. - 4+ years building production software, with direct experience working on—or securing—ML/LLM systems.
Apply at Propio

Posting record

Report an error
Status in our records
Live
First seen by us
Oct 15, 202637 days ago
Recorded sightings
38
Last seen by us
Nov 22, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 37

Up longer than most. Similar jobs usually come down between day 1 and day 15.

From 1,368 similar postings that came down. Coming down is not always a hire.

Propio is hiring

6 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Hiring has slowed
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
Where the posting names them
- 4+ years building production software, with direct experience working on—or securing—ML/LLM systems. - Strong software engineering skills with the ability to write production-grade code (primarily Python), beyond scripting or notebook prototyping. - Solid understanding of LLMs and ML: how models work, prompt engineering, and the safety implications of fine-tuning and RAG (e.g., unsafe retrieval, tool misuse, and data exfiltration).

Your next step

Complete your application on recruiting.paylocity.com. The employer’s form will show what is required.

Applying through Paylocity

  • “Upload your resume and we’ll automatically fill out your application”: the form fills itself from your file, so check it before sending.

Already applied? Track this application

Job description

View original posting ↗

Description

AI/LLM Safety Engineer


Own Your Impact. At Propio, we don't believe careers happen to people. We believe people create them.


Here, you're trusted to make decisions, challenge assumptions, drive innovation, and shape outcomes. Your success is not limited by hierarchy or tenure. It's fueled by your ambition, your curiosity, and your willingness to own your impact. If you're looking for a role where you can simply maintain the status quo, this probably isn't it, but if you're looking for a place where your ideas matter, your growth is accelerated, and your work creates meaningful impact across the world, we'd love to talk.


Why Propio?

Every day, communication changes lives. A patient receives care they otherwise couldn't access. A family gains critical information. A business connects with a customer. A community becomes more inclusive. These moments happen because barriers are removed. And behind those moments are Propio team members who show up every day to solve problems, innovate, and build the future. This isn't just work. This is world impact.


As a AI/LLM Safety Engineer, you'll have the opportunity to make a meaningful contribution to the continued growth and transformation of Propio.


You'll be empowered to:

  • Take ownership of important initiatives and outcomes.
  • Drive meaningful business results.
  • Influence decisions and contribute new ideas.
  • Partner with talented, high-performing team members.
  • Challenge yourself through continuous learning and growth.
  • Help shape the future of a rapidly growing organization.

What You'll Own

The AI/LLM Safety Engineer will join our AI team and take ownership of how safely our models and agents behave in production; with a focus on AI Safety, Trust & Safety, and Responsible AI. You will design the evaluations that catch unsafe behavior, build the guardrails that stop it, and lead the red-teaming that finds the gaps before our users—or attackers—do. Agent safety is the primary focus of this role: you will help ensure that as our systems gain the ability to call tools and take actions, they do so within well-defined, well-tested boundaries.


LLM Safety Evaluation & Red Teaming

  • Design and maintain a safety evaluation framework—adversarial prompt sets, scenario-based test suites, and regression suites—so that every model and agent update is validated before it ships.
  • Lead structured red-teaming exercises covering jailbreaks, prompt injection, tool misuse, and data exfiltration; document findings and drive each issue through to remediation and closure.

Guardrails & Runtime Controls

  • Build and iterate on guardrail logic, including input/output filtering, tool-boundary constraints, action validation, sensitive-data redaction, and policy prompting.
  • Integrate safety checks into CI/CD and runtime so that unsafe behavior is intercepted before it reaches users.

Agent Safety (primary focus of this role)

  • Perform threat modeling for agentic scenarios: tool-call boundaries, sandbox isolation, and least-privilege access, with particular attention to preventing agents from exfiltrating data or executing irreversible actions through chained tool calls.
  • Conduct safety reviews of reinforcement-learning (RL) environments and trajectory data, partnering with environment and agent engineering teams to embed safety constraints directly into the environments themselves.

Monitoring & Observability

  • Instrument AI features for safety with structured logging, tracing, and metrics, enabling detection of unsafe patterns and regressions in production.

Governance & Collaboration

  • Prepare evidence for governance reviews—test reports, evaluation summaries, and mitigation validation—aligned with internal Responsible AI standards.
  • Collaborate with Product and UX to improve safety interactions (warnings, confirmations, refusal messaging, and feedback collection), and align evaluation goals with the Research and Data teams.

What Makes Someone Successful Here

The most successful people at Propio aren't necessarily the ones with the longest resumes. They're the people who:

  • Take ownership instead of waiting for direction.
  • Embrace challenges as opportunities to grow.
  • Continuously seek better ways of working.
  • Turn ideas into action.
  • Hold themselves and others accountable to high standards.
  • Are driven by making a measurable impact.

Requirements

What You'll Bring

Required Qualifications

  • Bachelor's or Master's degree in Computer Science, Software Engineering, Cybersecurity, or a related technical field—or equivalent practical experience.
  • 4+ years building production software, with direct experience working on—or securing—ML/LLM systems.
  • Strong software engineering skills with the ability to write production-grade code (primarily Python), beyond scripting or notebook prototyping.
  • Solid understanding of LLMs and ML: how models work, prompt engineering, and the safety implications of fine-tuning and RAG (e.g., unsafe retrieval, tool misuse, and data exfiltration).
  • A security mindset with demonstrated threat-modeling ability; able to threat-model AI workflows and familiar with the fundamentals of access control, data retention, and incident response.
  • Familiarity with the LLM attack surface—prompt injection, jailbreaks, data poisoning, and supply-chain risk—and working knowledge of the OWASP LLM Top 10.
  • Hands-on experience with at least one of safety evaluation or red teaming, with the ability to walk through a real finding and how it was remediated.

Preferred Qualifications

  • Hands-on experience with industry safety tooling such as garak, PyRIT, promptfoo, Giskard, and NeMo Guardrails, and the ability to articulate the trade-offs between them.
  • Visible output in AI safety or security: publications at relevant venues (e.g., the NeurIPS AI Safety Workshop, USENIX Security, or DEF CON AI Village), open-source contributions, or responsible disclosures on frontier models with public write-ups.
  • Familiarity with AI governance and compliance frameworks (NIST AI RMF, ISO/IEC 42001, EU AI Act) and the ability to translate compliance requirements into concrete engineering tasks.
  • Engineering experience with agents, RL environments, and/or tool use.
  • Practical experience with threat-modeling methodologies such as MITRE ATLAS and STRIDE/PASTA.

Even if your experience doesn't perfectly match every qualification, we encourage you to apply. We're looking for potential, drive, and a commitment to growth as much as experience.


What You'll Gain

Own Your Growth: We invest in people who invest in themselves. You'll have opportunities to learn, develop, and expand your capabilities while building a meaningful career.


Own Your Impact: You'll see the connection between your work and our success. We believe great people deserve the opportunity to make a real difference.


Own Your Innovation: The best ideas can come from anywhere. We encourage curiosity, creativity, and challenging the way things have always been done.


Own Your Success: Whether you're building expertise, pursuing leadership opportunities, or expanding your career path, we'll give you room to grow and the support to get there.


At Propio, your work doesn't just move a company forward. It helps connect people, communities, and opportunities across the world. Ready to Build Something Bigger? Apply today and discover what happens when you own your success.

Demand for AI Engineer roles

New AI Engineer postings at the employers we follow are up 340% over the last 6 weeks compared with the 6 before.

Weekly new AI Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 78, the most 803, and the latest week had 438.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under AI Engineer, by the work the posting describes.

AI engineers build products on top of AI models that already exist, most often large language models: they connect a model to a company's data and tools and ship the assistants, agents and search features people use.

Postings ask for strong Python or TypeScript, experience with model APIs, retrieval and agent frameworks, and ways to test whether the model's answers are good enough.

An ML engineer trains and serves models; an AI engineer usually starts from a model someone else trained and builds the product around it, though titles often mix the two.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Remote

Working pattern and location restrictions need checking in the full posting.

Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.