Time since we first saw it
Day 3
Similar jobs usually come down between day 1 and day 15.
From 1,368 similar postings that came down. Coming down is not always a hire.
Location: 100% remote, seeking candidates located in LATAM. Contract Length: 1 year, with potential for extension.Read the full posting
Define testing and evaluation strategy for GenAI applications and AI agents
Build automated evaluation pipelines and add AI-quality checks to CI/CD
Design single-turn and multi-turn conversational test cases
Design adversarial tests for prompt injection, jailbreaks, and PII leakage
Perform root-cause analysis of incorrect AI outputs with AI engineers
Define the testing and evaluation strategy for Generative AI applications and AI agents.
Build automated evaluation pipelines and integrate AI-quality checks into CI/CD and release processes.
Design single-turn and multi-turn conversational test cases, including context retention and workflow transitions.
Design negative and adversarial test scenarios covering prompt injection, jailbreaks, hallucinations, PII leakage, unsafe responses, and out-of-scope requests.
Perform root-cause analysis for incorrect or undesirable AI outputs and collaborate with AI engineers on remediation.
5+ years of experience in software testing, quality engineering, or test automation with hands-on automation capabilities.
Experience with AI evaluation frameworks such as RAGAS, DeepEval, Promptfoo, LangSmith, TruLens, or equivalent tools.
Bachelor’s degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, Software Engineering, or a related technical field.
2+ years of experience with Amazon Bedrock.
Experience evaluating RAG systems, retrieval quality, or enterprise knowledge assistants.
Practical understanding of Generative AI / LLM behavior and the challenges of testing non-deterministic systems.
Education & Certifications: - Bachelor’s degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, Software Engineering, or a related technical field. Required Equipment: This role requires the use of a personal laptop. Candidates must be comfortable with company security software and device management tools being installed on their laptop as part of the onboarding process and for the duration of the engagement.
NTT DATA is a Top 5 global IT services provider with more than 190,000 professionals across 50+ countries.
In the employer’s words · Read in context
Actively hiringFirst seen by us 3 days ago, still listed, and not a repost of an earlier posting.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 3
Similar jobs usually come down between day 1 and day 15.
From 1,368 similar postings that came down. Coming down is not always a hire.
1 open job
Open jobs by the week we first saw them, last 12 weeks.
Hiring recordAdd your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
- Experience designing test scenarios for conversational applications, chatbots, virtual assistants, or AI agents. - Experience with Python and/or JavaScript / TypeScript for test automation and data processing. - Experience testing REST APIs and backend services.
- Experience evaluating RAG systems, retrieval quality, or enterprise knowledge assistants. - Experience with Amazon Bedrock and AWS services. - Experience testing Amazon Connect, Contact Center, agent-assist, or Customer Service applications.
Complete your application on job-boards.greenhouse.io. The employer’s form will show what is required.
Already applied? Track this application
Location: 100% remote, seeking candidates located in LATAM.
Contract Length: 1 year, with potential for extension.
Contract Rate: $20-25 per hour in USD. Final rate will be determined based on relevant experience and overall fit for the position.
NTT DATA is seeking an AI Testing Engineer, Amazon Bedrock to define and execute the quality strategy for Generative AI applications and AI agents used in Customer Service and Contact Center scenarios. This role will evaluate AI outputs across deterministic and non-deterministic scenarios using automation, semantic and statistical analysis, human review, and adversarial testing while partnering with AI engineers and functional QA teams.
Responsibilities:
Required Experience:
Nice-to-Have:
Education & Certifications:
Required Equipment: This role requires the use of a personal laptop. Candidates must be comfortable with company security software and device management tools being installed on their laptop as part of the onboarding process and for the duration of the engagement.
About NTT DATA
NTT DATA is a Top 5 global IT services provider with more than 190,000 professionals across 50+ countries. We combine industry expertise with capabilities in consulting, technology, AI, cloud, applications, infrastructure, and connectivity to help organizations innovate, optimize, and transform. Through responsible innovation, we deliver meaningful business outcomes, accelerate client success, and create a positive impact on society.
Equal Opportunity Employer
NTT DATA is committed to hiring and retaining a diverse workforce. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, creed, sex, sexual orientation, gender identity, marital status, national origin, age, veteran status, disability, or any other protected class. NTT DATA is an Equal Opportunity Employer Male/Female/Disabled/Veteran and a VEVRAA Federal Contractor.
Req ID: 1355
New AI Engineer postings at the employers we follow are up 340% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under AI Engineer, by the work the posting describes.
AI engineers build products on top of AI models that already exist, most often large language models: they connect a model to a company's data and tools and ship the assistants, agents and search features people use.
Postings ask for strong Python or TypeScript, experience with model APIs, retrieval and agent frameworks, and ways to test whether the model's answers are good enough.
An ML engineer trains and serves models; an AI engineer usually starts from a model someone else trained and builds the product around it, though titles often mix the two.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
LATAM
Location: 100% remote, seeking candidates located in LATAM. Contract Length: 1 year, with potential for extension.
No clear work-authorization passage found. Eligibility is unconfirmed.