Back to jobs

Agentic AI Systems Evaluation Engineer

Pay
USD 130,700–205,200/yearLocation-specific pay — pay source
Background in advanced systems validation with substantial depth in AI inference, performance analysis, and benchmarking automation. The pay range for this role is $130,700 to $205,200 USD annually with additional opportunities for pay in the form of bonus and/or equity (applies to United States of America candidates only). Pay varies by work location, job-related knowledge, skills, and experience. Compensation & Benefits (Full-Time Employees)
Read the full posting
Work setting
Not stated
Employment
Not stated

What you’ll work on

Full posting

We are looking for a Systems Evaluation Engineer to continuously assess whether our AI software delivers the customer experiences and system performance it is designed to achieve.

Working closely with product, architecture, and engineering teams, you will evaluate AI workloads, compare model and runtime implementations, and explain how technical performance affects the customer experience.

  • You will own the evaluation approach and interpretation of evidence, providing continuous feedback that shapes the developing solution.

From the employer’s posting
We are looking for a Systems Evaluation Engineer to continuously assess whether our AI software delivers the customer experiences and system performance it is designed to achieve. You will connect user stories and product goals to measurable outcomes, develop evaluation methodologies, and interpret results throughout development.
Working closely with product, architecture, and engineering teams, you will evaluate AI workloads, compare model and runtime implementations, and explain how technical performance affects the customer experience. Your findings will help teams understand trade-offs, identify gaps, and make better design and implementation decisions.
Working closely with product, architecture, and engineering teams, you will evaluate AI workloads, compare model and runtime implementations, and explain how technical performance affects the customer experience. Your findings will help teams understand trade-offs, identify gaps, and make better design and implementation decisions. This role requires deep technical understanding of AI inference and PC systems, combined with strong analytical judgment. You will own the evaluation approach and interpretation of evidence, providing continuous feedback that shapes the developing solution. What You Might Do

Before you apply

Employer’s visa record
249 H-1B petitions approved in fiscal 2024 and 2025For the employer as a whole. This posting doesn’t say whether the role is sponsored. US Citizenship and Immigration Services.

What you’ll bring

All qualifications

Core experience

  • 7+ years of experience in systems evaluation, performance engineering, software engineering, advanced validation, or a related field.
  • Strong understanding of AI systems and inference workloads, including model execution, prompt processing, and runtime behavior.
  • Experience designing benchmarking methodologies, selecting meaningful metrics, and interpreting complex system performance data.
  • Familiarity with model formats such as ONNX and GGUF, model runners, and the implications of different execution approaches.
  • Understanding of system-level performance, including compute utilization, memory constraints, concurrency, and interactions between AI workloads and the operating system.
  • Ability to translate technical measurements into customer impact and communicate actionable findings across product, architecture, and engineering teams.

Preferred experience

  • Experience evaluating AI workloads on Windows and Linux PCs or other edge devices.
  • Knowledge of attention mechanisms, KV caching, quantization, and other inference optimization concepts and their implications for evaluation.
  • Experience analyzing device telemetry and combining benchmark results with observations from representative customer scenarios.
  • Familiarity with CPU, GPU, and NPU execution and the challenges of comparing performance across heterogeneous hardware.
Qualification wording
7+ years of experience in systems evaluation, performance engineering, software engineering, advanced validation, or a related field.
Strong understanding of AI systems and inference workloads, including model execution, prompt processing, and runtime behavior.
Experience designing benchmarking methodologies, selecting meaningful metrics, and interpreting complex system performance data.
Familiarity with model formats such as ONNX and GGUF, model runners, and the implications of different execution approaches.
Understanding of system-level performance, including compute utilization, memory constraints, concurrency, and interactions between AI workloads and the operating system.
Ability to translate technical measurements into customer impact and communicate actionable findings across product, architecture, and engineering teams.
Experience evaluating AI workloads on Windows and Linux PCs or other edge devices.
Knowledge of attention mechanisms, KV caching, quantization, and other inference optimization concepts and their implications for evaluation.
Experience analyzing device telemetry and combining benchmark results with observations from representative customer scenarios.
Familiarity with CPU, GPU, and NPU execution and the challenges of comparing performance across heterogeneous hardware.
Education & alternatives
- Ability to translate technical measurements into customer impact and communicate actionable findings across product, architecture, and engineering teams. - Bachelor’s degree in computer science, statistics, software engineering, a related field, or equivalent practical experience. Preferred Skills

About HP

At HP, you’ll have a chance to create tools, technology, and solutions that reshape the way the world works in the future.

In the employer’s words · Read in context

Apply at HP

Posting record

Report an error
Status in our records
Live
First seen by us
Oct 19, 202629 days ago
Recorded sightings
45
Last seen by us
Nov 18, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 29

Up longer than most. Similar jobs usually come down between day 1 and day 11.

From 506 similar postings that came down. Coming down is not always a hire.

HP is hiring

56 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • 5% of open jobs were reposted
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • C++
  • Python
Where the posting names them
- Understanding of system-level performance, including compute utilization, memory constraints, concurrency, and interactions between AI workloads and the operating system. - Strong programming and automation skills in Python, C++, or another relevant language, with experience building evaluation tools or data analysis workflows. - Ability to translate technical measurements into customer impact and communicate actionable findings across product, architecture, and engineering teams.

Your next step

Complete your application on job-boards.greenhouse.io. The employer’s form will show what is required.

Applying through Greenhouse

  • No account needed: the form is one page, with an “Autofill my application” option that reads your resume.
  • Greenhouse doesn’t read resumes over 2.5 MB, or details placed in a header, footer, text box, table or second column. The file still attaches, but a recruiter has to type you in by hand.

Already applied? Track this application

Job description

View original posting ↗

About HP

At HP, you’ll have a chance to create tools, technology, and solutions that reshape the way the world works in the future. If you’re looking to join a company that allows you to connect with a network of professionals eager to support you in doing your best work, we want to talk to you. Our legendary culture guides every employee toward success—fostering collaboration and driving innovation.

Operating in over 170 countries, HP is always creating new services, products, and capabilities giving you more opportunities to advance your career. Here, innovation is the key to professional development and career mobility.

When you join HP, you’re joining a company that believes every voice matters and that we all deserve a seat at the table. From the boardroom to factory floor, we create a culture where everyone is respected and where people can be themselves. You will be part of a global laboratory where different perspectives and experiences will help you solve problems in new ways. This is where you can build a long and wide-ranging career.

About The Role 

We are looking for a Systems Evaluation Engineer to continuously assess whether our AI software delivers the customer experiences and system performance it is designed to achieve. You will connect user stories and product goals to measurable outcomes, develop evaluation methodologies, and interpret results throughout development. 

Working closely with product, architecture, and engineering teams, you will evaluate AI workloads, compare model and runtime implementations, and explain how technical performance affects the customer experience. Your findings will help teams understand trade-offs, identify gaps, and make better design and implementation decisions. 

This role requires deep technical understanding of AI inference and PC systems, combined with strong analytical judgment. You will own the evaluation approach and interpretation of evidence, providing continuous feedback that shapes the developing solution. 

What You Might Do 

  • Translate user stories and intended customer experiences into measurable success criteria, evaluation scenarios, and performance targets. 
  • Design repeatable benchmarking frameworks that assess AI workload effectiveness and system behavior throughout development. 
  • Compare model formats, model runners, runtime implementations, and hardware configurations using representative workloads and controlled measurements. 
  • Analyze inference behavior, including prompt processing, time to first token, token generation rate, latency, and memory consumption. 
  • Evaluate system responsiveness and resource utilization under individual and concurrent AI workloads, including effects on other PC activities. 
  • Interpret benchmark and telemetry data to distinguish meaningful improvements, regressions, measurement variability, and gaps in evaluation coverage. 
  • Provide actionable feedback to architects and engineers, connecting technical findings to customer impact and recommending further investigation. 
  • Build automation and reporting tools that make evaluations reproducible and results accessible to development teams. 
  • Communicate performance trends, trade-offs, and progress against customer outcomes through technical reviews and concise executive reports. 

 

Essential Qualifications 

  • 7+ years of experience in systems evaluation, performance engineering, software engineering, advanced validation, or a related field. 
  • Strong understanding of AI systems and inference workloads, including model execution, prompt processing, and runtime behavior. 
  • Experience designing benchmarking methodologies, selecting meaningful metrics, and interpreting complex system performance data. 
  • Familiarity with model formats such as ONNX and GGUF, model runners, and the implications of different execution approaches. 
  • Understanding of system-level performance, including compute utilization, memory constraints, concurrency, and interactions between AI workloads and the operating system. 
  • Strong programming and automation skills in Python, C++, or another relevant language, with experience building evaluation tools or data analysis workflows. 
  • Ability to translate technical measurements into customer impact and communicate actionable findings across product, architecture, and engineering teams. 
  • Bachelor’s degree in computer science, statistics, software engineering, a related field, or equivalent practical experience. 

 

Preferred Skills 

  • Experience evaluating AI workloads on Windows and Linux PCs or other edge devices. 
  • Knowledge of attention mechanisms, KV caching, quantization, and other inference optimization concepts and their implications for evaluation. 
  • Experience analyzing device telemetry and combining benchmark results with observations from representative customer scenarios. 
  • Familiarity with CPU, GPU, and NPU execution and the challenges of comparing performance across heterogeneous hardware. 
  • Experience with statistical analysis, experimental design, and methods for separating real performance changes from measurement noise. 
  • Background in advanced systems validation with substantial depth in AI inference, performance analysis, and benchmarking automation. 

The pay range for this role is $130,700 to $205,200 USD annually with additional opportunities for pay in the form of bonus and/or equity (applies to United States of America candidates only). Pay varies by work location, job-related knowledge, skills, and experience.

Compensation & Benefits (Full-Time Employees)

The salary range for this role is listed above. Final salary offered is based upon multiple factors including individual job-related qualifications, education, experience, knowledge and skills.

At HP, we offer a competitive and comprehensive benefits package, including:

  • Health insurance
  • Dental insurance
  • Vision insurance
  • Long term/short term disability insurance
  • Employee assistance program
  • Flexible spending account
  • Life insurance
  • Generous time off policies, including; 
    • 4-12 weeks fully paid parental leave based on tenure
    • 11 paid holidays
    • Additional flexible paid vacation and sick leave (US benefits overview)

Why join HP?

When you join HP, you’re investing in your future—and so are we. You have priorities beyond work. That’s why we offer flexibility, support, and benefits that help you shape the life you want. 

Equal Opportunity Employer (EEO) Statement

HP, Inc. provides equal employment opportunity to all employees and prospective employees, without regard to race, color, religion, sex, national origin, ancestry, citizenship, sexual orientation, age, disability, or status as a protected veteran, marital status, familial status, physical or mental disability, medical condition, pregnancy, genetic predisposition or carrier status, uniformed service status, political affiliation or any other characteristic protected by applicable national, federal, state, and local law(s).

Please be assured that you will not be subject to any adverse treatment if you choose to disclose the information requested. This information is provided voluntarily. The information obtained will be kept in strict confidence.

About HP

People
About 55,000 in 2025, down 5% from the year before.Annual report (SEC Form 10-K)
Company
Listed as HPQ.
H-1B visas
41 approved for new hires in fiscal 2025.US Citizenship and Immigration Services

About the role

We file this job under AI Engineer, by the work the posting describes.

AI engineers build products on top of AI models that already exist, most often large language models: they connect a model to a company's data and tools and ship the assistants, agents and search features people use.

Postings ask for strong Python or TypeScript, experience with model APIs, retrieval and agent frameworks, and ways to test whether the model's answers are good enough.

An ML engineer trains and serves models; an AI engineer usually starts from a model someone else trained and builds the product around it, though titles often mix the two.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay
Background in advanced systems validation with substantial depth in AI inference, performance analysis, and benchmarking automation. The pay range for this role is $130,700 to $205,200 USD annually with additional opportunities for pay in the form of bonus and/or equity (applies to United States of America candidates only). Pay varies by work location, job-related knowledge, skills, and experience. Compensation & Benefits (Full-Time Employees)
Location & working pattern

Houston, TX

Working pattern and location restrictions need checking in the full posting.

Work authorization
Equal Opportunity Employer (EEO) Statement HP, Inc. provides equal employment opportunity to all employees and prospective employees, without regard to race, color, religion, sex, national origin, ancestry, citizenship, sexual orientation, age, disability, or status as a protected veteran, marital status, familial status, physical or mental disability, medical condition, pregnancy, genetic predisposition or carrier status, uniformed service status, political affiliation or any other characteristic protected by applicable national, federal, state, and local law(s). Please be assured that you will not be subject to any adverse treatment if you choose to disclose the information requested. This information is provided voluntarily. The information obtained will be kept in strict confidence.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.