Back to jobs

Senior AI Engineer

Reads like an ML Engineer role: training models and running them in production.

Ho Chi Minh City, Ho Chi Minh City, Vietnam

Live, first seen by us 62 days ago
Pay
Pay not listed
Work setting
Not stated
Employment
Full-time — employment source
Employment type in the employer’s posting: Full-time
From the employer’s posting

What you’ll work on

Full posting
  • In this role, you will own the infrastructure layer that turns our AI models into fast, cost effective, production grade services.

  • Design mixed precision strategies and run calibration to minimize quality loss.

  • Build benchmarking for TTFT, inter token latency, throughput, GPU utilization, and cost per million tokens.

From the employer’s posting
OPSWAT is committed to harnessing the power of AI to drive meaningful advancements across all areas of our business. We are currently seeking a highly skilled Senior AI Engineer to lead the deployment and optimization of Large Language Models for efficient, high performance inference on GPU hardware. In this role, you will own the infrastructure layer that turns our AI models into fast, cost effective, production grade services. You will work closely with the AI/ML Team, the Enterprise & Data Team, and Infrastructure to design local LLM serving environments, maximize GPU utilization, and apply model compression techniques such as quantization from FP32 down to FP8/FP4. This is a chance to work at the deep intersection of models, serving software, and hardware within a global cybersecurity company, delivering the performance that powers OPSWAT's AI products at scale. What You Will Do
Local LLM Deployment & Hardware Setup: Design and build local LLM serving environments on GPU hardware, selecting GPU configurations based on VRAM, memory bandwidth, and workload. Install and maintain the full inference stack (NVIDIA drivers, CUDA, cuDNN, inference engines) across single and multi GPU deployments, including tensor and pipeline parallelism. Inference Optimization & Model Compression: Optimize LLM models for efficient GPU serving, balancing latency, throughput, memory, and cost. Apply quantization from FP32 to FP16/BF16, FP8, and FP4 using PTQ methods (GPTQ, AWQ, SmoothQuant) and QAT. Design mixed precision strategies and run calibration to minimize quality loss. Apply complementary techniques including pruning, distillation, and sparsity. Serving Engine & Performance Engineering: Deploy and tune high throughput serving engines (vLLM, TensorRT-LLM, TGI), optimizing KV cache (PagedAttention), continuous batching, and speculative decoding. Leverage optimized kernels (FlashAttention) and FP8 tensor cores. Build benchmarking for TTFT, inter token latency, throughput, GPU utilization, and cost per million tokens.
Inference Optimization & Model Compression: Optimize LLM models for efficient GPU serving, balancing latency, throughput, memory, and cost. Apply quantization from FP32 to FP16/BF16, FP8, and FP4 using PTQ methods (GPTQ, AWQ, SmoothQuant) and QAT. Design mixed precision strategies and run calibration to minimize quality loss. Apply complementary techniques including pruning, distillation, and sparsity. Serving Engine & Performance Engineering: Deploy and tune high throughput serving engines (vLLM, TensorRT-LLM, TGI), optimizing KV cache (PagedAttention), continuous batching, and speculative decoding. Leverage optimized kernels (FlashAttention) and FP8 tensor cores. Build benchmarking for TTFT, inter token latency, throughput, GPU utilization, and cost per million tokens. Production Serving Infrastructure: Deploy quantized models with autoscaling, load balancing, and observability. Establish quality regression gates and run A/B tests of quantized vs full precision models on real traffic.
Education & alternatives
What We Need From You - Bachelor's degree in Computer Science, Software Engineering, or a related field. - 5+ years of experience in software engineering, with a focus on ML infrastructure, LLM inference, or model optimization.
Apply at OPSWAT

Posting record

Report an error
Status in our records
Live
First seen by us
Sep 20, 202662 days ago
Recorded sightings
52
Last seen by us
Nov 21, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 62

Similar jobs usually come down between day 7 and day 76.

Well past OPSWAT’s usual: its jobs come down by about day 25.

From 6,389 similar postings that came down, and 9 of OPSWAT’s. Coming down is not always a hire.

OPSWAT is hiring

15 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Hiring is picking up
  • Its jobs tend to stay up a long time
  • 7% of open jobs were reposted
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • C++
  • Python
  • AWS
  • Azure
  • GCP
Where the posting names them
- Ability to work independently and as part of a team in a fast paced environment. - Proficiency in Python; familiarity with C++/CUDA is a strong plus. It Would Be Nice if You Have
- Experience with multi GPU and distributed inference. - Certifications in AWS or Azure architecture. - Experience with CI/CD and cloud production deployment (Azure, AWS, or GCP), including GPU backed instances.
- Certifications in AWS or Azure architecture. - Experience with CI/CD and cloud production deployment (Azure, AWS, or GCP), including GPU backed instances. OPSWAT is an equal opportunity employer. We celebrate diversity and are committed to providing an environment where equal employment opportunities are extended to all employees and applicants, free of discrimination and harassment of any type. All employment decisions are based on individual qualifications, job requirements, and business needs without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other category protected by federal, state, or local laws.

Your next step

Complete your application on www.opswat.com. The employer’s form will show what is required.

Applying through Greenhouse

  • No account needed: the form is one page, with an “Autofill my application” option that reads your resume.
  • Greenhouse doesn’t read resumes over 2.5 MB, or details placed in a header, footer, text box, table or second column. The file still attaches, but a recruiter has to type you in by hand.

Already applied? Track this application

Job description

View original posting ↗

OPSWAT, a global leader in IT, OT, and ICS critical infrastructure cybersecurity, delivers an end-to-end platform that gives public and private sector organizations and enterprises the critical advantage needed to protect their complex networks, secure their devices, and ensure compliance. Over the last 20 years our commitment to innovative technology has earned the trust of more than 1,700 organizations, governments, and institutions globally, solidifying our role in protecting the world’s critical infrastructure and securing our way of life.

The Position

OPSWAT is committed to harnessing the power of AI to drive meaningful advancements across all areas of our business. We are currently seeking a highly skilled Senior AI Engineer to lead the deployment and optimization of Large Language Models for efficient, high performance inference on GPU hardware.

In this role, you will own the infrastructure layer that turns our AI models into fast, cost effective, production grade services. You will work closely with the AI/ML Team, the Enterprise & Data Team, and Infrastructure to design local LLM serving environments, maximize GPU utilization, and apply model compression techniques such as quantization from FP32 down to FP8/FP4. This is a chance to work at the deep intersection of models, serving software, and hardware within a global cybersecurity company, delivering the performance that powers OPSWAT's AI products at scale.

What You Will Do

  • Local LLM Deployment & Hardware Setup: Design and build local LLM serving environments on GPU hardware, selecting GPU configurations based on VRAM, memory bandwidth, and workload. Install and maintain the full inference stack (NVIDIA drivers, CUDA, cuDNN, inference engines) across single and multi GPU deployments, including tensor and pipeline parallelism.
  • Inference Optimization & Model Compression: Optimize LLM models for efficient GPU serving, balancing latency, throughput, memory, and cost. Apply quantization from FP32 to FP16/BF16, FP8, and FP4 using PTQ methods (GPTQ, AWQ, SmoothQuant) and QAT. Design mixed precision strategies and run calibration to minimize quality loss. Apply complementary techniques including pruning, distillation, and sparsity.
  • Serving Engine & Performance Engineering: Deploy and tune high throughput serving engines (vLLM, TensorRT-LLM, TGI), optimizing KV cache (PagedAttention), continuous batching, and speculative decoding. Leverage optimized kernels (FlashAttention) and FP8 tensor cores. Build benchmarking for TTFT, inter token latency, throughput, GPU utilization, and cost per million tokens.
  • Production Serving Infrastructure: Deploy quantized models with autoscaling, load balancing, and observability. Establish quality regression gates and run A/B tests of quantized vs full precision models on real traffic.
  • Research & Propose Innovative Solutions: exploring and implementing novel inference optimization and model compression techniques.

What We Need From You

  • Bachelor's degree in Computer Science, Software Engineering, or a related field.
  • 5+ years of experience in software engineering, with a focus on ML infrastructure, LLM inference, or model optimization.
  • Hands on experience deploying and serving LLMs on GPU hardware in production.
  • Strong understanding of quantization and model compression (FP8/FP4/INT8/INT4, GPTQ, AWQ, SmoothQuant, QAT).
  • Experience with high throughput inference engines (vLLM, TensorRT-LLM, TGI, llama.cpp).
  • Solid understanding of GPU architecture, CUDA, and the memory bandwidth bound nature of LLM inference.
  • Familiarity with KV cache, continuous batching, PagedAttention, and speculative decoding.
  • Solid understanding of software development methodologies (Agile, Scrum, etc.).
  • Good communication, collaboration, and problem solving skills.
  • Ability to work independently and as part of a team in a fast paced environment.
  • Proficiency in Python; familiarity with C++/CUDA is a strong plus.

It Would Be Nice if You Have

  • Experience writing or tuning custom CUDA / Triton kernels.
  • Experience with multi GPU and distributed inference.
  • Certifications in AWS or Azure architecture.
  • Experience with CI/CD and cloud production deployment (Azure, AWS, or GCP), including GPU backed instances.

OPSWAT is an equal opportunity employer. We celebrate diversity and are committed to providing an environment where equal employment opportunities are extended to all employees and applicants, free of discrimination and harassment of any type. All employment decisions are based on individual qualifications, job requirements, and business needs without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other category protected by federal, state, or local laws.

Recruiting Agencies: we do not accept unsolicited resumes from third party agencies for any of our open positions. To submit resumes for our jobs, there must be a recruiting contract approved by our legal team and endorsed by both parties. We are currently not accepting additional 3rd party agencies at this time.

 

Demand for ML Engineer roles

New ML Engineer postings at the employers we follow are down 4% over the last 6 weeks compared with the 6 before.

Weekly new ML Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 508, the most 899, and the latest week had 508.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under ML Engineer, by the work the posting describes.

ML engineers put machine learning models into products. They write the code that trains, ships, serves and monitors models so a recommendation or prediction works for real users.

Postings ask for strong Python and software engineering, a framework such as PyTorch, cloud services, and more and more often experience with large language models.

A data scientist often works out which model to build; an ML engineer makes it run reliably in production.

US government figures for computer and information research scientists

Median yearly pay, May 2025
$140,300
Expected change, 2025 to 2035
21.8% more jobsagainst 3.5% for all jobs

The government has no code for ML engineers, so these figures are for the closest group, computer and information research scientists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1221.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Ho Chi Minh City, Ho Chi Minh City, Vietnam

Working pattern and location restrictions need checking in the full posting.

Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.