Back to jobs

AI Infrastructure Engineer – Generative AI Model Serving (Go / Rust / GPU)

Pay
Pay not listed
Work setting
Not stated
Employment
Not stated

What you’ll work on

Full posting
  • Own the full delivery path: CI/CD pipelines, model packaging and registries, canary and blue/green rollouts, infrastructure as code, and automated performance regression testing.

  • Support agentic and RAG workloads by serving embedding, reranking, and generation models alongside the Enterprise Vector Store, with predictable latency and graceful degradation.

  • Drive reliability, security, and cost efficiency: SLOs, load and chaos testing, capacity planning, GPU cost attribution, and secure model and tenant isolation.

From the employer’s posting
Build and manage GPU cluster infrastructure on Kubernetes (NVIDIA GPU Operator, device plugins, MIG/time-slicing, node pools, scheduling and bin-packing) across cloud and on-prem environments. Own the full delivery path: CI/CD pipelines, model packaging and registries, canary and blue/green rollouts, infrastructure as code, and automated performance regression testing. Build AI observability into everything: metrics, logs, and traces for token throughput, time-to-first-token, GPU utilization, queue depth, error and hallucination signals, using OpenTelemetry, Prometheus, and Grafana.
Provide clean APIs, SDKs, and developer tooling (OpenAI-compatible endpoints, internal services, and lightweight UIs or dashboards) so AI engineers and agent builders can deploy and monitor models self-serve. Support agentic and RAG workloads by serving embedding, reranking, and generation models alongside the Enterprise Vector Store, with predictable latency and graceful degradation. Drive reliability, security, and cost efficiency: SLOs, load and chaos testing, capacity planning, GPU cost attribution, and secure model and tenant isolation.
Support agentic and RAG workloads by serving embedding, reranking, and generation models alongside the Enterprise Vector Store, with predictable latency and graceful degradation. Drive reliability, security, and cost efficiency: SLOs, load and chaos testing, capacity planning, GPU cost attribution, and secure model and tenant isolation. Who You’ll Work With

What you’ll bring

All qualifications

Core experience

  • Strong communication skills and a bias toward measuring, benchmarking, and iterating.
  • Experience writing or tuning CUDA / Triton (OpenAI) / CUTLASS kernels, or contributing to inference engines.
  • Experience with on-device or edge ML inference (ONNX, TFLite, llama.cpp, quantized small language models), embedded or mobile deployment.
  • Knowledge of RAG pipelines, vector databases, embeddings, and agent frameworks (LangChain, LlamaIndex, ReAct-style agents).
  • Experience with model monitoring, drift detection, evaluation harnesses, and responsible AI practices.
  • Experience with service meshes, RDMA/InfiniBand/NCCL, Slurm, or Ray clusters.

Preferred experience

  • Familiarity with modern data platforms such as Teradata, Snowflake, or Databricks is a plus.
Qualification wording
Strong communication skills and a bias toward measuring, benchmarking, and iterating.
Experience writing or tuning CUDA / Triton (OpenAI) / CUTLASS kernels, or contributing to inference engines.
Experience with on-device or edge ML inference (ONNX, TFLite, llama.cpp, quantized small language models), embedded or mobile deployment.
Knowledge of RAG pipelines, vector databases, embeddings, and agent frameworks (LangChain, LlamaIndex, ReAct-style agents).
Experience with model monitoring, drift detection, evaluation harnesses, and responsible AI practices.
Experience with service meshes, RDMA/InfiniBand/NCCL, Slurm, or Ray clusters.
Familiarity with modern data platforms such as Teradata, Snowflake, or Databricks is a plus.
Education & alternatives
What You Bring - A Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience. - Genuine excitement for AI, LLMs, and the systems that make them efficient and affordable to run at scale.

About Teradata

At Teradata, we believe that people thrive when empowered with better information.

In the employer’s words · Read in context

Apply at Teradata

Posting record

Report an error

Actively hiringFirst seen by us 5 days ago, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 12, 20265 days ago
Recorded sightings
6
Last seen by us
Nov 17, 2026
Employer says posted
Oct 27, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 5

Similar jobs usually come down between day 3 and day 37.

From 88 similar postings that came down. Coming down is not always a hire.

Teradata is hiring

30 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • Databricks
  • Docker
  • Grafana
  • Kubernetes
  • Snowflake
  • Teradata
  • Terraform
  • Hugging Face
  • PyTorch
  • React
Where the posting names them
- Experience with containers and orchestration (Docker, Kubernetes, Helm) and CI/CD (GitHub Actions, Jenkins, Argo CD, or similar), plus infrastructure as code (Terraform). - Full-stack engineering experience: comfortable building REST/gRPC APIs and a pragmatic front end (React or similar) for dashboards, playgrounds, or admin tools; Python for glue, tooling, and ML integration. - Experience with observability practices and tools (OpenTelemetry, Prometheus, Grafana, tracing) applied to ML or LLM workloads.
- Strong communication skills and a bias toward measuring, benchmarking, and iterating. - Familiarity with modern data platforms such as Teradata, Snowflake, or Databricks is a plus. Bonus Points
- Strong system design skills and a deep understanding of distributed systems, networking, concurrency, and failure modes. - Experience with containers and orchestration (Docker, Kubernetes, Helm) and CI/CD (GitHub Actions, Jenkins, Argo CD, or similar), plus infrastructure as code (Terraform). - Full-stack engineering experience: comfortable building REST/gRPC APIs and a pragmatic front end (React or similar) for dashboards, playgrounds, or admin tools; Python for glue, tooling, and ML integration.
- Own the full delivery path: CI/CD pipelines, model packaging and registries, canary and blue/green rollouts, infrastructure as code, and automated performance regression testing. - Build AI observability into everything: metrics, logs, and traces for token throughput, time-to-first-token, GPU utilization, queue depth, error and hallucination signals, using OpenTelemetry, Prometheus, and Grafana. - Provide clean APIs, SDKs, and developer tooling (OpenAI-compatible endpoints, internal services, and lightweight UIs or dashboards) so AI engineers and agent builders can deploy and monitor models self-serve.
- Full-stack engineering experience: comfortable building REST/gRPC APIs and a pragmatic front end (React or similar) for dashboards, playgrounds, or admin tools; Python for glue, tooling, and ML integration. - Experience with observability practices and tools (OpenTelemetry, Prometheus, Grafana, tracing) applied to ML or LLM workloads. - You are obsessive about reliability, debuggability, and ensuring AI systems behave predictably and deterministically when needed.
At Teradata, we’re not just managing data; we’re unleashing its full potential. Our ClearScape Analytics™ platform and Enterprise Vector Store help the world’s largest enterprises derive value from their most complex data. Generative AI and agentic systems are the next frontier, and they run on infrastructure that has to be fast, efficient, and dependable at scale. As an AI Infrastructure Engineer, you will build the serving layer that runs large language models and other generative models inside the Teradata Vantage ecosystem. You will work on GPU-accelerated inference, high-performance Go and Rust services, and the cloud-native platform that lets teams ship models to production with confidence. You will own systems end to end, from a CUDA kernel profile to a Kubernetes rollout. What You’ll Do
- Integrate and extend inference engines such as vLLM, TensorRT-LLM, Triton Inference Server, Ray Serve, KServe, SGLang, llama.cpp, and ONNX Runtime; contribute fixes upstream where it makes sense. - Build and manage GPU cluster infrastructure on Kubernetes (NVIDIA GPU Operator, device plugins, MIG/time-slicing, node pools, scheduling and bin-packing) across cloud and on-prem environments. - Own the full delivery path: CI/CD pipelines, model packaging and registries, canary and blue/green rollouts, infrastructure as code, and automated performance regression testing.
Our Company: At Teradata, we believe that people thrive when empowered with better information. Teradata Autonomous Knowledge Platform activates enterprise intelligence by unifying data, knowledge and business context to achieve tangible outcomes. With Teradata, organizations can provide agents with full context for impact when it matters. Our solution lets businesses connect and scale on premises, in the cloud, or through a hybrid approach. Teradata delivers real business value with AI. Power the Platform Behind Enterprise Generative AI
Power the Platform Behind Enterprise Generative AI At Teradata, we’re not just managing data; we’re unleashing its full potential. Our ClearScape Analytics™ platform and Enterprise Vector Store help the world’s largest enterprises derive value from their most complex data. Generative AI and agentic systems are the next frontier, and they run on infrastructure that has to be fast, efficient, and dependable at scale. As an AI Infrastructure Engineer, you will build the serving layer that runs large language models and other generative models inside the Teradata Vantage ecosystem. You will work on GPU-accelerated inference, high-performance Go and Rust services, and the cloud-native platform that lets teams ship models to production with confidence. You will own systems end to end, from a CUDA kernel profile to a Kubernetes rollout.
Who You’ll Work With You will collaborate daily with AI researchers, ML engineers, data scientists, and platform and SRE teams who champion diversity, equity, and inclusion as fundamental to our success. You’ll be part of a cohesive team delivering high-quality, highly visible AI/ML functionality within the Teradata Vantage platform. You will report to the Sr. Manager, Software Engineering, who will champion your growth and empower your contributions. What Makes You a Qualified Candidate
- Experience serving LLMs or generative models in production, with practical knowledge of inference optimization techniques and the serving frameworks listed above. - Solid understanding of transformer architectures and the LLM inference lifecycle (prefill vs. decode, attention, KV cache, sampling), plus familiarity with PyTorch and Hugging Face tooling. - Strong system design skills and a deep understanding of distributed systems, networking, concurrency, and failure modes.
- Experience with on-device or edge ML inference (ONNX, TFLite, llama.cpp, quantized small language models), embedded or mobile deployment. - Knowledge of RAG pipelines, vector databases, embeddings, and agent frameworks (LangChain, LlamaIndex, ReAct-style agents). - Experience with model monitoring, drift detection, evaluation harnesses, and responsible AI practices.

Your next step

Complete your application on careers.teradata.com. The employer’s form will show what is required.

Already applied? Track this application

Job description

View original posting ↗

Our Company:

At Teradata, we believe that people thrive when empowered with better information. Teradata Autonomous Knowledge Platform activates enterprise intelligence by unifying data, knowledge and business context to achieve tangible outcomes. With Teradata, organizations can provide agents with full context for impact when it matters. Our solution lets businesses connect and scale on premises, in the cloud, or through a hybrid approach. Teradata delivers real business value with AI.

 

Power the Platform Behind Enterprise Generative AI

At Teradata, we’re not just managing data; we’re unleashing its full potential. Our ClearScape Analytics™ platform and Enterprise Vector Store help the world’s largest enterprises derive value from their most complex data. Generative AI and agentic systems are the next frontier, and they run on infrastructure that has to be fast, efficient, and dependable at scale.

As an AI Infrastructure Engineer, you will build the serving layer that runs large language models and other generative models inside the Teradata Vantage ecosystem. You will work on GPU-accelerated inference, high-performance Go and Rust services, and the cloud-native platform that lets teams ship models to production with confidence. You will own systems end to end, from a CUDA kernel profile to a Kubernetes rollout.

What You’ll Do

  • Design, build, and operate scalable, low-latency model-serving platforms for LLMs and generative models, covering batching, streaming, routing, autoscaling, and multi-tenant isolation.
  • Develop high-performance control-plane and data-plane services in Go and performance-critical components (tokenizers, schedulers, runtimes, gRPC/HTTP gateways, native extensions) in Rust.
  • Optimize GPU inference for throughput, latency, and cost using continuous batching, KV-cache management, quantization (INT8/FP8/AWQ/GPTQ), speculative decoding, tensor/pipeline parallelism, and efficient memory use.
  • Integrate and extend inference engines such as vLLM, TensorRT-LLM, Triton Inference Server, Ray Serve, KServe, SGLang, llama.cpp, and ONNX Runtime; contribute fixes upstream where it makes sense.
  • Build and manage GPU cluster infrastructure on Kubernetes (NVIDIA GPU Operator, device plugins, MIG/time-slicing, node pools, scheduling and bin-packing) across cloud and on-prem environments.
  • Own the full delivery path: CI/CD pipelines, model packaging and registries, canary and blue/green rollouts, infrastructure as code, and automated performance regression testing.
  • Build AI observability into everything: metrics, logs, and traces for token throughput, time-to-first-token, GPU utilization, queue depth, error and hallucination signals, using OpenTelemetry, Prometheus, and Grafana.
  • Provide clean APIs, SDKs, and developer tooling (OpenAI-compatible endpoints, internal services, and lightweight UIs or dashboards) so AI engineers and agent builders can deploy and monitor models self-serve.
  • Support agentic and RAG workloads by serving embedding, reranking, and generation models alongside the Enterprise Vector Store, with predictable latency and graceful degradation.
  • Drive reliability, security, and cost efficiency: SLOs, load and chaos testing, capacity planning, GPU cost attribution, and secure model and tenant isolation.

Who You’ll Work With

You will collaborate daily with AI researchers, ML engineers, data scientists, and platform and SRE teams who champion diversity, equity, and inclusion as fundamental to our success. You’ll be part of a cohesive team delivering high-quality, highly visible AI/ML functionality within the Teradata Vantage platform. You will report to the Sr. Manager, Software Engineering, who will champion your growth and empower your contributions.

What Makes You a Qualified Candidate

  • 3+ years of professional software engineering experience in backend systems, distributed systems, or AI/ML infrastructure.
  • Strong, production-level proficiency in Go (concurrency, gRPC, profiling, service design) and working proficiency, or a demonstrated passion for learning, Rust (ownership, async runtimes such as Tokio, FFI, performance tuning).
  • Hands-on experience with GPU computing for ML: CUDA fundamentals, GPU profiling (Nsight, nvidia-smi/DCGM), memory and bandwidth tradeoffs, and multi-GPU or multi-node inference.
  • Experience serving LLMs or generative models in production, with practical knowledge of inference optimization techniques and the serving frameworks listed above.
  • Solid understanding of transformer architectures and the LLM inference lifecycle (prefill vs. decode, attention, KV cache, sampling), plus familiarity with PyTorch and Hugging Face tooling.
  • Strong system design skills and a deep understanding of distributed systems, networking, concurrency, and failure modes.
  • Experience with containers and orchestration (Docker, Kubernetes, Helm) and CI/CD (GitHub Actions, Jenkins, Argo CD, or similar), plus infrastructure as code (Terraform).
  • Full-stack engineering experience: comfortable building REST/gRPC APIs and a pragmatic front end (React or similar) for dashboards, playgrounds, or admin tools; Python for glue, tooling, and ML integration.
  • Experience with observability practices and tools (OpenTelemetry, Prometheus, Grafana, tracing) applied to ML or LLM workloads.
  • You are obsessive about reliability, debuggability, and ensuring AI systems behave predictably and deterministically when needed.
  • You code daily, own systems end to end, and are knowledgeable about open-source tools and know how to leverage and extend them.

What You Bring

  • A Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
  • Genuine excitement for AI, LLMs, and the systems that make them efficient and affordable to run at scale.
  • Strong communication skills and a bias toward measuring, benchmarking, and iterating.
  • Familiarity with modern data platforms such as Teradata, Snowflake, or Databricks is a plus.

Bonus Points

  • Experience writing or tuning CUDA / Triton (OpenAI) / CUTLASS kernels, or contributing to inference engines.
  • Experience with on-device or edge ML inference (ONNX, TFLite, llama.cpp, quantized small language models), embedded or mobile deployment.
  • Knowledge of RAG pipelines, vector databases, embeddings, and agent frameworks (LangChain, LlamaIndex, ReAct-style agents).
  • Experience with model monitoring, drift detection, evaluation harnesses, and responsible AI practices.
  • Open-source contributions, technical writing, or public projects in ML systems, Go, or Rust.
  • Experience with service meshes, RDMA/InfiniBand/NCCL, Slurm, or Ray clusters.

#LI-VB1

About Teradata

People
About 1,709 in 2024.Benefit plan filings (Form 5500)
Company
Listed as TDC.
H-1B visas
1 approved for new hires in fiscal 2025.US Citizenship and Immigration Services

About the role

We file this job under AI Engineer, by the work the posting describes.

AI engineers build products on top of AI models that already exist, most often large language models: they connect a model to a company's data and tools and ship the assistants, agents and search features people use.

Postings ask for strong Python or TypeScript, experience with model APIs, retrieval and agent frameworks, and ways to test whether the model's answers are good enough.

An ML engineer trains and serves models; an AI engineer usually starts from a model someone else trained and builds the product around it, though titles often mix the two.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Hyderabad, Telangana

Our Company: At Teradata, we believe that people thrive when empowered with better information. Teradata Autonomous Knowledge Platform activates enterprise intelligence by unifying data, knowledge and business context to achieve tangible outcomes. With Teradata, organizations can provide agents with full context for impact when it matters. Our solution lets businesses connect and scale on premises, in the cloud, or through a hybrid approach. Teradata delivers real business value with AI. Power the Platform Behind Enterprise Generative AI
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.