Back to jobs

Senior AI Developer

Pay
Pay not listed
Work setting
Remote — work setting source
Location type: Remote
From the employer’s posting
Employment
Full-time — employment source
Employment type: Full-time
From the employer’s posting
Team
Engineering › Eng › 32010 Backend Engineering — team source
Department: Engineering › Eng › 32010 Backend Engineering
From the employer’s posting

What you’ll work on

Full posting
  • Design, build, and own ClickUp's AI evaluation system, including golden datasets, LLM-as-judge and automated scorers, regression detection, and quality tracking across AI features.

  • Build and evolve the core agent harness, and the AI that performs well on those evals, partnering closely with product and engineering to define what "great" looks like.

  • Build self-improvement loops where agents modify other agents or prompts, rerun evals, and iterate automatically.

From the employer’s posting
Key Responsibilities: Design, build, and own ClickUp's AI evaluation system, including golden datasets, LLM-as-judge and automated scorers, regression detection, and quality tracking across AI features. Build and evolve the core agent harness, and the AI that performs well on those evals, partnering closely with product and engineering to define what "great" looks like.
Design, build, and own ClickUp's AI evaluation system, including golden datasets, LLM-as-judge and automated scorers, regression detection, and quality tracking across AI features. Build and evolve the core agent harness, and the AI that performs well on those evals, partnering closely with product and engineering to define what "great" looks like. Build self-improvement loops where agents modify other agents or prompts, rerun evals, and iterate automatically.
Build and evolve the core agent harness, and the AI that performs well on those evals, partnering closely with product and engineering to define what "great" looks like. Build self-improvement loops where agents modify other agents or prompts, rerun evals, and iterate automatically. Benchmark new models as they're released and own model selection, routing, and fallback strategies across providers (e.g., OpenAI, Anthropic, Google, open-weight), balancing quality, latency, and cost.

Before you apply

Work authorization
Must already have the right to workPlease note we are unable to sponsor or take over sponsorship of an employment visa for roles outside of engineering and product at this time

What you’ll bring

All qualifications

Core experience

  • 5+ years of software engineering experience, including 2+ years building production LLM or ML systems.
  • Experience with eval tooling (e.g., Braintrust, LangSmith, Weave, Arize, or in-house frameworks).
  • Hands-on experience building LLM evaluation systems in production (golden datasets, LLM-as-judge, judge calibration, offline and online evals), tied to real product quality.
  • Experience with model routing, AI gateways, or inference optimization.
  • Proven experience integrating and managing multiple LLMs, including benchmarking, routing, and cost/latency/quality tradeoffs.
  • Deep expertise in backend engineering, distributed systems, and cloud-native technologies (e.g., Kubernetes, Docker, AWS/GCP/Azure).
Qualification wording
5+ years of software engineering experience, including 2+ years building production LLM or ML systems.
Experience with eval tooling (e.g., Braintrust, LangSmith, Weave, Arize, or in-house frameworks).
Hands-on experience building LLM evaluation systems in production (golden datasets, LLM-as-judge, judge calibration, offline and online evals), tied to real product quality.
Experience with model routing, AI gateways, or inference optimization.
Proven experience integrating and managing multiple LLMs, including benchmarking, routing, and cost/latency/quality tradeoffs.
Deep expertise in backend engineering, distributed systems, and cloud-native technologies (e.g., Kubernetes, Docker, AWS/GCP/Azure).
Apply at ClickUp

Posting record

Report an error
Status in our records
Live
First seen by us
Nov 7, 202614 days ago
Recorded sightings
16
Last seen by us
Nov 22, 2026
Reposted
once since Nov 6, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications. We count a repost when the same title and location comes back as a new listing within 60 days of the last one coming down. Some hiring systems expire listings on a schedule, so repost rates vary by system.

How this job compares

Time since we first saw it

Day 14

Similar jobs usually come down between day 1 and day 15.

From 1,376 similar postings that came down. Coming down is not always a hire.

ClickUp is hiring

5 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Takes jobs down at a usual pace
  • 20% of open jobs were reposted
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • TypeScript
  • AWS
  • Azure
  • Docker
  • GCP
  • Kubernetes
  • Airflow
  • Go
Where the posting names them
- Deep expertise in backend engineering, distributed systems, and cloud-native technologies (e.g., Kubernetes, Docker, AWS/GCP/Azure). - Strong programming skills in Python (preferred), Go, TypeScript, or similar languages used for backend and AI platform development. - Proficiency in orchestration frameworks and workflow engines (e.g., LangGraph, Airflow, Ray, or similar).
- Proven experience integrating and managing multiple LLMs, including benchmarking, routing, and cost/latency/quality tradeoffs. - Deep expertise in backend engineering, distributed systems, and cloud-native technologies (e.g., Kubernetes, Docker, AWS/GCP/Azure). - Strong programming skills in Python (preferred), Go, TypeScript, or similar languages used for backend and AI platform development.
- Strong programming skills in Python (preferred), Go, TypeScript, or similar languages used for backend and AI platform development. - Proficiency in orchestration frameworks and workflow engines (e.g., LangGraph, Airflow, Ray, or similar). - Familiarity with search and retrieval technologies and their integration into AI-driven applications.

Your next step

Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.

Applying through Ashby

  • No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.

Already applied? Track this application

Job description

View original posting ↗

At ClickUp, we're building the future of work: the first truly converged AI workspace unifying tasks, docs, chat, calendar, and enterprise search, all supercharged by context-driven AI. We are an AI-native company. Every team member is expected to leverage AI daily, and we evaluate AI fluency as part of our hiring process. Join us and help redefine what's possible. 🚀

We're looking for a highly skilled and experienced Senior AI Engineer to join our ClickUp Engineering team, with a focus on AI evaluations. You'll own how we measure and improve the quality of ClickUp's AI: building the eval systems that tell us how well our AI is performing, the core agent harness, and the AI that performs well on those evals. You'll work on backend systems that make AI-powered capabilities scalable, reliable, and secure, while working hands-on with LLMs to solve real user problems and drive product innovation.

 

Key Responsibilities:

  • Design, build, and own ClickUp's AI evaluation system, including golden datasets, LLM-as-judge and automated scorers, regression detection, and quality tracking across AI features.

  • Build and evolve the core agent harness, and the AI that performs well on those evals, partnering closely with product and engineering to define what "great" looks like.

  • Build self-improvement loops where agents modify other agents or prompts, rerun evals, and iterate automatically.

  • Benchmark new models as they're released and own model selection, routing, and fallback strategies across providers (e.g., OpenAI, Anthropic, Google, open-weight), balancing quality, latency, and cost.

  • Architect, design, and implement scalable AI platform services that support the deployment, orchestration, and lifecycle management of LLMs and other AI models.

  • Build and maintain robust APIs and backend systems that integrate AI-powered features into ClickUp's core platform.

  • Develop infrastructure for model serving, monitoring, logging, and automated evaluation to ensure high reliability and performance of AI services in production.

  • Drive best practices in AI privacy, security, and compliance, including data anonymization, secure data handling, and regulatory adherence.

  • Collaborate cross-functionally with product, frontend, and data teams to deliver seamless, reliable, and user-centric AI experiences.

 

Qualifications:

  • 5+ years of software engineering experience, including 2+ years building production LLM or ML systems.

  • Hands-on experience building LLM evaluation systems in production (golden datasets, LLM-as-judge, judge calibration, offline and online evals), tied to real product quality.

  • Strong experiment design and statistical rigor, including A/B testing, offline vs. online metrics, and separating real regressions from noisy or variable results.

  • Proven track record of applying LLMs and AI models to real-world, customer-facing product features.

  • Proven experience integrating and managing multiple LLMs, including benchmarking, routing, and cost/latency/quality tradeoffs.

  • Deep expertise in backend engineering, distributed systems, and cloud-native technologies (e.g., Kubernetes, Docker, AWS/GCP/Azure).

  • Strong programming skills in Python (preferred), Go, TypeScript, or similar languages used for backend and AI platform development.

  • Proficiency in orchestration frameworks and workflow engines (e.g., LangGraph, Airflow, Ray, or similar).

  • Familiarity with search and retrieval technologies and their integration into AI-driven applications.

  • Excellent collaboration and communication skills, with a track record of working effectively in cross-functional teams.

  • Passion for staying at the forefront of AI and applying new technologies to solve real-world problems at scale.

 

Nice to Have:

  • Background in data science, applied science, or machine learning engineering.

  • Experience with eval tooling (e.g., Braintrust, LangSmith, Weave, Arize, or in-house frameworks).

  • Experience with model routing, AI gateways, or inference optimization.

 

#LI-REMOTE

#LI-AK2
#LI-CC1

Equal Opportunity Employer

ClickUp is an Equal Opportunity Employer, and qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.

Privacy Notice

ClickUp collects and processes personal data in accordance with applicable data protection laws. You can find further details by viewing our Global Candidate Privacy Notice.

If you are a Philippine Job Applicant, please also see our Philippine Data Privacy Notice for further details.

Visa Sponsorship

Please note we are unable to sponsor or take over sponsorship of an employment visa for roles outside of engineering and product at this time. Sponsorship for engineering and product roles is not guaranteed, but is instead based on the business needs for that specific role at that time. Please reach out to the recruiter with any questions.

Fraud Alert

ClickUp Talent Acquisition will only initiate contact via an @clickup.com email or through our official careers portal on clickup.com. We will never request fees, payments, or sensitive personal information. Please disregard any offers received outside these channels and report them to support@clickup.com.

AI Processing Notice

ClickUp may use artificial intelligence and machine learning technologies to help review and screen candidates' employment applications against role-related criteria. These tools support, but do not replace, human decision‑making. If you have questions or need an accommodation in the recruitment process, please contact us at AskPeople@ClickUp.com.

Demand for AI Engineer roles

New AI Engineer postings at the employers we follow are up 340% over the last 6 weeks compared with the 6 before.

Weekly new AI Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 78, the most 803, and the latest week had 438.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under AI Engineer, by the work the posting describes.

AI engineers build products on top of AI models that already exist, most often large language models: they connect a model to a company's data and tools and ship the assistants, agents and search features people use.

Postings ask for strong Python or TypeScript, experience with model APIs, retrieval and agent frameworks, and ways to test whether the model's answers are good enough.

An ML engineer trains and serves models; an AI engineer usually starts from a model someone else trained and builds the product around it, though titles often mix the two.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

United States

- Experience with model routing, AI gateways, or inference optimization. #LI-REMOTE #LI-AK2 #LI-CC1
Work authorization
If you are a Philippine Job Applicant, please also see our Philippine Data Privacy Notice for further details. Visa Sponsorship Please note we are unable to sponsor or take over sponsorship of an employment visa for roles outside of engineering and product at this time. Sponsorship for engineering and product roles is not guaranteed, but is instead based on the business needs for that specific role at that time. Please reach out to the recruiter with any questions. Fraud Alert

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.