Extend the distributed dataset API with new operators and optimizations
Design ML pipelines covering training, tuning, serving, validation and retraining
Improve reliability, observability and multi-user features of compute infrastructure
Build abstractions that keep the platform flexible as compute backends change
Write, test and maintain code with unit tests, documentation and CI/CD
From the employer’s posting
Extend the proprietary distributed dataset API, a Polars-style, lazily evaluated, DAG-planned DataFrame surface, with new operators, optimizations, and a strong new-user experience
Design and implement machine learning pipelines spanning large-scale training and tuning, model serving, production validation, and the full retraining lifecycle for a growing model fleet
Improve the reliability, observability, and multi-user capabilities of the compute infrastructure, including resource management, workload isolation, and clean backend interfaces
Build durable abstractions that allow the platform to remain flexible as distributed compute backends evolve
Write, support, maintain, and test code using strong engineering practices, including unit testing, documentation, automation, and CI/CD workflows
Open 3 monthsWe first saw this posting 113 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
Status in our records
Live
First seen by us
Jul 28, 2026113 days ago
Recorded sightings
238
Last seen by us
Nov 19, 2026
Jul 28, 2026today
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
How this job compares
Time since we first saw it
Day 113
656
Up longer than most. Similar jobs usually come down between day 6 and day 56.
Well past Millennium’s usual: its jobs come down by about day 51.
From 14,133 similar postings that came down, and 33 of Millennium’s. Coming down is not always a hire.
Millennium is hiring
23 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
Bash
Python
Kubernetes
Spark
Dask
Polars
PySpark
PyTorch
XGBoost
Where the posting names them
What You'll Do • Extend the proprietary distributed dataset API, a Polars-style, lazily evaluated, DAG-planned DataFrame surface, with new operators, optimizations, and a strong new-user experience • Design and implement machine learning pipelines spanning large-scale training and tuning, model serving, production validation, and the full retraining lifecycle for a growing model fleet • Improve the reliability, observability, and multi-user capabilities of the compute infrastructure, including resource management, workload isolation, and clean backend interfaces • Build durable abstractions that allow the platform to remain flexible as distributed compute backends evolve • Write, support, maintain, and test code using strong engineering practices, including unit testing, documentation, automation, and CI/CD workflows • Partner with team members on the direction, design, and architecture of the platform • Collaborate closely with researchers and platform engineers to align compute and storage capabilities across the environment
What You Bring • 6+ years of Python and distributed systems experience in a quantitative finance environment, with a strong track record of building platform abstractions used by others in production • Bachelor’s degree in Computer Science, Mathematics, Financial Engineering, Operations Research, or a related field • Deep working knowledge of one or more distributed compute frameworks such as Ray, Dask, Spark, PySpark, or Slurm, with a clear understanding of their tradeoffs • Hands-on experience implementing machine learning and deep learning workflows using tools such as PyTorch and XGBoost, with comfort across the full training-to-serving lifecycle • Strong Python skills, and optionally kdb+, for platform and research-focused development • Strong working knowledge of Git, Unix/Linux, Bash, and modern CI/CD workflows • Strong communication skills and the ability to work effectively in a collaborative team environment • Experience with kdb+, PyKX, Polars, Arrow, columnar or lazy query engines, cash equities, live analytics, and/or cloud tooling including Kubernetes is a plus
Meet the Team The Execution Services team supports trading across products and geographies, with a focus on execution quality, trading efficiency, and the tools that help portfolio managers make informed decisions. Within this group, Central Liquidity Strategies develops and operates trading capabilities that support internal liquidity solutions across the platform. The team works across quantitative research, platform engineering, and production systems to build the compute, storage, and model infrastructure that supports research and live trading.
What You'll Do • Extend the proprietary distributed dataset API, a Polars-style, lazily evaluated, DAG-planned DataFrame surface, with new operators, optimizations, and a strong new-user experience • Design and implement machine learning pipelines spanning large-scale training and tuning, model serving, production validation, and the full retraining lifecycle for a growing model fleet • Improve the reliability, observability, and multi-user capabilities of the compute infrastructure, including resource management, workload isolation, and clean backend interfaces • Build durable abstractions that allow the platform to remain flexible as distributed compute backends evolve • Write, support, maintain, and test code using strong engineering practices, including unit testing, documentation, automation, and CI/CD workflows • Partner with team members on the direction, design, and architecture of the platform • Collaborate closely with researchers and platform engineers to align compute and storage capabilities across the environment
What You Bring • 6+ years of Python and distributed systems experience in a quantitative finance environment, with a strong track record of building platform abstractions used by others in production • Bachelor’s degree in Computer Science, Mathematics, Financial Engineering, Operations Research, or a related field • Deep working knowledge of one or more distributed compute frameworks such as Ray, Dask, Spark, PySpark, or Slurm, with a clear understanding of their tradeoffs • Hands-on experience implementing machine learning and deep learning workflows using tools such as PyTorch and XGBoost, with comfort across the full training-to-serving lifecycle • Strong Python skills, and optionally kdb+, for platform and research-focused development • Strong working knowledge of Git, Unix/Linux, Bash, and modern CI/CD workflows • Strong communication skills and the ability to work effectively in a collaborative team environment • Experience with kdb+, PyKX, Polars, Arrow, columnar or lazy query engines, cash equities, live analytics, and/or cloud tooling including Kubernetes is a plus
Quantitative Developer – Compute & ML Platform
About Millennium
Millennium is a global, diversified alternative investment firm, founded in 1989. Defined by evolution, innovation and focus, Millennium’s mission is to deliver results for our investors.
Our people are empowered with both independence and support: the autonomy to pursue ideas with conviction and the backing of a global network committed to collaboration, disciplined risk management and continuous learning. With opportunities to deepen expertise and accelerate development, talent at Millennium is equipped to adapt, evolve and build lasting impact over time. Discover how transformative growth accelerates impact.
Meet the Team
The Execution Services team supports trading across products and geographies, with a focus on execution quality, trading efficiency, and the tools that help portfolio managers make informed decisions. Within this group, Central Liquidity Strategies develops and operates trading capabilities that support internal liquidity solutions across the platform. The team works across quantitative research, platform engineering, and production systems to build the compute, storage, and model infrastructure that supports research and live trading.
What You'll Do
• Extend the proprietary distributed dataset API, a Polars-style, lazily evaluated, DAG-planned DataFrame surface, with new operators, optimizations, and a strong new-user experience
• Design and implement machine learning pipelines spanning large-scale training and tuning, model serving, production validation, and the full retraining lifecycle for a growing model fleet
• Improve the reliability, observability, and multi-user capabilities of the compute infrastructure, including resource management, workload isolation, and clean backend interfaces
• Build durable abstractions that allow the platform to remain flexible as distributed compute backends evolve
• Write, support, maintain, and test code using strong engineering practices, including unit testing, documentation, automation, and CI/CD workflows
• Partner with team members on the direction, design, and architecture of the platform
• Collaborate closely with researchers and platform engineers to align compute and storage capabilities across the environment
What You Bring
• 6+ years of Python and distributed systems experience in a quantitative finance environment, with a strong track record of building platform abstractions used by others in production
• Bachelor’s degree in Computer Science, Mathematics, Financial Engineering, Operations Research, or a related field
• Deep working knowledge of one or more distributed compute frameworks such as Ray, Dask, Spark, PySpark, or Slurm, with a clear understanding of their tradeoffs
• Hands-on experience implementing machine learning and deep learning workflows using tools such as PyTorch and XGBoost, with comfort across the full training-to-serving lifecycle
• Strong Python skills, and optionally kdb+, for platform and research-focused development
• Strong working knowledge of Git, Unix/Linux, Bash, and modern CI/CD workflows
• Strong communication skills and the ability to work effectively in a collaborative team environment
• Experience with kdb+, PyKX, Polars, Arrow, columnar or lazy query engines, cash equities, live analytics, and/or cloud tooling including Kubernetes is a plus
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 14% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.