Back to jobs

Data Engineer

London · London, United Kingdom

Actively hiring, first seen by us 6 days ago
Pay
Pay not listed
Work setting
Hybrid — work setting source
Location type: Hybrid
From the employer’s posting
Employment
Full-time — employment source
Employment type: Full-time
From the employer’s posting
Team
Tech — team source

What you’ll work on

Full posting

You will work very closely with the Data team lead and the other team members who will be assisting you whenever needed, making your integration in the company as smooth as possible.

The Data team works in an organised way using Agile methodologies and tools such as Jira and regular standups.

  • Develop and maintain ETL pipelines, some running in real time

  • Keep the data warehouses (Redshift and BigQuery) healthy

  • Develop and maintain Flask services and Postgres databases for the Data team

  • Build automated reporting pipelines for other teams

  • Train and deploy ML models for user segmentation and anomaly detection

From the employer’s posting
You will work very closely with the Data team lead and the other team members who will be assisting you whenever needed, making your integration in the company as smooth as possible.
The Data team works in an organised way using Agile methodologies and tools such as Jira and regular standups. You will find an environment where you have a clear engineering direction, can focus on your work and hone your skills as a data engineer through exciting projects. You will always be able to count on the support of many engineers across the company.
Developing and maintaining our Data ETL pipelines, some of which are real-time
Ensuring our data lake is kept in a healthy state, particularly our data warehouses: Redshift and Bigquery
Developing and maintaining Flask services and Postgres databases within the Data team
Assisting the different teams in the company with reporting, especially when it comes to automated reporting pipelines
training & deploying ML models used to perform different kinds of user segmentation, detect operational anomalies or estimate important business quantities
Education & alternatives
- You are a team player that enjoys contributing to the success of the team in a proactive and friendly environment; - You have a bachelor’s in Computer Science, Math or equivalent field, or possess relevant experience. Our Values
Apply at Smarkets

Posting record

Report an error

Actively hiringFirst seen by us 6 days ago, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 12, 20266 days ago
Recorded sightings
7
Last seen by us
Nov 18, 2026
Employer says posted
Nov 9, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 6

Similar jobs usually come down between day 6 and day 56.

From 14,133 similar postings that came down. Coming down is not always a hire.

Smarkets is hiring

1 open job

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • C++
  • Python
  • Rust
  • AWS
  • BigQuery
  • dbt
  • Docker
  • Grafana
  • Kafka
  • Kibana
  • Kubernetes
  • Redshift
  • Airflow
  • FastAPI
  • Flask
  • Luigi
  • NumPy
  • pandas
  • scikit-learn
  • PostgreSQL
Where the posting names them
Smarkets runs one of the world’s most sophisticated prediction markets, handling over £29 billion in volume since 2010 and engaging 200,000+ traders globally. We’re a technology company that happens to revolutionise betting - from sports to political markets, to delivering the sharpest prices and the fairest odds. Our stack is designed for scale, reliability, and speed: Linux, Kafka, Postgres, Kubernetes, with Python 3, C++, Rust and React powering our platform. We build infrastructure that institutions trust while keeping trading accessible to everyone. Our edge? We’ve thrived through every market cycle, competitor, and industry revolution. At the heart of our success are our people. We create a high-performance environment where exceptional talent can thrive, blending deep business experience with a commercial mindset to drive strategic growth.
In a typical week, a data engineer in the Data team would: - Add a new python ETL pipeline that segments users interested in specific sports through analysing behaviour which streamlines and tailors marketing communications to those users; - Develop a new endpoint to a Flask API, add unit tests, and deploy the new version of the API into our production Kubernetes cluster;
Team Tech Stack Our current technology stack primarily includes Linux, Docker, Kubernetes, Jenkins, Kafka, Python, Flask/FastAPI, Postgres, AWS Redshift, dbt, Google Bigquery, Prometheus, Grafana, Elastic Search, Kibana, Lightdash. About the Role:
- 3+ years of experience - You have an understanding of developing ETL pipelines using Python frameworks such as luigi or airflow; - You have experience with the development of Python-based REST APIs / services and their integration with databases (e.g. Postgres);
- You have an understanding of developing ETL pipelines using Python frameworks such as luigi or airflow; - You have experience with the development of Python-based REST APIs / services and their integration with databases (e.g. Postgres); - You are familiar with the key tools of the Python data science stack, e.g. Pandas, Numpy and scikit-learn;
- You have experience with the development of Python-based REST APIs / services and their integration with databases (e.g. Postgres); - You are familiar with the key tools of the Python data science stack, e.g. Pandas, Numpy and scikit-learn; - Some experience with training and deploying ML models;
The team’s responsibilities currently span across three different areas: - Data Engineering: development and maintenance of ETL pipelines, services and APIs, and data-related infrastructure like Redshift or BigQuery; - Data Science and Machine Learning: data exploration, ML models training and ML Ops to extract new insights from data;
- Developing and maintaining our Data ETL pipelines, some of which are real-time. The pipelines are fundamental to helping teams and stakeholders understand and drive business direction. Data components can also be user facing e.g. sending notifications to users; - Ensuring our data lake is kept in a healthy state, particularly our data warehouses: Redshift and Bigquery; - Developing and maintaining Flask services and Postgres databases within the Data team to provide access to data or manage certain business entities relevant to Data.
- Add a new python ETL pipeline that segments users interested in specific sports through analysing behaviour which streamlines and tailors marketing communications to those users; - Develop a new endpoint to a Flask API, add unit tests, and deploy the new version of the API into our production Kubernetes cluster; - Train and evaluate an ML model to identify certain user patterns and provide it as service to other engineering teams in a Flask API;
- Develop a new endpoint to a Flask API, add unit tests, and deploy the new version of the API into our production Kubernetes cluster; - Train and evaluate an ML model to identify certain user patterns and provide it as service to other engineering teams in a Flask API; Team Tech Stack
- Ensuring our data lake is kept in a healthy state, particularly our data warehouses: Redshift and Bigquery; - Developing and maintaining Flask services and Postgres databases within the Data team to provide access to data or manage certain business entities relevant to Data. - Assisting the different teams in the company with reporting, especially when it comes to automated reporting pipelines.

Your next step

Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.

Applying through Ashby

  • No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.

Already applied? Track this application

Job description

View original posting ↗

Who are we?

Smarkets: Predicting the Future of Betting

Smarkets runs one of the world’s most sophisticated prediction markets, handling over £29 billion in volume since 2010 and engaging 200,000+ traders globally. We’re a technology company that happens to revolutionise betting - from sports to political markets, to delivering the sharpest prices and the fairest odds.

Our stack is designed for scale, reliability, and speed: Linux, Kafka, Postgres, Kubernetes, with Python 3, C++, Rust and React powering our platform. We build infrastructure that institutions trust while keeping trading accessible to everyone. Our edge? We’ve thrived through every market cycle, competitor, and industry revolution.

At the heart of our success are our people. We create a high-performance environment where exceptional talent can thrive, blending deep business experience with a commercial mindset to drive strategic growth.

If you’re ready to help shape the future of prediction markets with cutting-edge technology and a customer-first philosophy, Smarkets is where you belong.

The Team

The Data Team is responsible for taking the wealth of data that Smarkets generates and using it to drive insights which improve the business. Since Smarkets produces a huge amount of data - including sports event data, payments information, order flow and user analytics - there are many opportunities for the team to add real business value.

The team’s responsibilities currently span across three different areas:

  • Data Engineering: development and maintenance of ETL pipelines, services and APIs, and data-related infrastructure like Redshift or BigQuery;

  • Data Science and Machine Learning: data exploration, ML models training and ML Ops to extract new insights from data;

  • Analytics and Reporting: creation of data models and dashboards as well as automation of reporting pipelines for different teams, stakeholders and third-parties.

In a typical week, a data engineer in the Data team would:

  • Add a new python ETL pipeline that segments users interested in specific sports through analysing behaviour which streamlines and tailors marketing communications to those users;

  • Develop a new endpoint to a Flask API, add unit tests, and deploy the new version of the API into our production Kubernetes cluster;

  • Train and evaluate an ML model to identify certain user patterns and provide it as service to other engineering teams in a Flask API;

Team Tech Stack

Our current technology stack primarily includes Linux, Docker, Kubernetes, Jenkins, Kafka, Python, Flask/FastAPI, Postgres, AWS Redshift, dbt, Google Bigquery, Prometheus, Grafana, Elastic Search, Kibana, Lightdash.

About the Role:

You will work very closely with the Data team lead and the other team members who will be assisting you whenever needed, making your integration in the company as smooth as possible.

The Data team works in an organised way using Agile methodologies and tools such as Jira and regular standups. You will find an environment where you have a clear engineering direction, can focus on your work and hone your skills as a data engineer through exciting projects. You will always be able to count on the support of many engineers across the company.

What you will do:

As a member of the data team, your responsibilities will include contributions to:

  • Developing and maintaining our Data ETL pipelines, some of which are real-time. The pipelines are fundamental to helping teams and stakeholders understand and drive business direction. Data components can also be user facing e.g. sending notifications to users;

  • Ensuring our data lake is kept in a healthy state, particularly our data warehouses: Redshift and Bigquery;

  • Developing and maintaining Flask services and Postgres databases within the Data team to provide access to data or manage certain business entities relevant to Data.

  • Assisting the different teams in the company with reporting, especially when it comes to automated reporting pipelines.

  • Doing data exploration and training & deploying ML models used to perform different kinds of user segmentation, detect operational anomalies or estimate important business quantities. In particular maintain and improve our existing recommender service that provides users with suggestions on new sport competitions.

Role Requirements:

What we are looking for in you:

  • 3+ years of experience

  • You have an understanding of developing ETL pipelines using Python frameworks such as luigi or airflow;

  • You have experience with the development of Python-based REST APIs / services and their integration with databases (e.g. Postgres);

  • You are familiar with the key tools of the Python data science stack, e.g. Pandas, Numpy and scikit-learn;

  • Some experience with training and deploying ML models;

  • You enjoy writing elegant, well-tested and maintainable code;

  • You are a team player that enjoys contributing to the success of the team in a proactive and friendly environment;

  • You have a bachelor’s in Computer Science, Math or equivalent field, or possess relevant experience.

Our Values

Our values are at the heart of everything we do - guiding how we work, collaborate, and innovate. They reflect what we expect of ourselves and each other to deliver the best results, while fostering a positive, high-performing environment:

  • Push to Win: We set ambitious goals and relentlessly pursue them, always striving for excellence.

  • Make Others Better: We lift each other up, share knowledge, and celebrate team success over individual achievement.

  • Give a Shit: We care deeply about our work, our users, and the impact we make.

  • Be a Pro: We take ownership, act with integrity, and consistently deliver to a high standard.

  • Bring the Energy: We bring positivity, curiosity, and enthusiasm to everything we do, inspiring those around us.

Culture Fit: These values define how we succeed as a team. If this isn’t you, this may not be the right place. We hire people who thrive in a fast-paced, collaborative, and ambitious environment.

Perks & Benefits
We offer a competitive salary and benefits package, alongside a dynamic, collaborative environment where your work truly makes an impact and your voice is heard. Our team is diverse, driven, and ambitious - united by a strong work ethic and a hunger to innovate and win.

Our benefits are designed around Health, Wealth,and Development, supporting you both professionally and personally.

These include:

  • Stock Options: Vest over 4 years - your success is our success.

  • Pension Scheme: Competitive plan via Aviva, with up to 6% matched contributions if you opt in.

  • Health Insurance: Comprehensive coverage to keep you and your family healthy.

  • Enhanced Parental Leave: Enhanced maternity and paternity leave to support you through life’s most important moments.

  • Stay Fuelled: Enjoy fresh, chef-made breakfast and lunch every day, plus a constant supply of fruit, snacks, tea, coffee and soft drinks — because great work starts with great food.

  • Cycle-to-Work Scheme: Support for sustainable commuting and staying active.

  • Learning & Development: £1,000 annual education budget for courses, conferences, books, or training.

  • Holiday: 25 days paid leave + bank holidays, with the option to carry over 5 days.

  • Flexible Working: Hybrid model with 3 days in the office and 2 days from home to fit your lifestyle.

  • Team Energy: Regular socials, hackathons, and events — because collaboration and fun go hand-in-hand.

Diversity & Inclusion

We’re an equal opportunities employer and celebrate diversity in all its forms. If you need any adjustments during the recruitment process, please let us know — we’re happy to accommodate your needs.

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 14% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 860, the most 1,353, and the latest week had 1,020.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

London, United Kingdom

- Holiday: 25 days paid leave + bank holidays, with the option to carry over 5 days. - Flexible Working: Hybrid model with 3 days in the office and 2 days from home to fit your lifestyle. - Team Energy: Regular socials, hackathons, and events — because collaboration and fun go hand-in-hand.
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.