Back to jobs

Senior Data Engineer

London, England, United Kingdom

Actively hiring, first seen by us 12 hours ago
Pay
Pay not listed
Work setting
Working pattern needs review — work setting source
State-of-the-art equipment (mac, iPhone, and mobile plan) Flexible WFH (Hybrid Model) - Fully Remote is also an option if you're not an Attica resident Our benefits package is designed to provide strong support to our team, but it may vary depending on location and type of employment (e.g., UK, Greece, or engagement through an Employer of Record).
Work setup in the employer’s posting: Remote
From the employer’s posting
Employment
Full-time — employment source
Employment type in the employer’s posting: Full-time
From the employer’s posting

What you’ll work on

Full posting
  • You'll work end-to-end on streaming and batch pipelines, from CDC and event ingestion through transformation, serving, and the feature layer that powers our ML and AI products.

  • You'll partner with ML engineers on feature pipelines, monitoring data drift, and keeping models well-fed and retrained as needed.

From the employer’s posting
✨The core mission of the Senior Data Engineer: You will own and evolve our data pipelines on GCP — building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation. You'll work end-to-end on streaming and batch pipelines, from CDC and event ingestion through transformation, serving, and the feature layer that powers our ML and AI products. Your day-to-day will include designing ELT/ETL processes on BigQuery and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark), orchestrating workflows with Airflow, and ensuring data is properly cleaned, modelled, and served for analytics, ML training, and online inference. You'll partner with ML engineers on feature pipelines, monitoring data drift, and keeping models well-fed and retrained as needed. You'll consume and build REST APIs, integrate with third-party SaaS sources, and treat infrastructure as code.
You will own and evolve our data pipelines on GCP — building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation. You'll work end-to-end on streaming and batch pipelines, from CDC and event ingestion through transformation, serving, and the feature layer that powers our ML and AI products. Your day-to-day will include designing ELT/ETL processes on BigQuery and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark), orchestrating workflows with Airflow, and ensuring data is properly cleaned, modelled, and served for analytics, ML training, and online inference. You'll partner with ML engineers on feature pipelines, monitoring data drift, and keeping models well-fed and retrained as needed. You'll consume and build REST APIs, integrate with third-party SaaS sources, and treat infrastructure as code. 🏢 Location & Work Mode:

About Hack The Box

We believe that diversity enriches our performance, services, and the communities we serve.

In the employer’s words · Read in context

Apply at Hack The Box

Posting record

Report an error

Actively hiringFirst seen by us today, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 18, 2026today
Recorded sightings
2
Last seen by us
Nov 19, 2026
Employer says posted
May 7, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 0

Early. Similar jobs usually come down between day 6 and day 56.

From 14,166 similar postings that came down. Coming down is not always a hire.

Hack The Box is hiring

1 open job

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
  • BigQuery
  • ClickHouse
  • dbt
  • Docker
  • GCP
  • Kafka
  • Kubernetes
  • Prefect
  • Redshift
  • Snowflake
  • Spark
  • Airflow
  • Dagster
Where the posting names them
- Analytical store: ClickHouse - Languages: Python, SQL - Modelling & quality: dbt, data quality gates
- Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub - Solid SQL and strong Python — you write production-quality code, not just notebooks - Experience with ClickHouse or another columnar OLAP engine in production
You will own and evolve our data pipelines on GCP — building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation. You'll work end-to-end on streaming and batch pipelines, from CDC and event ingestion through transformation, serving, and the feature layer that powers our ML and AI products. Your day-to-day will include designing ELT/ETL processes on BigQuery and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark), orchestrating workflows with Airflow, and ensuring data is properly cleaned, modelled, and served for analytics, ML training, and online inference. You'll partner with ML engineers on feature pipelines, monitoring data drift, and keeping models well-fed and retrained as needed. You'll consume and build REST APIs, integrate with third-party SaaS sources, and treat infrastructure as code. 🏢 Location & Work Mode:
⚔️ Technology tools & weapons you’ll be using: - Cloud & warehouse: GCP, BigQuery, Bigtable, Cloud Storage - Streaming & messaging: Pub/Sub, Kafka
🚀 The adventures that await you after becoming Senior Data Engineer at Hack The Box: - Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and ClickHouse - Help drive the migration off Snowflake onto our GCP-native stack — and retire shadow pipelines along the way
- Strong data modelling and warehouse architecture skills (dimensional modelling, event-driven, lakehouse patterns) - Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, Cloud Composer are strong pluses - Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub
- Orchestration: Airflow (Cloud Composer) - Analytical store: ClickHouse - Languages: Python, SQL
- Solid SQL and strong Python — you write production-quality code, not just notebooks - Experience with ClickHouse or another columnar OLAP engine in production - Workflow orchestration experience with Airflow (or Prefect/Dagster)
- Languages: Python, SQL - Modelling & quality: dbt, data quality gates - Containers & CI/CD: Docker, Kubernetes, GitHub Actions / equivalent
- Workflow orchestration experience with Airflow (or Prefect/Dagster) - Comfortable with dbt or equivalent transformation frameworks - Experience migrating off legacy warehouses (Snowflake, Redshift, Synapse) onto cloud-native stacks is a plus
- Modelling & quality: dbt, data quality gates - Containers & CI/CD: Docker, Kubernetes, GitHub Actions / equivalent - Legacy (being retired): Snowflake
- Working knowledge of ML in production — feature engineering, feature stores, model deployment, drift monitoring, retraining - Docker & Kubernetes experience - CI/CD mindset, infrastructure-as-code sensibility, and a bias for simple, observable systems
- Cloud & warehouse: GCP, BigQuery, Bigtable, Cloud Storage - Streaming & messaging: Pub/Sub, Kafka - Processing: Dataflow (Apache Beam), with Flink/Spark where appropriate
- Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, Cloud Composer are strong pluses - Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub - Solid SQL and strong Python — you write production-quality code, not just notebooks
- Experience with ClickHouse or another columnar OLAP engine in production - Workflow orchestration experience with Airflow (or Prefect/Dagster) - Comfortable with dbt or equivalent transformation frameworks
- Comfortable with dbt or equivalent transformation frameworks - Experience migrating off legacy warehouses (Snowflake, Redshift, Synapse) onto cloud-native stacks is a plus - Working knowledge of ML in production — feature engineering, feature stores, model deployment, drift monitoring, retraining
- Containers & CI/CD: Docker, Kubernetes, GitHub Actions / equivalent - Legacy (being retired): Snowflake 🚀 The adventures that await you after becoming Senior Data Engineer at Hack The Box:
- Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and ClickHouse - Help drive the migration off Snowflake onto our GCP-native stack — and retire shadow pipelines along the way - Own the orchestration layer in Airflow, including SLAs, retries, and data quality gates
- Processing: Dataflow (Apache Beam), with Flink/Spark where appropriate - Orchestration: Airflow (Cloud Composer) - Analytical store: ClickHouse
- Help drive the migration off Snowflake onto our GCP-native stack — and retire shadow pipelines along the way - Own the orchestration layer in Airflow, including SLAs, retries, and data quality gates - Model data for analytics and for ML — including feature pipelines that serve both training and low-latency online inference

Your next step

Complete your application on apply.workable.com. The employer’s form will show what is required.

Applying through Workable

  • No account needed. You can import your resume to fill in the form, or skip that and fill it in yourself.

Already applied? Track this application

Job description

View original posting ↗

Welcome! Super excited you dropped by 🥳
Let's redefine cyber security expertise standards and connect business - community through highly engaging hacking experiences. (Find out more insights about Hack The Box culture in our career site).

✨The core mission of the Senior Data Engineer:

You will own and evolve our data pipelines on GCP — building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation. You'll work end-to-end on streaming and batch pipelines, from CDC and event ingestion through transformation, serving, and the feature layer that powers our ML and AI products.

Your day-to-day will include designing ELT/ETL processes on BigQuery and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark), orchestrating workflows with Airflow, and ensuring data is properly cleaned, modelled, and served for analytics, ML training, and online inference. You'll partner with ML engineers on feature pipelines, monitoring data drift, and keeping models well-fed and retrained as needed. You'll consume and build REST APIs, integrate with third-party SaaS sources, and treat infrastructure as code.

🏢 Location &  Work Mode:

  • Europe/Greece
  • Choose between fully remote work across Europe or relocating to our Athens Tech Hub with relocation support.

Fully Remote / Hybrid (2 days in the office, 3 days remote, plus one month of work from anywhere) - If only in Greece:  When hiring in Greece, we’re open to candidates from all locations. Those based within 55 km of our Athens office will follow a hybrid work model. For candidates located beyond that radius, a fully remote arrangement is available.

✈️ Relocation

  • We offer a 10% relocation bonus on the annual salary to support your move.
  • In addition, you would benefit from a 50% tax reduction on your income as part of the relocation.
  • We'll support you with accommodation during the first few weeks

🍺 The fellowship you’ll be joining:

You will be part of the Data, Analytics & AI team, collaborating closely with Infrastructure, Software Engineering, Product, and ML/AI engineers. We're in the middle of a GCP-native modernisation — migrating away from Snowflake toward BigQuery, Bigtable, Pub/Sub, and Dataflow — so we're looking for someone who's opinionated about clean architecture, allergic to over-engineering, and comfortable owning systems end-to-end. If retiring a legacy warehouse and standing up its replacement sounds like a good time, you'll fit right in.

⚔️ Technology tools & weapons you’ll be using:

  • Cloud & warehouse: GCP, BigQuery, Bigtable, Cloud Storage
  • Streaming & messaging: Pub/Sub, Kafka
  • Processing: Dataflow (Apache Beam), with Flink/Spark where appropriate
  • Orchestration: Airflow (Cloud Composer)
  • Analytical store: ClickHouse
  • Languages: Python, SQL
  • Modelling & quality: dbt, data quality gates
  • Containers & CI/CD: Docker, Kubernetes, GitHub Actions / equivalent
  • Legacy (being retired): Snowflake

🚀 The adventures that await you after becoming Senior Data Engineer at Hack The Box:

  • Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and ClickHouse
  • Help drive the migration off Snowflake onto our GCP-native stack — and retire shadow pipelines along the way
  • Own the orchestration layer in Airflow, including SLAs, retries, and data quality gates
  • Model data for analytics and for ML — including feature pipelines that serve both training and low-latency online inference
  • Partner with ML engineers on feature stores, drift monitoring, and retraining workflows
  • Capture requirements from stakeholders and translate them into pragmatic, well-scoped data products
  • Continuously improve data quality, reliability, observability, and cost efficiency
  • Identify new data sources worth acquiring and integrate them cleanly

🏆 Skills, knowledge, and experience points required to unlock the role of the Senior Data Engineer at Hack The Box:

  • Strong data modelling and warehouse architecture skills (dimensional modelling, event-driven, lakehouse patterns)
  • Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, Cloud Composer are strong pluses
  • Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub
  • Solid SQL and strong Python — you write production-quality code, not just notebooks
  • Experience with ClickHouse or another columnar OLAP engine in production
  • Workflow orchestration experience with Airflow (or Prefect/Dagster)
  • Comfortable with dbt or equivalent transformation frameworks
  • Experience migrating off legacy warehouses (Snowflake, Redshift, Synapse) onto cloud-native stacks is a plus
  • Working knowledge of ML in production — feature engineering, feature stores, model deployment, drift monitoring, retraining
  • Docker & Kubernetes experience
  • CI/CD mindset, infrastructure-as-code sensibility, and a bias for simple, observable systems
  • Bonus: CDC tooling (Datastream, Debezium), Vertex AI / Feature Store

What your Hack The Box adventure will have in store: 

  • 🎯You'll have the exhilarating opportunity to contribute to a product that is highly appreciated by users and the cybersecurity community at large
  • 🎯 You'll experience a highly supportive and caring environment, fostering growth, flexibility, and autonomy
  • 🎯 You'll embark on an exciting journey of continuous learning and problem-solving, leveling up as our organization grows
  • 🎯 Most importantly, you'll have a blast at HTB 🥳 because fun is an essential ingredient in our recipe for success! Just wait until you see our global meet-ups! 

💰 The gems you’ll be enjoying as a Senior Data Engineer:

  • Private health care
  • Paid paternity leave
  • 25 annual leave days
  • Free lunch & snacks at the office
  • 120€ Ticket Restaurant by Edenred
  • Dedicated budget for training and professional development, participation in conferences
  • Full access to the Hack The Box lab offerings; so you can learn how to hack 😉
  • State-of-the-art equipment (mac, iPhone, and mobile plan)
  • Flexible WFH (Hybrid Model) - Fully Remote is also an option if you're not an Attica resident

Our benefits package is designed to provide strong support to our team, but it may vary depending on location and type of employment (e.g., UK, Greece, or engagement through an Employer of Record).

🗺️ The Quest of Becoming Hack The Box’s Senior Data Engineer:

  • Level 1: To complete level one’s objective, submit your application. 
  • Level 2: Meet the Talent Acquisition team. Level’s objective: highlight your past achievements, ambitions, and values.
  • Level 3: Meet the hiring team. Level’s objective: connect with the hiring team and share with them your achievements. 
  • Level 4: Complete 2 assignments that align with day-to-day job-related tasks and responsibilities. Part of the assignment is discussing it with the hiring team in a debriefing session, in order to walk the team through your thinking process. 
  • Level 5: Congratulations! Not many reach this level 💪. Level’s objective: have a constructive, final conversation with senior leadership to explore the role and your future at HTB. 
  • Level 6: You've officially received an offer from HTB! To complete the last level and the Quest, all you need to do is accept the offer. 
  • Quest complete. Congratulations, you’re officially one of us 🥳🎉🎇Your next quest: complete the onboarding.

Hack Your Career, Today. Join us in this epic adventure of cybersecurity at Hack The Box! 🚀🔒💻

At Hack The Box, we are on a quest to find the most exceptional and enthusiastic talent to join our team. Whether or not you consider yourself a gamer, we value what makes you unique and want to know more about you. This job post provides just a glimpse of the incredible gamified experience our business and consumer customers enjoy through our platforms. So, if you're ready to embark on a journey of growth and adventure, we can't wait to meet you!

ABOUT HACK THE BOX

Hack The Box is the Cyber Performance Center with the mission to provide a human-first platform to create and maintain high-performing cybersecurity individuals and organizations. 

Hack The Box is the only platform that unites upskilling, workforce development, and the human focus in the cybersecurity industry, and it’s trusted by organizations worldwide for driving their teams to peak performance. Offering an all-in-one environment for continuous growth, assessment, and recruitment, Hack The Box provides solutions for all cybersecurity domains. 

Launched in 2017, Hack The Box brings together the largest global cybersecurity community of more than 3 million platform members. Rapidly growing its international footprint and reach, Hack The Box is headquartered in the UK, with additional offices in the US, Australia, and Greece.

🚨 Exciting News:

At Hack The Box, we are committed to fostering a diverse, inclusive, and equitable workplace. We believe that diversity enriches our performance, services, and the communities we serve. As such, we ensure that all job applications are considered solely based on merit, skills, and qualifications. We do not discriminate on grounds of race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. We are dedicated to providing a fair and respectful work environment that reflects our values.

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 14% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 860, the most 1,353, and the latest week had 1,020.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

London, England, United Kingdom

- Europe/Greece - Choose between fully remote work across Europe or relocating to our Athens Tech Hub with relocation support. Fully Remote / Hybrid (2 days in the office, 3 days remote, plus one month of work from anywhere) - If only in Greece: When hiring in Greece, we’re open to candidates from all locations. Those based within 55 km of our Athens office will follow a hybrid work model. For candidates located beyond that radius, a fully remote arrangement is available. ✈️ Relocation
More source context
- State-of-the-art equipment (mac, iPhone, and mobile plan) - Flexible WFH (Hybrid Model) - Fully Remote is also an option if you're not an Attica resident Our benefits package is designed to provide strong support to our team, but it may vary depending on location and type of employment (e.g., UK, Greece, or engagement through an Employer of Record).
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.