Back to jobs

Data Engineering Leadership

Pay
Pay not listed
Work setting
Hybrid — work setting source
Work setup in the employer’s posting: Hybrid
From the employer’s posting
Employment
Full-time — employment source
Employment type in the employer’s posting: FULL_TIME
From the employer’s posting

What you’ll work on

Full posting
  • Design high-scale APIs for multidimensional reporting and analytical workloads

  • Build large-scale Spark pipelines and streaming systems using Kafka and Flink

  • Transform raw event data into canonical, production-grade datasets

  • Improve dynamic query generation, caching, response shaping, and execution performance

  • Support historical backfills, incremental processing, and real-time availability

From the employer’s posting
Design high-scale APIs for multidimensional reporting and analytical workloads
Build large-scale Spark pipelines and streaming systems using Kafka and Flink
Transform raw event data into canonical, production-grade datasets
Improve dynamic query generation, caching, response shaping, and execution performance
Support historical backfills, incremental processing, and real-time availability
Apply at Lutra

Posting record

Report an error

Actively hiringFirst seen by us 3 days ago, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 18, 20263 days ago
Recorded sightings
4
Last seen by us
Nov 21, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 3

Early. Similar jobs usually come down between day 6 and day 56.

From 14,766 similar postings that came down. Coming down is not always a hire.

Lutra is hiring

3 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Java
  • Scala
  • SQL
  • Hadoop
  • HDFS
  • Kafka
  • Kubernetes
  • Spark
  • Superset
  • Trino
  • Vertica
  • Airflow
  • Go
Where the posting names them
- Connect APIs to batch, streaming, and near real-time data sources - Build primarily in Go or a comparable backend language such as Java - Improve observability, incident response, reliability, and consumer experience
- Processing and streaming: Spark, Scala, Kafka, Flink, Airflow - Data services and APIs: Go, Java, REST, gRPC, SQL - Compute and storage: Kubernetes, Hadoop/HDFS, Ceph
Tech stack - Processing and streaming: Spark, Scala, Kafka, Flink, Airflow - Data services and APIs: Go, Java, REST, gRPC, SQL
- Help decompose centralized processing into domain-oriented streams and workloads - Move appropriate services and processing patterns from Hadoop toward Kubernetes - Improve data quality, correctness, observability, throughput, and compute efficiency
- Data services and APIs: Go, Java, REST, gRPC, SQL - Compute and storage: Kubernetes, Hadoop/HDFS, Ceph - Query and analytics: Trino, Vertica, Druid and StarRocks
Data processing and platform: This team builds and evolves large-scale batch and streaming pipelines that transform raw event data into clean, canonical, business-ready datasets; improving data models, workflow orchestration, correctness, observability, runtime, and compute efficiency. On this team you can expect to: - Build large-scale Spark pipelines and streaming systems using Kafka and Flink - Transform raw event data into canonical, production-grade datasets
- You have experience with large-scale data processing and modeling (the client leans on Spark currently) - You have experience with streaming technologies such as Kafka and Flink, including incremental processing, late data, deduplication, and replay - You have experience designing stable data contracts, schemas, canonical datasets, and transformation layers for business-critical use cases
- You have experience with large-scale storage, data warehouse, or analytical query systems and a strong grasp of query performance, data layout, and workload-aware optimization - You have experience with workflow orchestration and the production deployment or operation of services on Kubernetes - You have a track record of end-to-end ownership: navigating ambiguity, debugging complex cross-system issues, making sound trade-offs, and improving systems after they enter production
- You have a deep understanding of distributed-systems fundamentals, including partitioning, consistency, failure modes, state, throughput, latency, and cost trade-offs - You have experience with large-scale data processing and modeling (the client leans on Spark currently) - You have experience with streaming technologies such as Kafka and Flink, including incremental processing, late data, deduplication, and replay
- Query and analytics: Trino, Vertica, Druid and StarRocks - Reporting and BI: Looker, Superset, Redash The precise mix will vary by team and assignment. Depth in comparable technologies and the ability to reason from first principles matter more than matching every tool.
- Compute and storage: Kubernetes, Hadoop/HDFS, Ceph - Query and analytics: Trino, Vertica, Druid and StarRocks - Reporting and BI: Looker, Superset, Redash

Your next step

Complete your application on jobs.gem.com. The employer’s form will show what is required.

Already applied? Track this application

Job description

View original posting ↗

The search

Our client is seeking multiple technical leaders to contribute at both the Staff and Lead Engineer level. These are high-visibility, high-impact opportunities with a talent-dense, homegrown, bootstrapped, globally competitive Canadian tech success story. The company finds itself squarely in the middle of the disruption of search by AI/LLMs, the fragmentation of the media landscape and in a chapter of rapid, transformational growth.

The shared mandate is to modernize how massive volumes of data are processed, modeled, and made available to products and customers. The work spans petabyte-scale batch and streaming pipelines, low-latency APIs, analytical platforms, and the architecture that connects raw event data to reliable, production-grade data products.

Staff Engineer candidates may bring a stronger centre of gravity in backend and API systems or in data processing and platform engineering. For the Lead Engineer search, the client is seeking a hands-on technical and people leader with the affinity and aptitude to guide a distributed team through transformation.

The client

Our client builds a high throughput system powering hundreds of billions of daily transactions, each completed within milliseconds, across globally distributed infrastructure designed for reliability and efficiency. They have been quietly bootstrapping and growing in line with revenue for over 2 decades. They operate at a planetary scale with an exchange platform with thousands of nodes that handles nearly 500+ billion daily auctions and is trending towards 1 trillion daily auctions in the next ~4 years. To the present day, the company has not raised money from venture capitalists. It maintains its independence from external stakeholders, enabling it to chart its course, maintain a long-term perspective and build an enduring, sustainable business that currently employs ~600 team members globally.

Data is central to nearly every part of its platform. High-volume event and transactional data powers customer reporting, billing, marketplace analytics, experimentation, machine-learning workflows, and product experiences. The organization is evolving from centralized, batch-oriented reporting toward a platform-driven architecture combining batch, streaming, and asynchronous processing, with APIs becoming a primary interface to data.

This is not conventional business-intelligence work. The challenge is to process and serve enormous datasets with strong correctness guarantees, predictable latency, and disciplined infrastructure economics. Engineers work across the full path from edge and transport through compute, storage, query execution, and serving, balancing latency, cost, complexity, and reliability at every layer.

Our client has been stubbornly racking and stacking infrastructure around the world for the duration of their existence, a habit that allows for them to price themselves at the cost of electricity while their competitors are mired in rising cloud infrastructure costs. This long-term, somewhat contrarian, thinking puts them in a position to offer stability and career longevity evidenced by robust benefits that include RRSP matching.

If you’re looking for a role that offers an opportunity to innovate and optimize systems at a massive scale, creating a lasting impact in an environment that values technical excellence and resilience, this could be for you.

Role overviews

In our client’s context, both Staff Engineers and Leads remain hands-on. This is an intentional and informed cultural practice tailored to the challenge ahead of them. Leads are expected to understand the systems deeply, make informed architectural decisions, and help their teams execute.

As a Staff Engineer, you will be a primary owner and technical leader for data-intensive systems spanning processing, platform, and product-facing services. You will drive cross-team architecture, define reusable platform patterns, lead complex multi-system initiatives, and influence the long-term evolution of data processing and access across the organization. Depending on your background, your work may lean toward one or both of the following domains:

Data products and APIs: This team builds and operates high-scale services exposing aggregated and near real-time data for reporting, analytics, and customer-facing products; shaping API contracts, query abstraction and execution, caching, response design, performance, scalability, and reliability. This team operates as a full-stack data engineering group, owning the path from data generation and aggregation through query execution, APIs, and user-facing delivery. On this team you can expect to:

  • Design high-scale APIs for multidimensional reporting and analytical workloads
  • Define contracts for metrics, dimensions, filters, schemas, and aggregations
  • Improve dynamic query generation, caching, response shaping, and execution performance
  • Connect APIs to batch, streaming, and near real-time data sources
  • Build primarily in Go or a comparable backend language such as Java
  • Improve observability, incident response, reliability, and consumer experience
  • Lead initiatives involving Product, Data Platform, ML, and Application Engineering

Data processing and platform: This team builds and evolves large-scale batch and streaming pipelines that transform raw event data into clean, canonical, business-ready datasets; improving data models, workflow orchestration, correctness, observability, runtime, and compute efficiency. On this team you can expect to:

  • Build large-scale Spark pipelines and streaming systems using Kafka and Flink
  • Transform raw event data into canonical, production-grade datasets
  • Design partitioning, storage, aggregation, and query strategies for enormous datasets
  • Support historical backfills, incremental processing, and real-time availability
  • Help decompose centralized processing into domain-oriented streams and workloads
  • Move appropriate services and processing patterns from Hadoop toward Kubernetes
  • Improve data quality, correctness, observability, throughput, and compute efficiency
  • Participate in the long-term design of globally distributed data infrastructure

As a Lead Engineer you will lead a distributed team of data and software engineers responsible for high-scale, low-latency data processing pipelines, analytical platforms, and reporting products. In practice this means:

  • Set the technical direction for the team’s products, systems, and architectures while remaining comfortable working directly with the code and technology
  • Hire, mentor, and develop engineers, creating an environment of accountability, collaboration, and meaningful technical growth
  • Partner with Product to establish roadmaps, goals, forecasts, and measurable technical and business outcomes
  • Manage scope and sequencing across more opportunities than available resources, communicating progress and risks early and coordinating dependencies across teams
  • Own the quality, health, and outcomes of the team’s platforms, including operational response, technical debt, reliability, and continuous improvement

The shared mandate

  • Platform architecture: Define and evolve patterns across edge, transport, compute, storage, modeling, query execution, and serving
  • Batch and streaming systems: Build systems supporting historical processing, backfills, incremental updates, and near real-time availability without creating unnecessary architectural complexity
  • Data products and interfaces: Provide stable, flexible access to metrics, dimensions, filters, aggregations, and canonical datasets through well-designed APIs and analytical platforms
  • Performance and economics: Optimize query planning, data layout, partitioning, caching, execution, throughput, and infrastructure use so capacity does not need to grow linearly with data volume
  • Correctness and trust: Design for deduplication, late-arriving data, schema evolution, data contracts, reconciliation, and the integrity of business-critical reporting and billing datasets
  • Reliability and observability: Own monitoring, alerting, incident response, root-cause analysis, workflow health, and the mechanisms that make system behaviour visible
  • Cross-functional influence: Work across Data Engineering, Data Systems, Application Engineering, Product, analytics, machine learning, and experimentation teams to turn platform capabilities into business outcomes

Tech stack

  • Processing and streaming: Spark, Scala, Kafka, Flink, Airflow
  • Data services and APIs: Go, Java, REST, gRPC, SQL
  • Compute and storage: Kubernetes, Hadoop/HDFS, Ceph
  • Query and analytics: Trino, Vertica, Druid and StarRocks
  • Reporting and BI: Looker, Superset, Redash

The precise mix will vary by team and assignment. Depth in comparable technologies and the ability to reason from first principles matter more than matching every tool.

Key responsibilities

  • Vision and strategic direction: Identify high-leverage technical opportunities and translate them into a coherent direction for the platform and business
  • System design and delivery: Lead the design and implementation of distributed data systems operating under heavy workloads and demanding latency requirements
  • Data platform evolution: Help move the organization from centralized, batch-oriented systems toward workload-aware architectures combining batch, streaming, and asynchronous processing
  • API and query architecture: Define durable interfaces, schemas, query abstractions, and serving patterns for flexible access to large-scale multidimensional data
  • Pipeline and model ownership: Build canonical datasets and processing workflows supporting reporting, billing, products, analytics, experimentation, and machine-learning use cases
  • Operational excellence: Improve observability, service health, incident response, workflow reliability, technical debt management, and post-incident learning
  • Organizational leadership: Coordinate across teams, unblock dependencies, communicate trade-offs clearly, and build alignment around technical and product outcomes

Your know-how

  • You have significant experience designing, building, and operating data-intensive systems at scale
  • You have a deep understanding of distributed-systems fundamentals, including partitioning, consistency, failure modes, state, throughput, latency, and cost trade-offs
  • You have experience with large-scale data processing and modeling (the client leans on Spark currently)
  • You have experience with streaming technologies such as Kafka and Flink, including incremental processing, late data, deduplication, and replay
  • You have experience designing stable data contracts, schemas, canonical datasets, and transformation layers for business-critical use cases
  • You have experience with large-scale storage, data warehouse, or analytical query systems and a strong grasp of query performance, data layout, and workload-aware optimization
  • You have experience with workflow orchestration and the production deployment or operation of services on Kubernetes
  • You have a track record of end-to-end ownership: navigating ambiguity, debugging complex cross-system issues, making sound trade-offs, and improving systems after they enter production
  • You have an excellent command of English and experience collaborating with neighbouring engineering disciplines and stakeholders beyond R&D

For API-oriented work, you will call backend engineering skills (ideally but not necessarily with Go) and experience with REST/gRPC design, versioning, performance, and high-concurrency services.

Additional know-how for the Lead Engineer roles

  • You have 3+ years of experience leading and managing distributed teams
  • You have an affinity for mentorship and creating an environment that nurtures a balance of innovation and accountability
  • You have experience partnering with Product Management and internal stakeholders to shape roadmaps, forecasts, deliverables, timelines, and measurable outcomes
  • You are experienced making tradeoffs and experience exercising judgment to prioritize a broad portfolio when initiatives outnumber resources, including thoughtful sequencing and proactive communication of progress, delays, and risk

Interested in learning more? 

If you are interested in building data systems at a scale few Canadian companies can offer, while helping shape a consequential platform transformation, we would love to hear from you. Please send your resume or LinkedIn profile to talent@lutrapartners.com with “Data Engineering” as the subject or apply using the following link. One of our partners will be in touch shortly!

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 845, the most 1,344, and the latest week had 1,007.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Montréal, Québec

Working pattern and location restrictions need checking in the full posting.

Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.