Back to jobs

Senior Lead Data Engineer - Databricks

Pay
Pay not listed
Work setting
On-site — work setting source
Work setup in the employer’s posting: On-site
From the employer’s posting
Employment
Not stated

What you’ll work on

Full posting
  • Lead design and build of scalable data engineering solutions on Azure and Databricks

  • Build and optimize batch and streaming data pipelines

  • Design and optimize Delta Lake solutions

  • Gather client requirements and turn them into technical solutions

  • Mentor and coach junior data engineers

From the employer’s posting
Lead the design, development, and maintenance of scalable data engineering solutions using Azure and Databricks.
Build and optimize large-scale batch and streaming data pipelines.
Design, implement, and optimize Delta Lake solutions for reliable and performant data processing.
Collaborate with clients to gather requirements and translate business needs into technical solutions.
Mentor and coach junior data engineers while promoting engineering best practices.

What you’ll bring

All qualifications

Core experience

  • 10+ years of professional data engineering experience
  • 4+ years of hands-on Databricks experience
  • Strong PySpark and Apache Spark pipeline development
  • Lakehouse architecture and Delta Lake / Medallion experience
  • Strong SQL and Python skills
Qualification wording
Required 10+ years of professional experience in Data Engineering.
4+ years of hands-on experience with Databricks.
Strong experience developing data pipelines using PySpark and Apache Spark.
Strong experience designing and implementing Lakehouse architectures.
Strong knowledge of SQL and Python.
Apply at Integrant

Posting record

Report an error

Actively hiringFirst seen by us 3 days ago, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 18, 20263 days ago
Recorded sightings
4
Last seen by us
Nov 21, 2026
Employer says posted
Jul 27, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 3

Early. Similar jobs usually come down between day 6 and day 56.

From 14,771 similar postings that came down. Coming down is not always a hire.

Integrant is hiring

6 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
  • Azure
  • Databricks
  • dbt
  • Delta Lake
  • Docker
  • Kafka
  • Kubernetes
  • MLflow
  • Power BI
  • Snowflake
  • Spark
  • Airflow
  • PySpark
Where the posting names them
- Experience working with Delta Lake and Medallion Architecture. - Strong knowledge of SQL and Python. - Experience with Azure Data Factory.
- SQL - Python - Delta Lake
- Build and optimize large-scale batch and streaming data pipelines. - Develop ETL/ELT pipelines using Databricks, PySpark, SQL, and Azure Data Factory. - Design, implement, and optimize Delta Lake solutions for reliable and performant data processing.
- Experience with Unity Catalog. - Experience with Databricks SQL. - Experience with MLflow.
- PySpark - SQL - Python
Description We're on the lookout for exceptional individuals to join our team as Senior Lead Data Engineers – Databricks. As part of our team, you'll play a key role in designing and delivering modern, scalable data platforms for clients across multiple industries, leveraging Azure and Databricks technologies. You will lead the design, development, and optimization of end-to-end data engineering solutions built on modern Lakehouse architectures. Working closely with clients and cross-functional teams, you'll translate business requirements into scalable, high-performance data platforms while mentoring engineers and driving technical excellence.
- Responsibilities - Lead the design, development, and maintenance of scalable data engineering solutions using Azure and Databricks. - Design and implement modern Lakehouse architectures following industry best practices.
- Design and maintain enterprise-grade Data Warehouses, Data Lakes, and Lakehouses. - Implement data governance, security, and metadata management using Azure services such as Purview, Key Vault, and Microsoft Entra ID (Azure Active Directory). - Establish CI/CD pipelines and DevOps practices for data engineering workloads.
- Strong knowledge of SQL and Python. - Experience with Azure Data Factory. - Experience working with Azure Data Lake Storage Gen2 (ADLS Gen2).
- Experience with Azure Data Factory. - Experience working with Azure Data Lake Storage Gen2 (ADLS Gen2). - Experience with Azure Synapse Analytics.
- Experience working with Azure Data Lake Storage Gen2 (ADLS Gen2). - Experience with Azure Synapse Analytics. - Experience with Data Warehousing concepts and dimensional modeling.
- Experience optimizing Spark jobs, partitioning strategies, joins, caching, and workload performance. - Experience with Git and Azure DevOps CI/CD pipelines. - Experience implementing secure data platforms using Azure Key Vault, Microsoft Entra ID, and Microsoft Purview.
- Experience with Git and Azure DevOps CI/CD pipelines. - Experience implementing secure data platforms using Azure Key Vault, Microsoft Entra ID, and Microsoft Purview. - Experience building scalable enterprise data platforms in Azure.
- Experience implementing secure data platforms using Azure Key Vault, Microsoft Entra ID, and Microsoft Purview. - Experience building scalable enterprise data platforms in Azure. - Experience working directly with clients and gathering technical requirements.
- Experience with MLOps. - Experience with Azure Machine Learning. - Experience with Azure Cognitive Services.
- Experience with Azure Machine Learning. - Experience with Azure Cognitive Services. - Experience with Docker and Kubernetes.
- Core Technologies - Azure Databricks - Apache Spark
- Delta Lake - Azure Data Factory - Azure Synapse Analytics
- Azure Data Factory - Azure Synapse Analytics - ADLS Gen2
- Performance Optimization - Azure Ecosystem - Azure DevOps
- Azure Ecosystem - Azure DevOps - Microsoft Purview
- Microsoft Purview - Azure Key Vault - Microsoft Entra ID
- Microsoft Entra ID - Azure Functions - Event Hubs
- 10+ years of professional experience in Data Engineering. - 4+ years of hands-on experience with Databricks. - Strong experience developing data pipelines using PySpark and Apache Spark.
- Experience building real-time streaming solutions using Structured Streaming, Kafka, or Event Hubs. - Experience with Databricks Workflows. - Experience with Delta Live Tables.
- Experience with MLflow. - Experience with dbt. - Experience with Airflow or other orchestration platforms.
- Airflow - dbt - MLflow
- Develop ETL/ELT pipelines using Databricks, PySpark, SQL, and Azure Data Factory. - Design, implement, and optimize Delta Lake solutions for reliable and performant data processing. - Collaborate with clients to gather requirements and translate business needs into technical solutions.
- Strong experience designing and implementing Lakehouse architectures. - Experience working with Delta Lake and Medallion Architecture. - Strong knowledge of SQL and Python.
- Experience with Databricks Workflows. - Experience with Delta Live Tables. - Experience with Unity Catalog.
- Python - Delta Lake - Azure Data Factory
- Experience with Azure Cognitive Services. - Experience with Docker and Kubernetes. - Experience working with Microsoft Fabric.
- MLflow - Docker - Kubernetes
- Experience with Snowflake. - Experience building real-time streaming solutions using Structured Streaming, Kafka, or Event Hubs. - Experience with Databricks Workflows.
- Nice-to-Have Technologies - Kafka - Airflow
- Docker - Kubernetes - Power BI
- Experience with Databricks SQL. - Experience with MLflow. - Experience with dbt.
- dbt - MLflow - Docker
- Experience working with Microsoft Fabric. - Experience with Power BI. -
- Kubernetes - Power BI Benefits
- Strongly Preferred - Experience with Snowflake. - Experience building real-time streaming solutions using Structured Streaming, Kafka, or Event Hubs.
- ADLS Gen2 - Snowflake - Data Engineering
- 4+ years of hands-on experience with Databricks. - Strong experience developing data pipelines using PySpark and Apache Spark. - Strong experience designing and implementing Lakehouse architectures.
- Strong understanding of ETL and ELT design patterns. - Experience optimizing Spark jobs, partitioning strategies, joins, caching, and workload performance. - Experience with Git and Azure DevOps CI/CD pipelines.
- Azure Databricks - Apache Spark - PySpark
- Experience with dbt. - Experience with Airflow or other orchestration platforms. - Experience working in Agile environments.
- Kafka - Airflow - dbt
- Apache Spark - PySpark - SQL

Your next step

Complete your application on apply.workable.com. The employer’s form will show what is required.

Applying through Workable

  • No account needed. You can import your resume to fill in the form, or skip that and fill it in yourself.

Already applied? Track this application

Job description

View original posting ↗

Description

We're on the lookout for exceptional individuals to join our team as Senior Lead Data Engineers – Databricks. As part of our team, you'll play a key role in designing and delivering modern, scalable data platforms for clients across multiple industries, leveraging Azure and Databricks technologies.

You will lead the design, development, and optimization of end-to-end data engineering solutions built on modern Lakehouse architectures. Working closely with clients and cross-functional teams, you'll translate business requirements into scalable, high-performance data platforms while mentoring engineers and driving technical excellence.

Requirements

  • Responsibilities
    • Lead the design, development, and maintenance of scalable data engineering solutions using Azure and Databricks.
    • Design and implement modern Lakehouse architectures following industry best practices.
    • Build and optimize large-scale batch and streaming data pipelines.
    • Develop ETL/ELT pipelines using Databricks, PySpark, SQL, and Azure Data Factory.
    • Design, implement, and optimize Delta Lake solutions for reliable and performant data processing.
    • Collaborate with clients to gather requirements and translate business needs into technical solutions.
    • Optimize Spark workloads for performance, scalability, and cost efficiency.
    • Design and maintain enterprise-grade Data Warehouses, Data Lakes, and Lakehouses.
    • Implement data governance, security, and metadata management using Azure services such as Purview, Key Vault, and Microsoft Entra ID (Azure Active Directory).
    • Establish CI/CD pipelines and DevOps practices for data engineering workloads.
    • Mentor and coach junior data engineers while promoting engineering best practices.
    • Participate in architecture discussions, solution design, and technical leadership initiatives.
    • Contribute to internal knowledge sharing and continuous improvement.
  • Requirements
  • Required
    • 10+ years of professional experience in Data Engineering.
    • 4+ years of hands-on experience with Databricks.
    • Strong experience developing data pipelines using PySpark and Apache Spark.
    • Strong experience designing and implementing Lakehouse architectures.
    • Experience working with Delta Lake and Medallion Architecture.
    • Strong knowledge of SQL and Python.
    • Experience with Azure Data Factory.
    • Experience working with Azure Data Lake Storage Gen2 (ADLS Gen2).
    • Experience with Azure Synapse Analytics.
    • Experience with Data Warehousing concepts and dimensional modeling.
    • Strong understanding of ETL and ELT design patterns.
    • Experience optimizing Spark jobs, partitioning strategies, joins, caching, and workload performance.
    • Experience with Git and Azure DevOps CI/CD pipelines.
    • Experience implementing secure data platforms using Azure Key Vault, Microsoft Entra ID, and Microsoft Purview.
    • Experience building scalable enterprise data platforms in Azure.
    • Experience working directly with clients and gathering technical requirements.
    • Experience mentoring engineers and leading technical initiatives.
  • Strongly Preferred
    • Experience with Snowflake.
    • Experience building real-time streaming solutions using Structured Streaming, Kafka, or Event Hubs.
    • Experience with Databricks Workflows.
    • Experience with Delta Live Tables.
    • Experience with Unity Catalog.
    • Experience with Databricks SQL.
    • Experience with MLflow.
    • Experience with dbt.
    • Experience with Airflow or other orchestration platforms.
    • Experience working in Agile environments.
  • Nice to Have
    • MSc in Computer Science or a related field.
    • Experience with DataOps practices.
    • Experience with MLOps.
    • Experience with Azure Machine Learning.
    • Experience with Azure Cognitive Services.
    • Experience with Docker and Kubernetes.
    • Experience working with Microsoft Fabric.
    • Experience with Power BI.
  • Technical Skills
  • Core Technologies
    • Azure Databricks
    • Apache Spark
    • PySpark
    • SQL
    • Python
    • Delta Lake
    • Azure Data Factory
    • Azure Synapse Analytics
    • ADLS Gen2
    • Snowflake
  • Data Engineering
    • Data Warehousing
    • Data Lakes
    • Lakehouse Architecture
    • Medallion Architecture
    • Dimensional Modeling
    • ETL / ELT
    • Batch Processing
    • Streaming Data Pipelines
    • Data Governance
    • Performance Optimization
  • Azure Ecosystem
    • Azure DevOps
    • Microsoft Purview
    • Azure Key Vault
    • Microsoft Entra ID
    • Azure Functions
    • Event Hubs
  • Nice-to-Have Technologies
    • Kafka
    • Airflow
    • dbt
    • MLflow
    • Docker
    • Kubernetes
    • Power BI

Benefits

  • Salary paid in USD
  • Six-month career advancing opportunities
  • Supportive and friendly work environment
  • Premium medical insurance [employee +family]
  • English language development courses
  • Interest-free loans paid over 2.5 years
  • Technical development courses
  • Planned overtime program (POP)
  • Employment referral program
  • Premium location in Maadi
  • Social insurance

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 845, the most 1,344, and the latest week had 1,007.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Maadi, Al Qāhirah, Egypt

Working pattern and location restrictions need checking in the full posting.

Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.