Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
Python
SQL
Azure
BigQuery
Databricks
Delta Lake
Kafka
Redshift
Snowflake
Spark
Airflow
PySpark
Where the posting names them
- Strong hands-on experience (4+ years) in data engineering or building data processing platforms
- Strong SQL skills and solid experience with Python for building and maintaining data pipelines
- Experience working with distributed data processing frameworks such as Apache Spark/pySpark (Databricks experience is a strong advantage)
- Design and implement data processing workflows using distributed processing frameworks such as Apache Spark/pySpark (Databricks)
- Contribute to the evolution of the organization’s cloud data platform using Azure technologies (e.g., Azure Data Factory, Synapse, Microsoft Fabrik, Databricks, Event Hub, Azure Data Lake Storage)
- Strong background in data modelling to support analytics, reporting, and downstream data consumption
- Experience working with distributed data processing frameworks such as Apache Spark/pySpark (Databricks experience is a strong advantage)
- Practical experience designing and implementing data pipelines in cloud environments, preferably within the Azure ecosystem
- Experience designing data pipelines and data models for production-scale analytical systems
Nice to Have
- Experience with modern cloud data warehouses such as Snowflake, BigQuery, or Redshift
- Experience with streaming data technologies (e.g., Event Hub, Kafka, Spark Streaming)
- Design and develop scalable and reliable data platforms and/or data pipelines for analytical and operational use cases
- Design and implement data processing workflows using distributed processing frameworks such as Apache Spark/pySpark (Databricks)
- Contribute to the evolution of the organization’s cloud data platform using Azure technologies (e.g., Azure Data Factory, Synapse, Microsoft Fabrik, Databricks, Event Hub, Azure Data Lake Storage)
- Strong SQL skills and solid experience with Python for building and maintaining data pipelines
- Experience working with distributed data processing frameworks such as Apache Spark/pySpark (Databricks experience is a strong advantage)
- Practical experience designing and implementing data pipelines in cloud environments, preferably within the Azure ecosystem
- Experience with streaming data technologies (e.g., Event Hub, Kafka, Spark Streaming)
- Experience with Delta Lake or other lakehouse technologies
- Familiarity with CI/CD practices and Git-based development workflows
- Experience with modern cloud data warehouses such as Snowflake, BigQuery, or Redshift
- Experience with streaming data technologies (e.g., Event Hub, Kafka, Spark Streaming)
- Experience with Delta Lake or other lakehouse technologies
- Experience working with modern data lake architectures and data formats such as Parquet and JSON
- Experience with workflow orchestration tools (e.g., Airflow)
- Experience with database development and performance optimization
Design and develop scalable and reliable data platforms and/or data pipelines for analytical and operational use cases
Design and implement data processing workflows using distributed processing frameworks such as Apache Spark/pySpark (Databricks)
Contribute to the evolution of the organization’s cloud data platform using Azure technologies (e.g., Azure Data Factory, Synapse, Microsoft Fabrik, Databricks, Event Hub, Azure Data Lake Storage)
Strong background in data modelling to support analytics, reporting, and downstream data consumption
Integrate and process data from multiple internal and external sources while ensuring data consistency, quality, and reliability
Take ownership of data pipeline reliability, monitoring, and performance, ensuring scalable and maintainable data processing workflows
Collaborate with data analysts, data scientists, and engineering teams to translate business and analytical requirements into robust data solutions
Drive improvements in data engineering practices, including pipeline design standards, testing approaches, and operational reliability
Participate in evaluating and introducing new tools and technologies within the data engineering ecosystem
Qualifications
Required Qualifications
Strong hands-on experience (4+ years) in data engineering or building data processing platforms
Strong SQL skills and solid experience with Python for building and maintaining data pipelines
Experience working with distributed data processing frameworks such as Apache Spark/pySpark (Databricks experience is a strong advantage)
Practical experience designing and implementing data pipelines in cloud environments, preferably within the Azure ecosystem
Experience designing data pipelines and data models for production-scale analytical systems
Strong understanding of data engineering concepts including ETL/ELT design, dimensional modeling, and data warehousing principles
Experience working with modern data lake architectures and data formats such as Parquet and JSON
Experience with workflow orchestration tools (e.g., Airflow)
Experience with database development and performance optimization
Ability to design scalable and maintainable data solutions with minimal supervision
Strong collaboration and communication skills
Professional working proficiency in English
Nice to Have
Experience with modern cloud data warehouses such as Snowflake, BigQuery, or Redshift
Experience with streaming data technologies (e.g., Event Hub, Kafka, Spark Streaming)
Experience with Delta Lake or other lakehouse technologies
Familiarity with CI/CD practices and Git-based development workflows
Experience working in agile development environments
Additional Information
Company Description
We are looking for a Data Engineer to help design and build scalable data platforms that support analytics, reporting, and data-driven decision-making across the organization. In this role, you will develop robust data pipelines, contribute to the evolution of our cloud data platform, and ensure reliable and efficient data processing across multiple projects and teams.
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.