Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
SQL
Azure
Databricks
Delta Lake
PySpark
Where the posting names them
Job Summary: Key Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF). Build and optimize complex data transformation logic using Azure Databricks (PySpark and SQL). Lead and execute on-premises to Azure cloud migrations for data platforms, ETL workflows, and applications. Develop, deploy, and support applications using Azure App Service and Azure Functions. Design data models and perform query optimization and performance tuning using Azure SQL. Implement and manage data storage solutions using Azure Data Lake Storage Gen2 (ADLS Gen2) and Delta Lake. Apply robust data orchestration, scheduling, and error-handling patterns to ensure reliable data processing. Leverage core Azure services such as VNets, Storage Accounts, Key Vault, and IAM to build secure and scalable cloud solutions. Monitor, troubleshoot, and optimize data pipelines and Azure resources using appropriate monitoring tools. Required Skills & Qualifications Strong hands-on experience with Azure Data Factory (ADF). Proven expertise in Azure Databricks, including PySpark and SQL-based data transformations. Hands-on experience in developing and deploying solutions using Azure App Service and Azure Functions. Solid experience in on-premises-to-Azure migration projects, particularly for data platforms. Advanced SQL skills, including data modeling, query optimization, and performance tuning in Azure SQL. Working knowledge of Delta Lake, ADLS Gen2, and modern data lake architectures. Good understanding of core Azure services and cloud security best practices. Experience with monitoring and troubleshooting using Azure Monitor and Application Insights.
Job Summary: Key Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF). Build and optimize complex data transformation logic using Azure Databricks (PySpark and SQL). Lead and execute on-premises to Azure cloud migrations for data platforms, ETL workflows, and applications. Develop, deploy, and support applications using Azure App Service and Azure Functions. Design data models and perform query optimization and performance tuning using Azure SQL. Implement and manage data storage solutions using Azure Data Lake Storage Gen2 (ADLS Gen2) and Delta Lake. Apply robust data orchestration, scheduling, and error-handling patterns to ensure reliable data processing. Leverage core Azure services such as VNets, Storage Accounts, Key Vault, and IAM to build secure and scalable cloud solutions. Monitor, troubleshoot, and optimize data pipelines and Azure resources using appropriate monitoring tools. Required Skills & Qualifications Strong hands-on experience with Azure Data Factory (ADF). Proven expertise in Azure Databricks, including PySpark and SQL-based data transformations. Hands-on experience in developing and deploying solutions using Azure App Service and Azure Functions. Solid experience in on-premises-to-Azure migration projects, particularly for data platforms. Advanced SQL skills, including data modeling, query optimization, and performance tuning in Azure SQL. Working knowledge of Delta Lake, ADLS Gen2, and modern data lake architectures. Good understanding of core Azure services and cloud security best practices. Experience with monitoring and troubleshooting using Azure Monitor and Application Insights.
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.