Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
AWS
BigQuery
Databricks
Delta Lake
Elasticsearch
GCP
Kafka
S3
Spark
PySpark
Where the posting names them
Data Engineer – Cloud Big Data Role Summary: Responsible for designing, developing, and maintaining scalable cloud-based data pipelines and data platforms across AWS, GCP, Kafka, and Databricks. The role involves building reliable ETL/ELT pipelines, real-time data ingestion, data transformation, monitoring, and performance/cost optimization. In this role you will... Design and develop ETL/ELT data pipelines using AWS Glue, Databricks, and Spark. Build and maintain batch and real-time data pipelines using Apache Kafka / Confluent Kafka. Develop CDC-based data ingestion and event-driven data processing. Work with AWS services including Glue, S3, Lambda, IAM, KMS, Secrets Manager, and related services. Develop and optimize data solutions on GCP, including BigQuery, Cloud Storage, Cloud Functions, and related services. Implement data processing using Databricks, PySpark, Delta Lake, Bronze/Silver/Gold architecture, and DLT. Manage and optimize Elasticsearch/OpenSearch data ingestion, indexing, queries, and cluster performance. Implement data quality, validation, monitoring, logging, and alerting across data pipelines. Troubleshoot production issues involving Kafka, schema compatibility, CDC, Spark, Glue, Databricks, and cloud infrastructure. Optimize pipeline performance and cloud infrastructure costs. Implement secure data solutions using IAM, encryption, network controls, secrets management, and least-privilege access. Collaborate with application, DevOps, QA, analytics, and product teams to deliver reliable data solutions. You've got what it takes if you have… 5+ years of experience in Data Engineering / Big Data Engineering. Strong hands-on experience with AWS + Databricks + Kafka. Experience working in multi-cloud environments (AWS/GCP). Strong understanding of distributed systems, streaming, CDC, schema evolution, and data governance. Experience troubleshooting high-volume production data pipelines. Experience with cloud performance and cost optimization. #LI-Onsite
Data Engineer – Cloud Big Data Role Summary: Responsible for designing, developing, and maintaining scalable cloud-based data pipelines and data platforms across AWS, GCP, Kafka, and Databricks. The role involves building reliable ETL/ELT pipelines, real-time data ingestion, data transformation, monitoring, and performance/cost optimization. In this role you will... Design and develop ETL/ELT data pipelines using AWS Glue, Databricks, and Spark. Build and maintain batch and real-time data pipelines using Apache Kafka / Confluent Kafka. Develop CDC-based data ingestion and event-driven data processing. Work with AWS services including Glue, S3, Lambda, IAM, KMS, Secrets Manager, and related services. Develop and optimize data solutions on GCP, including BigQuery, Cloud Storage, Cloud Functions, and related services. Implement data processing using Databricks, PySpark, Delta Lake, Bronze/Silver/Gold architecture, and DLT. Manage and optimize Elasticsearch/OpenSearch data ingestion, indexing, queries, and cluster performance. Implement data quality, validation, monitoring, logging, and alerting across data pipelines. Troubleshoot production issues involving Kafka, schema compatibility, CDC, Spark, Glue, Databricks, and cloud infrastructure. Optimize pipeline performance and cloud infrastructure costs. Implement secure data solutions using IAM, encryption, network controls, secrets management, and least-privilege access. Collaborate with application, DevOps, QA, analytics, and product teams to deliver reliable data solutions. You've got what it takes if you have… 5+ years of experience in Data Engineering / Big Data Engineering. Strong hands-on experience with AWS + Databricks + Kafka. Experience working in multi-cloud environments (AWS/GCP). Strong understanding of distributed systems, streaming, CDC, schema evolution, and data governance. Experience troubleshooting high-volume production data pipelines. Experience with cloud performance and cost optimization. #LI-Onsite
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.