Design, build, test and maintain scalable data pipelines and database architectures
Design and optimize ETL/ELT processes into Amazon Redshift and other data platforms
Build data validation, quality and monitoring processes
Automate manual processes and improve data delivery
Enable pipelines that give analytics insights for marketing, campaigns and operations
From the employer’s posting
Design, develop, test, and maintain scalable data pipelines and database architectures.
Design and optimize ETL/ELT processes for extracting, transforming, and loading data from diverse sources into Amazon Redshift and other data platforms.
Develop and implement data validation, quality, and monitoring processes to ensure data accuracy, consistency, and reliability.
Identify and implement process improvements, including automation of manual processes and optimization of data delivery.
Support analytics initiatives by enabling data pipelines and solutions that provide actionable insights for marketing, campaigns, events, promotions, operational efficiency, and other key business performance metrics.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
C++
Java
Python
Scala
SQL
AWS
Hadoop
Kafka
NoSQL
PostgreSQL
Redshift
Spark
GCP
Where the posting names them
POSITION SUMMARY The Data Engineer is responsible for designing, building, expanding, and optimizing data pipelines and data architecture to support cross-functional applications and business initiatives. The role focuses on developing reliable, scalable, and secure data solutions that enable efficient data collection, processing, integration, and delivery. The Data Engineer will collaborate with Digital Developers, Data Architects, Solution Architects, and other stakeholders to support data initiatives and ensure consistent data delivery architecture across projects. The role requires a self-directed professional who can support the data needs of multiple teams, systems, and products while contributing to the development of future-generation data architecture and solutions. DUTIES AND RESPONSIBILITIES Collect, integrate, and structure datasets based on business and functional requirements. Design, develop, test, and maintain scalable data pipelines and database architectures. Build data transformation processes and algorithms that convert raw data into reliable, actionable information. Design and optimize ETL/ELT processes for extracting, transforming, and loading data from diverse sources into Amazon Redshift and other data platforms. Identify and implement process improvements, including automation of manual processes and optimization of data delivery. Develop and implement data validation, quality, and monitoring processes to ensure data accuracy, consistency, and reliability. Support analytics initiatives by enabling data pipelines and solutions that provide actionable insights for marketing, campaigns, events, promotions, operational efficiency, and other key business performance metrics. Collaborate with cross-functional teams to understand business objectives and translate requirements into effective technical solutions. Support the design and implementation of scalable, resilient, redundant, and secure data infrastructure across multiple data centers and AWS regions. Ensure data is appropriately protected throughout its lifecycle and remains compliant with applicable data governance, privacy, and security policies. Provide technical support and expertise for data-related initiatives across multiple teams, systems, and products. Requirements QUALIFICATIONS & REQUIREMENTS Bachelor’s degree in Computer Science, Information Technology, Information Management, Engineering, or related field, with at least 5 years of relevant experience in Data Engineering or a related discipline. Strong experience in designing, developing, and optimizing data pipelines, data architecture, ETL/ELT processes, and data integration solutions. Proficiency in Python, Java, Scala, or C++, and experience with SQL and NoSQL databases, including PostgreSQL and Cassandra. Hands-on experience with big data and cloud technologies, including Hadoop, Spark, Kafka, and AWS services such as EC2, EMR, RDS, and Redshift. Experience with data orchestration, stream processing, automation, data security, and governance. Strong analytical and problem-solving skills, with the ability to translate business requirements into scalable, reliable, and secure data solutions. Excellent communication and collaboration skills, with the ability to work effectively across technical and business teams. Self-directed, adaptable, and committed to continuous improvement, data quality, and operational excellence. Relevant AWS, Microsoft, or Google Cloud certifications are preferred but not required.
POSITION SUMMARY The Data Engineer is responsible for designing, building, expanding, and optimizing data pipelines and data architecture to support cross-functional applications and business initiatives. The role focuses on developing reliable, scalable, and secure data solutions that enable efficient data collection, processing, integration, and delivery. The Data Engineer will collaborate with Digital Developers, Data Architects, Solution Architects, and other stakeholders to support data initiatives and ensure consistent data delivery architecture across projects. The role requires a self-directed professional who can support the data needs of multiple teams, systems, and products while contributing to the development of future-generation data architecture and solutions. DUTIES AND RESPONSIBILITIES Collect, integrate, and structure datasets based on business and functional requirements. Design, develop, test, and maintain scalable data pipelines and database architectures. Build data transformation processes and algorithms that convert raw data into reliable, actionable information. Design and optimize ETL/ELT processes for extracting, transforming, and loading data from diverse sources into Amazon Redshift and other data platforms. Identify and implement process improvements, including automation of manual processes and optimization of data delivery. Develop and implement data validation, quality, and monitoring processes to ensure data accuracy, consistency, and reliability. Support analytics initiatives by enabling data pipelines and solutions that provide actionable insights for marketing, campaigns, events, promotions, operational efficiency, and other key business performance metrics. Collaborate with cross-functional teams to understand business objectives and translate requirements into effective technical solutions. Support the design and implementation of scalable, resilient, redundant, and secure data infrastructure across multiple data centers and AWS regions. Ensure data is appropriately protected throughout its lifecycle and remains compliant with applicable data governance, privacy, and security policies. Provide technical support and expertise for data-related initiatives across multiple teams, systems, and products. Requirements QUALIFICATIONS & REQUIREMENTS Bachelor’s degree in Computer Science, Information Technology, Information Management, Engineering, or related field, with at least 5 years of relevant experience in Data Engineering or a related discipline. Strong experience in designing, developing, and optimizing data pipelines, data architecture, ETL/ELT processes, and data integration solutions. Proficiency in Python, Java, Scala, or C++, and experience with SQL and NoSQL databases, including PostgreSQL and Cassandra. Hands-on experience with big data and cloud technologies, including Hadoop, Spark, Kafka, and AWS services such as EC2, EMR, RDS, and Redshift. Experience with data orchestration, stream processing, automation, data security, and governance. Strong analytical and problem-solving skills, with the ability to translate business requirements into scalable, reliable, and secure data solutions. Excellent communication and collaboration skills, with the ability to work effectively across technical and business teams. Self-directed, adaptable, and committed to continuous improvement, data quality, and operational excellence. Relevant AWS, Microsoft, or Google Cloud certifications are preferred but not required.
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.