Time since we first saw it
Day 0
Early. Similar jobs usually come down between day 5 and day 54.
From 10,736 similar postings that came down. Coming down is not always a hire.
Lead Data Engineer
- Pay
- Pay not listed
- Work setting
Remote — work setting source
Work setup in the employer’s posting: TELECOMMUTE
From the employer’s posting- Employment
- Not stated
What you’ll work on
Full postingBuild and maintain scalable data pipelines on AWS infrastructure
Build data processing solutions with Python, Pandas, and PySpark
Set up and oversee workflows on AWS MWAA (Airflow)
Automate infrastructure and deployments with Terraform and GitHub Actions
Add testing, monitoring, and data-quality safeguards to pipelines
From the employer’s posting
Create, construct, and sustain scalable data pipelines running on AWS infrastructure
Build dependable data processing solutions leveraging Python, Pandas, and PySpark
Establish and oversee workflows via AWS Managed Workflows for Apache Airflow (MWAA)
Streamline infrastructure setup and deployment workflows through automation using Terraform and GitHub Actions
Put in place testing, monitoring, troubleshooting practices, and data-quality safeguards to keep pipelines dependable
What you’ll bring
All qualificationsCore experience
- At least 5 years of applicable professional experience
- At least one year leading and managing development teams
- Strong PySpark skills for large-scale distributed data processing
- Core Python for data engineering, plus Pandas
- Experience with AWS MWAA or Apache Airflow
Qualification wording
At least 5 years of applicable professional background
A minimum of one year spent leading and managing development teams
Strong capability with PySpark for handling large-scale distributed data processing
Solid foundation in core Python programming applied to data engineering work
Background working with AWS MWAA or Apache Airflow to coordinate and manage data workflows
Tools in this posting
- Python
- AWS
- Terraform
- Airflow
- pandas
- PySpark
Source — Tool mentions in context
We're seeking a hands-on Lead Data Engineer to construct and maintain scalable data pipelines along with cloud-based data solutions. The right person for this role brings substantial background in Python, PySpark, AWS, Airflow, and DevOps, with particular attention to production-grade engineering and automation practices Responsibilities Create, construct, and sustain scalable data pipelines running on AWS infrastructure Build dependable data processing solutions leveraging Python, Pandas, and PySpark Establish and oversee workflows via AWS Managed Workflows for Apache Airflow (MWAA) Craft and fine-tune data queries through AWS Athena to enable efficient analytical work Streamline infrastructure setup and deployment workflows through automation using Terraform and GitHub Actions Put in place testing, monitoring, troubleshooting practices, and data-quality safeguards to keep pipelines dependable Take part in reviewing code and help promote sound engineering practices throughout the team Work alongside engineering, product, and business groups within an Agile setting Requirements At least 5 years of applicable professional background A minimum of one year spent leading and managing development teams Solid foundation in core Python programming applied to data engineering work Practical experience applying Python Pandas for manipulating and analyzing data Strong capability with PySpark for handling large-scale distributed data processing Background working with AWS MWAA or Apache Airflow to coordinate and manage data workflows Familiarity with AWS Athena for running queries and analyzing data housed within AWS Hands-on background using Terraform to automate infrastructure setup and management Working knowledge of GitHub Actions for constructing automated CI/CD pipelines Good grasp of DevOps principles as applied within data engineering contexts Strong capabilities in data engineering, troubleshooting, and resolving complex problems Upper-intermediate English language skills or higher (B1) Nice to have Background applying GitHub Copilot to assist with development, code reviews, testing, and release processes Knowledge of CI/CD methodologies and associated tools for smoother software delivery Background working within Agile/Scrum team structures Experience gained within financial services or investment management sectors Background in Gen AI application development and incorporating Gen AI practices into the software development lifecycle (SDLC)
Posting record
Report an errorActively hiringFirst seen by us today, still listed, and not a repost of an earlier posting.
- Status in our records
- Live
- First seen by us
- Nov 4, 2026today
- Recorded sightings
- 1
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
How this job compares
EPAM Systems is hiring
656 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Hiring recordYour next step
Check the employer’s posting for the current role and application details.
Already applied? Track this application
Has EPAM Systems posted this job before?
Yes. We’ve followed 10 other Lead Data Engineer jobs there since November 2026. All 10 are still up.
Large employers often post one title for several teams, so these can be separate openings.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
- Median yearly pay, May 2025
- $139,500
- Expected change, 2025 to 2035
- 9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Job description
We're seeking a hands-on Lead Data Engineer to construct and maintain scalable data pipelines along with cloud-based data solutions. The right person for this role brings substantial background in Python, PySpark, AWS, Airflow, and DevOps, with particular attention to production-grade engineering and automation practices Responsibilities Create, construct, and sustain scalable data pipelines running on AWS infrastructure Build dependable data processing solutions leveraging Python, Pandas, and PySpark Establish and oversee workflows via AWS Managed Workflows for Apache Airflow (MWAA) Craft and fine-tune data queries through AWS Athena to enable efficient analytical work Streamline infrastructure setup and deployment workflows through automation using Terraform and GitHub Actions Put in place testing, monitoring, troubleshooting practices, and data-quality safeguards to keep pipelines dependable Take part in reviewing code and help promote sound engineering practices throughout the team Work alongside engineering, product, and business groups within an Agile setting Requirements At least 5 years of applicable professional background A minimum of one year spent leading and managing development teams Solid foundation in core Python programming applied to data engineering work Practical experience applying Python Pandas for manipulating and analyzing data Strong capability with PySpark for handling large-scale distributed data processing Background working with AWS MWAA or Apache Airflow to coordinate and manage data workflows Familiarity with AWS Athena for running queries and analyzing data housed within AWS Hands-on background using Terraform to automate infrastructure setup and management Working knowledge of GitHub Actions for constructing automated CI/CD pipelines Good grasp of DevOps principles as applied within data engineering contexts Strong capabilities in data engineering, troubleshooting, and resolving complex problems Upper-intermediate English language skills or higher (B1) Nice to have Background applying GitHub Copilot to assist with development, code reviews, testing, and release processes Knowledge of CI/CD methodologies and associated tools for smoother software delivery Background working within Agile/Scrum team structures Experience gained within financial services or investment management sectors Background in Gen AI application development and incorporating Gen AI practices into the software development lifecycle (SDLC)
Source details
Source notes
Source excerptsSelected passages from the saved posting. Check the full description for conditions and exceptions.
- Pay
No pay amount identified in the saved description.
- Location & working pattern
Location not supplied.
Working pattern and location restrictions need checking in the full posting.
- Work authorization
No clear work-authorization passage found. Eligibility is unconfirmed.
AI answers unavailable
We couldn’t identify enough role detail in this saved description to support an AI answer. Read the full posting