Time since we first saw it
Day 4
Early. Similar jobs usually come down between day 6 and day 56.
From 14,770 similar postings that came down. Coming down is not always a hire.
Work setup in the employer’s posting: TELECOMMUTEFrom the employer’s posting
Migrate pandas, Koalas or pandas-on-Spark code to native PySpark
Decompose and refactor large legacy notebooks without existing documentation
Detect outliers and events in sensor time series using statistical methods
Package and manage ML models with MLflow, including tracking and Model Registry
Write unit and regression tests for PySpark pipelines using pytest
Migrate existing pandas, Koalas or pandas-on-Spark code to native PySpark implementations
Decompose and refactor large legacy notebooks without relying on existing documentation
Apply statistical techniques to detect outliers and events in sensor time series data
Package and manage machine learning models using MLflow, including experiment tracking, Model Registry and versioning
Develop unit and regression tests for PySpark pipelines using pytest
Actively hiringFirst seen by us 4 days ago, still listed, and not a repost of an earlier posting.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 4
Early. Similar jobs usually come down between day 6 and day 56.
From 14,770 similar postings that came down. Coming down is not always a hire.
892 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Hiring recordAdd your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
We are looking for a Lead Data Engineer to join our team and take ownership of complex data engineering challenges, from migrating legacy codebases to building robust, production-grade pipelines. In this role, you will work independently on large-scale PySpark systems, apply statistical methods to real-world sensor data, and leverage MLflow to manage the full lifecycle of machine learning models. Responsibilities Migrate existing pandas, Koalas or pandas-on-Spark code to native PySpark implementations Decompose and refactor large legacy notebooks without relying on existing documentation Apply statistical techniques to detect outliers and events in sensor time series data Package and manage machine learning models using MLflow, including experiment tracking, Model Registry and versioning Implement model explainability techniques such as SHAP for existing models Develop unit and regression tests for PySpark pipelines using pytest and tools like assertDataFrameEqual or chispa Work independently on data engineering projects with minimal oversight Integrate AI coding assistants into daily workflows and critically review their output Utilize Window functions to optimize data transformation logic Requirements 5+ years of experience in data engineering with deep knowledge of PySpark and pandas Proven experience migrating pandas, Koalas or pandas-on-Spark code to native PySpark, including Window functions Capability to read and decompose large legacy notebooks without documentation, and to work independently on them Expertise in applied statistics, including outlier detection and event detection in sensor time series Proficiency in MLflow for experiment tracking, Model Registry and versioning, with skills in packaging existing models including explainability techniques like SHAP Skills in writing unit and regression tests for PySpark using pytest and assertDataFrameEqual or chispa Familiarity with using AI coding assistants in daily work and reviewing their output critically English proficiency at B2 level or higher Nice to have Knowledge of incremental processing on Delta, including MERGE and Change Data Feed, and Databricks job triggers Background in working with sensor or IoT data Databricks Machine Learning Professional certification German proficiency
Complete your application on careers.epam.com. The employer’s form will show what is required.
Already applied? Track this application
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
Location not supplied.
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.