Senior Data Engineer

Chennai- Ascendas IT Park

Open 5 months, first seen by us 176 days ago

Pay: Pay not listed
Work setting: Not stated
Employment: Full-time — employment source
Employment type in the employer’s posting: Full time
From the employer’s posting

What you’ll work on

Full posting

Build, test and maintain ETL/ELT data pipelines on Azure Databricks and PySpark
Optimize performance and cost-efficiency of Spark jobs
Design and enforce access control policies using Unity Catalog
Ensure data quality through validation, monitoring and alerting

From the employer’s posting

Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark)

Optimize performance and cost-efficiency of Spark jobs.

Design and enforce access control policies using Unity Catalog.

Ensure data quality through validation, monitoring, and alerting mechanisms.

What you’ll bring

All qualifications

Preferred experience

Experience with MLflow for model tracking & deployment in Databricks.
Ability to work independently and in teams.
Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid).
Excellent communication and documentation skills.

Qualification wording

Experience with MLflow for model tracking & deployment in Databricks.

Ability to work independently and in teams.

Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid).

Excellent communication and documentation skills.

Apply at Toppanmerrill

Posting record

Report an error

Open 5 monthsWe first saw this posting 176 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.

Status in our records: Live
First seen by us: May 12, 2026176 days ago
Recorded sightings: 91
Last seen by us: Nov 4, 2026
Employer says posted: Sep 11, 2025

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 176

Up longer than most. Similar jobs usually come down between day 5 and day 54.

From 10,733 similar postings that came down. Coming down is not always a hire.

Toppanmerrill is hiring

1 open job

Its jobs tend to stay up a long time

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

Python
SQL
Azure
Databricks
dbt
Delta Lake
Kafka
MLflow
Spark
SQL Server
PySpark

Where the posting names them

- Azure Databricks & Apache Spark (PySpark) – Strong experience in building distributed data pipelines. - Python – Proficiency in writing optimized and maintainable Python code for data engineering. - Unity Catalog – Hands-on experience implementing data governance, access controls, and lineage tracking.

- Unity Catalog – Hands-on experience implementing data governance, access controls, and lineage tracking. - SQL – Strong knowledge of SQL for data transformations and optimizations. - Delta Lake – Understanding of time travel, schema evolution, and performance tuning.

- Certification in Databricks, Azure is a plus. - Experience with Azure Databricks Lakehouse connectors for SalesForce and SQL Server - Experience with Azure Synapse Link for Dynamics, dataverse

Responsibilities: - Develop & Optimize Data Pipelines- Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark). - Optimize performance and cost-efficiency of Spark jobs.

- Enable secure data sharing across teams and external stakeholders. - Integrate with Cloud Data Platforms- Work with Azure Data Lake Storage / Azure Blob Storage/ Azure Event Hub to integrate Databricks with cloud-based data lakes, data warehouses, and event streams. - Implement Delta Lake for scalable, ACID-compliant storage.

- Implement Delta Lake for scalable, ACID-compliant storage. - Automate & Orchestrate Workflows- Develop CI/CD pipelines for data workflows using Azure Databricks Workflows or Azure Data Factory. - Monitor and troubleshoot failures in job execution and cluster performance.

Required Skills & Experience: - Azure Databricks & Apache Spark (PySpark) – Strong experience in building distributed data pipelines. - Python – Proficiency in writing optimized and maintainable Python code for data engineering.

- Delta Lake – Understanding of time travel, schema evolution, and performance tuning. - Workflow Orchestration – Experience with Azure Databricks Jobs or Azure Data Factory. - CI/CD & Infrastructure as Code (IaC) – Familiarity with Databricks CLI, Databricks DABs, and DevOps principles.

- Experience with MLflow for model tracking & deployment in Databricks. - Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid). - Hands-on experience with dbt (Data Build Tool) for modular ETL development.

- Hands-on experience with dbt (Data Build Tool) for modular ETL development. - Certification in Databricks, Azure is a plus. - Experience with Azure Databricks Lakehouse connectors for SalesForce and SQL Server

- Experience with Azure Databricks Lakehouse connectors for SalesForce and SQL Server - Experience with Azure Synapse Link for Dynamics, dataverse - Familiarity with other data pipeline strategies, like Azure Functions, Fabric, ADF, etc

- Experience with Azure Synapse Link for Dynamics, dataverse - Familiarity with other data pipeline strategies, like Azure Functions, Fabric, ADF, etc Soft Skills:

- Workflow Orchestration – Experience with Azure Databricks Jobs or Azure Data Factory. - CI/CD & Infrastructure as Code (IaC) – Familiarity with Databricks CLI, Databricks DABs, and DevOps principles. - Security & Compliance – Knowledge of IAM, role-based access control (RBAC), and encryption.

Preferred Qualifications: - Experience with MLflow for model tracking & deployment in Databricks. - Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid).

- Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid). - Hands-on experience with dbt (Data Build Tool) for modular ETL development. - Certification in Databricks, Azure is a plus.

- Integrate with Cloud Data Platforms- Work with Azure Data Lake Storage / Azure Blob Storage/ Azure Event Hub to integrate Databricks with cloud-based data lakes, data warehouses, and event streams. - Implement Delta Lake for scalable, ACID-compliant storage. - Automate & Orchestrate Workflows- Develop CI/CD pipelines for data workflows using Azure Databricks Workflows or Azure Data Factory.

- SQL – Strong knowledge of SQL for data transformations and optimizations. - Delta Lake – Understanding of time travel, schema evolution, and performance tuning. - Workflow Orchestration – Experience with Azure Databricks Jobs or Azure Data Factory.

- Develop & Optimize Data Pipelines- Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark). - Optimize performance and cost-efficiency of Spark jobs. - Ensure data quality through validation, monitoring, and alerting mechanisms.

Upload resume Add LinkedIn

Your next step

Complete your application on toppanmerrill.wd5.myworkdayjobs.com. The employer’s form will show what is required.

Applying through Workday

You create an account with each employer before you can upload anything. Every Workday employer has its own account, so one sign-up doesn’t carry over.
“Autofill with Resume” comes right after sign-in, before the My Information and My Experience steps. Check every field it fills in those steps, since that’s what recruiters see.
Your application status lives in that employer’s Candidate Home, under the same sign-in.

Already applied? Track this application

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025: $139,500
Expected change, 2025 to 2035: 9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Job description

View original posting ↗

Job Description:

Responsibilities:

Develop & Optimize Data Pipelines
- Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark).
- Optimize performance and cost-efficiency of Spark jobs.
- Ensure data quality through validation, monitoring, and alerting mechanisms.
- Understand cluster types, configuration, and use-case for serverless
Implement Unity Catalog for Data Governance
- Design and enforce access control policies using Unity Catalog.
- Manage data lineage, auditing, and metadata governance.
- Enable secure data sharing across teams and external stakeholders.
Integrate with Cloud Data Platforms
- Work with Azure Data Lake Storage / Azure Blob Storage/ Azure Event Hub to integrate Databricks with cloud-based data lakes, data warehouses, and event streams.
- Implement Delta Lake for scalable, ACID-compliant storage.
Automate & Orchestrate Workflows
- Develop CI/CD pipelines for data workflows using Azure Databricks Workflows or Azure Data Factory.
- Monitor and troubleshoot failures in job execution and cluster performance.
Collaborate with Stakeholders
- Work with Data Analysts, Scientists, and Business Teams to understand requirements.
- Translate business needs into scalable data engineering solutions.
API expertise
- Ability to pull data from a wide variety of APIs using different strategies and methods

Required Skills & Experience:

Azure Databricks & Apache Spark (PySpark) – Strong experience in building distributed data pipelines.
Python – Proficiency in writing optimized and maintainable Python code for data engineering.
Unity Catalog – Hands-on experience implementing data governance, access controls, and lineage tracking.
SQL – Strong knowledge of SQL for data transformations and optimizations.
Delta Lake – Understanding of time travel, schema evolution, and performance tuning.
Workflow Orchestration – Experience with Azure Databricks Jobs or Azure Data Factory.
CI/CD & Infrastructure as Code (IaC) – Familiarity with Databricks CLI, Databricks DABs, and DevOps principles.
Security & Compliance – Knowledge of IAM, role-based access control (RBAC), and encryption.

Preferred Qualifications:

Experience with MLflow for model tracking & deployment in Databricks.
Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid).
Hands-on experience with dbt (Data Build Tool) for modular ETL development.
Certification in Databricks, Azure is a plus.
Experience with Azure Databricks Lakehouse connectors for SalesForce and SQL Server
Experience with Azure Synapse Link for Dynamics, dataverse
Familiarity with other data pipeline strategies, like Azure Functions, Fabric, ADF, etc

Soft Skills:

Strong problem-solving and debugging skills.
Ability to work independently and in teams.
Excellent communication and documentation skills.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern