Time since we first saw it
Day 3
Early. Similar jobs usually come down between day 6 and day 56.
From 14,771 similar postings that came down. Coming down is not always a hire.
Gurugram, HR, India
Actively hiring, first seen by us 3 days agoWe are seeking a Principal Data Engineer to serve as the most senior technical authority in our data and analytics practice.
You will set the technical direction for large-scale batch and streaming platforms, make the build-versus-buy and pattern decisions that shape delivery for years, and remain close enough to the code to review it, tune it, and unblock the team when it matters.
Own end-to-end architecture for AWS lakehouse, warehouse, and streaming platforms
Lead modernisation and migration from Hadoop, Teradata, and Informatica to AWS
Lead and line-manage a team of data engineers across multiple squads
Lead performance tuning across Redshift, Athena, Spark, and streaming workloads
Act as final reviewer on critical pull requests and raise the team's review bar
We are seeking a Principal Data Engineer to serve as the most senior technical authority in our data and analytics practice. This is a hands-on leadership role: you will own the architecture and engineering standards for AWS-native data platforms, lead and mentor multi-pod engineering teams, and act as the trusted technical counterpart to customer stakeholders.
You will set the technical direction for large-scale batch and streaming platforms, make the build-versus-buy and pattern decisions that shape delivery for years, and remain close enough to the code to review it, tune it, and unblock the team when it matters. Success in this role is measured as much by the capability you build in others as by the systems you build yourself.
Own end-to-end architecture for AWS-native data platforms — lakehouse, warehouse, streaming, and data product layers
lead modernisation and migration programmes (on-premises Hadoop/Teradata/Informatica → AWS; legacy ETL → Glue/EMR/Iceberg)
Lead and technically line-manage a team of data engineers across multiple squads
Lead deep performance engineering across Redshift, Athena, Spark on EMR/Glue, and streaming workloads
Act as final reviewer and approver on critical pull requests; run structured code review sessions and raise the review bar across the team.
================================== - Bachelor's degree in Computer Science, Information Technology, Data Analytics, or a related field. Master's degree is an advantage. - 10–15 years of overall IT experience, with 8+ years designing and delivering data platforms on AWS.
Actively hiringFirst seen by us 3 days ago, still listed, and not a repost of an earlier posting.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 3
Early. Similar jobs usually come down between day 6 and day 56.
From 14,771 similar postings that came down. Coming down is not always a hire.
20 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Hiring recordAdd your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
- Expert-level SQL, including complex analytical patterns and performance tuning on very large datasets. - Strong programming proficiency in Python and PySpark, with the ability to set code quality standards and review others' work critically. - Proven experience with distributed data processing internals (Spark execution model, partitioning, shuffles, memory management).
- Operations & Monitoring: Amazon CloudWatch, AWS CloudTrail, AWS X-Ray, AWS Cost Explorer, AWS Well-Architected Tool. - Expert-level SQL, including complex analytical patterns and performance tuning on very large datasets. - Strong programming proficiency in Python and PySpark, with the ability to set code quality standards and review others' work critically.
============== We are seeking a Principal Data Engineer to serve as the most senior technical authority in our data and analytics practice. This is a hands-on leadership role: you will own the architecture and engineering standards for AWS-native data platforms, lead and mentor multi-pod engineering teams, and act as the trusted technical counterpart to customer stakeholders. You will set the technical direction for large-scale batch and streaming platforms, make the build-versus-buy and pattern decisions that shape delivery for years, and remain close enough to the code to review it, tune it, and unblock the team when it matters. Success in this role is measured as much by the capability you build in others as by the systems you build yourself.
ARCHITECTURE & TECHNICAL DESIGN - Own end-to-end architecture for AWS-native data platforms — lakehouse, warehouse, streaming, and data product layers — from discovery through production hardening. - Produce and maintain architecture artefacts: solution design documents, HLD/LLD, data flow and lineage diagrams, ADRs (Architecture Decision Records), and reference implementations.
- Produce and maintain architecture artefacts: solution design documents, HLD/LLD, data flow and lineage diagrams, ADRs (Architecture Decision Records), and reference implementations. - Define the target-state roadmap and lead modernisation and migration programmes (on-premises Hadoop/Teradata/Informatica → AWS; legacy ETL → Glue/EMR/Iceberg). - Evaluate and select services, table formats, and third-party tooling with clear trade-off analysis on cost, performance, operability, and lock-in.
PLATFORM & PIPELINE ENGINEERING - Direct the design and build of scalable batch and streaming pipelines using AWS Glue, Amazon EMR, Amazon Kinesis, Amazon MSK, and AWS Lambda. - Model and optimise analytical data stores in Amazon Redshift and S3-based data lakes — partitioning strategy, file formats, compaction, distribution and sort keys, workload management.
- Model and optimise analytical data stores in Amazon Redshift and S3-based data lakes — partitioning strategy, file formats, compaction, distribution and sort keys, workload management. - Set the standards for CDC and ingestion patterns from operational databases and SaaS sources using AWS DMS, Glue connectors, and Kinesis. - Own orchestration patterns across Amazon MWAA, AWS Step Functions, and Amazon EventBridge, including idempotency, retry, backfill, and SLA-breach handling.
- Set the standards for CDC and ingestion patterns from operational databases and SaaS sources using AWS DMS, Glue connectors, and Kinesis. - Own orchestration patterns across Amazon MWAA, AWS Step Functions, and Amazon EventBridge, including idempotency, retry, backfill, and SLA-breach handling. - Own data cataloguing, lineage, and fine-grained access control via AWS Glue Data Catalog and AWS Lake Formation.
- Own orchestration patterns across Amazon MWAA, AWS Step Functions, and Amazon EventBridge, including idempotency, retry, backfill, and SLA-breach handling. - Own data cataloguing, lineage, and fine-grained access control via AWS Glue Data Catalog and AWS Lake Formation. - Define the data quality and observability framework — contracts, expectations, freshness and volume checks, reconciliation, and alerting.
- Own CI/CD for data pipelines — automated testing (unit, integration, data quality), linting, security scanning, environment promotion, and release management. - Champion infrastructure as code (Terraform, AWS CDK, or CloudFormation) and reusable, modular, well-tested platform components over bespoke one-off builds. - Drive technical debt visibility and remediation planning alongside feature delivery.
- Own PII discovery, classification, masking, tokenisation, and retention patterns; ensure designs meet applicable regulatory obligations (India DPDP Act 2023, GDPR where relevant) and audit requirements under ISO 27001 and SOC 2. - Define data governance operating model in partnership with security and compliance functions — data ownership, access request workflows, and audit logging via AWS CloudTrail and Lake Formation. REQUIRED SKILLS AND QUALIFICATIONS
- Bachelor's degree in Computer Science, Information Technology, Data Analytics, or a related field. Master's degree is an advantage. - 10–15 years of overall IT experience, with 8+ years designing and delivering data platforms on AWS. - Demonstrable track record as the lead architect or principal engineer on at least two large-scale AWS data platform builds or migrations.
- 10–15 years of overall IT experience, with 8+ years designing and delivering data platforms on AWS. - Demonstrable track record as the lead architect or principal engineer on at least two large-scale AWS data platform builds or migrations. - Deep expertise across AWS services and architectures, including:
- Demonstrable track record as the lead architect or principal engineer on at least two large-scale AWS data platform builds or migrations. - Deep expertise across AWS services and architectures, including: - Compute: EC2, Amazon EKS, Amazon ECS, AWS Lambda, AWS Fargate, AWS Batch.
- Deep expertise across AWS services and architectures, including: - Compute: EC2, Amazon EKS, Amazon ECS, AWS Lambda, AWS Fargate, AWS Batch. - Storage & Databases: Amazon S3, Amazon RDS, Amazon Aurora, Amazon Redshift, DynamoDB, Amazon Keyspaces, Amazon ElastiCache.
- Storage & Databases: Amazon S3, Amazon RDS, Amazon Aurora, Amazon Redshift, DynamoDB, Amazon Keyspaces, Amazon ElastiCache. - Data & Analytics: AWS Glue (ETL, Data Catalog, DataBrew), Amazon EMR, Amazon Athena, Amazon Kinesis (Data Streams, Firehose, Managed Service for Apache Flink), Amazon MSK, AWS DMS, AWS Lake Formation, Amazon MWAA, AWS Step Functions, Amazon QuickSight. - Operations & Monitoring: Amazon CloudWatch, AWS CloudTrail, AWS X-Ray, AWS Cost Explorer, AWS Well-Architected Tool.
- Data & Analytics: AWS Glue (ETL, Data Catalog, DataBrew), Amazon EMR, Amazon Athena, Amazon Kinesis (Data Streams, Firehose, Managed Service for Apache Flink), Amazon MSK, AWS DMS, AWS Lake Formation, Amazon MWAA, AWS Step Functions, Amazon QuickSight. - Operations & Monitoring: Amazon CloudWatch, AWS CloudTrail, AWS X-Ray, AWS Cost Explorer, AWS Well-Architected Tool. - Expert-level SQL, including complex analytical patterns and performance tuning on very large datasets.
=============================== - AWS Certified Data Engineer – Associate, AWS Certified Solutions Architect – Professional, or AWS Certified Data Analytics – Specialty. - Hands-on experience with modern lakehouse table formats — Apache Iceberg, Delta Lake, or Apache Hudi — including migration and optimisation at scale.
- Hands-on experience with modern lakehouse table formats — Apache Iceberg, Delta Lake, or Apache Hudi — including migration and optimisation at scale. - Experience with another hyperscaler (Azure or GCP), demonstrating breadth in data engineering. - Experience with dbt, Great Expectations, Soda, Monte Carlo, or equivalent transformation and data-quality tooling.
- Experience with another hyperscaler (Azure or GCP), demonstrating breadth in data engineering. - Experience with dbt, Great Expectations, Soda, Monte Carlo, or equivalent transformation and data-quality tooling. - Exposure to data mesh, data contracts, or domain-oriented data product operating models.
- AWS Certified Data Engineer – Associate, AWS Certified Solutions Architect – Professional, or AWS Certified Data Analytics – Specialty. - Hands-on experience with modern lakehouse table formats — Apache Iceberg, Delta Lake, or Apache Hudi — including migration and optimisation at scale. - Experience with another hyperscaler (Azure or GCP), demonstrating breadth in data engineering.
- Compute: EC2, Amazon EKS, Amazon ECS, AWS Lambda, AWS Fargate, AWS Batch. - Storage & Databases: Amazon S3, Amazon RDS, Amazon Aurora, Amazon Redshift, DynamoDB, Amazon Keyspaces, Amazon ElastiCache. - Data & Analytics: AWS Glue (ETL, Data Catalog, DataBrew), Amazon EMR, Amazon Athena, Amazon Kinesis (Data Streams, Firehose, Managed Service for Apache Flink), Amazon MSK, AWS DMS, AWS Lake Formation, Amazon MWAA, AWS Step Functions, Amazon QuickSight.
- Experience integrating ML and GenAI workloads — feature stores, Amazon SageMaker, Amazon Bedrock, vector stores, and RAG data pipelines. - Real-time and event-driven architecture experience with Apache Flink or Kafka Streams. - Prior experience in a consulting or professional services environment with direct client ownership.
- Direct the design and build of scalable batch and streaming pipelines using AWS Glue, Amazon EMR, Amazon Kinesis, Amazon MSK, and AWS Lambda. - Model and optimise analytical data stores in Amazon Redshift and S3-based data lakes — partitioning strategy, file formats, compaction, distribution and sort keys, workload management. - Set the standards for CDC and ingestion patterns from operational databases and SaaS sources using AWS DMS, Glue connectors, and Kinesis.
PERFORMANCE TUNING & COST OPTIMISATION - Lead deep performance engineering across Redshift, Athena, Spark on EMR/Glue, and streaming workloads: query plan analysis, skew and spill remediation, partition pruning, caching, concurrency scaling, and shuffle optimisation. - Establish benchmarking and profiling practice — define baselines, instrument workloads, and drive measurable improvements in latency, throughput, and job runtime.
- Establish benchmarking and profiling practice — define baselines, instrument workloads, and drive measurable improvements in latency, throughput, and job runtime. - Own the FinOps posture for the data estate: right-sizing, Spot and Graviton adoption, storage tiering and lifecycle policies, Redshift RA3/serverless sizing, and per-workload cost attribution and chargeback. - Set and enforce cost and performance SLOs, and run regular optimisation reviews with engineering and finance stakeholders.
- Exposure to data mesh, data contracts, or domain-oriented data product operating models. - Experience integrating ML and GenAI workloads — feature stores, Amazon SageMaker, Amazon Bedrock, vector stores, and RAG data pipelines. - Real-time and event-driven architecture experience with Apache Flink or Kafka Streams.
Complete your application on minfy.keka.com. The employer’s form will show what is required.
Already applied? Track this application
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
Gurugram, HR, India
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.