Time since we first saw it
Day 3
Early. Similar jobs usually come down between day 6 and day 56.
From 14,766 similar postings that came down. Coming down is not always a hire.
Bengaluru, KA, India
Actively hiring, first seen by us 3 days agoWe are hiring a Senior Data Engineer to own and scale a petabyte-scale data platform on Google Cloud Platform (GCP), processing TB scale of data every day with Apache Spark.
This is a product engineering role, not a services or consulting engagement.
Build and run batch and streaming pipelines on GCP that ingest hundreds of terabytes a day
Write and tune large Spark workloads, including skew handling and executor sizing
Lead and mentor 2–4 junior and mid-level data engineers
Own cloud cost of GCP data workloads and set up cost visibility and FinOps practices
Build AI-driven workflows for pipeline triage, root-cause analysis and runbook automation
We are hiring a Senior Data Engineer to own and scale a petabyte-scale data platform on Google Cloud Platform (GCP), processing TB scale of data every day with Apache Spark. You will lead a small team of data engineers, drive cloud cost optimisation across our data workloads, and build AI-driven workflows that change how we support and operate the platform.
This is a product engineering role, not a services or consulting engagement. You will own systems end to end, live with your design decisions in production, and help define data engineering culture in India.
Design, build and operate highly available, fault-tolerant batch and streaming pipelines on GCP
Write and tune large Spark workloads (PySpark, Spark SQL, Scala/Java) on Dataproc or Spark on Kubernetes (GKE)
Lead and mentor 2–4 junior and mid-level data engineers
Own the cost of our GCP data workloads
LLM-assisted triage of pipeline failures and data incidents, automated root-cause analysis, runbook automation
6 plus years in data engineering or backend/software engineering, with at least 4 years building big data pipelines in production.
Strong Python and Java with object-oriented design, design patterns and clean, testable, reusable code.
Factori is a rapidly growing location and consumer intelligence company built for an AI-first world.
In the employer’s words · Read in context
Actively hiringFirst seen by us 3 days ago, still listed, and not a repost of an earlier posting.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 3
Early. Similar jobs usually come down between day 6 and day 56.
From 14,766 similar postings that came down. Coming down is not always a hire.
2 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Hiring recordAdd your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
- Design, build and operate highly available, fault-tolerant batch and streaming pipelines on GCP that ingest hundreds of terabytes a day into a petabyte-scale data lake and warehouse. - Write and tune large Spark workloads (PySpark, Spark SQL, Scala/Java) on Dataproc or Spark on Kubernetes (GKE): partitioning, shuffle and join optimization, skew handling, memory and executor sizing. - Build real-time and near-real-time pipelines with Pub/Sub or Kafka and Spark Structured Streaming or Dataflow (Apache Beam), with clear guarantees on latency, ordering and exactly-once or idempotent processing.
Apache Spark at scale: - Deep hands-on PySpark and Spark SQL; Spark internals (DAG, stages, shuffle, catalyst, AQE), performance tuning, and handling data skew on multi-terabyte jobs. Scala or Java Spark is a strong plus. Google Cloud Platform:
Programming and OOP: - Strong Python and Java with object-oriented design, design patterns and clean, testable, reusable code. Data quality:
- Designed and run both, with a solid grasp of windowing, watermarks, late data, checkpointing, backfills and idempotency. Kafka or Pub/Sub plus Spark Structured Streaming, Flink or Dataflow. SQL: - Expert-level, including window functions, complex joins and query optimization on large analytical warehouses.
- Build real-time and near-real-time pipelines with Pub/Sub or Kafka and Spark Structured Streaming or Dataflow (Apache Beam), with clear guarantees on latency, ordering and exactly-once or idempotent processing. - Model data in BigQuery and the lakehouse (Parquet, Avro, Delta Lake or Apache Iceberg) into clean, simplified datasets that teams can self-serve. Own data quality
Lead cloud cost optimization - Own the cost of our GCP data workloads: measure cost per pipeline and per customer, then reduce it through right-sizing, autoscaling, preemptible/Spot VMs, storage tiering, BigQuery slot and query optimization, and job consolidation. - Set up cost visibility (billing export, labels, dashboards) and FinOps practices so cost is tracked and owned, not discovered after the fact.
Google Cloud Platform: - Production experience with BigQuery, Dataproc, Cloud Storage (GCS), Pub/Sub, and Cloud Composer or Airflow. Dataflow, GKE and Bigtable are a plus. Batch and stream processing:
- Google Cloud Professional Data Engineer or Professional Cloud Architect certification. - Lakehouse table formats: Delta Lake, Apache Iceberg or Apache Hudi. dbt, Trino/Presto or ClickHouse. - Terraform, Docker and Kubernetes (GKE) for data infrastructure.
- Lakehouse table formats: Delta Lake, Apache Iceberg or Apache Hudi. dbt, Trino/Presto or ClickHouse. - Terraform, Docker and Kubernetes (GKE) for data infrastructure. - Data observability and lineage tools such as Monte Carlo, OpenLineage or DataHub.
Databases and storage: - Parquet, Avro and ORC; partitioning and clustering; production experience with PostgreSQL, Elasticsearch and Bigtable (or comparable relational, search and NoSQL stores). Good to have :
About the Role: We are hiring a Senior Data Engineer to own and scale a petabyte-scale data platform on Google Cloud Platform (GCP), processing TB scale of data every day with Apache Spark. You will lead a small team of data engineers, drive cloud cost optimisation across our data workloads, and build AI-driven workflows that change how we support and operate the platform. This is a product engineering role, not a services or consulting engagement. You will own systems end to end, live with your design decisions in production, and help define data engineering culture in India.
- Deep hands-on PySpark and Spark SQL; Spark internals (DAG, stages, shuffle, catalyst, AQE), performance tuning, and handling data skew on multi-terabyte jobs. Scala or Java Spark is a strong plus. Google Cloud Platform: - Production experience with BigQuery, Dataproc, Cloud Storage (GCS), Pub/Sub, and Cloud Composer or Airflow. Dataflow, GKE and Bigtable are a plus.
Good to have : - Google Cloud Professional Data Engineer or Professional Cloud Architect certification. - Lakehouse table formats: Delta Lake, Apache Iceberg or Apache Hudi. dbt, Trino/Presto or ClickHouse.
- Write and tune large Spark workloads (PySpark, Spark SQL, Scala/Java) on Dataproc or Spark on Kubernetes (GKE): partitioning, shuffle and join optimization, skew handling, memory and executor sizing. - Build real-time and near-real-time pipelines with Pub/Sub or Kafka and Spark Structured Streaming or Dataflow (Apache Beam), with clear guarantees on latency, ordering and exactly-once or idempotent processing. - Model data in BigQuery and the lakehouse (Parquet, Avro, Delta Lake or Apache Iceberg) into clean, simplified datasets that teams can self-serve.
Batch and stream processing: - Designed and run both, with a solid grasp of windowing, watermarks, late data, checkpointing, backfills and idempotency. Kafka or Pub/Sub plus Spark Structured Streaming, Flink or Dataflow. SQL:
- Most of your experience at product or SaaS companies, owning platforms you built and ran in production, rather than consulting or IT services delivery. Apache Spark at scale: - Deep hands-on PySpark and Spark SQL; Spark internals (DAG, stages, shuffle, catalyst, AQE), performance tuning, and handling data skew on multi-terabyte jobs. Scala or Java Spark is a strong plus.
- Own production reliability: on-call, monitoring and observability, incident response, postmortems, and low-level debugging and performance tuning on large clusters. - Orchestrate pipelines with Airflow / Cloud Composer and ship through CI/CD and infrastructure as code (Terraform). What we are looking for :
Complete your application on lifesight.keka.com. The employer’s form will show what is required.
Already applied? Track this application
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
Bengaluru, KA, India
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.