Back to jobs

Senior Data Engineer

Pay
Pay not listed
Work setting
Not stated
Employment
Full-time — employment source
Employment type in the employer’s posting: FULL_TIME
From the employer’s posting

What you’ll work on

Full posting

We are looking for a self-sufficient BI Analyst / Data Engineer to join a small, high-impact analytics team.

The ideal candidate is comfortable operating independently across the data stack, takes ownership of their work, and communicates proactively with team members across time zones.

  • Design and maintain AWS Glue (PySpark) ETL jobs into a raw-core-presentation data lake

  • Orchestrate pipelines with AWS Step Functions, including retries and concurrency

  • Build and maintain Amazon QuickSight dashboards and Athena-backed datasets

  • Troubleshoot production pipeline failures and performance bottlenecks

  • Manage Apache Iceberg tables on S3, including MERGE, compaction and cleanup

From the employer’s posting
We are looking for a self-sufficient BI Analyst / Data Engineer to join a small, high-impact analytics team. In this role you will own the full lifecycle of business intelligence delivery—from building and maintaining ETL pipelines in AWS to designing interactive dashboards in Amazon QuickSight. You will work closely with onshore stakeholders to translate business requirements into reliable, well-documented data products that drive operational and strategic decision-making.
The ideal candidate is comfortable operating independently across the data stack, takes ownership of their work, and communicates proactively with team members across time zones.
Design, build, and maintain AWS Glue (PySpark) ETL jobs that ingest data from multiple source systems
Develop and optimize AWS Step Functions workflows for pipeline orchestration, including parallel execution, error handling, retry logic, and concurrency management.
Build, publish, and maintain Amazon QuickSight dashboards and analyses that serve operational and executive audiences.
Troubleshoot production pipeline failures, diagnose performance bottlenecks (OOM errors, API timeouts, DPU tuning)
Work with Apache Iceberg table format on S3 for ACID-compliant analytics tables, including MERGE operations, compaction, snapshot expiration, and orphan file cleanup.

What you’ll bring

All qualifications

Core experience

  • 5–10+ years of professional experience in data engineering, BI development, or analytics engineering.
  • Proficiency in Python and SQL; experience writing PySpark transformations and Athena queries.
  • Experience building and maintaining ETL pipelines that ingest data from REST APIs or SaaS platforms (e.g., Salesforce, JIRA, Zendesk).
  • Familiarity with columnar/lakehouse table formats such as Apache Iceberg, Delta Lake, or Apache Hudi.
  • Solid understanding of data warehouse concepts: dimensional modeling, slowly changing dimensions, incremental vs.

Preferred experience

  • Experience with infrastructure-as-code tools such as AWS CDK, CloudFormation, or Terraform.
  • Familiarity with CI/CD pipelines (GitHub Actions, CodePipeline) for data engineering workflows.
  • Prior experience translating Tableau workbooks or Excel-based reports into another BI platform.
  • Knowledge of EventBridge-driven architectures and SQS/SNS messaging patterns.
Qualification wording
5–10+ years of professional experience in data engineering, BI development, or analytics engineering.
Proficiency in Python and SQL; experience writing PySpark transformations and Athena queries.
Experience building and maintaining ETL pipelines that ingest data from REST APIs or SaaS platforms (e.g., Salesforce, JIRA, Zendesk).
Familiarity with columnar/lakehouse table formats such as Apache Iceberg, Delta Lake, or Apache Hudi.
Solid understanding of data warehouse concepts: dimensional modeling, slowly changing dimensions, incremental vs. full loads.
Experience with infrastructure-as-code tools such as AWS CDK, CloudFormation, or Terraform.
Familiarity with CI/CD pipelines (GitHub Actions, CodePipeline) for data engineering workflows.
Prior experience translating Tableau workbooks or Excel-based reports into another BI platform.
Knowledge of EventBridge-driven architectures and SQS/SNS messaging patterns.
Apply at HealthEdge

Posting record

Report an error

Open 4 monthsWe first saw this posting 141 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.

Status in our records
Live
First seen by us
Jun 25, 2026141 days ago
Recorded sightings
160
Last seen by us
Nov 13, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 141

Up longer than most. Similar jobs usually come down between day 6 and day 56.

Well past HealthEdge’s usual: its jobs come down by about day 26.

From 13,630 similar postings that came down, and 16 of HealthEdge’s. Coming down is not always a hire.

HealthEdge is hiring

3 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Hiring has slowed
  • Takes jobs down at a usual pace
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
  • AWS
  • Azure
  • Delta Lake
  • Excel
  • Iceberg
  • S3
  • Tableau
  • PySpark
  • Terraform
Where the posting names them
- Strong hands-on experience with AWS Glue (PySpark), Step Functions, S3, Athena, and IAM. - Proficiency in Python and SQL; experience writing PySpark transformations and Athena queries. - Experience building and maintaining ETL pipelines that ingest data from REST APIs or SaaS platforms (e.g., Salesforce, JIRA, Zendesk).
Position Summary We are looking for a self-sufficient BI Analyst / Data Engineer to join a small, high-impact analytics team. In this role you will own the full lifecycle of business intelligence delivery—from building and maintaining ETL pipelines in AWS to designing interactive dashboards in Amazon QuickSight. You will work closely with onshore stakeholders to translate business requirements into reliable, well-documented data products that drive operational and strategic decision-making. The ideal candidate is comfortable operating independently across the data stack, takes ownership of their work, and communicates proactively with team members across time zones.
ETL / Data Pipeline Development - Design, build, and maintain AWS Glue (PySpark) ETL jobs that ingest data from multiple source systems such as Salesforce, JIRA, Azure DevOps, and Zendesk into a layered data lake architecture (raw → core → presentation). - Develop and optimize AWS Step Functions workflows for pipeline orchestration, including parallel execution, error handling, retry logic, and concurrency management.
- Design, build, and maintain AWS Glue (PySpark) ETL jobs that ingest data from multiple source systems such as Salesforce, JIRA, Azure DevOps, and Zendesk into a layered data lake architecture (raw → core → presentation). - Develop and optimize AWS Step Functions workflows for pipeline orchestration, including parallel execution, error handling, retry logic, and concurrency management. - Implement incremental loading strategies, schema evolution handling, and data quality checks within Glue scripts.
- Work with Apache Iceberg table format on S3 for ACID-compliant analytics tables, including MERGE operations, compaction, snapshot expiration, and orphan file cleanup. - Integrate supporting AWS services such as EventBridge, SQS, SNS, Athena, and CloudWatch into pipeline architectures. - Troubleshoot production pipeline failures, diagnose performance bottlenecks (OOM errors, API timeouts, DPU tuning), and implement fixes with minimal guidance.
- 5–10+ years of professional experience in data engineering, BI development, or analytics engineering. - Strong hands-on experience with AWS Glue (PySpark), Step Functions, S3, Athena, and IAM. - Proficiency in Python and SQL; experience writing PySpark transformations and Athena queries.
Preferred Qualifications - Experience with infrastructure-as-code tools such as AWS CDK, CloudFormation, or Terraform. - Familiarity with CI/CD pipelines (GitHub Actions, CodePipeline) for data engineering workflows.
- Knowledge of EventBridge-driven architectures and SQS/SNS messaging patterns. - AWS certifications (e.g., Data Analytics Specialty, Solutions Architect Associate) are a plus. What We Offer
- Working knowledge of Amazon QuickSight, including dataset creation, calculated fields, parameters, and dashboard publishing. - Familiarity with columnar/lakehouse table formats such as Apache Iceberg, Delta Lake, or Apache Hudi. - Solid understanding of data warehouse concepts: dimensional modeling, slowly changing dimensions, incremental vs. full loads.
- Create well-modeled QuickSight datasets backed by Athena queries over Iceberg tables, applying appropriate joins, calculated fields, and filters. - Translate business requirements and existing Tableau/Excel reports into QuickSight visuals, ensuring accuracy and usability. - Optimize dashboard performance, evaluating SPICE vs. Direct Query modes and managing refresh schedules.
- Exposure to PII detection/redaction practices or data governance frameworks. - Prior experience translating Tableau workbooks or Excel-based reports into another BI platform. - Knowledge of EventBridge-driven architectures and SQS/SNS messaging patterns.
- Implement incremental loading strategies, schema evolution handling, and data quality checks within Glue scripts. - Work with Apache Iceberg table format on S3 for ACID-compliant analytics tables, including MERGE operations, compaction, snapshot expiration, and orphan file cleanup. - Integrate supporting AWS services such as EventBridge, SQS, SNS, Athena, and CloudWatch into pipeline architectures.
- Build, publish, and maintain Amazon QuickSight dashboards and analyses that serve operational and executive audiences. - Create well-modeled QuickSight datasets backed by Athena queries over Iceberg tables, applying appropriate joins, calculated fields, and filters. - Translate business requirements and existing Tableau/Excel reports into QuickSight visuals, ensuring accuracy and usability.

Your next step

Complete your application on careers-healthedge.icims.com. The employer’s form will show what is required.

Applying through iCIMS

  • You start with your email. New applicants then create a login, and returning ones sign in.
  • Upload your resume before you fill anything in: iCIMS fills some profile fields from it.
  • Check your status on the employer’s iCIMS Dashboard, under Past Job Submittals.

Already applied? Track this application

Has HealthEdge posted this job before?

Yes. We’ve followed one other Senior Data Engineer job there since August 2026. It is still up.

  1. This jobUp 141 days so far, first seen Jun 25
  2. UnavailableStill up after 78 days, first seen Aug 27

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 7% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 2. The fewest in a week was 860, the most 1,354, and the latest week had 1,335.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Job description

View original posting ↗

Overview

Sr Data Engineer

Position Summary 

We are looking for a self-sufficient BI Analyst / Data Engineer to join a small, high-impact analytics team. In this role you will own the full lifecycle of business intelligence delivery—from building and maintaining ETL pipelines in AWS to designing interactive dashboards in Amazon QuickSight. You will work closely with onshore stakeholders to translate business requirements into reliable, well-documented data products that drive operational and strategic decision-making. 

The ideal candidate is comfortable operating independently across the data stack, takes ownership of their work, and communicates proactively with team members across time zones. 

 

Core Responsibilities 

ETL / Data Pipeline Development 

  • Design, build, and maintain AWS Glue (PySpark) ETL jobs that ingest data from multiple source systems such as Salesforce, JIRA, Azure DevOps, and Zendesk into a layered data lake architecture (raw → core → presentation). 
  • Develop and optimize AWS Step Functions workflows for pipeline orchestration, including parallel execution, error handling, retry logic, and concurrency management. 
  • Implement incremental loading strategies, schema evolution handling, and data quality checks within Glue scripts. 
  • Work with Apache Iceberg table format on S3 for ACID-compliant analytics tables, including MERGE operations, compaction, snapshot expiration, and orphan file cleanup. 
  • Integrate supporting AWS services such as EventBridge, SQS, SNS, Athena, and CloudWatch into pipeline architectures. 
  • Troubleshoot production pipeline failures, diagnose performance bottlenecks (OOM errors, API timeouts, DPU tuning), and implement fixes with minimal guidance. 
  • Write and maintain clear technical documentation for all pipeline components. 

 

Dashboard & Reporting Development 

  • Build, publish, and maintain Amazon QuickSight dashboards and analyses that serve operational and executive audiences. 
  • Create well-modeled QuickSight datasets backed by Athena queries over Iceberg tables, applying appropriate joins, calculated fields, and filters. 
  • Translate business requirements and existing Tableau/Excel reports into QuickSight visuals, ensuring accuracy and usability. 
  • Optimize dashboard performance, evaluating SPICE vs. Direct Query modes and managing refresh schedules. 
  • Collaborate with stakeholders to iterate on dashboard designs, incorporating feedback and refining KPIs. 

 

General Team Responsibilities 

  • Participate in code reviews, sprint planning, and knowledge-sharing sessions with the broader BI and engineering teams. 
  • Follow established Git branching strategies and CI/CD practices for infrastructure and code deployment. 
  • Monitor data freshness and pipeline health; respond to alerts and escalate issues as appropriate. 
  • Proactively identify opportunities to improve data quality, reduce pipeline run times, and enhance reporting capabilities. 

 

Required Qualifications 

  • 5–10+ years of professional experience in data engineering, BI development, or analytics engineering. 
  • Strong hands-on experience with AWS Glue (PySpark), Step Functions, S3, Athena, and IAM. 
  • Proficiency in Python and SQL; experience writing PySpark transformations and Athena queries. 
  • Experience building and maintaining ETL pipelines that ingest data from REST APIs or SaaS platforms (e.g., Salesforce, JIRA, Zendesk). 
  • Working knowledge of Amazon QuickSight, including dataset creation, calculated fields, parameters, and dashboard publishing. 
  • Familiarity with columnar/lakehouse table formats such as Apache Iceberg, Delta Lake, or Apache Hudi. 
  • Solid understanding of data warehouse concepts: dimensional modeling, slowly changing dimensions, incremental vs. full loads. 
  • Comfortable working independently with minimal day-to-day supervision, managing your own task priorities, and communicating status asynchronously. 
  • Strong written and verbal English communication skills; able to participate in cross-timezone meetings and write clear documentation. 

 

Preferred Qualifications 

  • Experience with infrastructure-as-code tools such as AWS CDK, CloudFormation, or Terraform. 
  • Familiarity with CI/CD pipelines (GitHub Actions, CodePipeline) for data engineering workflows. 
  • Exposure to PII detection/redaction practices or data governance frameworks. 
  • Prior experience translating Tableau workbooks or Excel-based reports into another BI platform. 
  • Knowledge of EventBridge-driven architectures and SQS/SNS messaging patterns. 
  • AWS certifications (e.g., Data Analytics Specialty, Solutions Architect Associate) are a plus. 

 

What We Offer 

  • The opportunity to work on a greenfield BI platform with modern cloud-native tooling. 
  • A collaborative, small-team environment where your contributions have direct and visible impact. 
  • Exposure to a broad technology stack spanning data engineering, analytics, and cloud infrastructure. 
  • Mentorship and growth opportunities within a growing data organization. 

 

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Bangalore

Working pattern and location restrictions need checking in the full posting.

Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.