Pay
$127K–$189K is typical
The middle half of what similar US postings state. This one doesn’t list pay.
From 2,448 postings in the last year. Stated ranges, not salaries paid.
San Francisco
Open 5 months, first seen by us 172 days agoWork setup in the employer’s posting: IN_OFFICEFrom the employer’s posting
Employment type in the employer’s posting: FULL_TIMEFrom the employer’s posting
Design, build and scale data pipelines and data services for real-time and batch workloads
Operate ingest and transformation workflows with alerting, retries, SLAs and data quality checks
Model and optimize Snowflake and PostgreSQL schemas for high-volume, low-latency use
Build reusable frameworks and connectors for publishing and consuming data
Implement observability across pipelines and services, including lineage and incident playbooks
Design, build, and scale data pipelines and data services using Python, TypeScript, Apache Airflow, PySpark, AWS Glue, and Snowflake
Design, operationalize, and monitor ingest and transformation workflows, including DAGs, alerting, retries, SLAs, and robust data quality checks
Model data structures and implement efficient, scalable transformations in Snowflake and PostgreSQL
Build reusable frameworks, connectors, and internal libraries that standardize how data is published, discovered, and consumed
Implement and continuously improve observability across pipelines and services: structured logging, metrics, tracing, data quality monitoring, lineage, and incident response playbooks.
Open 5 monthsWe first saw this posting 172 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
$127K–$189K is typical
The middle half of what similar US postings state. This one doesn’t list pay.
From 2,448 postings in the last year. Stated ranges, not salaries paid.
Day 172
Up longer than most. Similar jobs usually come down between day 6 and day 56.
From 14,771 similar postings that came down. Coming down is not always a hire.
1 open job
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
Responsibilities - Design, build, and scale data pipelines and data services using Python, TypeScript, Apache Airflow, PySpark, AWS Glue, and Snowflake to support both real-time and batch workloads. - Design, operationalize, and monitor ingest and transformation workflows, including DAGs, alerting, retries, SLAs, and robust data quality checks for production environments.
- 5 plus years of production data engineering experience, including owning critical pipelines, datasets, and services in live environments. - Deep, hands-on experience with Apache Airflow, AWS Glue, PySpark, and Python-based data pipelines, including orchestration, monitoring, and troubleshooting at scale. - Solid SQL skills and experience working with PostgreSQL in production: schema design, query optimization, migration management, and handling concurrency in large-scale environments.
- Deep, hands-on experience with Apache Airflow, AWS Glue, PySpark, and Python-based data pipelines, including orchestration, monitoring, and troubleshooting at scale. - Solid SQL skills and experience working with PostgreSQL in production: schema design, query optimization, migration management, and handling concurrency in large-scale environments. - Strong understanding of cloud-native data and service workflows (AWS preferred), including data warehousing, storage, security, and cost-efficient architectures.
- Strong understanding of cloud-native data and service workflows (AWS preferred), including data warehousing, storage, security, and cost-efficient architectures. - Fluency in TypeScript and experience with a backend framework such as NestJS (or other Node.js frameworks), including designing decoupled services and robust enterprise interfaces; GraphQL experience is a significant plus. - Experience implementing observability for data and backend systems: logging, metrics, tracing, data validation, and automated alerts for pipeline and service health.
- Solid SQL skills and experience working with PostgreSQL in production: schema design, query optimization, migration management, and handling concurrency in large-scale environments. - Strong understanding of cloud-native data and service workflows (AWS preferred), including data warehousing, storage, security, and cost-efficient architectures. - Fluency in TypeScript and experience with a backend framework such as NestJS (or other Node.js frameworks), including designing decoupled services and robust enterprise interfaces; GraphQL experience is a significant plus.
- Partner closely with the core engineering team to shape ReadyOn’s Integration Platform, ensuring external systems (HCM, WFM, payroll, timekeeping, and other enterprise tools) integrate cleanly and are observable end to end in ReadyOn dashboards. - Model data structures and implement efficient, scalable transformations in Snowflake and PostgreSQL, including schema design, indexing, partitioning, and query optimization for high-volume, low-latency use cases. - Build reusable frameworks, connectors, and internal libraries that standardize how data is published, discovered, and consumed by backend services, analytics, and AI workloads.
Complete your application on jobs.gem.com. The employer’s form will show what is required.
Already applied? Track this application
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
San Francisco
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.