Back to jobs

Data Platform Engineer, Ingestion

New York City · New York City, New York, United States

Live, first seen by us 24 days ago
Pay
$170,000–236,000/year · BaseAnnual period assumed — pay source
COMPENSATION $170,000–$236,000 base salary, depending on level. Meaningful equity. Medical, dental and vision. HOW TO APPLY
Read the full posting
Work setting
On-site — work setting source
Location type: On-site
From the employer’s posting
Employment
Full-time — employment source
Employment type: Full-time
From the employer’s posting
Team
Research & Development › Engineering — team source
Department: Research & Development › Engineering
From the employer’s posting

What you’ll work on

Full posting
  • Build pipelines for messy APIs, files, database extracts and approved scrapers

  • Make incremental loads, backfills and retries safe from duplicates or corruption

  • Automate new-source onboarding from configuration to first delivery

  • Build reusable connectors and storage components other teams can use

  • Carry source metadata through pipelines and enforce privacy controls

From the employer’s posting
Build pipelines for messy sources.
Make reruns safe.
Automate onboarding.
Build modules other teams use.
Carry source metadata through the pipeline

Before you apply

Sponsorship
Visa sponsorship not confirmed — sponsorship source
Will you now or in the future require visa sponsorship to work in the United States?
From the employer’s application form

What you’ll bring

All qualifications

Core experience

  • Three or more years building and supporting production data pipelines
  • Experience bringing in data from many vendors and source types at scale
  • Practical judgment on incremental processing, backfills, failures and cost
  • Strong Python and SQL
  • Based in New York City or willing to relocate
Qualification wording
Typically three or more years building and supporting production data pipelines, or equivalent ownership.
You've brought in data from many vendors and source types, including messy data at scale
Practical judgment about incremental processing, backfills, failures and cost.
Strong Python and SQL.
You're in New York City or willing to relocate.

About Minerva

Minerva builds AI for marketing leaders.

In the employer’s words · Read in context · Company website ↗

Apply at Minerva

Posting record

Report an error
Status in our records
Live
First seen by us
Oct 24, 202624 days ago
Recorded sightings
26
Last seen by us
Nov 17, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Pay

Higher than most similar jobs

$170K–$236K against $127K–$190K, the middle half of what similar US postings state.

From 2,383 postings in the last year. Stated ranges, not salaries paid.

Time since we first saw it

Day 24

Similar jobs usually come down between day 6 and day 56.

From 14,041 similar postings that came down. Coming down is not always a hire.

Minerva is hiring

6 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
  • AWS
  • dbt
  • Iceberg
  • Snowflake
  • Trino
  • Airflow
  • Dagster
Where the posting names them
- Storage: Snowflake, plus a lakehouse on Spark, Iceberg, Trino and AWS Glue. - Code: Python on AWS. WHAT YOU HAVE
- You've automated repeated work or built components other engineers actually used. - Strong Python and SQL. - You pin down what a source owner actually needs, and you write down what you will and won't support.
- Transformation: dbt and SQLMesh. - Storage: Snowflake, plus a lakehouse on Spark, Iceberg, Trino and AWS Glue. - Code: Python on AWS.
- Orchestration: Dagster, with Airflow still running the workloads that haven't moved. - Transformation: dbt and SQLMesh. - Storage: Snowflake, plus a lakehouse on Spark, Iceberg, Trino and AWS Glue.
- Consumer, identity or contact data, and the privacy rules that come with it. - Dagster, Iceberg or Snowflake in production. YOU MIGHT BE A FIT IF YOU ARE…
OUR STACK - Orchestration: Dagster, with Airflow still running the workloads that haven't moved. - Transformation: dbt and SQLMesh.

Your next step

Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.

Applying through Ashby

  • No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.

Already applied? Track this application

Job description

View original posting ↗

ABOUT MINERVA

Minerva builds AI for marketing leaders. We work together with marketers to automate and accelerate their workflows with AI agents. Our agents augment the end-to-end workflow of the marketer by automating analysis, campaign planning, activation, measurement and reporting.

Customers like the NBA, Ramp, Wander, Hard Rock Stadium Group and Luxury Presence use Minerva to develop a better understanding of their customers and introduce AI specifically built to understand the customer into their commercial organizations. Our campaigns outperform other tools because we built the entire stack: from identity to enrichment to data integration as well as agents to action and report on the analysis.

We've raised $20M from The General Partnership, 8VC, Lingotto, NBA Investments, Topology Ventures, Future Positive, Background Capital and many others. Our team brings together operators and investors from Citadel, Dentsu, Bridgewater, Meta Superintelligence, Growthloop, Amazon and Lazard, alongside engineers and researchers from Berkeley, MIT, Stanford and Cambridge.

THE ROLE

Every new data source starts as someone else's problem: an API with rate limits, a vendor file that arrives late and gets corrected twice, a schema that changes without warning. Today, adding a new vendor feed still takes hand-built work.

You make new sources dependable and make the next one cheap. You build the ingestion pipelines for our customer and vendor data, automate onboarding from configuration to first delivery, and turn what you build into modules other Minerva teams can use without running platform infrastructure themselves.

This is a mid-level individual contributor seat on the data platform team, reporting to our Head of Data Platform. Our data product team decides which sources matter and what good data looks like. You own how it arrives and how it stays dependable.

WHAT YOU'LL DO

  • Build pipelines for messy sources. APIs, files, database extracts and approved scraping sources, at high volume, with inconsistent formats and schemas that change.

  • Make reruns safe. Incremental loads, backfills and retries that never duplicate or corrupt data. Bad records get quarantined, not loaded. Throughput and processing cost stay under control.

  • Automate onboarding. The path from source configuration and access checks through provisioning, validation, deployment and first delivery runs without hand-holding.

  • Build modules other teams use. Reusable connectors and storage components with tests, examples and clear interfaces, so a team building an API integration or a scraper gets managed ingestion, storage and monitoring without building them.

  • Keep the origin of every value. Carry source metadata through the pipeline (which vendor, which delivery, when) so we can always say where a value came from. Enforce privacy controls so restricted data can't come back through a new load or a replay.

  • Own your integrations in production. Connect each one to shared monitoring, write its recovery runbook and fix source-specific problems when they break.

OUR STACK

  • Orchestration: Dagster, with Airflow still running the workloads that haven't moved.

  • Transformation: dbt and SQLMesh.

  • Storage: Snowflake, plus a lakehouse on Spark, Iceberg, Trino and AWS Glue.

  • Code: Python on AWS.

WHAT YOU HAVE

  • Typically three or more years building and supporting production data pipelines, or equivalent ownership.

  • You've brought in data from many vendors and source types, including messy data at scale, and kept it running after launch.

  • Practical judgment about incremental processing, backfills, failures and cost. You can show the data stayed correct as volume grew.

  • You've automated repeated work or built components other engineers actually used.

  • Strong Python and SQL.

  • You pin down what a source owner actually needs, and you write down what you will and won't support.

  • You're in New York City or willing to relocate.

NICE TO HAVES

  • Connector frameworks, whether off the shelf or your own.

  • Onboarding automation or self-service data tooling.

  • Consumer, identity or contact data, and the privacy rules that come with it.

  • Dagster, Iceberg or Snowflake in production.

YOU MIGHT BE A FIT IF YOU ARE…

  • A data engineer at a data vendor or identity company who has onboarded source after source and wants to build the framework instead of the next one-off.

  • An engineer at a connector, integration or reverse-ETL company who has built connectors other people depend on.

  • A data engineer at a fintech, marketplace or adtech company who became the person every new integration ran through.

  • Or you don't fit any of these neatly, but you've taken data you didn't control and made it dependable. Apply.

WHAT SUCCESS LOOKS LIKE

  • First 90 days: a new source is live through automated onboarding you built, and another engineer has onboarded a different source with the same modules. You've measured the setup effort and tested recovery.

  • First 6 months: teams outside data platform bring in new sources through your modules, and every feed you own has alerting and a recovery runbook a second engineer has used.

  • The numbers you own: time to onboard a new source, and how often a feed needs hands-on recovery.

COMPENSATION

$170,000–$236,000 base salary, depending on level. Meaningful equity. Medical, dental and vision.

HOW TO APPLY

Skip the cover letter. Walk us through one pipeline you built for data you didn't control: the source, what broke, how you made reruns safe and how the next source got easier. A few paragraphs. Applications without it won't be reviewed.

WHERE WE WORK

We work in person from our office at 300 Kent Avenue in Williamsburg, Brooklyn.

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 14% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 861, the most 1,353, and the latest week had 1,022.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay
COMPENSATION $170,000–$236,000 base salary, depending on level. Meaningful equity. Medical, dental and vision. HOW TO APPLY
Location & working pattern

New York City, New York, United States

Working pattern and location restrictions need checking in the full posting.

Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.