Back to jobs

Staff Data Engineer

New York, NY · New York, NY, United States

Live, first seen by us 49 days ago
Pay
Pay not listedSimilar roles state $127K–$190K
Work setting
On-site — work setting source
Location type: On-site
From the employer’s posting
Employment
Full-time — employment source
Employment type: Full-time
From the employer’s posting
Team
Engineering › Sofware Engineering — team source
Department: Engineering › Sofware Engineering
From the employer’s posting

What you’ll work on

Full posting

You'll be based in Twenty's New York City office, building and operating the data integrations that feed Twenty's mission-critical platform.

If you're an experienced data engineer who wants your work in the hands of operators the same week you build it, and you'd rather own a production system end to end than ship features into a backlog, this role is for you.

  • Build PySpark AWS Glue pipelines that retrieve, parse, transform, and load new data sources

  • Design schemas and data models, including analytical models in ClickHouse

  • Monitor pipeline health and build dashboards and alerts in the LGTM stack

  • Diagnose and resolve production data-path incidents and lead root cause analysis

  • Package integrations so field teams can install and operate them in restricted environments

From the employer’s posting
You'll be based in Twenty's New York City office, building and operating the data integrations that feed Twenty's mission-critical platform. Twenty builds software for cyber operations, and the applications this role feeds are used by operators and analysts working against hard national security problems. This is a hands-on engineering role with real production ownership: you'll build the pipelines that retrieve, process, and integrate new data sources into our applications, monitor the health of the data flowing through the systems you can reach, and debug production issues when they surface. Where your work runs in environments you can't access, you'll partner with Twenty's forward deployed engineers, who operate on-site at customer facilities, to get it deployed and keep it healthy. You'll join the data platform team, reporting to its engineering manager. The data integration tooling you'll build on is young and under active development. You'll be among its first production users, and what you learn in operation will directly shape how it evolves.
If you're an experienced data engineer who wants your work in the hands of operators the same week you build it, and you'd rather own a production system end to end than ship features into a backlog, this role is for you.
primarily as AWS Glue jobs written in PySpark
Design schemas and data models that fit the platform's performance characteristics
Monitor the health of data pipelines and platform services using the LGTM stack
Diagnose and resolve production issues in the data path
Package pipelines and integrations destined for restricted environments

Before you apply

Work authorization
US citizens onlyUS Citizenship Required / No Active Clearance Required

What you’ll bring

All qualifications

Core experience

  • Experienced data engineer able to work through ambiguous problems independently
  • U.S. citizenship required
  • Experience with AWS Glue and PySpark
  • Participation in on-call rotation using PagerDuty
Qualification wording
If you're an experienced data engineer
U.S. Citizenship Required
primarily as AWS Glue jobs written in PySpark
Participate in on-call rotation using PagerDuty

About Twenty

America is under sustained cyber attack.

In the employer’s words · Read in context · Company website ↗

Apply at Twenty

Posting record

Report an error
Status in our records
Live
First seen by us
Sep 29, 202649 days ago
Recorded sightings
60
Last seen by us
Nov 17, 2026
Reposted
once since Jun 2, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications. We count a repost when the same title and location comes back as a new listing within 60 days of the last one coming down. Some hiring systems expire listings on a schedule, so repost rates vary by system.

How this job compares

Pay

$127K–$190K is typical

The middle half of what similar US postings state. This one doesn’t list pay.

From 2,383 postings in the last year. Stated ranges, not salaries paid.

Time since we first saw it

Day 49

Similar jobs usually come down between day 6 and day 56.

From 13,974 similar postings that came down. Coming down is not always a hire.

Twenty is hiring

4 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Its jobs tend to stay up a long time
  • 25% of open jobs were reposted
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
  • TypeScript
  • AWS
  • BigQuery
  • ClickHouse
  • Datadog
  • Delta Lake
  • Docker
  • Grafana
  • Iceberg
  • Kafka
  • PostgreSQL
  • Redshift
  • Spark
  • Trino
  • PySpark
  • Go
Where the posting names them
- 5+ years of experience in data engineering, or software engineering with a substantial data infrastructure focus. - Expert Python skills, with a track record of designing and maintaining production-grade codebases that other engineers extend and depend on. - Hands-on experience building and operating ETL/ELT pipelines with Spark (PySpark), ideally as AWS Glue jobs.
- Data integration: Twenty's purpose-built data platform and integration layer (in active development) - Languages in use across engineering: Go, TypeScript, Python Security / Work Environment
- Hands-on experience building and operating ETL/ELT pipelines with Spark (PySpark), ideally as AWS Glue jobs. - Strong SQL and schema design skills, including the ability to analyze access patterns and make sound partitioning and indexing decisions. - Experience with column-oriented analytical databases (example technologies: ClickHouse, Redshift, BigQuery).
Data Source Integration & Pipeline Development - Design and build data pipelines that retrieve, parse, transform, and load new data sources into Twenty's applications, primarily as AWS Glue jobs written in PySpark. - Investigate unfamiliar source systems and data formats, working with Twenty's engineers to understand semantics, access patterns, and constraints before writing code.
- Expert Python skills, with a track record of designing and maintaining production-grade codebases that other engineers extend and depend on. - Hands-on experience building and operating ETL/ELT pipelines with Spark (PySpark), ideally as AWS Glue jobs. - Strong SQL and schema design skills, including the ability to analyze access patterns and make sound partitioning and indexing decisions.
Tech Environment (You Might Work With) - Cloud: AWS (EC2, ECS, RDS, CloudWatch), including deployments into closed enclave environments - Data processing: AWS Glue jobs running PySpark
- Cloud: AWS (EC2, ECS, RDS, CloudWatch), including deployments into closed enclave environments - Data processing: AWS Glue jobs running PySpark - Analytics: ClickHouse, with ClickPipes for ingestion
- Strong SQL and schema design skills, including the ability to analyze access patterns and make sound partitioning and indexing decisions. - Experience with column-oriented analytical databases (example technologies: ClickHouse, Redshift, BigQuery). - Experience debugging production data systems: reading logs, tracing failures across components, and root-causing issues under time pressure.
- Investigate unfamiliar source systems and data formats, working with Twenty's engineers to understand semantics, access patterns, and constraints before writing code. - Design schemas and data models that fit the platform's performance characteristics and the shape of the data, including analytical models in ClickHouse. - Write code that is testable, debuggable, and maintainable by engineers operating it in environments you can't access. Tests carry unusual weight here: they're how the field trusts a change you'll never see run.
- Data processing: AWS Glue jobs running PySpark - Analytics: ClickHouse, with ClickPipes for ingestion - Containers: Docker, orchestrated on Amazon ECS
- Experience with streaming and message queue technologies (example technologies: Kafka, NATS, Kinesis, RabbitMQ). - Hands-on experience with observability tooling (Grafana, Datadog, Splunk, or similar). - Experience deploying and operating software in air-gapped, classified, or otherwise restricted environments.
Nice to Have - Experience with data lake and large-scale query technologies (example technologies: Apache Iceberg, Delta Lake, Trino, Presto, Athena). - Experience with streaming and message queue technologies (example technologies: Kafka, NATS, Kinesis, RabbitMQ).
- Experience deploying and operating software in air-gapped, classified, or otherwise restricted environments. - Familiarity with containerized environments (Docker and container orchestration) and CI/CD concepts. Tech Environment (You Might Work With)
- Analytics: ClickHouse, with ClickPipes for ingestion - Containers: Docker, orchestrated on Amazon ECS - Observability: Grafana, Loki, Tempo, Mimir (LGTM stack)
Production Health & Monitoring - Monitor the health of data pipelines and platform services using the LGTM stack (Grafana, Loki, Tempo, Mimir). - Track data quality, throughput, and freshness; identify anomalies and degradations before they become incidents.
- Containers: Docker, orchestrated on Amazon ECS - Observability: Grafana, Loki, Tempo, Mimir (LGTM stack) - Alerting / on-call: PagerDuty
- Experience with data lake and large-scale query technologies (example technologies: Apache Iceberg, Delta Lake, Trino, Presto, Athena). - Experience with streaming and message queue technologies (example technologies: Kafka, NATS, Kinesis, RabbitMQ). - Hands-on experience with observability tooling (Grafana, Datadog, Splunk, or similar).
- Messaging: NATS - Databases: PostgreSQL - Data integration: Twenty's purpose-built data platform and integration layer (in active development)

Your next step

Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.

Applying through Ashby

  • No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.

Already applied? Track this application

Job description

View original posting ↗

About the Company

America is under sustained cyber attack. Our adversaries infiltrate our networks, steal our IP, and degrade the digital infrastructure that modern life runs on. They’ve learned—correctly—that those attacks rarely produce consequences.

Twenty was founded to change that, by making our adversaries think twice before they attack us. Our vision is American and allied primacy in cyberspace—a future where they cannot contest us, deterrence is assured, and the free world remains secure.

Founded in 2024, Twenty Technologies (www.twenty.io) industrializes offensive cyber operations for the U.S. and its allies. Headquartered in Arlington, Virginia, Twenty has raised $168M from Khosla Ventures, Accel, Caffeinated Capital, Friends & Family Capital, Point72 Ventures, General Catalyst, and In-Q-Tel.

 
 
 
 

Mission | On Site | Full Time | U.S. Citizenship Required / No Active Clearance Required

 
 
 
 
 
 

Role Summary

You'll be based in Twenty's New York City office, building and operating the data integrations that feed Twenty's mission-critical platform. Twenty builds software for cyber operations, and the applications this role feeds are used by operators and analysts working against hard national security problems. This is a hands-on engineering role with real production ownership: you'll build the pipelines that retrieve, process, and integrate new data sources into our applications, monitor the health of the data flowing through the systems you can reach, and debug production issues when they surface. Where your work runs in environments you can't access, you'll partner with Twenty's forward deployed engineers, who operate on-site at customer facilities, to get it deployed and keep it healthy. You'll join the data platform team, reporting to its engineering manager. The data integration tooling you'll build on is young and under active development. You'll be among its first production users, and what you learn in operation will directly shape how it evolves.

 

If you're an experienced data engineer who wants your work in the hands of operators the same week you build it, and you'd rather own a production system end to end than ship features into a backlog, this role is for you.

 

What You'll Do

Data Source Integration & Pipeline Development

  • Design and build data pipelines that retrieve, parse, transform, and load new data sources into Twenty's applications, primarily as AWS Glue jobs written in PySpark.

  • Investigate unfamiliar source systems and data formats, working with Twenty's engineers to understand semantics, access patterns, and constraints before writing code.

  • Design schemas and data models that fit the platform's performance characteristics and the shape of the data, including analytical models in ClickHouse.

  • Write code that is testable, debuggable, and maintainable by engineers operating it in environments you can't access. Tests carry unusual weight here: they're how the field trusts a change you'll never see run.

  • Write integration code that meets the security standards of classified environments: no hardcoded secrets, disciplined credential handling, and audit-ready practices throughout, so your work moves through customer approval processes without friction.

Deployment & Field Support

  • Deploy, operate, and iterate on integrations against the systems reachable from the office, end to end.

  • Package pipelines and integrations destined for restricted environments so forward deployed engineers and customer deployment teams can install, verify, and operate them without you in the room.

  • Support field deployments remotely: reproduce issues, cut fixes, and move them through the pipeline quickly when an on-site engineer is blocked.

  • Operate and improve pipelines built elsewhere on the team, and feed operational learnings back into their design.

Production Health & Monitoring

  • Monitor the health of data pipelines and platform services using the LGTM stack (Grafana, Loki, Tempo, Mimir).

  • Track data quality, throughput, and freshness; identify anomalies and degradations before they become incidents.

  • Build and improve the dashboards and alerts that make pipeline health visible to the team and to the forward deployed engineers who depend on it.

Incident Response & Debugging

  • Diagnose and resolve production issues in the data path: failed ingests, malformed source data, pipeline stalls, performance degradation.

  • Own resolution of incidents within your sphere of responsibility, escalating with a clear picture of what you found, not just what triggered the alert.

  • Drive root cause analysis and post-incident reviews, and turn findings into runbooks, fixes, or upstream design changes.

  • Participate in on-call rotation using PagerDuty, with clear escalation paths across the engineering team.

Field & Platform Collaboration

  • Work closely with forward deployed engineers on-site at customer facilities, sharing context in both directions so field reality and platform design stay connected.

  • Turn operational observations, data source opportunities, and recurring patterns from the field into platform capabilities rather than one-off fixes.

 

Who You Are

  • You independently identify the right solution to ambiguous, open-ended problems. New data sources arrive undocumented and messy; you investigate, form a design, and deliver without waiting for a spec.

  • You respond with urgency to operational issues and own resolution within your sphere. You're unafraid to call an incident when the signal warrants it.

  • You proactively create and update runbooks and documentation for the components you own. Some of the environments your code runs in are ones you'll never touch, and what you write down is what the engineers there have to work with.

  • You make informed decisions by consulting the right stakeholders and balancing detail with the big picture, and you execute against the spirit of the requirement, not just the letter.

  • You actively seek out and eliminate sources of toil. Repetitive manual work is a design problem, and you treat it like one.

  • You understand the customer and the mission well enough to know which work has the greatest impact, and you redirect your focus when what you're doing isn't moving the needle.

  • You tailor your communication to your audience, whether that's a forward deployed engineer at a customer site, a product engineer in Arlington, or a teammate across the desk, and you proactively share information so the right people stay informed and aligned.

  • You're comfortable building for restricted environments where deployment is gated, feedback loops are longer than you'd like, and disciplined engineering practice is the price of admission.

 

Must Have

  • 5+ years of experience in data engineering, or software engineering with a substantial data infrastructure focus.

  • Expert Python skills, with a track record of designing and maintaining production-grade codebases that other engineers extend and depend on.

  • Hands-on experience building and operating ETL/ELT pipelines with Spark (PySpark), ideally as AWS Glue jobs.

  • Strong SQL and schema design skills, including the ability to analyze access patterns and make sound partitioning and indexing decisions.

  • Experience with column-oriented analytical databases (example technologies: ClickHouse, Redshift, BigQuery).

  • Experience debugging production data systems: reading logs, tracing failures across components, and root-causing issues under time pressure.

  • Demonstrated ability to work independently on production systems, including sound judgment about when to decide and when to escalate.

  • Ability to work on-site full-time at Twenty's New York City office, with an initial onboarding period at our Arlington, VA office and occasional travel there afterward.

 

Nice to Have

  • Experience with data lake and large-scale query technologies (example technologies: Apache Iceberg, Delta Lake, Trino, Presto, Athena).

  • Experience with streaming and message queue technologies (example technologies: Kafka, NATS, Kinesis, RabbitMQ).

  • Hands-on experience with observability tooling (Grafana, Datadog, Splunk, or similar).

  • Experience deploying and operating software in air-gapped, classified, or otherwise restricted environments.

  • Familiarity with containerized environments (Docker and container orchestration) and CI/CD concepts.

 

Tech Environment (You Might Work With)

  • Cloud: AWS (EC2, ECS, RDS, CloudWatch), including deployments into closed enclave environments

  • Data processing: AWS Glue jobs running PySpark

  • Analytics: ClickHouse, with ClickPipes for ingestion

  • Containers: Docker, orchestrated on Amazon ECS

  • Observability: Grafana, Loki, Tempo, Mimir (LGTM stack)

  • Alerting / on-call: PagerDuty

  • Messaging: NATS

  • Databases: PostgreSQL

  • Data integration: Twenty's purpose-built data platform and integration layer (in active development)

  • Languages in use across engineering: Go, TypeScript, Python

 

Security / Work Environment

This role requires U.S. citizenship and eligibility to obtain a U.S. Government security clearance; no active clearance is required to start. Work is performed at Twenty's New York City office. Onboarding begins with a stint of roughly two weeks at Twenty's Arlington, VA office, and travel to Arlington after that is occasional and as needed. Some of the systems this role builds for are restricted, and tooling constraints apply when working with them.

Benefits
What's on the table:

  • Health. Medical, dental, and vision plan options. Life / AD&D, disability coverage options.

  • Family. Paid parental leave for eligible full-time employees. 12 weeks for birthing parents, 4 for non-birthing parents, 6 weeks for adoptive, foster, or intended parents through surrogacy.

  • Vacation. Paid holidays and flexible PTO. Take what you need.

  • Retirement. 401(k) with pre-tax and Roth options. HSA/FSA options, dependent care FSA.

Benefits vary by location, role, and eligibility. Full plan details provided during the interview and offer process.


If this role sounds like you, apply and share with us your interest

Due to U.S. government contract and security requirements, this role is limited to U.S. citizens. Some positions may also require eligibility to obtain and maintain a U.S. Government security clearance. Any active clearance requirement will be listed in the role description.

Twenty is an equal opportunity employer. We consider all qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, veteran status, disability, or any other protected status, consistent with applicable law.

If you need a reasonable accommodation during the hiring process, let us know and we will work with you.

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 14% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 861, the most 1,353, and the latest week had 1,022.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

New York, NY, United States

- Package pipelines and integrations destined for restricted environments so forward deployed engineers and customer deployment teams can install, verify, and operate them without you in the room. - Support field deployments remotely: reproduce issues, cut fixes, and move them through the pipeline quickly when an on-site engineer is blocked. - Operate and improve pipelines built elsewhere on the team, and feed operational learnings back into their design.
More source context
Field & Platform Collaboration - Work closely with forward deployed engineers on-site at customer facilities, sharing context in both directions so field reality and platform design stay connected. - Turn operational observations, data source opportunities, and recurring patterns from the field into platform capabilities rather than one-off fixes.

More relevant text appears in the full description.

Work authorization
Founded in 2024, Twenty Technologies (www.twenty.io) industrializes offensive cyber operations for the U.S. and its allies. Headquartered in Arlington, Virginia, Twenty has raised $168M from Khosla Ventures, Accel, Caffeinated Capital, Friends & Family Capital, Point72 Ventures, General Catalyst, and In-Q-Tel. Mission | On Site | Full Time | U.S. Citizenship Required / No Active Clearance Required Role Summary
More source context
Security / Work Environment This role requires U.S. citizenship and eligibility to obtain a U.S. Government security clearance; no active clearance is required to start. Work is performed at Twenty's New York City office. Onboarding begins with a stint of roughly two weeks at Twenty's Arlington, VA office, and travel to Arlington after that is occasional and as needed. Some of the systems this role builds for are restricted, and tooling constraints apply when working with them. Benefits What's on the table:

More relevant text appears in the full description.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.