Implement robust checkpointing, state recovery, schema evolution, and deduplication to safeguard against silent data corruption and ordering anomalies.
Develop internal developer agents and automation loops (scaffolding, change detection, quality circuit breakers, and alert triage).
From the employer’s posting
Model complex event streams through bronze/silver/gold medallion architectures using modern open table formats (Delta Lake, Apache Iceberg).
Implement robust checkpointing, state recovery, schema evolution, and deduplication to safeguard against silent data corruption and ordering anomalies.
Develop internal developer agents and automation loops (scaffolding, change detection, quality circuit breakers, and alert triage).
Implement robust checkpointing, state recovery, schema evolution, and deduplication to safeguard against silent data corruption and ordering anomalies.
Develop internal developer agents and automation loops (scaffolding, change detection, quality circuit breakers, and alert triage).
Spot platform bottlenecks, architectural debt, or emerging technologies before anyone files a ticket—scope the solution and champion it to production.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
Python
AWS
Delta Lake
Iceberg
Spark
Where the posting names them
- Experience owning production Spark streaming workloads, including orchestration, checkpointing, monitoring, alerting and failure recovery.
- Python you’d put your name on: tested, typed, reviewed and shipped through CI.
- AWS fluency and comfort reading and changing infrastructure as code.
This is a Data Platform Engineering seat. Your mission is to build the tooling, infrastructure, and pipelines that allow engineering and product teams to self-serve while reliably delivering high-integrity data to customer-facing applications, internal reporting, ML models, and autonomous AI agents.
You will join the core team behind our data platform: an event-driven lakehouse on AWS handling tens of thousands of events per second. You will design, build, and operate the near-real-time pipelines that model product and service events through medallion layers using open table formats and analytical engines, delivered as strictly tested, peer-reviewed, and version-controlled software.
We build with agents, for agents: Our platform is itself agentic. We actively develop and maintain skills and AI agents that scaffold new pipelines, monitor upstream repositories and tickets for data-contract changes before they break downstream systems, autonomously triage alerts, and assist with cross-repo code reviews.
Key Responsibilities
- Architect, deploy, and operate production near real-time streaming workloads on AWS from ingestion to analytical serving layers.
- Model complex event streams through bronze/silver/gold medallion architectures using modern open table formats (Delta Lake, Apache Iceberg).
- Python you’d put your name on: tested, typed, reviewed and shipped through CI.
- AWS fluency and comfort reading and changing infrastructure as code.
- Hands-on experience with modern table formats, including upserts, schema evolution and the maintenance they need.
- Architect, deploy, and operate production near real-time streaming workloads on AWS from ingestion to analytical serving layers.
- Model complex event streams through bronze/silver/gold medallion architectures using modern open table formats (Delta Lake, Apache Iceberg).
- Implement robust checkpointing, state recovery, schema evolution, and deduplication to safeguard against silent data corruption and ordering anomalies.
Requirements & Qualifications
- Experience owning production Spark streaming workloads, including orchestration, checkpointing, monitoring, alerting and failure recovery.
- Python you’d put your name on: tested, typed, reviewed and shipped through CI.
Complete your application on job-boards.greenhouse.io. The employer’s form will show what is required.
Applying through Greenhouse
No account needed: the form is one page, with an “Autofill my application” option that reads your resume.
Greenhouse doesn’t read resumes over 2.5 MB, or details placed in a header, footer, text box, table or second column. The file still attaches, but a recruiter has to type you in by hand.
KnowBe4 empowers the modern workforce to make smarter security decisions every day. Trusted by more than 70,000 organizations worldwide, KnowBe4 is the pioneer of digital workforce security, securing both AI agents and humans. The KnowBe4 Platform provides attack simulation and training, collaboration security, and agent security powered by AIDA (Artificial Intelligence Defense Agents) and a proprietary Risk Score. The platform leverages 15-years of behavioral data to combat advanced threats including social engineering, prompt injection, and shadow AI. By securing humans and agents, KnowBe4 leads the industry in workforce trust and defense.
Position open to candidates located in Brazil. Please submit your resume in English.
This is a Data Platform Engineering seat. Your mission is to build the tooling, infrastructure, and pipelines that allow engineering and product teams to self-serve while reliably delivering high-integrity data to customer-facing applications, internal reporting, ML models, and autonomous AI agents.
You will join the core team behind our data platform: an event-driven lakehouse on AWS handling tens of thousands of events per second. You will design, build, and operate the near-real-time pipelines that model product and service events through medallion layers using open table formats and analytical engines, delivered as strictly tested, peer-reviewed, and version-controlled software.
We build with agents, for agents: Our platform is itself agentic. We actively develop and maintain skills and AI agents that scaffold new pipelines, monitor upstream repositories and tickets for data-contract changes before they break downstream systems, autonomously triage alerts, and assist with cross-repo code reviews.
Key Responsibilities
Architect, deploy, and operate production near real-time streaming workloads on AWS from ingestion to analytical serving layers.
Model complex event streams through bronze/silver/gold medallion architectures using modern open table formats (Delta Lake, Apache Iceberg).
Implement robust checkpointing, state recovery, schema evolution, and deduplication to safeguard against silent data corruption and ordering anomalies.
Develop internal developer agents and automation loops (scaffolding, change detection, quality circuit breakers, and alert triage).
Spot platform bottlenecks, architectural debt, or emerging technologies before anyone files a ticket—scope the solution and champion it to production.
Requirements & Qualifications
Experience owning production Spark streaming workloads, including orchestration, checkpointing, monitoring, alerting and failure recovery.
Python you’d put your name on: tested, typed, reviewed and shipped through CI.
AWS fluency and comfort reading and changing infrastructure as code.
Hands-on experience with modern table formats, including upserts, schema evolution and the maintenance they need.
Strong data modelling judgement, including knowing the tradeoffs and limits of a design.
Experience with event-stream or CDC ingestion, and the de-duplication and ordering problems that come with it.
The ability to take a vague stakeholder request, work out the real question, come back with a design, and explain the result and its caveats to people who don’t read code.
The judgement to spot what the platform needs before anyone files a ticket, scope it and make the case for it, including new technologies where they would serve us better.
A track record of raising the bar around you: reviewing, mentoring and setting conventions a team actually follows.
Interest in or experience building, agentic tooling for engineering work itself: scaffolding, change detection, triage and review loops.
Clear verbal and written English communication skills to collaborate effectively across international teams.
Our Fantastic Benefits
We offer company-wide bonuses based on monthly sales targets, employee referral bonuses, adoption assistance, tuition reimbursement, certification reimbursement, and certification completion bonuses - all in a modern, high-tech, and fun work environment. For more details about our benefits in each office location, please visit www.knowbe4.com/careers/benefits.
Note: An applicant assessment and background check may be part of your hiring procedure.
Individuals seeking employment at KnowBe4 are considered without prejudice to race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, sexual orientation or any other characteristic protected under applicable federal, state, or local law. If you require reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employee selection process, please visit www.knowbe4.com/careers/request-accommodation.
No recruitment agencies, please.
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.