Back to jobs

Data Engineer

Pay
Pay not listed
Work setting
Hybrid — work setting source
Details Location: Melbourne, VIC (or Hybrid/On-Site across Australia). Type: Full time, permanent.
Location type: Hybrid
From the employer’s posting
Employment
Full-time — employment source
Employment type: Full-time
From the employer’s posting
Team
Data — team source

What you’ll work on

Full posting
  • Build, run and scale crawlers and scrapers for hard-to-reach country-specific sources

  • Develop pipelines for extraction, filtering, cleaning, deduplication and formatting

  • Build agentic pipelines where AI agents generate, annotate, critique and verify data

  • Turn market requirements into dataset specifications, data mixtures and evaluations

  • Track provenance, licensing, privacy and security for every dataset shipped

From the employer’s posting
Build, run, and scale crawlers and scrapers for web scale collection
Develop robust pipelines for extraction, filtering, cleaning, deduplication, and formatting
Build agentic pipelines where AI agents generate, annotate, critique, and verify data at scale
Turn a market’s requirements, from its laws and regulators to its cultural norms and governance expectations, into dataset specifications, data mixtures, and evaluations we can measure against
Track provenance, licensing, privacy, and security for every dataset you ship

Before you apply

Work authorization
Must already have the right to work in AustraliaMust have the right to work in Australia

About Maincode

Maincode makes Matilda, an Australian-made AI stack we own end to end, from tin to token.

In the employer’s words · Read in context · Company website ↗

Apply at Maincode

Posting record

Report an error
Status in our records
Live
First seen by us
Sep 30, 202652 days ago
Recorded sightings
47
Last seen by us
Nov 21, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 52

Similar jobs usually come down between day 6 and day 56.

From 14,771 similar postings that came down. Coming down is not always a hire.

Maincode is hiring

1 open job

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • C++
  • JavaScript
  • Python
  • Rust
  • Spark
  • Go
Where the posting names them
- A track record of building, scaling, and running large scale web crawling or scraping systems in production. - Strong engineering in Python and at least one of Go, Rust, or C++, with real experience in distributed data processing (Spark, Ray, Beam, or similar) and petabyte scale storage. - Hands on knowledge of the hard parts of crawling: JavaScript heavy sites, headless browsers, rate limiting, robots.txt, and access and licensing constraints.
- Strong engineering in Python and at least one of Go, Rust, or C++, with real experience in distributed data processing (Spark, Ray, Beam, or similar) and petabyte scale storage. - Hands on knowledge of the hard parts of crawling: JavaScript heavy sites, headless browsers, rate limiting, robots.txt, and access and licensing constraints. - Experience using LLMs and AI agents inside data pipelines for classification, extraction, synthetic data generation, or quality scoring, and a clear sense of where they fail.

Your next step

Complete your application on jobs.ashbyhq.com. The employer’s form will show what is required.

Applying through Ashby

  • No account needed. “Autofill from resume” at the top of the form fills the key fields from your file.

Already applied? Track this application

Job description

View original posting ↗

About Maincode

Maincode makes Matilda, an Australian-made AI stack we own end to end, from tin to token. We own and operate our own infrastructure here in Australia, we train and maintain our own frontier models, and we run multiple surfaces on top of them, including chat, coding and API. Everything we ship is built on capability we control ourselves. We are hiring someone who cares about that. You should be passionate about Australian-made technology, a builder first, and keen to execute at the frontier of data engineering and model training.

The role

A frontier model is only as capable as the data it learns from, and every model carries the worldview of the data it was trained on. We did this for Australia with Matilda’s voice. Now we are building the capability to do it for any country or market: training open weights models on custom datasets that reflect the cultural, regulatory, governance, and ideological requirements of the people who will use them.

You will be the hands on engineer who builds the system that makes those datasets. You will own the pipeline end to end, from web scale crawling of hard to reach, country specific sources through to filtering, deduplication, quality control, and the agent driven pipelines that turn raw material into training sets. We need someone who moves fast, writes code every day, and knows how to use AI agents to multiply what a small team can produce without letting quality slip.

What you will do

  • Build, run, and scale crawlers and scrapers for web scale collection, including specialized crawlers for high value, hard to reach sources such as legislation, regulator guidance, government publications, local media, and community forums.

  • Develop robust pipelines for extraction, filtering, cleaning, deduplication, and formatting to ensure the highest quality inputs for model training.

  • Build agentic pipelines where AI agents generate, annotate, critique, and verify data at scale, with the evaluation and spot checks needed to trust what they produce.

  • Turn a market’s requirements, from its laws and regulators to its cultural norms and governance expectations, into dataset specifications, data mixtures, and evaluations we can measure against.

  • Work directly with our core training team to iterate on data mixtures, evaluate data quality, and respond to model performance feedback.

  • Track provenance, licensing, privacy, and security for every dataset you ship.

What you will bring

  • A track record of building, scaling, and running large scale web crawling or scraping systems in production.

  • Strong engineering in Python and at least one of Go, Rust, or C++, with real experience in distributed data processing (Spark, Ray, Beam, or similar) and petabyte scale storage.

  • Hands on knowledge of the hard parts of crawling: JavaScript heavy sites, headless browsers, rate limiting, robots.txt, and access and licensing constraints.

  • Experience using LLMs and AI agents inside data pipelines for classification, extraction, synthetic data generation, or quality scoring, and a clear sense of where they fail.

  • A bias for shipping. You get a working pipeline running in days, then make it reliable.

  • Experience leading the technical direction of a project and bringing other engineers along with you.

Highly regarded

  • Direct experience building pre-training or post-training datasets for large language models, including SFT, preference, or reinforcement learning data.

  • Experience with multilingual, regional, or culturally specific data, or with evaluating models against local norms and regulation.

  • Experience fine-tuning or post-training open weights models.

  • Open source contributions to crawling, data processing, or dataset tooling.

  • A strong network within the Australian technology, academic, or media ecosystem to facilitate unique data partnerships.

Eligibility

  • Must have the right to work in Australia.

  • Willingness to undergo background checks as required.

Details

  • Location: Melbourne, VIC (or Hybrid/On-Site across Australia).

  • Type: Full time, permanent.

  • Remuneration: Negotiable based on experience, plus super and equity.

Who thrives here

Engineers who understand that high quality data is the true moat of any frontier model, and that it is also where a model’s values come from. If you care about Australian-made technology and you would rather spend your week shipping crawlers, tuning agent pipelines, and reading samples of your own data than sitting in steering committee meetings, you will do well here.

 

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 845, the most 1,344, and the latest week had 1,007.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Australia

Details - Location: Melbourne, VIC (or Hybrid/On-Site across Australia). - Type: Full time, permanent.
Work authorization
Eligibility - Must have the right to work in Australia. - Willingness to undergo background checks as required.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.