Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
Python
SQL
AWS
Databricks
Delta Lake
Docker
MLflow
S3
Spark
Terraform
PySpark
scikit-learn
SciPy
Great Expectations
Where the posting names them
- Strong expertise in medallion architecture and event-driven asynchronous pipeline design
- Advanced proficiency in Python, PySpark, and SQL
- Deep hands-on experience with Databricks, including Unity Catalog, Lakeflow Jobs/Workflows, and job optimization
- Deep hands-on experience with Databricks, including Unity Catalog, Lakeflow Jobs/Workflows, and job optimization
- Strong AWS expertise including S3, IAM, KMS, and services such as AWS Batch, ECR, EventBridge, Lambda, or API Gateway
- Experience with infrastructure as code and CI/CD using Terraform and Databricks Asset Bundles
WILL BE A PLUS
- Experience packaging workloads in Docker containers and running batch jobs on AWS
- Familiarity with experiment optimization tools such as scikit-optimize, BoTorch, Ax, and scipy
Join an ambitious deep-tech initiative shaping the future of clean energy. We are looking for a Senior Data Engineer to design and scale cloud-native data platforms for a pioneering fusion energy company working on the world’s first commercial laser fusion power plant.
In this role, you will work on greenfield data engineering initiatives, build near-real-time pipelines, and modernize large-scale data infrastructure using Databricks and AWS technologies.
We at Sigma Software create opportunities to work on innovative global projects where technology directly impacts the future of science and sustainability. This role offers the chance to contribute to breakthrough research while driving engineering excellence, automation, and AI adoption within the data platform ecosystem.
- 5+ years of experience in data engineering
- 3+ years of production experience with Databricks and Delta Lake
- Strong expertise in medallion architecture and event-driven asynchronous pipeline design
- Advanced proficiency in Python, PySpark, and SQL
- Deep hands-on experience with Databricks, including Unity Catalog, Lakeflow Jobs/Workflows, and job optimization
- Strong AWS expertise including S3, IAM, KMS, and services such as AWS Batch, ECR, EventBridge, Lambda, or API Gateway
- Strong AWS expertise including S3, IAM, KMS, and services such as AWS Batch, ECR, EventBridge, Lambda, or API Gateway
- Experience with infrastructure as code and CI/CD using Terraform and Databricks Asset Bundles
- Practical experience with Git workflows, pull request reviews, and automated testing
- Strong understanding of data quality, lineage, provenance, schema contracts, and versioning
- Experience with data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks Labs DQX
- Experience with monitoring, observability, and cloud cost optimization
- Experience configuring secure CI pipelines using OIDC or service principals
- Practical MLflow experience including experiment tracking, model registration, and reproducibility practices
- Strong understanding of data quality, lineage, provenance, schema contracts, and versioning
- Define and enforce data platform standards including Data Lake/Lakehouse principles, medallion architecture, and data contracts
- Refactor, optimize, and modernize Spark and PySpark scripts for performance and maintainability
- Introduce best practices for code quality, testing, and CI/CD across data pipelines
- Experience packaging workloads in Docker containers and running batch jobs on AWS
- Familiarity with experiment optimization tools such as scikit-optimize, BoTorch, Ax, and scipy
- Experience building feature sets and train/test splits that avoid data leakage
Design and build scalable cloud-native data platforms from greenfield to production
Design and implement near-real-time ingestion pipelines using event-driven patterns
Define and enforce data platform standards including Data Lake/Lakehouse principles, medallion architecture, and data contracts
Refactor, optimize, and modernize Spark and PySpark scripts for performance and maintainability
Introduce best practices for code quality, testing, and CI/CD across data pipelines
Drive adoption of AI tooling and agentic workflows within the data engineering team
Ensure data quality, observability, and reliability across pipelines and platforms
Design and deliver self-service tooling and microservices that abstract platform complexity
Collaborate directly with the Customer’s scientific and engineering teams
Monitor platform performance, reliability, and cloud infrastructure costs
Qualifications
5+ years of experience in data engineering
3+ years of production experience with Databricks and Delta Lake
Strong expertise in medallion architecture and event-driven asynchronous pipeline design
Advanced proficiency in Python, PySpark, and SQL
Deep hands-on experience with Databricks, including Unity Catalog, Lakeflow Jobs/Workflows, and job optimization
Strong AWS expertise including S3, IAM, KMS, and services such as AWS Batch, ECR, EventBridge, Lambda, or API Gateway
Experience with infrastructure as code and CI/CD using Terraform and Databricks Asset Bundles
Practical experience with Git workflows, pull request reviews, and automated testing
Experience configuring secure CI pipelines using OIDC or service principals
Practical MLflow experience including experiment tracking, model registration, and reproducibility practices
Strong understanding of data quality, lineage, provenance, schema contracts, and versioning
Experience with data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks Labs DQX
Experience with monitoring, observability, and cloud cost optimization
Upper-Intermediate English level or higher
WILL BE A PLUS
Experience packaging workloads in Docker containers and running batch jobs on AWS
Familiarity with experiment optimization tools such as scikit-optimize, BoTorch, Ax, and scipy
Experience building feature sets and train/test splits that avoid data leakage
Experience with regulated or high-security environments including data residency requirements
Experience disabling or restricting AI features in secured environments
Additional Information
PERSONAL PROFILE
Strong analytical mindset
Ownership and accountability
Ability to communicate complex technical concepts clearly
Proactive approach to problem-solving
Attention to scalability and maintainability
Collaborative and team-oriented attitude
Company Description
Join an ambitious deep-tech initiative shaping the future of clean energy. We are looking for a Senior Data Engineer to design and scale cloud-native data platforms for a pioneering fusion energy company working on the world’s first commercial laser fusion power plant.
In this role, you will work on greenfield data engineering initiatives, build near-real-time pipelines, and modernize large-scale data infrastructure using Databricks and AWS technologies.
We at Sigma Software create opportunities to work on innovative global projects where technology directly impacts the future of science and sustainability. This role offers the chance to contribute to breakthrough research while driving engineering excellence, automation, and AI adoption within the data platform ecosystem.
CUSTOMER
Our Customer is a German-American deep-tech company building the world’s first commercial laser fusion power plant. The company develops direct-drive laser fusion technology and is commercializing the 2021 breakthrough at Lawrence Livermore, where scientists achieved fusion ignition for the first time.
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.