We are seeking a highly skilled Software Engineering Lead – Data Platform to lead the development of Caterpillar’s next-generation Digital Manufacturing Data Platform.
The ideal candidate will bring deep expertise in data engineering, large-scale data ingestion, AWS-based architectures, and Snowflake, along with strong leadership and software engineering discipline.
Lead and mentor a team of data engineers and platform developers
Design scalable ingestion pipelines for batch and near real-time data
Build and optimize Snowflake-based data platforms for performance and cost
Architect scalable and secure data platforms on AWS
Implement CI/CD pipelines using Azure DevOps / Jenkins
From the employer’s posting
We are seeking a highly skilled Software Engineering Lead – Data Platform to lead the development of Caterpillar’s next-generation Digital Manufacturing Data Platform. This platform enables large-scale data ingestion, transformation, and analytics across manufacturing, supply chain, and engineering ecosystems.
The ideal candidate will bring deep expertise in data engineering, large-scale data ingestion, AWS-based architectures, and Snowflake, along with strong leadership and software engineering discipline.
Lead and mentor a team of data engineers and platform developers
Design robust data ingestion frameworks for batch and near real-time pipelines
Build and optimize Snowflake-based data platforms for performance and cost
Architect scalable and secure data platforms on AWS
Implement CI/CD pipelines using Azure DevOps / Jenkins
Bachelor’s or Master’s degree in Computer Science / Engineering
Strong leadership, communication, and stakeholder management skills
Qualification wording
Bachelor’s or Master’s degree in Computer Science / Engineering
Strong leadership, communication, and stakeholder management skills
Education & alternatives
Qualifications
- Bachelor’s or Master’s degree in Computer Science / Engineering
- Proven track record in leading engineering teams and delivering at scale
Open 3 monthsWe first saw this posting 99 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
Status in our records
Live
First seen by us
Aug 11, 202699 days ago
Recorded sightings
67
Last seen by us
Nov 15, 2026
Aug 11, 2026today
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
How this job compares
Time since we first saw it
Day 99
656
Up longer than most. Similar jobs usually come down between day 6 and day 56.
Well past Caterpillar’s usual: its jobs come down by about day 12.
From 13,984 similar postings that came down, and 218 of Caterpillar’s. Coming down is not always a hire.
Caterpillar is hiring
41 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
Python
SQL
AWS
Azure
Kafka
Neo4j
Redshift
S3
Snowflake
Spark
Where the posting names them
- Deep expertise in Snowflake (architecture, optimization, data modeling)
- Strong programming skills in Python and SQL
- Extensive experience with data ingestion pipelines and ETL/ELT frameworks
We are seeking a highly skilled Software Engineering Lead – Data Platform to lead the development of Caterpillar’s next-generation Digital Manufacturing Data Platform. This platform enables large-scale data ingestion, transformation, and analytics across manufacturing, supply chain, and engineering ecosystems.
The ideal candidate will bring deep expertise in data engineering, large-scale data ingestion, AWS-based architectures, and Snowflake, along with strong leadership and software engineering discipline.
Key Responsibilities
- Data Platform & Architecture
- Architect scalable and secure data platforms on AWS
- Design robust data ingestion frameworks for batch and near real-time pipelines
- 10+ years of experience in Data Engineering / Data Platform roles
- Strong experience in AWS data ecosystem (S3, Glue, Lambda, EMR, Redshift)
- Deep expertise in Snowflake (architecture, optimization, data modeling)
- Exposure to real-time streaming (Kafka, Spark Streaming)
- Experience with CI/CD tools (GitHub, Jenkins, AWS CloudFormation etc.)
- Solid understanding of distributed systems and scalable architectures
- Leverage AWS services (S3, Glue, Lambda, EMR, Redshift, etc.) for end-to-end pipelines
- Implement CI/CD pipelines using Azure DevOps / Jenkins
- Ensure system scalability, resiliency, and operational readiness
- Extensive experience with data ingestion pipelines and ETL/ELT frameworks
- Exposure to real-time streaming (Kafka, Spark Streaming)
- Experience with CI/CD tools (GitHub, Jenkins, AWS CloudFormation etc.)
Nice-to-Have Skills
- Experience with Graph Databases (Neo4j, Neptune)
- Experience with Vector Databases (Milvus, OpenSearch)
- Lead design and development of scalable ingestion pipelines (structured and unstructured data)
- Build and optimize Snowflake-based data platforms for performance and cost
- Enable ingestion of diverse sources (databases, APIs, files, streaming data)
- Strong experience in AWS data ecosystem (S3, Glue, Lambda, EMR, Redshift)
- Deep expertise in Snowflake (architecture, optimization, data modeling)
- Strong programming skills in Python and SQL
Complete your application on cat.wd5.myworkdayjobs.com. The employer’s form will show what is required.
Applying through Workday
You create an account with each employer before you can upload anything. Every Workday employer has its own account, so one sign-up doesn’t carry over.
“Autofill with Resume” comes right after sign-in, before the My Information and My Experience steps. Check every field it fills in those steps, since that’s what recruiters see.
Your application status lives in that employer’s Candidate Home, under the same sign-in.
When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.
Software Engineering Lead – Data Platform
About the Role
We are seeking a highly skilled Software Engineering Lead – Data Platform to lead the development of Caterpillar’s next-generation Digital Manufacturing Data Platform. This platform enables large-scale data ingestion, transformation, and analytics across manufacturing, supply chain, and engineering ecosystems.
The ideal candidate will bring deep expertise in data engineering, large-scale data ingestion, AWS-based architectures, and Snowflake, along with strong leadership and software engineering discipline.
Key Responsibilities
Leadership & Delivery
Lead and mentor a team of data engineers and platform developers
Drive Agile execution and ensure predictable, high-quality delivery
Establish engineering best practices, code quality, and CI/CD standards
Data Platform & Architecture
Architect scalable and secure data platforms on AWS
Design robust data ingestion frameworks for batch and near real-time pipelines
Define best practices in data modeling, governance, and metadata management
Data Engineering & Ingestion
Lead design and development of scalable ingestion pipelines (structured and unstructured data)
Build and optimize Snowflake-based data platforms for performance and cost
Enable ingestion of diverse sources (databases, APIs, files, streaming data)
About 118,000 in 2025, up 5% from the year before.Annual report (SEC Form 10-K)
Company
Listed as CAT.
H-1B visas
63 approved for new hires in fiscal 2025.US Citizenship and Immigration Services
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 14% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.