Pay
Higher than most similar jobs
$196K–$293K against $127K–$190K, the middle half of what similar US postings state.
From 2,383 postings in the last year. Stated ranges, not salaries paid.
San Jose, CA, US, 95128
Live, first seen by us 42 days agoStrong analytical and problem-solving skills with a passion for performance optimization. Compensation: The target salary range for this position is 196,350 - 292,600 USD. The salary offered will be determined by the candidate's location, qualifications, experience, and education and may be outside of this range. Final compensation packages are competitive and in line with industry standards, reflecting a variety of factors, and include a comprehensive benefits package. This may cover Health Insurance, Life Insurance, Retirement or Pension Plans, Paid Time Off, various Leave options, Performance-Based Incentives, employee stock purchase plan, and/or restricted stocks (RSU’s), with all offerings subject to regional variations and governed by local laws, regulations, and company policies. Benefits may vary by country and region, and further details will be provided as part of the recruitment process.Read the full posting
Lead design of performance and functional enhancements to Apache Gluten and Velox
Design, implement, test, and optimize data processing engine components
Influence how the engine interfaces with open-source platforms and engines
Analyze open-source technology roadmaps for opportunities to enhance the engine
Foster best practices in code reviews, testing, CI/CD, and issue resolution
Lead design of functional and performance enhancements to open-source platforms such as Apache Gluten and Velox
Individually design, implement, test, optimize, and maintain components of the data processing engine.
Influence the architecture of how our data processing engine interfaces with open-source platforms and engines.
Analyze technology roadmap of Apache Gluten, Velox, and equivalent platforms and identify opportunities for our engine to enhance technology and product leadership.
Foster best practices in design and code reviews, testing, CI/CD, and issue resolution
What You'll Bring: - 12+ years of relevant experience Bachelor's degree in Computer Science, or a related field OR a Master's degree in Computer Science. - 3+ years of deep technical experience in developing core components of Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion, Apache DataFusion Comet, or equivalent platforms designed for large-scale data processing.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Higher than most similar jobs
$196K–$293K against $127K–$190K, the middle half of what similar US postings state.
From 2,383 postings in the last year. Stated ranges, not salaries paid.
Day 42
Similar jobs usually come down between day 6 and day 56.
Well past NetApp’s usual: its jobs come down by about day 21.
From 14,040 similar postings that came down, and 36 of NetApp’s. Coming down is not always a hire.
5 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
- Demonstrated experience in the design, development, and successful release of high-performance data processing engines for large production deployments. - Exceptional programming skills in C, C++, and Java. - Extensive development experience in Linux environments.
Job Summary DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for specialists to join our engineering team and shape the future of accelerated data processing. DataPelago Nucleus is a universal data processing engine that is designed to accelerate the processing of diverse data – structured through unstructured – with any parallel processing framework – e.g., Apache Spark – on any infrastructure – including vectorized CPU and GPU. DataPelago Accelerator for Spark (DPA-S), our product based on DataPelago Nucleus, is running large-scale production applications of many globally renowned customers.
DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for specialists to join our engineering team and shape the future of accelerated data processing. DataPelago Nucleus is a universal data processing engine that is designed to accelerate the processing of diverse data – structured through unstructured – with any parallel processing framework – e.g., Apache Spark – on any infrastructure – including vectorized CPU and GPU. DataPelago Accelerator for Spark (DPA-S), our product based on DataPelago Nucleus, is running large-scale production applications of many globally renowned customers. The Opportunity:
The Opportunity: As a Senior/Principal Data Processing Engineer - OSS, you will be a key individual contributor in adopting and advancing capabilities of open-source software (OSS) platforms such as Apache Gluten, Velox, Apache Spark, and Apache Flink in the context of DataPelago’s data processing engine. You will enhance functional breadth, performance, scale, and reliability of the DataPelago engine through downstream and upstream contributions. You will have the opportunity to engage with community working on these platforms. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers. What You'll Do:
- 12+ years of relevant experience Bachelor's degree in Computer Science, or a related field OR a Master's degree in Computer Science. - 3+ years of deep technical experience in developing core components of Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion, Apache DataFusion Comet, or equivalent platforms designed for large-scale data processing. - 3+ years of deep technical experience in instrumenting, analyzing, and optimizing the performance of data processing engine components on benchmark and customer workloads.
- 3+ years of deep technical experience in instrumenting, analyzing, and optimizing the performance of data processing engine components on benchmark and customer workloads. - Sound knowledge of the architecture and internal operation of one or more of Apache Spark, Apache Flink, Presto/Trino. - Demonstrated experience in the design, development, and successful release of high-performance data processing engines for large production deployments.
Complete your application on jobs.netapp.com. The employer’s form will show what is required.
Already applied? Track this application
DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for specialists to join our engineering team and shape the future of accelerated data processing.
DataPelago Nucleus is a universal data processing engine that is designed to accelerate the processing of diverse data – structured through unstructured – with any parallel processing framework – e.g., Apache Spark – on any infrastructure – including vectorized CPU and GPU. DataPelago Accelerator for Spark (DPA-S), our product based on DataPelago Nucleus, is running large-scale production applications of many globally renowned customers.
The Opportunity:
As a Senior/Principal Data Processing Engineer - OSS, you will be a key individual contributor in adopting and advancing capabilities of open-source software (OSS) platforms such as Apache Gluten, Velox, Apache Spark, and Apache Flink in the context of DataPelago’s data processing engine. You will enhance functional breadth, performance, scale, and reliability of the DataPelago engine through downstream and upstream contributions. You will have the opportunity to engage with community working on these platforms. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.
Compensation:
The target salary range for this position is 196,350 - 292,600 USD. The salary offered will be determined by the candidate's location, qualifications, experience, and education and may be outside of this range. Final compensation packages are competitive and in line with industry standards, reflecting a variety of factors, and include a comprehensive benefits package. This may cover Health Insurance, Life Insurance, Retirement or Pension Plans, Paid Time Off, various Leave options, Performance-Based Incentives, employee stock purchase plan, and/or restricted stocks (RSU’s), with all offerings subject to regional variations and governed by local laws, regulations, and company policies. Benefits may vary by country and region, and further details will be provided as part of the recruitment process.
New Data Engineer postings at the employers we follow are up 14% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
Strong analytical and problem-solving skills with a passion for performance optimization. Compensation: The target salary range for this position is 196,350 - 292,600 USD. The salary offered will be determined by the candidate's location, qualifications, experience, and education and may be outside of this range. Final compensation packages are competitive and in line with industry standards, reflecting a variety of factors, and include a comprehensive benefits package. This may cover Health Insurance, Life Insurance, Retirement or Pension Plans, Paid Time Off, various Leave options, Performance-Based Incentives, employee stock purchase plan, and/or restricted stocks (RSU’s), with all offerings subject to regional variations and governed by local laws, regulations, and company policies. Benefits may vary by country and region, and further details will be provided as part of the recruitment process.
San Jose, CA, US, 95128
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.