3+ years of systems programming (Rust, C++, C, or similar)
Experience with data processing systems such as YT, Kafka, or Hadoop
Deep understanding of multithreading and concurrency trade-offs
Solid understanding of Linux internals
Ownership of the full project lifecycle, including deployment and scaling in production
Qualification wording
3+ years of hands-on experience in systems programming (Rust, C++, C, or similar)
Knowledge of Data processing systems (YT/Kafka/Hadoop)
Deep understanding of multithreading, including various approaches, challenges, and trade-offs
Solid understanding of Linux internals (syscalls, networking stack, memory model, kernel tuning)
Ownership of full project lifecycle you don't just write a fast inner loop, you care about how the system is built, deployed, operated, and scaled in production
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
C
C++
Python
Rust
AWS
Hadoop
Kafka
Kubernetes
Where the posting names them
Qualifications
- 3+ years of hands-on experience in systems programming (Rust, C++, C, or similar)
- Ownership of full project lifecycle you don't just write a fast inner loop, you care about how the system is built, deployed, operated, and scaled in production
- Ownership of full project lifecycle you don't just write a fast inner loop, you care about how the system is built, deployed, operated, and scaled in production
- Knowledge of Python or other scripting languages
- Knowledge of Data processing systems (YT/Kafka/Hadoop)
- Experience with trading systems or other latency-sensitive real-time systems
- Familiarity with cloud services, Kubernetes, and AWS infrastructure
- Knowledge of Python or other scripting languages
- Knowledge of Data processing systems (YT/Kafka/Hadoop)
- Passion for writing clean, efficient, and scalable systems-level code
Perplexity is looking for a highly skilled Expert Systems Engineer to join our Search Core team. For the indexing and data processing stream. This role is critical to building next-generation search products and technologies.
Responsibilities
Design and build core search engine components, especially indexing pipelines, data storage formats that operate at the scale of billions of pages
Develop streaming and batch data processing with YT systems for search index construction in a high-load environment
Push the limits of hardware performance through low-level optimizations and systems-level tuning
Tackle challenges in multithreading, concurrency, and system-level optimization
Design and build core search engine components, including indexing pipelines, retrieval algorithms, and ranking systems that operate at the scale of billions of pages
Develop streaming and batch data processing systems for search index construction in a high-load environment
Push the limits of hardware performance through low-level optimizations and systems-level tuning
Tackle challenges in multithreading, concurrency, and system-level optimization
Qualifications
3+ years of hands-on experience in systems programming (Rust, C++, C, or similar)
Ownership of full project lifecycle you don't just write a fast inner loop, you care about how the system is built, deployed, operated, and scaled in production
Knowledge of Python or other scripting languages
Knowledge of Data processing systems (YT/Kafka/Hadoop)
Passion for writing clean, efficient, and scalable systems-level code
Strong knowledge of algorithms and data structures, and the ability to apply them effectively
Deep understanding of multithreading, including various approaches, challenges, and trade-offs
Experience building high-load, distributed, and hardware-adjacent services
Solid understanding of Linux internals (syscalls, networking stack, memory model, kernel tuning)
Experience developing core components of search engines, databases, or information retrieval systems
Understanding of search fundamentals: indexing, query parsing, ranking, and relevance
Experience with trading systems or other latency-sensitive real-time systems
Familiarity with cloud services, Kubernetes, and AWS infrastructure
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.