Staff Big Data Engineer
Pune
- Pay
- Pay not listed
- Work setting
- Not stated
- Employment
Full-time — employment source
Employment type in the employer’s posting: Full time
From the employer’s posting
What you’ll work on
Full postingDefine the technical vision and long-term strategy for the data platform
Design and build scalable data processing systems with Apache Spark and Kafka
Lead architecture decisions across multiple engineering teams
Troubleshoot and resolve complex performance bottlenecks in pipelines and systems
Mentor engineers on distributed systems and big data technologies
From the employer’s posting
Define and drive the technical vision, architecture, and long-term strategy for the data platform and data pipeline ecosystem.
Design, build, and maintain highly scalable, high-availability data processing systems using Apache Spark, Apache Kafka, and distributed data technologies.
Lead architecture and design decisions across multiple engineering teams
Troubleshoot, optimize, and resolve complex performance bottlenecks across data pipelines, distributed systems, and platform infrastructure.
Mentor and guide engineers on distributed systems design, performance optimization, and big data technologies.
What you’ll bring
All qualificationsPreferred experience
- Experience working with high-volume, multi-tenant enterprise platform environments.
- Proficiency in Scala, Java, or Python for data processing applications.
- Proven experience mentoring junior or mid-level engineering team members.
Qualification wording
Experience working with high-volume, multi-tenant enterprise platform environments.
Proficiency in Scala, Java, or Python for data processing applications.
Proven experience mentoring junior or mid-level engineering team members.
Tools in this posting
- Java
- Python
- Scala
- Kafka
- Spark
Source — Tool mentions in context
- Experience working with high-volume, multi-tenant enterprise platform environments. - Proficiency in Scala, Java, or Python for data processing applications. - Proven experience mentoring junior or mid-level engineering team members.
● Define and drive the technical vision, architecture, and long-term strategy for the data platform and data pipeline ecosystem. ● Design, build, and maintain highly scalable, high-availability data processing systems using Apache Spark, Apache Kafka, and distributed data technologies. ● Lead architecture and design decisions across multiple engineering teams, ensuring solutions meet requirements for scalability, reliability, performance, security, and cost efficiency.
● Minimum 6 years of experience building and operating large-scale data pipelines processing billions of events or transactions per day. ● Minimum 4 years of experience administering and operating Apache Kafka in production environments. ● Minimum 4 years of experience designing and implementing event-driven or streaming data architectures.
● Minimum 12 years of experience in software engineering, data engineering, distributed systems engineering, or big data platform development. ● Minimum 6 years of hands-on experience designing, developing, and optimizing Apache Spark-based data processing solutions. ● Minimum 6 years of experience building and operating large-scale data pipelines processing billions of events or transactions per day.
Posting record
Report an errorOpen 3 monthsWe first saw this posting 97 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
- Status in our records
- Live
- First seen by us
- Jul 23, 2026 · 97 days ago
- Recorded sightings
- 170
- Last seen by us
- Oct 29, 2026
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Your next step
- Have your resume and examples of relevant work ready.
- Check the listed location, eligibility and core experience before starting.
- Ask the employer about the pay range before committing time to the process.
Complete your application on qualys.wd5.myworkdayjobs.com. The employer’s form will show what is required.
Already applied? Track this application
Job description
Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!
Staff Big Data Engineer
Job Description
We are seeking a Staff Big Data Engineer to define, design, and drive the evolution of the data platform and pipeline architecture powering Qualys' Enterprise TruRisk Platform. This role will be responsible for building and scaling distributed data systems that process billions of events and transactions daily while providing technical leadership across multiple engineering teams.
Responsibilities
● Define and drive the technical vision, architecture, and long-term strategy for the data platform and data pipeline ecosystem.
● Design, build, and maintain highly scalable, high-availability data processing systems using Apache Spark, Apache Kafka, and distributed data technologies.
● Lead architecture and design decisions across multiple engineering teams, ensuring solutions meet requirements for scalability, reliability, performance, security, and cost efficiency.
● Design and implement event-driven, streaming, and large-scale data processing architectures to support real-time and batch workloads.
● Troubleshoot, optimize, and resolve complex performance bottlenecks across data pipelines, distributed systems, and platform infrastructure.
● Partner with Product Management, Professional Services, and Sales Engineering teams to evaluate technical solutions, architecture trade-offs, and platform capabilities.
● Establish engineering standards, architectural guidelines, documentation practices, and operational best practices for the data platform.
● Research, evaluate, and recommend emerging technologies that improve platform scalability, performance, and operational efficiency.
● Mentor and guide engineers on distributed systems design, performance optimization, and big data technologies.
● Support architecture reviews, technical governance initiatives, and engineering excellence programs.
Required Qualifications
● Minimum 12 years of experience in software engineering, data engineering, distributed systems engineering, or big data platform development.
● Minimum 6 years of hands-on experience designing, developing, and optimizing Apache Spark-based data processing solutions.
● Minimum 6 years of experience building and operating large-scale data pipelines processing billions of events or transactions per day.
● Minimum 4 years of experience administering and operating Apache Kafka in production environments.
● Minimum 4 years of experience designing and implementing event-driven or streaming data architectures.
● Strong expertise in distributed data processing systems, data lake architectures, and large-scale platform design.
● Experience delivering highly available, scalable, and fault-tolerant distributed systems in production environments.
● Demonstrated experience leading architecture initiatives and influencing technical direction across multiple engineering teams.
● Bachelor's degree in Computer Science, Computer Engineering, Information Technology, or a related technical discipline.
Preferred Qualifications
- Experience working with high-volume, multi-tenant enterprise platform environments.
- Proficiency in Scala, Java, or Python for data processing applications.
- Proven experience mentoring junior or mid-level engineering team members.
Source details
Source notes
Source excerptsSelected passages from the saved posting. Check the full description for conditions and exceptions.
- Pay
No pay amount identified in the saved description.
- Location & working pattern
Pune
Working pattern and location restrictions need checking in the full posting.
- Work authorization
No clear work-authorization passage found. Eligibility is unconfirmed.
See how this role fits your experience
Add your resume to compare the role’s scope, tools and requirements with your experience.