Data Engineer
Pune Maharashtra India
- Pay
- Pay not listed
- Work setting
- Not stated
- Employment
Full-time — employment source
Time Type: Full time------------------------------------------------------ Most Relevant Skills
Employment type in the employer’s posting: Full time
From the employer’s posting
What you’ll work on
Full postingDesign, develop, test and maintain PySpark and Spark batch and streaming pipelines
Manage and monitor Spark streaming jobs doing eligibility and data quality checks
Manage and monitor Spark batch jobs writing audit and inbound/outbound data to Hive
Respond to ITRS alerts and job failures, kills, or late triggers
Configure JDBC connections from SAS, Python and R to sandbox data platform environments
From the employer’s posting
Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads.
Manage and monitor Spark Streaming Jobs (running Sunday evening through Saturday afternoon) performing Eligibility and Data Quality (DQ) checks
Manage and monitor Spark Batch Jobs (running hourly or every 2 hours) that store audit, inbound, and outbound data into Hive tables
Respond to ITRS Alerts and system monitoring events (e.g., job failures, kills, or jobs not triggered within 30 minutes of scheduled time).
Configure and manage JDBC connections from SAS, Python, and R to dedicated Sandbox/data platform environments.
What you’ll bring
All qualificationsCore experience
- 2-5 years of relevant experience
- Bachelor’s degree/University degree on information technology or Engineer or equivalent experience
- Experience in programming/debugging used in business applications
Qualification wording
2-5 years of relevant experience
Bachelor’s degree/University degree on information technology or Engineer or equivalent experience
Experience in programming/debugging used in business applications
Education & alternatives
Education: - Bachelor’s degree/University degree on information technology or Engineer or equivalent experience This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
Tools in this posting
- Python
- R
- SAS
- SQL
- Hadoop
- Hive
- Kafka
- Oracle
- Spark
- PySpark
Source — Tool mentions in context
Work with large-scale data sources across Oracle, Hadoop, Impala, and SAS Server. Configure and manage JDBC connections from SAS, Python, and R to dedicated Sandbox/data platform environments. Collaborate on schema creation during code deployment; raise CMP tickets for permission grants and country-specific schema access.
Tool / Technology - Role in Platform Python / PySpark - Primary language for data pipeline development Apache Spark - Distributed compute engine for large-scale data workloads
Must Have: - Proficiency in Python and PySpark for distributed data engineering. - Experience with Apache Spark (Streaming and Batch) in production environments.
Respond to ITRS Alerts and system monitoring events (e.g., job failures, kills, or jobs not triggered within 30 minutes of scheduled time). Work with large-scale data sources across Oracle, Hadoop, Impala, and SAS Server. Configure and manage JDBC connections from SAS, Python, and R to dedicated Sandbox/data platform environments.
Spark Batch - Scheduled ETL jobs writing to Hive tables SAS Server - Main analytical tool with Oracle/Hadoop/Impala access Hadoop / HDFS / Hive - Distributed storage, batch processing, and table management
Good to Have: - Experience with SAS or equivalent enterprise analytical platforms. - Familiarity with SDLC practices across Development, UAT, and Production environments. ( GitHub / Light speed )
Hadoop / HDFS / Hive - Distributed storage, batch processing, and table management Impala - Fast SQL querying on Hadoop data Kafka - Event streaming and messaging platform
- Experience with Apache Spark (Streaming and Batch) in production environments. - Strong understanding of SQL and ability to work with relational (Oracle) and columnar (Impala, Hive) databases. - Familiarity with Kafka or event-driven architectures.
SAS Server - Main analytical tool with Oracle/Hadoop/Impala access Hadoop / HDFS / Hive - Distributed storage, batch processing, and table management Impala - Fast SQL querying on Hadoop data
Manage and monitor Spark Streaming Jobs (running Sunday evening through Saturday afternoon) performing Eligibility and Data Quality (DQ) checks, publishing data to Kafka sender topics, and processing exception/XMEV requests. Manage and monitor Spark Batch Jobs (running hourly or every 2 hours) that store audit, inbound, and outbound data into Hive tables (e.g., gfolyreg_managed.SWXPTT_audit). Respond to ITRS Alerts and system monitoring events (e.g., job failures, kills, or jobs not triggered within 30 minutes of scheduled time).
Spark Streaming - Real-time data processing and Kafka integration Spark Batch - Scheduled ETL jobs writing to Hive tables SAS Server - Main analytical tool with Oracle/Hadoop/Impala access
Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads. Manage and monitor Spark Streaming Jobs (running Sunday evening through Saturday afternoon) performing Eligibility and Data Quality (DQ) checks, publishing data to Kafka sender topics, and processing exception/XMEV requests. Manage and monitor Spark Batch Jobs (running hourly or every 2 hours) that store audit, inbound, and outbound data into Hive tables (e.g., gfolyreg_managed.SWXPTT_audit).
Apache Spark - Distributed compute engine for large-scale data workloads Spark Streaming - Real-time data processing and Kafka integration Spark Batch - Scheduled ETL jobs writing to Hive tables
Impala - Fast SQL querying on Hadoop data Kafka - Event streaming and messaging platform Oracle / JDBC - Relational database connectivity
- Strong understanding of SQL and ability to work with relational (Oracle) and columnar (Impala, Hive) databases. - Familiarity with Kafka or event-driven architectures. - Experience with job scheduling and monitoring tools (e.g., Autosys, ITRS).
Kafka - Event streaming and messaging platform Oracle / JDBC - Relational database connectivity Autosys - Job scheduling, alerting, and monitoring
The Applications Development Programmer Analyst Citi is seeking a Applications Development Programmer Analyst 2 (C10) to join its Technology – Data Engineering function, focused on the PySpark / Spark-based Data Platform. In this role, you will design, develop, and maintain scalable data pipelines and analytical solutions, supporting large-scale data operations across enterprise systems. You will work within a full software development lifecycle (SDLC) environment spanning Development (NAM), UAT, and Production. activities.
Core Technical Duties Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads. Manage and monitor Spark Streaming Jobs (running Sunday evening through Saturday afternoon) performing Eligibility and Data Quality (DQ) checks, publishing data to Kafka sender topics, and processing exception/XMEV requests.
Python / PySpark - Primary language for data pipeline development Apache Spark - Distributed compute engine for large-scale data workloads Spark Streaming - Real-time data processing and Kafka integration
- Proficiency in Python and PySpark for distributed data engineering. - Experience with Apache Spark (Streaming and Batch) in production environments. - Strong understanding of SQL and ability to work with relational (Oracle) and columnar (Impala, Hive) databases.
Posting record
Report an errorActively hiringFirst seen by us 4 days ago, still listed, and not a repost of an earlier posting.
- Status in our records
- Live
- First seen by us
- Oct 26, 2026 · 4 days ago
- Recorded sightings
- 13
- Last seen by us
- Oct 30, 2026
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Your next step
- Have your resume and examples of relevant work ready.
- Check the listed location, eligibility and core experience before starting.
- Ask the employer about the pay range before committing time to the process.
Complete your application on citi.wd5.myworkdayjobs.com. The employer’s form will show what is required.
Already applied? Track this application
Job description
The Applications Development Programmer Analyst
Citi is seeking a Applications Development Programmer Analyst 2 (C10) to join its Technology – Data Engineering function, focused on the PySpark / Spark-based Data Platform. In this role, you will design, develop, and maintain scalable data pipelines and analytical solutions, supporting large-scale data operations across enterprise systems. You will work within a full software development lifecycle (SDLC) environment spanning Development (NAM), UAT, and Production.
activities.
Core Technical Duties
Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads.
Manage and monitor Spark Streaming Jobs (running Sunday evening through Saturday afternoon) performing Eligibility and Data Quality (DQ) checks, publishing data to Kafka sender topics, and processing exception/XMEV requests.
Manage and monitor Spark Batch Jobs (running hourly or every 2 hours) that store audit, inbound, and outbound data into Hive tables (e.g., gfolyreg_managed.SWXPTT_audit).
Respond to ITRS Alerts and system monitoring events (e.g., job failures, kills, or jobs not triggered within 30 minutes of scheduled time).
Work with large-scale data sources across Oracle, Hadoop, Impala, and SAS Server.
Configure and manage JDBC connections from SAS, Python, and R to dedicated Sandbox/data platform environments.
Collaborate on schema creation during code deployment; raise CMP tickets for permission grants and country-specific schema access.
Support the retirement of legacy applications and champion adoption of modern data engineering technologies.
Technical Stack
Tool / Technology - Role in Platform
Python / PySpark - Primary language for data pipeline development
Apache Spark - Distributed compute engine for large-scale data workloads
Spark Streaming - Real-time data processing and Kafka integration
Spark Batch - Scheduled ETL jobs writing to Hive tables
SAS Server - Main analytical tool with Oracle/Hadoop/Impala access
Hadoop / HDFS / Hive - Distributed storage, batch processing, and table management
Impala - Fast SQL querying on Hadoop data
Kafka - Event streaming and messaging platform
Oracle / JDBC - Relational database connectivity
Autosys - Job scheduling, alerting, and monitoring
Required Skills & Qualifications
Must Have:
- Proficiency in Python and PySpark for distributed data engineering.
- Experience with Apache Spark (Streaming and Batch) in production environments.
- Strong understanding of SQL and ability to work with relational (Oracle) and columnar (Impala, Hive) databases.
- Familiarity with Kafka or event-driven architectures.
- Experience with job scheduling and monitoring tools (e.g., Autosys, ITRS).
- Knowledge of data governance, access management, and information security standards.
- Strong problem-solving skills for diagnosing and resolving complex data pipeline issues.
Good to Have:
- Experience with SAS or equivalent enterprise analytical platforms.
- Familiarity with SDLC practices across Development, UAT, and Production environments. ( GitHub / Light speed )
Qualifications:
- 2-5 years of relevant experience
- Experience in programming/debugging used in business applications
- Working knowledge of industry practice and standards
- Comprehensive knowledge of specific business area for application development
- Working knowledge of program languages
- Consistently demonstrates clear and concise written and verbal communication
Education:
- Bachelor’s degree/University degree on information technology or Engineer or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
------------------------------------------------------
Job Family Group:
Technology------------------------------------------------------
Job Family:
Applications Development------------------------------------------------------
Time Type:
Full time------------------------------------------------------
Most Relevant Skills
Please see the requirements listed above.------------------------------------------------------
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.
Source details
Source notes
Source excerptsSelected passages from the saved posting. Check the full description for conditions and exceptions.
- Pay
No pay amount identified in the saved description.
- Location & working pattern
Pune Maharashtra India
Working pattern and location restrictions need checking in the full posting.
- Work authorization
No clear work-authorization passage found. Eligibility is unconfirmed.
See how this role fits your experience
Add your resume to compare the role’s scope, tools and requirements with your experience.