Senior Data Engineer

Key Responsibilities Data Engineering & Development • Design, develop, test, and deploy high-quality, secure, scalable, and resilient data pipelines using Apache Spark, Java/Scala across Hadoop and cloud-native object storage platforms. • Build and maintain batch and near real-time data processing frameworks capable of supporting petabyte-scale workloads. • Develop reusable engineering components and frameworks that accelerate data product delivery while maintaining enterprise standards. Architecture & Platform Engineering • Design and implement a "build once, run anywhere" architecture supporting seamless deployment across on-premises and public cloud environments without code changes. • Implement data lineage, metadata management, data cataloguing, data quality controls, and observability capabilities across the data ecosystem. • Collaborate with architects and platform teams to establish scalable design patterns and engineering best practices. Cloud Modernisation • Contribute to migration initiatives moving legacy ETL and data warehouse workloads from on-premises environments to cloud-native architectures. • Leverage cloud services such as Amazon S3, EMR, Glue, and related data services to improve scalability, reliability, and operational efficiency. • Drive adoption of modern lakehouse and distributed compute architectures. Delivery & Technical Leadership • Lead end-to-end development activities including requirement analysis, solution design, coding, testing, deployment, and production support. • Mentor and guide junior engineers through code reviews, technical coaching, and engineering best practices. • Partner with product owners, analysts, architects, and business stakeholders to deliver high-quality solutions within committed timelines. Operational Excellence • Troubleshoot complex production incidents and perform root cause analysis to identify and implement long-term remediation strategies. • Ensure compliance with Mastercard's engineering, security, quality assurance, and operational governance standards. • Continuously identify opportunities to improve performance, automation, monitoring, and process efficiency. Innovation • Evaluate emerging data technologies and conduct proof-of-concept (POC) initiatives to determine their applicability within Mastercard's data ecosystem. • Contribute to engineering innovation and continuous improvement initiatives across the organisation.

Key Responsibilities Data Engineering & Development • Design, develop, test, and deploy high-quality, secure, scalable, and resilient data pipelines using Apache Spark, Java/Scala across Hadoop and cloud-native object storage platforms. • Build and maintain batch and near real-time data processing frameworks capable of supporting petabyte-scale workloads. • Develop reusable engineering components and frameworks that accelerate data product delivery while maintaining enterprise standards. Architecture & Platform Engineering • Design and implement a "build once, run anywhere" architecture supporting seamless deployment across on-premises and public cloud environments without code changes. • Implement data lineage, metadata management, data cataloguing, data quality controls, and observability capabilities across the data ecosystem. • Collaborate with architects and platform teams to establish scalable design patterns and engineering best practices. Cloud Modernisation • Contribute to migration initiatives moving legacy ETL and data warehouse workloads from on-premises environments to cloud-native architectures. • Leverage cloud services such as Amazon S3, EMR, Glue, and related data services to improve scalability, reliability, and operational efficiency. • Drive adoption of modern lakehouse and distributed compute architectures. Delivery & Technical Leadership • Lead end-to-end development activities including requirement analysis, solution design, coding, testing, deployment, and production support. • Mentor and guide junior engineers through code reviews, technical coaching, and engineering best practices. • Partner with product owners, analysts, architects, and business stakeholders to deliver high-quality solutions within committed timelines. Operational Excellence • Troubleshoot complex production incidents and perform root cause analysis to identify and implement long-term remediation strategies. • Ensure compliance with Mastercard's engineering, security, quality assurance, and operational governance standards. • Continuously identify opportunities to improve performance, automation, monitoring, and process efficiency. Innovation • Evaluate emerging data technologies and conduct proof-of-concept (POC) initiatives to determine their applicability within Mastercard's data ecosystem. • Contribute to engineering innovation and continuous improvement initiatives across the organisation. Qualifications Required Experience • 7+ years of experience delivering enterprise-scale Data Warehouse, Data Lake, or Data Lakehouse solutions. • Proven experience implementing multiple end-to-end data engineering projects within large-scale distributed computing environments. • Hands-on experience migrating ETL and analytics workloads from on-premises platforms to cloud-native environments. Technical Expertise • Strong development experience using:Apache Spark, Scala or Java, Hadoop ecosystem technologies, Object Storage platforms • Experience building orchestration and workflow solutions using: Apache Airflow/ Apache NiFi and Similar enterprise scheduling frameworks • Strong SQL expertise and experience with Relational and NoSQL database technologies: Oracle/ SQL Server, Cassandra, Dynamo DB etc. • Working knowledge of cloud platforms, preferably AWS: Amazon S3, EMR, AWS Glue, Cloud-native data services Professional Skills • Strong analytical and problem-solving capabilities. • Experience operating within Agile delivery environments. • Excellent written and verbal communication skills. • Proven ability to collaborate within geographically distributed and matrix-based teams. • Self-starter with strong ownership, accountability, and execution focus. • Ability to learn emerging technologies quickly and apply them effectively to business challenges. Corporate Security Responsibility