Cloud Data Engineer

Work experience Minimum 4-5 years related working experience as a Dataengineer using Azure, Databricks, Python, Azure Data Factory or Synapse WHAT WE OFFER

At BASF Digital Hub Hyderabad we develop innovative digital solutions for BASF, create new exciting customer experiences and business growth, and drive efficiencies in processes, helping to strengthen BASF´s position as the digital leader in the chemical industry. We believe the right path is through creativity, trial and error and great people working and learning together. Become part of our team and develop the future with us - in a global team that embraces diversity and equal opportunities. Objective of the position :- At our unit “Data Foundation - Big Data Management” we aim to offer organizations a robust and scalable solution for managing and deriving insights from vast quantities of data. By utilizing Azure's PaaS components, our platform streamlines the deployment and handling of Big Data workloads, empowering our clients to make data-driven decisions and propel business expansion. Our team is accountable for the creation and upkeep of the Big Data platform built on Azure PaaS components EDL (Enterprise Data Lake). We collaborate intensively with stakeholders to comprehend their needs and devise solutions that fulfill their expectations. The team is also in charge of guaranteeing the platform's scalability, dependability, and security, and for staying abreast with the newest technologies and trends in the big data domain About the Job:

About the Job: We are seeking a highly motivated and detail-oriented candidate who will be responsible for all data engineering aspects of the data product layer of our Enterprise Datalake, working with Azure and Databricks, maintaining/developing our framework and CI/CD pipelines and processes in an ambitious team. Main Task:-

- Troubleshoot data and software related issues and optimize pipelines for performance and scalability. - Work with Azure Databricks, Azure Data Factory/Synapse keeping up to date with the latest features and suggest their implementation if beneficial. - Demonstrate fluent communication skills in English (spoken and written)

- Strong data engineering and development skills to maintain and extend the data product layer of our datalake - Strong knowledge in automation & scripting with tools like Github actions, Azure DevOps pipelines, Azure Data Factory, etc. Technical & Professional Knowledge (additional plus/nice to have skills)

Technical & Professional Knowledge (additional plus/nice to have skills) - Experience with Azure Datafactory/Synapse data flows and pipelines, which are the basis of a legacy application for data products - Ideally, automating code generation and code management with the help of AI tools

- Experience in Agile way of working with a DevOps mindset - In-depth know how with Lakehouse concepts, Databricks and Unity Catalog. - Strong data engineering and development skills to maintain and extend the data product layer of our datalake

- Integration knowledge, how metadata can be exchanged with other applications via APIs e.g. with Colibra datacatalog or an inhouse solution for data product definition - Ideally, backed up by vendor certifications (e.g. Microsoft, Databricks). - Team player with strong interpersonal, written, and verbal communication skills

- Ideally, automating code generation and code management with the help of AI tools - PowerBI as one of the main tools of our users - Integration knowledge, how metadata can be exchanged with other applications via APIs e.g. with Colibra datacatalog or an inhouse solution for data product definition