← back to jobs
> job detail
B
⚙️Data Engineer

GDB/ZE

Basf · Hyderabad, IN
// classified as
Data Engineer (Pipelines, infra, ingestion, ETL.)
posted
1d ago
location
Hyderabad, IN
languages
python
tools
azure, databricks, powerbi
> stack
pythonazuredatabrickspowerbi
> description
<div> <p><span>Objective of the position</span><span> </span></p> </div> <div> <p><span> </span></p> </div> <div> <p><span>Describe your Product Mission here / objective of the role. </span> <br><span>At our unit “Data Foundation - Big Data Management” we aim to offer organizations a robust and scalable solution for managing and deriving insights from vast quantities of data. By utilizing Azure's PaaS components, our platform streamlines the deployment and handling of Big Data workloads, empowering our clients to make data-driven decisions and propel business expansion. Our team is accountable for the creation and upkeep of the Big Data platform built on Azure PaaS components EDL (Enterprise Data Lake). We collaborate intensively with stakeholders to comprehend their needs and devise solutions that fulfill their expectations. The team is also in charge of guaranteeing the platform's scalability, dependability, and security, and for staying abreast with the newest technologies and trends in the big data domain</span><span> </span></p> </div> <div> <p><span> </span></p> </div> <div> <p><span>About the Job:</span><span> </span></p> </div> <div> <p><span>We are seeking a highly motivated and detail-oriented candidate who will be responsible for all data engineering aspects of the data product layer of our Enterprise Datalake, working with Azure and Databricks, maintaining/developing our framework and CI/CD pipelines and processes in an ambitious team.</span><span> </span></p> </div> <div> <p><span> </span></p> </div> <div> <p><span>Main tasks</span><span> </span></p> </div> <div> <div> <div id="{2ea7c093-19cd-42c4-a8dd-f40b265abb3d}{112}"></div> <table dir="ltr" style="width:0.0px" border="1"> <tbody> <tr style="height:77.0px"> <td style="width:686.0px"> <div> <div> <p><span> </span></p> </div> <div> <ul style="list-style-type:disc"> <li> <p><span>Develop, extend and optimize the data product layer of our Enterprise Datalake with high code standards, tests and suitable documentation to meet both user and internal needs.</span><span> </span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:disc"> <li> <p><span>Collaborate with the product owner and architect to understand data requirements and provide suitable solutions.</span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:disc"> <li> <p><span>Troubleshoot data and software related issues and optimize pipelines for performance and scalability.</span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:disc"> <li> <p><span>Work with Azure Databricks, Azure Data Factory/Synapse keeping up to date with the latest features and suggest their implementation if beneficial.</span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:disc"> <li> <p><span>Demonstrate fluent communication skills in English (spoken and written)  </span><span> </span></p> </li> </ul> </div> <div> <p><span> </span></p> </div> </div> </td> </tr> <tr style="height:77.0px"> <td style="width:686.0px"> <div> <div> <p><span> </span></p> </div> </div> </td> </tr> </tbody> </table> </div> </div> <div> <p><span> </span></p> </div> <div> <p><span> </span></p> </div> <div> <p><span>Job Requirements </span><span> </span></p> </div> <div> <p><span>minimum requirement to carry out the work – you might want to point out “additional plus/nice to have skills” that are not mandatory</span><span> </span></p> </div> <div> <p><span> </span></p> </div> <div> <div> <div id="{2ea7c093-19cd-42c4-a8dd-f40b265abb3d}{113}"></div> <table dir="ltr" style="width:0.0px" border="1"> <tbody> <tr style="height:36.0px"> <td style="width:160.0px"> <div> <div> <p><span><span>Edu</span><span>c</span><span>a</span><span>t</span><span>i</span><span>o</span><span>n</span></span><span> </span></p> </div> </div> </td> <td style="width:523.0px"> <div> <div> <ul style="list-style-type:square"> <li> <p><span>Bachelor's degree in computer science, Information Technology, Engineering, Business, or related fields.</span><span> </span></p> </li> </ul> </div> </div> </td> </tr> <tr style="height:36.0px"> <td style="width:160.0px"> <div> <div> <p><span>Work experience</span><span> </span></p> </div> </div> </td> <td style="width:523.0px"> <div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Minimum 7-8 years related working experience as a </span><span>Dataengineer</span><span> using</span></span><span> Azure, Databricks, Python, Azure Data Factory or Synapse</span><span> </span></p> </li> </ul> </div> </div> </td> </tr> <tr style="height:33.0px"> <td style="width:160.0px"> <div> <div> <p><span><span>T</span><span>e</span><span>c</span><span>hn</span><span>i</span><span>c</span><span>a</span><span>l</span><span> </span><span>&amp;</span><span> </span><span>P</span><span>r</span><span>o</span><span>f</span><span>e</span><span>ss</span><span>i</span><span>o</span><span>n</span><span>a</span><span>l</span><span> </span><span>Knowledge</span><span> (mandatory)</span></span><span> </span></p> </div> </div> </td> <td style="width:523.0px"> <div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Experienc</span><span>e in</span><span> Agile way of working </span><span>with a DevOps mindset</span></span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:square"> <li> <p><span><span>In-depth </span><span>know how</span><span> with </span><span>Lakehouse </span><span>concepts</span><span>, </span><span>Databricks</span><span> and Unity </span><span>Catalog</span><span>.</span></span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Strong </span><span>data</span><span> </span><span>engineering and development skills to </span><span>maintain</span><span> and extend </span><span>the </span><span>data product layer</span><span> of our </span><span>datalake</span></span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Strong knowledge in automation &amp; scripting with tools like </span><span>Github</span><span> actions</span><span>, </span><span>Azure DevOps pipelines, Azure Data Factory, etc.</span></span><span> </span></p> </li> </ul> </div> </div> </td> </tr> <tr style="height:33.0px"> <td style="width:160.0px"> <div> <div> <p><span><span>T</span><span>e</span><span>c</span><span>hn</span><span>i</span><span>c</span><span>a</span><span>l</span><span> </span><span>&amp;</span><span> </span><span>P</span><span>r</span><span>o</span><span>f</span><span>e</span><span>ss</span><span>i</span><span>o</span><span>n</span><span>a</span><span>l</span><span> </span><span>Knowledge (</span><span>additional</span><span> plus/nice to have skills)</span></span><span> </span></p> </div> </div> </td> <td style="width:523.0px"> <div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Experience with </span><span>Azure </span><span>Datafactory</span><span>/Synapse</span><span> data flows</span><span> and pipelines, which are the basis of a </span><span>legacy application</span><span> for data products</span><span> </span></span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Ideally, </span><span>automating code generation</span><span> and code management with the help of AI</span><span> </span><span>tools</span></span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:square"> <li> <p><span><span>PowerBI</span><span> </span><span>as one of the main tools of our users</span></span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Integration knowledge</span><span>,</span><span> </span><span>how</span><span> metadata can be exchanged with other applications via API</span><span>s</span><span> e.g. with </span><span>Colibra</span><span> </span><span>datacatalog</span><span> </span><span>or an inhouse </span><span>solution for data product definition</span></span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Ideally, backed up by vendor certifications (e.g. Microsoft, </span><span>Databricks</span><span>).</span></span><span> </span></p> </li> </ul> </div> <div> <ul style="list-style-type:square"> <li> <p><span><span>Team player with strong interpersonal, </span><span>written,</span><span> and verbal communication skills </span></span><span> </span></p> </li> </ul> </div> </div> </td> </tr> </tbody> </table> </div> </div> <p> </p>