> job detail
C
👽Other
Sr. Data Engineer
C1Exchange · Chennai, India
// classified as
Other (Adjacent or hard to classify.)
posted
1d ago
location
Chennai, India
languages
java, python, scala
tools
aws, docker, dynamodb
> stack
javapythonscalasqlawsdockerdynamodbelasticsearchhadoophivekafkamysqlnosqls3sparkairflow
> education
master
> description
<p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"><strong>Strong skills in Python - Scala or Java. //</strong></span><strong><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> </span></strong><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"><strong>4 years experience on full life cycle Big Data production projects. // </strong><strong>E</strong></span><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"><strong>xperience with AWS services like EMR, Lambda, S3, DynamoDb. // </strong></span><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"><strong>Strong Experience in processing Big Data and analyzing the data using Spark, Map Reduce, Hadoop, Sqoop, Apache Airflow, HDFS, Hive, Zookeeper, etc.</strong></span></p><p><br></p><p><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">C1X Inc. is a fast-growing, global technology company headquartered in San Jose, US, and with offices in Chennai and Tokyo. Our mission is to simplify and innovate in digital marketing by building unique and large-scale data products. We are a world-class engineering team that encompasses frontend (Angular), backend (API / Java / Node.js), mobile (Android/IOS), and Big Data engineering to deliver compelling products.</span></p><p><br></p><p><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">You will be a key member of the engineering team, responsible for shaping data products. You’ll have the opportunity to work with the C-level staff, business stakeholders, product managers, and engineering teams to define the next-generation customer data management and marketing software. </span></p><p><br></p><p><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#434343;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Responsibilities</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Design and implement distributed data processing pipelines using Spark, Hive, Python, and other tools and languages prevalent in the Hadoop ecosystem on AWS. Ability to design and implement end-to-end solutions.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Build utilities, user-defined functions, and frameworks to better enable data flow patterns.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Work with architecture/engineering leads and other teams to ensure quality solutions are implemented, and engineering best practices are defined and adhered to. </span><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Create and maintain data documentation and definitions.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Work with the product owner and scrum master to understand requirements, help the team plan, and execute sprint tickets, work with other technical teams to develop new features.</span></p><p dir="ltr" style="line-height:1.2;margin-top:10pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Help drive best practices in continuous integration and deliver data quality.</span></p><p dir="ltr" style="line-height:1.2;margin-top:10pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Manage data flows and set up automation between various data sources.</span></p><p dir="ltr" style="line-height:1.2;margin-top:10pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Help drive optimization, testing, and tooling of the data products.</span></p><p><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#434343;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Qualifications</span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">B.E. in Computer Science/Engineering or equivalent.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Strong demonstrable skills in two of the following programming languages – Python, Scala or Java.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Minimum 4 years experience working on full life cycle Big Data production projects. </span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Should have experience with AWS services like EMR, Lambda, S3, DynamoDb. </span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Strong Experience in processing Big Data and analyzing the data using Spark, Map Reduce, Hadoop, Sqoop, Apache Airflow, HDFS, Hive, Zookeeper, etc.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Familiarity with Docker, Airflow, or equivalent data pipeline and workflow management tools, </span><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">distributed Stream Processing frameworks for Fast & Big Data like ApacheSpark, Flink, Kafka streams.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Strong skills around developing RESTful APIs.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Experience with version control using GitHub and Jenkins CI/CD pipelines.</span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Intermediate to advanced knowledge of SQL.</span><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> </span><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Relational SQL and NoSQL databases, including Elasticsearch, Postgres, MySql. Experience in SQL tuning, schema design, or analytical programming.</span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Experience with different storage formats like Parquet, CSV and JSON.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Experience in unit, functional and automated testing.</span></p><p dir="ltr" style="line-height:1.2;background-color:#ffffff;margin-top:0pt;margin-bottom:0pt;padding:0pt 0pt 10pt 0pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Strong expertise in troubleshooting production issues.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Comfortable working </span><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">across a wide array of technologies,</span><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> fast-paced, and results-oriented environment.</span></p><p dir="ltr" style="line-height:1.2;margin-top:3pt;margin-bottom:10pt;"><span style="font-size:11pt;font-family:'Open Sans',sans-serif;color:#666666;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Excellent verbal and writing skills to effectively collaborate with both business and technical teams.</span></p>