Data Engineer - GCP

The purpose of this role is to create solution architectures for systems and platforms ensuring all relevant options are explored for time, cost and compliance to the company’s standards, policies and practices whilst meeting the business’ requirements. The role will oversee the work of other architects and delivery of the solution for clients both internal and external, shepherding it through the relevant governance.

Job Description:

We are looking for a highly skilled GCP Data Engineer with strong programming fundamentals and cloud-native architecture experience. The candidate should have hands-on expertise in building scalable data pipelines on Google Cloud Platform, integrating external APIs, and applying software engineering principles to data workflows. 

Business Title

Data Engineer - GCP

Years of Experience

10+ Years

Key responsibilities

• Architect, Design and build Cloud solutions using GCP.
• Design and implement modular, object-oriented Python applications for data ingestion and transformation.
• Strong Object-Oriented Programming skills. Proficient in designing and implementing scalable, maintainable data pipelines using core OOP principles such as encapsulation, inheritance, polymorphism, and abstraction. Experienced in building modular systems with well-defined class hierarchies, applying SOLID principles to ensure clean architecture and separation of concerns. Skilled in leveraging design patterns to solve complex problems and enhance code readability and testability.
• Build and maintain ETL/ELT pipelines using GCP services: BigQuery, Cloud Functions, Cloud Run, Cloud Composer, and DataForm.
• Integrate with external APIs to extract and load data into BigQuery.
• Develop event-driven architectures using Pub/Sub and Eventarc to trigger workflows and manage real-time data streams.
• Automate deployment and testing workflows using Cloud Build and CI/CD pipelines.
• Optimize performance and cost of data processing and storage across GCP services.
• Collaborate with cross-functional teams to translate business requirements into scalable data solutions.
• Ensure data security, governance, and compliance across all pipelines and storage layers.

Must have skills

1. 10 years of experience in data engineering, with at least 4+ years on Google Cloud Platform.
2. Strong proficiency in Python, including OOP principles, modular design, and clean coding standards.
3. Hands-on with GCP services: BigQuery, Cloud Functions, Cloud Run, Cloud Build, Dataform, Pub/Sub, Eventarc, Cloud Storage, and Cloud Composer.
4. Solid understanding of SQL, data modeling, and data warehousing concepts.
5. Familiarity with CI/CD pipelines, Git-based workflows etc.
6. Strong problem-solving skills and ability to work independently in a fast-paced environment.

Good to have skills

1. UNIX shell scripting, SnowFlake, Reshift, Familiar with NoSQL such as MongoDB, etc
2. ETL tool
3. Experience in Cloud platforms - AWS / Azure
4. Client-facing skills & Technical Team management
5.Experience with CI/CD pipelines
6. Requirement gathering 

Location:

DGS India - Pune - Indiqube Orchid

Brand:

Merkle

Time Type:

Full time

Contract Type:

Permanent