Time since we first saw it
Day 1
Early. Similar jobs usually come down between day 6 and day 56.
At Capco, jobs usually come down by about day 39.
From 14,771 similar postings that came down, and 240 of Capco’s. Coming down is not always a hire.
India - Bengaluru
Actively hiring, first seen by us 1 day agoDesign, build and maintain scalable data pipelines and transformation workflows in Databricks
Develop and maintain Delta Lake tables with sound partitioning and optimisation
Optimise Spark workloads, queries and cluster configuration
Build data ingestion, validation and reconciliation processes across multiple sources
Maintain CI/CD pipelines for deploying Databricks code across environments
Design, develop, test, and maintain scalable data pipelines and transformation workflows using Databricks.
Develop and maintain Delta Lake tables, applying appropriate data modelling, partitioning, and optimisation strategies.
Optimise Spark workloads through efficient transformations, query tuning, cluster configuration, and resource management.
Develop data validation, reconciliation, and quality control processes to ensure data accuracy and consistency.
Develop and maintain CI/CD pipelines for deploying Databricks code and configurations across environments.
Experience-6-12 yrs
Databricks Development and Data Engineering
Build and optimise data processing solutions using Apache Spark, PySpark, and SQL.
Use Git, Azure DevOps or equivalent tools for source control, collaboration, and release management.
“Capco, a Wipro company, is a global technology and management consulting firm.
In the employer’s words · Read in context
Actively hiringFirst seen by us 1 day ago, still listed, and not a repost of an earlier posting.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 1
Early. Similar jobs usually come down between day 6 and day 56.
At Capco, jobs usually come down by about day 39.
From 14,771 similar postings that came down, and 240 of Capco’s. Coming down is not always a hire.
102 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
- Strong practical experience developing data pipelines and transformation workloads using Databricks. - Advanced proficiency in Python, PySpark, and SQL. - Strong understanding of Apache Spark architecture, distributed data processing, and performance optimisation.
- Design, develop, test, and maintain scalable data pipelines and transformation workflows using Databricks. - Build and optimise data processing solutions using Apache Spark, PySpark, and SQL. - Develop batch and, where required, streaming data pipelines to support enterprise data processing requirements.
- Databricks Certified Data Engineer Associate or Professional certification. - Experience with Unity Catalog, Delta Live Tables / Lakeflow Spark Declarative Pipelines, and Databricks SQL. - Exposure to structured streaming and real-time data processing using Spark Structured Streaming or Kafka.
Job Description: Role: AWS Data Engineer Location : Bangalore, Pune
- Design and develop data ingestion and integration pipelines connecting structured, semi-structured, and unstructured data sources. - Implement data orchestration workflows using Databricks Workflows, AWS, or equivalent technologies. - Support integration with enterprise databases, APIs, cloud storage, and downstream analytical platforms.
- Good understanding of lakehouse architecture, data modelling, and data warehousing concepts. - Strong knowledge and hands-on experience of AWS development in some of the following : Lambda, Glue, Step Functions, IAM roles, Lakeformation, EventBridge, SNS, SQS, EC2,Security Groups, CloudFormation, RDS, DynamoDB, Redshift - Experience with data orchestration tools such as Databricks Workflows or Azure Data Factory.
- Develop and maintain CI/CD pipelines for deploying Databricks code and configurations across environments. - Use Git, Azure DevOps or equivalent tools for source control, collaboration, and release management. - Implement unit testing, integration testing, and data quality testing for data engineering solutions.
- Strong knowledge and hands-on experience of AWS development in some of the following : Lambda, Glue, Step Functions, IAM roles, Lakeformation, EventBridge, SNS, SQS, EC2,Security Groups, CloudFormation, RDS, DynamoDB, Redshift - Experience with data orchestration tools such as Databricks Workflows or Azure Data Factory. - Practical experience implementing data quality checks, error handling, logging, and monitoring.
Experience-6-12 yrs Databricks Development and Data Engineering - Design, develop, test, and maintain scalable data pipelines and transformation workflows using Databricks.
Databricks Development and Data Engineering - Design, develop, test, and maintain scalable data pipelines and transformation workflows using Databricks. - Build and optimise data processing solutions using Apache Spark, PySpark, and SQL.
- Apply software engineering best practices to data pipeline development, including modular design, code reviews, automated testing, and documentation. - Develop and maintain CI/CD pipelines for deploying Databricks code and configurations across environments. - Use Git, Azure DevOps or equivalent tools for source control, collaboration, and release management.
- Implement unit testing, integration testing, and data quality testing for data engineering solutions. - Support automated deployment processes using Databricks Asset Bundles, Terraform, or equivalent tools where appropriate. - Troubleshoot and resolve defects across development, testing, and production environments.
- Data Governance, Security and Reliability - Implement appropriate data access controls, security configurations, and governance standards within Databricks. - Work with Unity Catalog or equivalent governance capabilities to support data discovery, access management, and lineage.
Essential Technical Skills - Typically 6+ years of experience in data engineering, software engineering, or related technology roles, including substantial hands-on experience with Databricks. - Strong practical experience developing data pipelines and transformation workloads using Databricks.
- Typically 6+ years of experience in data engineering, software engineering, or related technology roles, including substantial hands-on experience with Databricks. - Strong practical experience developing data pipelines and transformation workloads using Databricks. - Advanced proficiency in Python, PySpark, and SQL.
- Understanding of data security, access controls, and governance principles. - Strong troubleshooting and performance-tuning capabilities for Spark and Databricks workloads. - Experience working within Agile software delivery environments.
- Experience delivering data engineering solutions within banking, capital markets, insurance, or other financial services environments. - Databricks Certified Data Engineer Associate or Professional certification. - Experience with Unity Catalog, Delta Live Tables / Lakeflow Spark Declarative Pipelines, and Databricks SQL.
- Implement robust data ingestion, cleansing, transformation, and enrichment processes across multiple data sources. - Develop and maintain Delta Lake tables, applying appropriate data modelling, partitioning, and optimisation strategies. - Implement reusable, modular, and maintainable data engineering frameworks and components.
- Strong understanding of Apache Spark architecture, distributed data processing, and performance optimisation. - Hands-on experience with Delta Lake, including data ingestion, transformation, optimisation, and data management. - Experience designing and implementing ETL/ELT pipelines for large-scale data processing.
- Experience with Unity Catalog, Delta Live Tables / Lakeflow Spark Declarative Pipelines, and Databricks SQL. - Exposure to structured streaming and real-time data processing using Spark Structured Streaming or Kafka. - Experience with infrastructure as code using Terraform or equivalent technologies.
- Experience with infrastructure as code using Terraform or equivalent technologies. - Familiarity with Microsoft Fabric, Snowflake, Synapse Analytics, or other modern data platforms. - Experience with cloud data migration and legacy data platform modernisation programmes.
- Advanced proficiency in Python, PySpark, and SQL. - Strong understanding of Apache Spark architecture, distributed data processing, and performance optimisation. - Hands-on experience with Delta Lake, including data ingestion, transformation, optimisation, and data management.
- Exposure to structured streaming and real-time data processing using Spark Structured Streaming or Kafka. - Experience with infrastructure as code using Terraform or equivalent technologies. - Familiarity with Microsoft Fabric, Snowflake, Synapse Analytics, or other modern data platforms.
Complete your application on job-boards.greenhouse.io. The employer’s form will show what is required.
Already applied? Track this application
Job Title: Data Engineer
About Us
“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.
WHY JOIN CAPCO?
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
DIVERSITY & INCLUSION
We believe that diversity of people and perspective gives us a competitive advantage.
Job Description:
Role: AWS Data Engineer
Location : Bangalore, Pune
Experience-6-12 yrs
Databricks Development and Data Engineering
Data Platform Design and Optimisation
Data Integration and Orchestration
Software Engineering and DevOps
Consulting, Technical Leadership and Stakeholder Engagement
Required Skills and Experience
Essential Technical Skills
Consulting and Professional Skills
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
India - Bengaluru
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.