Time since we first saw it
Day 146
Up longer than most. Similar jobs usually come down between day 6 and day 56.
From 14,771 similar postings that came down. Coming down is not always a hire.
Hyderabad, Indore and Ahmedabad, India
Open 4 months, first seen by us 146 days agoWork setup in the employer’s posting: On-siteFrom the employer’s posting
Employment type in the employer’s posting: Full timeFrom the employer’s posting
Design, build, and optimize ETL/ELT data pipelines on Databricks with PySpark and Spark SQL
Build incremental ingestion pipelines with Auto Loader and structured streaming
Build reusable accelerators, templates, and demo environments for pre-sales and delivery
Tune performance and cost on Databricks clusters and jobs, and fix production pipeline issues
Mentor junior engineers as the team scales
Design, build, and optimize ETL/ELT data pipelines on Databricks using PySpark and Spark SQL.
Build and maintain incremental ingestion pipelines using Auto Loader and structured streaming
Build reusable accelerators, templates, and demo environments to support pre-sales and speed up future client delivery.
Perform data quality checks, performance tuning, and cost optimization on Databricks clusters and jobs.
Mentor junior engineers as the team scales
From the employer’s posting.
is a premier global software products and services firm that specializes in providing innovative solutions and services for data-driven enterprises.
In the employer’s words · Read in context
Open 4 monthsWe first saw this posting 146 days ago and it is still up. Roles open this long are often slow to fill or kept open to collect applications.
These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.
Day 146
Up longer than most. Similar jobs usually come down between day 6 and day 56.
From 14,771 similar postings that came down. Coming down is not always a hire.
11 open jobs
Open jobs by the week we first saw them, last 12 weeks.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
· Experience with at least one major cloud platform (Azure, AWS, or GCP); Azure strongly preferred. · Experience with Python for data engineering tasks beyond Spark (scripting, automation, testing). · Familiarity with CI/CD practices for data pipelines (Git-based workflows, automated testing/deployment).
Key Responsibilities · Design, build, and optimize ETL/ELT data pipelines on Databricks using PySpark and Spark SQL. · Build and maintain incremental ingestion pipelines using Auto Loader and structured streaming, including checkpoint management and schema evolution handling.
· 5–9 years of data engineering experience, with at least 2 years of hands-on Databricks experience in production. · Strong proficiency in PySpark and/or Spark SQL for large-scale data processing. · Practical experience with Delta Lake, Unity Catalog, and Databricks Workflows/job orchestration.
· Practical experience with Delta Lake, Unity Catalog, and Databricks Workflows/job orchestration. · Solid SQL skills and experience with data modeling for analytics/lakehouse architectures. · Experience with at least one major cloud platform (Azure, AWS, or GCP); Azure strongly preferred.
· Databricks Certified Data Engineer Associate/Professional certification. · Experience with Lakeflow Declarative Pipelines (formerly Delta Live Tables), Lakeflow Connect for managed ingestion, MLflow for experiment tracking and model registry, Databricks SQL, and AI/BI Genie. · Exposure to data governance and security frameworks (row/column-level security, data masking).
· Solid SQL skills and experience with data modeling for analytics/lakehouse architectures. · Experience with at least one major cloud platform (Azure, AWS, or GCP); Azure strongly preferred. · Experience with Python for data engineering tasks beyond Spark (scripting, automation, testing).
· Prior experience in a consulting/IT services environment delivering to multiple clients. · Familiarity with orchestration tools (Airflow) and ingestion tools (Fivetran, Kafka, Azure Data Factory). What Success Looks Like
Role: As one of the founding members of Kanerika's Databricks delivery practice, you will be a hands-on builder responsible for designing, developing, and optimizing data pipelines on Databricks for client engagements. You'll work closely with the Practice Lead to translate architecture into working solutions, build reusable accelerators ahead of client demand, and help establish engineering best practices for the growing team. Key Responsibilities
· Build and maintain incremental ingestion pipelines using Auto Loader and structured streaming, including checkpoint management and schema evolution handling. · Implement and maintain Delta Lake tables (including Change Data Feed and liquid clustering), Unity Catalog structures, and Databricks Workflows for client and internal projects. · Build reusable accelerators, templates, and demo environments to support pre-sales and speed up future client delivery.
· Collaborate with the Practice Lead/Architect on solution design for client engagements, providing engineering-level input on feasibility and effort. · Perform data quality checks, performance tuning, and cost optimization on Databricks clusters and jobs. · Participate in client-facing technical discussions as needed, including discovery sessions and technical walkthroughs.
· Troubleshoot and resolve production data pipeline issues across client environments. · Stay current with Databricks platform releases across Unity Catalog, Lakeflow Declarative Pipelines, and share learnings with the team through internal knowledge sessions and documentation. Required Skills & Experience
Required Skills & Experience · 5–9 years of data engineering experience, with at least 2 years of hands-on Databricks experience in production. · Strong proficiency in PySpark and/or Spark SQL for large-scale data processing.
· Strong proficiency in PySpark and/or Spark SQL for large-scale data processing. · Practical experience with Delta Lake, Unity Catalog, and Databricks Workflows/job orchestration. · Solid SQL skills and experience with data modeling for analytics/lakehouse architectures.
PEFERRED/ NICE TO HAVE · Databricks Certified Data Engineer Associate/Professional certification. · Experience with Lakeflow Declarative Pipelines (formerly Delta Live Tables), Lakeflow Connect for managed ingestion, MLflow for experiment tracking and model registry, Databricks SQL, and AI/BI Genie.
· Within 12 months: Recognized as a go-to senior engineer on the team, mentoring newer hires and contributing to architecture decisions. · Leading databricks partnership upgrade to Gold level. Benefits
· Familiarity with CI/CD practices for data pipelines (Git-based workflows, automated testing/deployment). · Strong debugging and performance-tuning skills for Spark jobs (partitioning, caching, cluster sizing). · Good communication skills and comfort working directly with client stakeholders when needed.
Complete your application on kanerika.zohorecruit.com. The employer’s form will show what is required.
Already applied? Track this application
About
Kanerika
Who
we are:
Kanerika
Inc. is a premier global software products and services firm that specializes
in providing innovative solutions and services for data-driven enterprises. Our
focus is to empower businesses to achieve their digital transformation goals
and maximize their business impact through the effective use of data and AI. We leverage cutting-edge technologies in data
analytics, data governance, AI-ML, GenAI/ LLM and industry best practices to
deliver custom solutions that help organizations optimize their operations,
enhance customer experiences, and drive growth.
Kanerika has won several awards over the years,
including:
· CMMI
Level 3 Appraised in 2024.
· Best
Place to Work 2023 by Great Place to Work
· Top
10 Most Recommended RPA Start-Ups in 2022 by RPA today.
· Frost
& Sullivan India 2021 Technology Innovation Award for its Kompass
composable solution architecture.
· Kanerika
has also been recognized for its commitment to customer privacy and data
security, having achieved, ISO 9001, ISO 27701, SOC2, and GDPR compliances.
Working
for us
Kanerika
is rated 4.6/5 on Glassdoor, for many good reasons. We truly value our
employees' growth, well-being, and diversity, and people’s experiences bear
this out. At Kanerika, we offer a host of enticing benefits that create an
environment where you can thrive both personally and professionally. From our
inclusive hiring practices and mandatory training on creating a safe work
environment to our flexible working hours and generous parental leave, we
prioritize the well-being and success of our employees. Our commitment to
professional development is evident through our mentorship programs, job training
initiatives, and support for professional certifications. Additionally, our
company-sponsored outings and various time-off benefits ensure a healthy
work-life balance. Join us at Kanerika and become part of a vibrant and diverse
community where your talents are recognized, your growth is nurtured, and your
contributions make a real impact. See the benefits section below for the perks
you’ll get while working for Kanerika.
Locations
We
are located in Austin (USA), Singapore, Hyderabad, Indore and Ahmedabad
(India).
Job
Location: Hyderabad, Indore and Ahmedabad (India)
Role:
As
one of the founding members of Kanerika's Databricks delivery practice, you
will be a hands-on builder responsible for designing, developing, and
optimizing data pipelines on Databricks for client engagements. You'll work
closely with the Practice Lead to translate architecture into working
solutions, build reusable accelerators ahead of client demand, and help
establish engineering best practices for the growing team.
Key
Responsibilities
· Design,
build, and optimize ETL/ELT data pipelines on Databricks using PySpark and
Spark SQL.
· Build
and maintain incremental ingestion pipelines using Auto Loader and structured
streaming, including checkpoint management and schema evolution handling.
· Implement
and maintain Delta Lake tables (including Change Data Feed and liquid
clustering), Unity Catalog structures, and Databricks Workflows for client and
internal projects.
· Build
reusable accelerators, templates, and demo environments to support pre-sales
and speed up future client delivery.
· Collaborate
with the Practice Lead/Architect on solution design for client engagements,
providing engineering-level input on feasibility and effort.
· Perform
data quality checks, performance tuning, and cost optimization on Databricks
clusters and jobs.
· Participate
in client-facing technical discussions as needed, including discovery sessions
and technical walkthroughs.
· Write
clean, well-documented, and testable code following team engineering standards.
· Mentor
junior engineers as the team scales, and contribute to internal
knowledge-sharing and best-practice documentation.
· Troubleshoot
and resolve production data pipeline issues across client environments.
· Stay
current with Databricks platform releases across Unity Catalog, Lakeflow
Declarative Pipelines, and share learnings with the team through internal
knowledge sessions and documentation.
Required
Skills & Experience
· 5–9
years of data engineering experience, with at least 2 years of hands-on
Databricks experience in production.
· Strong
proficiency in PySpark and/or Spark SQL for large-scale data processing.
· Practical
experience with Delta Lake, Unity Catalog, and Databricks Workflows/job
orchestration.
· Solid
SQL skills and experience with data modeling for analytics/lakehouse
architectures.
· Experience
with at least one major cloud platform (Azure, AWS, or GCP); Azure strongly
preferred.
· Experience
with Python for data engineering tasks beyond Spark (scripting, automation,
testing).
· Familiarity
with CI/CD practices for data pipelines (Git-based workflows, automated
testing/deployment).
· Strong
debugging and performance-tuning skills for Spark jobs (partitioning, caching,
cluster sizing).
· Good
communication skills and comfort working directly with client stakeholders when
needed.
PEFERRED/
NICE TO HAVE
· Databricks
Certified Data Engineer Associate/Professional certification.
· Experience
with Lakeflow Declarative Pipelines (formerly Delta Live Tables), Lakeflow
Connect for managed ingestion, MLflow for experiment tracking and model
registry, Databricks SQL, and AI/BI Genie.
· Exposure
to data governance and security frameworks (row/column-level security, data
masking).
· Prior
experience in a consulting/IT services environment delivering to multiple
clients.
· Familiarity
with orchestration tools (Airflow) and ingestion tools (Fivetran, Kafka, Azure
Data Factory).
What
Success Looks Like
· Within
3 months: Comfortable with Kanerika's delivery standards; has built or
contributed to at least one reusable accelerator/demo asset.
· Within
6 months: Independently delivering core engineering work on client
engagement(s) with minimal oversight.
· Within
12 months: Recognized as a go-to senior engineer on the team, mentoring newer
hires and contributing to architecture decisions.
· Leading
databricks partnership upgrade to Gold level.
Why
join us?
· Work
with a passionate and innovative team in a fast-paced, growth-oriented
environment.
· Gain
hands-on experience in content marketing with exposure to real-world projects.
· Opportunity
to learn from experienced professionals and enhance your marketing skills.
· Contribute
to exciting initiatives and make an impact from day one.
· Competitive
stipend and potential for growth within the company.
· Recognized
for excellence in data and AI solutions with industry awards and accolades.
1. Culture:
i. Open
Door Policy: Encourages open communication and accessibility to management.
ii. Open
Office Floor Plan: Fosters a collaborative and interactive work environment.
iii. Flexible
Working Hours: Allows employees to have flexibility in their work schedules.
iv. Employee
Referral Bonus: Rewards employees for referring qualified candidates.
v. Appraisal
Process Twice a Year: Provides regular performance evaluations and feedback.
2. Inclusivity and Diversity:
a. Hiring practices that promote diversity: Ensures a
diverse and inclusive workforce.
b. Mandatory POSH training: Promotes a safe and
respectful work environment.
3. Health Insurance and Wellness Benefits:
a. GMC and Term Insurance: Offers medical coverage
and financial protection.
b. Health Insurance: Provides coverage for medical
expenses.
c. Disability Insurance: Offers financial support in
case of disability.
4. Child Care & Parental Leave Benefits:
a. Company-sponsored family events: Creates
opportunities for employees and their families to bond.
b. Generous Parental Leave: Allows parents to take
time off after the birth or adoption of a child.
c. Family Medical Leave: Offers leave for employees
to take care of family members' medical needs.
5. Perks and Time-Off Benefits:
a. Company-sponsored outings: Organizes recreational
activities for employees.
b. Gratuity: Provides a monetary benefit as a token
of appreciation.
c. Provident Fund: Helps employees save for
retirement.
d. Generous PTO: Offers more than the industry
standard for paid time off.
e. Paid sick days: Allows employees to take paid time
off when they are unwell.
f. Paid
holidays: Gives employees paid time off for designated holidays.
g. Bereavement Leave: Provides time off for employees
to grieve the loss of a loved one.
6. Professional Development Benefits:
a. L&D with FLEX- Enterprise Learning Repository:
Provides access to a learning repository for professional development.
b. Mentorship Program: Offers guidance and support
from experienced professionals.
c. Job Training: Provides training to enhance
job-related skills.
d. Professional Certification Reimbursements: Assists
employees in obtaining professional certifications.
e. Promote from Within: Encourages internal growth
and advancement opportunities.
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
No pay amount identified in the saved description.
Hyderabad, Indore and Ahmedabad, India
Working pattern and location restrictions need checking in the full posting.
No clear work-authorization passage found. Eligibility is unconfirmed.