Lifted, an Upwork Company™

#111 - Data Engineer

Hyderabad, TS, India

Live, first seen by us 27 days ago

Pay: Pay not listed
Work setting: Not stated
Employment: Contract — employment source
Employment type in the employer’s posting: Contract
From the employer’s posting

What you’ll work on

Full posting

Migrate pipelines from Apache Airflow 2 to Apache Airflow 3
Migrate pipelines from Hive to Iceberg using Snowpark Connect
Migrate Confluent connections between cloud regions
Adapt existing data pipelines to Snowflake while keeping data quality and reliability
Support workflow scheduling and monitoring in Apache Airflow

From the employer’s posting

Migrate pipelines from Apache Airflow 2 to Apache Airflow 3.

Migrate pipelines from Hive to Iceberg using Snowpark Connect.

Migrate Confluent connections between cloud regions.

Adapt existing data pipelines to Snowflake while maintaining data quality and reliability.

Support workflow scheduling and monitoring in Apache Airflow.

What you’ll bring

All qualifications

Core experience

2+ years of experience with large-scale datasets, including Airflow and Spark
Advanced SQL, including window functions and performance tuning
Hands-on Python development experience
Apache Airflow pipeline development and testing
Apache Kafka and hands-on PySpark experience

Qualification wording

At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.

Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.

Hands-on Python development experience.

Apache Airflow experience developing and testing pipelines in Airflow and Python.

Apache Kafka experience.

Apply at Lifted, an Upwork Company™

Posting record

Report an error

Status in our records: Live
First seen by us: Oct 8, 202627 days ago
Recorded sightings: 136
Last seen by us: Nov 4, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 27

Similar jobs usually come down between day 5 and day 54.

From 10,733 similar postings that came down. Coming down is not always a hire.

Lifted, an Upwork Company™ is hiring

6 open jobs

Open jobs by the week we first saw them, last 12 weeks.

Data Engineer hiring is steady
Takes jobs down quickly

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

Python
SQL
AWS
Iceberg
Kafka
S3
Snowflake
Spark
Airflow
PySpark

Where the posting names them

- Strong software engineering fundamentals. - Hands-on Python development experience. - Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.

- Snowflake data modeling. - Apache Airflow experience developing and testing pipelines in Airflow and Python. - Apache Kafka experience.

Required Tools & Platforms - Python and SQL. - Snowflake, Apache Airflow, Apache Kafka, and Apache Spark/PySpark.

- Hands-on Python development experience. - Advanced SQL, including complex joins, CTEs, window functions, and performance tuning. - Snowflake data modeling.

- Experience with Iceberg and Parquet data formats. - AWS experience, particularly S3 data lakes, partitioning, and lifecycle policies. - Testing strategy experience and the ability to debug and test code hands-on.

- Iceberg and Parquet. - AWS S3. - The migration environment also includes Confluent, Hive, and Snowpark Connect.

- Apache Spark experience, including hands-on PySpark. - Experience with Iceberg and Parquet data formats. - AWS experience, particularly S3 data lakes, partitioning, and lifecycle policies.

- Apache Airflow experience developing and testing pipelines in Airflow and Python. - Apache Kafka experience. - Apache Spark experience, including hands-on PySpark.

- Python and SQL. - Snowflake, Apache Airflow, Apache Kafka, and Apache Spark/PySpark. - Iceberg and Parquet.

Job Description We are seeking a hands-on Data Engineer to support a large-scale data platform modernization initiative. You will migrate approximately 1,000 datasets to Snowflake, adapt existing pipelines, and help deliver reliable orchestration with Apache Airflow 3. The architecture and technical approach are already defined, with the role focused on implementation and migration delivery through March 2027. Enterprise experience strongly preferred.

- Migrate pipelines from Hive to Iceberg using Snowpark Connect. - Adapt existing data pipelines to Snowflake while maintaining data quality and reliability. - Support workflow scheduling and monitoring in Apache Airflow.

- Advanced SQL, including complex joins, CTEs, window functions, and performance tuning. - Snowflake data modeling. - Apache Airflow experience developing and testing pipelines in Airflow and Python.

Nice-to-Have Skills - Hands-on Snowflake migration experience, including data warehouse modernization and performance and cost optimization. - Apache Airflow migration experience, including version upgrades, executor changes, or platform migrations.

- Apache Kafka experience. - Apache Spark experience, including hands-on PySpark. - Experience with Iceberg and Parquet data formats.

Key Responsibilities - Migrate pipelines from Apache Airflow 2 to Apache Airflow 3. - Migrate Confluent connections between cloud regions.

- Adapt existing data pipelines to Snowflake while maintaining data quality and reliability. - Support workflow scheduling and monitoring in Apache Airflow. - Partner with the team to execute time-sensitive data migrations.

- Hands-on Snowflake migration experience, including data warehouse modernization and performance and cost optimization. - Apache Airflow migration experience, including version upgrades, executor changes, or platform migrations. - Experience using AI tools for development.

Upload resume Add LinkedIn

Your next step

Complete your application on jobs.smartrecruiters.com. The employer’s form will show what is required.

Applying through SmartRecruiters

Uploading your resume fills in the application form for you, so check what it filled before you send.

Already applied? Track this application

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025: $139,500
Expected change, 2025 to 2035: 9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Job description

View original posting ↗

Job Description

We are seeking a hands-on Data Engineer to support a large-scale data platform modernization initiative. You will migrate approximately 1,000 datasets to Snowflake, adapt existing pipelines, and help deliver reliable orchestration with Apache Airflow 3. The architecture and technical approach are already defined, with the role focused on implementation and migration delivery through March 2027.

Enterprise experience strongly preferred.

Key Responsibilities

- Migrate pipelines from Apache Airflow 2 to Apache Airflow 3.

- Migrate Confluent connections between cloud regions.

- Migrate pipelines from Hive to Iceberg using Snowpark Connect.

- Adapt existing data pipelines to Snowflake while maintaining data quality and reliability.

- Support workflow scheduling and monitoring in Apache Airflow.

- Partner with the team to execute time-sensitive data migrations.

Qualifications

Must-Have Skills

- 2+ years of experience.

- Strong software engineering fundamentals.

- Hands-on Python development experience.

- Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.

- Snowflake data modeling.

- Apache Airflow experience developing and testing pipelines in Airflow and Python.

- Apache Kafka experience.

- Apache Spark experience, including hands-on PySpark.

- Experience with Iceberg and Parquet data formats.

- AWS experience, particularly S3 data lakes, partitioning, and lifecycle policies.

- Testing strategy experience and the ability to debug and test code hands-on.

- At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.

- Strong communication skills.

- Ability to work independently.

Nice-to-Have Skills

- Hands-on Snowflake migration experience, including data warehouse modernization and performance and cost optimization.

- Apache Airflow migration experience, including version upgrades, executor changes, or platform migrations.

- Experience using AI tools for development.

Additional Information

Required Tools & Platforms

- Python and SQL.

- Snowflake, Apache Airflow, Apache Kafka, and Apache Spark/PySpark.

- Iceberg and Parquet.

- AWS S3.

- The migration environment also includes Confluent, Hive, and Snowpark Connect.

Location, Time & Engagement

- Open to candidates in eligible EMEA and APAC locations.

- Coverage until 12:00 PM Pacific Time is required.

- Remote contract engagement for 40 hours per week.

- Expected engagement through March 31, 2027; start date to be confirmed.

- Two openings are available.

- This is not currently a contract-to-hire opportunity.

Source details