Back to jobs

Senior Data Engineer

Malaysia - Kuala Lumpur

Live, first seen by us 37 days ago
Pay
Pay not listed
Work setting
Hybrid — work setting source
This role is not eligible for relocation Remote Type: This position is a hybrid of office/remote working Skills: Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}.
Read the full posting
Employment
Full-time — employment source
Employment type in the employer’s posting: Full time
From the employer’s posting

What you’ll work on

Full posting
  • Lead the design, development, migration, and operation of scalable data pipelines and data products using Databricks, PySpark, Spark SQL, and AWS.

  • Own the squad’s data engineering domain, providing technical direction and ensuring solutions are reliable, supportable, secure, and aligned with business needs.

  • Design and implement scalable data ingestion and migration frameworks using technologies such as Databricks Auto Loader and Delta Live Tables (DLT).

From the employer’s posting
Key Responsibilities Lead the design, development, migration, and operation of scalable data pipelines and data products using Databricks, PySpark, Spark SQL, and AWS. Own the squad’s data engineering domain, providing technical direction and ensuring solutions are reliable, supportable, secure, and aligned with business needs.
Lead the design, development, migration, and operation of scalable data pipelines and data products using Databricks, PySpark, Spark SQL, and AWS. Own the squad’s data engineering domain, providing technical direction and ensuring solutions are reliable, supportable, secure, and aligned with business needs. Design and implement scalable data ingestion and migration frameworks using technologies such as Databricks Auto Loader and Delta Live Tables (DLT).
Own the squad’s data engineering domain, providing technical direction and ensuring solutions are reliable, supportable, secure, and aligned with business needs. Design and implement scalable data ingestion and migration frameworks using technologies such as Databricks Auto Loader and Delta Live Tables (DLT). Develop and maintain data models, transformation pipelines, and storage solutions with strong focus on data quality, integrity, performance, and scalability.

About BP

At bp, we provide the following environment and benefits to you:

In the employer’s words · Read in context

Apply at BP

Posting record

Report an error
Status in our records
Live
First seen by us
Oct 15, 202637 days ago
Recorded sightings
29
Last seen by us
Nov 16, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 37

Similar jobs usually come down between day 6 and day 56.

Well past BP’s usual: its jobs come down by about day 13.

From 14,770 similar postings that came down, and 129 of BP’s. Coming down is not always a hire.

BP is hiring

8 open jobs

Open jobs by the week we first saw them, last 12 weeks.

  • Data Engineer hiring has slowed
  • Takes jobs down quickly
Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • Python
  • SQL
  • AWS
  • Databricks
  • Delta Lake
  • NoSQL
  • S3
  • Spark
  • PySpark
Where the posting names them
- 8+ years of experience in Data Engineering, Software Engineering, or related technical disciplines. - 5+ years of hands-on Python experience and 3+ years working with PySpark and distributed data processing. - Strong hands-on expertise in Databricks, PySpark, Spark SQL, SQL, Delta Lake, and data modelling.
Key Responsibilities - Lead the design, development, migration, and operation of scalable data pipelines and data products using Databricks, PySpark, Spark SQL, and AWS. - Own the squad’s data engineering domain, providing technical direction and ensuring solutions are reliable, supportable, secure, and aligned with business needs.
- 5+ years of hands-on Python experience and 3+ years working with PySpark and distributed data processing. - Strong hands-on expertise in Databricks, PySpark, Spark SQL, SQL, Delta Lake, and data modelling. - Experience designing and operating enterprise-scale data ingestion, transformation, migration, and processing frameworks.
- Build and maintain pipeline orchestration and deployment capabilities using Databricks Workflows, Databricks Asset Bundles (DAB), CI/CD, and Infrastructure as Code. - Integrate data platforms with the AWS ecosystem, including IAM, storage, security, networking, and supporting cloud services. - Implement effective governance, security, lineage, and access controls using Unity Catalog and enterprise standards.
- Hands-on experience with Databricks Auto Loader, Delta Live Tables (DLT), Workflows, and Databricks Asset Bundles (DAB). - Strong AWS experience, including IAM, S3 and integration with AWS cloud services and infrastructure. - Experience with Unity Catalog, including data governance, access control, metadata, and lineage.
- Own the squad’s data engineering domain, providing technical direction and ensuring solutions are reliable, supportable, secure, and aligned with business needs. - Design and implement scalable data ingestion and migration frameworks using technologies such as Databricks Auto Loader and Delta Live Tables (DLT). - Develop and maintain data models, transformation pipelines, and storage solutions with strong focus on data quality, integrity, performance, and scalability.
- Develop and maintain data models, transformation pipelines, and storage solutions with strong focus on data quality, integrity, performance, and scalability. - Build and maintain pipeline orchestration and deployment capabilities using Databricks Workflows, Databricks Asset Bundles (DAB), CI/CD, and Infrastructure as Code. - Integrate data platforms with the AWS ecosystem, including IAM, storage, security, networking, and supporting cloud services.
- Implement effective governance, security, lineage, and access controls using Unity Catalog and enterprise standards. - Establish observability, monitoring, alerting, and cost optimisation across data pipelines and Databricks workloads. - Drive operational excellence through incident resolution, root cause analysis, automation, runbooks, and continuous reliability improvements.
- Experience designing and operating enterprise-scale data ingestion, transformation, migration, and processing frameworks. - Hands-on experience with Databricks Auto Loader, Delta Live Tables (DLT), Workflows, and Databricks Asset Bundles (DAB). - Strong AWS experience, including IAM, S3 and integration with AWS cloud services and infrastructure.
This position is a hybrid of office/remote working Skills: Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}. 
 Legal Disclaimer:

Your next step

Complete your application on bpinternational.wd3.myworkdayjobs.com. The employer’s form will show what is required.

Applying through Workday

  • You create an account with each employer before you can upload anything. Every Workday employer has its own account, so one sign-up doesn’t carry over.
  • “Autofill with Resume” comes right after sign-in, before the My Information and My Experience steps. Check every field it fills in those steps, since that’s what recruiters see.
  • Your application status lives in that employer’s Candidate Home, under the same sign-in.

Already applied? Track this application

Has BP posted this job before?

Yes. We’ve followed 2 other Senior Data Engineer jobs there since May 2026. All 2 are still up.

  1. São PauloStill up after 182 days, first seen May 24
  2. SunburyStill up after 45 days, first seen Oct 7
  3. This jobUp 38 days so far, first seen Oct 15

Job description

View original posting ↗

Entity:

Technology


Job Family Group:

IT&S Group


Job Description:

Let me tell you about the role

As the primary Data Engineer within the squad, this role provides end-to-end technical ownership of the data domain. The Senior Data Engineer will design, build, migrate, operate, and continuously improve data products and pipelines, ensuring they are scalable, reliable, secure, observable, and cost-effective.


Key Responsibilities

  • Lead the design, development, migration, and operation of scalable data pipelines and data products using Databricks, PySpark, Spark SQL, and AWS.
  • Own the squad’s data engineering domain, providing technical direction and ensuring solutions are reliable, supportable, secure, and aligned with business needs.
  • Design and implement scalable data ingestion and migration frameworks using technologies such as Databricks Auto Loader and Delta Live Tables (DLT).
  • Develop and maintain data models, transformation pipelines, and storage solutions with strong focus on data quality, integrity, performance, and scalability.
  • Build and maintain pipeline orchestration and deployment capabilities using Databricks Workflows, Databricks Asset Bundles (DAB), CI/CD, and Infrastructure as Code.
  • Integrate data platforms with the AWS ecosystem, including IAM, storage, security, networking, and supporting cloud services.
  • Implement effective governance, security, lineage, and access controls using Unity Catalog and enterprise standards.
  • Establish observability, monitoring, alerting, and cost optimisation across data pipelines and Databricks workloads.
  • Drive operational excellence through incident resolution, root cause analysis, automation, runbooks, and continuous reliability improvements.
  • Provide technical leadership through architecture decisions, code reviews, engineering standards, mentoring, and knowledge sharing.

What you will need to be successful (experience and qualifications)

  • 8+ years of experience in Data Engineering, Software Engineering, or related technical disciplines.
  • 5+ years of hands-on Python experience and 3+ years working with PySpark and distributed data processing.
  • Strong hands-on expertise in Databricks, PySpark, Spark SQL, SQL, Delta Lake, and data modelling.
  • Experience designing and operating enterprise-scale data ingestion, transformation, migration, and processing frameworks.
  • Hands-on experience with Databricks Auto Loader, Delta Live Tables (DLT), Workflows, and Databricks Asset Bundles (DAB).
  • Strong AWS experience, including IAM, S3 and integration with AWS cloud services and infrastructure.
  • Experience with Unity Catalog, including data governance, access control, metadata, and lineage.
  • Experience with CI/CD, Infrastructure as Code, automated testing, Git, and deployment automation.
  • Strong understanding of observability, data quality, performance optimisation, cost monitoring, reliability, and production support.
  • Good understanding of information security, data governance, privacy, resilience, and operational best practices.
  • Strong technical leadership, problem-solving, communication, stakeholder management, and mentoring skills.


About bp

At bp, we provide the following environment and benefits to you:

  • A company culture where we respect our diverse and unified teams, where we are proud of our achievements and where fun and the attitude of giving back to our environment are highly valued.
  • Possibility to join our social communities and networks
  • Learning opportunities and other development opportunities to craft your career path
  • Life and health insurance, medical care package


And many other benefits. We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform crucial job functions, and receive other benefits and privileges of employment.


Travel Requirement

No travel is expected with this role


Relocation Assistance:

This role is not eligible for relocation


Remote Type:

This position is a hybrid of office/remote working


Skills:

Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}

.





Legal Disclaimer:

We are an equal opportunity employer. We do not discriminate on the basis of protected characteristics like race, religion, color, sex, national origin, sexual orientation, veteran status or disability status. Individuals with an accessibility need may request an adjustment/accommodation related to bp’s recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodation related to the recruitment process, please contact us.

If you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 845, the most 1,344, and the latest week had 1,007.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay

No pay amount identified in the saved description.

Location & working pattern

Malaysia - Kuala Lumpur

No travel is expected with this role Relocation Assistance: This role is not eligible for relocation Remote Type: This position is a hybrid of office/remote working Skills: Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}.
Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.