Back to jobs

Lead Data Engineer

Pay
£56,058–60,393/year — pay source
Lead Data Engineer Salary: £56,058 –£60,393 per annum (Grade 9) Directorate: IT, Digital and Customer Services Service: Digital Reports to: Head of User-Centred Design & Digital Experience
Read the full posting
Work setting
Not stated
Employment
Not stated

What you’ll work on

Full posting

Working within our Digital function, you'll take a leading, hands-on role in developing and modernising Havering's data estate, with Microsoft Fabric as our strategic data and analytics platform.

You'll work closely with data engineers, analysts, architects, software developers, service teams and external partners to deliver reliable solutions, improve engineering practices and support the transition from legacy technologies.

  • Develop and maintain the Microsoft Fabric environment, including pipelines, lakehouses and warehouses

  • Assess and migrate legacy SQL Server warehouses, SSIS packages and data marts in phases

  • Write and tune complex T-SQL, stored procedures and views; investigate data issues

  • Improve engineering practice with Git, Azure DevOps, CI/CD and automated testing

  • Mentor colleagues and review technical work across the data estate

From the employer’s posting
Working within our Digital function, you'll take a leading, hands-on role in developing and modernising Havering's data estate, with Microsoft Fabric as our strategic data and analytics platform.
You'll work closely with data engineers, analysts, architects, software developers, service teams and external partners to deliver reliable solutions, improve engineering practices and support the transition from legacy technologies.
Developing, optimising and maintaining our Microsoft Fabric environment, including OneLake, Data Factory pipelines, lakehouses, warehouses
Assessing and prioritising existing SQL Server data warehouses, SSIS packages, data marts and integrations
Writing and tuning complex T-SQL, stored procedures and views, improving performance and investigating data issues through root-cause analysis
Improving engineering practices through Git, Azure DevOps, CI/CD, automated testing and Microsoft Fabric deployment pipelines
mentoring colleagues, reviewing technical work and promoting consistent, maintainable engineering practices across the data estate

What you’ll bring

All qualifications

Core experience

  • Strong experience developing and supporting SQL Server data warehouses, data marts and ETL/ELT in production
Qualification wording
Strong experience developing and supporting SQL Server data warehouses, data marts and ETL/ELT processes in production environments.
Apply at Elfy

Posting record

Report an error

Actively hiringFirst seen by us 1 day ago, still listed, and not a repost of an earlier posting.

Status in our records
Live
First seen by us
Nov 20, 20261 day ago
Recorded sightings
3
Last seen by us
Nov 22, 2026

These dates show when we found the posting. Check the employer’s website to confirm it is still accepting applications.

How this job compares

Time since we first saw it

Day 1

Early. Similar jobs usually come down between day 6 and day 56.

From 14,771 similar postings that came down. Coming down is not always a hire.

Elfy is hiring

1 open job

Open jobs by the week we first saw them, last 12 weeks.

Hiring record

See how this role fits your experience

Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.

  • C#
  • Python
  • SQL
  • T-SQL
  • Azure
  • Databricks
  • Delta Lake
  • Power BI
  • Spark
  • SQL Server
  • SSIS
  • PySpark
Where the posting names them
- SQL development and optimisation: Writing and tuning complex T-SQL, stored procedures and views, improving performance and investigating data issues through root-cause analysis. - Master data and existing applications: Supporting person-matching services, golden-record management, UPRN-based address matching and the Local Land and Property Gazetteer, alongside existing ASP.NET/C# applications and opportunities for modernisation. - Data governance and security: Embedding data quality, ownership, metadata management, lineage and appropriate access controls into engineering solutions. Working with Information Governance and Data Protection colleagues to protect sensitive information, support DPIAs and meet UK GDPR requirements, while developing the use of Microsoft Fabric and Purview governance capabilities.
- SQL Server Agent, stored procedures and data marts - ASP.NET/C# web applications - Master data management, person matching and address gazetteer services
- Dynamics 365, Dataverse, Power Apps or Power Automate. - ASP.NET/C# applications connected to SQL Server. - Master data management, person matching, UPRN and address data management.
- Master data management, person matching, UPRN and address data management. - Python, PowerShell or Azure Logic Apps. - Local government or other public sector data environments.
- Microsoft Fabric and data engineering: Developing, optimising and maintaining our Microsoft Fabric environment, including OneLake, Data Factory pipelines, lakehouses, warehouses, Spark notebooks, Delta tables and Power BI semantic models. - Legacy migration and modernisation: Assessing and prioritising existing SQL Server data warehouses, SSIS packages, data marts and integrations; designing appropriate migration and modernisation approaches; and delivering phased improvements with robust testing, data validation and service continuity. - Data estate structure and engineering standards: Helping develop consistent approaches to workspace organisation, development, test and production environments, access management, source control, automated deployment, monitoring and operational management.
- Data estate structure and engineering standards: Helping develop consistent approaches to workspace organisation, development, test and production environments, access management, source control, automated deployment, monitoring and operational management. - Data warehouse and integration: Developing, maintaining and improving SQL Server data warehouses and ETL/ELT processes, integrating data from cloud and on-premises databases, Dynamics 365, Dataverse, APIs, SFTP and other business systems. - SQL development and optimisation: Writing and tuning complex T-SQL, stored procedures and views, improving performance and investigating data issues through root-cause analysis.
- Data warehouse and integration: Developing, maintaining and improving SQL Server data warehouses and ETL/ELT processes, integrating data from cloud and on-premises databases, Dynamics 365, Dataverse, APIs, SFTP and other business systems. - SQL development and optimisation: Writing and tuning complex T-SQL, stored procedures and views, improving performance and investigating data issues through root-cause analysis. - Master data and existing applications: Supporting person-matching services, golden-record management, UPRN-based address matching and the Local Land and Property Gazetteer, alongside existing ASP.NET/C# applications and opportunities for modernisation.
Future data environment - Microsoft Fabric, including OneLake, Data Factory, Lakehouse, Warehouse, PySpark, Spark SQL and Delta tables - Microsoft Azure data and integration services
Existing data environment - Microsoft SQL Server, T-SQL and SSIS - SQL Server Agent, stored procedures and data marts
- Microsoft SQL Server, T-SQL and SSIS - SQL Server Agent, stored procedures and data marts - ASP.NET/C# web applications
You'll bring: - Strong experience developing and supporting SQL Server data warehouses, data marts and ETL/ELT processes in production environments. - Advanced T-SQL skills, including query optimisation, stored procedures and troubleshooting.
- Strong experience developing and supporting SQL Server data warehouses, data marts and ETL/ELT processes in production environments. - Advanced T-SQL skills, including query optimisation, stored procedures and troubleshooting. - Experience building and maintaining reliable data pipelines, including SSIS or comparable integration technologies.
Experience in the following areas would be advantageous: - Hands-on Microsoft Fabric experience, including lakehouses, warehouses, PySpark, Spark SQL and Delta Lake. - Microsoft Purview and data governance tooling.
- Platform performance and reliability: Monitoring and optimising data pipelines, workloads and platform performance, supporting capacity and cost efficiency, troubleshooting production issues and implementing reliable operational processes. - DevOps and automation: Improving engineering practices through Git, Azure DevOps, CI/CD, automated testing and Microsoft Fabric deployment pipelines. - Technical leadership and collaboration: Taking ownership of engineering deliverables, contributing to technical decisions, mentoring colleagues, reviewing technical work and promoting consistent, maintainable engineering practices across the data estate.
- Microsoft Fabric, including OneLake, Data Factory, Lakehouse, Warehouse, PySpark, Spark SQL and Delta tables - Microsoft Azure data and integration services - Power BI and semantic models
- Microsoft Purview - Azure DevOps, Git and CI/CD - Dynamics 365, Dataverse and Power Platform
- Experience building and maintaining reliable data pipelines, including SSIS or comparable integration technologies. - Experience with Microsoft Fabric, Azure Data Factory, Databricks or another modern cloud data engineering platform. - Experience modernising or migrating legacy data solutions to cloud-based environments.
- Experience integrating data from databases, business applications, APIs and file-based sources. - Familiarity with Git, Azure DevOps or equivalent source control and deployment practices. - An understanding of data governance, information security and UK GDPR, including handling sensitive information.
Your key areas of focus will include: - Microsoft Fabric and data engineering: Developing, optimising and maintaining our Microsoft Fabric environment, including OneLake, Data Factory pipelines, lakehouses, warehouses, Spark notebooks, Delta tables and Power BI semantic models. - Legacy migration and modernisation: Assessing and prioritising existing SQL Server data warehouses, SSIS packages, data marts and integrations; designing appropriate migration and modernisation approaches; and delivering phased improvements with robust testing, data validation and service continuity.
- Technical leadership and collaboration: Taking ownership of engineering deliverables, contributing to technical decisions, mentoring colleagues, reviewing technical work and promoting consistent, maintainable engineering practices across the data estate. - Future digital capabilities: Establishing trusted, reusable data foundations for analytics, Power BI, Power Platform, joined-up digital services, automation and responsible AI adoption. Technology you'll work with
- Microsoft Azure data and integration services - Power BI and semantic models - Microsoft Purview
- Advanced T-SQL skills, including query optimisation, stored procedures and troubleshooting. - Experience building and maintaining reliable data pipelines, including SSIS or comparable integration technologies. - Experience with Microsoft Fabric, Azure Data Factory, Databricks or another modern cloud data engineering platform.

Your next step

Complete your application on elfy.fa.em3.oraclecloud.com. The employer’s form will show what is required.

Applying through Oracle Recruiting

  • Applying starts by asking for your email address.
  • Each employer sets which files it accepts, so read the note beside the upload field.

Already applied? Track this application

Job description

View original posting ↗

Lead Data Engineer

Salary: £56,058 –£60,393 per annum (Grade 9)

Directorate: IT, Digital and Customer Services
Service: Digital
Reports to: Head of User-Centred Design & Digital Experience

Job Description Help shape Havering's modern, connected and secure data estate

The London Borough of Havering is modernising how we manage, connect and use data across our services, with Microsoft Fabric at the heart of our future data and analytics architecture.

We're looking for an experienced, hands-on Lead Data Engineer to help develop and mature our Microsoft data environment, drive the phased migration and consolidation of legacy data platforms, and embed modern engineering and governance practices.

Our existing corporate data warehouse brings together information from more than 30 council and supplier systems, supporting essential services including housing, social care, education, council tax, benefits, electoral registration and parking. This data underpins day-to-day operations, statutory reporting, fraud prevention and emergency response.

Our ambition is to create a more integrated, scalable and well-governed data estate, reducing complexity, improving data quality and accessibility, and enabling better use of information across the Council.

This is an opportunity to contribute to meaningful data modernisation, combining hands-on engineering with the chance to influence how Havering's data capabilities develop over the coming years.

About the role

Working within our Digital function, you'll take a leading, hands-on role in developing and modernising Havering's data estate, with Microsoft Fabric as our strategic data and analytics platform.

You'll work closely with data engineers, analysts, architects, software developers, service teams and external partners to deliver reliable solutions, improve engineering practices and support the transition from legacy technologies.

You'll take ownership of technical deliverables, contribute to solution design and engineering decisions, and help ensure our data services remain reliable, secure and maintainable as the environment evolves.

Your key areas of focus will include:

  • Microsoft Fabric and data engineering: Developing, optimising and maintaining our Microsoft Fabric environment, including OneLake, Data Factory pipelines, lakehouses, warehouses, Spark notebooks, Delta tables and Power BI semantic models.
  • Legacy migration and modernisation: Assessing and prioritising existing SQL Server data warehouses, SSIS packages, data marts and integrations; designing appropriate migration and modernisation approaches; and delivering phased improvements with robust testing, data validation and service continuity.
  • Data estate structure and engineering standards: Helping develop consistent approaches to workspace organisation, development, test and production environments, access management, source control, automated deployment, monitoring and operational management.
  • Data warehouse and integration: Developing, maintaining and improving SQL Server data warehouses and ETL/ELT processes, integrating data from cloud and on-premises databases, Dynamics 365, Dataverse, APIs, SFTP and other business systems.
  • SQL development and optimisation: Writing and tuning complex T-SQL, stored procedures and views, improving performance and investigating data issues through root-cause analysis.
  • Master data and existing applications: Supporting person-matching services, golden-record management, UPRN-based address matching and the Local Land and Property Gazetteer, alongside existing ASP.NET/C# applications and opportunities for modernisation.
  • Data governance and security: Embedding data quality, ownership, metadata management, lineage and appropriate access controls into engineering solutions. Working with Information Governance and Data Protection colleagues to protect sensitive information, support DPIAs and meet UK GDPR requirements, while developing the use of Microsoft Fabric and Purview governance capabilities.
  • Platform performance and reliability: Monitoring and optimising data pipelines, workloads and platform performance, supporting capacity and cost efficiency, troubleshooting production issues and implementing reliable operational processes.
  • DevOps and automation: Improving engineering practices through Git, Azure DevOps, CI/CD, automated testing and Microsoft Fabric deployment pipelines.
  • Technical leadership and collaboration: Taking ownership of engineering deliverables, contributing to technical decisions, mentoring colleagues, reviewing technical work and promoting consistent, maintainable engineering practices across the data estate.
  • Future digital capabilities: Establishing trusted, reusable data foundations for analytics, Power BI, Power Platform, joined-up digital services, automation and responsible AI adoption.

Technology you'll work with

Our strategic direction is centred on the Microsoft ecosystem, with Microsoft Fabric forming the foundation of our future data and analytics platform.

Future data environment

  • Microsoft Fabric, including OneLake, Data Factory, Lakehouse, Warehouse, PySpark, Spark SQL and Delta tables
  • Microsoft Azure data and integration services
  • Power BI and semantic models
  • Microsoft Purview
  • Azure DevOps, Git and CI/CD
  • Dynamics 365, Dataverse and Power Platform

Existing data environment

  • Microsoft SQL Server, T-SQL and SSIS
  • SQL Server Agent, stored procedures and data marts
  • ASP.NET/C# web applications
  • Master data management, person matching and address gazetteer services
  • APIs, SFTP, files and integrations across on-premises and cloud systems

About you

We're looking for a skilled and proactive data engineer who enjoys solving complex problems, takes ownership of technical delivery and is motivated by the opportunity to modernise a significant data environment.

You'll bring:

  • Strong experience developing and supporting SQL Server data warehouses, data marts and ETL/ELT processes in production environments.
  • Advanced T-SQL skills, including query optimisation, stored procedures and troubleshooting.
  • Experience building and maintaining reliable data pipelines, including SSIS or comparable integration technologies.
  • Experience with Microsoft Fabric, Azure Data Factory, Databricks or another modern cloud data engineering platform.
  • Experience modernising or migrating legacy data solutions to cloud-based environments.
  • Good knowledge of data modelling, integration patterns, data quality and performance optimisation.
  • Experience integrating data from databases, business applications, APIs and file-based sources.
  • Familiarity with Git, Azure DevOps or equivalent source control and deployment practices.
  • An understanding of data governance, information security and UK GDPR, including handling sensitive information.
  • Strong problem-solving and communication skills, with the ability to work collaboratively across technical and service teams.

Experience in the following areas would be advantageous:

  • Hands-on Microsoft Fabric experience, including lakehouses, warehouses, PySpark, Spark SQL and Delta Lake.
  • Microsoft Purview and data governance tooling.
  • Dynamics 365, Dataverse, Power Apps or Power Automate.
  • ASP.NET/C# applications connected to SQL Server.
  • Master data management, person matching, UPRN and address data management.
  • Python, PowerShell or Azure Logic Apps.
  • Local government or other public sector data environments.
  • Relevant Microsoft certifications, such as DP-600 or DP-700.

We don't expect you to be an expert in every technology listed. What matters most is strong data engineering expertise, sound technical judgement, a commitment to delivering reliable solutions and a willingness to embrace new technologies as our data estate evolves.

Why join us?

You'll have the opportunity to help shape Havering's future data estate, working with modern Microsoft technologies while contributing to the migration and modernisation of existing systems.

Your work will support essential public services, improve how information is shared and used, and help establish the foundations for better analytics, digital services and responsible AI adoption.

You'll be part of a collaborative Digital function, working alongside engineers, architects, analysts and service teams, with opportunities to develop your skills, influence technical decisions and deliver meaningful improvements.

If you enjoy solving complex data challenges, working with modern technologies and making a practical difference through better use of data, we'd welcome your application.

Please view Job Profile here

For an informal discussion about the post, please contact Norbert.Stawczyk@havering.gov.uk

The closing date for applications is 24 December 2026. We reserve the right to close the vacancy early if there is significant interest.

Successful applicants will be contacted with details of the next stage of the recruitment process.

Additional Information

We want everyone to choose Havering. When you apply for a job with us, your application is considered on its merits regardless of your age, disability, ethnicity, faith, gender identity or sexual orientation. Our residents and service users come from all walks of life, and so do our employees. Find out what it means to Choose Havering.

The London Borough of Havering has important responsibilities for safeguarding and promoting the welfare of children, young people and vulnerable adults. If you are appointed to a job that involves working with these groups, you may be subject to a Disclosure and Barring Service (DBS) check.

Please attach your supporting statement, explaining in no more than two A4 sides why and how you meet the criteria for this role.

We practise anonymised recruitment. Please ensure that you remove all personal information from any documents that you upload.

Demand for Data Engineer roles

New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.

Weekly new Data Engineer postings at the same employers, from the week of August 17 to the week of November 9. The fewest in a week was 845, the most 1,344, and the latest week had 1,007.
August 17Last week

Each bar is one week of new postings. The darkest bars are the last 6 weeks.

Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.

About the role

We file this job under Data Engineer, by the work the posting describes.

Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.

Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.

An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.

US government figures for database architects

Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs

The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.

Source: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics (May 2025) and Employment Projections (2025 to 2035). Occupation code 15-1243.

Source details

View original posting ↗

Source notes

Source excerpts

Selected passages from the saved posting. Check the full description for conditions and exceptions.

Pay
Lead Data Engineer Salary: £56,058 –£60,393 per annum (Grade 9) Directorate: IT, Digital and Customer Services Service: Digital Reports to: Head of User-Centred Design & Digital Experience
Location & working pattern

Romford, United Kingdom

Working pattern and location restrictions need checking in the full posting.

Work authorization

No clear work-authorization passage found. Eligibility is unconfirmed.

Find answers in the posting

AI
How answers work

AI selects complete passages from this posting. Check them for conditions and exceptions.

Uses this posting and your question. No profile needed.