Design, build and maintain ETL/ELT pipelines feeding a central data platform
Lead development and improvement of the cloud data platform in Azure
Build datasets for reporting, analytics, dashboards and data products
Set engineering practices such as CI/CD, automated testing and pipeline alerting
Manage integrations with insurer portals and external data providers
From the employer’s posting
Design, develop and maintain enterprise ETL/ELT pipelines, integrating data from case management, finance, CRM and third-party systems into a central data platform.
Lead the development and continuous improvement of our cloud-based data platform within Azure.
Build and maintain datasets that support business reporting, analytics, dashboards and data-enabled products.
Establish and champion engineering best practices, including CI/CD, automated testing, version control, monitoring and alerting on pipeline failures.
Manage integrations with insurer portals and external data providers, ensuring high-quality, timely and secure data flows.
Add your resume and we’ll mark which of these tools you already use, and point out anything in the posting that could rule you out.
Python
SQL
Azure
Where the posting names them
- Advanced SQL development skills.
- Proficiency in Python and modern data pipeline orchestration tools, including their monitoring and automation.
- Track record of building reliable, monitored production-grade data pipelines.
- Azure Data Lake
- Advanced SQL development skills.
- Proficiency in Python and modern data pipeline orchestration tools, including their monitoring and automation.
- Design, develop and maintain enterprise ETL/ELT pipelines, integrating data from case management, finance, CRM and third-party systems into a central data platform.
- Lead the development and continuous improvement of our cloud-based data platform within Azure.
- Ensure data pipelines are scalable, reliable, cost-effective and resilient.
- Proven experience in a senior Data Engineering or Lead Data Engineer role.
- Strong hands-on experience with Microsoft Azure data technologies, including: - Azure Data Factory
- Azure Synapse Analytics
- Strong hands-on experience with Microsoft Azure data technologies, including: - Azure Data Factory
- Azure Synapse Analytics
- Microsoft Fabric
- Microsoft Fabric
- Azure Data Lake
- Advanced SQL development skills.
- Play a key role in our major transformation initiatives and the creation of our modern cloud data platform.
- Work with cutting-edge Azure and AI technologies.
- Influence strategic data decisions across the business.
We are looking for an experienced Data Engineering Lead to play a pivotal role in shaping and delivering our enterprise data platform. This is an exciting opportunity to lead the development of scalable, secure, and AI-ready modern data platform that enable advanced analytics, reporting, automated and AI-enabled processes, and the creation of data powered digital products.
Working closely with Data Architecture, Analytics, and wider business teams, you will be responsible for designing and maintaining robust data pipelines, integrating multiple business systems, and ensuring data remains trusted, accessible, consistent, and fit for purpose.
This role is ideal for a technically strong data engineering professional who enjoys combining hands-on delivery with leadership and continuous improvement.
What you will do?
As our Data Engineering Lead, you will:
Design, develop and maintain enterprise ETL/ELT pipelines, integrating data from case management, finance, CRM and third-party systems into a central data platform.
Lead the development and continuous improvement of our cloud-based data platform within Azure.
Ensure data pipelines are scalable, reliable, cost-effective and resilient.
Build and maintain datasets that support business reporting, analytics, dashboards and data-enabled products.
Establish and champion engineering best practices, including CI/CD, automated testing, version control, monitoring and alerting on pipeline failures.
Work closely with Data Architects to implement target-state data models and modern data engineering principles.
Partner with data scientists and analysts to operationalise advanced analytics and predictive models.
Manage integrations with insurer portals and external data providers, ensuring high-quality, timely and secure data flows.
Support the organisation's ambition to become increasingly data-driven and AI-enabled.
Who you are
Essential Experience & Skills
Proven experience in a senior Data Engineering or Lead Data Engineer role.
Strong hands-on experience with Microsoft Azure data technologies, including:
Azure Data Factory
Azure Synapse Analytics
Microsoft Fabric
Azure Data Lake
Advanced SQL development skills.
Proficiency in Python and modern data pipeline orchestration tools, including their monitoring and automation.
Track record of building reliable, monitored production-grade data pipelines.
Experience integrating data from multiple business-critical applications and legacy systems.
Good understanding of data security, data protection and access control principles within cloud-based data platforms.
Comfortable working in regulated data environments with sensitive personal and litigation data.
Excellent stakeholder management and communication skills.
Desirable Experience
Exposure to predictive analytics, machine learning operationalisation or AI platforms.
Experience working with Dynamics CRM, case management platforms such as VisualFiles, and finance systems such as Elite 3E.
Understanding of data modelling and data architecture best practices.
Experience leading or mentoring data engineering teams.
Experience within insurance, legal services, claims management or financial services is desirable but not mandatory.
Why Join Us?
This is an exciting opportunity to shape the future of data engineering within a growing, data-driven organisation.
You'll have the opportunity to:
Be part of an exciting journey reshaping how data and AI are used in claims management.
Play a key role in our major transformation initiatives and the creation of our modern cloud data platform.
Work with cutting-edge Azure and AI technologies.
Influence strategic data decisions across the business.
Deliver solutions that directly improve operational performance and customer outcomes.
Join a collaborative and forward-thinking Data & Analytics function.
Success Measures
Success in this role will be measured by:
Reliability and performance of data pipelines and platform services.
Delivery of high-quality, trusted data products.
Reduction in data-related incidents and support issues.
Improved data accessibility and adoption across the business.
Successful implementation of engineering best practices and automation.
Enablement of analytics, reporting and AI initiatives through scalable data pipelines and architecture.
Ready to Apply?
If you are passionate about building modern data platforms and turning complex data challenges into business value, we'd love to hear from you. Join us and play a key role in shaping the future of data across the organisation.
What's in it for you?
High levels of flexibility and a great work life balance - https://www.dacbeachcroft.com/en/Work-with-us/Whats-in-it-for-you
A well-rounded remuneration package (which includes private medical insurance, income protection insurance and discounted gym membership, amongst many other benefits)
Opportunities for growth and progression including professional funding
In person and remote social events
Opportunity to get involved in a range of Environmental, Social and Governance (ESG) activities
We are dedicated to building a diverse, inclusive and authentic workplace, which aligns closely to our cultural principles (Determined, Clear, Creative and Supportive). If you are excited about this role and being part of our culture, but your past experience does not align perfectly with every qualification in the job description, we encourage you to apply anyway. You may be just the right candidate for this or other roles!
Employment type
Permanent
Demand for Data Engineer roles
New Data Engineer postings at the employers we follow are up 13% over the last 6 weeks compared with the 6 before.
August 17Last week
Each bar is one week of new postings. The darkest bars are the last 6 weeks.
Counted from the hiring pages we’ve followed for at least five weeks, so employers we add don’t look like growth. Updated weekly.
About the role
We file this job under Data Engineer, by the work the posting describes.
Data engineers build and run the pipelines that copy data out of apps and outside services into a warehouse or lake, and fix them when a source changes or a load fails.
Expect SQL, Python, a cloud platform, a scheduler such as Airflow, and often tools for large data such as Spark or Kafka.
An analytics engineer works one step later, turning the raw data a data engineer lands into clean tables people can query.
US government figures for database architects
Median yearly pay, May 2025
$139,500
Expected change, 2025 to 2035
9.4% more jobsagainst 3.5% for all jobs
The government has no code for data engineers, so these figures are for the closest group, database architects, which includes data integration and data warehousing specialists.
Selected passages from the saved posting. Check the full description for conditions and exceptions.
Pay
No pay amount identified in the saved description.
Location & working pattern
London, City of London, London - Woolgate
- Opportunities for growth and progression including professional funding
- In person and remote social events
- Opportunity to get involved in a range of Environmental, Social and Governance (ESG) activities
Work authorization
No clear work-authorization passage found. Eligibility is unconfirmed.