Senior AI Data Engineer

We never stop learning. We never stop challenging the norm. We push for better, and we celebrate the wins along the way. Here, you’ll get flexibility that’s real, benefits you can count on, and a team that succeeds together. Because at UKG, your work matters—and so do you. Role Overview We are looking for a Lead Data Engineer to lead the design, development, and operation of reliable, scalable data platforms and products. This role combines hands-on engineering with technical leadership, guiding a team to deliver secure, well-governed data pipelines and curated datasets for analytics, reporting, and AI use cases. The ideal candidate brings deep experience with cloud data platforms, distributed processing, and modern data architecture. They can translate business and analytical needs into clear technical designs, set engineering standards, mentor engineers, and work across teams to deliver production-ready data solutions. Experience with GCP, Azure, Databricks, Python, PySpark, SQL, and Generative AI data patterns is valuable. Key Responsibilities • Lead the architecture and delivery of scalable batch and streaming data pipelines, data products, and lakehouse solutions across cloud environments. • Translate business, reporting, analytics, and AI requirements into technical designs, data models, interfaces, and delivery plans. • Design ingestion and transformation patterns for relational databases, APIs, event streams, files, and cloud storage, including structured and semi-structured data. • Build and review production-grade data solutions using Python, PySpark, SQL, Spark, and platforms such as GCP BigQuery, Azure Databricks, and Azure Data Lake. • Establish reusable engineering patterns for orchestration, modular transformations, metadata, schema evolution, incremental processing, and backfills. • Guide data modeling across raw, refined, and curated layers, ensuring datasets are understandable, reusable, performant, and aligned with domain needs. • Define and uphold standards for coding, peer review, testing, version control, CI/CD, release management, and operational readiness. • Own reliability and performance outcomes for critical pipelines, including monitoring, alerting, recovery, capacity planning, and cost optimization. • Implement data quality controls, reconciliation, lineage, and observability so that completeness, accuracy, freshness, and consistency can be measured. • Partner with security, governance, architecture, platform, analytics, and business teams to address access controls, privacy, retention, and compliance requirements. • Lead migration and modernization work from legacy data platforms to cloud environments, including dependency analysis, parity validation, cutover planning, and decommissioning support. • Mentor and coach data engineers; provide technical direction, unblock delivery, and help the team grow its engineering and platform skills. • Break down complex initiatives into milestones, estimate effort, surface risks and dependencies early, and communicate delivery progress to stakeholders. • Investigate and resolve complex production issues, conduct root-cause analysis, and ensure corrective actions prevent recurrence. • Prepare governed, well-documented datasets for AI and Generative AI use cases, including retrieval, feature, and semantic search workflows where appropriate. • Evaluate tools and design options pragmatically, documenting trade-offs and recommending approaches that meet scale, security, cost, and maintainability needs.

Key Responsibilities • Lead the architecture and delivery of scalable batch and streaming data pipelines, data products, and lakehouse solutions across cloud environments. • Translate business, reporting, analytics, and AI requirements into technical designs, data models, interfaces, and delivery plans. • Design ingestion and transformation patterns for relational databases, APIs, event streams, files, and cloud storage, including structured and semi-structured data. • Build and review production-grade data solutions using Python, PySpark, SQL, Spark, and platforms such as GCP BigQuery, Azure Databricks, and Azure Data Lake. • Establish reusable engineering patterns for orchestration, modular transformations, metadata, schema evolution, incremental processing, and backfills. • Guide data modeling across raw, refined, and curated layers, ensuring datasets are understandable, reusable, performant, and aligned with domain needs. • Define and uphold standards for coding, peer review, testing, version control, CI/CD, release management, and operational readiness. • Own reliability and performance outcomes for critical pipelines, including monitoring, alerting, recovery, capacity planning, and cost optimization. • Implement data quality controls, reconciliation, lineage, and observability so that completeness, accuracy, freshness, and consistency can be measured. • Partner with security, governance, architecture, platform, analytics, and business teams to address access controls, privacy, retention, and compliance requirements. • Lead migration and modernization work from legacy data platforms to cloud environments, including dependency analysis, parity validation, cutover planning, and decommissioning support. • Mentor and coach data engineers; provide technical direction, unblock delivery, and help the team grow its engineering and platform skills. • Break down complex initiatives into milestones, estimate effort, surface risks and dependencies early, and communicate delivery progress to stakeholders. • Investigate and resolve complex production issues, conduct root-cause analysis, and ensure corrective actions prevent recurrence. • Prepare governed, well-documented datasets for AI and Generative AI use cases, including retrieval, feature, and semantic search workflows where appropriate. • Evaluate tools and design options pragmatically, documenting trade-offs and recommending approaches that meet scale, security, cost, and maintainability needs.

Required Skills • 6+ years of experience in data engineering, data platform engineering, or a closely related discipline, including ownership of production data systems. • Demonstrated experience leading technical delivery, mentoring engineers, and coordinating work across multiple teams or domains. • Strong programming skills in Python and advanced SQL; hands-on experience with PySpark or Spark for distributed data processing. • Deep understanding of cloud data architecture and lakehouse concepts, including storage, compute, partitioning, file formats, and workload design. • Hands-on experience with one or more major cloud data platforms; GCP and Azure experience is preferred, including BigQuery, Databricks, and Azure Data Lake. • Experience designing and operating reliable ETL/ELT pipelines, workflow orchestration, data transformations, and reusable ingestion frameworks. • Strong data modeling skills, including dimensional modeling and the design of curated datasets for analytics and downstream applications. • Experience with data quality, reconciliation, observability, lineage, metadata, and production monitoring practices. • Working knowledge of security and governance practices such as role-based access, encryption, sensitive data handling, and auditability. • Experience with Git-based development, automated testing, CI/CD, infrastructure or configuration management, and controlled deployments. • Ability to troubleshoot performance and reliability issues across distributed data systems and explain technical findings clearly. • Strong communication, planning, prioritization, and stakeholder management skills; able to make technical topics accessible to non-engineering partners.