Senior Data Engineer

 

Position Summary

The Senior Data Engineer owns the data platform end to end — designing, building, and maintaining the infrastructure, pipelines, and governed data layers that power analytics and downstream consumers across the organization. This is a hands-on engineering role centered on cloud data platform development, pipeline orchestration, and data modeling at scale.

The guiding principle is to build once, consume everywhere — define transformations and metrics a single time, governed and version-controlled in the warehouse, then serve them consistently to every downstream consumer: BI tools, client-facing reporting, and AI systems.

The ideal candidate is comfortable building data platforms and systems from scratch, thrive in a platform development and maintenance environment, and brings strong hands-on experience with Snowflake and workflow orchestration tools such as Apache Airflow and dbt (or equivalent platforms). This role does not own BI visualization — your work spans raw ingestion through the governed gold layer.

Key Responsibilities

Data Platform Development & Maintenance

• Design, build, and maintain scalable data pipelines and platform infrastructure on Snowflake.
• Architect and implement end-to-end data flows from source ingestion through the silver and gold layers.
• Own platform reliability, performance tuning, and ongoing maintenance — treating the data platform as a product.
• Build systems from the ground up where needed; evaluate existing systems and refactor for scale, cost efficiency, and maintainability.
• Establish engineering standards: version control, CI/CD, testing frameworks, and documentation for all platform components.

Pipeline Orchestration & Workflow Management

• Design and maintain orchestration workflows using Apache Airflow (or equivalent) to schedule, monitor, and manage data pipelines.
• Ensure pipeline reliability, observability, and failure recovery through alerting, logging, and automated retries.
• Coordinate dependencies across upstream sources and downstream consumers with well-defined SLAs.

Gold-Layer Modeling & Metrics

• Design, build, and maintain gold-layer data models and a governed metrics layer in Snowflake using dbt — clean, tested, documented, and version-controlled.
• Establish metric definitions as the single source of truth: calculated once, reused everywhere, with clear ownership and change management.
• Push transformation and calculation logic upstream into the warehouse so the consumption layer stays simple, consistent, and high-trust.
• Partner with analytics and BI stakeholders on the boundary between pipelines (data engineering) and consumption (analytics/reporting).
• Structure and document data models so they meet the governance and consistency requirements of AI agents and autonomous analytical systems.

AI-Native Development

• Build with coding agents and AI-assisted tooling as a primary part of the workflow — accelerating pipeline development, data modeling, documentation, and testing.
• Independently set up and manage development environments, repositories, and the tooling needed to work effectively with agentic systems.
• Help establish team practices and standards for building data infrastructure in the AI era.

Quality, Governance & Platform Judgment

• Champion data quality, consistency, and trust across all platform layers — ensuring data holds up under scrutiny from any downstream consumer.
• Contribute to platform architecture decisions; identify opportunities for automation, templating, and process improvement.
• Apply sound engineering judgment to evaluate build vs. buy decisions and tooling trade-offs.

Qualifications

Required

• 5+ years of hands-on experience in data engineering — building pipelines, data models, and platform infrastructure in a data-driven organization.
• Snowflake: Proven, hands-on production experience building data models and transformations in Snowflake. Deep familiarity with Snowflake architecture, performance optimization, and cost management.
• Workflow Orchestration (must have): Hands-on experience with Apache Airflow or an equivalent DAG-based orchestration platform (e.g., Prefect, Dagster, Mage). Ability to design, deploy, and operate complex pipeline workflows.
• dbt: Demonstrated experience using dbt for transformation modeling, testing, and documentation in a production warehouse environment. Equivalent semantic layer tooling experience considered.
• Strong SQL proficiency — able to write, optimize, and review complex queries across large datasets.
• Experience building data platforms or systems from scratch — not just extending existing infrastructure but designing and standing up new ones.
• Comfort with platform development and maintenance as a long-term ownership model — not just project delivery.
• Fluency working in the AI era: building with coding agents, setting up development environments, and using AI tooling as a core productivity multiplier.
• Familiarity with software engineering practices: version control (Git), CI/CD pipelines, automated testing, and code review.
• Strong communication skills — able to translate business requirements into technical data platform decisions.

Preferred

• Experience in pharmacy benefits management, healthcare technology, or a related regulated industry.
• Experience with cloud infrastructure and data platform tooling (AWS, Azure, or GCP) alongside the data stack.
• Familiarity with data governance frameworks, data contracts, or data mesh principles.
• Experience with streaming or near-real-time data pipelines (e.g., Kafka, Kinesis, Spark Streaming).
• Exposure to semantic/metrics layer tooling (e.g., MetricFlow, Cube, LookML).
• Experience building demo environments or client-facing data products.

Salary Range: 145,000 - 165,000