← back to jobs
> job detail
U
👽Other

Principal Data & AI Platform Architect – Azure Databricks

United Vein & Vascular Centers · Tampa, Florida, United States
// classified as
Other (Adjacent or hard to classify.)
posted
1d ago
location
Tampa, Florida, United States
languages
python, sql
tools
azure, databricks, delta
> stack
pythonsqlazuredatabricksdeltasparkpyspark
> education
master
> description
<p><span style="font-family: georgia, palatino, serif;">Reporting to the Senior Director of Data &amp; AI, the Senior Data Architect is a hands-on technical leader responsible for the architecture, engineering, governance, and optimization of UVVC’s enterprise data platform. This role will define technical standards and design scalable data pipelines, medallion data products, semantic models, and governed analytics solutions that enable trusted self-service analytics and AI-driven insights across UVVC.</span></p> <p><span style="text-decoration: underline;"><strong><span style="font-family: georgia, palatino, serif;">Responsibilities&nbsp;</span></strong></span></p> <p><span style="font-family: georgia, palatino, serif;"><strong>Data Pipeline Development &amp; Orchestration</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Design, build, and optimize batch and streaming ETL/ELT pipelines and reusable ingestion frameworks using Azure Data Factory and Databricks across APIs, databases, SaaS platforms, and internal systems.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Build scalable Delta Lake transformation frameworks using medallion architecture, Spark, and SQL.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Implement CI/CD, parameterization, triggers, and pipeline automation best practices.</span></li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>Azure Data Platform Engineering</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Architect, manage, and optimize enterprise data environments across Azure Data Lake Storage Gen2 (ADLS Gen2), Azure SQL, and Databricks, including serverless and classic compute strategies, cost governance, and workload isolation strategies.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Implement DataOps practices including testing, version control, monitoring, and documentation.</span></li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>&nbsp;</strong></span><span style="font-family: georgia, palatino, serif;"><strong>Unity Catalog, Security &amp; Data Governance </strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Design and administer the enterprise Unity Catalog structure, including catalogs, schemas, external locations, storage credentials, groups, service principals, and ownership models.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Implement least-privilege access, governed tags, attribute-based access-control policies, row-level filters, and column-level masking for PHI, PII, financial, and other sensitive information.</span></li> </ul> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Establish standards for data classification, lineage, auditability, stewardship, retention, certification, and access reviews.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Partner with Security, Compliance, Privacy, and business data owners to ensure data solutions align with HIPAA and organizational security requirements.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Govern tables, views, volumes, functions, metric views, dashboards, models, and Genie Agents through Unity Catalog.</span></li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>AI/BI, Semantic Models &amp; Genie Agents&nbsp;</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Design and develop Databricks AI/BI Dashboards and domain-specific Genie Agents for clinical, operational, financial, RCM, marketing, and executive use cases.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Configure trusted datasets, joins, business terminology, instructions, example queries, dimensions, measures, synonyms, and approved KPI definitions.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Develop and govern Unity Catalog metric views and semantic definitions to ensure consistent reporting across Databricks AI/BI and Power BI.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Establish testing and monitoring processes for Genie Agent accuracy, data grounding, security, explainability, performance, and user adoption.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Ensure that AI-generated results respect Unity Catalog permissions and approved business definitions.</span></li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>Databricks Architecture &amp; Platform Ownership</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Design and implement enterprise-grade Databricks Lakehouse Medallion architecture (Bronze, Silver, Gold layers).</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Define and enforce data engineering standards, naming conventions, and architectural patterns across all pipelines.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Lead the architecture of Delta Lake design patterns, including partitioning, optimization, and data lifecycle management.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Establish scalable serverless and classic compute strategies, job orchestration frameworks, and workspace organization.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Evaluate and implement new Databricks capabilities and ensure alignment with enterprise data strategy<strong>.</strong></span></li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>&nbsp;</strong></span><span style="font-family: georgia, palatino, serif;"><strong>Cross-Functional Collaboration</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Work closely with clinical, sales, marketing, finance, RCM, operations, HR, and IT teams to understand business needs.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Provide technical guidance on data engineering patterns and platform capabilities.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Clearly communicate progress, risks, and technical decisions to data stakeholders and leadership.</span></li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>Data Modeling and Enterprise Metrics</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Develop conceptual, logical, dimensional, and physical data models supporting clinical, operational, financial, marketing, RCM, and workforce analytics.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Establish conformed dimensions, master and reference data standards, governed KPIs, and reusable semantic definitions.</span></li> <li style="font-family: georgia, palatino, serif;">Reduce conflicting calculations and duplicate business logic across Databricks AI/BI, Power BI, and downstream applications.</li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>Data Quality and Observability</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Establish data-quality rules, reconciliation controls, data SLAs, pipeline observability, alerting, incident response, and root-cause-analysis processes.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Define standards for schema evolution, change data capture, late-arriving data, historical tracking, retries, and recovery.</span></li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>Performance and FinOps</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Monitor and optimize Databricks consumption using system tables, workload tagging, budget controls, query profiling, serverless and classic compute selection, SQL warehouse configuration, and data-layout optimization.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Establish cost allocation and accountability by environment, data product, department, and workload.</span></li> </ul> <p><span style="text-decoration: underline;"><strong><span style="font-family: georgia, palatino, serif;">Qualifications&nbsp;</span></strong></span></p> <p><span style="font-family: georgia, palatino, serif;"><strong>Required Qualifications</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">8+ years of progressive experience in data engineering, data architecture, analytics engineering, or cloud data platforms, including demonstrated ownership of production Databricks architecture.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Demonstrated technical leadership and mentoring experience within data engineering or architecture teams.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Hands-on expertise with Azure Data Factory, including pipelines, mapping data flows, Integration Runtime configuration and management, triggers, and monitoring.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Hands-on expertise with Azure Databricks, including notebooks, Apache Spark, Delta Lake, Databricks SQL, Lakeflow Jobs, Lakeflow pipelines, and workflow orchestration.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Advanced SQL expertise, including complex transformations, dimensional and semantic modeling, query-plan analysis, Delta Lake optimization, Databricks SQL, and SQL warehouse performance tuning.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Advanced proficiency in Python and PySpark for data engineering, reusable framework development, automation, testing, and performance optimization.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Deep expertise in Medallion Lakehouse architecture (Bronze/Silver/Gold) and Delta Lake optimization techniques.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Demonstrated experience designing, implementing, and operating enterprise Databricks environments across development, testing, and production, including security, governance, deployment, performance, and cost management responsibilities.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Strong understanding of Databricks Unity Catalog, data governance, and security models.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Strong understanding of HIPAA, PHI/PII safeguards, least-privilege access, data retention, auditability, and secure healthcare data integration.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Experience defining data platform standards, frameworks, and best practices</span></li> </ul> <p><span style="font-family: georgia, palatino, serif;"><strong>Preferred Qualifications</strong></span></p> <ul> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Experience with AI/ML workflows, feature engineering, or model enablement.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Experience integrating data across EHR/EMR, CRM, patient-engagement, contact-center, marketing, finance/ERP, HRIS, and revenue-cycle platforms.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Experience designing enterprise data models and governed KPIs for healthcare operations, including patient volume, referrals, scheduling, conversion, provider productivity, revenue cycle, payer performance, denials, collections, labor, and clinic-level financial performance.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Familiarity with real-time processing (Structured Streaming) within Databricks.</span></li> <li style="font-family: georgia, palatino, serif;"><span style="font-family: georgia, palatino, serif;">Experience with master data management, reference data, and entity-resolution strategies across patients, providers, locations, payers, legal entities, and acquired practices.</span></li> </ul><div class="content-conclusion"><p><span style="font-size: 12pt; font-family: georgia, palatino, serif;"><strong>About us:&nbsp;&nbsp;</strong></span></p> <p><span style="font-size: 12pt; font-family: georgia, palatino, serif;">UVVC, is a leading provider of comprehensive vein and vascular care with over 60 clinics across Arizona, Illinois, Colorado, Florida, Georgia, Texas, and expanding. Our mission is to revolutionize vascular care by delivering an all-inclusive clinic experience that addresses every aspect of lower extremity vein, vascular, and wound conditions.&nbsp;</span></p> <p><span style="font-size: 12pt; font-family: georgia, palatino, serif;">United Vein &amp; Vascular Centers (UVVC) is distinguished by its innovative approach to diagnosing and treating a variety of vascular conditions that affect the pelvis and lower extremities. With a team of committed specialists, cutting-edge medical technology, and a patient-centric approach that emphasizes minimally invasive procedures, UVVC ensures superior care and optimal outcomes for it’s patients. &nbsp;</span></p> <p>&nbsp;</p></div>