Senior AI Data Engineer | LATAM

- Production experience orchestrating data pipelines with Apache Airflow or Dagster, including dependencies, backfills, retries, and alerting; you understand the trade-off between task-based and asset-based orchestration and work comfortably in platform-native schedulers such as Lakeflow Jobs and Fabric Data Factory. - Expert SQL and strong Python and PySpark, with production experience in dbt or platform-native pipelines and in ingestion tools (Fivetran, Lakeflow Connect, Azure or Fabric Data Factory, or dlt); you use coding agents such as Claude Code daily and review everything they generate. - Experience authoring a semantic or metrics layer (dbt Semantic Layer, Power BI semantic models, Cube, LookML) and turning business definitions into governed metrics that people and agents both rely on.

- Lead data readiness assessments: inventory source systems, map fields and business logic, and document gaps and remediation steps before engineering starts. - Expose governed data to Claude and other agents through MCP, so every agent, dashboard, and text-to-SQL query answers from the same definitions. - Use AI to draft documentation and definitions, then own their correctness together with the business owner.

- Working ML fluency: you understand data leakage, time-based validation, class imbalance, and precision versus recall, and you have built datasets that trained or evaluated a model or retriever, evaluated with Langfuse, Arize Phoenix, Braintrust, or MLflow and versioned in the MLflow registry. - Hands-on experience building agents on Databricks (Agent Bricks or Mosaic AI Agent Framework) and running coding agents such as Claude Code or Codex under governed harnesses with skills, policies, and guardrails; familiarity with Databricks’ newer agentic tooling (Omnigent, Genie Migrate, Agent Bricks CLI) and with how agents consume data through RAG, text-to-SQL, and MCP. - Governance depth across Unity Catalog, Purview, or Snowflake RBAC; you treat access control as a design input for pipelines and retrieval alike.

- Choose the right retrieval architecture for each use case, from hybrid search with reranking to graph-based and agentic retrieval, and justify the cost and maintenance burden of each to the client. - Operate vector, full-text, and hybrid search infrastructure (Databricks AI Search, formerly Vector Search; Azure AI Search; OpenSearch; pgvector), including incremental indexing, permission-aware retrieval, and citations back to the source. - Build and deploy retrieval-backed agents on Databricks Agent Bricks, using its code-first tooling (Agent Bricks CLI, Agent Runtime on Databricks Apps) and governing model access through Unity AI Gateway.

You will build the data foundations that make our clients’ AI systems trustworthy: the lakehouses that unify their ERPs and SaaS tools, the retrieval systems that ground agents in their documents and records, the semantic layers that give every metric one definition, and the evaluation datasets that prove an answer is right. Every agent, dashboard, and workflow Cadre ships sits on top of the data layer you design. Your week moves between architecture and execution: mapping a manufacturer’s ERP tables with their finance team one day, building Databricks medallion layers or tuning a hybrid retrieval pipeline the next. You thrive here if you keep up with retrieval research, know why a random split leaks in a time series, and can explain a GraphRAG trade-off to a CFO in plain language. You will work in client pods with AI engineers, solution architects, and AI strategists, and shape the reusable data and retrieval blocks every engagement starts from.

Lakehouse and Pipeline Engineering - Design and build medallion (bronze, silver, gold) lakehouses on Databricks (Delta Lake, Unity Catalog, Lakeflow Spark Declarative Pipelines) and Microsoft Fabric (OneLake, Data Factory), and on Snowflake where a client already runs it. - Own ingestion from ERPs, CRMs, and SaaS platforms using platform-native connectors (Lakeflow Connect, Fabric Data Factory), Fivetran, or dlt, including change data capture and schema drift handling.

- Own ingestion from ERPs, CRMs, and SaaS platforms using platform-native connectors (Lakeflow Connect, Fabric Data Factory), Fivetran, or dlt, including change data capture and schema drift handling. - Model and transform data with dbt, orchestrate it with Apache Airflow, Dagster, or Lakeflow Jobs, and ship it through CI/CD with GitHub Actions and Declarative Automation Bundles (formerly Databricks Asset Bundles). - Build data quality checks and data contracts into every pipeline so downstream agents never consume silent errors.

- Build data quality checks and data contracts into every pipeline so downstream agents never consume silent errors. - Run agentic data engineering safely in client Databricks repositories: set up coding agents (Claude Code, Codex) with shared skills, policies, and guardrails, using Omnigent where it is available, and accelerate legacy-to-Databricks migrations with Genie Migrate. Retrieval Systems: RAG and GraphRAG

- Operate vector, full-text, and hybrid search infrastructure (Databricks AI Search, formerly Vector Search; Azure AI Search; OpenSearch; pgvector), including incremental indexing, permission-aware retrieval, and citations back to the source. - Build and deploy retrieval-backed agents on Databricks Agent Bricks, using its code-first tooling (Agent Bricks CLI, Agent Runtime on Databricks Apps) and governing model access through Unity AI Gateway. Semantic Layers and Business Ontologies

Semantic Layers and Business Ontologies - Translate how a client defines a customer, an order, or an invoice into governed metric and entity definitions using dbt Semantic Layer (MetricFlow), Power BI semantic models, Cube, Databricks metric views, or Snowflake semantic views. - Lead data readiness assessments: inventory source systems, map fields and business logic, and document gaps and remediation steps before engineering starts.

- Build golden datasets, output and label pairs, and backtesting splits that measure whether an agent, retriever, or model is right. - Instrument retrieval and agent pipelines with dedicated tracing and evaluation tools (Langfuse, Arize Phoenix, Braintrust), or with MLflow 3 GenAI evaluation and LLM judges on Databricks, and track recall, precision, and groundedness. - Track experiments and version models and evaluation runs in the MLflow registry, so every change to a retriever or agent is traceable.

What You Need to Succeed - 5+ years of data engineering experience, including at least one production lakehouse you designed and operated on Databricks or Microsoft Fabric with a medallion architecture; experience on both is a plus, since our clients run both. - A Master’s degree or PhD in Computer Science, Mathematics, Engineering, or a related quantitative field, with the theoretical grounding to read current research and apply it to client problems.

- A Master’s degree or PhD in Computer Science, Mathematics, Engineering, or a related quantitative field, with the theoretical grounding to read current research and apply it to client problems. - Hands-on experience shipping RAG pipelines to production (hybrid retrieval on Databricks AI Search or an equivalent engine, reranking, chunking strategy, and permission-aware indexing over messy enterprise documents), with command of the current state of the art: you follow the literature and can explain when hybrid, graph-based, or agentic retrieval fits a problem. - Experience designing and running GraphRAG or knowledge-graph retrieval (Microsoft GraphRAG, LightRAG, or equivalent, on graph stores such as Neo4j), and the judgment to know when a graph earns its maintenance cost and when hybrid search is enough.

- Expert SQL and strong Python and PySpark, with production experience in dbt or platform-native pipelines and in ingestion tools (Fivetran, Lakeflow Connect, Azure or Fabric Data Factory, or dlt); you use coding agents such as Claude Code daily and review everything they generate. - Experience authoring a semantic or metrics layer (dbt Semantic Layer, Power BI semantic models, Cube, LookML) and turning business definitions into governed metrics that people and agents both rely on. - Working ML fluency: you understand data leakage, time-based validation, class imbalance, and precision versus recall, and you have built datasets that trained or evaluated a model or retriever, evaluated with Langfuse, Arize Phoenix, Braintrust, or MLflow and versioned in the MLflow registry.

- Design and build medallion (bronze, silver, gold) lakehouses on Databricks (Delta Lake, Unity Catalog, Lakeflow Spark Declarative Pipelines) and Microsoft Fabric (OneLake, Data Factory), and on Snowflake where a client already runs it. - Own ingestion from ERPs, CRMs, and SaaS platforms using platform-native connectors (Lakeflow Connect, Fabric Data Factory), Fivetran, or dlt, including change data capture and schema drift handling. - Model and transform data with dbt, orchestrate it with Apache Airflow, Dagster, or Lakeflow Jobs, and ship it through CI/CD with GitHub Actions and Declarative Automation Bundles (formerly Databricks Asset Bundles).

- Instrument retrieval and agent pipelines with dedicated tracing and evaluation tools (Langfuse, Arize Phoenix, Braintrust), or with MLflow 3 GenAI evaluation and LLM judges on Databricks, and track recall, precision, and groundedness. - Track experiments and version models and evaluation runs in the MLflow registry, so every change to a retriever or agent is traceable. - Design leakage-free, time-aware splits for forecasting, scoring, and risk ranking.

- Experience authoring a semantic or metrics layer (dbt Semantic Layer, Power BI semantic models, Cube, LookML) and turning business definitions into governed metrics that people and agents both rely on. - Working ML fluency: you understand data leakage, time-based validation, class imbalance, and precision versus recall, and you have built datasets that trained or evaluated a model or retriever, evaluated with Langfuse, Arize Phoenix, Braintrust, or MLflow and versioned in the MLflow registry. - Hands-on experience building agents on Databricks (Agent Bricks or Mosaic AI Agent Framework) and running coding agents such as Claude Code or Codex under governed harnesses with skills, policies, and guardrails; familiarity with Databricks’ newer agentic tooling (Omnigent, Genie Migrate, Agent Bricks CLI) and with how agents consume data through RAG, text-to-SQL, and MCP.

- Build production RAG pipelines end to end: document parsing and OCR, chunking, embeddings, hybrid retrieval (BM25 plus dense vectors), and cross-encoder reranking. - Design GraphRAG systems that extract entities and relationships into a knowledge graph, using frameworks such as Microsoft GraphRAG or LightRAG on graph stores such as Neo4j, for questions that span documents, systems, and multiple entity hops. - Choose the right retrieval architecture for each use case, from hybrid search with reranking to graph-based and agentic retrieval, and justify the cost and maintenance burden of each to the client.

- Hands-on experience shipping RAG pipelines to production (hybrid retrieval on Databricks AI Search or an equivalent engine, reranking, chunking strategy, and permission-aware indexing over messy enterprise documents), with command of the current state of the art: you follow the literature and can explain when hybrid, graph-based, or agentic retrieval fits a problem. - Experience designing and running GraphRAG or knowledge-graph retrieval (Microsoft GraphRAG, LightRAG, or equivalent, on graph stores such as Neo4j), and the judgment to know when a graph earns its maintenance cost and when hybrid search is enough. - Production experience orchestrating data pipelines with Apache Airflow or Dagster, including dependencies, backfills, retries, and alerting; you understand the trade-off between task-based and asset-based orchestration and work comfortably in platform-native schedulers such as Lakeflow Jobs and Fabric Data Factory.

Governance and Client Delivery - Implement role-based access, row-level security, and masking in Unity Catalog, Microsoft Purview, or Snowflake roles, and carry those permissions into retrieval and agent context. - Ensure every data path meets SOC 2 expectations and the client’s own AI governance policies, including the handling of PII, PHI, and payroll data.

- Hands-on experience building agents on Databricks (Agent Bricks or Mosaic AI Agent Framework) and running coding agents such as Claude Code or Codex under governed harnesses with skills, policies, and guardrails; familiarity with Databricks’ newer agentic tooling (Omnigent, Genie Migrate, Agent Bricks CLI) and with how agents consume data through RAG, text-to-SQL, and MCP. - Governance depth across Unity Catalog, Purview, or Snowflake RBAC; you treat access control as a design input for pipelines and retrieval alike. - Client-facing consulting or forward-deployed experience and professional English fluency, because you will work daily with US executives and turn technical trade-offs into business decisions.

- Client-facing consulting or forward-deployed experience and professional English fluency, because you will work daily with US executives and turn technical trade-offs into business decisions. - Strong plus: Snowflake Cortex, Palantir Foundry ontologies, RDF/OWL knowledge graphs, DSPy prompt optimization, or LoRA fine-tuning of open-weight models. Why Cadre AI

- Experience designing and running GraphRAG or knowledge-graph retrieval (Microsoft GraphRAG, LightRAG, or equivalent, on graph stores such as Neo4j), and the judgment to know when a graph earns its maintenance cost and when hybrid search is enough. - Production experience orchestrating data pipelines with Apache Airflow or Dagster, including dependencies, backfills, retries, and alerting; you understand the trade-off between task-based and asset-based orchestration and work comfortably in platform-native schedulers such as Lakeflow Jobs and Fabric Data Factory. - Expert SQL and strong Python and PySpark, with production experience in dbt or platform-native pipelines and in ingestion tools (Fivetran, Lakeflow Connect, Azure or Fabric Data Factory, or dlt); you use coding agents such as Claude Code daily and review everything they generate.