Sr. Data Platform Engineer

About Us

The role

What you’ll do

• Continue our migration of pipeline orchestration to Prefect
• Own day-to-day operations of our data infrastructure
• Extend the pipeline to include new data sources and transformations
• Maintain, expand and optimize our postgres database and Iceberg datalake
• Create the 'connective tissue' for data at Aarden

• Partner with product on new feature-driven datasets.
• Collaborate with the ML/analytics team to close the loop: anomaly detection → ticket → fix → validation → promotion to production
• Develop cross-team tooling/infra to keep GitHub, Notion, Linear, and Slack connected so pipeline issues, docs, and fixes stay linked

• Implement run-over-run data observability (row counts, key column distributions) to catch anomalies and bugs
• Expose accuracy/quality metrics as first-class artifacts so changes can be evaluated automatically, by a human or an agent
• Write and maintain AI-context documentation (schema docs, pipeline architecture, known patterns/quirks, "what not to do")

You might be a good fit if you…

• Have strong Python skills & are comfortable with PySpark or similar distributed data processing
• Have a strong sense of how the data you’re working with impacts the end-user
• Are curious and excited about AI and the impact it can have on our ways of working as developers
• Have experience with geospatial data (GeoParquet, PostGIS, Apache Sedona, or similar)
• Have worked with table formats like Apache Iceberg and lakehouse architectures
• Have worked on workflow orchestration (Prefect, Airflow, Dagster, or similar)
• Are comfortable working in a git-based, CI-friendly workflow

• Have worked in full-stack environments, where your work can directly impact the application layer
• Have experience with Apache Sedona or other cloud spatial-compute platforms
• Have built observability/logging layers for data pipelines (not just app services)
• Have experience with property, parcel, real estate, or land data specifically

• Have experience using AI agents to improve data architecture in a real production codebase
• Experience in real estate/land, energy, forestry, or agriculture tech

Our Stack

• Prefect 3 for pipeline orchestration (YAML/config-driven flows, retries, logging)
• Coiled for elastic EC2 workers on GDAL-heavy and batch Python jobs
• Wherobots (managed Apache Sedona / PySpark) for large-scale spatial joins, parcel ingest, and lakehouse work

Working at Aarden

• At least 2 in-person days per week at our office in Capitol Hill | We’ve found that while heads-down time at home is fantastic for task-related productivity, in-person time is magic for longer-form productivity. Our in-person days are used to plan, troubleshoot, and check-in with each other on progress and questions. Expect team lunches and whiteboarding.
• Focused ownership in your role | The rest of the team is here to help you and cares deeply about the long-term functionality of our applications. With that said, we’ll be looking to you to own your lane, go deep, and develop a strong stance on what it takes to make our applications best-in-class.
• Dedicated monthly AI tooling budget | We’re in a golden era of AI-powered developer tooling. We strongly encourage augmenting your output with AI tools, and have a dedicated & flexible budget for every team member to support that setup. We care about what you ship, not how.