Senior Data Engineer, Unified Platform

DRW · Singapore

// classified as

Other (Adjacent or hard to classify.)

> stack

javapythonsqlclickhousekafkakubernetesnosqlprefects3sparkairflowdagsterdaskpolars

> description

DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to operate in markets around the world. We value autonomy and the ability to quickly pivot to capture opportunities, so we operate using our own capital and trading at our own risk. Headquartered in Chicago with offices throughout the U.S., Canada, Europe, and Asia, we trade a variety of asset classes including Fixed Income, ETFs, Equities, FX, Commodities and Energy across all major global markets. We have also leveraged our expertise and technology to expand into three non-traditional strategies: real estate, venture capital and cryptoassets. We operate with respect, curiosity and open minds. The people who thrive here share our belief that it’s not just what we do that matters–it's how we do it. DRW is a place of high expectations, integrity, innovation and a willingness to challenge consensus. As a Senior Data Engineer on our Data Experience team, you will play an integral role in bringing vendor datasets into our data platform, governing our centralized data pipelines, supporting rapid data product development, and working alongside individual Traders, Quantitative Researchers, and Back-Office personnel to best utilize the firm’s data and platform tools.  Technical Requirements Summary:   <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1">Have experience designing and building data pipelines </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1">Have experience working within modern batch or streaming data ecosystems </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1">An expert in SQL and have experience in Java or Python </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1">Can apply data modeling techniques </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1">Able to own the delivery of data products, working with analysts and stakeholders to understand requirements and implement solutions </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1">Able to contribute to project management and project reporting </li> </ul> What you will do in this role:   <ul> <li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1">Help model, build, and manage data products built atop DRW’s Unified Data Platform. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1">Work closely with Data Strategists to determine appropriate data sources and implement processes to onboard and manage new data sources for trading, research, and back-office purposes. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1">Contribute to data governance processes that enable discovery, cost-sharing, usage tracking, access controls, and quality control of datasets to address the needs of DRW trading teams and strategies. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1">Continually monitor data ingestion pipelines and data quality to ensure stability, reliability, and quality of the data. Contribute to the monitoring and quality control software and processes. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1">Own the technical aspects of vendor ingestion pipelines, coordinating with vendor relationship managers on upcoming changes, performing routine data operations without breaking internal users, and contributing to the team’s on-call rotation to respond to unanticipated changes. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1">Rapidly respond to user requests, identifying platform gaps and self-service opportunities that make the user experience more efficient. </li> </ul> What you will need in this role:  <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1">5+ years of experience working with modern data technologies and/or building data-first products. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1">Excellent written and verbal communication skills. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1">Proven ability to work in a collaborative, agile, and fast-paced environment, prioritizing multiple tasks and projects, and efficiently handle the demands of a trading environment. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1">Proven ability to deliver rapid results within processes that span multiple stakeholders. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1">Strong technical problem-solving skills. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1">Extensive familiarity with SQL and Java or Python, with a proven ability to develop and deliver maintainable data tranformations for production data pipelines. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="7" data-aria-level="1">Experience leveraging data modeling techniques and ability to articulate the trade-offs of different approaches. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="8" data-aria-level="1">Experience with one or more data processing technologies (e.g. Flink, Spark, Polars, Dask, etc.) </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="9" data-aria-level="1">Experience with multiple data storage technologies (e.g. S3, RDBMS, NoSQL, Delta/Iceberg, Cassandra, Clickhouse, Kafka, etc.) and knowledge of their associated trade-offs. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="10" data-aria-level="1">Experience with multiple data formats and serialization systems (e.g. Arrow, Parquet, Protobuf/gRPC, Avro, Thrift, JSON, etc.) </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="11" data-aria-level="1">Experience managing data pipeline orchestration systems (e.g. Kubernetes, Argo Workflows, Airflow, Prefect, Dagster, etc.) </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="12" data-aria-level="1">Proven experience in managing the operational aspects of large data pipelines such as backfilling datasets, rerunning batch jobs, and handling dead-letter queues. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="13" data-aria-level="1">Prior experience triaging data quality control processes, correcting data gaps and inaccuracies. </li> </ul>   For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at <a href="https://drw.com/privacy-notice">https://drw.com/privacy-notice</a>. California residents, please review the California Privacy Notice for information about certain legal rights at <a href="https://drw.com/california-privacy-notice">https://drw.com/california-privacy-notice</a>. #LI-PR1