ADBC in production: zero-copy, non-zero problems
Columnar co-founder Matt Topol on the year of type-system chaos behind building production ADBC drivers for the standard dbt v2 adopted.
Overview
Explores real-world challenges running ADBC (Arrow Database Connectivity) in production: observability gaps for failed queries, type-mapping inconsistencies, driver discovery issues, and ecosystem immaturity, despite promised zero-copy, vendor-neutral performance. Shares experience deploying ADBC with dbt and Power BI.
What you'll take away
- 01
ADBC gives applications one vendor-neutral API across many database drivers, handing data back in Apache Arrow's columnar format instead of the row-oriented protocols behind ODBC and JDBC, and dbt v2 now uses it as a core communication layer.
- 02
Type systems are where most of the pain lives: SQL's numeric precision alone diverges from 38 digits (SQL Server) to more than 1,000 (Postgres), forcing a driver to choose its type mapping before it ever sees real data.
- 03
Closing advice for anyone adopting ADBC: test your schemas early, pin your driver versions, and always know exactly which driver version you're running.
Session speaker
Matt Topol
Columnar