All aboard: rebuilding dbt in the open
dbt v2 bundles adapters into a single binary, parses large projects several times faster, and replaces the sprawling manifest.json with small, queryable files built for both people and AI agents.
Overview
Rewriting a 9 year old code base is hard. Rewriting it in the open over ~1 billion dbt installs later? Harder. Why did we do it? For faster runtimes, SQL comprehension, shared adapters, better docs, and a modernized developer experience.
What you'll take away
- 01
dbt v2 gives dbt real semantic comprehension of SQL and Jinja for the first time, by rewriting the templating engine in Rust, rather than just orchestrating a DAG without understanding the SQL inside it.
- 02
On a synthetic 1,800-model project, v1 parsed in 10-13 seconds versus about 3 for v2, and the new linter runs 50 to 100 times faster than standard SQLFluff.
- 03
Docs v2 replaces a single, half-a-million-line manifest.json with small, queryable Parquet files, enabling plain-language lineage and "blast radius" queries that used to mean manually parsing a massive file.
Session speaker
Hope Watson
dbt Labs