Revolutionizing the staging layer: automating dbt generation for 700+ sources
WebstaurantStore replaced a manual, source-by-source migration with an AI-assisted, human-checked dbt staging layer for over 700 sources.
Overview
Describes migrating a large warehouse from SQL Server to dbt and automating staging components (source definitions, model SQL, YAML config, docs) across 700+ enterprise sources, with governance built into the migration rather than bolted on after.
What you'll take away
- 01
Snowflake Cortex LLM functions proposed normalized column names and business definitions from raw legacy metadata, but every suggestion landed in a human-reviewed table before any production model ever read it.
- 02
That checkpoint is what let a three-person team move fast without losing control: automating a staging layer for 700-plus sources in 90 days, a roughly tenfold increase in ingestion speed, with no added headcount.
- 03
Automation only pays off if the people relying on it can still trust what it produces — the human review step was the actual unlock, not the AI alone.
Session speaker
Cathy Huang
Webstaurantstore