How Checkr cut an hours-long metrics investigation to minutes by engineering AI context on dbt and Omni

Chris Merrick, co-founder and CTO of Omni, and Sarah Fischbach, staff analytics engineer at Checkr, framed this session around a single question: is context engineering becoming the new analytics engineering? Sarah's answer came from two years of hands-on experience building Checkr's AI-ready data foundation, working alongside a small central analytics engineering team and roughly 50 analysts embedded across the business.

The problem she opened with is a familiar one at scale: Checkr's Snowflake instance and dbt project span more than 6,000 models. When she connected an AI agent directly to Snowflake through an MCP and asked the same question multiple times, she got a different wrong answer every time, a pattern she called metric drift.

She sharpened the example with FRT (First Reply Time), the core SLA Checkr's operations team is measured against. Asked "what queues drove FRT below 90%?", the agent had no idea what FRT meant, what a queue was, or what the 90% threshold referred to, because it could only see field names and table structures, not the business logic behind them.

Her framework breaks AI-readiness into three layers: the data warehouse (the dictionary of every data point that exists), the semantic layer (the grammar that defines valid joins, calculations, and metrics, and prevents metric drift), and AI context (the conversational bridge that most teams skip entirely). Omni's version of that third layer is five structured fields: synonyms that map business language to field names, sample values so an agent can parse natural language into valid filters, field descriptions (often already written in a dbt project), free-text behavioral guidance for edge cases, and pre-built sample queries for complex, multi-step questions.

Her core principle for filling all of this in: use the context that already exists in Confluence pages, meeting transcripts, PRDs, and Slack threads, and don't manufacture new documentation from scratch.

To do this at scale, Checkr built an automated feedback loop: a golden set of evals (real stakeholder questions paired with the correct query plan) runs against Omni, a Claude-based agent clusters the failures, a second skill pulls relevant context from a vector store of company knowledge to draft a fix, and a third skill re-runs the failing evals to verify the fix actually worked before opening a merge request for a subject-matter expert to review.

The simpler, manual version of the same loop: take a Confluence page or meeting transcript, paste it into an AI assistant alongside the relevant dbt model code, ask it to extract synonyms and behavioral guidance, then review and add that context to Omni by hand.

The payoff showed up when Christian, one of Checkr's operations leads, asked the agent what queues drove FRT below 90%. With synonyms, channel-specific SLA targets, and edge-case rules already encoded, the agent identified the underperforming week, broke it down by queue and channel, and correctly ruled out the high-volume queue everyone assumed was the culprit, pointing instead to international queues.

In Christian's words: "What would have taken hours got done in minutes, and with enough directional clarity to actually change how we responded."

One audience question, from a data professional at OutSchool, surfaced a useful caution: adding AI context broadly, without a specific failure to fix, made their agent's answers slower and roughly doubled token costs without improving accuracy. Sarah and Chris's advice matched Checkr's own experience: match the fix to the failure, prune fields nobody actually queries, and add context surgically where there's a measurable precision or recall problem, not as a blanket pass over every model.

Watch every keynote and breakout from dbt Summit 2026 on demand. Watch dbt Summit 2026 on demand →