How to Design a Data Agent People Can Verify
Summary
The video argues that current internal 'data agents' (AI chatbots that query databases to answer business questions) represent a significant design flaw if they only provide a final answer without showing their work. To build trustworthy AI products, the design must prioritize verifiability, allowing domain experts to trace the data's provenance, review the underlying logic (e.g., SQL queries, definitions, intermediate calculations), and confirm the result's accuracy. This shift requires adopting design patterns similar to literate programming and notebooks.
Key takeaways
-
The Flaw in Current Data Agents
2:05
Simply receiving an answer (e.g., 'net revenue is X') from a data agent is insufficient because the user cannot verify its correctness. The lack of transparency makes the output untrustworthy.
-
The Need for Provenance and Traceability
2:30
A proper data agent must show the 'working'—the path taken to reach the number. This includes the underlying SQL query, the definitions used (e.g., what constitutes 'net revenue'), the filters applied, and the specific calendar period used.
-
Designing for Evaluation (Evals)
3:50
The product design should mimic a data scientist's thought process, utilizing notebooks or semantic layers to display intermediate calculations and the source of data. This makes the product inherently easier to evaluate and debug.
-
Benefits of Verifiable Design
4:40
Designing for verifiability not only builds user confidence but also provides rich data signals, making it easier to automatically classify errors and collect better feedback for model improvement.
Technical details
-
Data Agent Architecture
100s
The current trend involves internal agents that query databases to replace data analysts. The core challenge is that these agents often fail to provide the necessary evidence (e.g., the SQL query or calculation steps) to validate their output.
-
Verification Requirements
150s
Verification requires exposing the full analytical chain: the definition of complex metrics (like 'net revenue'), the source tables, necessary joins, and the specific date range ('last quarter').
-
Best Practices (Literate Programming)
230s
The ideal solution involves linking the answer to a 'notebook' or analysis that spells out the thought process, showing the provenance of the data and the steps taken by the AI.
-
Industry Example
340s
Hex is cited as a real-world data product that successfully implements this design, providing not only an answer but also evidence, a notebook, and the underlying SQL query.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.