How to Design a Data Agent People Can Verify
The video argues that current internal 'data agents' (AI chatbots that query databases to answer business questions) represent a significant design flaw if they only provide a final answer without showing their work. To build trustworthy AI products, the design must prioritize verifiability, allowing domain experts to trace the data's provenance, review the underlying logic (e.g., SQL queries, definitions, intermediate calculations), and confirm the result's accuracy. This shift requires adopting design patterns similar to literate programming and notebooks.
Key takeaways
-
The Flaw in Current Data Agents
2:05
Simply receiving an answer (e.g., 'net revenue is X') from a data agent is insufficient because the user cannot verify its correctness. The lack of transparency makes the output untrustworthy.
-
The Need for Provenance and Traceability
2:30
A proper data agent must show the 'working'—the path taken to reach the number. This includes the underlying SQL query, the definitions used (e.g., what constitutes 'net revenue'), the filters applied, and the specific calendar period used.
-
Designing for Evaluation (Evals)
3:50
The product design should mimic a data scientist's thought process, utilizing notebooks or semantic layers to display intermediate calculations and the source of data. This makes the product inherently easier to evaluate and debug.
-
Benefits of Verifiable Design
4:40
Designing for verifiability not only builds user confidence but also provides rich data signals, making it easier to automatically classify errors and collect better feedback for model improvement.
