How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase
Summary
This talk outlines a framework for integrating AI agents into highly regulated, policy-driven workflows, such as those found in enterprise financial services. The core methodology involves modeling processes as Directed Acyclic Graphs (DAGs) for the overall flow, and Directed Cyclic Graphs (DCGs) for internal nodes (loops). Trust in agents is established not by the model itself, but by implementing robust confidence mechanisms—specifically, rigorous evaluation (Evals) and tracing of runtime logs—to ensure outputs meet defined policy thresholds. Agents should be deployed in a phased manner: fully supervised $\rightarrow$ sampled $\rightarrow$ exceptions only.
Key takeaways
-
Policy as the System Prompt
The organizational policy serves as the 'original institutional intent' or the 'original system prompt,' defining rules using language like 'must,' 'may,' and 'should' to govern how work is done.
-
The Loop is only as good as the Check
17:23
In any process loop (DCG), the reliability of the entire workflow depends entirely on the quality and rigor of the check mechanism used to determine if the work is 'good enough' to proceed.
-
Phased Agent Adoption
Agent deployment should follow a structured progression: 1) Fully supervised (manual verification); 2) Sampling (humans review a slice of output); 3) Exceptions only (humans handle only failures).
Technical details
-
Workflow Modeling (DAGs/DCGs)
0s
Workflows are modeled as Directed Acyclic Graphs (DAGs) to represent the overall, gated flow (e.g., approval processes). Individual steps within the flow are often loops, represented by Directed Cyclic Graphs (DCGs). Tools like Apache Airflow manage DAGs, while LangChain/LangGraph are suggested for managing DCG state.
-
Confidence Mechanisms (Evals and Traces)
Since agents are probabilistic, confidence is built using statistical analysis through 'Evals' (evaluations on representative datasets) and 'Traces' (runtime logs of what happens). This complements traditional software confidence mechanisms like unit tests.
-
System State Separation
0s
It is crucial to separate the 'build state' (defining or changing policy) from the 'run state' (the system constantly operating and moving).
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.