# The Agent Works. Does Anyone Trust It? — Michele Grant, Leading at Scale

## Executive summary

While an AI agent's technical accuracy (e.g., matching analyst reports) proves functionality, it does not guarantee organizational trust or measurable business value. Michele Grant outlines a framework for moving agents from pilot to core workflow by answering three critical questions: What value does it create? Who is accountable for it? And what evidence permits greater autonomy? Successful deployment requires establishing pre-deployment baselines, implementing robust governance (like the Three Lines Model), and maintaining rigorous technical artifacts to ensure the agent operates within defined risk and human oversight boundaries.

## Key takeaways

- Value Requires a Baseline: Before deployment, measurable value must be established using pre-deployment baselines for three metrics: Cycle Time (time to compile reports), Analyst Capacity (hours saved, net of compute/licensing costs), and Detection Time (time from data slip to committee visibility).
- Accountability via Three Lines Model: Accountability must be applied to the agent using the Three Lines Model: Management owns the controls (the integration lead); Risk and Compliance oversee the agent's exposure; and Internal Audit provides assurance based on material risk.
- Governance through Artifacts: To grant autonomy, three artifacts are required: an Auditable Decision Log (recording raw data, tool calls, and human sign-off); a Model Inventory Record (documenting data lineage, dependencies, and regression triggers); and an Autonomy Scorecard (tracking progress through stages like Shadow Mode and Exception Ensemble).

## Technical details

- Deterministic Code Requirements: Core mathematical calculations and reconciliation rules must run in deterministic, finance-approved code, rather than being generated as conversational prose from an LLM. The model's written summary should be treated as a helpful explanation, not proof.
- Agent Autonomy Stages: Autonomy advances through four stages: Stage 1 (Shadow Mode - agent runs alongside manual process); Stage 2 (Supervised Draft - senior analyst reviews every line item); Stage 3 (Exception Ensemble - analyst reviews lag initiatives plus random sample); and Stage 4 (Exceptions Only - analyst reviews only high-risk anomalies).
- Risk Monitoring: When operating in advanced stages, monitoring error severity and tracking the override rate is critical. A falling override rate may indicate a slip in the depth of review rather than improved performance.

## Practical implications

- For build engineers, focus on building observability into the agent's governance layer, ensuring all tool calls, data lineage, and model versions are logged in an auditable decision log.
- Implement mandatory pre-deployment baselining for key operational metrics (cycle time, capacity, detection time) to prove measurable value before granting autonomy.
- Design the system architecture to separate deterministic, finance-approved code (for calculations) from the LLM's interpretive layer (for narrative/explanation).

## Topics

AI Governance, Enterprise Architecture, Risk Management, M&A Integration, Process Automation, Leading at Scale

Source: https://www.youtube.com/watch?v=-a2oTS9J67U