Agents Are Where Microservices Were in 2015 — Roberto Milev & Uday Kanagala, Navan
The talk outlines that AI agents represent a paradigm shift comparable to microservices in 2015, requiring entirely new architectural patterns for reliable production deployment. Key areas of focus include managing agent statefulness (moving beyond stateless APIs), implementing advanced observability via hooks and traces, adopting skills as the primary unit of context, and establishing robust governance through guardrails at the policy layer. The industry is moving toward scoring non-deterministic trajectories rather than asserting fixed outputs.
Key takeaways
-
Agents are Stateful by Nature
3:54
Unlike traditional stateless API services, agents require persistent sessions and state management, necessitating specialized agentic runtimes (e.g., AWS Agent Core Runtime) [2:34].
-
Skills as Context Unit
6:04
To manage context effectively, the focus should be on treating 'skills'—which include instructions and tool execution logic—as pluggable units of work that allow for progressive disclosure of context [6:04].
-
Observability via Hooks and Traces
7:16
Because agents emit excessive thinking output, traditional logging fails. Operational reliability requires intercepting every step (pre-tool/post-tool, pre-decision/post-decision) to capture auto-traces, goals, reasoning, belief status, and confidence scores [7:16].
-
Testing Non-Deterministic Systems
9:32
Since agents are non-deterministic, testing must shift from asserting fixed outputs to scoring trajectories—evaluating the efficiency and completeness of the path taken toward a goal [9:32].
-
Guardrails for Authorization
13:44
The blurring line between an agent acting on behalf of a user versus using its own service account requires fine-grained authorization policies (guardrails) applied before and after every tool call to prevent sensitive data leakage [13:04].