Agents Are Where Microservices Were in 2015 — Roberto Milev & Uday Kanagala, Navan
Summary
The talk outlines that AI agents represent a paradigm shift comparable to microservices in 2015, requiring entirely new architectural patterns for reliable production deployment. Key areas of focus include managing agent statefulness (moving beyond stateless APIs), implementing advanced observability via hooks and traces, adopting skills as the primary unit of context, and establishing robust governance through guardrails at the policy layer. The industry is moving toward scoring non-deterministic trajectories rather than asserting fixed outputs.
Key takeaways
-
Agents are Stateful by Nature
3:54
Unlike traditional stateless API services, agents require persistent sessions and state management, necessitating specialized agentic runtimes (e.g., AWS Agent Core Runtime) [2:34].
-
Skills as Context Unit
6:04
To manage context effectively, the focus should be on treating 'skills'—which include instructions and tool execution logic—as pluggable units of work that allow for progressive disclosure of context [6:04].
-
Observability via Hooks and Traces
7:16
Because agents emit excessive thinking output, traditional logging fails. Operational reliability requires intercepting every step (pre-tool/post-tool, pre-decision/post-decision) to capture auto-traces, goals, reasoning, belief status, and confidence scores [7:16].
-
Testing Non-Deterministic Systems
9:32
Since agents are non-deterministic, testing must shift from asserting fixed outputs to scoring trajectories—evaluating the efficiency and completeness of the path taken toward a goal [9:32].
-
Guardrails for Authorization
13:44
The blurring line between an agent acting on behalf of a user versus using its own service account requires fine-grained authorization policies (guardrails) applied before and after every tool call to prevent sensitive data leakage [13:04].
Technical details
-
Runtime Architecture
70s
The industry is standardizing layers for agentic flows, including runtime, memory, context management, and orchestration. Agents require stateful handling, which cloud providers are addressing with dedicated runtimes [1:25].
-
Memory Management
222s
Memory has evolved from basic Retrieval-Augmented Generation (RAG) to a pipeline that incorporates short-term conversational memory and long-term/episodic memories, which must be managed by the user [3:42].
-
Context Management
364s
The recommended approach is focusing on 'skills' as the unit of context. Skills are composed dynamically and allow for progressive disclosure, starting with a limited scope before expanding metadata [6:04].
-
Observability & Tracing
306s
To debug complex agentic flows, auto-traces must capture specific signals (current goal, reasoning, belief status) at critical points like tool calls and decisions. This provides confidence scores to determine if an answer is inferred or certain [8:24].
-
Authorization & Guardrails
290s
A policy layer must enforce guardrails before and after every tool call, addressing the ambiguity of whether an agent's action belongs to a user or the service account itself. This is critical for enterprise governance [14:15].
Mentioned resources
- AWS Agent Core Runtime
- OTEL (OpenTelemetry)
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.