Your agent architecture has a half-life of 6 months — Dan Farrelly, CTO, Inngest
Summary
The talk addresses the rapid obsolescence of AI agent architectures, noting that their 'half-life' can be as short as six months due to evolving models and frameworks. The core thesis is that engineers must build systems by focusing on three decoupled conceptual layers—Execution, Context, and Compute—to ensure stability. The Execution Layer is identified as the most stable component, responsible for managing flow, state durability, retries, and coordinating complex workflows regardless of changes in LLMs or prompts.
Key takeaways
-
Agent Architecture Volatility
AI agent architectures are highly volatile; components like prompts may last weeks, models months, and frameworks can quickly become outdated. This rapid change necessitates an architectural approach that decouples core primitives to prevent technical debt.
-
The Three Conceptual Layers
3:30
Effective agent design requires considering three discrete layers: the Execution Layer (the 'brain,' handling flow, state, and durability); the Context Layer (the 'knowledge,' including models, prompts, tools, and memory); and the Compute Layer (the 'hands,' involving sandboxes, runtimes, or browsers).
-
Focus on the Execution Layer
5:20
The execution layer must be designed to remain stable over years. It is responsible for managing the full life cycle of an agent—including planning, model calls, code running, and sub-agent invocation—independent of the context or compute used.
Technical details
-
Architectural Layers
210s
The system should be viewed through three decoupled layers: Execution (flow/state/durability), Context (models/prompts/tools/memory), and Compute (sandboxes/runtimes). Coupling these layers leads to high technical debt.
-
Execution Layer Requirements
420s
The execution layer must provide: 1) **Resumability:** The ability to pick up after failures (e.g., a failed step 38) without restarting the entire process, requiring state to be durable and external. 2) **Invocation Patterns:** Support for diverse patterns like CRON jobs, event triggers, APIs, human-in-the-loop processes, and asynchronous/synchronous calls. 3) **Observability:** Full session tracing across all components (LLM calls, tool calls, database errors, permissions issues) to facilitate debugging and improvement.
-
Sandboxes vs. Execution Layer
580s
Sandboxes are ephemeral and stateless by design; using them for durability is an anti-pattern. The execution layer must provide the context, sequence, and durability that the sandbox lacks. The execution layer acts as the 'brain,' while the sandbox acts as the 'hands.'
-
Advanced Agent Patterns
720s
Emerging long-running patterns (e.g., background agents, loop architectures) require robust infrastructure that can handle asynchronous, delegated tasks and provide full observability across multiple processes.
Mentioned resources
- Inngest
- durable serverless functions
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.