The Video Signal

Topic

Idempotency

All digests tagged Idempotency

Brains vs Hands: How to Run AI Agents Safely in Production — Viren Baraiya thumbnail

· 16:50

Brains vs Hands: How to Run AI Agents Safely in Production — Viren Baraiya

Running AI agents in production requires a fundamental architectural shift: the agent's role must be separated from the execution mechanism. The speaker emphasizes that the LLM (the 'brain') should only be responsible for planning the next steps, while a deterministic, durable workflow engine (the 'hands' or harness) must handle the actual execution. This approach treats agent harnesses as 'late-bound sagas,' ensuring reliability, managing side effects, and guaranteeing deterministic outcomes, which is crucial for mission-critical systems like SRE or payment processing.

Key takeaways

  1. Agent Scope Beyond Chatbots 0:01

    In production, agents are not limited to simple chatbots. They must handle background tasks, run on schedules, react to events (e.g., logs, alerts), and coordinate in multi-agent systems (1:52, 2:27).

  2. Harness as the Application 0:03

    An agent, when combined with a controlling harness, functions as an application, much like a set of microservices. The harness is responsible for delivering the overall business goal, integrating databases, internal systems, and human-in-the-loop approvals (3:02, 3:12).

  3. The Brain and the Hands Separation 0:09

    The core principle is the clear split: the LLM plans what should happen next (non-deterministic), but the harness executes the plan using deterministic code. This ensures reliable execution, especially for critical tasks like cluster restarts (9:56).

  4. Agentic Workflows as Late-Bound Sagas 0:10

    Agent harnesses are described as 'late-bound sagas.' They gain the benefits of traditional sagas (visibility, control) while allowing the agent to propose and build the workflow at runtime, rather than requiring the entire sequence to be defined upfront (10:21).

Watch on YouTube Full article

AI Agents Are Just Distributed Systems Now — Salman Munaf, TikTok thumbnail

· 19:48

AI Agents Are Just Distributed Systems Now — Salman Munaf, TikTok

The core argument is that AI agents, once they interact with external services (tools/APIs), cease being purely model problems and become complex distributed systems challenges. Building reliable agents requires incorporating robust distributed systems thinking—focusing on failure modes, state management, transactionality, and security controls—rather than just improving the LLM itself.

Key takeaways

  1. AI Agents as Probabilistic Coordinators 5:46

    Unlike traditional deterministic workflows that follow a predefined decision tree, AI agents act as probabilistic coordinators. This variability necessitates strong external controls (e.g., circuit breakers, spending ceilings) to ensure determinism and prevent severe consequences.

  2. Handling Failure: Timeout vs. Failure 11:59

    A timeout does not mean failure; it means unknown. To prevent unsafe side effects (like double refunds), agents must utilize request IDs and idempotency keys, allowing for status lookups to confirm if an action already succeeded.

  3. State Management and Context 17:37

    Context that influences an action is state. This state can become stale or conflict with the authoritative source of truth, requiring explicit invalidation and provenance tracking, much like managing a cache.

  4. Security and Control Boundaries

    Agents must operate with scoped credentials (separate read/write permissions) and limited tool access. Furthermore, human approval should be bound to specific actions, actors, and expiry times, not blanket privileges.

Watch on YouTube Full article