# AI Agents Need a Babysitter

## Executive summary

The discussion explores the architectural challenges of deploying autonomous AI agents, focusing on moving beyond simple API calls to reliable, auditable, and intent-driven workflows. Key solutions discussed include using intent-based tool selection mechanisms like Jev, enhancing agent capabilities via WebMCP, and implementing robust, outcome-based testing (e.g., Cucumber) to manage the increasing volume of AI-generated code contributions. The overarching theme is the need for 'babysitters'—verification layers—to ensure AI agents operate reliably and maintain business integrity.

## Key takeaways

- Intent-Based Tool Selection (Jev): Instead of having an agent select a tool based on its name, a more robust approach is for the agent to define its desired *intention* (the outcome). A mechanism like Jev then recommends the appropriate tool based on this intent, providing an auditable trail of *why* the tool was called, not just *that* it was called. This is crucial for improving reliability and fine-tuning agent behavior. [02:40]
- WebMCP for Workflow Enablement: WebMCP (Web Machine Capability Protocol) allows web-hosted sites to expose tools to agents. This enables complex, multi-stage workflows—like navigating an e-commerce site—where each page or step surfaces new, context-specific tools, creating a 'skill tree' effect. [16:04].
- Shifting Focus to Business Outcomes: The industry is shifting from designing APIs that expose raw data/functions (CRUD) to designing APIs and systems that enable entire workflows or 'jobs to be done.' This requires thinking about the desired end state rather than the specific tools needed to achieve it. [20:38].
- Verifying Business Intent in Code: To manage AI-generated code, verification must move beyond testing if the code *works* (unit tests) to testing *why* the code exists. Frameworks like Cucumber allow developers to encode business requirements and intended outcomes directly into tests, ensuring the code solves the correct problem. [36:31].

## Technical details

- Jev (Tool Finding): A tool designed to classify tool calls based on a desired action and effect (intention). It allows agents to define what they want to achieve, and Jev finds the appropriate tool, making the process auditable at the intent level. [02:40]
- MCP / WebMCP: MCP (Machine Capability Protocol) and its web-hosted counterpart, WebMCP, are protocols that allow agents to interact with and utilize tools exposed by websites. WebMCP enables dynamic tool surfacing, where new tools appear as the agent progresses through a workflow (e.g., moving from a pet store landing page to a dog food aisle). [16:04].
- Hybrid/Blended Agents: A proposed method for increasing agent reliability is using a hybrid approach, such as combining a specialized classifier model (like Jev) with a smaller, more robust LLM (like Luna or Haiku) as a fallback mechanism when the primary classifier fails. [11:50].
- Outcome-Based Testing (Cucumber): A testing framework (Java-based) that shifts testing focus from method functionality to business outcomes. It allows developers to encode high-level business requirements into tests, verifying the intended result of the code. [36:31].

## Practical implications

- Implement robust, multi-layered verification systems (QA agents) that test for business outcomes, not just code syntax.
- Adopt workflow-centric API design, focusing on the 'job to be done' rather than raw data access.
- Integrate intent-based classification layers (like Jev) into agent architectures to provide auditable decision-making paths.
- Utilize WebMCP to model complex, multi-step user journeys, allowing the agent to dynamically discover tools at each stage of the interaction.

## Topics

AI Agents, MLOps, Workflow Automation, Software Testing, API Design, Grok, Pi (Harness), builder.io (Steve)

Source: https://www.youtube.com/watch?v=YIDKxCgrRjs