How to trace your vibe-coded agent with W&B Weave
Summary
The video demonstrates how to implement comprehensive observability for AI agents using Weights & Biases (W&B) Weave and the W&B MCP server. By leveraging the `weave for agents SDK`, engineers can add full tracing—including conversations, turns, LLM calls, and tool executions—to an existing agent's logic without modifying its core code. This instrumentation allows developers to monitor performance metrics, track resource usage (tokens, cost), and debug complex interactions, such as identifying model hallucinations.
Key takeaways
-
Weave provides deep observability for AI agents
The tracing structure follows a clear hierarchy: Agent $\to$ Conversation $\to$ Turn $\to$ LLM Call + Tool Call. This detailed view is crucial for understanding agent behavior and performance.
-
Non-invasive instrumentation using W&B MCP
Observability can be added by prompting a coding assistant (like Claude Code) to inject the necessary tracing logic via the `weave for agents SDK`, avoiding changes to existing application code.
-
Debugging and Evaluation Capabilities
The Weave UI allows engineers to inspect individual conversations and turns, providing step-by-step visibility into tool usage (e.g., Tavali search) and LLM decisions. This is critical for debugging hallucinations or unexpected agent behavior.
Technical details
-
Weave Data Model Structure
225s
The tracing model is hierarchical: Agent (top level) $\to$ Conversation (tied by ID) $\to$ Turn (user message + agent response) $\to$ LLM Call (single API hit) or Tool Call (single tool execution).
-
Instrumentation Command Flow
320s
The process involves prompting the coding assistant to add observability using the `weave for agents SDK` and specifying that each run should be a conversation, provider workflows as turns, and search/LLM calls as tool/LLM spans.
-
W&B MCP Server Functionality
16s
The W&B MCP server is used to integrate observability into the IDE environment, allowing the agent to pull traces and documentation directly within the development workflow.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.