# How Harmonic 4x'd User Retention by Building on Deep Agents

## Executive summary

Harmonic transitioned its natural language interface, Scout, from a brittle query parsing graph to an architecture built on Deep Agents and a simple model-plus-tools loop. This shift quadrupled week one to week four user retention. The core technical lesson is that robust agent design requires managing context via a 'harness contract,' ensuring that all artifacts (like visualizations or large search result sets) are visible to the model—either in the message list or offloaded through file system tools—to prevent the UX from becoming an invisible black box.

## Key takeaways

- Deep Agents significantly boost retention: Switching to Deep Agents resulted in a fourfold increase in week one to week four user retention for Scout. (1:24)
- The agent architecture simplified from graphs to loops: Scout evolved from complex, multi-node query parsing graphs (LangGraph) into a simpler model and tools loop, mediated by middleware. (2:41)
- Context management is handled by the harness: Deep Agents manage context overload using mechanisms like compaction for long message lists and file system abstraction to store large results, returning only pointers to the model. (4:56)
- UX must respect the agent's context contract: For a product UX to be useful, any rendered element (e.g., charts) must either reside in the message list or be discoverable by the model via tools/file system pointers; otherwise, it is invisible to the agent. (7:04)

## Technical details

- Architectural Shift: The initial Scout 1.0 used a complex query parsing graph where each node had its own model and required extensive maintenance. The new architecture simplifies this to a Model + Tools loop, utilizing Deep Agents as the harness. (2:41)
- Deep Agent Context Management: Deep Agents manage context by running compaction when message lists get too long and using file system tools to store large artifacts, allowing the model to access chunks via pointers rather than receiving massive data dumps. (4:56)
- The Harness Contract: The contract dictates that anything offloaded by the harness must be made available to the model through tools, ensuring progressive disclosure of context. (7:04)
- Handling Large Results (Search): For large datasets (e.g., thousands of search results), the best practice is to use a dedicated tool that returns an ID, allowing subsequent tools to fetch specific batches or artifacts from a shared file system, rather than dumping all data into the message list. (13:43)

## Practical implications

- When building LLM-powered applications, treat the agent's context window as a finite resource; offload large data sets to persistent storage (like a file system) and expose retrieval tools instead.
- Design UX elements (visualizations, sidebars) that are not just visually appealing but are also programmatically accessible or discoverable by the underlying LLM model via tool calls or message content.
- Prioritize robust context management mechanisms (e.g., compaction, file system abstraction) over simply passing all raw output into the chat history.

## Topics

AI Agents, LLMs, Natural Language Processing, Context Management, Software Architecture, Build Engineering, LangGraph, LangSmith, LangChain Academy

Source: https://www.youtube.com/watch?v=pGdZBK___jM