How Harmonic 4x'd User Retention by Building on Deep Agents

Summary

Harmonic transitioned its natural language interface, Scout, from a brittle query parsing graph to an architecture built on Deep Agents and a simple model-plus-tools loop. This shift quadrupled week one to week four user retention. The core technical lesson is that robust agent design requires managing context via a 'harness contract,' ensuring that all artifacts (like visualizations or large search result sets) are visible to the model—either in the message list or offloaded through file system tools—to prevent the UX from becoming an invisible black box.

Technical details

• Architectural Shift 241s

  The initial Scout 1.0 used a complex query parsing graph where each node had its own model and required extensive maintenance. The new architecture simplifies this to a Model + Tools loop, utilizing Deep Agents as the harness. (2:41)

• Deep Agent Context Management 496s

  Deep Agents manage context by running compaction when message lists get too long and using file system tools to store large artifacts, allowing the model to access chunks via pointers rather than receiving massive data dumps. (4:56)

• The Harness Contract 704s

  The contract dictates that anything offloaded by the harness must be made available to the model through tools, ensuring progressive disclosure of context. (7:04)

• Handling Large Results (Search)

  For large datasets (e.g., thousands of search results), the best practice is to use a dedicated tool that returns an ID, allowing subsequent tools to fetch specific batches or artifacts from a shared file system, rather than dumping all data into the message list. (13:43)

Mentioned resources

• LangGraph (Library/Framework)
• LangSmith (Platform)
• LangChain Academy (Educational Resource)

Channel & topics

• LangChain
• AI Agents
• LLMs
• Natural Language Processing
• Context Management
• Software Architecture
• Build Engineering
• LangGraph
• LangSmith
• LangChain Academy