We Built an AI Support Agent That Resolves 80% of Tickets — AssemblyAI
Summary
Matt Lawler of AssemblyAI details the creation of 'Joey,' an advanced AI support agent designed to solve the scaling bottleneck inherent in the Forward Deployed Engineer (FDE) model. Faced with 1,000 API signups daily and limited human resources, the team replaced an ineffective off-the-shelf bot (10% resolution) with Joey. Joey, built on the Claude Agent SDK, achieves an 80% end-to-end resolution rate for approximately $700/month. The architecture leverages local markdown documentation, external embeddings, agentic file search, and a large `CLAUDE.md` system prompt. The agent's capabilities were further enhanced by integrating AssemblyAI's Voice Agent API, allowing for real-time, low-latency voice interactions.
Key takeaways
-
Scaling the FDE Model
5:28
The FDE model, while effective for building trust, does not scale to handle high volume (e.g., 1,000 API signups/day). Automation is necessary to maintain a high-quality customer experience at scale.
-
Agentic Architecture Improvement
8:37
Joey was built to overcome limitations of vendor bots by giving the agent full control over its infrastructure, including file system access, tool calling, and the ability to write/debug code.
-
Performance Gains
13:22
The agent increased the end-to-end resolution rate from 10% to 80% in the first week of deployment, costing around $700/month.
-
Voice Integration
15:31
The agent was given a voice interface using the Voice Agent API, which strings together speech-to-text, LLM, and text-to-speech over a single websocket connection for real-time, low-latency interaction.
Technical details
-
Agent Architecture
612s
Joey is built on the Claude Agent SDK and utilizes four major components: 1) Local markdown documentation (ensuring the agent has access to all current docs, even if the main doc site is down); 2) Embeddings (from Voyage) for efficient retrieval; 3) Agentic file search for deep resource stringing; and 4) A large `CLAUDE.md` file containing guardrails and operational advice.
-
Deployment and CI/CD
722s
The system was deployed on Railway, enabling rapid iteration. Fixes can be written as a PR and deployed live within approximately 30 seconds, allowing for real-time bug fixes during a live session.
-
Voice Agent Implementation
931s
The voice functionality uses the Voice Agent API, which manages the entire pipeline (speech-to-text $\rightarrow$ LLM $\rightarrow$ text-to-speech) over a single websocket connection, handling pauses and interruptions.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.