# Universal 3.5 Pro Demo: Smarter Speech-to-Text with Contextual Awareness

## Executive summary

This demo introduces Universal 3.5 Pro, an advanced Speech-to-Text (STT) model designed to significantly boost transcription accuracy through enhanced contextual awareness. Key features include passing domain-specific prompts (e.g., 'cardiology consultation'), applying context to key terms to prevent misapplication, and supporting dynamic mid-call prompt updates via API calls. Furthermore, the model retains conversation history (agent context), allowing it to accurately transcribe user input even in poor audio conditions by understanding the situational flow of a voice agent interaction.

## Key takeaways

- Contextual Prompting for Domain Accuracy: Passing detailed information about the audio content (e.g., 'cardiology consultation between Dr. Smith and elderly patient regarding chest pain...') dramatically improves model accuracy within specific domains. The more specific the prompt, the better the results.
- Contextual Key Terms: Unlike previous methods where key terms were applied blindly, Universal 3.5 Pro allows users to define what a key term represents (e.g., 'The user's name is Zachary Klebanoff'). This prevents the model from incorrectly applying terminology based solely on acoustic similarity.
- Dynamic Mid-Call Prompt Updates: The prompt can be updated in real time via the API (not available in the playground demo). This is crucial for voice agents, allowing tool calls or external data to adjust the model's context mid-conversation.
- Conversation/Agent Context: The model retains previous transcriptions and accepts LLM-generated responses from a voice agent as context. This provides situational awareness, improving accuracy even in poor audio conditions and reducing the Word Error Rate (WER) on voice agent datasets.

## Technical details

- Contextual Prompting: The model accepts descriptive prompts to guide transcription, ranging from general domains (e.g., 'medical consultation call') to highly specific scenarios.
- Key Term Context Application: Contextual prompting allows the model to distinguish between acoustically similar phrases and defined key terms, ensuring accurate transcription even when ambiguity exists (e.g., differentiating a name from common speech).
- API Integration for Dynamic Context: Mid-call prompt updates are achieved through the API, enabling systems to inject new contextual information (e.g., 'the caller's name is Lance Armstrong and his bike tire is popped') as events occur.
- Agent Context Flow: The system automatically retains speech-to-text transcripts within the session. Developers can pass LLM-generated agent responses directly to the model, providing a complete conversational history for improved accuracy.

## Practical implications

- Build engineers can integrate Universal 3.5 Pro into voice agent pipelines to significantly improve the reliability and accuracy of real-time transcription, especially in noisy or complex conversational environments.
- The ability to dynamically update prompts mid-call allows for sophisticated state management within IVR/voice bot systems, improving user experience by maintaining context awareness.
- Developers can leverage Agent Context to build robust voice applications that handle multi-turn conversations and tool-based data lookups (e.g., retrieving product names after a background API call).

## Topics

Speech-to-Text, Voice AI, Contextual Awareness, LLM Integration, API Development, Natural Language Processing, AssemblyAI Website, AssemblyAI Twitter

Source: https://www.youtube.com/watch?v=gygb4kqf2vk