Universal 3.5 Pro Demo: Smarter Speech-to-Text with Contextual Awareness
This demo introduces Universal 3.5 Pro, an advanced Speech-to-Text (STT) model designed to significantly boost transcription accuracy through enhanced contextual awareness. Key features include passing domain-specific prompts (e.g., 'cardiology consultation'), applying context to key terms to prevent misapplication, and supporting dynamic mid-call prompt updates via API calls. Furthermore, the model retains conversation history (agent context), allowing it to accurately transcribe user input even in poor audio conditions by understanding the situational flow of a voice agent interaction.
Key takeaways
-
Contextual Prompting for Domain Accuracy
Passing detailed information about the audio content (e.g., 'cardiology consultation between Dr. Smith and elderly patient regarding chest pain...') dramatically improves model accuracy within specific domains. The more specific the prompt, the better the results.
-
Contextual Key Terms
2:00
Unlike previous methods where key terms were applied blindly, Universal 3.5 Pro allows users to define what a key term represents (e.g., 'The user's name is Zachary Klebanoff'). This prevents the model from incorrectly applying terminology based solely on acoustic similarity.
-
Dynamic Mid-Call Prompt Updates
2:55
The prompt can be updated in real time via the API (not available in the playground demo). This is crucial for voice agents, allowing tool calls or external data to adjust the model's context mid-conversation.
-
Conversation/Agent Context
3:30
The model retains previous transcriptions and accepts LLM-generated responses from a voice agent as context. This provides situational awareness, improving accuracy even in poor audio conditions and reducing the Word Error Rate (WER) on voice agent datasets.