Workshop: Building and optimizing dictation features

Published 2026-09-09 · Duration 36:06

Summary

This workshop details the architecture for building low-latency dictation features using AssemblyAI's APIs. The core strategy involves leveraging the Sync API—a single POST request—to achieve speed superior to traditional async or streaming methods. The process is optimized through three key stages: using key terms prompting for accuracy, implementing connection warming to minimize network overhead (DNS, TCP, TLS), and running a cleanup pass via LLM Gateway (e.g., Qwen3.5 4B Fast) to refine raw transcripts into polished, intent-preserving text. The goal is to deliver results on screen in under one second.

Download summary

Key takeaways

Sync API Preference 5:35
The Sync API is preferred for dictation because it avoids the overhead of maintaining a WebSocket connection (streaming) and bypasses the inherent latency floor of traditional async endpoints, which is unsuitable for short, burst dictation sessions.
Latency Optimization via Warming 11:40
To minimize latency, it is critical to call a warm endpoint (`client.sync.warm`) while the user is recording. This pre-pays the networking costs (DNS, TCP, TLS handshake), ensuring the subsequent transcription request goes straight to inference.
Cleanup Pass with LLM Gateway 18:20
A cleanup step using an LLM (like Qwen3.5 4B Fast) is necessary to transform raw, spoken text (e.g., 'I think we should meet in 5 minutes') into polished, corrected text while preserving the original intent. This is achieved by prompting the model to act as a function in a pipeline, not an assistant.
Upcoming Dictation API 30:00
AssemblyAI is rolling out a dedicated Dictation API (`client.dictation.describe`) that will wrap the entire loop (STT + Cleanup) into a single, simplified call, making the process easier for developers.

Technical details

Sync API Usage 335s
The Sync API allows transcription via a single POST request, making it ideal for dictation bursts (estimated 10-12 seconds) where the overhead of continuous WebSocket connections would introduce unacceptable latency.
Key Terms Prompting 530s
This technique improves transcription accuracy by providing the model with an explicit vocabulary list (e.g., proper names or product terms) before the main transcription pass.
Connection Warming 700s
Calling `client.sync.warm` while recording pays the networking debt (DNS, TCP, TLS) upfront, significantly reducing the total turnaround time and improving perceived speed.
Cleanup Model Selection 1100s
For cleanup, the speaker recommends starting with Qwen3.5 4B Fast due to its excellent balance of speed and cost efficiency. The LLM Gateway supports plug-and-play integration with various providers.
Latency Budgeting 1400s
The goal is to achieve results on screen in less than one second. Latency is budgeted across STT inference (~140ms), transport (~100ms after warming), and cleanup (< 600ms).

Mentioned resources

AssemblyAI SDK (Library)
AssemblyAI Website (Website)
Dictation API Waitlist (Product Page)

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.