# Workshop: Building and optimizing dictation features

## Executive summary

This workshop details the architecture for building low-latency dictation features using AssemblyAI's APIs. The core strategy involves leveraging the Sync API—a single POST request—to achieve speed superior to traditional async or streaming methods. The process is optimized through three key stages: using key terms prompting for accuracy, implementing connection warming to minimize network overhead (DNS, TCP, TLS), and running a cleanup pass via LLM Gateway (e.g., Qwen3.5 4B Fast) to refine raw transcripts into polished, intent-preserving text. The goal is to deliver results on screen in under one second.

## Key takeaways

- Sync API Preference: The Sync API is preferred for dictation because it avoids the overhead of maintaining a WebSocket connection (streaming) and bypasses the inherent latency floor of traditional async endpoints, which is unsuitable for short, burst dictation sessions.
- Latency Optimization via Warming: To minimize latency, it is critical to call a warm endpoint (`client.sync.warm`) while the user is recording. This pre-pays the networking costs (DNS, TCP, TLS handshake), ensuring the subsequent transcription request goes straight to inference.
- Cleanup Pass with LLM Gateway: A cleanup step using an LLM (like Qwen3.5 4B Fast) is necessary to transform raw, spoken text (e.g., 'I think we should meet in 5 minutes') into polished, corrected text while preserving the original intent. This is achieved by prompting the model to act as a function in a pipeline, not an assistant.
- Upcoming Dictation API: AssemblyAI is rolling out a dedicated Dictation API (`client.dictation.describe`) that will wrap the entire loop (STT + Cleanup) into a single, simplified call, making the process easier for developers.

## Technical details

- Sync API Usage: The Sync API allows transcription via a single POST request, making it ideal for dictation bursts (estimated 10-12 seconds) where the overhead of continuous WebSocket connections would introduce unacceptable latency.
- Key Terms Prompting: This technique improves transcription accuracy by providing the model with an explicit vocabulary list (e.g., proper names or product terms) before the main transcription pass.
- Connection Warming: Calling `client.sync.warm` while recording pays the networking debt (DNS, TCP, TLS) upfront, significantly reducing the total turnaround time and improving perceived speed.
- Cleanup Model Selection: For cleanup, the speaker recommends starting with Qwen3.5 4B Fast due to its excellent balance of speed and cost efficiency. The LLM Gateway supports plug-and-play integration with various providers.
- Latency Budgeting: The goal is to achieve results on screen in less than one second. Latency is budgeted across STT inference (~140ms), transport (~100ms after warming), and cleanup (< 600ms).

## Practical implications

- For optimal performance, always implement connection warming while the user is recording to minimize network overhead.
- When designing the cleanup prompt, specify that the model is operating as a function in a pipeline, not an assistant, and provide few-shot examples for both desired fixes and guardrails (what not to change).
- To manage latency, focus on optimizing the cleanup step by using smaller, faster models (like Qwen3.5 4B Fast) and keeping prompts concise.
- The upcoming Dictation API will simplify the entire workflow by combining STT and cleanup into a single API call.

## Topics

Speech-to-Text, Natural Language Processing, Low-Latency Systems, API Design, Prompt Engineering, AssemblyAI SDK, AssemblyAI Website, Dictation API Waitlist

Source: https://www.youtube.com/watch?v=ldH50XCGNXg