Act, Confirm, or Stop? Smarter behavior for AI assistants, wearables & robots — Amit Desai, Roku
The presentation argues that improving voice AI user experience requires focusing on a second, often neglected dimension: system behavior under uncertainty. While increasing accuracy (Knob One) is critical, the system's ability to intelligently decide what to do when it is unsure (Knob Two) can yield greater user satisfaction. This is quantified using the Outcome User Cost Heuristic (OUCH), which minimizes the total user effort by assigning differential costs to various bad outcomes (e.g., playing the wrong song vs. simply stating 'I did not understand').
Key takeaways
-
The Two Knobs of Voice AI Improvement
0:03
User satisfaction can be improved by increasing technical accuracy (Knob One) or by optimizing the system's decision-making process when confidence is low (Knob Two). The latter is often overlooked.
-
The Outcome User Cost Heuristic (OUCH)
0:10
Instead of treating all errors equally, OUCH minimizes the total user cost by quantifying the relative pain of different bad outcomes (e.g., the effort required to stop a wrong song vs. the time taken to hear 'Sorry, I did not understand').
-
Adding Conversational Behavior
0:13
Introducing a third behavior—confirming the guess out loud (e.g., 'Did you mean ABC?')—splits the confidence range into three regions (Stop, Confirm, Act) and further lowers the overall user cost.
