The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend - David vonThenen
This talk details acoustic keystroke logging—a method of intercepting typed information purely from sound rather than physical interception. The speaker outlines how deep learning models can be trained using spectrographic images derived from recorded key presses to classify specific letters (e.g., 'S'). While demonstrating the high accuracy of single-keyboard attacks (100%), the presentation shows that multi-keyboard logging is challenging but feasible, especially when combined with context prediction and spell-checking algorithms. The session concludes by emphasizing defensive measures, including two-factor authentication using physical keys and implementing strong, unique, offline password policies.
Key takeaways
-
Acoustic Keystroke Logging Mechanism
16:22
The attack relies on machine learning audio classification. Audio files (linear 16 format) are converted into spectrographic images (frequency over time, visualized as a heatmap), which serve as the input for training models like Convolutional Neural Networks (CNNs).
-
Multi-Keyboard Attack Complexity
28:12
While single-keyboard classification can achieve 100% accuracy, using multiple keyboards significantly lowers confidence scores. The problem is made solvable by decomposing the text based on space delimiters and employing spell-checking/context prediction (e.g., predicting 'hello people' from partial sound inputs).
-
Defensive Strategies
40:50
Defense requires layered security: use physical two-factor authentication keys (like YubiKey) instead of SMS; utilize unique, complex passwords that are not known to the user; and be aware of potential signal interference or noise.