Between the Layers– Interpreting Large Language Models - Michelle Frost - NDC Toronto 2026
This talk provides a deep dive into AI Interpretability, arguing that for Large Language Models (LLMs) to achieve reliability and trust, we must move beyond external explanations (explainability) and understand the internal mechanisms (interpretability). The speaker outlines advanced research—particularly Anthropic's work on mechanistic interpretability—that focuses on decomposing model activations into sparse, single-meaning features. This shift allows engineers to treat LLMs not as black boxes, but as auditable systems that can be tested, monitored, and debugged at the circuit level.
Key takeaways
-
Interpretability vs. Explainability
1:48
Interpretability is the 'inside view' (understanding which internal parts wrote the answer), while explainability is the 'outside view' (a human-readable reason for the output). Mechanistic evidence comes from ablating suspected units and observing predictable behavior changes, whereas a salency map or local surrogate provides an account.
-
The Need for Operational Trust
3:50
To build trust in LLMs, we must make concepts like truthfulness and calibration operational. This requires understanding the model's internal mechanisms to ensure consistency (robustness) and prevent unsafe answers or agreement bias (syncopancy).
-
The Interpretability Litmus Test
5:20
A system should achieve 'understanding' if it can be able to: Name the units, Test their causal importance, Predict counterfactual failures (e.g., misinformation), Instrument them in production, and allow another engineer to Reproduce findings.
-
Modern Mechanistic Approaches
9:20
Advanced research uses tools like Sparse Autoencoders (SAEs) and overcomplete dictionaries to decompose model activations into 'features'—internal signals that track a single concept (monosemanticity). This allows for the mapping of complex behaviors to specific, nameable circuits.
-
The Evolution of AI Safety
11:20
Interpretability is presented as 'ethical infrastructure.' By understanding internal mechanisms (e.g., default circuits that trigger refusals), we can move from simply measuring outputs to actively debugging and governing the system's behavior.