Between the Layers– Interpreting Large Language Models - Michelle Frost - NDC Toronto 2026
Summary
This talk provides a deep dive into AI Interpretability, arguing that for Large Language Models (LLMs) to achieve reliability and trust, we must move beyond external explanations (explainability) and understand the internal mechanisms (interpretability). The speaker outlines advanced research—particularly Anthropic's work on mechanistic interpretability—that focuses on decomposing model activations into sparse, single-meaning features. This shift allows engineers to treat LLMs not as black boxes, but as auditable systems that can be tested, monitored, and debugged at the circuit level.
Key takeaways
-
Interpretability vs. Explainability
1:48
Interpretability is the 'inside view' (understanding which internal parts wrote the answer), while explainability is the 'outside view' (a human-readable reason for the output). Mechanistic evidence comes from ablating suspected units and observing predictable behavior changes, whereas a salency map or local surrogate provides an account.
-
The Need for Operational Trust
3:50
To build trust in LLMs, we must make concepts like truthfulness and calibration operational. This requires understanding the model's internal mechanisms to ensure consistency (robustness) and prevent unsafe answers or agreement bias (syncopancy).
-
The Interpretability Litmus Test
5:20
A system should achieve 'understanding' if it can be able to: Name the units, Test their causal importance, Predict counterfactual failures (e.g., misinformation), Instrument them in production, and allow another engineer to Reproduce findings.
-
Modern Mechanistic Approaches
9:20
Advanced research uses tools like Sparse Autoencoders (SAEs) and overcomplete dictionaries to decompose model activations into 'features'—internal signals that track a single concept (monosemanticity). This allows for the mapping of complex behaviors to specific, nameable circuits.
-
The Evolution of AI Safety
11:20
Interpretability is presented as 'ethical infrastructure.' By understanding internal mechanisms (e.g., default circuits that trigger refusals), we can move from simply measuring outputs to actively debugging and governing the system's behavior.
Technical details
-
Model Architecture Components
480s
Transformers process text by mapping tokens into vectors using an embedding matrix. Within a transformer block, each token creates three versions of itself: the Query (what it looks for), the Key (what it offers), and the Value (its content). Information is passed through attention mechanisms and Multi-Layer Perceptrons (MLPs) into a residual stream.
-
Mechanistic Interpretability Tools
560s
Key tools include: **Sparse Autoencoders (SAEs)**, which are used to unmix overlapped signals; **Attribution Graphs**, which map features influencing tokens across layers; and **Default Circuits**, which represent inherent safety behaviors or refusals.
-
Causal Testing
320s
The core method involves generating a mechanistic story (a hypothesis) and then using probes or steering ablations to test if the predicted behavior change occurs when internal units are manipulated. This provides causal evidence.
-
Fairness Assessment
420s
Counterfactual edits—the smallest realistic changes that flip a model's decision (e.g., flipping gender pronouns)—are crucial for proving bias, as demonstrated by the Compass dataset analysis.
Mentioned resources
- Anthropic Research Papers
- Neuronipedia
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.