Continual Learning — The Video Signal

Aug 12, 2026 · 20:02

Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain

The continuous improvement of autonomous agents requires shifting focus from code determinism to data mining agent traces. The speaker argues that observability and continual learning are fundamentally linked: an agent's actions in an environment generate a trace record that serves as the substrate for all future improvements. Techniques like harness engineering, distillation (SFT), and analyzing counterfactual traces allow developers to systematically improve agents at lower costs than relying solely on frontier models.

Key takeaways

Shipping is the First Step
To gather data for improvement, an agent must be deployed into a real-world environment (shipping it). This process generates valuable trace data from tool calls, API usage, and CLIs.
Observability = Continual Learning 4:04
There is a tight coupling between observability and continual learning for agents. Both require comprehensive traces—the record of actions taken in the environment—to allow the agent to update its internal knowledge or definition.
The Value of Traces 6:00
Traces capture fine-grained behavior that simple pass/fail benchmarks miss. They are crucial for proving counterfactuals (e.g., comparing GPT 5.5 vs. GLM 5.2) and understanding how agents behave at a granular level.
Improvement Loop Strategy 13:00
For agent improvement, the recommended loop is: Start with Harness Engineering (fast feedback, ~2 minutes) $\rightarrow$ Saturate this ceiling $\rightarrow$ Fine-tune the model to break through it $\rightarrow$ Return to Harness Engineering.
The Future of Data 17:00
Agent activity will generate data at an exponential rate, potentially eclipsing all human-produced data in history. Managing this requires building systems that can efficiently mine and process traces.

Watch on YouTube Full article

Aug 12, 2026 · 19:31

Lessons from Studying Every Memory System — Shlok Khemani, Independent

The talk provides a deep dive into the evolution and architectural differences of memory systems in consumer AI applications (ChatGPT, Claude, Gemini). The core thesis is that 'memory' is not a standardized technology but rather a function of compute, requiring careful trade-offs between profile size, update frequency, and context window cost. Speakers highlight that while general architectures are converging toward running profiles, the specific implementation details remain unique to each product, meaning memory cannot be outsourced.

Key takeaways

Memory is not standardized
There is no single way to implement AI memory; products evolve independently (e.g., ChatGPT uses dense keywords/running profile; Claude uses full sentences/tools).
The Compute Trade-off
A running profile requires balancing two costs: the cost to maintain (update frequency/compute) and the serving cost (profile length in context window). This trade-off dictates product design.
Memory is a Product Problem
The biggest limitation of current AI memory systems is not technology, but product design. They often fail to reason over rich external sources like emails or calendars, leading to conflicts (e.g., conflicting travel dates).

Watch on YouTube Full article

Aug 12, 2026 · 19:03

Bringing Continual Learning into Enterprises — Samuel Denton, Applied Compute

Applied Compute presents a framework for implementing continual learning in enterprise AI agents by mapping out a 'distillation spectrum.' This model uses a two-axis grid: the type of production trace (Offline vs. Online) and the source of hints (Static/Offline vs. Dynamic/Online). The goal is to improve agent behavior without relying on a 'golden answer.' Key findings demonstrate that online hinting paired with online traces offers the highest ceiling for continuous improvement, while offline methods provide immediate value using historical data dumps.

Key takeaways

The Continual Learning Spectrum
Continual learning can be approached across four quadrants defined by trace type (Offline/Online) and hint source (Static/Dynamic). The most scalable method is Quadrant 4: Online hints paired with online production traces, enabling continuous improvement as the model serves live traffic.
Improving Agents Without Golden Answers 5:20
The framework emphasizes improving agents without requiring a 'golden answer' or perfect rubric. This approach allows for targeted behavior changes (e.g., encouraging tool calls) by conditioning the rollout on existing production traces and injecting hints, rather than forcing specific tokens.
Online Hinting Achieves High Performance Gains 12:00
In a fully online setting, dynamic hints based on the model's own recent rollout can dramatically improve highly specialized behaviors (e.g., correct hyperlink formatting), raising performance from 15% to 80%, significantly outperforming static offline hinting methods.

Watch on YouTube Full article

Aug 12, 2026 · 19:43

Intelligence + Continual Learning = Expertise — Yu Su, NeoCognition

The talk distinguishes between 'Intelligence' (the capacity to reason through unfamiliar problems from available context) and 'Expertise' (accumulated, situated competence). While modern LLM agents excel at symbolic tasks like coding because code is a structured language-native world, they struggle in heterogeneous real-world digital environments. The speaker posits that this difficulty represents a modern Moravec's paradox. To scale AI beyond basic capability, systems must implement continual learning to acquire specialized expertise for each 'microworld,' leading toward 'unbounded expertise from bounded intelligence.'

Key takeaways

Intelligence vs. Expertise Distinction 3:50
Intelligence is the ability to reason through novel problems given context, while expertise is accumulated competence that allows for efficient action and judgment in a specific domain (e.g., recognizing constraints beyond just finding a shared calendar slot).
The Coding Agent Advantage 5:26
Coding is an ideal first market for LLM agents because code is already symbolic and structured, providing clear tests and rewards. Leaving this 'privileged world of code' introduces significant brittleness.
The Role of Continual Learning 10:44
Continual learning is defined as the adaptive compression of experience into reusable structures for future behavior. It is presented as the critical bridge needed to transition from raw intelligence (brute-forcing solutions) to specialized expertise (compressing the search space).

Watch on YouTube Full article

Aug 12, 2026 · 19:42

Scaling Compute on Context — Jack Morris, Engram

This talk introduces the concept of 'Scaling Compute on Context,' addressing the fundamental limitation that current Large Language Models (LLMs) are trained exclusively on public data. While traditional deep learning progress relies on scaling three axes—more data, more compute, and bigger models—the inability to scale with private corporate or personal data limits model depth. The speaker reviews several advanced techniques (e.g., KV compaction, on-policy distillation, continued pretraining) aimed at transferring domain expertise from a fixed, unstructured corpus ($D$) into the model's weights, ultimately seeking a self-improving training mechanism analogous to AlphaGo.

Key takeaways

The Knowledge Gap: Public vs. Private Data 5:59
Current LLMs are superb on public data (Wikipedia, GitHub) but lack deep knowledge of private corporate or personal contexts (emails, internal meetings). This is the core problem that limits model depth and expertise acquisition.
Scaling Compute on Context 8:40
Since scaling with new public data is limited, the focus shifts to maximizing compute power given a fixed, private dataset ($D$). This approach aims to imbue the pre-trained model ($ heta$) with deep domain knowledge from $D$.
The Goal: Self-Improvement and Depth Scaling 11:45
Most current methods eventually saturate (hitting a 'data wall'). The desired property is self-improvement, where getting better makes the training questions harder, allowing compute to continuously buy depth rather than flattening out.

Watch on YouTube Full article

Jul 31, 2026 · 19:27

Reinforcement Learning without Verifiable Rewards — Will Brown, Prime Intellect

Will Brown discusses extending Reinforcement Learning (RL) into complex, real-world tasks that lack clean, verifiable rewards. The core thesis is that 'environments' must serve as the anchor for learning. Techniques like grounding in source material, using LLM judges to audit actions, and employing a reverse direction trick are necessary to generate reliable reward signals when ground truth is unavailable. The ultimate goal is enabling continual learning—allowing deployed agents to autonomously improve by observing and correcting mistakes in messy production settings.

Key takeaways

The Shift from Verifiable Rewards 6:53
Traditional RL thrives on verifiable rewards (e.g., math, code test cases). However, most real-world tasks (like writing reports or handling refunds) are fuzzy and lack clean best answers, requiring new methods to generate reliable signals.
Environments as the Learning Anchor
An 'environment' is defined by a task, a harness (e.g., Docker image, codebase), and a scoring rule/verifier. These objects can be used not only for RL but also for Supervised Fine-Tuning (SFT) or prompt optimization.
Mitigating Reward Hacking
Since loose proxies for objectives can be exploited, careful design is crucial. Techniques include inspecting traces, running small experiments, and using judges to audit rollouts in hindsight.
Generating Signal via Reverse Direction 17:26
A powerful technique involves working backward: starting from a known solution or artifact (like a completed PR) and training the model to find it again, providing verifiable steps for an initially hard problem.

Watch on YouTube Full article

Jul 31, 2026 · 48:41

Building AI That Learns and Adapts: A Case Study in MRI Diagnostics - Agata Chudzińska

This technical talk explores Continual Learning (CL), positioning it as a critical missing piece for building robust, adaptive AI systems. Using a case study in MRI diagnostics for detecting Focal Cortical Dysplasias (FCDs) in epilepsy patients, the speaker details how CL techniques mitigate issues like data drift and catastrophic forgetting. The presentation covers advanced ML architectures (Encoder-Decoder), preprocessing steps (e.g., segmentation, Junction/Extension filters), and three main CL methods: regularization-based, memory-based, and architecture-based. The goal is to enable models to adapt continuously in privacy-sensitive environments without requiring full retraining.

Key takeaways

Continual Learning (CL) addresses core ML deployment challenges. 3:30
CL allows AI models to learn and adapt over time from new data streams without forgetting previously acquired knowledge, solving issues like catastrophic forgetting and data drift. This is crucial for high-stakes fields like medicine.
Medical imaging requires specialized preprocessing and architectures. 6:50
For FCD detection from MRI scans, the process involves brain tissue segmentation and applying feature enhancement filters (e.g., Junction and Extension) before feeding data into an Encoder-Decoder variant of a neural network architecture.
CL methods offer solutions for resource constraints. 10:05
Instead of retraining from scratch (which is costly, time-consuming, and raises privacy concerns), CL utilizes regularization-based, memory-based, or architecture-based approaches to maintain performance while adapting to new tasks.
The future of AI requires continuous adaptation. 20:05
The speaker emphasizes that CL is not just a 'nice to have' feature but a strategic architectural choice for any production-level ML system dealing with evolving data and strict privacy requirements.

Watch on YouTube Full article

Jul 13, 2026 · 49:44

The AI Memory Problem: Why Long Context Isn’t Enough — Dan Biderman, Engram Co-founder & CEO

The talk addresses the limitations of current AI architectures—specifically that simply increasing context window size or relying solely on Retrieval-Augmented Generation (RAG) is insufficient for building truly intelligent, long-horizon agents. Dan Biderman introduces a paradigm shift focusing on 'continual learning' and knowledge compression. Key solutions include using specialized knowledge representations called 'cartridges,' implementing 'test-time training' (or test-time compute), and achieving high token efficiency to enable models to handle the anticipated petabytes of proprietary enterprise data.

Key takeaways

Beyond RAG: The Need for Internalized Knowledge 15:14
Current methods like RAG are limited because they only provide external, textual context. True intelligence requires embedding knowledge into the model's parameters (weights) to achieve 'intuition,' allowing the model to generalize and extrapolate beyond explicit notes or recipes.
The Problem of Scale: Context Rot and Token Limits 23:30
As companies accumulate trillions of tokens of proprietary data, simple context management fails due to 'context rot' (the model becoming less accurate the more context it reads) and extreme token consumption. This necessitates methods that are both highly efficient and scalable.
The Solution Stack: Cartridges, Training, and Memory 30:05
Engram proposes a multi-faceted approach combining knowledge compression via 'cartridges' (compact capsules of knowledge), gradient-based updates during inference ('test-time training'), and advanced memory layers to achieve superior token efficiency and model accuracy.
The Future: Autonomous, Personalized AI
The ultimate goal is a system where the model autonomously determines what knowledge should be internalized (in weights) versus what should remain external (in text/RAG), creating personalized models that improve continuously with user interaction, similar to nurturing a Tamagotchi.

Watch on YouTube Full article