# Bringing Continual Learning into Enterprises — Samuel Denton, Applied Compute

## Executive summary

Applied Compute presents a framework for implementing continual learning in enterprise AI agents by mapping out a 'distillation spectrum.' This model uses a two-axis grid: the type of production trace (Offline vs. Online) and the source of hints (Static/Offline vs. Dynamic/Online). The goal is to improve agent behavior without relying on a 'golden answer.' Key findings demonstrate that online hinting paired with online traces offers the highest ceiling for continuous improvement, while offline methods provide immediate value using historical data dumps.

## Key takeaways

- The Continual Learning Spectrum: Continual learning can be approached across four quadrants defined by trace type (Offline/Online) and hint source (Static/Dynamic). The most scalable method is Quadrant 4: Online hints paired with online production traces, enabling continuous improvement as the model serves live traffic.
- Improving Agents Without Golden Answers: The framework emphasizes improving agents without requiring a 'golden answer' or perfect rubric. This approach allows for targeted behavior changes (e.g., encouraging tool calls) by conditioning the rollout on existing production traces and injecting hints, rather than forcing specific tokens.
- Online Hinting Achieves High Performance Gains: In a fully online setting, dynamic hints based on the model's own recent rollout can dramatically improve highly specialized behaviors (e.g., correct hyperlink formatting), raising performance from 15% to 80%, significantly outperforming static offline hinting methods.

## Technical details

- Distillation Spectrum Definition: The spectrum ranges from 'offline distillation' (learning from a single batch of production traces) to the 'holy grail' of continual learning, which is a unified engine where inference and training occur in the same loop.
- Online vs. Offline Hinting: Hints can be static/offline (e.g., general priors like preventing over-refunds) or dynamic/online, meaning they are constructed based on what the model just did during its live rollout.
- SWE-bench Tool Call Improvement: Using offline hints with a production trace, Qwen 3.5 was successfully guided to increase its task complete tool call rate from ~22% to 60% on SWE-bench without degrading the overall test pass rate. This was achieved by forcing the reasoning path toward the tool call rather than changing the tool call tokens themselves.
- Per Step Hinting and Masking: For optimal distillation, it is critical to inject hints at specific points (per step hinting) in the rollout, focusing on only the next few steps forward. Additionally, using a relevance mask during self-distillation prevents the teacher model's irrelevant preferences for connector words from degrading learning ability.

## Practical implications

- Enterprises can implement continuous learning loops for AI agents using historical production data (offline) or by integrating model updates directly into the serving pipeline (online).
- The framework allows targeted behavioral improvements (e.g., formatting, reasoning pathing) even when a definitive 'golden answer' is unavailable.
- By focusing on online hints, organizations can achieve scalable, high-ceiling performance gains in live production environments.

## Topics

Continual Learning, LLM Distillation, Applied Compute, SWE-bench, Online/Offline AI, Applied Compute Blog Post

Source: https://www.youtube.com/watch?v=ZTA0GwpAUak