# Hugging Face Journal Club: Kimi K3

## Executive summary

The discussion summarizes the Kimi K3 tech report, detailing a highly complex and scaled training pipeline for large language models. Key innovations include specialized agentic structures (general, deep research, coding), multi-tier on-policy distillation across nine domain experts, and advanced infrastructure techniques like partial rollout schemes and dynamic resource scheduling. The talk emphasizes that the model's performance is achieved through sophisticated engineering efforts rather than a single breakthrough concept.

## Key takeaways

- Multi-Tier Expert Specialization: The training pipeline involves generating nine specialized domain experts (3 reasoning levels: low, high, max) for three general domains (general tasks, deep research, coding). These are combined using multi-tier on-policy distillation into a single student model.
- Partial Rollout Scheme: A novel RL technique where the system samples $k$ rollouts from $n$ prompts. It uses an additional parameter $\lambda$ to measure completed rollouts within a budget, updating weights on that subset and pushing incomplete rollouts into subsequent iterations.
- Extensible Chat Template: The Kimi K3 model utilizes an 'extensible token markup language' for its chat template. This design aims to be easy to learn during SFT while remaining flexible enough to accommodate future modalities without re-engineering the core template.
- Advanced Infrastructure Scaling: The system employs sophisticated resource management, including a dynamic rollout auto throttling scheduler based on KV cache pressure and a method for collocating RL training and inference by automatically balancing GPU resources between the two tasks.

## Technical details

- Training Pipeline Architecture: The process involves Supervised Fine-Tuning (SFT) on a base model, followed by training specialized experts using Reinforcement Learning (RL). The final step is multi-tier on-policy distillation, where the student model learns from these multiple expert teachers.
- Reasoning Effort Tuning: The reasoning effort (low, high, max) is discretized and tuned via a stage-wise curriculum. Initial budgets are estimated from the SFT model, and training progresses by annealing the token budget multiplier to generate distinct expert models.
- Quantization Aware Training: The method involves quantizing both weights and activations of the Mixture of Experts (MOE) model to save memory. Crucially, they ensure that the trainer and inference server use the same quantization schema (e.g., MXFP4 FP8 format) to eliminate mismatch.
- Agentic Reward Modeling: The reward model is designed as an agent itself, moving beyond simple pairwise classifiers. It generates a rubric on the fly based on the policy's output and scores against it, allowing for fine-grained feedback.

## Practical implications

- The multi-tier distillation approach provides a clean paradigm for improving general models by delegating complex RL tasks to specialized experts.
- Implementing dynamic resource scheduling (e.g., balancing GPU time between training and inference) is critical for scaling large, mixed-mode LLMs.
- Adopting extensible chat template standards minimizes engineering overhead when adding new modalities or capabilities to future model generations.

## Topics

Large Language Models, Reinforcement Learning, Mixture of Experts (MOE), Model Quantization, Agentic AI, Training Infrastructure, Kimi K3 Tech Report, OpenAI GPT-5.6 models

Source: https://www.youtube.com/watch?v=MW8-kqd2SD8