Adaption Labs: Gradient-Free Continual Learning — Sara Hooker, Adaption
The talk argues that the frontier of AI development is shifting away from relying solely on massive, co-located pre-training compute (the 'Death of Scaling'). Instead, democratization and innovation are being driven by automating model training loops and optimizing for distributed, post-training customization. The speaker introduces AutoScientist, a system designed to co-optimize the entire model lifecycle—from data quality through alignment—allowing builders to achieve frontier capabilities with greater efficiency and accessibility.
Key takeaways
-
The Narrow Path of AI Research
2:36
Historically, contributing to AI breakthroughs required following a narrow path (right PhD, right lab), which was compounded by the need for enormous compute resources. This has created significant barriers to entry globally.
-
AutoScientist Automates Model Training
10:45
The system automates model training by co-optimizing the entire loop (data $ ightarrow$ alignment). It is designed to self-evolve based on domain and data type, outperforming research staff because it searches across a broader space of architectures (dense vs. Mixture of Experts) than human researchers typically consider.
-
Shifting Compute Paradigms
17:12
Pre-training size is no longer the most rewarding axis for scale; instead, distributed inference compute and post-training customization are becoming more lucrative. This shift means that algorithms and recipes matter more than simply hoarding GPUs.