Autoresearch Made Our Models 3x Faster — Tejas Bhakta, Morph
The video details how combining autoresearch frameworks with custom GPU kernels and bare-metal hardware tweaks can achieve significant model inference speedups (up to 3x). Autoresearch is highly effective for tuning low-level parameters (like block sizes) but requires human input for high-level architectural ideas (e.g., pipelining). Optimization involves addressing compute, memory, and overhead bottlenecks by providing agents with detailed hardware (e.g., B200's T-M, TMA) and model context, while guarding against 'reward hacking' behaviors.
Key takeaways
-
Autoresearch Mechanism
Autoresearch is a framework where an agent iteratively proposes solutions and refines them towards a defined goal (e.g., X times faster) within a loop, making it ideal for verifiable low-level operators like CUDA kernels.
-
Human vs. Agent Roles
0:01
Humans must provide the groundbreaking ideas (e.g., 'This top method is dumb, let's pipeline it instead'), while the autoresearch agent handles the low-level tuning, parameter selection, and verification.
-
Optimization Stacking
0:05
Speed gains are cumulative: custom kernels, bare-metal tweaks (e.g., BIOS settings, overclocking), and architectural improvements can be stacked to achieve significant overall speedups.