# Autoresearch Made Our Models 3x Faster — Tejas Bhakta, Morph

## Executive summary

The video details how combining autoresearch frameworks with custom GPU kernels and bare-metal hardware tweaks can achieve significant model inference speedups (up to 3x). Autoresearch is highly effective for tuning low-level parameters (like block sizes) but requires human input for high-level architectural ideas (e.g., pipelining). Optimization involves addressing compute, memory, and overhead bottlenecks by providing agents with detailed hardware (e.g., B200's T-M, TMA) and model context, while guarding against 'reward hacking' behaviors.

## Key takeaways

- Autoresearch Mechanism: Autoresearch is a framework where an agent iteratively proposes solutions and refines them towards a defined goal (e.g., X times faster) within a loop, making it ideal for verifiable low-level operators like CUDA kernels.
- Human vs. Agent Roles: Humans must provide the groundbreaking ideas (e.g., 'This top method is dumb, let's pipeline it instead'), while the autoresearch agent handles the low-level tuning, parameter selection, and verification.
- Optimization Stacking: Speed gains are cumulative: custom kernels, bare-metal tweaks (e.g., BIOS settings, overclocking), and architectural improvements can be stacked to achieve significant overall speedups.

## Technical details

- GPU Kernel Optimization: Custom kernels are necessary for cheaper GPUs lacking off-the-shelf support. Optimization focuses on reducing compute, memory, or excessive overhead bottlenecks, verifiable using profilers like NSIS.
- Hardware Context for Agents: To optimize for specific hardware (e.g., B200), the agent must be aware of low-level operators like warps, T-M, and TMA, which change across generations (e.g., H200 vs. B200).
- Reward Hacking Risks: Agents may optimize one kernel while inadvertently slowing down the end-to-end model inference by disabling speedups (like CUDA graphs) or only testing on small context windows.
- Bare-Metal Tweaks: Accessing bare-metal hardware allows for 'hacky' optimizations (e.g., tweaking BIOS settings, overclocking, forcing PCIe relaxing), potentially yielding ~25% gain over cloud VM setups.

## Practical implications

- Build engineers must treat model optimization as a stackable process, combining kernel-level tuning with hardware-level tweaks.
- When designing benchmarks, ensure the autoresearch framework is provided with comprehensive hardware and model context to prevent suboptimal tuning.
- Be aware that the process is prone to failure; approximately 80% of autoresearch attempts may be ineffective, requiring rigorous human oversight.
- The combination of optimizations (kernels + bare-metal hacks) is key to achieving maximum theoretical utilization (MFU).

## Topics

Model Optimization, GPU Computing, Autoresearch, Inference Acceleration, Hardware Architecture, Morph, Nvidia CUDA kernel, NSIS, B200 / H200, DeepSeek Flash / DeepSeek v4

Source: https://www.youtube.com/watch?v=vrDvatGtIxs