Small Models, Big Results: Training a Finance Agent for Under $500 — Charles Dickens, Snorkel AI
Summary
This talk details how a specialized, smaller Language Model (Qwen3 4B) was trained using Reinforcement Learning (RL) on a domain-specific dataset (FinQA) derived from SEC 10-K filings. The resulting model significantly outperformed its much larger, generalist counterpart (235B parameters) on complex financial question-answering tasks, achieving approximately 60% pass@1 versus 51%. The core finding suggests that for enterprise AI workloads, reliability and specialization can outweigh raw parameter scale, especially when utilizing high-quality, domain-specific data and simple binary rewards.
Key takeaways
-
Specialization Over Scale
For complex, domain-specific tasks (like financial analysis), specialized workflows do not require generalist, polymath models. A smaller, focused model can achieve parity with, or exceed, much larger models, leading to better deployment economics.
-
The Power of Simple Rewards
13:20
The most effective training setup was surprisingly simple: using a single binary reward signal (correct/incorrect) rather than complex, multi-component reward engineering. The bottleneck was identified as disciplined tool use, not reasoning depth.
-
Generalization of Tool Discipline
The tool-use discipline learned on simple, single-table question-answer pairs successfully generalized to harder, multi-table questions requiring sequential decision-making and planning, without needing explicit multi-table training.
Technical details
-
FinQA Dataset Creation
506s
The FinQA benchmark was built from SEC 10-K filings (annual reports) pulled from the Edgar system. The process involved generating SQL tables and question-answer pairs using the Qwen 3 30B model, followed by a three-layer verification process: programmatic consistency checks, automated reviews from independent agents, and expert manual review. This resulted in a dataset of expert-validated financial Q&A.
-
Model Training and Architecture
920s
The Qwen3 4B model was trained using the open-source rLLM framework (developed by UC Berkeley Sky Computing Lab collaborators). The training utilized Reinforcement Learning (RL) with a binary correctness reward, running in a REACT loop environment that provided access to the 7,000 generated tables. The entire process was completed for under $500.
-
Failure Modes in LLM Agents
620s
The analysis of frontier models revealed common failure modes: 1) Schema hallucination (assuming non-existent tables/columns), 2) Context flooding (overwhelming context with poorly formed queries like `SELECT *`), and 3) Poor recovery (repeating failed strategies instead of adapting to error messages).
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.