# Small Models, Big Results: Training a Finance Agent for Under $500 — Charles Dickens, Snorkel AI

## Executive summary

This talk details how a specialized, smaller Language Model (Qwen3 4B) was trained using Reinforcement Learning (RL) on a domain-specific dataset (FinQA) derived from SEC 10-K filings. The resulting model significantly outperformed its much larger, generalist counterpart (235B parameters) on complex financial question-answering tasks, achieving approximately 60% pass@1 versus 51%. The core finding suggests that for enterprise AI workloads, reliability and specialization can outweigh raw parameter scale, especially when utilizing high-quality, domain-specific data and simple binary rewards.

## Key takeaways

- Specialization Over Scale: For complex, domain-specific tasks (like financial analysis), specialized workflows do not require generalist, polymath models. A smaller, focused model can achieve parity with, or exceed, much larger models, leading to better deployment economics.
- The Power of Simple Rewards: The most effective training setup was surprisingly simple: using a single binary reward signal (correct/incorrect) rather than complex, multi-component reward engineering. The bottleneck was identified as disciplined tool use, not reasoning depth.
- Generalization of Tool Discipline: The tool-use discipline learned on simple, single-table question-answer pairs successfully generalized to harder, multi-table questions requiring sequential decision-making and planning, without needing explicit multi-table training.

## Technical details

- FinQA Dataset Creation: The FinQA benchmark was built from SEC 10-K filings (annual reports) pulled from the Edgar system. The process involved generating SQL tables and question-answer pairs using the Qwen 3 30B model, followed by a three-layer verification process: programmatic consistency checks, automated reviews from independent agents, and expert manual review. This resulted in a dataset of expert-validated financial Q&A.
- Model Training and Architecture: The Qwen3 4B model was trained using the open-source rLLM framework (developed by UC Berkeley Sky Computing Lab collaborators). The training utilized Reinforcement Learning (RL) with a binary correctness reward, running in a REACT loop environment that provided access to the 7,000 generated tables. The entire process was completed for under $500.
- Failure Modes in LLM Agents: The analysis of frontier models revealed common failure modes: 1) Schema hallucination (assuming non-existent tables/columns), 2) Context flooding (overwhelming context with poorly formed queries like `SELECT *`), and 3) Poor recovery (repeating failed strategies instead of adapting to error messages).

## Practical implications

- For build engineers designing agentic systems, the focus should shift from maximizing model size to maximizing data quality and domain specificity.
- The reliability and auditability of agents operating in complex, structured environments (like legacy APIs or financial schemas) are paramount and can be improved through rigorous, multi-stage verification pipelines.
- RL fine-tuning with simple, binary rewards can be a highly cost-effective method for improving agent reliability in specialized domains, potentially reducing reliance on massive, expensive generalist models.

## Topics

Large Language Models (LLMs), Reinforcement Learning (RL), Agentic Systems, Data Engineering, Financial Technology (FinTech), rLLM framework, FinQA open-source benchmark, rLLM-FinQA-4B on Hugging Face

Source: https://www.youtube.com/watch?v=TyPpSRXGhbc