Post-Training at the Frontier: How You.com, NVIDIA, and CoreWeave Improved Nemotron’s Agentic Search

Summary

You.com, NVIDIA, and CoreWeave collaborated to significantly improve Nemotron 3.5 Lightning's agentic web search and browsing capabilities through a rigorous post-training process. The effort focused on addressing the bottleneck of model tool usage proficiency. By leveraging You.com's proprietary web graph and index to generate specialized, decontaminated training data, the team applied Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on CoreWeave's dedicated infrastructure. This resulted in an 8.5 absolute increase (23% relative increase) in accuracy on the BrowseComp benchmark and drastically improved efficiency, reducing average tool calls per task from 168 to 115.

Technical details

• Model Architecture & Benchmarking 750s

  The base model was Nemotron 3.5 Lightning (a 30 billion parameter, 3 billion active parameter Mixture of Experts model). Evaluation was conducted using the BrowseComp benchmark, which requires sourcing facts and web browsing. The initial model achieved 37% on the benchmark.

• Failure Analysis 800s

  Initial model failures were attributed to weaker tool use, particularly with unique output schemas from web search tools. Patterns included 'second guessing' (over-searching after finding a correct answer) and 'death loops' (searching repeatedly without finding the correct information).

• Data Generation (Web Graph) 900s

  You.com utilized its web graph to generate data by focusing on pages in the middle band of popularity (avoiding boilerplate or spammy URLs). They then used an LLM to generate verifiable question-answer pairs from these pages, creating a dataset of retrieval questions.

• Training Pipeline 1000s

  The pipeline involved: 1) Filtering data by difficulty; 2) Decontaminating the dataset (ensuring training data did not contain answers to benchmark questions); 3) Running eight iterations of each question inside the Nemo harness with a binary reward judge; 4) Routing correct traces to SFT and remaining tasks to RL.

Mentioned resources

• Nemotron 3.5 Lightning (LLM Model)
• BrowseComp (Benchmark)
• Nemo gym (RL Platform)

Channel & topics

• Weights & Biases
• Large Language Models (LLMs)
• Reinforcement Learning (RL)
• Supervised Fine-Tuning (SFT)
• Web Search Infrastructure
• Agentic AI
• Nemotron 3.5 Lightning
• BrowseComp
• Nemo gym