Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
This workshop details the use of NVIDIA Data Designer, an open-source framework, to build synthetic data pipelines for training advanced AI agents. The core focus is moving beyond simple memory retrieval to teaching agents complex, multi-step behaviors like web search and reasoning. Data Designer allows users to create structured, diverse, and scalable training data by defining a Directed Acyclic Graph (DAG) that governs data generation, incorporating seed data, samplers, and tool calls (via Model Context Protocol, MCP).
Key takeaways
-
Synthetic Data Generation Pipeline
10:23
Data Designer allows users to create synthetic data by defining a DAG that generates data column by column, using seed data (e.g., Wikipedia) and sampler columns (e.g., difficulty level) to inject diversity and ground the data in real-world distributions. This process is designed to be version-controllable and shareable, treating data creation as code.
-
Teaching Agentic Search Behavior
28:10
To teach a model to search, the pipeline must incorporate tools (like Tavi search) and structure the data to require tool calling. The process involves creating 'search riddles' from graph paths (e.g., Wikidata hops) that are difficult enough to prevent the model from simply recalling the answer, forcing it to use the search tool.
-
Data Quality and Evaluation
43:20
Data quality is paramount. The process involves significant filtering and validation, often funneling tens of thousands of seeds into thousands of high-quality records. Evaluation requires a multi-faceted approach, combining LLM as a judge, deterministic checks, and human review, especially since source graphs (like Wikidata) can become stale.
-
Advanced Data Types and Personas
50:00
Data Designer supports creating complex data types beyond simple text, including structured outputs (JSON, YAML, XML), multi-turn tool calls, and data adhering to specific demographic distributions using 'Personas' (e.g., Neotron Personas) to learn from aggregate, sensitive information.