Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA

Published 2026-10-11 · Duration 55:30

Summary

This workshop details the use of NVIDIA Data Designer, an open-source framework, to build synthetic data pipelines for training advanced AI agents. The core focus is moving beyond simple memory retrieval to teaching agents complex, multi-step behaviors like web search and reasoning. Data Designer allows users to create structured, diverse, and scalable training data by defining a Directed Acyclic Graph (DAG) that governs data generation, incorporating seed data, samplers, and tool calls (via Model Context Protocol, MCP).

Download summary

Key takeaways

Synthetic Data Generation Pipeline 10:23
Data Designer allows users to create synthetic data by defining a DAG that generates data column by column, using seed data (e.g., Wikipedia) and sampler columns (e.g., difficulty level) to inject diversity and ground the data in real-world distributions. This process is designed to be version-controllable and shareable, treating data creation as code.
Teaching Agentic Search Behavior 28:10
To teach a model to search, the pipeline must incorporate tools (like Tavi search) and structure the data to require tool calling. The process involves creating 'search riddles' from graph paths (e.g., Wikidata hops) that are difficult enough to prevent the model from simply recalling the answer, forcing it to use the search tool.
Data Quality and Evaluation 43:20
Data quality is paramount. The process involves significant filtering and validation, often funneling tens of thousands of seeds into thousands of high-quality records. Evaluation requires a multi-faceted approach, combining LLM as a judge, deterministic checks, and human review, especially since source graphs (like Wikidata) can become stale.
Advanced Data Types and Personas 50:00
Data Designer supports creating complex data types beyond simple text, including structured outputs (JSON, YAML, XML), multi-turn tool calls, and data adhering to specific demographic distributions using 'Personas' (e.g., Neotron Personas) to learn from aggregate, sensitive information.

Technical details

NVIDIA Data Designer 120s
An open-source framework used to build synthetic data pipelines. It generates data by defining a DAG, allowing for structured, reproducible data creation. Installation is done via `pip install data designer`.
Model Context Protocol (MCP) 1370s
A standard that allows an agent to connect to external tools. In the context of search, it enables the model to use a search tool (like Tavi) to find answers rather than relying solely on internal knowledge.
Search Riddle Creation 1750s
The process of generating complex questions from graph paths (e.g., Wikidata hops) that require multi-step reasoning and external search to solve. This is crucial for training agents to perform real-time information retrieval.
Data Generation Workflow 120s
The workflow involves: 1) Defining a seed data set (CSV, JSONL, Parquet); 2) Adding sampler columns (e.g., difficulty); 3) Using prompt templates to guide LLM generation; 4) Utilizing structured columns (JSON/Markdown) for reliable output; 5) Running in 'preview mode' before batch execution.
Data Quality & Evaluation 2600s
Data quality is assessed using multiple methods: LLM as a judge, deterministic checks, and human review. The process emphasizes filtering and refining data (e.g., reducing 50,000 seeds to 7,000 records) to ensure accuracy, even when source data is stale.

Mentioned resources

NVIDIA Data Designer (Framework)
Tavi Search API (Search Tool)
Neotron (Foundation Models)

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.