Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA

The workshop is about something more generic and more powerful. It's about how do we build data pipelines, um, using the tool that we created, NeMo, uh, Data Designer, uh, to be able to create or add any kind of functionality to a model or an agent. So let's get started. And by the way, I have a GitHub repo, uh, where I have some notebooks. So if you're interested, you can scan the QR code, or you can go to that GitHub repo. Um, and I'll have the same QR code on all the other slides as well, so if

you miss it, there'll be more chances to get it.

So what is the workshop about, and what will we learn here? Um, the first thing we're gonna learn is how do you actually take a target agent behavior that you want and turn it into a data problem, right? So basically, can we create data for this targeted agentic behavior? And then can we train a model to learn how to, um, uh, do that task? And so we built a tool called Data Designer to build, uh, synthetic data pipelines, uh, which does

exactly that. And the use case that we're gonna target today is search. Um, and by the way, this is a very reusable framework, so if you wanna create data for something apart from search, this is exactly how you would do it. And there's two ways to participate in today's workshop. So, um, we have, you know, of course, you can follow live, try to run the notebooks. But since we just have, like, probably an hour or less than an hour, um, you could also listen now and then run the notebooks later. Um,

so yeah, make sure you get the GitHub link. And if you want the slides, I also have the slides on the, on the, uh, GitHub repo.

So you're gonna need four things if you're gonna follow along either now or after the workshop. Obviously, you need a GPU-enabled machine, so a cloud VM or a workstation. Um, you also need a Tavily search API key. Tavily is a company that does, uh, search for agents. If you sign up on their website, they'll give you a thousand credits for free. And you need Data Designer, which is our tool for creating data. It's fully open source. Anyone can download it, use it, do whatever they want with it.

And all the notebooks that I've provided, um, on the GitHub repo.

So here's the agenda for today. So we'll first spend a few minutes talking about what is Data Designer, and why do we actually need a framework like this. And then we'll talk about, uh, the basics. Um, there's three notebooks. I'm gonna actually be showing you code. Um, the first notebook is about the basics of Data Designer. How do you actually create data that you want? And then the second notebook is about model context protocol or MCP. How do you actually, uh, create agentic data? And then the

third notebook, which is what, um, is the super interesting use case of search. How do you actually teach a model to do search? And then we'll leave some time at the end for questions.

So I wanna start off with a quote from Andrej Karpathy, and he said, "Internet data is not the data you want for your transformer." So this is, um, a very, uh, valid point because internet data could be, you know, high-quality sources like Wikipedia, or it could be, you know, someone's opinion on Twitter or X or some other platform, right? So how do we actually know, um, what data is useful for training AI models and what data is not useful? So, uh, we need to process this

internet data, curate it, and then, uh, create training data from it. So that's the traditional approach. What we're proposing is that we could actually synthetically generate data, uh, either from scratch or from a seed dataset. And so that's what Data Designer is for. So Data Designer is a framework to create synthetic data. And what do I mean by, um, seed data? So seed data, like you could use Wikipedia data as

a seed and then try to create synthetic training data from Wikipedia. So your synthetic data is not just made up by a model, it is grounded in real-world, uh, distribution and real-world data.

So Data Designer is a completely open-source framework. We have a large team at NVIDIA that works on this. Um, and, uh, you can just install it by, uh, running pip install datadesigner.

And I think, um, what's really, um, what we're really proud of is Data Designer has been used to create six and a half trillion tokens of, um, training data, and this is increasing every single day. Um, and yeah, we're keeping track of the GitHub stars as well. So if you like our repo, give us a star.

So, uh, what is synthetic data generation, right? Um, well, one way to do synthetic data generation is take a loop, run it a million times, and then if you wanna create a product review dataset, just, you know, continually prompt a LLM saying, "Give me a product, give me a review, and then give me a customer profile." That's one way to create a dataset. But you're gonna run into a bunch of issues if you do that. So what's missing? Um, well, diversity, diversity, diversity. That's the most

important thing, right? If you continually prompt a LLM after ten, twelve, maybe twenty tries, it's gonna start repeating stuff. It's not gonna give you what you need. Um, you're also missing correlations between all the features that you're looking for in your dataset. Um, you don't have multimodality. Um, you know, you're not really able to steer the creation of that dataset, and there's no validation, verification, uh, and so on. And if I create a dataset, I can't really share it with you

because how do you actually take the, uh, process that I use to create my dataset to create something similar? So that's why we created Data Designer. So how does it work? Uh, we have many different columns or features in Data Designer. Um, these columns are essentially, um, a Data Designer abstraction. In Data Designer, we generate data column by column. So first you have your seed column, which is

completely optional. So I gave the example of Wikipedia, which could be a seed dataset. Um, we could use any dataset as a seed, so anything which is a CSV, JSONL, Parquet file. So you could use your own company data as seed, and then you can try to, uh, add sampler columns to inject diversity. So let's say you are trying to create a coding dataset. Well, a very simpler sampler column would be like easy, medium, and difficult coding questions. So

that's a sampler column you could add, which is difficulty. You could also add sub-samplers. So for easy, you could map it to a specific type of problem. For medium, you could ma-map it to a specific type of problem, and so on. So we'll get into all of this. I know it feels a bit abstract, but we'll get into all of this once we look at the code. Um, and then we have columns for generating text, for generating structured data, for generating code. Uh, we also-- uh, Data Designer is now multimodal, so you can generate images as well. Um,

and you can validate your generations too.

So the whole idea is we build a DAG or a graph which describes how your dataset should be, and then this DAG, once you're ready to execute on it, you can just hit one click and get a dataset out of it. So there's three different notebooks we'll walk through today. Um, I know some of you are thinking, "Well, in twenty twenty-six, he's gonna be showing us code, right? I mean, now there's Codex and Claude Code and all these amazing tools." So I think if you understand the

code, it'll be much easier to use Codex or Claude Code to write your own, uh, synthetic data workflows for you. Uh, so we'll just take a brief look at each notebook now.

And by the way, this is our repo. So this is our Data Designer repo. It's fully open source. Um, and this is the GitHub repo that, um, I have created for this workshop, where we have notebooks. Um, so you can try out notebook one, two, three, and then we also have, um, the slides here if you wanna follow along, or if you want access to the slides after the workshop. So let's go to the first

notebook. So this is our intro to Data Designer notebook. Can everyone see okay, or should I zoom it?

It's good?

Okay. So first things first, we set up our environment. Um, there's a bunch of, uh, prompts in here that we're importing. And then what we do is we just, you know, import Data Designer. That's it, literally. Uh, and once you do that, you see in Data Designer, there's a concept of model providers. Um, so we're currently using, uh, vLLM, so that's local inference. So I'm running this on a s-- on a, on a NVIDIA GPU

cluster, and then we have access to different types of models, right? So on my local cluster, I have access to gpt-oss-120b. You can use any model or any open model to create synthetic data. Um, some closed models prevent you, because of licensing issues, uh, from creating data from it and using it for training models. But open models are usually fair game. So I'm using this model and, um, the goal here is can I actually take a dataset of Wikipedia topics,

uh, for this notebook? So like, you know, different, like, uh, what-- Wikipedia has so many different pages and, um, uh, many different topics. So we just made a list of those, put it in a CSV, and then we basically just call our config builder with seed dataset. And so essentially now we're, um, trying to create a Data Designer workflow, but with Wikipedia, uh, topics. Um, so once you do that, you can add question difficulty. You can say easy, medium, hard, however you want

your dataset to be. And then there's weights you can specify, so you can have more medium questions than, um, you know, easy or hard. And then there's all kinds of sampler columns you can add about question difficulty. So, for example, for easy, you might do, like, direct lookup or a simple fact or definition, right? But difficult might be you want something in more detail or obscure. Um, and then once we get to the LLM-generated columns, you just call this function

add_column, and then you have your prompt, which is essentially gonna, um, look something like this.

Just trying to find it.

Yep. So you are an expert trivia question writer. Um, so here's a topic, difficulty, question type. So whenever you put these, uh, Jinja templating for topic, what it's gonna do is the topics you created in the previous, um, or in the seed that you had from your Wikipedia, it's gonna insert those at runtime into this prompt. So essentially every prompt that the LLM sees is now unique and diverse, and so the model is gonna give you diverse data when you

actually hit generate. Um, and then, um, you, you can also have like a structured column. So a structured column is when instead of trying to generate like text data, um, you can try to generate like a JSON or Markdown format or any custom format that you are interested in. Um, and, uh, we know that LLMs, you know, we have to be nice to them, we have to talk to them, but sometimes they may not give you that structured format that you want, so then we just end up dropping those records.

So the way, um, data creation works is we have something called preview mode in Data Designer. So essentially when you hit preview, what's gonna happen is it's gonna try to create three records for you, and then you can go and look at those records. So let's look at some records. So all of this will make a lot more sense. So the topic here is the World Wide Web, um, and the question-- or I guess some consortium of the World Wide Web, and the question is: Which W3C

recommendation is identified by the same three-letter abbreviation that serves as a airport code for some city in Norway? So there's no way I would know the answer to this question, right? That's why these questions are essentially hard, 'cause we want to invoke the model to do search, right? But in this first notebook, we're just trying to create question answer pairs. So, uh, which is a very simple dataset. So I guess this is the answer the model came up with. And then you can also add LLM as a judge to score how your model is doing. So this is the question the model came up with,

this is the answer it came up with, and then, um, this is the question quality. So we can have another model judge what, uh, it thinks of the question that the, uh, previous model created, um, and so on. So in the end, you end up with a dataset like this. And by the way, we have a, a function that you can just push your datasets directly to Hugging Face. Um, so it's super easy to use. And, um, you can also visualize all of your questions here, as you can see.

And then you can get some stats about your dataset. Obviously, we generated only three records here, so, uh, you know, these stats are not very meaningful. But if you generate like hundreds or thousands of records, you could do that. And then we have something called batch mode, so which is data designer.create. And here, instead of a hundred, I could just, you know, put how many other records I need, and then I can go and create all these records. And as I mentioned previously, we generate data column by column. So in the batch mode, first you generate

the questions, and then you generate the answers, right? So this is obviously a very simple example. And then you have the LLM as a judge. Um, and here is your final dataset. That's what it looks like. So that was the first notebook. I think it's quite straightforward once you look at it and try it out. Um, we have a second notebook which focuses more on MCP, and then we're gonna build to the third notebook, which I think everyone will find interesting 'cause that's where we'll actually talk

about search. Sorry?

Can you go back to one slide where

Yeah.

And also the first GitHub link.

Oh, yeah, the-- all, all the GitHub links are the same. I'm just putting them in every single slide. So if you missed the first one, you can, you can get it. But I'll pause here for a minute if you wanna scan.

So, um, let's talk about MCP. So MCP is a standard which allows, um, an agent to essentially connect to tools, right? So the tool here is Tavily, which is nothing but a search tool for agents. There are many other search tools as well. Um, so that's basically the, um, uh... What, what we need, the mo-- What we're trying to teach the model here is how do you do search, right? So, uh, Tavily is a good tool

for the model to use to learn how to do that search. Um, so going back to the second notebook.

So here what we'll do is we'll just build upon some simple concepts from the first notebook, and we'll add, um, how to create agentic data. So again, you have the imports, you have the, the model configs, and then the same seed dataset, the same sampler column, so question difficulty, question type, and so on. And then you actually create the question, and then at the end, you get like a config, right? I don't know why it's printed, printed this in like, uh,

light color. But yeah, this is essentially your graph, um, that you're creating. And then you have the answer. So it's the exact same thing, except now when you actually create the answer, you add this tool alias here. And so the goal is, instead of just trying to answer the question, the model is now gonna use a Tavily search tool to try to find the answer on the internet, right? So that's-- We're build,

we're building upon the first notebook where we were just like, "Okay, please generate a question, and then please generate an answer." Now it's, "Please generate a question," and then, "Hey, you have this tool available. Can you use this tool to go to try to find the answer rather than trying to generate the answer?" So that's, that's the difference between the first and the second notebook.

And then again, you have the preview mode where you can generate a few records, and then you look at your records. And by the way, data creation is very iterative. It's not like one and done. So when you create some data, look at it. If you're not happy with it, then probably you go change your workflow, change your prompts, and then, you know, iterate on it. And then once you're happy with it, you can send it off to batch execution. So let's look at what, um, the second notebook produced. So let's say

the question is- Uh, to which German city did Telefunken move its headquarters in 1949? Does anyone know the answer? No, right? I mean, I don't know the answer. So, um, what we want to do is create questions that are difficult, so it invokes reasoning and tool calling from the model. And so when the model is actually trying to respond to this, um, it's-- as you can see here, it's using Tavily. So the

first search it made is, Telefunken moved its headquarters in 1949 to which company? So essentially, it's using one search tool call here. And then, um, it's gonna get an answer. So this is the query from the model, and then here are the results. These are all the URLs. And then for each URL, it's gonna give you a title, and then it's also giving, giving you the content. So that's what Tavily gives us, right? If you make a query. Or that's what it gives the agent when the agent makes the query. And then you end up with the response. And

then finally, at the end, you can create your dataset with batch mode, just like we did last time. And if you're thinking, "He just blazed through this whole first two notebooks," that's because the crux of it is in the third notebook, right? So, um, that's what I really want us to focus on today. So let's talk about search, um, 'cause we just went through a couple of notebooks showing us how to use Data Designer, and that's fine. But what's the problem we're actually trying to solve?

So, um, like I mentioned before, I'm a research scientist at NVIDIA, and we're also part of the Nemotron team. So Nemotron is a collection of found-- open source foundation models, um, that we've created. Uh, we just released our latest model, Ultra, but we also have two other models. So we have three different sizes of models. And so as part of that work, we actually had to teach Nemotron how to do search, and that's very important, right? 'Cause maybe this feels like a long time ago now, but maybe two

or three years ago, I'm sure you all remember looking at, like, I don't know, ChatGPT, Claude, or Gemini, and then looking at the web UI, and it says, "The knowledge cutoff date is December twenty twenty-three." So the reason it said that is because, well, you have to start training the model at some point. So you have to stop collecting and creating data at some point, right? So let's say they're training the model in January twenty-four. By December twenty-three, they need to, like, um, essentially freeze the data. So

anything that the model knows after, um, December twenty-three is not part of its training data. So if you had to a-ask a question like, "What is the weather today?" There's no way the model can answer that question from its training data, right? Uh, question?

Yeah. So yeah, definitely scan the QR code. So the idea is, how do you actually, um, teach the model about current events? Because learning about current events seems as important, if not more important, than, you know, knowing about, uh, things from the past, right? So, um, that's why search exists, and that's why models need to learn how to do search. Um, so coming back to the slide, um, so teaching Nemotron

models how to use search tools, that was our motivation. And, um, um, we-- essentially, the model needs access to real-time information to be able to do useful things. So how did we actually do this? So before I get to how did we do this, we need something to measure progress on, right? So, um, in the foundation model community, when you need to measure progress on something, you use a benchmark, which is like a standardized test set without the answers, right? Like, we don't have the

answers. So one of the benchmarks for search is BrowseComp, uh, which is a really difficult benchmark. And, uh, the FIFA World Cup is going on right now, so I don't know how many soccer fans there are here, but I'm, I'm a big soccer fan. But we don't call it soccer. But, um, between nineteen ninety and nineteen ninety-four, which teams played in a soccer match with a Brazilian referee, had four yellow cards, two for each team, three in total, blah, blah, blah. I won't read the rest of the question, but it's, it's quite

complicated question, right? There's no way that any of us know the answer to that, or maybe there's like, you know, someone who's, um, really well-versed in this who has the answer at the top of their head. What we would do back in the day, like, you know, three, four years ago, is we'd actually try to break a problem down if we had a complicated, uh, query like this, and then we would reason about what queries to actually make. We would go and search on Google or elsewhere, uh, a-get access to more information,

and then reason about what was our original goal, what information did we get access to, and then proceed from there, and then finally arrive at the answer, right? So the answer is Ireland versus Romania. I, I didn't know that. Um, so how did we actually create a dataset that allows us to train a model to go from zero percent to thirty percent on this BrowseComp benchmark? Um, so that's what we're gonna learn about today. So to do that, we do-- we did

something, um, uh, we, we used a seed dataset, which I described earlier. Except our seed dataset is not just Wikipedia. It's something that, um, is essentially a graph of Wikipedia topics. So it's called Wikidata. And what we did is we did a few hops along this graph, and that is our seed dataset. And I know the diagram on the right looks like a tree. It's not a tree, it's a graph. So, um, if it looks like a tree, that's my fault.

Uh, it should, it should, it should be a graph. Um, and essentially, um, you can see, um, on the right, the first topic is NVIDIA, and then NVIDIA is connected to Jensen, who is our CEO, and then that is connected to Oregon State University, 'cause Wiki- Wikidata says that that's where he went to college. And then- That college is located in Benton County, and Benton County is loca-- is actually named after a person called

Thomas Hart Benton. So if you took, um... If you randomly sampled an entity in Wikidata and then did five to eight hops along it, and I'll explain why five to eight, you'll end up, um, with a hop like this, like the green, you know, set of entities with, like, arrows in between. So that is essentially our seed dataset to create, uh, search riddles. So one search riddle that I could

create from this, and I'm gonna have to read this from my speaker notes, is, um:

Which company was founded by the person who went to college where the county was named after Thomas Hart Benton? So there's no way the LLM knows the answer to that in the training data, right? 'Cause if we give an agent the chance to cheat, it's gonna cheat. It's gonna u- just give us the answer. It's not gonna use the tool calls or the tools, sorry, like search that we want it to use, right? So we wanna make the questions as difficult and as, I guess, um, riddle-like as possible,

so we can invoke the search behavior from the model. So this is our seed. So I, I, I spoke about the first step, which is create random walks with seed. And why four to eight or five to eight hops? Sometimes it just, you know, if you do these hops, it just goes completely off-topic. Like, it will say, um, I don't know, like, um, a person's name, and then it will say, well, human being. Well, sure, that is true, but, you know, it's not really related. Like, basically, your hops will go

off-topic, so that's why we want to, as a heuristic, stay between four and eight. That's what worked well for me. And then you transform these seeds into search riddles, and then, um, you execute the search via the search tool, which is Tavily. So that's how you create a dataset with a prompt and a, with a search prompt and a search response, right? And then we evaluate our data, and then something that's not mentioned here is, you know, we train our models, right? So once you have the dataset, you train the model

with it, and then you measure performance on a standardized benchmark. So let's look at the notebook and see how we can do this. Any questions so far?

Yes.

So the reason, uh, it was important to do-- I guess it's not difficult questions, it's questions that the model, uh, cannot find on the internet readily, uh, cannot find answers readily on the internet to. So, like, if you took that full blob of text that we had in the BrowseComp benchmark, and you put that on the web, you're not gonna get the answer. I mean, you will get the answer 'cause of the AI search thing with Gemini now, but that's what we're trying to teach the model, essentially. Yeah.

Okay. So, um, so this is the tutorial for the end-to-end agentic search.

So we already spoke about the way we do this and how we create the data. We already spoke about how we import all these, um, prompts, and I wrote a function to, like, use Tavily with MCP. You can go and look at the code later. Um, and

here is where we have our seed entities. So this is the example I showed you. This is just one seed entity. So the way, uh, my seed collection algorithm works is, like, you can run it a million times, and you get a million different hops along Wikidata, right? So those are just topics we wanna ground our search riddle creation on. So another topic is, um, we like modern art and, like, I'm not really an art buff, but-- So I don't

understand most of these words, but essentially, this is, like, another seed hop that you can see.

So the way to create this dataset is, um, well, you are an expert search agent that uses web search to answer questions accurately, provide valid JSON with the final answer, supporting URLs, and a short justification. And here are the tools you have available. You have one tool, which is Tavily search, and then here are some usage guidelines for the tool. So, like, we're like: Okay, please, please don't use more than, like, you know... Don't, don't, like, call the tool, like, a hundred times and spend,

you know, thousand dollars or something. I guess those are, like, all kinds of instructions you can put in there. But essentially, um, uh, jokes aside, what you want here is, um, your agent shouldn't go crazy and try to, like, you know, spend way too much money answering a question that's very easy or, you know, something like that. So that's why we have all these instructions, and we also have an example. So this is, um, um, something

like in-context learning, right? So you put an example into the prompt, and then the model can essentially, um, learn from that example how to actually call the s-search tool. So here's your prompt that you want to create. So here what we do is, um, we collected the Wikidata hops. We can insert those hops into the prompt here, uh, which is our, um, you know, seed entity. And then, uh,

we ask the model to essentially come up with a question, um, using any or all of the seed entities from this hop. And then we ask the model to create a response using, uh, this AgentSolution raw, which I showed earlier.

So in the, in the second notebook. So once you do this, you'll end up with a preview dataset. And so here's our preview dataset. So Indian Council of Medical Research is the start entity for the seed, and then apparently that's located in a city called Bhubaneswar, which is a capital of some state in India. Uh, so that's our seed path. And then here is the question that the model came

up with. And then this agent solution row is not displayed in here, but if you look at it, we can, we can see what it looks like here. So it did a bunch of tool calls, it reasoned about the question, and then it's trying to find the answer on the internet. So that's how you create, um, a search tool called dataset. And I guess I'll pause here for questions, and then we can move on to the next

few slides after that. Any questions if something was-- Yes.

Yeah. Um, so this was, uh, this is, uh, for doing some similarity search. The tag search

Yeah. Absolutely.

On the BIRD benchmark.

Yes. So we moved the BIRD benchmark from twenty-six to forty-one percent. So you asked the, you know, exact right question. And we have a blog post on this as well. Um, you can read about it. Um, so all the datasets we've created with Data Designer, um, we have blog posts, and you can see right here. So I was involved in this work as well. Uh, so yeah, this talks about SQL, and I can also share the link with you after. Um,

but essentially, Data Designer can be used to create any kind of dataset for training AI models, possibly other datasets as well, if you're looking for test datasets not related to model training and so on. Um, and we have a bunch of data tools that we've created in our team apart from Data Designer that I'll talk about in the end.

Follow-up question. Can you, uh, can you comment on why we only need a

percent? Uh, because the BIRD benchmark is SOTA is

So the SOTA is seventy on BIRD is because it's agentic. Our, um, current SQL dataset is not agentic. And, um, also a SOTA on benchmarks is very much model dependent. We're talking about moving, uh, Nemotron Nano, which is our small size, um, LLM or SLM. I don't know, I don't know if you can call it an SLM because it's an MOE, but it's an LLM. And so, um, for that model size, it's a pretty

respectable performance on BIRD. Yeah. Question?

Yeah. No, this is a deterministic algorithm to create the seed data, and then the, uh, tool calls and the agentic stuff is in creating the actual-- like performing the actual search once you have the question and creating the trajectory. We call it a trajectory. So creating the search trajectory. This is deterministic. This is just something I came up with to create seed data. Yeah. You can have any kind of seed data. Maybe you, like, you know, you can come up with a better way of creating seed data

than this as well.

Did you use fine-tuning or did you use pre-- Uh, did you take the synthetic data and pre-train the model with it?

Oh, great question. So we could do that. Um, we have created datasets that can be used in pre-training and post-training. My main focus is post-training, um, or, you know, creating datasets for post-training. So this specific, uh, um, I guess project was SFT, uh, in post-training. Yes.

Any particular reason for using Tavily?

There's no reason apart from the fact that, uh, I think someone at NVIDIA did a deal with Tavily to get credits. You can use Exa. Exa is another search agent, uh, provider. You can write your own search tool, um, I guess. So yeah, there's that as well. Yeah.

So this is a SFT agent, right?

Essentially.

So what-- I'm sorry. What is generating the golden responses?

So the golden responses are generated by, um, Data Designer using the model that you choose. And so, um, in the first notebook, um... Yeah. And so I think, I think, um, you know, I went a bit quick 'cause it's-- we just have an hour for this workshop. A previous version of this workshop was two hours. So the model that I used is GPT-OSS one twenty B to create the data. Um, but Data Designer is not just a collection of LLMs that you can use to create

data. It has a bunch of LLM and non-LLM models as well working together to help you create diverse data.

So is this basically just the scaling then?

Say that again.

Is this basically the scaling then?

This is sca-- Yeah. Uh, what do you mean?

So, like if, if this thing is an orchestration to create-

Yes.

I know you can't really

Yeah.

But if this thing is an orchestration to create, uh, golden data that gets pushed on from another model-

Yeah.

Essentially what we're doing here is distilling-

Yes.

GPT-OSS one twenty B into another model.

Yes,

you are distilling, but in some cases you can even go beyond distillation. And that's the case that I'm trying to make, 'cause we have built this, uh, tool that allows you to create diverse data. So for example, a completely different use case that I was working on, GPT-OSS gets Some percentage on this, uh, benchmark called StructEval, which is a way to measure how models do on JSON, Markdown, XML, YAML, and CSV formats. And Nemotron actually distilled from GPT-OSS,

but went beyond GPT-OSS's performance. So it's distillation, but it can also be more powerful than that because of all the things we've built into Data Designer. Yes.

Uh, how can I use Data Designer for RL?

You can use Data Designer for RL.

Yeah. How? Like-

So let's say, uh... And by the way, we're doing this for search right now, or, or the team at NVIDIA is doing this for search. So instead of trying to create the golden solution, like how he described it, or, or, you know, um, executing the search riddles as written on my slide, you just don't do that step, and then you essentially end up with a dataset full of prompts. And then we have a tool called NeMo Gym, which is essentially, um, an RL, um, reward function. You know, uh, I guess it's

an, it's a, it's a tool to basically teach your models how to do RL. But the thing is, in NeMo Gym, you write like, um, a function that essentially gives you a reward, and then you can use our, uh, any RL training framework to execute in real time with those tools, and then provide a reward in RL.

I was wondering if there's like a way to take the golden reward function rather than-

That's a great point. We're thinking about how to do this actually currently. But to build something generic is the, is the challenge, right? Like, we don't wanna build something specific to one use case. Yeah. Yeah.

Is this reliable and accurate for

Is this reliable and accurate for?

Uh, ETL.

ETL.

Or like ETL pipelines and

So I haven't tried it myself, but I think it is because it, it should be. Because, um, we've created a diverse set of datasets with Data Designer, and I think at the start we showed about six and a half trillion tokens. And if it's not reliable, you should come talk to us and see how we can help you.

I can't hear you, sorry.

Uh, like for two or five end times.

Sorry, I'm not able to hear you.

Two or five end user

Oh, I, I don't know about that. I'll, I'll, I'll come back to you at the end, and we can, we can chat more. Yeah, sorry about that. Okay. Um, we have about fifteen more minutes, so let me move on, and we can have more questions at the end, w-- to something really interesting, which is evaluating your synthetic data, right? And that's not an easy thing to do because without this evaluation step, your synthetic data is not that useful. So when I started, just

to give you a sense of how SDG, as we call it, works, synthetic data generation, I started with fifty thousand seeds, and in the end, I ended up only with seven thousand high quality end-to-end SFT records. And then I used that to train the model, and we moved the benchmark from zero to thirty, right? So that's great, but fifty thousand to seven thousand, that's a lot of, you know, um, um, funneling and filtering for high quality.

So what, like, I'm trying to focus on here is that data quality is the most important thing when you're trying to create data. And, um, what do I mean by quality? It can be task-specific, but it can also be generic. Like diversity is one way to, um, you know, one axis of quality, I would say. Um, so after that, I created this function to help us, um, show, like this is what the LLM-as-a-judge scores for each

trajectory, um, and what is like your, um, you know, number of words in your, uh, question or search query, and then here is your keep percentage, sixty-six percent, and so on. So what's really-- So you can take a look at that function later. But what's really interesting is how do you actually check if the answers that the model got are correct or not? So I came across this really interesting example, and the search riddle was quite simple. It said: Which coun-- which

country owns U.S. Steel? And I thought the answer is the U.S. And so, um, I, I, you know, I, I guess the agent predicted U.S. in the training dataset, and, um, my verifier said that it was correct. But then I went and Googled it, and apparently the U.S. sold U.S. Steel to Japan, I think, if I remember correctly, a couple of years ago. So even my Wikidata seed can become stale,

right? And so that's the real challenge is how do you actually evaluate and, um, uh, tell if the data that you've created is, um, good, accurate, you know, relevant, and so on. Um, so I guess if you go and implement a determinist, uh, I guess, uh, like a simple matching function like how I did, you'll end up with like twenty-four percent, and then you'll be like: "Well, my data is terrible.

Why did I do all of this?" But that's not entirely true because here are low percentages I expected 'cause I don't know how to actually create that function. So this is an active area of research, is how do you actually evaluate this data? So I think evaluation from my perspective is a very multifaceted problem. So you can't just take LLM-as-a-judge and put it in there and hope that that's gonna, you know, give you everything. Evaluation is a problem where if you check for a hundred problems, and if you see that those hundred problems

do-- all you, all you can tell from checking for a hundred problems is those hundred issues don't exist in your dataset. There could be a thousand other issues. So that's the challenge, is how do you actually evaluate the data? Um, so we have a few different ways that we do that. Um, one of them is, like I mentioned earlier, LLM as a judge, which is generic. Uh, something I did really interesting is I actually did LLM as a judge with, uh... And I took the judgments, and I sent them to human reviewers, and I have those reviewers,

um, cross-check the judgments of the LLM, and essentially give me confidence that the LLM is judging my responses correctly. So that's one way to do it. And then in search, you could look at real-time information. So you have to basically find a union of all these different, different, uh, layers of evaluation that's gonna essentially give you enough confidence in your dataset. So that's it for the code. I know I blazed through this, but hopefully I've, you

know, um, given access to all of this in my GitHub repo so you all can take a look. Um, so let's talk about a few other use cases. So like I mentioned earlier, we are the, um, you know, part of a much larger team that creates Nemotron, right? So, um, my focus or my team's focus is essentially creating datasets or creating a framework for creating datasets and-- or I guess both of those. So there's all kinds of datasets we've created. The first one that we spoke about today, search tool

calling, um, and then we also have a dataset to do multi-turn tool calls. So I think the search one was very much single turn, right? Like we're going from prompt to response. But what if you had multiple turns? Or what if you had multiple tools, um, or, you know, not just search tools, but like you might have a, I don't know, a weather tool or a coding tool and so on. Um, so we're working on multiple different projects here. Another

really interesting one is structured output generation. I briefly spoke about it. There's a benchmark called StructEval, which measures your performance on TOML, JSON, YAML, CSV, XML. And all of these are super important 'cause everyone wants to use, um, you know, agents to do everything now. I'm sure you all have seen the, the, uh, the blowtorch meme, uh, on LinkedIn or something. But, but, uh, if we're gonna be using agents as, um, the central

piece in software applications, then they need to reliably give us, um, like a JSON format or a YAML format and so on. So that's what we were focused on there. Um, we've also worked on SQL, um, that someone asked about here. And, uh, we've also worked on scientific reasoning. Um, there's also pre-training datasets that we've created using Data Designer for STEM performance and so on. Um, and so I can talk about the best practices for

generating data. Um, I won't go over this in detail. There's-- Uh, 'cause you can always look at the slides later. But essentially diversity before volume, I think the Kimi K two paper showed something similar, which is ten strategies, one epoch is better than one epoch... Sorry, ten strategies, one epoch is better than ten epochs, one strategy. And, um, you always wanna use the cheapest correct column type. So, like, using a LLM to create a sampler makes no sense, right? It's a complete waste of

resources. But using LLM to generate text data makes a lot of sense. So our samplers are written in Python, of course. Um, and then you generate-- you overproduce data, and then you filter down, like I mentioned earlier. And then you verify what you can, uh, iterate on the preview. I think the best way to know if your data is any good is to actually look at the data. That's the best way. Uh, if you can look-- review a few samples. And, um, of course, what is this idea of Data Designer? Essentially, you're taking the data creation

process and making it, um, something, you know, version controllable, uh, shareable like code. So essentially we're creating code that can create data. So now you can share this code with anyone 'cause, you know, we already built all this tooling for sharing code and having version control and all of these other things. So if you're trying to do search, but not exactly the way I did it and just slightly differently, you take my code, you modify it, and you can create your own dataset from it. Um,

so something that's super interesting, I'm getting into the extra slides, which I didn't think I would. I think I have ten more minutes, right? So, um, this is something that's really interesting. We've created, uh, something called Personas. So Nemotron Personas USA, um, it's also inspired from this other paper, which is scaling synthetic data creation with one billion personas. So what is a persona, right? A persona is essentially, um... So we're trying to solve the problem here of

how do you know if the data you've created actually, um, adheres to, like, demographic information or, you know, grounded in some real world distributions, right? That's very important. Uh, so if I were to create a dataset of healthcare records in San Francisco, California, well, I wanna make sure that the, uh, the demograph-- the actual demographic of San Francisco is reflected in my dataset, right? So, like, the number of, uh... You know, there's all kinds of

demographic features. Um, so we created a dataset called Nemotron Personas, uh, using Data Designer, but it's also now a seed in Data Designer to create more datasets. And, um, this essentially allows you to learn from user data without actually leaking any single person's information out there. So now you can train AI models, and you can learn from the aggregate of- Um, you know, people's

information, but you don't actually have to have like, for example, my healthcare records, you know, directly, um, uh, being used in training an AI model. I, I keep using the healthcare example because I wanna find a sensitive industry which is like, you know, there's HIPAA compliance, there's all kinds of regulations there, right? That we have to respect. So, um, so this is a persona. Um, this person is a fifty-year-old pharmacist with a free spirit, balances her relaxed approach, blah, blah, blah. But

essentially, there's a persona, there's a professional persona. Um, first name, last name, middle name, gender, age, um, all kinds of things. And we have a million of these. You can take this, you can use it, um, however you please as seed to create more data. So, yeah, I think with the personas, um, I will probably stop here. So we have two more minutes. I'll talk about these other tools, and then we'll leave time for questions.

Oh, you want a question? Yeah. Yeah. Yeah.

For? Like that set

of Yeah. Yeah, yeah, yeah. So essen-- That's essentially why we created it, is to, um, have a way to learn from aggregate information without actually like, you know, having individual people. So, like, you could create synthetic Social Security numbers. Now, they're not actually gonna be, like, your Social Security number, right? But I think the idea is, like, if there's, um, for example, um, um... I don't know. What is a

good example that I-

Yeah. Yeah. Yeah, yeah. Exactly. So, for example, we, we actually created some tax data for, um, you know, a customer. And this tax data, I realized there's exactly, uh, [REDACTED] people in the [REDACTED] who are below [REDACTED] earning more than [REDACTED] a year on their tax return. But there's, like, way more people who are older, and they're earning, you know, X or Y or Z amount of money. And so the tax data that we create, let's say we wanna create a

synthetic W-2 form. We wanna make sure that that form is not just random values from the LLM, 'cause the LLM is not gonna give you, um... If you-- H-Have you asked a LLM to tell a joke? It's gonna tell you the same joke every time. So similarly, it's not gonna give you, like, numbers that are realistic, right? Like, it's always gonna come back to the average of things. And so when you create data, you wanna make sure that that synthetic W-2 form or ten forty form or whatever is actually, um, adhering

to real-world distribution. So that's why we created this persona dataset.

So we have a bunch of tools. Uh, just two minutes and then more questions. Sorry about that. Um, so we have a bunch of tools that we've created. Data Designer is only the tip of the iceberg, I guess. Uh, but it's our most successful tool so far 'cause it's our oldest tool. Uh, we also have, um, you know, at NVIDIA, we have data curation tools, so which is NeMo Curator, which I'm sure there are other talks about this I, I believe I saw on the schedule. Um, it is very interesting. You

know, you have to take data, uh, real-world data, curate it, modify it, post-process it, get it to a state where it's usable to train in LLMs, or you can create synthetic data. So I think it depends on your use case. Like, I'm not gonna stand here and say synthetic data is the best thing ever for everything. But I think there are specific use cases where you wanna curate real-world data or where you wanna, you know, generate synthetic data. Um, and so we have data, uh, curation tools. We have Data Designer, which is our synthetic data generation tool. We have a

new tool called Anonymizer. And by the way, all these tools are completely open source. So everything that our org does at NVIDIA is fully open source, including Nemotron, including all these tools. Um, so you can use it however you want. Um, so this data tool is called NeMo Anonymizer, and the goal here is can we actually anonymize data? So I don't know about you guys, but I, I don't know if I should be saying this when the talk is recorded, but I've been putting all kinds of things into my agents. Like, you know, I

accidentally put my API keys in there once, or like, you know, some other sensitive information, right? All this log data for, like, all your agents that you're using, we need some way to anonymize it and before it's usable, right? So that's what Anonymizer does. So that's just an example, but there's obviously many other ways to use Anonymizer. And then we have NeMo Safe Synthesizer, which is allowing us to generate safe synthetic versions of sensitive datasets, um, as well. So yeah, come talk to me if you have any questions.

Um, here's my LinkedIn if you wanna stay in touch. Uh, again, you can use Data Designer. I would love for you guys to try out the notebooks. I'm so sorry that we had to-- We had only one hour, so I had to just, like, blaze through all the code. But you can try out the notebooks. We also have the blogs if you're interested in text to SQL or some, some other use case. So, um, this is, this is showing us how to use the personas. Um, I think there's a blog somewhere on how we created the personas dataset. For the

US, we took the census information, and we used a PGM, a probabilistic graphical model, to essentially, um, create, um, like, a connection between all-- B-basically take that demographic information, process it, and then try to, uh, uh, create a dataset from it, uh, essentially. So that's what we did. Um, and so every country pro-- uh, publishes this, uh, demographic information, so it's not really like, um...

I'm looking at other slides now. We-we've done a bunch of these workshops recently. Um, but yeah, there's all kinds of personas. If you're not-- If you're trying to build for something outside the US, if you're building something for Japan, India, Singapore, Brazil, I'm sure we have more now. So we, we have all kinds of personas datasets. So, um, it's giving us, um, um, a way to, uh, create customized AI for whichever country you're interested in.