Datasets

A dataset is a collection of test cases for an agent — real examples of what people will ask, paired with the answer you'd consider correct. Datasets are how you turn "the agent feels good" into a number you can measure and improve against. They belong to a specific agent and live alongside its knowledge.

What a test case is

Each test case is a pair:

• a query — something a user might ask the agent, and
• an expected answer — what a good response looks like.

You build up a dataset by collecting cases that matter to you: common questions, tricky edge cases, things the agent got wrong, and examples that capture your tone or policy.

What you can do with a dataset

• Add cases one at a time as you think of them or as real questions come in.
• Edit the dataset's name and description.
• Export the dataset (as a file) to back it up or reuse it.
• Page through large datasets, and see the total case count at a glance.

Why datasets matter

Datasets are the fuel for measuring and improving an agent:

• Evaluation runs the agent against every case in a dataset and scores how well its answers match, using configured evaluators. See Evaluation & optimization.
• Optimization uses those scores to decide whether a proposed new version of the agent is actually better than the current one before it's promoted.

You find an agent's datasets in its Knowledge area. From there you can add cases, attach evaluators, and launch an evaluation run (an "experiment") to see how the agent scores.