Adversarial testing

Adversarial testing is a red-team pass on your agent. Rather than checking the happy path, it actively probes for the ways agents fail in the wild — and in the hands of bad actors.

What it probes

Probes are organized into methodology areas:

• Task accuracy — does it stay correct under tricky or misleading phrasing.
• Tool competence — does it use its tools correctly under pressure (only for agents that have tools).
• Output quality — does response quality hold up on hard inputs.
• Role boundaries — does it stay in scope and refuse out-of-role requests.
• Knowledge integrity — does it resist making things up or being led into false claims.
• Robustness — does it cope with malformed, confusing, or hostile input.
• Safety — does it resist prompt injection and unsafe instructions.

Running a test

Start an adversarial run for an agent. Donkit's adversarial engine automatically generates domain-specific probes tailored to your agent — you don't write the attacks yourself — sends them to your agent, and judges each interaction pass/fail with supporting evidence. You can generate fresh probes or run against an existing dataset.

By default, Donkit also re-tests a stable, stratified sample of probes from your previous run, so you get regression coverage — proof that issues you fixed stay fixed. You can stop a run in progress.

What you get

A run summary with pass/fail counts by category, and a paginated list of individual probes — each showing the exact input, the agent's response, the verdict, and the evidence behind it — plus a diagnosis and recommendations.