Offensive Red-Team Frameworks for AI Agents: Garak, PyRIT, and Promptfoo in the Lab

If you are standing up a red-team practice for AI agents, three open-source frameworks keep coming up: Garak (NVIDIA), PyRIT (Microsoft), and Promptfoo. They sound interchangeable. They are not. Each one answers a different question, and reaching for the wrong one wastes weeks before you realize the gaps it was never built to cover.

Here is the short version, and the rest of this article is the long version.

Which red-team tool should you use, and when?

Use Garak when you want broad, automated vulnerability scanning of a model endpoint. It ships with a large library of canned attacks — jailbreaks, prompt injection, encoding bypasses, toxicity, data leakage, package hallucination — and you point it at a model to get a coverage report fast. Think of it as a vulnerability scanner for LLMs: low setup, wide surface, shallow customization.

Use PyRIT when you need programmable, multi-turn attack orchestration in Python. It is a framework for security researchers, not a one-shot scanner. PyRIT excels at scripted, adaptive, multi-turn attacks — Crescendo, TAP, PAIR, Skeleton Key — with composable converters, scorers, and targets you assemble yourself. The power is in the orchestration; the cost is that you write code.

Use Promptfoo when you want application- and agent-layer red teaming wired into CI/CD. It generates adversarial test cases from declarative YAML configs — the same engine behind a tool-call fuzzing harness — maps findings to frameworks like the OWASP LLM Top 10 and the OWASP Agentic Top 10, and runs as a regression gate on every deploy. It is the strongest of the three at testing the application around the model — RAG pipelines, tools, memory, and MCP servers.

In practice, mature teams run more than one. A common pattern is Garak plus PyRIT for offensive depth and breadth, and Promptfoo as the always-on regression layer in the pipeline. The three are complements far more than substitutes — and once you see what each actually tests, that becomes obvious.

The rest of this guide breaks down what each tool tests, where the boundaries sit, how they differ on the dimensions that matter, and how to choose based on your team and threat model. Wherever a fact is load-bearing, it is sourced.

Why does “red-teaming an AI agent” need different tools than testing a model?

Before comparing tools, it helps to be precise about the target, because the word “AI agent” quietly changes the threat model.

A bare language model takes text in and produces text out. Red-teaming it means probing the weights and the prompt: can you jailbreak it, make it leak training data, coax toxic output, or trick it into following injected instructions? That is a contained surface, and it is mostly what classic LLM scanners were built for.

An AI agent is a different beast. It wraps a model in a loop that can call tools, read from a vector database, write to and read from memory across turns, hit external APIs, and act on the results — sometimes autonomously, sometimes by orchestrating other models. Each of those capabilities is a new attack surface that does not exist in a bare model:

• Tool and API access means a successful injection can trigger real-world side effects, not just bad text. This is the “excessive agency” problem.
• Persistent memory means an attacker can poison state in one turn and harvest the consequences several turns later — the failure mode you reproduce in a memory-poisoning RAG lab.
• Retrieval (RAG) means the agent ingests untrusted documents that may carry hidden instructions — indirect prompt injection through the knowledge base.
• Tool descriptions and connectors (increasingly the Model Context Protocol, MCP) become an injection vector in their own right: hidden instructions can ride inside a tool’s own description.

This is why a “best red-team tool” question has no single answer. A scanner that is excellent at jailbreaking a model may be blind to a memory-poisoning attack that only manifests across turns. A framework that orchestrates beautiful multi-turn exploits may not map its findings to the compliance framework your auditor wants to see. The three tools below sit at different points on this spectrum — model layer, attack-orchestration layer, and application/agent layer — and that placement is the single most useful lens for choosing between them.

What does Garak test?

Garak is NVIDIA’s open-source LLM vulnerability scanner. Its tagline says it plainly: it checks whether an LLM can be made to fail in ways you do not want. You give it a model, it runs a battery of attacks, and it reports where the model broke.

The mental model is the closest thing the LLM world has to a Nessus or an OpenVAS: broad, automated, low-effort to get a first signal.

How is Garak built?

Garak’s architecture is four moving parts working in sequence. A generator is the connection to the model under test. Probes assemble and orchestrate prompts designed to elicit a specific failure. A harness runs the probes against the generator. Detectors then inspect the responses to decide whether the attack succeeded, and an evaluator scores the run. You pick a model type and name on the command line, and Garak figures out which probes and detectors to run.

The corpus is the headline feature. At time of writing, Garak ships more than 150 distinct probe modules and on the order of 3,000 prompts and prompt templates, spanning categories such as prompt injection, DAN-style jailbreaks, encoding-based bypasses, hallucination, toxicity generation, data leakage, misinformation, malware generation, cross-site scripting, package hallucination, and glitch-token exploitation. (Garak GitHub)

It supports three probing styles: static probes that replay known exploits, dynamic probes that evaluate responses for emergent weaknesses, and adaptive probes that iterate on failures to build more sophisticated attempts.

Where does Garak stop?

Garak’s strength is also its boundary. It tests model endpoints, not full application stacks. It will tell you a lot about how robust a base model is to a wide spread of known attack classes, and it gives the broadest automated coverage of base-model vulnerabilities of any open-source tool. But it is not built to reason about your RAG retriever, your agent’s tool permissions, or a memory-poisoning attack that only surfaces across a conversation. Its agentic-attack coverage exists but is early-stage. And while its probe categories overlap heavily with the OWASP LLM Top 10, it does not ship native OWASP report mapping — so framing findings for an auditor is on you. (QAwerk comparison)

Reach for Garak when you want a fast, wide first-pass scan of a model — your own fine-tune, a vendor model you are evaluating, or a base model before you build on it.

What does PyRIT test?

PyRIT — the Python Risk Identification Tool — is Microsoft’s open-source framework for security risk identification and red teaming in generative AI systems, MIT-licensed and built by the Microsoft AI Red Team out of their own operational practice. (Note: the original Azure/PyRIT repository was archived in March 2026 and now redirects to microsoft/PyRIT.)

PyRIT is not a scanner you point and shoot. It is a library of composable building blocks you assemble into an attack. That distinction is the whole point: PyRIT is for the red teamer who needs to construct a specific, often multi-turn, adversarial campaign rather than run a fixed checklist.

How is PyRIT built?

PyRIT’s design is deliberately modular so that pieces can be reused and extended to new models and modalities. The core abstractions are:

• Targets — the model or system under test, model- and platform-agnostic.
• Converters — transformations applied to a prompt before it is sent (encoding, translation, obfuscation, and many others). Converters are how you mutate an attack into forms that slip past filters.
• Orchestrators — the logic that drives an attack, from “apply a converter and send one prompt” all the way up to using half a dozen generative models to compose and iteratively refine a multi-turn attack against a target.
• Scorers — how PyRIT decides whether an attempt succeeded, using true/false, Likert-scale, classification, and custom scorers, powered by an LLM, Azure AI Content Safety, or your own logic.
• Datasets — seed prompts and attack material.

The attack techniques it is known for live in those orchestrators: Crescendo (gradually escalating an innocuous conversation toward a harmful goal), TAP (Tree of Attacks with Pruning), PAIR, and Skeleton Key are all multi-turn or iterative strategies that a one-shot scanner simply cannot express. PyRIT is also multimodal, aimed at probing for novel harms across modalities, and it categorizes harms including fabricated/ungrounded content (hallucination), misuse and bias, and prohibited content such as harassment. (PyRIT paper, arXiv 2410.02828)

Where does PyRIT stop?

PyRIT’s flexibility is its tax. It assumes you can write Python and design an attack; there is no “run everything” button that gives a meaningful score with zero configuration. It is the right tool for a researcher building bespoke orchestration, and the wrong tool if what you wanted was a quick coverage report on a Tuesday afternoon. PyRIT has since been integrated into Azure AI Foundry’s AI Red Teaming Agent, which wraps some of this in a more guided experience — but the open-source library itself rewards expertise. (Microsoft Learn)

Reach for PyRIT when you need custom, multi-turn, possibly multimodal attack orchestration — when “did Crescendo eventually break this agent?” is a question you actually need answered, and you have the engineering to build it.

What does Promptfoo test?

Promptfoo is an open-source CLI and library for evaluating and red-teaming LLM applications. It started life as an eval tool — comparing prompts and models — and grew a serious red-teaming arm. Its calling card is that it tests the application and agent layer and slots into CI/CD as a regression gate. It is widely used, including inside OpenAI and Anthropic. (Promptfoo GitHub)

If Garak tests the model and PyRIT scripts the attacker, Promptfoo tests the thing you shipped — the agent with its tools, its retriever, and its memory — and keeps testing it on every commit.

How is Promptfoo built?

Promptfoo’s red-team system has three concepts:

• Targets (a.k.a. providers) — the endpoint, app, or agent under test.
• Plugins — adversarial input generators that produce malicious payloads. Promptfoo ships a large library (157 plugins across categories such as security and access control, trust and safety, compliance and legal, brand, and dataset-derived test sets). (Red Team Plugins, Promptfoo)
• Strategies — the techniques for delivering those payloads. Static strategies apply deterministic transformations (cheap, but easy for foundation models to have already patched); dynamic strategies use an attacker agent to mutate inputs through iterative refinement. (Red Team Strategies, Promptfoo)

The configuration is declarative YAML, which is exactly what makes it CI-friendly: the same config that a security engineer authors runs unattended in a pipeline.

What about agent-specific attacks?

This is where Promptfoo pulls ahead for agent work. It has dedicated plugins for the attack surfaces unique to agents:

• Memory poisoning — tests whether a stateful agent can have its memory corrupted in one turn to alter behavior later, mapped to T1 of the OWASP Agentic Top 10. (Memory Poisoning Plugin, Promptfoo)
• MCP and tool poisoning — tests Model Context Protocol servers for hidden instructions smuggled inside tool descriptions and for data-exfiltration paths. (MCP Security Testing, Promptfoo)
• RAG/context poisoning — for agents that retrieve from document stores, tests whether poisoned context can manipulate the agent into leaking data.
• Excessive agency / privilege escalation — tests whether an agent can be steered into escalating privilege or reaching resources it should not.

It also ships compliance presets that map findings directly onto frameworks: owasp:llm for the OWASP LLM Top 10, the newer OWASP Top 10 for Agentic Applications (announced at Black Hat Europe 2025), and nist:ai:measure for the NIST AI Risk Management Framework. That mapping is the feature your GRC team will care about most. (OWASP Agentic, Promptfoo)

Where does Promptfoo stop?

Promptfoo’s adversarial generation leans on its plugin-and-strategy model, which is excellent for breadth and for CI repeatability, but it is not designed to be the deeply scripted, researcher-driven attack lab that PyRIT is. If you need to invent a brand-new multi-turn orchestration from primitives, PyRIT gives you more rope. Promptfoo’s bet is that most teams want curated, framework-mapped coverage that runs automatically — not a blank Python canvas.

Reach for Promptfoo when you are securing a shipped agent or LLM application, want OWASP/NIST-mapped reports, and need red-team checks to run as a gate in your pipeline rather than as an occasional manual exercise.

How do Garak, PyRIT, and Promptfoo differ at a glance?

The table below summarizes the practical differences. Read it as “what is each tool for,” not “which one wins” — they target different layers.

Dimension	Garak (NVIDIA)	PyRIT (Microsoft)	Promptfoo
Primary identity	LLM vulnerability scanner	Attack-orchestration framework	App/agent red-team + eval harness
Main layer tested	Model endpoint	Model + scripted campaigns	Application & agent layer
Setup effort	Low (CLI, point-and-scan)	High (write Python)	Low–medium (declarative YAML)
Attack breadth	Very broad (150+ probes, ~3,000 prompts)	Whatever you build	Broad via plugin library (150+ plugins)
Multi-turn attacks	Limited / adaptive probes	Strong (Crescendo, TAP, PAIR, Skeleton Key)	Via dynamic strategies
Agent-specific tests (memory, MCP, RAG, excessive agency)	Early-stage	Buildable, not turnkey	Strong, dedicated plugins
Multimodal	Partial	Yes	Partial
Compliance mapping (OWASP/NIST)	Not native	Not native	Native presets
CI/CD integration	Possible, not the focus	Possible via code	First-class
Best fit	Fast broad model scan	Researcher building bespoke attacks	Securing a shipped agent in a pipeline
License / source	Open source	Open source (MIT)	Open source

A few honest caveats about any comparison table: tool capabilities move fast in this space, so probe counts and plugin counts are point-in-time figures, and all three projects are actively adding agentic coverage. Treat the table as a map of intent and center of gravity, then verify current capabilities against each project’s docs before you commit.

How do you actually choose? A decision guide

The cleanest way to choose is to start from your situation rather than from the tools.

“I’m evaluating a model before building on it.”

Start with Garak. One command gives you a wide read on jailbreaks, injection, leakage, and toxicity for that model. It is the fastest way to know whether a base model or fine-tune is a liability before you wrap an agent around it. If the model is going into something sensitive, follow up with PyRIT for the multi-turn attacks Garak does not deeply probe.

“I’m a security researcher hunting novel attacks.”

Reach for PyRIT. You need converters, scorers, and orchestrators you can recombine, and multi-turn strategies like Crescendo and TAP that express an evolving conversation. PyRIT is the lab bench. The price of admission is Python and time, and for this use case that is a fair trade.

“I shipped an agent and need to keep it safe over time.”

Make Promptfoo the backbone. Its agent-specific plugins (memory poisoning, MCP/tool poisoning, RAG, excessive agency) test the surfaces that only exist once a model is wrapped in an agent, and the OWASP/NIST presets turn raw findings into something your auditors and engineers can act on. Crucially, it runs in CI/CD, so a regression in a prompt, a tool, or a retriever gets caught on the commit that introduced it rather than in production.

“I want comprehensive coverage and have the budget for it.”

Layer them. The combination that comes closest to comprehensive is Garak plus PyRIT for offensive depth and breadth, Promptfoo for ongoing regression in CI/CD, with a production defense layer (guardrails) on top. (QAwerk comparison) This is not gold-plating — the three genuinely cover different layers, and a gap in one is a strength in another.

What none of these tools do for you

A red-team framework finds problems; it does not fix them, and it does not replace judgment. Three things worth keeping in mind so the tooling does not lull you into false confidence.

Automated red teaming is necessary, not sufficient. These tools surface known and emergent attack classes efficiently, but a determined human adversary reasoning about your specific business logic — the discount the agent can be talked into, the record it can be coerced to update — will find things no scanner enumerates. The same instinct underpins running CTF-style agent security challenges, where humans probe paths an automated suite never anticipates. Use automation to clear the wide, repeatable surface so your humans spend their time on the bespoke, high-value attacks.

Findings need triage and a fix loop. A 4,000-line report of “model said something bad” is noise until someone separates genuine risk from artifact, maps it to a control, and routes it to an owner. The frameworks with compliance mapping (Promptfoo) shorten this; none eliminate it.

Red teaming is not a one-time event. Models update, prompts drift, tools get added, and new attack techniques get published monthly. The reason CI/CD integration matters is that yesterday’s clean scan says nothing about today’s agent. Treat red teaming as a continuous control, not a launch checkbox.

The bottom line

The three frameworks are not rivals so much as three lenses on the same problem. Garak scans the model. PyRIT scripts the attacker. Promptfoo guards the application. If you only remember one rule of thumb: match the tool to the layer you are worried about — and if your agent has tools, memory, and retrieval, the application layer is almost certainly where your real risk lives, which is exactly the layer the bare-model scanners were never built to see.

---

This article references open-source tools and their public documentation. Capabilities, probe counts, and plugin counts are point-in-time and change frequently; verify current details against each project’s official repository and docs before relying on them for security decisions.

Sources

• NVIDIA/garak — GitHub
• PyRIT — Microsoft GitHub
• PyRIT: A Framework for Security Risk Identification and Red Teaming in Generative AI Systems — arXiv 2410.02828
• AI Red Teaming Agent — Microsoft Learn
• promptfoo/promptfoo — GitHub
• Red Team Plugins — Promptfoo docs
• Red Team Strategies — Promptfoo docs
• Memory Poisoning Plugin — Promptfoo docs
• MCP Security Testing — Promptfoo docs
• OWASP Top 10 for Agentic Applications — Promptfoo docs
• LLM Red Teaming Tools Compared — QAwerk