Open source AI pentesting tools: what each does and where it stops

Open source AI pentesting tools are free, publicly licensed programmes that use a large language model to plan and run security tests against a web app or API, and in 2026 the main ones are PentestGPT, Strix, Shannon, PentAGI, CAI, HexStrike AI, Nuclei and ZAP. They differ a lot: some are autonomous agents, some are tool servers, some are classic scanners with an AI feature added. The right pick depends on whether you have the code, how much you trust a model with a live target, and who will read the results.

An AI pentesting tool is software where a model decides which request to send next, runs it, reads the response and tries to prove a finding. This guide uses only what each project says in its own repository or documentation, read on 7 October 2026. It gives no prices, because the licences are free and the model usage you pay for varies. See each provider's pricing page.

What are open source AI pentesting tools, and what kinds exist?

They fall into four kinds, and mixing them up is the most common buying mistake. A scanner that fires known checks is not an agent that reasons, and a tool that tests a chatbot is not a tool that tests your login page.

• Autonomous agents: a model drives tools, reads results and decides the next step. Examples: PentestGPT, Strix, Shannon, PentAGI.
• Frameworks and tool servers: they give a model access to many security tools and leave the planning to you or the model. Examples: CAI, HexStrike AI.
• Scanners with an AI layer: a fixed engine that can also take help from a model. Examples: Nuclei with its AI template option, and ZAP as a long-standing scanner.
• Tools that test AI apps: they probe a model or an LLM feature for prompt injection and leakage. Examples: garak and promptfoo.

Why does any of this matter now? The Verizon 2026 Data Breach Investigations Report page reports that 31% of breaches now begin with a software vulnerability, ahead of stolen passwords as the most common way in. It adds that generative AI now boosts 15% of attack techniques. Testing your own software before someone else does is the point of all these tools.

Open source AI pentesting tools at a glance

The table restates each repository's own words so you can scan the differences before reading the detail.

Which open source AI pentesting tools are worth knowing in 2026?

Ten projects are worth knowing, grouped by what they do. Every statement below comes from the project's own README or documentation.

Autonomous agents

PentestGPT

PentestGPT describes itself as an "AI-Powered Autonomous Penetration Testing Agent" that uses a model's reasoning for penetration testing and capture-the-flag challenges. Its README says the current version works through staged phases: reconnaissance, exploit and walkthrough for CTFs, and asset discovery, vulnerability identification and report for pentests. It drives the Claude Code or Codex command-line tools, and needs Python 3.12 or later and the uv package manager for local runs.

It started as research, "Published at USENIX Security 2024", and is MIT licensed. The README reports an 86.5% success rate (90 of 104 benchmarks) on an XBOW validation-suite experiment in December 2025, then states plainly that this is "a historical research result, not a current pentestgpt-agent regression guarantee." Read that sentence before you quote the number anywhere. It suits people who want to learn how an agent plans, or who work on lab and CTF-style targets.

Strix

Strix calls itself an "Open-source AI penetration testing tool to find and fix your app's vulnerabilities". It runs in a Docker sandbox, needs an LLM API key from a supported provider, and is licensed Apache-2.0. Its documentation lists three usage patterns: scanning a local codebase, a black-box web application assessment, and a security review of a GitHub repository. A -n (non-interactive) flag runs it headless, and the README includes GitHub Actions examples for pull requests.

The README claims "real exploit validation" with working proofs of concept. The documentation does not publish a measured false-positive rate, so treat that as the project's claim and test it on an app you know. Its warning is blunt: "Authorized use only", and only against systems you own or have explicit, written permission to test.

Shannon

Shannon is described as an autonomous AI pentester for web applications and APIs. It reads your source code, identifies attack paths and "executes real exploits" to prove a flaw before it reaches production. It needs source access. It starts with npx @keygraph/shannon@latest, mounts the target repository read-only inside an ephemeral worker container and writes results to a local workspace. The open-source edition is under the GNU Affero General Public License v3.0, and the project offers commercial licensing separately.

The README is candid about limits. A full run "can take roughly 1 to 1.5 hours and may incur LLM API costs", findings "still require human review", and it says "Do not run Shannon against production systems." That last line is the most useful sentence in any tool in this guide: it tells you where to point it.

PentAGI

PentAGI stands for "Penetration testing Artificial General Intelligence". The README says all operations run "in a sandboxed Docker environment with complete isolation" and lists "20+ professional security tools including nmap, metasploit, sqlmap, and more". It supports more than ten model providers, keeps long-term memory of past results and is self-hosted. It is a broad infrastructure-and-web tool rather than a web-app specialist, and it ships with an end-user licence agreement, so read that before commercial use.

Frameworks and tool servers

CAI

CAI (Cybersecurity AI) was a framework from Alias Robotics for building AI-powered offensive and defensive security automation, released in April 2025. Its README now says it is archived as of August 2026 and warns that "An archived offensive-security framework is unmaintained attack tooling." The licence mixes MIT components with proprietary additions licensed for research only. It is worth knowing for its published research, not for a new production pipeline.

HexStrike AI

HexStrike AI describes itself as an MCP server that lets AI agents "autonomously run 150+ cybersecurity tools". The README claims "12+ Specialized AI Agents". It is MIT licensed. It also carries its own safety note: "AI agents can execute arbitrary security tools - ensure proper oversight", and it recommends isolated environments or dedicated testing machines. The more tools a model can reach, the more your scope rules matter. See the pentest scope term.

Scanners with an AI layer

Nuclei

Nuclei calls itself "a fast, customizable vulnerability scanner", built on a YAML-based template language and fed by a large security community. Detection logic lives in YAML templates, so a check is a file you can read, review and version. The README lists an -ai option to "generate and run template using ai prompt", and the documentation describes generating templates "using natural language descriptions powered by ProjectDiscovery's AI capabilities". It is MIT licensed, and the README notes it is under active development with possible breaking changes.

This is the most conservative way to add AI: the model writes a check, a person can read it, and the scan itself stays deterministic. Because the documentation credits a ProjectDiscovery service for the generation, check what is sent out before you describe an internal target in a prompt. Nuclei's strength is breadth: applications, APIs, networks, DNS and cloud configuration.

ZAP

ZAP calls itself "the world's most widely used web app scanner" and "free and open source". It presents itself as an independent project, and its site is marked "by Checkmarx". Its home page does not describe AI features, so it appears here as the baseline most teams already know, not as an AI agent. Its Automation Framework is "one YAML file" that controls ZAP, ships in the stable Docker image, runs with the -autorun option and exits with 0, 1 or 2 so a pipeline can react. It imports OpenAPI, GraphQL and SOAP definitions.

ZAP's own getting-started page carries a rule worth repeating: use it to attack only an application you have permission to test, because active scanning can damage data and function. Our Burp Suite vs ZAP comparison goes deeper.

Tools that test the AI itself

garak and promptfoo

garak says it checks "if an LLM can be made to fail", probing for prompt injection, data leakage, jailbreaks and similar weaknesses; it is Apache 2.0. promptfoo is "a CLI and library for evaluating and red-teaming LLM-based apps", is MIT licensed and states that evals run locally. Neither tests your login, your invoices or your API's authorisation. They test the model features you have shipped, against the OWASP Top 10 for LLM Applications 2025, which lists prompt injection first and excessive agency sixth.

How do open source AI pentesting tools compare on what matters?

Licence and star count tell you little. What matters is who controls the test, what the model can touch and what you get at the end. This table sets the points a reader can check in each project's own documentation.

Are open source AI pentesting tools safe to run?

They are safe only when you contain them. Every agent in this guide carries a warning, because each one actively sends exploit traffic and a model can choose an action nobody expected. Treat the first run like a change to a production system: write the scope, pick the environment, limit the credentials.

A containment checklist before the first run

Use this table as a sign-off sheet. A "no" in the right-hand column is a reason to stop, not a reason to proceed carefully.

What about the model provider?

This is the question teams skip. Shannon's README says the vendor "never proxies your model traffic", which means traffic goes to the provider whose credentials you supplied. Strix and PentestGPT likewise need a model provider or a signed-in command-line tool. If you handle regulated data, decide before testing whether your code, requests and responses may go to that provider. A local model, where the project supports one, keeps the data in your network though you should test the result quality yourself.

What about the tool's own supply chain?

These are young, fast-moving repositories. Nuclei's README says to expect breaking changes, and CAI shows what happens at the end of a project's life: archived and, in its own words, unmaintained, with known weaknesses that will not be fixed. Pin versions, read what you install and keep it away from the machines that hold secrets. See software supply chain security.

What do open source AI pentesting tools still miss?

They give you a capable tester and leave the programme to you. A pentest is useful because someone owns the scope, the evidence, the fix and the check that the fix worked. Free tools supply the first step of that chain. The rest is your team's time, and it is where most trials stall.

Evidence you can hand to an auditor or a developer

An agent's transcript is not a report. A developer needs the request, the response, the affected endpoint and, ideally, the line of code. An auditor needs scope, dates and a retest. Strix and Shannon say they validate with working proofs of concept, which is the right idea. Ask whether the output is stable enough to attach to a ticket without rewriting it. Our guide to reading a pentest report lists what good evidence looks like.

Business logic and cross-account access

The flaws that cost real money are usually in how your app behaves, not in a known pattern: a coupon applied twice, one customer reading another customer's invoice. A template scanner can only check what someone wrote a template for. An agent can try more, but it needs several real accounts and a clear idea of who should see what. Test that deliberately; see business logic flaws, IDOR and the access control testing checklist.

Proving the fix, not just finding the bug

Finding is half the work. After a developer changes code, someone must replay the same exploit against the new build. A fresh scan can pass because it never repeated the original request. Our closed-loop retest entry explains the difference.

Triage cost: the hidden bill

Prioritising matters because the volume is high. The CISA Known Exploited Vulnerabilities Catalog showed 1,734 entries when we read it on 7 October 2026, and its own guidance says to use it as an input to prioritisation. A tool that reports ten suspected issues, three of them false positives, costs someone an afternoon. The IBM Cost of a Data Breach Report 2026 describes the global average cost of a breach as "a 12% increase over last year and a record high", so a missed real issue is the more expensive mistake.

How do you choose an open source AI pentesting tool?

Choose by four facts about your situation: whether you can share source, where the test will run, how much a person will review, and what you must show afterwards. Run the same small app through two tools before committing.

Choosing by situation

Questions to answer first

1. Who owns the finding? Name the person who reads, rejects or accepts each one.
2. Where does code go? Check the model provider's terms before pasting in source.
3. Is the target safe? A staging copy with fake data and test accounts.
4. Is it repeatable? Can a colleague run it next month and get comparable output?
5. Can you prove the fix? Keep the exact request that exploited the bug.

How do you trial an open source AI pentesting tool in a week?

Run it on an app where you already know the bugs, and score it against that list. Five steps are enough.

1. Pick a known target: a staging app with two or three issues your team has already found.
2. Write the scope: hosts, accounts, off-limits actions and a stop time.
3. Isolate the run: container or VM, test credentials, no production secrets.
4. Score the output: real findings, misses, false alarms and the time spent checking each.
5. Replay one fix: change the code, then re-run the exact exploit to see it fail.

Where does Cyberlop fit next to open source tools?

Cyberlop is a managed option for teams that want the evidence and the retest without building the harness. It finds vulnerabilities in web apps and their APIs with black-box, grey-box and white-box pentesting. In white-box mode it reads your code while testing the running app, so a finding can point to the file and line. It reports only what it exploited, with the request, the response and, where a person can see it, a screenshot.

It also keeps severity and confidence apart, carries CWE and OWASP references, checks business-logic flaws and cross-account access, and proves a fix by re-running the exact exploit against the new code. It can run inside your own network, on a schedule or in CI/CD. Open source tools are often the right start; use the section above to decide when you have outgrown them. See private scanning and what the evidence looks like.

Related reading: best AI pentesting tools, AI penetration testing, best DAST tools, DAST vs SAST vs IAST, AI agent security testing and XBOW alternatives. Terms: DAST, automated pentesting and proof of exploit.