T3MP3ST Framework Turns AI Coding Agents Into Autonomous Security Testing Tools

T3MP3ST is a new open-source offensive security framework that aims to turn existing AI coding agents into autonomous red-teaming assistants for authorized testing. The project, published on the T3MP3ST GitHub repository, does not ship its own AI model. Instead, it wraps a local or connected coding agent with reconnaissance, testing, exploitation support, evidence tracking, and reporting workflows.

The framework is designed for security researchers who already use tools such as Claude Code, OpenAI Codex-style coding agents, Hermes, or local model runners. T3MP3ST acts as the operational layer around those agents, letting users point the system at an approved target through a browser-based War Room interface or a command-line workflow.

The most important detail is scope. T3MP3ST is not proof that fully autonomous AI hacking is solved. Its own documentation says the strongest benchmark claims come from a single-agent tool-backed loop and recon engine, while coordinated eight-operator swarm exploitation remains experimental.

What T3MP3ST Does

T3MP3ST positions itself as a multi-agent offensive security harness. It connects an AI coding agent to a controlled toolset and guides it through a red-team workflow that starts with discovery and ends with evidence-backed reporting.

The project says it includes 35 built-in tools by default and can expand to a larger optional arsenal when users enable additional adapters. Its stable pieces include the mission engine, War Room interface, local-agent connection, MCP server support, HTTP API, egress-scope containment, and a coordinated-disclosure pipeline.

The framework also attempts to reduce accidental out-of-scope activity. According to the project README, once a mission target is set, built-in networked tools should reject off-scope public hosts. This does not remove legal responsibility from the operator, but it shows that the developers built scope control into the platform rather than treating it as an afterthought.

How the Framework Is Structured

T3MP3ST uses an eight-operator architecture that mirrors common red-team phases. The named operators are Recon, Scanner, Exploiter, Infiltrator, Exfiltrator, Ghost, Coordinator, and Analyst.

The mapping borrows from established adversary behavior models. MITRE ATT&CK describes adversary tactics and techniques based on real-world observations, and T3MP3ST uses that style of phase-based thinking to organize its agent roles.

Operator	Main role	Current maturity
Recon	Asset discovery, OSINT, DNS and HTTP checks	Stable and benchmarked
Scanner	Service fingerprinting and vulnerability checks	Partly stable
Exploiter	Exploit attempt and validation	Experimental in swarm mode
Analyst	Finding review and reporting	Supported through reporting workflows

This structure gives T3MP3ST a clear roadmap, but the benchmarked results should not be read as proof that every operator works equally well. The project states that later-stage operators run real tool-backed loops, yet the full swarm has not been proven at scale.

Benchmark Claims and What They Mean

The headline benchmark claim is a 90.1% pass@1 score on XBOW’s 104-challenge web security benchmark. XBOW said its validation benchmark was created to test offensive security tools across realistic web vulnerability classes and reported an 85% success rate for its own system in earlier testing.

The XBOW benchmark write-up says the suite was designed around realistic vulnerability classes such as SQL injection, insecure direct object reference, and server-side request forgery. It also notes that the original benchmark set contained 104 tasks and was intended to give security teams a clearer way to compare AI-powered offensive tools.

There is an important caveat. The public XBOW validation benchmarks repository now warns that the 2024 benchmark set has become less useful for comparing frontier systems because performance has improved and the dataset is considered saturated by mid-2026.

Cybench Results Need a Careful Reading

T3MP3ST also discusses Cybench, a 40-task academic benchmark built from professional capture-the-flag competitions. The Cybench paper introduced the benchmark to measure cybersecurity capabilities and risks in language-model agents, with tasks that require command execution, file analysis, and exploit reasoning in controlled environments.

The project’s own Cybench documentation is more cautious than the sample headline numbers circulating online. It says T3MP3ST has fully imported the standalone file-only subset, while service-required tasks still need more runner work. That means the most careful description is not “T3MP3ST solved Cybench,” but that it has measured and is still extending its approach on reachable subsets.

• XBOW web benchmark: T3MP3ST claims 90.1% pass@1 on the 104-challenge suite.
• Cybench: The project describes an honest pass@1 methodology, but full benchmark coverage is still in progress.
• CVE-Zero test: The developers claim a single agent pinned 8 of 10 held-out 2026 CVEs to exact file, line, and CWE, while the broader pack surfaced all 10.
• Swarm testing: The coordinated eight-operator workflow is still not the source of the strongest published numbers.

This distinction matters because security benchmarks can become misleading when teams blend black-box, white-box, single-agent, best-of-N, and swarm results into one headline. T3MP3ST’s documentation separates these claims more clearly than many early AI security demos.

Why Security Teams Are Paying Attention

The appeal of T3MP3ST is not only the score. The framework connects existing AI coding agents to a repeatable workflow that includes target scoping, tool execution, evidence capture, and reporting. For defensive teams, that could make internal testing faster when used under a written authorization plan.

The framework also reflects a wider shift in cybersecurity. AI agents can now write code, inspect projects, run tools, and reason across long testing sessions. The Cybench research showed why this trend needs measurement, since autonomous agents can create both useful security testing workflows and real cyber risk.

At the same time, the public XBOW benchmark repository shows how quickly evaluation targets can age. A benchmark that once separated strong systems from weak ones may lose value once models improve, benchmark data spreads, or systems tune their workflows around known tasks.

Legal and Safety Limits

T3MP3ST is explicitly presented as a tool for authorized testing, research, and education. The project tells users to test only systems they own or systems where they have written permission. That warning is essential because the same automation that helps defenders can also enable abuse when used outside scope.

The project uses the GNU Affero General Public License v3.0, which is commonly used for network-facing open-source software because it includes requirements related to modified versions made available over a network.

The license does not grant permission to attack third-party systems. It controls software rights, not testing authorization. Security teams still need rules of engagement, written scope, logging, human review, and a responsible disclosure process before using any autonomous security agent.

What Ships Now and What Is Still Experimental

Area	Status	Practical meaning
Web app testing	Stable and benchmarked	Strongest public evidence comes from XBOW-style web tasks
Recon engine	Stable	Uses real tool output for discovery and evidence
Source code analysis	Experimental	White-box workflows exist, but ingest support remains limited
Smart contracts	Experimental	Focused on reproducing known exploit classes
Cloud, mobile, AD and binary RE	Roadmap or early scaffolding	Not yet the basis for strong public claims

The framework’s documentation makes one thing clear: T3MP3ST is ambitious, but not finished. Its strongest current value appears to be as a tool-backed AI security harness for controlled web testing, benchmark experimentation, and structured vulnerability research.

For defenders, the release is worth watching because it shows how quickly AI coding agents are moving from code completion into security operations. For organizations, it also raises a governance question: if AI agents can automate parts of offensive testing, internal policies must define who can use them, where they can run, and how findings get reviewed.

T3MP3ST’s strongest contribution may be its insistence on evidence-backed claims. By tying benchmark numbers to re-derivable artifacts and by marking unfinished areas as experimental, it gives security teams a clearer way to judge the project without relying only on hype. The XBOW benchmark discussion, MITRE ATT&CK framework, and AGPL-3.0 license context all point to the same conclusion: autonomous security testing is becoming more practical, but it still needs strict boundaries.

FAQ