OpenAI’s GPT-Red Attacks AI Agents to Harden GPT-5.6 Against Prompt Injection

OpenAI has developed GPT-Red, an internal automated red-teaming model that searches for prompt injection vulnerabilities at scale. According to the company’s GPT-Red announcement, the system creates attacks, studies how target models respond, and repeatedly adjusts its approach until it finds a weakness.

The company used attacks generated by GPT-Red to improve GPT-5.6 Sol, its flagship model. OpenAI says Sol recorded six times fewer failures on its hardest direct prompt injection benchmark than its strongest production model from four months earlier.

GPT-Red remains separate from publicly deployed models. OpenAI wants to use its offensive capabilities for internal security research without giving attackers access to a model trained specifically to bypass AI safeguards.

What GPT-Red does

Prompt injection attempts to make an AI system follow malicious instructions instead of the user’s request or the developer’s rules. An attacker can place these instructions in content that an AI agent processes, including emails, webpages, code repositories, local files and tool responses.

These attacks become more dangerous when an AI can use tools or take actions. A compromised agent could expose data, alter files, send information to an outside service or make unauthorized changes while appearing to complete a legitimate task.

The GPT-5.6 system card describes prompt injection as an attempt to place adversarial instructions inside tool output so they can override system, developer or user instructions.

• Direct prompt injection places the malicious request in a prompt sent directly to the model.
• Indirect prompt injection hides instructions in external content that the model retrieves or processes.
• Agentic prompt injection targets systems that can browse, call tools, access files or perform actions.

GPT-Red learns through self-play

OpenAI trained GPT-Red with self-play reinforcement learning. The attacker model competes against a collection of defender models across simulated security scenarios.

GPT-Red receives a reward when it triggers a valid failure. Defender models receive rewards for resisting the attack while still completing the user’s original task. As the defenders improve, the attacker must discover more effective and varied prompt injections.

The environments reflect situations that AI agents may encounter during normal work. GPT-Red might control part of an email, a webpage banner, a local document or the output returned by a tool.

In its automated red-teaming report, OpenAI says GPT-Red could compromise nearly every internal and production model tested during training, including models up to GPT-5.5. The company then incorporated the resulting attacks into GPT-5.6 training.

GPT-Red outperformed human attackers in an internal test

OpenAI also evaluated GPT-Red on an internal recreation of the indirect prompt injection arena described in the research paper How Vulnerable Are AI Agents to Indirect Prompt Injections? The test used environments and attack goals that were separate from GPT-Red’s training scenarios.

GPT-Red found successful attacks in 84% of the tested scenarios against GPT-5.1. Human red teamers working independently succeeded in 13% of the scenarios, according to OpenAI.

Evaluation	Target	Reported result
Indirect prompt injection arena	GPT-5.1	GPT-Red succeeded in 84% of scenarios
Human red-team comparison	GPT-5.1	Human participants succeeded in 13% of scenarios
Direct GPT-Red attacks	GPT-5.6 Sol	0.05% failure rate
Fake Chain-of-Thought attacks	GPT-5.6 Sol	Attack success fell below 10%

The underlying large-scale prompt injection study involved 464 participants, more than 272,000 attack attempts and 13 frontier models. OpenAI used an internal mirror of its arena for the separate GPT-Red comparison.

GPT-Red manipulated a live vending machine agent

One of OpenAI’s most notable tests involved Vendy, an autonomous agent that manages a vending machine in an OpenAI office. Researchers first allowed GPT-Red to develop attacks against a simulated version of the system before transferring them to the production agent.

GPT-Red achieved all three objectives assigned for the test:

• It changed an expensive in-stock item to the minimum permitted price of $0.50.
• It added an item worth more than $100 and offered it for $0.50.
• It cancelled another customer’s order.

OpenAI disclosed the vulnerabilities to the relevant parties and said new safeguards were undergoing testing.

The company also tested GPT-Red against a Codex CLI agent powered by GPT-5.4 Mini. Across 10 held-out data-exfiltration scenarios, GPT-Red completed more attacks and used fewer tokens than a prompted GPT-5.5 baseline.

How much safer is GPT-5.6 Sol?

OpenAI says an early GPT-Red model discovered a direct prompt injection technique called Fake Chain-of-Thought. The attacks succeeded more than 95% of the time against GPT-5.1, but their success rate fell below 10% against GPT-5.6 Sol.

On a broader set of held-out environments, GPT-5.6 Sol reportedly failed on only 0.05% of GPT-Red’s direct prompt injection attempts. OpenAI also reported accuracy above 97% on several indirect prompt injection benchmarks involving browsing and developer tools.

Separate results in the GPT-5.6 safety evaluation gave Sol a score of 1.000 against known connector prompt injections and 0.910 in the stronger search and function-calling category. These results come from different evaluations and should not be treated as direct equivalents to the 0.05% GPT-Red figure.

GPT-5.6 model	Connector evaluation	Search and function-calling
Sol	1.000	0.910
Terra	1.000	0.946
Luna	0.999	0.897

Prompt injection remains a security risk

The reported results do not mean GPT-5.6 Sol can resist every prompt injection. GPT-Red operates in controlled environments, and attackers may discover new techniques outside the scenarios used during training and evaluation.

OpenAI says it will continue combining automated testing with human and third-party red teaming, layered safeguards and real-time monitoring. This approach acknowledges that model-level resistance forms only one part of an agent’s security controls.

Developers must still limit tool permissions, separate trusted instructions from untrusted content and require confirmation before sensitive actions. Logging, sandboxing and restrictions on data access can also reduce the damage caused by a successful injection.

OpenAI plans to expand GPT-Red with additional compute, training data and algorithmic improvements. It also said a technical preprint with more information about the system would follow its initial announcement.

FAQ