Countering AI Hacking Agents: The Challenge of Prompt Injection Attacks

Prompt injections, the harmful commands that attackers embed in content to manipulate large language models (LLMs), have become a favored tactic for turning AI platforms against their users. Just a cleverly crafted command hidden in an email or calendar invitation can often lead an LLM to leak sensitive information or engage in other detrimental actions.

Recently, however, defenders have started to adopt prompt injection strategies as well.

Researchers from Tracebit revealed on Monday that placing prompt injections alongside passwords, cryptographic keys, and various secrets stored on Amazon Web Services can effectively neutralize attacks from AI hacking agents. These prompts push the attacking LLM to execute actions that its guardrails—safety measures put in place by AI developers to prevent harmful actions—are meant to stop. As a result, the LLM shuts down.

For instance, one prompt might instruct the LLM to outline steps for producing inhalable Anthrax spores, or, specifically for LLMs developed in China, to reference the iconic Tank Man from the 1989 Tiananmen Square incident. When the LLM encounters these restricted commands, it ceases to follow its prior instructions. The researchers have labeled this strategy as context bombing.

“Ultimately we’re triggering a refusal mechanism in the context,” explained Andy Smith, cofounder and CEO of Tracebit, regarding the nomenclature. “What we aim to convey is the powerful, immediate effect this has, which can be challenging for the agents to reverse. Once they encounter this context, they will continue to refuse.”

According to Tracebit, initial tests indicate that context bombing holds significant promise. They evaluated models including Opus 4.8, Gemini 3.1 Pro, GLM 5.2, DeepSeek 4 Pro, and Kimi 2.6 by instructing them to perform routine developer tasks, which led the models to enumerate resources and inadvertently discover the planted strings. These trials were conducted within a simulated AWS environment.

“Across five leading models and 152 attack attempts, embedding one of these strings within a decoy secret reduced the rate at which agents attained full account admin from 57 percent to 5 percent, and total compromise (including leaving a persistent foothold) from 36 percent to 1 percent,” Monday’s update reported. “The most effective agent in our tests, Opus 4.8, failed every time when faced with a context bomb, compared to achieving admin access in 93 percent of runs previously.”

On average, across the five models and 152 trials, the findings included:

• Admin privilege escalation dropped from 57 percent to 5 percent
• Admin escalation with a persistent foothold decreased from 36 percent to 1 percent
• Successful attack paths diminished from 91 percent to 15 percent
• On average, runs completed 1.53 attack paths successfully, reduced to just 0.16
• No runs managed to complete an attack path without triggering at least one canary detection

This research builds upon findings from May, when Tracebit unveiled a method for defenders to receive alerts when their infrastructure faces threats from AI agentic adversaries. This approach involves deploying AWS resources that mimic legitimate ones but are, in reality, inactive. They coexist with active resources. When probing occurs from an agentic AI, defenders are alerted. Similar to “canaries” in coal mines, these resources assist in threat detection before serious consequences arise.