Indirect Prompt Injection: Attacks Hidden in the Page
In February 2023, a team led by researcher Kai Greshake published work with a title that captured the problem exactly: “Not what you've signed up for.” They demonstrated that an AI assistant connected to external content — like Bing's chat mode reading the web page you were viewing — could be hijacked by instructions hidden in that content, without the user typing anything malicious at all. They named the class of attack indirect prompt injection.
Direct prompt injection is when a user types a malicious instruction. Indirect prompt injection is subtler and, for many real deployments, more dangerous: the malicious instruction lives in data the assistant consumes on the user's behalf — a web page, a document, an email. The user asks an innocent question; the assistant reads a poisoned source; the hidden instructions inside that source take over.
An attack with no visible attacker
The Greshake team showed the technique against the Edge browser's Bing chat sidebar, which could answer questions about the page in front of you. They planted instructions in a web page as text invisible to a human reader — zero-size or hidden font. When the assistant ingested the page to help the user, it also ingested the attacker's instructions and adopted a secret agenda: in one demonstration, steering the conversation to extract the user's name and then exfiltrate it through a crafted link. The user saw a helpful assistant; they never saw the payload.
Why it resists easy fixes
The reason indirect prompt injection is so stubborn is fundamental to how current LLMs work: they do not reliably separate instructions from data. Everything in the context window is just text, and text that looks like a command tends to be followed. There is no robust in-band way for the model to know that the words in a retrieved page are content to summarise rather than orders to obey. Presented at Black Hat USA 2023, the research made clear this is an architectural property, not a bug in one product.
Designing around an unsolved problem
- Treat all retrieved content as untrusted. Anything the assistant reads from the web, a file, or a mailbox can carry instructions. Never grant it the same trust as the user's direct input.
- Constrain capability, not just content. Limit what the assistant can do after reading external data — restrict outbound requests, tool calls, and actions so a hijack cannot reach far.
- Isolate high-trust actions from untrusted context. Do not let a session that has ingested arbitrary web content also hold the authority to move money, change settings, or read other users' data.
- Monitor for hijacked behaviour. Sudden shifts in an assistant's goals — asking for data it shouldn't need, emitting unexpected links — are signals worth catching.
Years on, indirect prompt injection remains one of the defining unsolved problems of AI security, sitting at the top of the OWASP risk list for LLM applications. The Greshake work was the early, rigorous warning. Its enduring message for defenders is that every data source you connect to an assistant expands the set of parties who can put words in its mouth. You cannot account for those injection paths — or contain what a hijacked assistant might do — if you have not mapped what your AI systems are allowed to read and to reach.