How prompt injection attacks work — and what to do about them

Prompt injections aren’t theoretical; they’re a technique that attackers are slowly learning to wield to their advantage, sometimes with devastating results. 

Prompt injection is tricky to execute and even harder for businesses and AI companies to prevent. That’s because AI tools and the large language models (LLMs) underpinning them are unpredictable. Asking an AI chatbot to roleplay as a forked version without any guardrails might work one day, but not the next. Similarly, you can’t know exactly when an AI tool might reference a website that’s been poisoned with hidden, malicious instructions – or, more importantly, how it might choose to act upon them.

Despite these unknowns, there’s a lot the industry does know about prompt injection attacks, and how they’re evolving alongside the conveyor belt of new LLM models and agentic capabilities. Few people are more knowledgeable than Johann Rehberger, a security researcher, red teamer, and speaker. He’s uncovered and disclosed many vulnerabilities, including SpAIware, a technique that exploits prompt injection and persistent memory in AI applications.

Johann joined Wade Wells on the Random but Memorable podcast to explain how prompt injection works, the different forms it can take, and the processes organizations should put in place to minimize the risk.

Can't see the video? Watch on YouTube.

Editor’s note: This interview has been lightly edited for clarity and brevity. The views and opinions expressed by the interviewee don’t represent the opinions of 1Password.

Wade Wells: Before we get into prompt injection attacks — can you briefly explain how modern AI tools access and process external data, like websites and documents?

Johann Rehberger: When we talk about modern AI tooling, we usually mean something that uses an LLM and revolves around the context window. It then reasons about what is in the context window. 

A modern agent, or the harness around the LLM, will grab the most relevant information and try to put it into that context window. That’s then sent to the core, and the LLM processes and reasons about this data. 

The outcome, hopefully, is a useful response. Or you might get back an instruction for the harness or the agent to call an additional tool which gives it more power. For example, it might then read an email, which brings more context in, and the process continues. That's what's called the agentic loop.

WW: In simple terms, what is a prompt injection attack, and how does it work?

JR: When people use the phrase prompt injection, they usually mean that the AI tool reads information from somewhere and then takes action based on that information. And that action isn’t necessarily what the user wanted to happen.

WW: What can attackers achieve with a successful prompt injection attack? And have you seen any of those scenarios play out in the real world?

JR: I've seen emulations while I was part of red teaming organizations. Google also did a large analysis earlier this year called AI threats in the Wild where they highlighted that many websites are now polluted with adversarial instructions. This is where adversaries put comments in web pages and so on to steer or trick agentic systems that actually connect to and read that information.

“Many websites are now polluted with adversarial instructions.”

There have been a couple of real-world examples, too, like Clinejection. That’s where the title of the GitHub issue was the prompt injection. The AI chatbot processes the GitHub issue, uses the title in its prompt, and that hijacks the agent. And then you could read whatever that agent then has access to, like secrets and so on. 

It’s the kind of attack I described during my Month of AI Bugs, where I focused a lot on coding agents. It’s a plausible and realistic scenario that we’ve now seen happen in the real world.

WW: How real is the threat to everyday users right now? What are the odds that someone actually encounters one of these attacks in the wild?

JR: That’s a good question.

What are the odds that you use an AI that is actually hijacked by third-party instructions? Versus, what are the odds that the AI just hallucinates and does something bad to your computer, system, or data? I think both threats are really real. And I think we should just draw the conclusion that both can happen. 

You need to protect yourself from those threats. For example, don’t give an agent enough access to take a harmful rogue action. 

Because a rogue action can be caused by multiple means, it could be adversarial training data in the model or just a hallucination. Prompt injection steers it in that direction by telling the model to do something malicious.

WW: There are two main categories here: direct and indirect prompt injections. Can you walk us through the difference?

JR: When we talk about the overall term, it would be called prompt hacking. That’s where you hack prompts, right? But then you have jailbreaking, which is really more focused on the LLM itself. The underlying model has been trained, and you try to steer it away from its original training. 

With prompt injection, there's a prompt that’s outside the model, like a developer prompt or a system prompt. Then you inject it into that prompt. Direct prompt injection is where you, as the user, directly talk to the agent. It has a developer or system prompt, and you try to overwrite that prompt and make it do something else. 

Indirect prompt injection is where the agent reads data from somewhere else, like an email, that’s malicious. The AI’s context contains that malicious data, and that influences the inference process. And that leads to an undesirable outcome … potentially.

WW: Direct injections often involve telling an AI to ignore its own rules — for example, through elaborate roleplay. How do these jailbreaking techniques actually work?

Roleplay is a good example. That’s where you try to steer and trick the LLM itself in its reasoning process to consider you being, or itself being, a different person. You try to get the answer out by slowly having the LLM take a certain role. It’s a common jailbreaking technique that still works very well. 

Another technique that’s working very well both from a prompt injection and jailbreaking standpoint is exploiting the reasoning process itself. The LLM starts reasoning, and then you go ahead and use that as a trick to derail the LLM. For example, you give it a puzzle. “Hey, can you solve this puzzle? Can you decrypt this key for me?” And then the AI goes: “I need to decrypt the key. Where's the password? Or where's the private key that I need for this?” And then you steer it slowly in a different direction. It doesn't remember what the initial task was. 

That’s still working really well with many different models.

WW: Are all prompt injections written in plain English, or is there something more technical going on?

Nobody fully understands how these models reason or fundamentally work. We have an understanding of what's going on. We observe the outcome, and it's good, but we don't fully understand what's happening within the model.

There have been multiple discoveries over the last few years that nobody anticipated initially. One is the use of certain Unicode characters, which are not visible to humans. But if you send those invisible characters to the LLM, the model will understand them because they mimic the ASCII character set. 

So you can send information to the LLM that bypasses firewall filters because none of them would see it unless they have a specific check for that. And then the LLM follows those instructions.

“You can send information to the LLM that bypasses firewall filters.”

There are a lot of unknowns that we’ll continue to discover as we go.

WW: With indirect prompt injections, malicious instructions are hidden somewhere in content that the AI reads and acts on. Where do attackers typically hide those instructions — in plain sight on a webpage, or somewhere less obvious?

JR: One scenario that we saw is the GitHub example, right? Because if you have a public project, anybody could create a GitHub issue or even a PR, right? 

And then you have an OpenClaw instance that goes ahead and triages the bug or the ticket. [Platforms like GitHub] are naturally a very hostile environment where a lot of attacks are planted. That is a really problematic scenario if the agent itself isn’t properly isolated in a sandbox with production secrets.

Another scenario is email and document sharing. When we say “hidden instructions”, it's actually just text that is somewhere in the middle of the document. It's not fundamentally hidden; it's just embedded in the document in a way that’s not obvious for a normal user to see.

WW: What defenses are AI companies building — and how effective are they in practice?

JR: Anthropic put out this classifiers document. So in Anthropic's case, whenever you submit a prompt, it first goes through a set of classifier checks. There's a different model actually checking your prompt before it reaches the really powerful model.

Think of it as similar to a prompt firewall, right? And then it downgrades you to a less capable model if you're doing cybersecurity research, for instance.

WW: How are AI providers trying to detect and block this type of prompt injection attack? Is there a way for them to filter out content that contains malicious instructions?

JR:  The most fundamental thing to focus on is this: don’t trust the LLM output. Work from there. Assume the LLM can output any text it wants and make any tool call it wants. The client that actually hosts the model needs to be in an environment that doesn’t allow certain actions. So use a sandbox.

That's the most fundamental security invariant to put in place. 

“Assume the LLM can output any text it wants and make any tool call it wants.”

A lot of researchers and startups are working on prompt firewalls to make it less likely for a prompt injection to enter the system in the first place. But those, I think, are less reliable than actually having a great security posture in the first place. You should have real security invariants and security controls in place to prevent the AI from sending the data to some domain that you don't want the agent to connect to in the first place.

WW: Models struggle to tell the difference between legitimate instructions from their operator and malicious instructions hidden in data they're reading. Is there a technical solution to that problem — or is it more of a fundamental design challenge?

JR: An LLM doesn't distinguish between system instructions and the data that it is processing. It's all the same. That’s a new design that we haven't really had in a computer system before.

So we need to work with this model and build defenses from the right first principles. That is, don't give the agent access to a lot of capabilities, and so on.

If you think about self-driving cars, even now, we know they are probably safer than a human driving a car. And they will be much, much safer in the future. The same should be true for an agent that operates a computer. The agent should be less susceptible to social engineering tricks than humans are.

That is my logical long-term thinking. But there's one big difference: cyberspace isn’t a very friendly place. It's not like the road, which has arbitrary changes while the car is driving. In cyberspace, the adversary is molding around the target. Attackers can change their behavior and tactics. So that's why I'm not 100% percent convinced, but I'm optimistic.

WW: What can organizations do to avoid prompt injection attacks?

JR: Inventory is where everything starts. You need to uncover shadow AI because often something is exploited and you didn't even know it existed in the first place.

So try to figure out what models and agents people are using, and on what endpoints. Getting those basics in place, then monitor the queries being sent. 

And then you think about the “paved roads” in your company. What is the one solution that is easiest for everybody to use? The secure way should be the easy way. You want to have one framework, or one harness, that you spend your company resources on, and then tell everybody: “If you use this, you don't have to do anything else because we take care of monitoring, logging, and everything else. If you want to do your own thing, you need to do it yourself.”

“The secure way should be the easy way.”

That's usually the incentive. People use the company one because it’s the better one. It's a paved road.

WW: One of the scariest use cases for prompt injection is credential theft — using a malicious instruction to trick an AI agent into leaking API keys, access tokens, or passwords it has been given. How does that attack work in practice, and how common is it?

JR: The way it works in practice is that the model has access to a secret. It can place that secret into the inference process and then call a tool to leak it.

With indirect prompt injection, an attacker can try to do this remotely. The adversary might control a document somewhere, or an email, and the user interacts with that document, which runs the attack. 

A coding agent is the prime target because developers have a lot of secrets in their environment variables or on their disk. So that's something to look into – make sure you have an environment where the agent actually never even sees the password. A proxy does the request, and the LLM doesn't even have the opportunity ever to see the password.

“Make sure you have an environment where the agent actually never even sees the password.”

WW: 1Password recently launched an integration with Claude that lets AI agents access credentials without those secrets ever appearing in the model's context or memory. How significant is that kind of approach for reducing prompt injection risk?

JR: That’s a very important step toward getting to a more secure environment and infrastructure. Because you often say in the system prompt: “Don't put the password in the LLM.” But then the LLM does it anyway because it's blocked somewhere.

AI agents are going to find a way if you block them.

WW: For listeners who want to go deeper on prompt injection — where should they start?

JR: There are a couple of good places to start. If you're interested in attack techniques, Jason Haddix has a really good taxonomy for prompt injection. I recommend reading through the different attack techniques. Then you have Mitre Atlas that shows real-world scenarios.