Home / Knowledge hub / Prompt injection
Knowledge hub
Prompt injection
Text that tricks an AI model into ignoring its instructions and doing something else.
What it means
Prompt injection is when text given to a language model changes what the model does, against the wishes of the developer. The model receives instructions and data in one stream of words, and it cannot reliably tell which words are the rules and which are just content to work on.
It comes in two forms. In direct injection, a user types something such as “ignore your previous instructions”. In indirect injection, the instruction hides in content the model reads on the user’s behalf, such as a web page, a document or an email. The second is harder, because nobody typed anything suspicious.
Why it matters
The risk is what the model can do or reveal. If it only writes text, the harm may be embarrassment. If it can search internal documents, send email or call your API, an injected instruction can pull out data or trigger actions with the permissions of the person using it. The OWASP list for language model applications ranks it first.
There is no complete fix today. Filters help and they get bypassed. That puts weight on design: what the model is allowed to do, whose permissions it runs with, and what happens to the business if its answer is wrong or hostile.
How it shows up
It shows up in chat assistants, support bots, summarisers, and agents connected to tools. A document with a hidden line telling the assistant to send its contents to a web address is a typical test. So is a support ticket that instructs the bot to refund an order.
The sensible defences are structural. Give the model the smallest set of permissions it needs, check its requests on the server like any other untrusted input, require human approval for sensitive actions, and treat output as unsafe before it reaches a browser or database. See insecure output handling and excessive agency.
Questions people ask
Prompt injection, answered
Is prompt injection the same as a jailbreak?
They are related but not identical. A jailbreak tries to make a model break its safety rules. Prompt injection tries to make it follow an attacker’s instructions inside an application, often to reach data or tools.
Both use text as the weapon.
Can prompt injection be fully prevented?
Not reliably. Because instructions and data share the same channel, no filter catches everything. The practical approach is to limit what a successful injection can do.
Assume the model can be fooled and design around it.
What is indirect prompt injection?
The hidden instruction sits in content the model fetches or is given, such as a web page, a PDF or an email, rather than in the user’s own message.
The user may never see it.
How do we test an AI feature for it?
Feed the feature content containing instructions, try to reach data from other accounts, and see what actions it will take. Then check each tool call is authorised on the server, not by the model.
Record the exact input and output.
Does a system prompt protect us?
No. A system prompt shapes behaviour but is not a security boundary, since a model can be talked out of it. Real controls live outside the model: permissions, checks and approvals.
Do not put secrets in it.
Sources
Where this comes from
Related
Terms that sit next to this one
Indirect prompt injection
Instructions hidden in a web page or file that an AI assistant then obeys.
Jailbreak
A prompt crafted to push an AI model past its safety rules.
OWASP LLM Top 10
The OWASP list of the main security risks in applications built on language models.
Excessive agency
Giving an AI agent more tools and permissions than its task needs.
Insecure output handling
Trusting what a model writes and passing it on to code or a browser unchecked.
What can the model reach on its own?
Prompt injection is dangerous where the model has real reach. We check the web app and API around it, including cross-account access and business-logic flaws, and report only what we exploited. Ask us to run it on one of your applications.
Or start with a $199 pilot on one application: thirty days, success criteria agreed before day one, credited against the annual if you convert.