Prompt injection
An attack that sneaks instructions into content the model will read (docs, email, web pages) so those instructions override the developer’s system intent.
Learn
When to use it
Use prompt injection when testing the security of AI models against unintended commands. Prompt injection attacks reveal how easily external content can override a model's intended behavior, helping developers identify vulnerabilities in systems like document processing workflows.
Quick example
In a scenario where Copilot for Word is used to assist with document editing, an attacker might embed hidden instructions within a Word document. When Copilot reads the document, these instructions could override its intended behavior, executing the attacker's commands instead. In this case, the Word document acts as the vector for prompt injection, exploiting the AI's input processing.
Word document → prompt injection → Copilot for Word → unintended behavior
Ecosystem
Prompt injection interacts with various components in AI systems, particularly those involving document processing and input validation. It affects how models interpret and execute instructions within a broader workflow.
┌─ document processing ─┐
input →│ prompt injection │→ model execution
└─ input validation ────┘
Misconceptions
| Misconception | Rebuttal |
|---|---|
| It's a bug in the model | It's an exploitation of input handling |
| Only affects text-based AI | Can target any model processing external input |
Trade-offs
- Security testing — risk of exposing sensitive data
- Vulnerability awareness — requires careful handling to avoid misuse