Issue Details
Prompt Injection
llm01-2024 • Rank #1 in OWASP 2025
Explanation
Prompt injection exploits the model instruction channel by inserting adversarial content that competes with or overrides trusted directives. Attackers embed malicious instructions in user prompts, retrieved documents, tool outputs, or web content to force unsafe responses and actions. The risk grows in agent systems where model output can trigger tools and external side effects.
Remedy
Separate trusted control instructions from untrusted content, and enforce policy checks before any sensitive action. Constrain tool usage with allowlists and context-aware guards, and apply output moderation for actionability and policy compliance. Include adversarial prompt testing in CI to validate defenses against known injection patterns.
| Year | Rank | Category |
|---|---|---|
| 2024 | #1 | llm01-2024 - Prompt Injection |