Developers working with AI application security and prompt injection risks

Prompt injection happens when untrusted input changes an AI system’s behavior in a way the application did not intend. It is especially important for AI agents because a manipulated model may have access to tools, data and external systems.

Direct vs indirect prompt injection

Prompt Injection & LLM Security video thumbnail — Official explainer: Prompt Injection & LLM Security — Microsoft Academy Hub.

Direct injection is supplied by the user in the prompt. Indirect injection is hidden or embedded in content the model is asked to process—such as a web page, email, document, retrieved knowledge or tool output. An agent can encounter hostile instructions without the user typing them.

Why this is different from ordinary software input

Traditional software tries to separate code from data with strict syntax and parsers. LLMs consume natural language where instructions and data can look similar. A system prompt can tell a model to ignore malicious instructions, but that alone is not a strong security boundary.

What an attacker may try to achieve

Override the application’s intended task.
Extract sensitive context.
Convince an agent to call an unsafe tool.
Redirect data to an external destination.
Influence downstream code or workflow decisions.
Cause excessive spending or repeated actions.

OWASP’s 2026 AI-security context

OWASP’s 2026 guidance continues to treat prompt injection and related agent-control problems as major application-security concerns. Its 2026 exploit round-up describes a shift from theoretical risks toward incidents involving agent identities, orchestration layers, excessive autonomy and malicious context.

Why “just filter the prompt” is not enough

Attackers can vary wording, encoding, language, formatting and the location of malicious content. More importantly, the model may need to read untrusted text to perform its job. The goal is therefore not to make every hostile string disappear; it is to ensure that untrusted content cannot directly obtain authority.

Seven practical defenses

Least privilege: limit what tools and data the model can access.
Separate instructions from data: label retrieved content as untrusted and do not treat it as policy.
Validate tool arguments: enforce schemas, allowlists and business rules outside the model.
Require approval: gate payments, deletions, messages, deployments and permission changes.
Sanitize outputs before execution: never execute generated code, SQL, shell commands or HTML blindly.
Constrain destinations: restrict outbound URLs, recipients and data-transfer paths.
Log and test: keep an audit trail and red-team the complete workflow, not only the model.

Prompt injection and RAG

Retrieval-augmented generation can introduce untrusted text from search indexes, documents or websites. Retrieval improves grounding but does not automatically make content trustworthy. Provenance, access controls and output validation remain necessary.

Prompt injection and AI agents

The danger grows when a model can act. A malicious web page that merely changes a summary is one problem; a page that influences an agent with email, cloud or code permissions is a different risk class. This is why OWASP separately highlights excessive agency, tool misuse and agent control.

Security checklist for developers

Map every data source the model can read.
Map every action the model can take.
Remove unnecessary credentials.
Use scoped, short-lived tokens.
Validate tool calls with deterministic code.
Require confirmation for irreversible actions.
Test malicious content in documents, websites and tool results.
Monitor unusual action sequences.

FAQ

Can a system prompt stop prompt injection?

It can help shape behavior, but it should not be treated as the only security control. Authorization and validation should live outside the model.

Is prompt injection the same as jailbreak?

They overlap but are not identical. Jailbreaking typically tries to bypass model behavior restrictions; prompt injection targets an application’s instruction hierarchy or workflow, often using untrusted external content.

Threat-model the agent, not just the prompt

Prompt injection becomes more serious when the model can reach valuable data or tools. The right question is therefore not only “Can the model be tricked?” but “What can happen if it is tricked?” A summarization assistant with no external permissions has a different risk profile from an agent that can send email, modify cloud resources or approve payments.

Use deterministic controls around probabilistic models

Security-sensitive decisions should not depend only on whether a model “understands” a policy. Enforce important rules in ordinary application code: which tools are available, which recipients are permitted, maximum transaction values, allowed file types, data-classification restrictions and whether an action requires approval. The model can propose an action; the surrounding system should decide whether that action is allowed.

Indirect prompt injection is a data-provenance problem

An agent may encounter instructions inside a web page, document, email, ticket or retrieved knowledge base. Treat that material as untrusted content even when it looks authoritative. Keep provenance metadata, distinguish system policy from retrieved text and prevent content from silently gaining the same authority as developer instructions.

Design approval gates around impact

Action	Recommended control
Read public information	Automatic with logging
Read sensitive internal data	Scoped permissions and access checks
Send external message	Recipient allowlist or human confirmation
Delete, deploy, pay or change permissions	Explicit human approval plus audit log

How to test an agent for prompt injection

Map every source of untrusted text.
Insert malicious instructions into each source type, not only the chat box.
Test whether the agent leaks secrets, changes destinations or calls unintended tools.
Test encoded, multilingual and indirect instructions.
Verify that application-side policies block prohibited actions even when the model tries them.
Record failures and add them to regression tests.

Prompt injection is unlikely to be solved by one universal filter. The practical goal is to reduce the authority of untrusted text and limit the damage any single model mistake can cause. That approach aligns naturally with Zero Trust principles and least-privilege design.