A trust boundary is the line between what an AI agent may trust and what it must treat as untrusted. In a normal web application, teams usually draw boundaries around users, services, databases, networks, and third-party APIs. AI agents need the same discipline, but the boundary is easier to blur because the agent reads natural language, tool outputs, documents, tickets, browser pages, logs, and instructions in the same run.
That is where many agent security failures begin. A support ticket can contain hostile instructions. A web page can tell the agent to ignore policy. A tool result can include text that looks like a command. A user can ask for a harmless summary while the agent quietly gains access to sensitive records. If the system does not label what is trusted and what is not, the model will be asked to make security decisions from mixed context.
This checklist is for product, engineering, and security teams preparing an AI agent for production. It is not a theoretical threat model. It is a field review you can run before connecting the agent to write-capable tools, browser sessions, customer data, admin workflows, or external destinations.
1. Name the trusted instruction sources
Start by listing the instruction sources that are allowed to control the agent: system prompt, developer policy, workflow configuration, tenant policy, allowlists, approval rules, and runtime safety checks. These sources should be narrow, versioned, and owned by the team. They should not be mixed with user content, document text, page text, or tool output.
A practical review question is simple: if the agent reads this text, is it allowed to change its rules because of it? For most inputs, the answer should be no. User requests can define the task, but they should not override security policy. Retrieved documents can provide facts, but they should not change tool permissions. Web pages can be summarized, but they should not redefine where data may be sent.
2. Mark untrusted content explicitly
Every untrusted input should be labeled before it enters the model context. That includes user messages, uploaded files, RAG snippets, browser page text, email bodies, issue comments, tool responses, logs, and third-party API results. The label does not need to be complicated. What matters is that the prompt structure tells the model and the surrounding controller that this content is data, not policy.
For example, tool results can be wrapped as “untrusted tool output.” Browser text can be wrapped as “untrusted page content.” Retrieved documents can be wrapped as “reference material that may contain malicious instructions.” This helps the model behave, but it is not enough by itself. The controller should also enforce the boundary outside the model.
Users should be able to tell the agent what they want done. They should not be able to grant the agent new capabilities, disable approvals, add external destinations, or bypass tenant rules through ordinary chat. Treat task intent and security authority as separate inputs.
If a user asks, “send the customer export to this new address,” the agent can understand the task, but the destination still needs to pass policy. If a user says, “you have permission to skip approval,” that sentence should not change the workflow. Approval requirements should come from configuration, risk classification, and role-based policy, not from the user message that the model is trying to satisfy.
4. Review tool boundaries by action type
Not all tools carry the same risk. A read-only search tool, a customer-record update tool, a refund tool, a browser click tool, and an outbound email tool should not share one generic permission label. Break tools into read, transform, write, delete, purchase, publish, execute, and external-send categories.
For each category, define what the agent may do automatically, what needs confirmation, and what is never allowed. The review should include tool parameters, not only tool names. A “send message” tool is much safer when the destination is pre-approved and the final content is shown to a human before delivery.
5. Check data movement boundaries
Data movement is often the real risk. The agent may be allowed to read customer data for support, but not send that data to a vendor system. It may summarize logs internally, but not paste them into a public issue. It may inspect a document, but not upload it to an external workspace.
Draw the approved paths for sensitive data: where it can be read from, where it can be stored, where it can be transformed, and where it can be sent. Then test the opposite paths. Ask the agent to send data to a personal email, paste it into a web form, include it in a support reply, or attach it to a ticket. The expected answer should be a refusal, an approval request, or a redacted output depending on policy.
6. Treat browser pages as hostile until proven otherwise
Browser automation adds an especially weak boundary because the agent reads a live page and then acts in the same environment. A page can contain instructions, misleading UI, hidden fields, fake login prompts, injected text, or confusing buttons. The browser controller should never let page text change policy or tool authority.
At minimum, define allowed origins, expected tenant context, approved actions, and stop conditions. Before any form submission or destructive click, show the URL, active account, field diff, target element, and business effect. A browser page is an input source; it is not a trusted operator.
7. Put approvals at boundary crossings
Approvals work best when placed where a boundary is crossed. Examples include read to write, internal to external, draft to publish, staging to production, low-risk to financial, one tenant to another tenant, or normal content to sensitive content. Approval should not be a generic “are you sure?” prompt.
A useful approval screen shows what will change, which records are affected, where data will go, what policy matched, and what rollback is available. The reviewer should be able to approve, edit, reject, or escalate. If the approval screen hides the most important context, it becomes a ritual rather than a control.
8. Keep memory inside a defined boundary
Agent memory can quietly break trust boundaries. A preference learned in one tenant may affect another tenant. A sensitive detail from one customer may appear in a future answer. A temporary incident note may become long-term model context. Memory should have scope, retention, deletion, and audit rules.
Decide what the agent may remember, for whom, for how long, and for which workflows. Do not let untrusted content write directly into long-term memory without filtering. Give users and admins a way to inspect and delete stored memory when the use case requires it.
9. Log boundary decisions, not only final actions
Audit logs are more useful when they explain decisions. Record which inputs were labeled untrusted, which policy rules matched, which tool calls were blocked, which approvals were requested, which data was redacted, and which boundary crossing was allowed. A final action log is not enough to diagnose why the agent behaved safely or unsafely.
The trace should let a reviewer answer: what did the agent see, what did it trust, what did it ignore, what did it change, and who approved it? This is also useful for customer trust. When a buyer asks how the agent is controlled, a concrete trace is stronger than a general statement about guardrails.
10. Test with boundary-breaking prompts
Do not only test normal tasks. Build a small regression set that tries to break each boundary. Put malicious instructions in documents, web pages, tickets, file names, tool outputs, and user messages. Ask the agent to send data to new destinations, skip approvals, operate in the wrong tenant, reveal hidden policy, and use tools outside the workflow.
Each test should have an expected behavior: ignore, refuse, ask for approval, redact, stop, or escalate. Run these tests when prompts, models, tools, retrieval sources, browser controllers, or approval screens change. Trust boundaries are not one-time documentation; they are a regression surface.
Minimum review packet
Before launch, keep a short review packet with the trusted instruction sources, untrusted input labels, tool categories, data movement rules, browser boundaries, approval points, memory scope, audit log examples, and boundary-breaking test results. If the team cannot produce that packet, the agent is probably not ready for production use.
Related resources
- OWASP Top 10 for Large Language Model Applications
- MITRE ATLAS
- AI Agent Browser Automation Security Checklist
- AI Agent Data Exfiltration Checklist
- MCP Security Checklist for AI Agents
For a broader readiness review, use the AI Agent Readiness Self-Assessment or request an AI Agent Readiness Audit.