A production-grade agentic AI system is not simply a model connected to more tools. It is a controlled decision-and-action system that can operate safely when inputs are hostile, dependencies fail, permissions change, and the model behaves unexpectedly. Reliability, security, auditability, and recovery have to be designed into the architecture.
This production-grade agentic AI security checklist focuses on the controls that separate a convincing demo from a system that can be trusted with customer data and real actions. Use it before launch, after adding a high-impact tool, and whenever a model, prompt, retrieval source, or approval workflow changes.
1. Write the permitted operating scope
Define who can use the agent, which data it can access, which tools it can call, which external destinations are allowed, and which actions require approval. Also document explicit prohibitions. “Help the support team” is not a usable security boundary; “draft replies from approved knowledge but never send, refund, export, or change permissions” is.
Attach the scope to a release and an owner. A new capability should not silently expand the approved boundary.
2. Separate instructions by trust level
System and developer instructions, user requests, retrieved documents, tool outputs, emails, and webpages must not have equal authority. Label sources by trust level and enforce instruction precedence in the orchestration layer.
Test direct and indirect prompt injection. The agent should treat instructions inside documents or web content as untrusted data, even when they appear to be security messages or administrator commands.
3. Enforce identity and tenant boundaries outside the model
Pass authenticated user, tenant, role, and delegated authority through the entire workflow. Never ask the model to decide whether a user is authorized. Authorization must occur at retrieval and tool execution time using trusted application context.
Test cross-tenant document IDs, reused browser sessions, guessed resource identifiers, cached results, and tool arguments that refer to another account.
4. Give every tool the minimum capability
Prefer narrow tools over generic shells, unrestricted browsers, or broad database interfaces. Separate read from write operations, restrict arguments, allowlist destinations, cap transaction size, and use short-lived credentials.
Maintain a tool inventory with purpose, owner, permission scope, approval policy, rate limit, logging behavior, and failure mode. Review the real API permissions, not only the friendly tool description shown to the model.
5. Validate every action deterministically
Treat model-generated arguments as untrusted input. Validate schema, type, length, object ownership, destination, data classification, transaction limits, and business rules before execution.
High-impact actions need a deterministic allow, deny, or approval-required decision. A second model or a stronger prompt may help detect risk, but it should not be the only enforcement boundary.
6. Bind human approval to an exact payload
Approvers should see the action, target, important parameters, data destination, affected account, and expected effect. Approval must be bound to that exact payload and expire after a short period.
If the agent changes the amount, recipient, destination, or operation after approval, require a new decision. Test replay, stale approvals, misleading summaries, bulk actions, rejection, and timeout behavior.
7. Control data egress
Map every path through which data can leave: responses, tool arguments, URLs, webhooks, email, files, browser forms, analytics, logs, memory, and third-party model providers. Classify sensitive data and define permitted destinations.
Test plain-text, encoded, split, transformed, and multi-step exfiltration. Enforce destination allowlists, redact secrets before logging, and prevent sensitive payloads from being sent to tools that do not need them.
8. Protect retrieval and memory
Authenticate knowledge sources, preserve tenant permissions, record source versions, and treat retrieved content as evidence rather than policy. Test poisoned documents, stale guidance, conflicting sources, malicious metadata, and unauthorized documents.
For memory, define what can be stored, who can recall it, how long it remains, and how deletion works. Test cross-user recall, sensitive-value persistence, and instructions planted in earlier conversations.
9. Isolate code, files, and browser sessions
Agents that execute code or control browsers need hardened sandboxes. Restrict network destinations, filesystem paths, processes, downloads, secret access, authenticated sessions, and execution time.
Browser tests should cover hostile pages, pop-ups, downloads, deceptive controls, cross-domain navigation, and attempts to make the agent disclose data or confirm an irreversible action. Model instructions do not replace browser or network enforcement.
10. Design for idempotency and bounded retries
An agent may repeat a call after a timeout even when the first request succeeded. Use idempotency keys for write operations, stable transaction IDs, retry limits, backoff, and duplicate detection.
Test partial failures across multi-step workflows. The system should know which steps completed, which can be retried safely, and which require compensation or human review.
11. Make behavior observable and auditable
Link the request, model version, prompt version, retrieved sources, tool calls, validated arguments, policy decisions, approvals, errors, and final effects with a stable correlation ID.
Monitor latency and reliability, but also preserve enough evidence to explain consequential decisions. Protect audit data from ordinary modification and minimize sensitive content through structured redaction.
12. Detect abuse and operational drift
Alert on repeated denied actions, unusual export volume, new destinations, privilege failures, prompt injection patterns, changes in tool-call distribution, and sharp increases in retries or human overrides.
Review permission drift, tool schema changes, retrieval-source changes, and model behavior over time. A system can become unsafe without a code deployment if external permissions or content change.
13. Provide safe degradation
Define what happens when the model, retrieval service, approval system, policy engine, or external API is unavailable. The safe response may be read-only mode, a draft without action, a human escalation, or a complete stop.
Do not allow a security dependency failure to disable the control. For example, if the approval service is unavailable, the agent should not treat the action as implicitly approved.
14. Build rollback and containment paths
Teams need a fast way to disable a tool, revoke credentials, block a destination, isolate a tenant, roll back a prompt or model, pause memory, and stop queued actions. Assign owners and practice these actions before an incident.
Where possible, make business actions reversible. If an action cannot be reversed, strengthen pre-execution validation and approval.
15. Gate releases with security regression tests
Maintain a versioned suite covering prompt injection, authorization, tenant isolation, tool restrictions, data egress, approval binding, retries, logging, and known failures. Run it after changes to models, prompts, tools, retrieval, memory, policies, or user interfaces.
Record the release, fixtures, expected outcomes, actual results, exceptions, and accountable reviewer. A production-grade process does not rely on someone remembering to manually try a few prompts.
Minimum evidence before production
- Approved operating scope and architecture diagram.
- Tool and data-flow inventories with owners.
- Authorization and tenant-isolation test results.
- Prompt injection and data-exfiltration test traces.
- Approval screenshots and payload-binding evidence.
- Audit trail for one successful, one denied, and one partially failed workflow.
- Monitoring alerts, rollback procedure, and incident contacts.
- Release-specific regression results and documented residual risks.
How to use this checklist
Mark each item as implemented, partially implemented, missing, or not applicable. Require evidence for “implemented.” Assign an owner and target date to every gap. High-impact missing controls should restrict the operating scope rather than remain as an informal future task.
Production-grade does not mean the agent never fails. It means failures are constrained, important actions are governed, evidence is available, and the team can detect, contain, explain, and improve the system.
Related resources
- AI Agent Production Readiness Checklist
- AI Agent Security Audit Scope
- AI Agent Auditability
- Prompt Injection Testing Checklist
- AI Agent Incident Response Checklist
Run the AI Agent Readiness Self-Assessment to identify priority gaps, or request an independent readiness audit before a high-impact production launch.