- enforcement traffic planes
- protocol-independent evidence
- spawn and descendant revoke
- a fail-closed halt when the authorizer is unreachable. Fail-closed means the agent stops if the system that approves actions cannot be reached.
How to read this page
If you assign security work for an agent, start with sections 1 through 11.
If you check the model, the data, or the security program around the agent, start with sections 12 through 20.
Sections 21 and 22 name questions the 68 papers do not answer.
This page uses 68 public papers. Forty-one are the original MOSAIC set. MOSAIC means Multi-Organization Secure AI Coordination.
MOSAIC organizations:
- BIML (Berryville Institute of Machine Learning)
- CIS (Center for Internet Security)
- CSA (Cloud Security Alliance)
- CoSAI (Coalition for Secure AI)
- NIST (National Institute of Standards and Technology)
- OWASP (Open Worldwide Application Security Project)
- SANS (the SANS Institute)
In September 2026, eleven papers were added. Some came from those same groups. Others came from ETSI, ENISA, the Frontier Model Forum, OpenSSF, and the AI Verify Foundation.
In November 2026, seventeen papers were added from later CoSAI, CSA, NIST, OpenSSF, OWASP, SANS, FMF, and CISA publications. The SANS Critical AI Security Guidelines v1.2 source code was retired; v1.4 remains as SANS-CAISG14.
Sections 1 through 11 cover agent-specific security: identity, delegation, autonomy, and the Model Context Protocol (MCP). MCP is how an agent finds and calls tools.
Sections 12 through 20 cover supply chain, data, governance, and application security that agent systems sit on. Those rules also apply outside agent systems.
Sections 21 and 22 came from a check against an older security engineering catalog. That catalog includes Saltzer and Schroeder, NIST SP 800-27, ISO 27001, CIS design principles, and Microsoft's immutable laws of security.
Those two sections name a question the older catalog asks and the AI papers do not yet answer. They do not repeat sections 1 through 20.
Read sections 1 through 11 with sections 12 through 20. Agent rules in sections 1 through 11 assume that foundation is already in place.
If you narrow a delegation chain and skip model provenance, you secure only half the stack. Provenance records where an artifact came from and whether someone changed it.
This page groups repeated axioms, invariant rules, and principles by topic.
- An axiom is a claim the sources treat as a starting rule.
- An invariant rule is a condition that must stay true.
- A principle is a design rule that shows up again and again.
One idea is cited once, even when three or four organizations reached it on their own.
Each claim names its sources with a short code. Look up codes in the Source documents table before you see them used.
Each section ends with two notes.
- The OpenCRE crosswalk names OpenCRE topics the section's sources map to. OpenCRE (Open Common Requirement Enumeration) is a shared list of security topics.
- The Gap note checks the section against the older principles catalog and names what that catalog asks that the AI papers do not yet answer.
Three Gap notes, in sections 2, 12, and 18, also state an editorial position. In those notes this page takes a side.
Where sources disagree, or where one organization goes further than the others, this page records the difference and leaves it visible.
Several sections note that a rule first appeared in an older, narrower paper and was later restated in stronger form. Both papers are cited.
The earlier paper is often more specific about the mechanism. The newer paper often states the principle more forcefully.
In March 2026, six claims from seven talks at the first [un]prompted conference were added. They use UNPROMPTED1- codes.
In October 2026, nine claims from seven talks at [un]prompted II were added. They use UNPROMPTED2- codes.
The talks are conference material, not papers, so the count of 68 papers stays the same. Any generalization or Gap note that names the 68 sources describes the papers only.
Paraphrase: a talk citation records our takeaway from that session. It is not a direct quote from the speaker, and it may include our own conclusions. The Claim entry below defines the word.
A research-status note under a gap points at arXiv papers that touch that gap.
- Tested means a lab, a benchmark, or a simulation.
- Theoretical means a proposal, or a survey with no experiment of its own.
- Deployed means the work ran in production or on a real incident.
Each note says the tag comes from the abstract only and the paper is unverified. That limit applies even when you open the page at a single section.
That tag is this page's paraphrase of an abstract. The monitor inventory uses a separate evidence level, defined under Terms.
Terms used on this page
- Agentic system. Software that can plan and take actions. It does more than answer a question.
- Corpus. The 68 papers this page synthesizes. Conference talks are cited separately and are not part of the 68.
- Claim. A short statement this page records and cites with a short code. For a conference talk, the statement is our paraphrased takeaway from the talk. It is not a direct quote from the speaker, and it may include our own conclusions. An identity claim is a different use on this page: it is what an agent says about who it is.
- Source. Two uses. A source of a principle is a paper or a talk that the principle list cites. A monitor record is one row in the whitepaper inventory. Metadata only, duplicate, and marked for removal are labels for other monitor records. Those records are not sources of principles.
- Cited in principles. The agentic catalog lists the paper as a source of a principle on this page.
- Research the monitor found. An arXiv row the monitor stored because a monitored paper cites it. That list is separate from organization papers and is not the default source list.
- Included. On a scope button, the monitor decided to keep that organization paper in the inventory. In a table status line, "in this list" means the rows the current filter shows.
- Evidence level. Two uses. In a research-status note, the tag is this page's paraphrase of an abstract, and the note says the paper is unverified. In the monitor inventory, the evidence level is the tag the monitor stored after an automated read of the abstract or the PDF. The filter definitions paraphrase the monitor's tags. Neither tag is an extracted principle. The monitor tag "tested, possibly deployed (needs review)" is not the deployed tag.
- Non-human identity. An identity for software, such as an agent, not for a person.
- IAM. Identity and access management. The system that decides who or what may access a resource.
- MCP. Model Context Protocol. The protocol an agent uses to find and call tools.
- LLM. Large language model. A model that generates text from a prompt.
- SRF. The CoSAI AI Shared Responsibility Framework. It names one accountable party for each activity.
- Provenance. A record of where an artifact came from and whether it was changed.
- Attestation. A signed statement that a claim about a system or an artifact is true.
- OpenCRE. Open Common Requirement Enumeration. A shared list of security topics. Each section ends with the OpenCRE topics its sources map to.
Source documents
The principles on this page come from the papers the principles catalog cites. That list is the default.
The whitepaper monitor also stores organization papers this page does not cite, and arXiv papers that a monitored paper cites.
Research the monitor found is its own list. It is not part of the default. Its count is separate from the organization-paper count.
In this list, a source of principles means the principles catalog cites that paper. Metadata only, duplicate, and marked for removal label other monitor records. Those records are not sources of principles.
When the selected list contains an arXiv record, the evidence filter opens. Its definitions paraphrase the monitor's tags. An evidence tag is not an extracted principle.
Organization papers and research the monitor found do not overlap. Research the monitor found is not part of the default list. A metadata-only record is not a source of principles. The evidence filter opens when the selected list contains an arXiv record.
| Count | Value |
|---|---|
| Organization papers | 105 |
| Research the monitor found | 400 |
| Organization papers that are metadata only | 3 |
| Research records with extracted principles | 0 |
| Organization arXiv records outside the research count | 2 |
Machine-readable list: /data/whitepaper-inventory.slim.json. The page does not load the full monitor export.
Agent security (sections 1 through 11)
| Title | Organization | Date | Link |
|---|---|---|---|
| Artificial Intelligence (AI) Agents Companion Guide (CIS Controls v8.1) | Center for Internet Security | April 20, 2026 | Link |
| Model Context Protocol (MCP) Companion Guide (CIS Controls v8.1) | Center for Internet Security | April 20, 2026 | Link |
| Agentic AI Identity & Access Management: A New Approach | Cloud Security Alliance | 2026 | Link |
| CoSAI Principles for Secure-by-Design Agentic Systems | Coalition for Secure AI | July 14, 2025 | Link |
| Model Context Protocol (MCP) Security | Coalition for Secure AI | Jan 27, 2026 | Link |
| The Future of Agentic Security: From Chatbots to Autonomous Swarms | Coalition for Secure AI | March 16, 2026 | Link |
| Agentic Identity and Access Management | Coalition for Secure AI | April 17, 2026 | Link |
| CheatSheet: A Practical Guide for Securely Using Third-Party MCP Servers 1.0 | OWASP GenAI Security Project | Nov 4, 2025 | Link |
| OWASP Top 10 for Agentic Applications 2026 | OWASP GenAI Security Project | Dec 9, 2025 | Link |
| A Practical Guide for Secure MCP Server Development | OWASP GenAI Security Project | Feb 16, 2026 | Link |
| State of Agentic AI Security and Governance 2.01 | OWASP GenAI Security Project | June 1, 2026 | Link |
| AI Shared Responsibility Framework (SRF) v1.0 | Coalition for Secure AI | May 26, 2026 | Link |
| Model Context Protocol (MCP) Security V2.0 | Coalition for Secure AI | September 2026 | Link |
| Zero Trust for AI Systems | Coalition for Secure AI | September 2026 | Link |
| Addendum: Post-Quantum Cryptography Considerations for MCP | Coalition for Secure AI | September 2026 | Link |
| Treat Your Agent Like an Insider Threat: Why AI Sandboxing Can't Wait | Coalition for Secure AI | August 25, 2026 | Link |
| MCP Tool Exposure is a Privilege Grant: A Two-Gate Authorization Model for Agentic AI | Cloud Security Alliance | October 15, 2026 | Link |
| Shadow AI Does Not Read Your Org Chart: Rethinking Identity Governance for Autonomous Agents | Cloud Security Alliance | September 3, 2026 | Link |
| Design Tenets for Agent Identity Management Systems | National Institute of Standards and Technology | October 27, 2026 | Link |
| SAF-MCP: Secure Agentic Framework for Model Context Protocol (formerly SAFE-MCP), Framework Model v2 | OpenSSF | 2026 | Link |
Supply chain and general AI security (sections 12 through 20)
| Title | Organization | Date | Link |
|---|---|---|---|
| Security Engineering for Machine Learning (IEEE Computer) | Berryville Institute of Machine Learning | Aug 2019 | Link |
| An Architectural Risk Analysis of Machine Learning Systems (BIML-78) | Berryville Institute of Machine Learning | Jan 20, 2020 | Link |
| An Architectural Risk Analysis of Large Language Models (BIML-LLM24) | Berryville Institute of Machine Learning | Jan 24, 2024 | Link |
| 23 Security Risks in Black-Box Large Language Model Foundation Models (IEEE Computer) | Berryville Institute of Machine Learning | April 2024 | Link |
| No Security Meter for AI | Berryville Institute of Machine Learning | May 13, 2026 | Link |
| Artificial Intelligence and Large Language Models (LLM) Companion Guide (CIS Controls v8.1) | Center for Internet Security | April 20, 2026 | Link |
| AI Controls Matrix (AICM) v1 | Cloud Security Alliance | July 10, 2025 | Link |
| AI Controls Matrix (AICM) v1.1 | Cloud Security Alliance | June 22, 2026 | Link |
| The State of Cloud and AI Security 2025 | Cloud Security Alliance | 2025 | Link |
| Data Security within AI Environments | Cloud Security Alliance | 2025/2026 | Link |
| AI Vulnerability: Security Program Guide for CISOs | Cloud Security Alliance | Sept/Oct 2025 | Link |
| Establish Risks and Controls for the AI Supply Chain | Coalition for Secure AI | 2024 (v1) | Link |
| Signing ML Artifacts: Building Towards Tamper-Proof ML Metadata Records | Coalition for Secure AI | Sept 29, 2025 | Link |
| Preparing Defenders of AI Systems | Coalition for Secure AI | 2024 | Link |
| AI Incident Response Framework | Coalition for Secure AI | Oct 30, 2025 | Link |
| AI Risk Management Framework (AI RMF 1.0) | National Institute of Standards and Technology | Jan 26, 2023 | Link |
| AI RMF Playbook | National Institute of Standards and Technology | March 30, 2023 | Link |
| AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1) | National Institute of Standards and Technology | July 26, 2024 | Link |
| Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2) | National Institute of Standards and Technology | 2024, updated 2025 | Link |
| AI Exchange (owaspai.org): AI Security Overview & Controls Guide | OWASP AI Exchange | Living document, 2022-2026 | Link |
| OWASP Top 10 for LLM Applications & Generative AI (2025) | OWASP GenAI Security Project | 2025 | Link |
| GenAI Incident Response Guide 1.0 | OWASP GenAI Security Project | July 28, 2025 | Link |
| Solutions Reference Guide Q2/Q3'25 | OWASP GenAI Security Project | Nov 4, 2025 | Link |
| Vendor Evaluation Criteria for AI Red Teaming Providers & Tooling v1.0 | OWASP GenAI Security Project | Feb 4, 2026 | Link |
| OWASP GenAI Data Security Risks & Mitigations 2026 | OWASP GenAI Security Project | March 17, 2026 | Link |
| Critical AI Security Guidelines v1.0 | SANS Institute | March 31, 2025 | Link |
| Critical AI Security Guidelines v1.1 (draft) | SANS Institute | April 2025 | Link |
| Own AI Securely: The SANS Secure AI Blueprint | SANS Institute | 2025 | Link |
| Critical AI Security Guidelines v1.4 | SANS Institute | March 2026 | Link |
| SANS AI Security Maturity Model eBook | SANS Institute | May 11, 2026 | Link |
| OWASP GenAI LLM Top 10 2026 | OWASP GenAI Security Project | August 3, 2026 | Link |
| OWASP Agentic Skills Top 10 (AST10) | OWASP GenAI Security Project | 2026 | Link |
| Secure Software Development Practices for Generative AI and Dual-Use Foundation Models: An SSDF Community Profile (SP 800-218A) | National Institute of Standards and Technology | July 2024 | Link |
| AIMM: Artifact Integrity Maturity Model for AI/ML Supply Chain Trust and Provenance | Coalition for Secure AI | September 7, 2026 | Link |
| Securing Agentic AI with Zero Trust Microsegmentation | Cloud Security Alliance | September 29, 2026 | Link |
| Zero Trust Microsegmentation Guidance | Cloud Security Alliance | September 8, 2026 | Link |
| OWASP GenAI Data Security Best Practices v2, Part 1 | OWASP GenAI Security Project | October 2026 | Link |
| OWASP GenAI Red Teaming Manual Version 1.0 | OWASP GenAI Security Project | October 2026 | Link |
| Digital Forensics & AI Investigations Framework | SANS Institute | 2026 | Link |
| Incident Response & AI Investigations Framework | SANS Institute | 2026 | Link |
| Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation | CISA | October 2026 | Link |
Peer standards development organizations (SDOs) and coalitions
| Title | Organization | Date | Link |
|---|---|---|---|
| ETSI EN 304 223 V2.1.1: Baseline Cyber Security Requirements for AI Models and Systems | ETSI | December 2025 | Link |
| ENISA's view on Cybersecurity in the Frontier AI Era | ENISA | July 2026 | Link |
| Emerging Security Practices for AI Agents | Frontier Model Forum | June 3, 2026 | Link |
| Managing Advanced Cyber Risks in Frontier AI Frameworks | Frontier Model Forum | February 13, 2026 | Link |
| Security-Focused Guide for AI Code Assistant Instructions | OpenSSF | August 1, 2025 | Link |
| Model AI Governance Framework for Agentic AI v1.5 | AI Verify Foundation | May 20, 2026 | Link |
| Agents for Cyber Defense | Frontier Model Forum | 2026 | Link |
Conference talks
The Claims column lists paraphrased takeaways we drew from each talk. The text repeats the principle statements on this page that cite the talk. These are not direct quotes from the speaker, and they may include our own conclusions. If no statement cites the talk, the cell says Not yet recorded. On a narrow screen, the claim text stays closed until you open it.
| Code | Organization | Date | Link | Title | Claims |
|---|---|---|---|---|---|
| UNPROMPTED1-RYCIAK | [un]prompted | March 2026 | Link | Vibe Check: Security Failures in AI-Assisted IDEs (Piotr Ryciak) | Show the claimHide the claim
|
| UNPROMPTED1-MCCARTHY | [un]prompted | March 2026 | Link | Zeal of the Convert: Taming Shai-Hulud with AI (Rami McCarthy) | Show the claimHide the claim
|
| UNPROMPTED1-NABEEL | [un]prompted | March 2026 | Link | Detecting GenAI Threats at Scale with YARA-Like Semantic Rules (Mohamed Nabeel) | Show the claimHide the claim
|
| UNPROMPTED1-EPP | [un]prompted | March 2026 | Link | 8 Minutes to Admin. We Caught It in the Wild. Welcome to VibeHacking (Sergej Epp) | Show the claimHide the claim
|
| UNPROMPTED1-RITTINGHOUSE-HUANG | [un]prompted | March 2026 | Link | 1.8M Prompts, 30 Alerts: Hunting Abuse in a User-Defined Agent Ecosystem (Matt Rittinghouse, Millie Huang) | Show the claimHide the claim
|
| UNPROMPTED1-ISAK-GILL | [un]prompted | March 2026 | Link | Developing & Deploying AI Fingerprints for Advanced Threat Detection (Natalie Isak, Waris Gill) | Show the claimHide the claim
|
| UNPROMPTED1-POLLEY | [un]prompted | March 2026 | Link | Training BrowseSafe: Lessons from Detecting Prompt Injection in Production Browser Agents (Kyle Polley) | Show the claimHide the claim
|
| UNPROMPTED2-EVRON | [un]prompted II | October 2026 | Link | Opening Words Wednesday (Gadi Evron) | Show the claimHide the claim
|
| UNPROMPTED2-BARANOVSKYI | [un]prompted II | October 2026 | Link | Who Authorized the Agent Making the AI Harness an Enforceable Compliance Boundary (Oleksii Baranovskyi) | Show the claimsHide the claims
|
| UNPROMPTED2-NAJEEB | [un]prompted II | October 2026 | Link | Metrics for Agentic Systems What Good Looks Like for a Security Agent (Shahanas Najeeb) | Show the claimHide the claim
|
| UNPROMPTED2-SEVERI-YANG | [un]prompted II | October 2026 | Link | Memory Is the New Attack Surface Benchmarking Poisoning Through Agent Memory (Giorgio Severi, David Yang) | Show the claimHide the claim
|
| UNPROMPTED2-BROSSARD-AUMASSON | [un]prompted II | October 2026 | Link | Hacking Truth Forged Lean Proofs and the Risk of Verifier Bugs (Jonathan Brossard, Jean-Philippe Aumasson) | Show the claimHide the claim
|
| UNPROMPTED2-NEJAD | [un]prompted II | October 2026 | Link | Opened Is Not Fixed The Last Mile of AI Vulnerability Remediation (Shayaun Nejad) | Show the claimsHide the claims
|
| UNPROMPTED2-CHENNABASAPPA-PUDERBAUGH | [un]prompted II | October 2026 | Link | Permission Fatigue Is a Security Bug Dynamic Guardrails for Coding Agents (Sahana Chennabasappa, Chris Puderbaugh) | Show the claimHide the claim
|
1. Give each agent its own identity
Give each agent its own identity. Do not treat it as a kind of human user, and do not put it on a shared service account.
CSA-IAM states this most directly. An agent is a first-class identity, separate from a human user and from a traditional service account. The identity has to record what the agent can do, its provenance, and its security posture. A name the agent claims is not enough.
CSA-IAM also says there is no implicit or default trust between autonomous agents. "It is an internal agent" is not a reason to trust it.
COSAI-IAM treats a shared account for an agent as a failure of accountability. It treats the fix as an extension of the identity and access management (IAM) system you already run for people and service accounts. That identity plane is the starting point. You do not need a second, separate system.
OWASP-TOP10 names what you get if you skip this. Most current architectures still have no distinct, governed agent identity. OWASP-TOP10 calls that the attribution gap. It treats the gap as a structural problem across the industry, not as a mistake in one deployment.
OWASP-STATE adds the scale. Non-human identities already outnumber human identities in production, and the gap keeps widening. The attribution gap is already the majority-case identity plane in most environments.
CIS-AGENTS and CIS-MCP treat this as lifecycle management. Issue, rotate, and retire a non-human identity with the same discipline you use for a human credential. An agent identity, once created, does not need less ongoing governance than a human one.
- Every agent needs its own distinct, verifiable identity, never a shared account or reused human credential (CSA-IAM, COSAI-IAM, OWASP-MCP-DEV, OWASP-STATE, NIST-AGENT-ID, CSA-SHADOW)
- Identity claims must be cryptographically verifiable and attestable (CSA-IAM, OWASP-TOP10, OWASP-STATE, NIST-AGENT-ID)
- Credentials must be short-lived and task-scoped rather than standing (COSAI-IAM, OWASP-TOP10, OWASP-STATE, NIST-AGENT-ID, CSA-SHADOW)
- Non-human identities require the same lifecycle management (issuance, rotation, retirement) as any other governed credential (CIS-AGENTS, CIS-MCP, OWASP-MCP-DEV)
- No implicit or default trust exists between agents simply because both are internal to the same organization (CSA-IAM)
- Agentic IAM extends existing identity infrastructure rather than requiring a parallel, separate system (COSAI-IAM)
- Each agent instance receives its own short-lived workload identity rather than a shared static key reused across instances (CSA-SHADOW)
- The task originator signs a mandate that states the intended outcome and limits for the agent's actions (NIST-AGENT-ID)
- How quickly an organization can halt an agent once it starts behaving unexpectedly is a measured control, not an informal hope (CSA-SHADOW)
OpenCRE crosswalk: Access Control, Authentication, Identity Management, Secrets Management, Session Management.
Gap: None of the 68 sources require you to record the approver's authenticator strength with the agent identity that person created (Microsoft's Immutable Law 5, weak passwords). Identity standards outside these papers define that strength: NIST SP 800-63B and FIDO2 (unverified). FIDO2 is a standard for authenticators that resist phishing. A strong agent credential can still be created by a person who uses a weak or reused password.
Research status: arXiv work covers part of this gap. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2608.04292 is tagged tested (a lab prototype). arXiv:2501.09674 is tagged theoretical (a proposal only).
2. Do not leave access standing
Least privilege appears in every source in this set. Least privilege means you grant only the access the task needs. The agent version goes further, and the sources repeat the same extra step. They reject static privilege that you grant once and leave in place.
COSAI-IAM names this Zero Standing Privilege. It pairs the name with a capability-risk classification. Scale the controls to what a given agent can actually do and to how sensitive the data is. Do not apply one policy to every agent and ignore its blast radius. Blast radius is how much damage that agent can do if it is misused.
CSA-IAM requires you to re-verify access continuously. Do not grant it permanently. The control is fine-grained, dynamic, and based on attributes and policy. It is not the role-based model most human IAM systems still use. Attribute-based access control decides from facts about the agent, the data, and the request.
OWASP-TOP10 carries the same idea from access into behavior. It coins "Least-Agency" for the limit on what an agent may decide and do, not only on what it can technically reach. An agent with narrow data access and broad freedom to decide is still under-constrained.
COSAI-FUTURE applies the same logic to the reasoning layer. Least privilege has to reach the agent's cognitive process. It does not stop at network and file permissions. Restore elevated context only through explicit just-in-time provisioning. Just-in-time means you grant the access at the moment of the task. Do not grant it up front.
OWASP-MCP-CHEAT applies the same rule at the MCP server boundary. Require just-in-time access with automatic expiration. Do not leave a standing grant that sits unused most of the time.
- Access is continuously re-verified against current context, not granted once and left standing (CSA-IAM, OWASP-STATE, COSAI-ZT)
- Credentials and permissions are short-lived, scoped, and automatically revocable (COSAI-IAM, OWASP-TOP10, OWASP-STATE, OWASP-MCP-CHEAT, COSAI-ZT)
- Least privilege extends to the agent's decision-making layer, not only its infrastructure access (COSAI-FUTURE, OWASP-TOP10)
- Controls scale with an agent's actual capability and the sensitivity of what it touches, rather than applying uniformly to every agent (COSAI-IAM)
- Access control uses fine-grained, attribute- and policy-based models rather than coarse role-based grants (CSA-IAM)
- An agent's delegated permissions are a subset of the initiating user's permissions, so the agent cannot reach data or systems the user cannot (COSAI-ZT)
- Every MCP tool call, RAG lookup, and data or API access is a separate authorization event checked against a policy decision point with the user plus agent identity (COSAI-ZT)
- When the decision point is silent, missing, or lacks a required identity or telemetry signal, the call is denied rather than allowed (OPENSSF-SAF-MCP, CSA-ZT-SEG, COSAI-ZT)
- Zero Trust does not stop misaligned reasoning: an agent can still take any unintended action the user is permitted to take (COSAI-ZT)
- When a policy blocks an agent, return an explanation so the agent can adapt instead of looping or working around it (UNPROMPTED2-BARANOVSKYI)
- When a user denies trust, anything already started must actually stop; a denial that leaves processes running is not a control (UNPROMPTED1-RYCIAK)
- Each agent action must record three things and pass them to the next agent:
- who first made the request,
- when that person's approval runs out, and
- how much the agent may do on its own.
OpenCRE crosswalk: Access Control, Authentication, Authorization, Configuration Management.
Gap: COSAI-ZT, OPENSSF-SAF-MCP, and CSA-ZT-SEG require deny when the decision point is silent or a required identity or telemetry signal is missing. NIST-GAI still says a system can fail safely. None of the sources say what to do when the policy decision point returns an ambiguous or conflicting answer rather than silence or a clean deny (Saltzer and Schroeder's fail-safe defaults still apply to that case).
Research status: Tested arXiv work addresses a fail-closed default. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2606.28679 is tagged tested (a lab proof of concept). arXiv:2606.26057 is tagged theoretical (a proposal only).
Gap and editorial position: None of the sources require agent actions to carry these three facts. None require the facts to survive when a request moves between agents run by different organizations. We think they should.
Databases solved a version of this long ago. They can undo, redo, and replay work because every logged change records its transaction details. When agent chains lack those details, here is what goes wrong:
- You can't undo. No one can tell which later actions went past the first limit.
- You can't redo. After an outage, no one knows which pending actions were still approved.
- You can't replay. An investigator can't compare the chain to what the person actually approved.
- Power grows with each step. Each agent sees only the one before it, so it inherits that agent's full access instead of the person's narrower intent.
- Approvals outlive their owner. A request can keep acting at a partner company after the person who approved it cancels it or leaves.
- Old requests can come back. A request queued, retried, scheduled, or saved in agent memory can run weeks or months later. If it carries no expiry, nothing stops it, and no one can trace it to the person who asked.
- No one holds the full story. Each company keeps part of the log, and nothing ties the parts together.
Some tools exist outside the AI sources. OAuth token exchange (RFC 8693) can record a chain of agents acting for someone. OpenID Connect can record how the approver signed in. Neither sets a limit on what an agent may do alone, and the AI guidance requires neither.
3. Narrow authority at every handoff
Once one agent calls another, or acts for a user, the sources agree on one rule. Authority can only narrow as it passes down the chain. Every hop has to enforce that on its own. A hop is one step from one agent to the next. Do not trust that an earlier check already covered this hop.
COSAI-IAM states this as an invariant. Delegation scope never expands at any hop. Revoke a delegation anywhere in the chain and everything downstream of it becomes invalid. One revocation can cut off a whole multi-agent workflow. You do not need a separate revocation at every node.
COSAI-IAM calls the full model "delegation with full lineage." On-behalf-of tokens carry the agent's identity and the originating user's identity together. Narrow the scope at each hop. Do not pass the same token unchanged.
OWASP-MCP-DEV names the failure this design prevents. Never pass a raw upstream token to a downstream API. That recreates the confused deputy problem. A confused deputy is a service that acts with more authority than the caller has, because it cannot tell a correctly scoped request from an inherited, overly broad one.
COSAI-FUTURE requires that a downstream call one agent makes for another stay traceable to the exact thread or event that authorized it. It also requires that a retrieval agent never query as a super-admin. Query scope must stay inside the invoking identity's actual entitlements. Do not use the retrieval agent's own broad access for convenience.
OWASP-STATE states the same requirement from the audit side. Delegated authority must stay traceable to the originating human principal at every step. A principal is the person or identity the action is for. A downstream action has to trace back to a specific person's original request. It must not dead-end at an intermediate agent.
- Delegated authority narrows at every hop and never expands (COSAI-IAM, COSAI-MCP, COSAI-MCP2)
- Raw upstream tokens are never forwarded downstream; only newly scoped credentials pass along the chain (COSAI-IAM, OWASP-MCP-DEV, COSAI-MCP2)
- Each hop enforces authorization independently rather than assuming a perimeter check elsewhere already covered it (COSAI-IAM, OWASP-MCP-DEV)
- Delegation chains stay traceable back to the human principal that originated them (OWASP-STATE, COSAI-FUTURE)
- Revoking a delegation at any point in a chain automatically invalidates every downstream delegation built on it (COSAI-IAM)
- A retrieval or query agent never operates with broader entitlements than the identity that invoked it, regardless of the agent's own standing access (COSAI-FUTURE)
- An authoring agent never self-approves its own pull request or bypasses CI/CD to deploy a unilateral change; write access stops at the declarative repository, with production deployment enforced through immutable GitOps rather than direct agent write access to the runtime (COSAI-FUTURE)
OpenCRE crosswalk: Access Control, Authentication, Session Management, Authorization.
Gap: COSAI-IAM names agent collusion. None of the sources say how to test two correctly scoped agents for combined leakage (NIST SP 800-27's principle of addressing overlapping information domains). Research shows the combined leak happens (arXiv:2509.14284, unverified).
Research status: Tested arXiv work shows combined leakage across agents. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2509.14284, arXiv:2603.05520, and arXiv:2602.11510 are tagged tested (lab, benchmark, or simulation).
4. Destroy the run when it ends
Several sources treat the lifetime of an agent's execution environment as a control of its own. It is not only a cost choice.
COSAI-FUTURE calls this Zero-State Initialization. Every autonomous loop starts from a pristine image that has been checked cryptographically, and the environment is destroyed immediately after it runs. Nothing persists between executions for an attacker to re-enter. A compromise from one run cannot carry into the next.
COSAI-FUTURE pairs that with "guaranteed teardown" as its own operational requirement. Destruction of the execution environment is a property the platform has to enforce. It is not best-effort cleanup.
OWASP-TOP10 and OWASP-MCP-CHEAT both require just-in-time access that expires automatically. The access does not persist past the task that needed it. The same short lifetime applies to the credentials those environments use while they are alive.
The logic connects to Treat what the agent reads as untrusted. No source in this set of papers claims prompt injection can be fully prevented. Prompt injection means an attacker hides an instruction in content the model is supposed to read. Ephemeral execution is a containment strategy for an attack you cannot stop at the input. Ephemeral means the environment exists only for that run. If you cannot reliably filter the injected instruction before it reaches the model, limit how long a compromised execution context can persist, and require that it be destroyed instead of reused.
- Autonomous execution loops start from a verified image and are destroyed immediately after each run (COSAI-FUTURE)
- Access grants expire automatically instead of persisting past the task that required them (OWASP-TOP10, OWASP-MCP-CHEAT)
- Environment teardown is a guaranteed platform property, not best-effort cleanup (COSAI-FUTURE)
- Ephemeral execution functions as a containment strategy for attacks that can't be reliably filtered at the input layer (COSAI-FUTURE, cross-referenced with OWASP-STATE)
- Network egress from an autonomous execution environment defaults to deny, with swarm topology bounded rather than left open-ended, pairing network-level containment with the compute-level containment ephemerality already provides (COSAI-FUTURE)
- Isolation is enforced independently of the agent, because an agent that can reason about its sandbox can reason around it (COSAI-SANDBOX)
- Every permitted channel, including the LLM inference endpoint, is treated as a potential exfiltration path (COSAI-SANDBOX)
- Before an agent harness starts, a launch gate verifies that all required controls are in place and working, and agents run under compiled default policies (UNPROMPTED2-BARANOVSKYI)
OpenCRE crosswalk: Secure Product Design, Configuration Management, Access Control.
Gap: COSAI-FUTURE and COSAI-MCP require default-deny egress. Egress is traffic leaving the environment. None of the sources tie an ephemeral agent's allowlist to how critical the backend systems are that it can still reach (NIST SP 800-27's principle of isolating public-access systems from mission-critical resources). An environment that is destroyed after one run can still damage a critical system it was allowed to reach during that run.
Research status: Deployed arXiv work bounds which systems an agent can reach. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2607.05743 is tagged theoretical (a survey, with no experiment of its own). arXiv:2603.17419 is tagged deployed (nine production agents).
5. Keep a person able to stop the agent
Every source keeps a human override somewhere in the loop. None of them apply oversight the same way to every agent. Oversight scales with how autonomous the agent is.
COSAI-SRF is the most precise source on this point. It is the only source in the 68 papers that ties oversight to a numbered autonomy scale. It requires immediate human intervention regardless of autonomy level. It requires formal documentation and a quarterly review once an agent reaches autonomy level L3. It requires a contractual accountability clause from the provider once an agent reaches full autonomy at L5.
Its autonomy scale is adapted from SAE J3016, the Society of Automotive Engineers standard used to classify self-driving vehicle autonomy. COSAI-SRF pairs that scale with a separate scale of Human Override Tiers. The tiers run from automated soft guidance up to an emergency shutdown authorized by the chief information security officer (CISO) or another executive. The papers do more than say a kill switch must exist. A kill switch is a control that stops the agent now. The scale says who is authorized to use it at each severity.
COSAI-SBD requires agentic systems to stay under meaningful human control. Tie the degree of oversight to the risk of the business outcome. Do not fix it in advance. Systems must be continuously validated against their intended purpose and their predefined failure modes. Do not check once at launch and stop.
OWASP-TOP10 and OWASP-STATE both require human approval before any high-impact, irreversible, or goal-changing action. Both require a kill switch or circuit breaker that can halt a compromised agent. A circuit breaker stops a repeating failure. OWASP-STATE adds a budget limit for fully autonomous agents. It treats a spending or resource cap as its own emergency brake, distinct from an outright shutdown.
COSAI-MCP requires human-in-the-loop consent as a fail-safe default for actions that go through MCP. That places the same rule at the protocol layer, not only at the application layer.
AIVERIFY-MAGF states the scaling limit directly. Continuous human approval of every agent step does not scale, so put checkpoints on high-stakes and irreversible actions. ETSI-304223 Principle 4 requires that human responsibility for AI systems stay enabled across the lifecycle. That is the same override authority COSAI-SRF assigns by name.
- Human intervention capability exists regardless of an agent's autonomy level (COSAI-SRF)
- Human approval is required before high-impact, irreversible, or goal-changing actions (OWASP-TOP10, OWASP-STATE, COSAI-MCP, OWASP-DATASEC2, SANS-IR-AI)
- A kill-switch, circuit-breaker, or budget limit exists for fully autonomous agents (OWASP-STATE, CISA-SI-IR)
- Oversight requirements scale with autonomy level and business risk; checkpoints concentrate on high-stakes and irreversible actions, since continuous approval of every agent step does not scale (COSAI-SRF, COSAI-SBD, AIVERIFY-MAGF)
- Authority to invoke each tier of human override is explicitly assigned, up to CISO- or executive-level authorization for emergency shutdown (COSAI-SRF, ETSI-304223, SANS-DFIR, SANS-IR-AI)
- The human examiner remains the final authority for forensic conclusions, attribution, and legally sensitive sign-off; AI does not own those conclusions (SANS-DFIR, SANS-IR-AI)
- Systems are continuously validated against their intended purpose and predefined failure modes, not checked once at launch and left alone (COSAI-SBD)
- Putting a human in the loop is not by itself a meaningful control at agent speed and scale (UNPROMPTED2-EVRON)
OpenCRE crosswalk: Access Control, Governance Risk and Compliance, Log Management.
Gap: COSAI-MCP names approval fatigue. None of the sources weigh what a human approval gate costs you in decision delay against the risk it prevents (NIST SP 800-27's principle of identifying trade-offs between risk reduction and operational effectiveness). An escalation tier that looks sound on paper and is too slow to use in a live incident gets bypassed in practice. That is the same failure named in A control people bypass does not work. This section's oversight tiers are not yet connected back to that failure.
Research status: Tested arXiv work measures oversight cost and approval fatigue. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2606.08919, arXiv:2603.24582, arXiv:2604.04918, and arXiv:2610.22987 are tagged tested (lab, benchmark, or simulation).
6. Log the decision, and protect the log
In these papers, logging is a security control. It is not an operational extra. Several sources require the log itself to be tamper-evident. A log that merely exists is not enough.
OWASP-TOP10 requires immutable, tamper-evident logging of agent actions, tool calls, and messages between agents. It also requires observability into agent goals and decision paths. A log of actions, with no record of the reasoning behind them, is forensically incomplete.
COSAI-IAM requires the chain of authority behind a delegated action to be preserved in immutable logs. That connects to Narrow authority at every handoff. The log is the record of what happened. It is also the mechanism that lets you enforce the narrowing rule after the fact.
CIS-AGENTS and CIS-MCP extend existing CIS Controls v8.1 logging requirements to agent actions. They do not write a new control set. They keep the logs bounded and auditable. Across the CIS material, agentic AI is a new surface for old controls. It is not a domain that needs its own control catalog from scratch.
COSAI-SBD goes further than the rest. It requires telemetry across the full decision path: inputs, plans, decisions, communications, and outputs together, not only the final action. An auditor can reconstruct what an agent did, and also what it considered and rejected along the way.
OWASP-STATE frames the goal as parity with existing software supply chain practice. Keep execution provenance as traceable as a static software inventory. That bar is higher than what most organizations currently hold their AI systems to. A static inventory is checked on a schedule. Execution provenance has to be captured continuously.
AIVERIFY-MAGF names the components that inventory has to cover for an agent: model, instructions, memory, planning, tools, protocols, access controls, guardrails, approvals, and logging.
- Logging of agent actions, tool calls, and inter-agent messages is immutable and tamper-evident (OWASP-TOP10, COSAI-IAM, NIST-AGENT-ID, CISA-SI-IR)
- Activity logs that the agent itself can reach or change cannot serve as reliable evidence (NIST-AGENT-ID)
- Agents must not be able to access or interfere with the monitoring processes and logs that watch them (CISA-SI-IR)
- Monitored model output is treated as possibly containing exploits aimed at the monitors (CISA-SI-IR)
- Telemetry covers the full decision path, not just the final action taken (COSAI-SBD)
- Execution provenance is kept as traceable as a static software inventory (OWASP-STATE)
- Observability extends to agent goals and decision paths, not only the actions ultimately taken (OWASP-TOP10)
- Existing CIS Controls v8.1 logging requirements are extended to agent actions rather than replaced with a new control set (CIS-AGENTS, CIS-MCP)
- An agent's security-relevant components, meaning model, instructions, memory, planning, tools, protocols, access controls, guardrails, approvals, and logging, are each in scope for inventory and controls (AIVERIFY-MAGF)
- Run new AI detection rules in shadow production for a period before enforcing, because real feeds behave differently (UNPROMPTED1-NABEEL)
- Deception such as honey tokens favors defenders against AI-driven attackers and is underused (UNPROMPTED1-EPP)
OpenCRE crosswalk: Log Management, Error Handling, Governance Risk and Compliance.
Gap: Every source here treats logging as a purely technical control. None discuss whether the people expected to review an audit trail are trained to recognize what they are looking at (NIST SP 800-27's requirement that developers, and by extension operators, be trained to work securely). An immutable log that nobody knows how to read produces the same outcome as no log at all.
Research status: No arXiv work was found. This gap is open in the 68 sources and open on arXiv.
7. Treat what the agent reads as untrusted
This is the one place where the sources admit an open, unsolved problem. They do not offer a settled control.
COSAI-FUTURE states it plainly. Proving that a large language model (LLM) will never execute an injected instruction hidden in untrusted content is currently considered theoretically impossible. It adds a point about visibility. Being able to read a chat log is not the same as having cryptographic control over code integrity. Monitoring an agent's conversation is not a substitute for controlling what it is allowed to execute.
OWASP-STATE calls prompt injection an unsolved architectural flaw. An LLM merges its data plane and its control plane into a single token stream. The same channel that carries the content the model is supposed to process also carries anything an attacker smuggles in as an instruction.
OWASP-TOP10 adds that agents and models cannot reliably tell instructions apart from the untrusted content around them. It frames this as amplification of an existing problem. Agents mostly amplify existing LLM vulnerabilities by giving a model with this flaw the ability to act on the world. They do not introduce an entirely new class of vulnerability. OWASP-LLMTOP10-2026 splits the same idea across two lists. Model-as-component risks stay on the LLM Top 10. Model-as-actor risks pair with the Agentic Top 10.
The flaw resists a permanent fix, so the sources agree on a different rule. Treat everything an agent reads as untrusted input, and validate it at every boundary it crosses. That includes tool descriptions, retrieved documents, and other agents' output. Do not trust it once and pass it along.
COSAI-MCP and OWASP-MCP-DEV both flag tool descriptions and metadata as untrusted unless they come from an already-trusted server. That closes a specific hole. A tool's own self-description is content an attacker can poison. If you trust it by default because it comes from the tool's own manifest, you defeat the broader rule.
OWASP-STATE's "lethal trifecta" and "Rule of Two" give this problem its most concrete operational form. An agent that has private data access, is exposed to untrusted content, and has an outbound communication channel, all three at once, is treated as exploitable from input to output, no matter how well any single piece is defended. The Rule of Two caps an agent at no more than two of the three properties unless a person approves the step.
FMF-AGENTS restates the same trifecta and adds the architectural consequence. Enforce tool, filesystem, egress, and credential limits in the harness, outside the reasoning loop. A harness is the software around the model that starts the agent, applies policy, and calls tools. Probabilistic model defenses cannot carry that load. FMF-AGENTS frames those harness limits as one layer in a four-layer split: model, system guardrails, harness orchestration, and tool access. Each layer has its own controls. Tool output that re-enters the reasoning loop, and content written to persistent memory, sit in the same untrusted class as tool descriptions.
- All tool descriptions, retrieved content, inter-agent messages, tool outputs that re-enter the reasoning loop, and content written to persistent memory are treated as untrusted input, never trusted by default (COSAI-MCP, OWASP-MCP-DEV, OWASP-TOP10, FMF-AGENTS)
- Input is validated at every trust boundary it crosses, not only at the outermost one (COSAI-MCP)
- The "Lethal Trifecta," meaning private data access plus exposure to untrusted content plus outbound communication, is treated as exploitable end to end; the "Rule of Two" caps an agent at satisfying no more than two of the three without human approval (OWASP-STATE, FMF-AGENTS)
- Visibility into an agent's conversation or chat log is not equivalent to cryptographic control over what it can execute (COSAI-FUTURE)
- Agents are treated as amplifying existing LLM-level vulnerabilities rather than introducing an entirely new vulnerability class; model-as-component risks stay on the LLM Top 10, and model-as-actor risks pair with the Agentic Top 10 (OWASP-TOP10, OWASP-LLMTOP10-2026)
- Tool, filesystem, egress, and credential limits are enforced in the harness, outside the model's reasoning loop; probabilistic model defenses are not treated as the control (FMF-AGENTS, OPENSSF-SAF-MCP, COSAI-ZT)
- System and developer prompts are never used as a secret store for keys, credentials, or hidden rules (OWASP-RT-MANUAL)
- Agent security spans the model, system guardrails, harness orchestration, and tool access; each layer has its own controls (FMF-AGENTS)
- Agent memory is an attack surface: test over long trajectories after injection, and inspect memory writes and reads alongside actions (UNPROMPTED2-SEVERI-YANG)
- Using the latest, most robust model for an agent is itself a layer of prompt injection defense (UNPROMPTED1-POLLEY)
OpenCRE crosswalk: Input Validation, Injection Prevention, Data Protection, Threat Modeling.
Gap: No source claims prompt injection can be fully prevented, so the practical question moves from "how do we stop this" to "where do we spend limited defensive effort first." None of the 68 papers rank injection vectors by how much attention they deserve (CIS's design principle of focus: identify the most critical actions against the most important attacks, and do not try to solve everything at once).
Research status: arXiv work measures injection vectors. It does not rank them for where to spend defense first, so this gap stays open on that point. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2506.09956 is tagged tested (a public challenge). arXiv:2605.24309 is tagged theoretical (a survey, with no experiment of its own).
8. Secure the Model Context Protocol
Five of the sources address the Model Context Protocol (MCP) by name. MCP has become the default way agents discover and invoke tools. COSAI-MCP and CIS-MCP both treat MCP as a distinct third layer, separate from the model and the agent, with its own risk profile.
COSAI-MCP is explicit that existing risk frameworks were built on an assumption of predictable component behavior and do not fully address MCP's failure modes. It names MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems), the NIST AI Risk Management Framework (AI RMF), and MAESTRO (Multi-Agent Environment, Security, Threat, Risk, and Outcome).
It cites three real incidents, at Asana, WordPress AI Engine, and Supabase, as evidence that MCP threats are active. It states that containers alone are not a strong security boundary for MCP execution. A container isolates the process. It does not isolate the trust decisions the process makes for the model.
OWASP-MCP-DEV requires OAuth 2.1 or OpenID Connect (OIDC) for every remote MCP server. Validate issuer, audience, expiry, and signature on every request. It forbids passing a token through to a downstream API unchanged. That is the confused-deputy rule from Narrow authority at every handoff, applied to the MCP transport.
It adds MCP hygiene the other papers do not cover. Fully isolate users, sessions, and execution contexts. Keep no global or shared state for user data. Never expose secrets to the LLM itself. Error responses returned to the model must never include stack traces, tokens, file paths, or other tool internals. A verbose error is itself a disclosure channel an attacker can use to map the system.
OWASP-MCP-CHEAT adds a governance layer. A security reviewer and a domain owner both have to sign off before a third-party server is enabled. Pin approved versions and verify the checksum so a silent upgrade cannot slip in. Treat an MCP server the way you would treat any other third-party dependency that can execute code.
- Every capability or tool is granted individually and explicitly, never through a broad or opaque grant (CIS-MCP, OPENSSF-SAF-MCP, CSA-MCP-GATE)
- Remote MCP servers require OAuth 2.1/OIDC with per-request validation of issuer, audience, expiry, and signature (OWASP-MCP-DEV)
- A security reviewer and a domain owner both sign off before a third-party MCP server is enabled (OWASP-MCP-CHEAT)
- Containers alone are not treated as a sufficient security boundary for MCP execution (COSAI-MCP, COSAI-MCP2, COSAI-SANDBOX)
- User sessions and execution contexts are fully isolated, with no shared global state across users (OWASP-MCP-DEV)
- Error responses returned to the model never include stack traces, tokens, file paths, or other internal details (OWASP-MCP-DEV)
- Real, documented MCP incidents are cited as evidence the threat is active, not a hypothetical worst case (COSAI-MCP, COSAI-MCP2)
- Discovering a capability and being authorized to use it are different decisions; discovery filtering is risk reduction, not authorization (CSA-MCP-GATE)
- A tool call is a request, not a command; a deterministic enforcement point outside the model decides whether this agent may use this tool on this resource now (CSA-MCP-GATE)
- The authorization artifact for a privileged tool call names the specific tool, its arguments or their digest, the requesting session, and a short expiry (OPENSSF-SAF-MCP)
- Per-call capability brokering issues a single-use token scoped to that call's declared filesystem, network, and syscall manifest (OPENSSF-SAF-MCP)
- MCP tool and server authors do not write cryptographic code; TLS libraries, proxies, and managed platforms supply it, and hybrid PQ TLS terminates at a reverse proxy, sidecar, or gateway (COSAI-MCP-PQC)
- Operators re-check MCP server and tool definitions after each change in testing and production, to catch a rug pull where a trusted tool turns malicious (OWASP-RT-MANUAL)
OpenCRE crosswalk: Access Control, Authentication, Input Validation, Secure Communication, Supply Chain, Log Management.
Gap: COSAI-MCP-PQC requires that MCP authors not write cryptographic code and that hybrid PQ TLS terminate at a proxy or gateway. CSA-DATA still names a vault with keys in a key management service (KMS) or a hardware security module (HSM) for data tokenization, not for MCP. None of the sources say how the proxy's own keys are stored, rotated, or protected once cryptography has been externalized (Microsoft's law that encrypted data is only as secure as its decryption key, and its related risk law that encryption alone is not a data protection solution).
Research status: arXiv work covers part of MCP key handling. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2609.22573 is tagged tested (lab, benchmark, or simulation). arXiv:2608.10760 is tagged deployed (an industry deployment).
9. Track where agent parts came from
Treat models, tools, and MCP servers as supply-chain components that need provenance. Runtime controls applied after the fact are not the whole job.
COSAI-SBD requires provenance, model integrity, and runtime validation as part of its secure AI supply chain principle. It treats supply chain security as part of the rest of its secure-by-design agent principles. It is not a bolt-on compliance checkbox.
OWASP-TOP10 and OWASP-STATE both call for software bill of materials (SBOM) or AI bill of materials (AIBOM) records, and for signed manifests, for agent and tool components. That extends a practice from traditional software supply chain security into agents.
OWASP-MCP-DEV requires cryptographic integrity and signing for tools, manifests, and dependencies, not only for the deployed application. Signing the final agent application is not enough if the tools it calls at runtime were never verified.
Sign the model and the data covers the signing and attestation mechanics in more depth. Attestation is a signed statement that a claim about an artifact is true. The rules in this section are the agent-specific use of that discipline. They apply to the tools and MCP servers an agent depends on at runtime, not to the model or the training data it was built from.
OWASP-AST10 extends the same discipline to agent skills. A skill is a natural-language-plus-code package that gives an agent an executable capability, across author, publish, install, load, and execute. An unsandboxed skill inherits the agent's full security context. An external instruction file that is not pinned can be swapped after install.
ETSI-304223 Principle 8 requires data, models, and prompts to be documented as tracked assets. Provenance is not limited to model weights and datasets.
- Model and tool provenance is verifiable through signed manifests or SBOM/AIBOM-style records covering data, models, and prompts (COSAI-SBD, OWASP-TOP10, OWASP-STATE, OWASP-MCP-DEV, ETSI-304223, COSAI-AIMM)
- Approved server and tool versions are pinned and checksum-verified to block a silent malicious upgrade (OWASP-MCP-CHEAT)
- Cryptographic signing extends to tools and dependencies invoked at runtime, not only the deployed agent application (OWASP-MCP-DEV)
- Supply chain security is treated as inseparable from the rest of an agent's secure-by-design posture, not a separate compliance step (COSAI-SBD)
- Agent skills, meaning natural-language-plus-code packages that give an agent executable capability, are inventoried, pinned, sandboxed, and provenance-checked across author, publish, install, load, and execute; an unsandboxed skill inherits the agent's full security context (OWASP-AST10)
OpenCRE crosswalk: Supply Chain, Cryptography, Secure Product Design.
Gap: Signing and provenance protect an artifact's chain of custody once it exists in digital form. None of the sources discuss physical access controls over the systems that build or store that artifact (the principle NIST SP 800-27 and Microsoft share: physical access defeats software-level guarantees). A signed model manifest does not help if anyone who walked up to the build server could use it.
Research status: No arXiv work was found. This gap is open in the 68 sources and open on arXiv. Physical access to a build server is a general security control, and these AI papers do not cover it.
10. Name one accountable party
The most complete treatment in these papers is COSAI-SRF, the CoSAI AI Shared Responsibility Framework (SRF). Exactly one party is accountable for each activity or component. Security and governance requirements cascade down from the business layer through every supporting layer. As an agent's autonomy level rises, accountability shifts from the end user toward the application developer and the agentic provider.
Its five-layer model is L1 AI Business & Usage, L2 AI Information, L3 AI Application, L4 AI Platform, and L5 AI Model Provider. The model is calibrated by operating model as well as by layer. An organization running an AI-SaaS product carries less infrastructure-layer responsibility than one running its own Agent-PaaS or IaaS deployment. The same activity can land on a different accountable party depending on how the system is operated, not only on which layer it belongs to. The operating models in the SRF are AI-SaaS, AI-PaaS, Agent-PaaS, and IaaS.
The dependencies encoded in those five layers stay the same no matter how you structure your teams. Renaming groups does not move an obligation from one layer to another.
COSAI-SRF is explicit about other frameworks. NIST AI RMF defines what to do. ISO 42001, the AI management-system standard, defines how to manage it. CoSAI's SRF defines who is accountable. Those are three answers to three questions.
COSAI-SBD stated a narrower version earlier. It required accountability to be shared among stakeholders and scaled to risk. It did not have the five-layer model, and it did not shift responsibility with autonomy. COSAI-SRF later formalized both.
OWASP-STATE draws the line more narrowly. Model-level safety belongs to the model provider. Deployment-layer security controls belong to the organization doing the deploying. The two cannot be governed as a single function. That is a two-party version of the principle COSAI-SRF generalizes across five layers and four operating models.
- Exactly one party is accountable per activity or component; overlapping accountability is not permitted (COSAI-SRF)
- Accountability shifts toward the application developer and agentic provider as autonomy increases (COSAI-SRF)
- Model-level safety and deployment-layer security are distinct responsibilities that cannot be governed as one function (OWASP-STATE, COSAI-SRF)
- The same activity can land on a different accountable party depending on the operating model in use (AI-SaaS, AI-PaaS, Agent-PaaS, IaaS), not only on which architectural layer it belongs to (COSAI-SRF)
- Shared-responsibility frameworks are treated as complementary to, not competitive with, process frameworks like NIST AI RMF and management-system standards like ISO 42001 (COSAI-SRF)
- The dependencies encoded in the five architecture layers remain invariant regardless of how an organization structures its teams (COSAI-SRF)
OpenCRE crosswalk: Governance Risk and Compliance, Access Control, Log Management.
Gap: COSAI-SRF assigns exactly one accountable party per activity. None of the sources require that accountability to be backed by a written security policy that states what that party is supposed to do (NIST SP 800-27's principle of establishing a sound security policy as the foundation for design). Naming who is accountable is not the same as documenting what they are accountable for.
Research status: arXiv work on written runtime policy is only a proposal. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2510.24383 and arXiv:2603.16586 are tagged theoretical (a proposal only).
11. Find agents you do not govern
Two named ideas show up again without a settled fix.
OWASP-STATE says every organization already has agents running that have not been discovered or governed. It calls this Shadow AI. It treats Shadow AI as a present condition. Shadow IT was never a hypothetical problem an organization could choose to prepare for on its own schedule. Shadow AI follows the same logic.
OWASP-TOP10 names the attribution gap, the absence of a distinct, governed agent identity discussed in Give each agent its own identity. It treats the gap as a structural problem across current architectures, not as an oversight in one deployment. The same gap drives Shadow AI. You cannot govern an agent you cannot attribute to a specific, accountable identity.
OWASP-STATE also says deployment-layer safety and deployment-layer security cannot be governed separately for autonomous agents. That is an early diagnosis of a coordination failure, not a finished risk model. Security teams and safety teams that operate independently for traditional software often cannot simply be merged for agent systems. The merge takes real organizational change. None of the sources claim a proven template for that change.
- Shadow AI is treated as already present in every organization, not a hypothetical future risk (OWASP-STATE)
- The attribution gap, the lack of a distinct governed agent identity, is named as a structural problem across current architectures (OWASP-TOP10)
- Deployment-layer safety and deployment-layer security cannot be governed as separate functions for autonomous agents, though no source claims a proven organizational template for merging them (OWASP-STATE)
OpenCRE crosswalk: Threat Modeling, Access Control, Governance Risk and Compliance.
Gap: OWASP's ASI08 (Cascading Failures) and COSAI-IAM's revocation cascade name the risk. None of the sources map how far one compromised identity's delegations reach once that identity is discovered (the cloud architecture corollary that identity compromise cascades through an entire environment).
Research status: Tested arXiv work shows a compromised agent cascading through other agents. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2604.07775, arXiv:2605.19240, and arXiv:2402.08567 are tagged tested (lab, benchmark, or simulation).
12. Do not treat a benchmark as proof of security
The oldest material here predates the agentic wave. It addresses machine learning (ML) security at the design level, before an agent or an MCP server was part of the picture.
BIML-SE opens with a point the rest of the papers mostly assume. Machine learning is "sophisticated associative-learning technology," not magic. Its recent progress owes more to faster hardware and larger datasets than to a scientific breakthrough in how the algorithms work. That argues against treating ML systems as unknowable black boxes that are exempt from ordinary security engineering.
BIML-ARA and BIML-NOMETER both treat security as an emergent, system-wide property, not as a score. BIML-NOMETER states that no benchmark or leaderboard can certify a system as secure. A benchmark only measures the absence of known badness. It never measures the presence of security. Published benchmarks also risk contamination or leakage into training data, which undermines the benchmark over time even when the system itself has not changed.
BIML-ARA adds that a system is only as secure as its weakest of nine generic ML components. Logical or pipeline boundaries are not automatically trust boundaries unless someone establishes them. A module line on a system diagram does not mean any security enforcement happens at that line.
BIML-LLM24 and BIML-23RISKS extend this to foundation models. A foundation model is an opaque black box. Complete verification or control through prompting alone is never achievable. Accountability for an LLM application's risk rests with whoever builds or fields it, regardless of what is hidden inside the model.
Both papers, with BIML-NOMETER, treat LLMs as auto-associative predictive generators. They are not reasoning or understanding systems. Anthropomorphizing a model misleads evaluation.
BIML-23RISKS is specific about what that accountability requires in practice. Training data must not be legally encumbered by copyright, by child sexual abuse material (CSAM), or by General Data Protection Regulation (GDPR) right-to-be-forgotten obligations. Randomness used for weights and thresholds must use cryptographic-grade generators. Evaluation criteria and results must be real and open to scrutiny. Do not keep them as trade secrets.
BIML-NOMETER closes with a method. Do not score a model's output directly. Follow software security's path from ad hoc testing toward process-based assurance models such as BSIMM (Building Security In Maturity Model). Measure the rigor of the process that produced the system. Do not use the system's output as a proxy for its security.
- Architectural risk analysis, meaning design-level threat modeling, comes before operational bolt-on fixes, not after (BIML-SE, BIML-ARA, BIML-LLM24)
- Classic security design principles (fail securely, least privilege, compartmentalize, defense in depth, keep it simple, be reluctant to trust) apply directly to ML systems (BIML-ARA)
- Training data curation, provenance, and integrity are a foundational control, not an afterthought (BIML-LLM24, BIML-23RISKS)
- Red-teaming and anthropomorphizing a model are treated with explicit skepticism as sufficient validation on their own, because LLMs are auto-associative predictive generators rather than reasoning systems (BIML-LLM24, BIML-23RISKS, BIML-NOMETER)
- Recursive pollution, meaning a model trained on its own or another model's output, degrades integrity across generations and has to be guarded against directly (BIML-LLM24, BIML-23RISKS)
- Reusing a pretrained or transfer-learned model carries the risk of an inherited backdoor or Trojan planted in the original training run, which no amount of downstream fine-tuning reliably removes (BIML-ARA)
- Randomness used for model weights and thresholds relies on cryptographic-grade generators, never naive or predictable ones (BIML-23RISKS)
- Security assurance follows a process-based model, measuring the rigor of development rather than scoring model output as a proxy for security (BIML-NOMETER)
OpenCRE crosswalk: Threat Modeling, Secure Product Design, Data Protection, Supply Chain, Vulnerability Management, Training and Awareness.
Gap and editorial position: Kerckhoffs's open-design principle holds that security should never depend on secrecy of design. This page takes an explicit position. The papers' general acceptance of closed, black-box foundation models is an industry-accepted risk, tolerated for commercial and safety reasons. It is not a competing security principle of equal standing. BIML-LLM24 and BIML-23RISKS argue for regulating the black box. They require disclosure of training data, evaluation criteria, and results. That is the same preference for openness, aimed at the model.
Research status: A counter-view is on arXiv. arXiv:2610.35199, "Can Open Be Too Open?", argues that more transparency does not by itself produce more security. The tag is theoretical (a proposal only). The tag is provisional because it comes from the abstract only, and the paper is unverified. This editorial position is unchanged.
13. Sign the model and the data
CoSAI's original supply-chain workstream treats the AI supply chain as four interdependent dimensions: data, model, application, and infrastructure (COSAI-SUPPLY). A vulnerability introduced at any one stage propagates through the rest. The paper rejects the assumption that an internally developed or internally stored model is safer than a third-party one simply because it is kept in-house.
Its companion paper on artifact signing says what a fix looks like (COSAI-SIGN). Every model artifact carries a signature that ties a producer's identity to its exact contents. A single manifest has to cover the full collection of files, so what was signed and what ships do not drift apart. A consumer authenticates the signer's identity against its own deployment policy. It does not trust a central authority to make that call.
COSAI-SIGN also describes a maturity path: basic integrity checks, then signature chaining, then fully structured attestations. That gives you a staged path. It does not require the strictest form of verification on day one.
NIST-ADVML and the SANS guidelines come at the same concern from different angles. They treat an AI bill of materials as a governance artifact on par with a traditional software bill of materials (SANS-CAISG14, SANS-BLUEPRINT). CSA-AICM11 requires that model weights, training data, and inference pipelines stay protected against poisoning, manipulation, and unauthorized access at all times, not only at deployment. It frames this as an extension of existing cloud supply chain practice.
OWASP-DATASEC adds a rule the signing papers leave implicit. Classification labels on a dataset have to propagate to every derivative of that dataset. A model trained on restricted data does not lose that restriction because the data passed through a transformation.
- Every model artifact carries a signature tying producer identity to its exact content, verified before use (COSAI-SIGN, COSAI-AIMM)
- The producer signs at the release boundary when the artifact is immutable, and the consumer verifies on download before the model enters a pipeline or production (COSAI-AIMM)
- Unsafe serialization formats, such as pickle, are never used for model persistence or exchange (COSAI-SUPPLY)
- Unsigned datasets, checkpoints, or pipeline artifacts are never loaded, executed, or promoted (OWASP-DATASEC, COSAI-AIMM)
- Model and data provenance stay tamper-proof, traceable end to end, and continuously monitored rather than vetted once at intake (COSAI-SUPPLY, NIST-GAI, CSA-AICM11)
- An internally developed or internally stored model is never assumed safer by default than a third-party one (COSAI-SUPPLY)
- Signature verification follows a staged maturity path, from basic integrity checks to full structured attestation, rather than requiring the strictest form immediately (COSAI-SIGN)
- An AI Bill of Materials is treated as a governance artifact on par with a traditional software bill of materials, with disclosure of technical supply-chain detail restricted so that an AIBOM does not itself become an adversary aid (NIST-ADVML, SANS-CAISG14, SANS-BLUEPRINT)
- Data classification labels propagate to every derivative of a dataset, surviving transformation steps rather than being lost along the way (OWASP-DATASEC)
OpenCRE crosswalk: Supply Chain, Cryptography, Data Protection, Access Control, Secure Product Design, Log Management.
Gap: Signing every artifact and manifest is specified here in exacting detail. None of the sources address whether you can do this in an organization that runs thousands of models and constantly retrained checkpoints. They only say it should happen (CIS's design principle that every safeguard must be specific and feasible to implement, not only correct in theory).
Research status: Tested arXiv work measures whether signatures and model bills of materials can be checked across public repositories. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2610.19498 and arXiv:2607.17242 are tagged tested (a measurement of public model files).
14. Classify each attack by what it breaks
NIST-ADVML gives this its most rigorous treatment. Map every adversarial attack's objective to exactly one of three violations: an availability breakdown, an integrity violation, or a privacy compromise. Any attack classification has to specify the learning stage and the attacker's goals, capabilities, and knowledge together. Do not describe an attack only by the technique used.
NIST-ADVML insists that you evaluate mitigations against strong, adaptive, worst-case adversaries, not against weak or static ones. It is explicit that a robustness-accuracy trade-off is inherent to current mitigation techniques. No defense offers an absolute guarantee against a future, evolved attack. A system tuned for maximum robustness against known attacks will typically sacrifice some accuracy on clean, non-adversarial input. That sacrifice is the cost of the robustness.
It also draws a distinction the rest of the papers mostly treat as one problem. Predictive AI (PredAI) and generative AI (GenAI) need related but distinct taxonomies. The attack surface of a classifier that makes a discrete prediction differs from the attack surface of a generative model that produces open-ended output.
OWASP-REDTEAM makes a related point from the testing side. Jailbreaks are only a small subset of the risk in tool-using or agentic systems. A red-teaming vendor that tests jailbreaks alone offers superficial coverage of what NIST-ADVML's taxonomy describes. That taxonomy spans availability, integrity, and privacy attacks across the full ML lifecycle, not only the prompt-based jailbreaks that get the most public attention.
- Every attack's objective maps to availability breakdown, integrity violation, or privacy compromise, with no fourth category (NIST-ADVML)
- Defenses are evaluated against adaptive, worst-case adversaries, never against a fixed or weak attacker model (NIST-ADVML)
- No mitigation offers an absolute guarantee; empirical defenses remain vulnerable to attacker techniques that haven't been invented yet (NIST-ADVML)
- A robustness-accuracy trade-off is treated as inherent to current mitigation techniques, not a solvable engineering gap (NIST-ADVML)
- Predictive AI and generative AI require related but distinct attack taxonomies rather than one taxonomy applied uniformly to both (NIST-ADVML)
- A standardized, common terminology is used across the ML and cybersecurity communities, so an attack described by a data scientist and a security engineer resolves to the same classification rather than two incompatible vocabularies (NIST-ADVML)
OpenCRE crosswalk: Threat Modeling, Data Protection, Supply Chain, Vulnerability Management, Secure Product Design, Training and Awareness.
Gap: Extraction and query-flooding attacks against a model often show up first as an unusual pattern at the network layer. None of the sources tie NIST-ADVML's attack taxonomy back to network infrastructure monitoring as a detection point before an attack reaches the model (CIS Control 12, Network Infrastructure Management). The taxonomy classifies what happened. Nothing here says where to watch for it happening.
Research status: Tested arXiv work detects model extraction from query or API traffic. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2606.05725, arXiv:1805.02628, and arXiv:2201.09243 are tagged tested (lab, benchmark, or simulation).
15. Separate the risk process from the control list
Name one accountable party covers who is accountable for a given AI activity. This section covers the process frameworks that govern risk management more broadly.
NIST-RMF requires that its Govern function run through Map, Measure, and Manage continuously. Govern does not sit beside them as a fourth, separate step. NIST-RMF is explicit that risk equals likelihood times magnitude of impact, that residual risk always remains after mitigation and must be disclosed, and that a system presenting unacceptable or catastrophic risk has to stop, regardless of business pressure to ship it.
It frames Valid & Reliable as the necessary base condition under every other trustworthiness characteristic in its model. A system that is not reliable in the first place cannot meaningfully be called safe, secure, or fair, no matter how it scores on those other dimensions one by one.
CSA-AICM11 says it fills a gap that NIST AI RMF and the EU AI Act leave open on purpose. Both name risk categories and do not specify the operational security controls that satisfy them. A vendor-agnostic control matrix is built to fill that hole. CSA-AICM11 positions itself as a superset of the Cloud Controls Matrix. Its premise is that cloud security is the foundation AI security has to build on.
NIST-GAI extends the same RMF functions to generative AI. It adds that some generative AI risks are known from evidence and others remain speculative, because generative AI measurement science is still immature. It requires content provenance tracking across the lifecycle, using mechanisms such as cryptographic watermarking. Provenance belongs inside the risk management framework. It is not a separate technical add-on.
NIST-PLAYBOOK turns the RMF into practice, and it is explicit about what it is not. Its suggestions are voluntary and non-exhaustive. They are not a mandatory checklist. It pushes you to consider non-AI alternatives. Do not assume an AI system is the right tool by default. Most of the rest of these papers assume the AI system is already being built and ask how to secure it. The Playbook also asks whether to build it.
- The Govern function runs through Map, Measure, and Manage continuously; it is not a separate fourth step (NIST-RMF)
- A system presenting unacceptable or catastrophic risk stops, regardless of business pressure to proceed (NIST-RMF)
- Risk management frameworks and operational control catalogs serve different purposes and don't substitute for each other: a risk framework names what to manage, a control matrix specifies how (CSA-AICM11)
- Control ownership is assigned per stakeholder role, never left ambiguous, across every pillar of a governance framework (CSA-AICM1, CSA-AICM11)
- Valid & Reliable is treated as the necessary base condition underlying every other trustworthiness characteristic, not one criterion among equals (NIST-RMF)
- Governance guidance explicitly considers whether an AI system is the right tool at all, not only how to secure one already being built (NIST-PLAYBOOK)
OpenCRE crosswalk: Governance Risk and Compliance, Threat Modeling, Secure Product Design, Data Protection, Training and Awareness.
Gap: None of the governance frameworks require their own risk categories to be revised from real, observed attacker behavior, as opposed to a static, predefined taxonomy of AI risks (CIS's design principle that offense informs defense). NIST AI RMF and the AI Controls Matrix both describe what to manage. Neither describes a feedback loop from live incidents back into the framework itself.
Research status: arXiv work built from real incident reports revises a risk taxonomy. The tag is provisional because it comes from the abstract only, and the paper is unverified. arXiv:2603.04259 is tagged deployed (real incident data).
16. Protect data at each stage
CSA-DATA applies the CIA triad, confidentiality, integrity, and availability, across the full AI data lifecycle: collection, annotation, storage, processing, training, deployment, and monitoring. It treats each stage as its own exposure point. Do not assume one perimeter control covers all of them. That differs from traditional data security, where a smaller number of storage and transit points usually cover most of the exposure.
OWASP-DATASEC identifies a failure mode inside that lifecycle that traditional software did not have. The context window fuses multiple trust domains into one flat namespace with no internal access control. A conventional application can enforce access boundaries between data sources in code. An LLM's context window cannot, once everything has been concatenated into a single prompt.
OWASP-DATASEC echoes BIML-LLM24 on foundation models. Machine unlearning is technically limited, so data you believe you deleted can persist inside model weights or embeddings. A deletion request can be satisfied at the storage layer and remain unsatisfied at the model layer. Most data governance policies were not written for that distinction.
- Sensitive data stays encrypted at rest, in transit, and in use, without exception (CSA-DATA)
- Operators rotate non-human identities that interact with data on a schedule, with no standing exemption (CSA-DATA)
- Classification labels propagate to every derivative of a dataset, not just the raw source (OWASP-DATASEC)
- Deleting source data does not guarantee removal from a trained model's weights or embeddings (OWASP-DATASEC, BIML-LLM24)
- Lifecycle controls such as retention and erasure must reach every derived asset; a deletion that leaves embeddings, traces, or adapter weights behind does not meet the erasure requirement (OWASP-DATASEC2)
- Each stage of the AI data lifecycle, from collection through monitoring, is treated as its own exposure point rather than covered by a single perimeter control (CSA-DATA)
- The LLM context window is treated as a flat namespace with no internal access control between the trust domains it merges, a risk category traditional software security did not previously need to address (OWASP-DATASEC)
- Detect abuse across services and customers without reading or training on their raw prompt data (UNPROMPTED1-RITTINGHOUSE-HUANG, UNPROMPTED1-ISAK-GILL)
OpenCRE crosswalk: Access Control, Cryptography, Data Protection, Secrets Management, Log Management, Supply Chain, Secure Product Design.
Gap: Data minimization and classification propagation are covered in depth here. None of the sources caution that anonymization or de-identification of training or inference data is never a complete guarantee (Microsoft's law that absolute anonymity is not practically achievable, online or offline). That caveat matters because this section's own point is that deleted data can persist inside model weights, whatever the deletion request assumed.
Research status: Tested arXiv work shows anonymized text can be re-identified. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2610.05586 and arXiv:2602.12806 are tagged tested (lab, benchmark, or simulation).
17. Stop the incident and keep the evidence
COSAI-DEFEND explains why this needed its own framework. Defenses are only as strong as their weakest point. AI systems expand the attack surface well past traditional software. Securing them is "a team sport": coordinated effort across roles, not a single security function acting alone.
It treats separation of code and data as a core principle. It adds an accountability point for AI-assisted contributions. Contributors remain personally responsible for content produced with AI assistance under their own signed contributor agreements, regardless of how much of the work the AI tool generated.
COSAI-IR builds the response mechanics on that foundation. It splits AI incidents into five domains: data, model, deployment, infrastructure, and user interaction. General incident response is already documented elsewhere. This framework covers the AI-specific gaps. It requires raw prompts, system prompts, parameters, and completions to be logged and preserved for forensic reconstruction. Containment must never destroy evidence while it stops the incident. That is a real tension. The fastest way to stop an active incident is often to kill the process or wipe the environment. COSAI-IR says speed cannot come at the cost of forensic soundness.
OWASP-IR adds a governance detail COSAI-IR does not specify. Every critical AI system or model needs an assigned owner at all times. Non-human identities that interact with AI systems must be inventoried and controlled as part of the response plan. An unaccounted-for agent identity is a blind spot during an incident. Root-cause analysis can become impossible if nobody can say which agent identities had access to what at the time of compromise.
CSA-VULN adds urgency to the timeline. The discovery-to-weaponization window for AI-assisted exploit generation has collapsed from weeks to hours. That is why it argues a permanent Vulnerability Operations capability, not a periodic patch cycle, is now the baseline. It names blast-radius containment through segmentation as the primary control once discovery volume rises past what any patch cadence can keep up with. Blast radius here is how far one compromise can spread.
ENISA-FRONTIER states the same containment requirement under a sharper condition. Exploit windows can be negative, meaning an attacker may already hold a working exploit before a vendor fix ships. It also states the starting posture: treat environments as potentially compromised, including agentic endpoints and enterprise browsers.
- Securing AI systems is treated as a coordinated, cross-role effort, not a task for the security team alone (COSAI-DEFEND)
- AI incidents are classified into five domains: data, model, deployment, infrastructure, user interaction (COSAI-IR)
- Raw prompts, parameters, and completions are logged and preserved for forensic reconstruction; containment never destroys evidence (COSAI-IR)
- Every critical AI system has an assigned owner, and non-human identities are inventoried and controlled as part of incident response (OWASP-IR)
- A permanent Vulnerability Operations capability replaces the periodic patch cycle, since the discovery-to-weaponization window has collapsed to hours (CSA-VULN)
- Post-incident reviews are always blameless, and remediation addresses the underlying vulnerability, not just its symptom (COSAI-IR)
- Contributors remain personally accountable for AI-assisted work product under their own signed agreements, regardless of how much the AI tool generated (COSAI-DEFEND)
- Blast-radius containment through segmentation is the primary control once vulnerability discovery volume outpaces any realistic patch cadence, including when exploit windows are negative and an attacker holds a working exploit before a vendor fix ships (CSA-VULN, ENISA-FRONTIER, CSA-ZT-MICRO, CSA-ZT-SEG)
- Defenders treat environments as potentially compromised, including agentic endpoints and enterprise browsers (ENISA-FRONTIER)
- A defensive agent is itself a high-value target; fundamentals such as strong authentication, timely patching, and vulnerability management come before greater agent autonomy (FMF-DEFENSE)
- Connection-defined, topology-defined, and nano-segmentation controls scale with agent autonomy, tool power, and data sensitivity (CSA-ZT-MICRO, CSA-ZT-SEG)
- Build variant analysis into hunting and patching pipelines; an opened fix is not a finished fix (UNPROMPTED2-NEJAD)
- Security advisories give adversarial AI a map of how to attack you; be careful what you disclose (UNPROMPTED2-NEJAD)
OpenCRE crosswalk: Log Management, Access Control, Data Protection, Vulnerability Management, Secrets Management, Threat Modeling.
Gap: COSAI-IR measures containment effectiveness in general. None of the sources require you to confirm that a quarantined agent has actually stopped acting (Microsoft's cybersecurity risk law that isolated networks are not automatically secure). Research shows framework stop controls often fail to do that (arXiv:2607.14166, unverified).
Research status: Tested arXiv work shows a stop or quarantine command often fails to halt side effects. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2607.14166, arXiv:2610.35486, and arXiv:2602.11749 are tagged tested (lab, benchmark, or simulation).
18. Test with people in the loop
OWASP-REDTEAM defines AI red teaming as adversarial testing across safety, security, misuse, robustness, ethics, and alignment failures together. It is not a synonym for jailbreak testing. It separates guidance by system maturity. Most organizations currently run simple generative AI systems. A smaller, fast-growing share run advanced, tool-using, or agentic ones. Both categories carry risks that need specialized adversarial testing. The sophistication of that testing has to differ between them.
It states that no vendor can legitimately claim full coverage or one-click red teaming. Critical or emergent-behavior findings always require human verification. Do not let an LLM judge score itself. Using an LLM to grade another LLM's safety failures inherits the same blind spots the test was designed to catch.
It also requires metrics that are quantitative, reproducible, and tied to real business risk. Do not score on qualitative "vibes." Destructive tool testing has to happen in a sandboxed, dry-run environment, not against production. A red team that exercises an agent's real tool-calling capability against live systems can cause the damage the test is meant to evaluate.
OWASP-REDTEAM also requires that engagements be explicitly authorized, scoped, and bounded by defined rules of engagement. Data handling during testing has to follow clear retention, isolation, and access-control policies, whichever vendor runs the engagement. Govern the red-teaming provider with the same rigor you use for any other third party with privileged access to the system under test.
NIST-218A and FMF-CYBER place a related requirement earlier in the lifecycle. Systematic evaluation for hazardous dual-use capabilities, including cyber uplift and end-to-end autonomous operations, is a release gate for a frontier model. It is not a post-release assurance activity. Dual-use here means a capability that can be turned to harm. Cyber uplift means the model makes cyber attacks easier or more capable.
- No vendor legitimately claims full coverage or one-click red teaming (OWASP-REDTEAM)
- Critical or emergent-behavior findings require human verification, not LLM-judge-only scoring (OWASP-REDTEAM)
- Tool calls, MCP outputs, and agent actions are never assumed inherently safe during testing (OWASP-REDTEAM)
- Testing guidance is calibrated to system maturity, since simple and advanced or agentic GenAI systems carry different risk profiles requiring different testing sophistication (OWASP-REDTEAM)
- Destructive tool testing happens in a sandboxed, dry-run environment rather than against production systems (OWASP-REDTEAM, OWASP-RT-MANUAL)
- Red-teaming vendors themselves are governed under explicit authorization, scope, and data-handling policy, the same as any other privileged third party (OWASP-REDTEAM, OWASP-RT-MANUAL)
- Systematic evaluation for hazardous dual-use capabilities, including cyber uplift and end-to-end autonomous operations, is a release gate for a frontier model rather than a post-release assurance activity (NIST-218A, FMF-CYBER)
- When using an LLM to judge agent work, check that specific investigation steps were followed rather than asking whether the output is right; beware lucky runs (UNPROMPTED2-NAJEEB)
- Do not blindly trust automated verifiers (proof checkers, scanners); soundness bugs in verifiers are security issues (UNPROMPTED2-BROSSARD-AUMASSON)
OpenCRE crosswalk: Threat Modeling, Vulnerability Management, Access Control, Log Management, Data Protection, Supply Chain.
Gap and editorial position: BIML-NOMETER's claim that no benchmark can certify a system as secure is not, and should not be read as, an argument against red teaming's value. Read it with OWASP-REDTEAM's insistence on rigorous, reproducible, human-verified testing. The papers' position is to test to the fullest extent practical, knowing that no test suite will find every vulnerability. Do not treat incompleteness as a reason not to test. CIS's design principle that every control be measurable, and BIML's rejection of a security "meter," are in tension only if you mistake measurability for certification. Used as an incomplete but necessary check, both positions hold at once.
Research status: arXiv work is consistent with this position. arXiv:2503.16431 reports OpenAI external red teaming in practice. The tag is deployed (a practice report). The tag is provisional because it comes from the abstract only, and the paper is unverified. The position above is unchanged.
19. Do not let the model decide access
OWASP-LLMTOP10 restates the untrusted-input problem in application terms. An LLM cannot reliably distinguish trusted instructions from untrusted content, so always segregate and label untrusted external content apart from trusted instructions. Enforce authorization downstream. Do not leave the model to police itself.
It also names the condition behind most of the OWASP Top 10 risks. Developers commonly grant LLM-based systems agency, meaning function calls, extensions, and tools. That grant of agency, not the language model itself, turns a text-generation risk into an action-taking risk.
OWASP-AIEXCHANGE states the same access-control rule as a flat prohibition. Never delegate access control to the generative AI model or agent. It frames that as a specific case of a broader axiom: an AI model can always be wrong or manipulated. Treat that as a constant condition you design around. It is not an edge case you handle when it shows up.
CIS-LLM requires a strict separation between system prompts, user input, and retrieved context. It treats prompt and guardrail changes as configuration items that go through version control and change approval, like any other production configuration. A prompt edit gets the same operational seriousness as a firewall rule change. It is not a lightweight content update a developer can push without review.
OWASP-SOLUTIONS ties these rules to one frame. It maps vendor tooling against the OWASP LLM Top 10 across the lifecycle of LLM operations (LLMOps) and LLM security operations (LLMSecOps). The dependency is explicit. You cannot secure agents until you have secured the models and infrastructure those agents sit on. Agent-specific controls in sections 1 through 11 hold only if the model and infrastructure controls in this section are already in place.
OWASP-LLMTOP10-2026 restates the model-wrong-or-manipulated condition as a design consequence. Build systems so that when the model is fooled, nothing important breaks, because the prompt-injection surface cannot be closed. The same edition splits rankings across two lists. Model-as-component risks stay on the LLM Top 10. Model-as-actor risks pair with the Agentic Top 10.
FMF-AGENTS adds that harm also occurs with no external attacker, when a misaligned benign request meets overly broad scope and permissions. OPENSSF-ASSIST applies the same access-control and validation defaults to AI code assistants. Project instruction files encode those defaults. They do not replace verification of the generated code against ASVS (Application Security Verification Standard), SCVS (Software Component Verification Standard), or AISVS (Artificial Intelligence Security Verification Standard).
OWASP-AIEXCHANGE also names the "lethal trifecta": private data exposure, untrusted content, and an exfiltration channel together. The same concept reappears, almost unchanged, in OWASP-STATE's agent paper, covered in Treat what the agent reads as untrusted. That repetition is evidence of a cross-cutting risk pattern.
- Untrusted external content is segregated and labeled apart from trusted instructions, never trusted by default (OWASP-LLMTOP10)
- Access control is never delegated to the model or agent itself (OWASP-LLMTOP10, OWASP-AIEXCHANGE, SANS-CAISG1, SANS-CAISG11, SANS-CAISG14, COSAI-ZT, CSA-MCP-GATE, NIST-AGENT-ID)
- Extension and tool permissions never exceed what the specific function requires (OWASP-LLMTOP10)
- A strict separation exists between system prompts, user input, and retrieved context, enforced as a configuration item under change control (CIS-LLM)
- An AI model being wrong or manipulated is treated as a constant design condition, not an edge case handled separately when it occurs; systems are built so that when the model is fooled, nothing important breaks, including when a misaligned benign request meets overly broad scope and permissions (OWASP-AIEXCHANGE, OWASP-LLMTOP10-2026, FMF-AGENTS)
- Agent-level security controls are only meaningful once the underlying model and infrastructure security they depend on is already in place (OWASP-SOLUTIONS)
- Project instruction files for AI code assistants encode validation, secrets handling, and least-privilege defaults; they do not replace ASVS, SCVS, or AISVS verification (OPENSSF-ASSIST)
- Agent security controls must work out of the box across many fast-changing agent frameworks (UNPROMPTED2-CHENNABASAPPA-PUDERBAUGH)
- Use AI to generate deterministic scripts and signatures for scale and repeatability, and keep existing deterministic methods rather than replacing them (UNPROMPTED1-MCCARTHY)
OpenCRE crosswalk: Access Control, Input Validation, Injection Prevention, Secure Product Design, Log Management, Supply Chain.
Gap: A growing share of LLM applications are embedded inside browser extensions or email-integrated assistants. None of the sources discuss that delivery surface as distinct from the API-level application security the rest of this section covers (CIS Control 9, Email and Web Browser Protections). The OWASP Top 10 for LLM applications addresses what the model does once it is invoked. None of the sources address the browser or mail client as the thing that invoked it.
Research status: arXiv work covers browser-extension and email delivery, including one real incident. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2509.10540 (EchoLeak) is tagged deployed (a real incident). arXiv:2511.20597 and arXiv:2507.05307 are tagged tested (lab, benchmark, or simulation).
20. Secure adoption instead of blocking AI
SANS-CAISG1 opens with a stance the rest of the papers assume and rarely state. The biggest risk of AI is not using it, so a security team's job is to secure adoption, not to block it. The guidance itself is explicitly point-in-time, because published guidance ages fast. A guidelines document in this space has a shorter useful shelf life than most security guidance has had.
SANS-CAISG14 restates that opening as an operational axiom. AI adoption is already embedded in enterprise products, so blocking AI use is not a viable security control. It names zero model trust: models cannot be fully trusted, so you need external controls, validation, and guardrails at runtime.
It also states a design axiom that cuts against solving agent risk by adding more controls. Complexity is the enemy of security. A simpler agent architecture, with fewer moving parts, is a security advantage of its own.
It is blunt that a model's guardrails are never a sufficient boundary on their own. A model must never have access to data or actions it is not permitted to disclose or take, however strong its guardrails are. That restates the rule from Do not let the model decide access as an architectural requirement. The access should not exist for the model to disclose. Do not create the access and then suppress it with a guardrail.
SANS-BLUEPRINT treats model registries as non-negotiable for production AI governance. It places ultimate accountability for AI risk outcomes on the board and the executive team, not on the security function alone. It frames AI as the most transformative technology shift since the internet. It organizes guidance around three tracks: protect AI, utilize AI, and govern AI. Run them as distinct, coordinated tracks. Do not fold them into one undifferentiated AI security effort.
SANS-MATURITY makes the ceiling explicit. Overall maturity cannot exceed the Govern score. Self-reported capability without documentary evidence is not a stage claim. It also states a position phrased more aggressively than anywhere else in these papers. Security teams need to "fight fire with fire," using AI defensively to match the speed of AI-assisted attacks. A purely human-paced defensive process cannot keep up with an AI-accelerated offensive one.
CSA-STATE looks at the broader cloud environment AI now runs in. It finds a widening gap between how fast AI adoption is moving and how ready security programs are. It names identity, not infrastructure, as the primary attack surface for both cloud and AI workloads. Most current security programs remain reactive and incident-focused by default. The organizations surveyed generally already know that a preventive posture would serve them better.
- A model's guardrails are never treated as a sufficient security boundary on their own (SANS-CAISG1, SANS-CAISG11, SANS-CAISG14)
- Complexity is treated as the enemy of security in its own right; a simpler architecture is a security advantage, not just an operational one (SANS-CAISG1)
- Blocking AI use is not a viable security control; the job is to secure adoption already embedded in products (SANS-CAISG1, SANS-CAISG14)
- Model registries are a non-negotiable governance requirement for production AI (SANS-BLUEPRINT)
- Ultimate accountability for AI risk outcomes sits with the board and executive team, not the security function alone (SANS-BLUEPRINT)
- Identity, not infrastructure, is the primary attack surface for both cloud and AI workloads (CSA-STATE)
- AI security guidance is treated as explicitly point-in-time given how fast published guidance ages, rather than written as a stable, long-lived standard (SANS-CAISG1)
- Governance is organized around three distinct, coordinated tracks: protecting AI, utilizing AI, and governing AI; overall maturity cannot exceed the Govern score, and self-reported capability without documentary evidence is not treated as a stage claim (SANS-BLUEPRINT, SANS-MATURITY)
- Current security programs remain reactive and incident-focused by default, despite general awareness that a preventive posture would be more effective (CSA-STATE)
OpenCRE crosswalk: Access Control, Data Protection, Log Management, Secure Product Design, Supply Chain, Training and Awareness, Governance Risk and Compliance.
Gap: The enterprise-readiness guidance here measures governance maturity and control completeness. None of the sources frame program success as the economic cost imposed on an attacker (Microsoft's cybersecurity risk law that security success means ruining the attacker's return on investment). A fully governed AI program and an expensive-to-attack program are not automatically the same thing.
Research status: arXiv work covers part of attacker cost. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2605.06713 is tagged theoretical (a proposal only). arXiv:2201.09243 is tagged tested (lab, benchmark, or simulation).
21. Plan how you turn an AI system off
NIST-GAI requires that a mechanism always exist to deactivate or decommission a generative AI system found to pose unacceptable risk. Treat shutdown as a designed capability. Do not improvise it after a system is already flagged as dangerous. ETSI-304223 Principle 13 requires proper data and model disposal at end of life.
CSA-AICM1 names Denial of Service as its own threat category inside the AI Controls Matrix, alongside model manipulation and data poisoning. It does not fold availability risk into a generic "resilience" catch-all, which is what several other sources in this set tend to do.
COSAI-IR covers backups of models and embeddings, restoration testing, business continuity, and recovery-time objectives. None of the sources set how fast an AI system must be deactivated once that deactivation is triggered. CIS Control 11 (Data Recovery) and NIST SP 800-27's contingency-planning principle ask for recovery-time and recovery-point targets. This set of papers still does not set those targets for AI shutdown.
- A deactivation or decommissioning mechanism is a designed system capability, not an improvised response built after a system is already flagged as unacceptably risky (NIST-GAI, ETSI-304223)
- Denial of Service is treated as its own named threat category for AI systems, not folded into a generic resilience catch-all (CSA-AICM1)
- COSAI-IR covers backups, restoration testing, business continuity, and recovery-time objectives. None of the sources set how fast an AI system must be deactivated once that is triggered
OpenCRE crosswalk: Governance Risk and Compliance, Secure Product Design, Vulnerability Management.
Gap: COSAI-IR covers backups, restoration testing, and recovery-time objectives. None of the sources set how fast an AI system must be deactivated once that is triggered (CIS Control 11, Data Recovery, and NIST SP 800-27's contingency-planning principle, as noted at the start of this section).
Research status: arXiv work here is only a proposal about risk left after decommissioning, not a required shutdown time. The tag is provisional because it comes from the abstract only, and the paper is unverified. arXiv:2606.12432 is tagged theoretical (a proposal only).
22. A control people bypass does not work
The finding is mainly an absence. Classic security engineering treats a control's usability as part of its effectiveness. Saltzer and Schroeder's psychological acceptability, NIST SP 800-27's call to strive for operational ease of use, and Microsoft's "productivity always wins" all warn you about the same failure. A control that is too painful for a real operator to live with gets bypassed in practice, however sound it looks on paper.
Almost none of the 68 sources engage with that warning directly. The one partial exception is SANS-CAISG1, covered in Secure adoption instead of blocking AI. Its axiom that complexity is the enemy of security points at the same concern from the design side. A simpler system is easier to secure partly because it is easier for the people running it to understand and operate. SANS-CAISG1 frames this as an architectural virtue. It does not directly study control adoption or operator burnout.
The rest of the papers are silent on the question. Zero Standing Privilege, continuous re-verification, per-hop delegation checks, dual sign-off for MCP servers, and quarterly autonomy-level reviews are real, repeatedly cited controls on this page. Every one of them adds friction for the people who have to operate around them.
None of the 68 sources ask whether that friction causes those controls to get quietly disabled, worked around, or rubber-stamped under deadline pressure. A generation of classic security engineering warned that it would. That is a structural blind spot in the current AI security literature. It is not a choice made while writing this page. It deserves your attention because the rest of this page is built almost entirely out of controls that could fail this way.
- A control's effectiveness is inseparable from whether the humans operating it can tolerate using it, per classic security engineering; this consideration is almost entirely absent from the AI-specific corpus synthesized here
- A simpler architecture is treated as easier to secure in part because it is easier to operate correctly, though only as an implicit design virtue rather than a direct study of control adoption or bypass (SANS-CAISG1)
- None of the 68 sources examine whether the friction created by their own recommended controls, continuous re-verification, dual sign-off, per-hop checks, leads operators to disable or route around them under real-world pressure
OpenCRE crosswalk: Training and Awareness, Governance Risk and Compliance.
Gap: This page names control-adoption friction as a blind spot in the field. None of the 68 sources, and this page itself, propose a way to measure that friction. Examples are time-to-bypass, help-desk ticket volume against a new control, or the abandonment rate of a mandated review step. The rest of these papers are comfortable measuring technical metrics. The absence is acknowledged here and is still not instrumented.
Research status: Tested arXiv work measures reviewer fatigue and approval settings in production systems. It does not define time-to-bypass, help-desk volume, or review abandonment. The tags are provisional because they come from abstracts only, and the papers are unverified. arXiv:2605.24309 and arXiv:2606.08919 are tagged tested (lab, benchmark, or simulation).