AI governance defines how an organization should manage artificial intelligence. AI security examines how that intelligence can be compromised. Building trustworthy AI requires both.
Artificial intelligence is moving from experimentation into business-critical workflows. Organizations are deploying AI assistants, integrating large language models into products, automating decisions, and introducing autonomous agents that interact with APIs, browsers, databases, and enterprise systems.
Yet adoption is advancing faster than many organizations’ ability to govern and secure these systems.
A model can produce impressive results while operating on untrusted data. An AI agent can complete a business task while exposing confidential information. A carefully designed governance policy can exist on paper while the production system remains vulnerable to manipulation.
This is where two international standards become particularly relevant: ISO/IEC 42001:2023, which establishes requirements for an Artificial Intelligence Management System (AIMS), and ISO/IEC 27090:2026, which addresses security threats and compromises specific to AI systems.
They address different dimensions of the same engineering challenge: how to develop, deploy, operate, and improve AI responsibly without overlooking its security risks.
1. ISO/IEC 42001: Making AI Governance an Operational Capability
Published in December 2023, ISO/IEC 42001 provides requirements and guidance for establishing, implementing, maintaining, and continually improving an AI management system.
Its scope extends beyond model development. It is relevant to organizations that develop, provide, or use AI-based products and services.
The central idea is straightforward: responsible AI should not depend exclusively on individual judgment, informal guidelines, or a collection of disconnected controls. It requires a management system with defined responsibilities, objectives, processes, risk assessments, monitoring, and continual improvement.
In practical terms, an organization should be able to answer questions such as:
- Which AI systems are being used, and for what purposes?
- Who is accountable for their development, approval, deployment, and monitoring?
- What risks can affect users, customers, business operations, and other stakeholders?
- How are requirements translated into technical controls?
- How are incidents, deviations, and lessons learned incorporated into improvements?
ISO/IEC 42001 provides the organizational framework for answering these questions systematically.
However, governance alone does not demonstrate that an AI system resists attacks. A documented risk-management process must be connected to effective technical safeguards and evidence that those safeguards work.
2. ISO/IEC 27090: Addressing the Security Risks Specific to AI
Published in October 2026, ISO/IEC 27090 focuses on understanding, detecting, and mitigating security threats affecting AI systems throughout their lifecycle.
Traditional cybersecurity remains essential, but AI introduces additional attack surfaces and failure modes.
Consider several examples:
Data poisoning. An attacker manipulates training data or other data used in model development or updating, potentially influencing model behavior.
Prompt injection. Malicious instructions embedded in user input or retrieved content attempt to override an AI application’s intended behavior.
Model theft and extraction. Repeated or strategically designed queries may expose information about a model or enable an attacker to approximate its behavior.
Compromised AI workflows. An agent may interpret untrusted content as instructions, misuse available tools, or perform actions beyond the intended business scope.
Sensitive information exposure. A system may reveal confidential data through generated responses, retrieval mechanisms, logs, or improperly authorized tools.
These examples do not all represent the same type of vulnerability, and their applicability depends on the architecture. The important principle is that AI security must consider the model, its data, its interfaces, its surrounding infrastructure, and the actions it can perform.
ISO/IEC 27090 complements established information-security practices, including those associated with ISO/IEC 27001 and ISO/IEC 27002, by addressing AI-specific threats.
It does not make conventional security controls obsolete. It helps organizations identify where additional AI-focused analysis and mitigation are necessary.
3. Why the Two Standards Are Stronger Together
ISO/IEC 42001 and ISO/IEC 27090 should not be treated as competing frameworks.
They answer complementary questions.
| Engineering question | Primary contribution |
|---|---|
| How should the organization govern its AI activities? | ISO/IEC 42001 |
| Who owns AI-related risks and decisions? | ISO/IEC 42001 |
| Which security threats are specific to AI systems? | ISO/IEC 27090 |
| How can AI security threats be detected and mitigated? | ISO/IEC 27090 |
| How should security controls be embedded into organizational processes? | Both, supported by the wider security management framework |
| How can effectiveness be demonstrated continuously? | Governance, engineering controls, testing, monitoring, and evidence |
The distinction is important.
An organization may have a mature approval process but insufficient adversarial testing. Another may conduct sophisticated penetration tests without a clear ownership model, risk acceptance process, or mechanism for tracking remediation.
Neither approach is sufficient on its own.
Governance without technical validation risks becoming procedural compliance. Security testing without governance risks becoming a collection of isolated technical activities.
The objective is to connect policy, risk, implementation, verification, and accountability.
4. A Practical Example: Securing an AI Agent That Uses Enterprise Tools
Imagine an AI agent that receives a support request, searches internal documentation, retrieves customer information, and invokes an API to update a business record.
The agent appears to work correctly in functional testing. But what happens when a retrieved document contains malicious instructions? What if the agent can access records unrelated to the current customer? What if a tool call is repeated after a timeout? What if the model changes and begins producing different tool arguments?
A practical implementation can connect governance and security as follows.
Step 1 — Define the intended use and accountability
Establish the agent’s purpose, permitted users, business boundaries, accountable owners, and expected outcomes.
Document prohibited actions and identify decisions that require human approval.
This supports the management-system perspective of ISO/IEC 42001.
Step 2 — Map the attack surface
Identify the model provider, prompts, retrieval sources, connected tools, APIs, identity mechanisms, secrets, logs, and data flows.
Determine which components process sensitive information and which actions could create material business impact.
Use ISO/IEC 27090 to inform the analysis of AI-specific security threats.
Step 3 — Translate risks into enforceable controls
Do not rely exclusively on instructions written in a system prompt.
Implement authorization in the application and underlying services. Validate tool arguments against schemas. Restrict access to the minimum required permissions. Separate untrusted content from privileged instructions where the architecture permits, and require explicit approval for sensitive or irreversible operations.
Apply rate limits, secret management, audit logging, and isolation appropriate to the deployment.
The model should not be the sole enforcement mechanism for security-critical decisions.
Step 4 — Design security-focused tests
Build tests around realistic attack scenarios:
- Can a malicious retrieved document induce an unauthorized tool call?
- Can the agent access another user’s data?
- Can an API request bypass server-side authorization?
- Does the application reject malformed or unexpected tool arguments?
- Can confidential information leak through the response or diagnostic logs?
- Does a model update change security-relevant behavior?
- Are destructive operations blocked when approval is required?
A successful response to a normal prompt does not establish resilience against these scenarios.
Step 5 — Verify and monitor in production
Collect evidence from automated tests, access-control checks, security assessments, deployment approvals, and monitoring.
Track security-relevant events, unexpected tool usage, policy violations, and model or configuration changes.
Define incident-response procedures and establish when a system must be restricted, rolled back, or taken out of service.
Step 6 — Feed the results back into governance
Security findings should update the risk register, control implementation, test suites, acceptance criteria, and operational procedures.
This closes the loop between management-system governance and engineering execution.
The result is not simply a compliant document or a one-time security assessment. It is a repeatable process for managing AI risk as the system evolves.
5. The QA and DevSecOps Opportunity
Quality engineering has a central role in making these principles measurable.
For conventional applications, teams commonly validate functional behavior, performance, accessibility, integration, and security. AI-enabled systems require these foundations plus additional dimensions of verification.
An AI-focused test strategy should consider:
Behavioral robustness: Does the system remain within its intended boundaries when inputs are ambiguous, adversarial, or unexpected?
Authorization integrity: Are permissions enforced by trusted application components rather than inferred from model responses?
Data protection: Can sensitive information be exposed through prompts, retrieval, outputs, tool results, or logs?
Tool-use safety: Are tool calls validated, authorized, bounded, and traceable?
Model and configuration changes: Do updated models, prompts, retrieval indexes, or agent workflows introduce regressions?
Operational resilience: Can the organization detect anomalous behavior, investigate incidents, and recover safely?
These tests should be risk-based. A read-only internal assistant and an autonomous agent capable of transferring funds, modifying production infrastructure, or changing customer records do not require identical assurance levels.
For teams using CI/CD, AI security checks can be integrated into the software delivery lifecycle. High-risk findings can block promotion, while approved exceptions should be documented, time-bound where appropriate, and assigned to an accountable owner.
Automation can provide repeatability, but human review remains important for assessing business impact, interpreting ambiguous findings, and approving residual risk.
6. An Implementation Roadmap
Organizations do not need to begin with a large, abstract compliance exercise. A focused pilot can establish the foundations.
Phase 1 — Discover. Inventory AI use cases, models, agents, data sources, third-party services, and connected tools.
Phase 2 — Assess. Identify business, operational, privacy, and security risks. Prioritize systems according to their exposure and potential impact.
Phase 3 — Govern. Define responsibilities, approval criteria, acceptable-use rules, risk ownership, and evidence requirements under the AI management system.
Phase 4 — Engineer. Implement access controls, input and output validation, tool restrictions, monitoring, and security safeguards appropriate to the threat model.
Phase 5 — Verify. Build functional, adversarial, integration, and regression tests around the identified risks. Record results and remediation decisions.
Phase 6 — Improve. Monitor production behavior, review incidents, reassess risks after material changes, and update controls and tests.
Useful indicators include the proportion of AI systems with an assigned owner, the coverage of identified high-risk scenarios, the rate of unresolved critical findings, the time required to remediate security issues, and the percentage of material model or configuration changes subjected to risk-based validation.
These indicators should support decisions, not become compliance metrics pursued without regard to actual risk.
7. What These Standards Do Not Guarantee
Neither standard should be interpreted as a guarantee that an AI system is safe, secure, accurate, or free from bias.
ISO/IEC 42001 establishes requirements for an AI management system; it does not certify the correctness of every model output.
ISO/IEC 27090 provides guidance on AI-specific security threats and their treatment; applying it does not eliminate all attack possibilities.
Likewise, a certification claim should be distinguished from a statement that an organization has implemented selected controls or adopted recommendations. The scope of any assurance claim matters.
Organizations must still account for their applicable legal obligations, contractual requirements, threat environment, technical architecture, and risk tolerance.
The goal is not to promise zero risk. It is to make risk visible, assign responsibility, implement proportionate controls, and demonstrate that those controls are being evaluated and improved.
Trust Must Be Engineered
AI governance and AI security are not separate destinations. They are connected parts of a lifecycle that begins with intended use and continues through development, deployment, monitoring, and change.
ISO/IEC 42001 helps organizations establish the management discipline required to govern AI responsibly. ISO/IEC 27090 brings a focused perspective on threats that can compromise AI systems and the measures used to address them.
For QA and DevSecOps professionals, the practical implication is clear: turn AI policies into testable requirements, security risks into verifiable controls, and test results into evidence that informs governance.
The next stage of AI adoption will not be defined solely by how capable our models become. It will also be defined by how reliably we can constrain, test, monitor, and account for the systems built around them.
Trustworthy AI is not a property we should assume from a model’s performance. It is an engineering outcome that organizations must continually earn.
References
- ISO/IEC 42001:2023 — Artificial intelligence management systems: https://www.iso.org/standard/42001
- ISO/IEC 27090:2026 — Cybersecurity: Addressing security threats and compromises to AI systems: https://www.iso.org/standard/27090
- ISO/IEC 27001 — Information security management systems: https://www.iso.org/standard/27001.html
This article provides a practical engineering perspective and is not a substitute for the full standards, a formal conformity assessment, or organization-specific security advice.