A Prudent Approach to AI Governance
There is a cacophony of discussion, rhetoric, and noise about AI proliferation every day, ranging from AI utopia to apocalypse. Beyond these philosophical discussions, however, business leaders and rank and file workers continue to implement AI in a steady and regular cadence. For example, the Stanford HAI’s AI Index and McKinsey’s enterprise surveys show that nearly 90% of 10 enterprise organizations utilize AI in at least one business function (e.g., marketing content, customer service chatbots, code assistance, etc.). Additionally, the New York Times reports that while 30% of workers reported using AI as part of their job in 2023, this percentage has increased to 76% in 2026. Finally, a survey by JetBrains in April 2026 found that a vast majority (90%) of software developers regularly use one or more AI tools, demonstrating how AI is impacting how modern software is built today.
Clearly, enterprise organizations and knowledge workers are embracing AI to bolster productivity, automate processes, and lower costs. While organizations push AI towards these positive outcomes, industry research points to a darker side of the story as many enterprises are also lagging behind when it comes to AI governance (See Figure 1).
According to research from IBM, 87% of enterprises claim to have established clear AI governance frameworks, but fewer than 25% have operationalized the actual technical controls needed to monitor or enforce those frameworks. Additionally, Deloitte reports that while 74% of enterprise organizations plan to deploy autonomous or agentic AI within the next two years, only 21% currently have a mature governance model equipped to oversee AI systems that act without direct human prompts. Finally, a qualitative research study from theCUBE and Qualitate indicates that 11 of 25 respondents said their own AI adoption creates greater operational pressure today than defending against adversarial AI. Governance requirements such as risk assessment, policy development, identity controls, and vendor scrutiny drive the need for thorough and complex governance policy decisions.
The gap between aggressive AI proliferation and tepid AI governance adds incremental and compounding risk to organizations. For example, data might leak as employees use the AI tool du jour to help them get their jobs done, outside of the knowledge or oversight of IT and security teams. Business processes might be disrupted by excessive agent capabilities, as poorly governed agents access sensitive data, disrupt supply chains, or compromise systems in an effort to achieve their objectives. And there is the risk of regulatory compliance violations if sensitive credit card, intellectual property, and patient record data is inadvertently shared with frontier LLMs.
This article is the first in a series examining how organizations can govern these risks as AI becomes more deeply embedded in enterprise operations. We begin with an overall approach to AI governance, followed by deeper examinations of two areas where agentic AI creates new control requirements: data and identity.
A Prudent Approach to AI Governance
The risky situation described above should set off alarm bells in executive offices and board rooms – especially since rapidly evolving AI technologies and use cases can be complex and confusing, leaving organizations wondering where to begin and how to proceed.
While AI can be somewhat scary, organizations can overcome this trepidation and bridge the AI governance gap through a 3-phased plan for AI governance:
- Establishing organizational buy-in, structure, and commitment
- Forming and implementing basic core controls and operations
- Promoting processes and skills for continuous monitoring and change
Establishing organizational buy-in, structure, and commitment
Former U.S. Secretary of State & Chairman of the Joint Chiefs of Staff, Colin Powell, is quoted as saying, “When you choose to be a leader, you accept the responsibility of being the catalyst for change, the standard-bearer for quality, and the persistent force against inertia.” This serves as a good analogy for a top-down approach to AI governance. CEOs and other executives must champion the effort with an appropriate AI governance budget, clear objectives for AI projects , and ongoing training across the organization. The goal? Embed AI governance into the corporate culture (i.e., create departmental champions, establish internal promotion, encourage participation and suggestions from individuals, etc.).
With executive leadership established, organizations should create an AI steering committee including participants from line of businesses, legal/risk/compliance, IT, and cybersecurity at a minimum. The steering committee must create a governance structure that accommodates innovation, risk management, technical feasibility, and business value. AI projects themselves should follow a risk classification (including data classification), establishing stringent guardrails for high-risk projects and limited harnesses for low-risk projects (i.e., a “fast path” with minimal gates). For help with AI project classification, organizations should consult the EU AI Act, a pyramid model consisting of 4 distinct risk classes (i.e., minimal/low risk, limited risk, high risk, unacceptable risk).
The steering committee is also responsible for creating and implementing enterprise policies that align with their industry, geography, or regulatory compliance requirements (for example: an acceptable use policy for knowledge workers utilization of AI services). Formal policies and contractual requirements should extend to third-party risk management as well (i.e., AI and/or agentic data access through customers, partners, and vendors).
As organizations institute AI governance, they often focus solely on policies, processes, and technical controls. These are critical areas, but it’s important to make sure to create governance success metrics based on security and ROI. For example, what is the business objective for a particular AI project and how can the organizations measure progress? Security success metrics can include things like shadow AI ingestion, data leakage tracking, runtime security metrics, and red teaming results.
Forming and implementing basic core controls and operations
NBA great, Michael Jordan, said: “If you get the fundamentals down, the level of everything you do will rise.” This same sentiment applies here as AI governance is built upon a strong foundation.
In this instance, organizations should start by seeking out established AI guidelines, frameworks, and best practices. Some examples include the National Institute of Standards and Technologies (NIST) AI Risk Management Framework (AI RMF 1.0, released in January 2023), the International Organization for Standardization (ISO) ISO/IEC 42001:2023 (Artificial Intelligence — Management System), released in December 2023 by the ISO and the International Electrotechnical Commission (IEC), and the Open Worldwide Application Security Project (OWASP) LLM & GenAI Security and Governance Checklist. Most organizations use a combination of these and other guidelines to create an AI governance structure customized for their individual needs.
With a structure in place, organizations should create a register of all technical assets that will be used in all AI projects (i.e., data, applications, APIs, identities etc.) and then define technical controls and guardrails for AI project development, test, and production. It is especially important to set up harnesses for nondeterministic AI agents that may exhibit anomalous behavior as they seek to meet objectives. Technical AI guardrails will likely depend upon existing and perhaps new types of security tools, therefore tools and their configuration settings should also be recorded and tracked. To prevent some type of unanticipated AI catastrophe, technical teams should also implement AI “kill switches” for high-risk projects. Governance should also include the implementation of a tamperproof audit trail as well as the establishment of auditing schedules and guidelines.
As AI systems become more agentic, governance also has to operate at runtime. An approved AI project can encounter data, conditions or requested actions that change its risk during execution. Technical controls need to continuously evaluate what an agent is attempting to do, the data and authority involved, and whether the action remains within established boundaries. Security guardrails can then block anomalous/malicious actions or raise an alert to a human-in-the-loop.
Beyond a comprehensive AI foundation and runtime controls, AI governance should also include active and continuous AI red teaming and penetration testing. This will require broad knowledge of AI vulnerabilities, AI-centric threat intelligence, and a complete list of all internal AI projects. Organizations may need to work with specialized service providers here.
Promoting processes and skills for continuous monitoring and change
According to the AI Security Institute, the length of complex, multi-step technical tasks (e.g., software engineering, vulnerability evaluation, and systems operation) that frontier AI models can complete autonomously has a doubling time of roughly 4 to 4.2 months. This illustrates AI technology’s unprecedented advancement. Given this pace, AI governance must be flexible and continuously adaptable to change.
First and perhaps foremost, organizations should commit to continuous education and training – especially for developers, IT operations, and cybersecurity staff. These groups must keep in lockstep with technology innovations, vendor offerings, and potential risks. Forward-looking organizations with aggressive AI plans may want to a chief AI technology officer (CAITO) position to lead and manage personnel. This effort should also include establishing an AI Center of Excellence to monitor technology advances, new products, vendor AI implementation, etc.). Under the CAITO, teams should actively research, test, and report on new products, technology progress, and potential business use cases to the AI steering committee.
AI governance should include quarterly governance and policy review requirements to ensure that the organization is keeping up with rapid change. This should also be aligned with any changes in regulatory, geographic, or industry requirements.
The pragmatic AI approach described here is necessarily high-level. Nevertheless, it can act as a blueprint, helping organizations move beyond AI confusion and establish the organizational, technical and operational foundations for governing AI as adoption expands.
In upcoming research, we will take a closer look at two control domains that become especially important as AI systems gain greater access and agency: data and identity. We will examine how AI changes the practical meaning of authorized data access, as well as how organizations can govern the identity, permissions and authority of agents acting on behalf of people and business processes.
