Autonomous Agents Are Creating a New Enterprise Accountability Problem
Enterprise AI governance is entering a fundamentally different phase.
The first wave of governance focused largely on controlling access to models, monitoring usage, protecting sensitive data, and documenting compliance. Those challenges have not disappeared, but autonomous AI agents introduce a more difficult problem. Agents can now make decisions, invoke tools, access APIs, and delegate tasks to other agents without requiring a human decision at every step.
That changes the accountability model.
According to ECI Research, 48% of respondents identified navigating compliance documentation and audit evidence collection as a major source of cognitive load for developers. Another 26% selected secure machine-to-machine secret management and API tokens as the biggest hurdle to implementing zero-trust applications. Those challenges become more significant when machines begin delegating authority to other machines.
In this episode of AppDevANGLE, I spoke with Sudeep Goswami, CEO of Traefik Labs, and Andreas Prins, who leads sovereignty strategy at SUSE, about why enterprise AI governance needs to evolve beyond simply observing what agents do.
Our conversation explored agent-to-agent authorization, runtime policy enforcement, cryptographically verifiable evidence, sovereign AI infrastructure, and why enterprises increasingly need to prove what an agent was authorized to do rather than reconstruct its behavior after the fact.
Observability Does Not Prove Authorization
Enterprise technology teams have spent years improving their ability to observe applications. Logs, metrics, traces, dashboards, and audit records provide increasingly detailed visibility into what systems are doing. Agentic AI creates a different requirement. Knowing that an agent performed an action does not necessarily establish whether that agent was authorized to perform it.
Goswami described the governance challenge as a progression of increasingly difficult questions. The first is visibility. “Which agents are doing what?” Goswami said. “This is where your visibility and your dashboards are able to answer this question.”
The next question is authority. “What authority is each agent carrying?” he said. “Because when you have an agent that is handing a task to another agent, that authority should shrink and not leak out.”
That distinction becomes critical in multi-agent environments. A human may authorize one agent to perform a task. That agent may delegate part of the task to another agent, which may then access an API, database, model, or additional tool.
At every step, the distance between the original human intent and the final machine action increases. Traditional observability can reconstruct that chain. Governance increasingly needs to prove that every action within the chain was permitted.
Agent-to-Agent Delegation Changes the Least-Privilege Model
Agentic systems also complicate traditional approaches to identity and access management. Possessing valid credentials does not mean an agent should be authorized to perform every action those credentials technically allow. Context matters.
“Just because an agent has some credentials, is that agent allowed to make this specific action right now, given the surrounding context around it?” Goswami asked. He compared the distinction to physical access. “Your badge will get you into a building, but that doesn’t mean your badge is going to allow you to approve a fifty thousand dollar transaction,” he said.
That is an important distinction for enterprise AI governance. Identity answers who or what is making a request. Authorization needs to determine whether that specific action should be permitted under the current circumstances. Agent-to-agent delegation makes that more difficult because permissions can potentially propagate through increasingly complex chains of autonomous systems.
Prins compared the challenge to an earlier transformation in software development, when enterprises moved from manual production releases toward automated CI/CD pipelines. Organizations eventually learned to encode approvals, security requirements, secrets management, and release controls directly into the software delivery process.
Agentic AI may require a similar transition. “Let’s rethink, and more important, let’s articulate as code,” Prins said. The goal is to establish clear boundaries around what individual agents can do and how authority changes as tasks move through an autonomous workflow.
Policy Enforcement Needs to Happen at Runtime
Defining an AI governance policy is only part of the problem. Enterprises also need to know whether those policies are being enforced when agents actually take action. That makes runtime enforcement increasingly important. “You can have your policies defined in a system, but then how do you know that your policies are actually being enforced at the right place at the right time?” Goswami said.
The challenge applies to both successful and unsuccessful actions. Organizations need visibility into what an agent was allowed to do, but they also need evidence of actions that were denied. “It’s equally important to know what an agent was allowed to do and did it do it well,” Goswami explained. “But then also scenarios where the agent tried to do something that it wasn’t authorized for.”
Denied actions can become an important governance signal. If agents repeatedly attempt actions outside their intended authority, organizations may have poorly defined workflows, insufficient context, excessive permissions, or gaps in their policy design.
That creates a feedback loop between policy definition and enforcement. “There needs to be a feedback mechanism from your policy engine to the enforcement engine, and then back to the policy engine,” Goswami said.
This moves AI governance closer to the execution layer. Instead of treating governance as documentation surrounding an AI system, policy becomes part of the infrastructure through which agents operate.
Audit Logs May Not Be Enough for Autonomous Systems
The next challenge is proving that those controls actually worked. Traditional audit systems rely heavily on logs. But agentic environments raise an important trust question: what happens when the system responsible for producing the evidence is also the system being audited?
Goswami used the analogy of a vehicle odometer. If someone could change the odometer without an independent record of its previous state, there would be no reliable mechanism for determining whether the number had been altered. The same principle can apply to system-generated audit evidence.
“This is the problem with logs,” Goswami said. “They’re all self-signed by the vendors that generate them. They can be tampered with, and there’s no third-party way to know when and how it was tampered.” That does not make traditional logging obsolete. It means higher-risk agentic systems may require stronger evidence mechanisms.
Goswami argued that organizations need high-fidelity records of agent actions and gateway decisions, including whether an action was allowed or denied, combined with cryptographic mechanisms that make changes detectable. “You need the logging capability at a cryptographic level, but then you also need a third-party verification mechanism to be able to check against it,” he said.
The distinction is between recording activity and proving the integrity of the record. As autonomous agents begin making more consequential decisions, that difference could become increasingly important for audit, compliance, security, and accountability.
Sovereignty Is Becoming Part of AI Governance Architecture
The governance challenge becomes even more complicated when enterprises operate across disconnected, regulated, or air-gapped environments. According to ECI Research, 47% of respondents said their organizations support a mixture of connected and disconnected or air-gapped software environments. Another 11% are deploying generative AI specifically within on-premises or air-gapped infrastructure.
That means enterprise AI governance cannot universally depend on external SaaS services or remotely hosted control planes. For some organizations, the governance infrastructure itself must operate inside the environment they control.
“The moment you become dependent on a third-party SaaS service that you don’t control or it’s not in your own soil, then all bets are off,” Goswami said.
Prins framed sovereignty as a risk-based architectural decision. Organizations increasingly have multiple choices for how they consume AI, ranging from closed frontier models to open-weight models and fully open-source approaches. Each comes with different tradeoffs around convenience, transparency, customization, and control.
“There are actually two ways that you need to decide on,” Prins said. “One is on the model.” The second question is where that model runs. “Do you own and control the entire stack?” Prins asked. “Is that a stack that you manage and you run yourself, and you have full transparency of every step along the way?”
The answer will vary by workload. Lower-risk applications may be appropriate for externally hosted AI services. Highly regulated workloads involving confidential or proprietary data may require substantially more infrastructure control. “The more regulated, the more control you should have,” Prins said.
This expands the definition of AI sovereignty beyond model selection or data residency. It increasingly includes the infrastructure, authorization systems, policy enforcement points, audit evidence, and other governance mechanisms surrounding the model.
AI Governance Has to Extend Across the Entire Stack
No single technology is likely to solve this problem. Enterprise agentic systems span models, compute infrastructure, APIs, gateways, identity systems, policy engines, observability platforms, and the applications agents ultimately interact with.
Each layer contributes part of the governance model. The model determines behavior. Infrastructure determines where that behavior executes. Identity establishes who or what is interacting with the system. Policy defines what should be allowed. Gateways can enforce those decisions as agents interact with APIs, tools, and models. Observability provides visibility, and provenance or evidence mechanisms establish a record of what occurred.
Goswami argued that these capabilities need to work together rather than becoming another collection of disconnected governance silos. “There is no single vendor that can deliver sovereignty all by themselves,” he said.
That is an important consideration for enterprise architecture teams. The objective should not be finding a single AI governance product that claims to solve every layer of the problem. It should be creating an architecture in which authority, policy, enforcement, visibility, and evidence remain connected as agentic workloads move across infrastructure.
“Security is a big thing. Control is a big thing,” Goswami said. “So you’ve got to bring all of these things together, from the model to the underlying infrastructure, to the control layer, to the visibility layer, and now increasingly important is the trust layer.”
Analyst Take
Enterprise AI governance is moving from a visibility problem to an accountability problem. That distinction matters.
Observability tells organizations what an AI agent did. Agent governance increasingly needs to establish what the agent was authorized to do, where that authority originated, how it changed when work was delegated, whether policies were enforced at runtime, and whether the resulting evidence can be trusted. This is a much larger architectural problem than monitoring AI activity.
As agent-to-agent workflows become more common, traditional identity and access models will need to account for delegated authority that changes based on context. An agent should not inherit unlimited permissions simply because another authorized system asked it to perform a task.
The same shift is happening with auditability. Logs will remain important, but higher-risk environments will increasingly require evidence that can demonstrate the integrity of both allowed and denied actions. That pushes governance closer to runtime infrastructure and creates a stronger connection between policy, enforcement, observability, and provenance.
Sovereignty adds another dimension. Organizations operating in regulated, disconnected, or air-gapped environments need governance systems that can function under the same infrastructure constraints as the AI workloads themselves. The ability to control where models run is only part of that equation. Enterprises also need control over where authorization happens, where policies are enforced, and where evidence is generated and retained.
AI governance can no longer stop at knowing what an agent did. Enterprises need to know what the agent was allowed to do, why it was allowed to do it, and whether they can prove that the controls worked. As autonomous agents take on more consequential enterprise workflows, provable trust will become an increasingly important part of the architecture required to move agentic AI safely into production.
