Agent Experience Is Becoming the New Measure of AI Performance
Enterprise adoption of agentic AI is moving quickly, creating an operational challenge that traditional observability systems were not designed to address. According to theCUBE Research, 71% of organizations are sourcing agentic AI capabilities through platform vendors, while 68% engage IT or consulting service providers. Only 32% are primarily building these capabilities in-house.
At the same time, two-thirds of enterprise AI leaders are already implementing multi-agent collaboration in production or pilot environments. The question is shifting from whether enterprises will deploy AI agents to something more immediate: Who is watching the agents?
In this episode of AppDevANGLE, I spoke with Keith Zubchevich, President and CEO of Conviva, about why real-time observability must evolve as AI agents become a larger part of customer and employee experiences.
Our conversation explored why traditional application monitoring does not provide enough visibility into agent behavior, why conversational interactions require new forms of behavioral analysis, and why agent experience, or AX, could become an important competitive differentiator as enterprises move agentic AI into production.
What emerged is a broader lesson for enterprise AI: knowing that an agent technically worked is no longer enough. Organizations increasingly need to understand the experience the agent created while getting there.
Real-Time Visibility Becomes Critical at Agentic Scale
Traditional application environments already present observability challenges. theCUBE Research found that 32% of enterprises take hours to become aware of production problems. Agentic systems make that delay increasingly problematic because agents can operate continuously, interact with large numbers of users, and execute workflows at a scale no human operations team can manually observe.
Zubchevich sees parallels with Conviva’s history in streaming media. Conviva built its business around measuring digital experiences in real time, including video resolution, buffering, and other factors affecting the actual viewer experience rather than simply determining whether a stream was technically available. Agentic AI creates a similar challenge. “It is not enough to just understand the quality of service,” Zubchevich said. “You have to understand the quality of experience.”
An AI agent can technically complete a transaction while still frustrating the customer. It can reach the correct outcome after unnecessary prompts. It can successfully answer a question while requiring enough back-and-forth interaction that the user never wants to use it again. Traditional availability metrics may classify all of those interactions as successful, but from an experience perspective, they may represent failure. This distinction becomes increasingly important as agents move into customer support, commerce, internal operations, sales, and other business-critical workflows.
Traditional Observability Was Built for Deterministic Software
The observability challenge extends beyond scale. Agents behave differently from conventional software. Traditional applications are largely deterministic. Product teams design interfaces, establish workflows, define buttons and actions, and instrument known events.
Agentic interactions are conversational and dynamic. A user may express the same intent in dozens of ways. Conversations can follow unpredictable paths, and the agent itself can dynamically determine how it should respond or which actions it should take. That makes traditional clickstream and funnel-based analytics less effective.
Zubchevich argues that enterprises will increasingly need to think in terms of behavioral patterns. “You can no longer have a deterministic path,” he said. “You have to unpack a conversation and convert that to a pattern.” That means extracting signals such as user intent, topics, sentiment, interaction patterns, and the progression from intent to outcome.
The important question is no longer simply whether the customer clicked the expected button. It becomes: What was the customer trying to accomplish? Did the agent understand that intent? How efficiently did it move toward an outcome? And did the user ultimately get what they wanted?
This requires a different observability architecture than the telemetry systems enterprises have historically used for applications and infrastructure.
Quality of Service Does Not Equal Quality of Experience
This distinction between technical performance and user experience is particularly important for AI agents. Traditional observability frequently emphasizes quality of service. These metrics remain useful, but they do not tell organizations whether an agent created a good experience.
Zubchevich argues that agent experience requires organizations to evaluate the interaction itself. “Measuring an agent outcome is just one small piece of the puzzle,” he said. “It could frustrate people that get to an outcome, which means they’re never coming back.”
Consider a customer service agent that eventually resolves a billing issue. Operationally, the interaction may be classified as successful. But if the customer had to restate the problem multiple times, correct the agent repeatedly, or spend ten minutes reaching an answer that should have taken two, the business has a very different problem.
That is why agentic observability will increasingly need to connect technical telemetry with experience intelligence. The enterprise needs visibility not only into whether an agent works, but whether people actually want to continue using it.
Agent Experience and AI Economics Are Connected
The experience problem also has a direct financial dimension. Every unnecessary agent interaction consumes compute resources and, depending on the architecture, additional tokens. An inefficient agent can therefore create two problems simultaneously. It frustrates the use, and it costs more to operate. Zubchevich pointed to token consumption as one of the emerging operational challenges facing organizations as AI usage expands.
The relationship between experience and cost creates an important optimization opportunity. If an agent requires repeated prompts to understand user intent, those additional interactions consume resources. If enterprises can identify where those conversations break down and improve the agent’s ability to recognize intent earlier, they can potentially improve both customer experience and AI economics.
“If I can make the experience more efficient, and I can cut down the time it takes an agent to understand the intent…not only are you improving the experience, you’re reducing token consumption at the same time,” Zubchevich said.
That changes the way enterprises should think about AI cost optimization. Token consumption cannot be evaluated entirely as an infrastructure metric. Some of that cost is directly connected to experience design. Poorly performing agents may generate unnecessary consumption because users need additional interactions to accomplish the same task. Better AX can therefore become part of the FinOps conversation around enterprise AI.
Agent Experience Is Becoming a Competitive Surface
As organizations deploy more customer-facing AI agents, the quality of those experiences will increasingly affect brand perception. theCUBE Research found that 51% of organizations continue to rely on public AI tools such as ChatGPT and Microsoft Copilot, while only 20% report enterprise-wide AI deployments built on governed frameworks. Agent adoption is moving faster than operational maturity. That creates a risk as enterprises rush customer support agents, sales agents, commerce agents, and other AI-powered experiences into production without equivalent visibility into what customers are actually experiencing.
Zubchevich again sees parallels with streaming. Early streaming providers learned that simply delivering video over the internet was not enough. Consumers expected an experience comparable to television. Buffering, poor resolution, and unreliable playback could quickly drive viewers elsewhere.
Agentic AI may follow a similar trajectory. The novelty of interacting with an AI agent will fade and what remains will be the experience. “That experience is the competitive differentiator for businesses to build lasting consumer or user relationships,” Zubchevich said.
The agents that understand intent quickly, minimize unnecessary interaction, deliver accurate outcomes, and create low-friction experiences will have an advantage over agents that merely function. This makes AX more than another observability metric. It becomes part of product strategy.
Multi-Agent Systems Raise the Stakes Further
The observability challenge becomes even more complex as enterprises move from individual assistants toward multi-agent architectures. Two-thirds of enterprise AI leaders are already implementing multi-agent collaboration in live or pilot workflows. These environments create chains of interaction in which one agent may interpret intent, another retrieve information, another execute an action, and additional agents validate or coordinate the result.
A successful customer outcome may therefore depend on multiple autonomous systems operating together. Traditional monitoring approaches can tell enterprises whether individual components are available, but they may struggle to explain why the overall experience failed.
That means organizations will increasingly need visibility across the entire agent journey—from initial intent through agent interactions and ultimately to the business outcome. As agent architectures become more distributed, experience intelligence becomes an important layer for connecting technical performance with what the customer actually experienced.
Analyst Take
Agentic AI is creating an observability problem that enterprises cannot solve simply by adding another dashboard to their existing monitoring stack. The fundamental unit being observed is changing. Traditional application observability measures deterministic systems. Agentic systems introduce dynamic conversations, probabilistic behavior, evolving intent, and increasingly complex interactions between multiple autonomous components.
That requires enterprises to expand the definition of observability. Infrastructure health still matters. Application performance still matters. Model performance still matters. But none of those independently answer the question that ultimately determines whether customer-facing agents succeed: Did the agent create a good experience?
Keith Zubchevich’s distinction between quality of service and quality of experience is particularly important. An agent can be technically available, successfully execute an API call, and eventually complete a transaction while still creating a poor customer experience. Enterprises therefore need to begin measuring agentic systems around intent, sentiment, interaction efficiency, outcome quality, and behavioral patterns alongside traditional telemetry.
There is also a meaningful economic implication. Agent experience and AI cost optimization are not separate problems. Every unnecessary prompt, repeated question, or failed attempt can increase both customer friction and token consumption. Improving the path from intent to outcome can potentially improve experience and reduce operating costs simultaneously.
As agentic interfaces become more common, AX could develop into a competitive surface in much the same way digital experience became critical to streaming, e-commerce, and mobile applications. The next stage of enterprise AI will not be defined simply by which organizations deploy agents first. It will be defined by which organizations can see what those agents are doing, understand the experiences they create, and improve those experiences in real time.
