Enterprise AI initiatives are running into a wall, and most organizations are looking in the wrong direction for the cause. Investments in model selection, fine-tuning, and prompt engineering are consuming significant budget and attention, yet AI deployments continue to underperform. The root cause is not the model. It is the data infrastructure connecting enterprise systems to the AI workloads that sit on top of them. The model becomes a scapegoat for what is fundamentally a connectivity and context management problem.
In this episode of AppDevANGLE, I spoke with Amit Naik, VP of AI Architecture at CData Software, about the structural data connectivity failures shaping enterprise AI outcomes. CData has spent years solving enterprise data connectivity for systems like Salesforce and Workday, and has extended that focus to what Naik describes as the AI connectivity problem, specifically through a managed MCP server product called Connect AI.
Our conversation explored several interconnected themes: why context management is now the dominant variable in AI quality, how token economics are becoming a board-level budget concern, why agentic AI raises the stakes for data governance in ways that conversational AI never did, and why the open source approach to MCP infrastructure introduces enterprise compliance risk that most teams have not fully priced in.
The broader signal is this: enterprise AI strategy that treats the model as the primary investment lever while leaving data infrastructure fragmented and ungoverned is structurally set up to fail, regardless of which model is selected.
Context Engineering Is the Real Differentiator in Enterprise AI
The conversation about enterprise AI has focused heavily on model capability for the past three years. That framing made sense early in the cycle, when model quality was genuinely inconsistent and use case coverage was limited. That period has passed. As Naik put it directly: “The current generation of models is good enough. They are excellent at all of the AI use cases.” The implication for enterprise teams is significant: if the model is no longer the bottleneck, attention must shift to what feeds the model.
Context management is now the variable that determines AI output quality. The context window — the information presented to the model at inference time — is where AI systems succeed or fail. Overload it with irrelevant data, and the model’s reasoning degrades. Feed it stale or duplicate records, and the output reflects those structural problems. Naik described this clearly: if you give incorrect or excessive information to a highly capable person, the outcome is still polluted by that bad input. Models behave exactly the same way.
The consequences of poor context management compound in agentic scenarios. Naik explained that when models are given access to tools via protocols like MCP, poor context quality leads directly to bad tool selection and bad tool call sequences. Each bad call injects worse information into the next step, and the downstream outcome diverges sharply from what the enterprise intended. This failure mode looks, from the outside, like a model problem, but it is not.
The practical engineering discipline that follows from this insight is scope management: limiting the fields, records, and data sources that enter the context to exactly what is needed for a given task. The enterprises that internalize this discipline early will have a structural quality advantage over those still trying to compensate with larger context windows and more powerful models.
Token Economics Are Forcing AI Projects Into Budget Crises
Token consumption is a cost structure most enterprise technology teams did not plan for, and the economics are more consequential than they appear at initial deployment. When AI systems are fed broad, unfiltered datasets, the cost penalty is immediate and continuous. Every conversation with the model carries the full context. Every irrelevant token in that context is wasted spend. At scale, this dynamic is capable of exhausting annual AI budgets in a fraction of the planned timeline.
Naik offered a pattern he sees repeatedly: enterprises commit to a year-long AI initiative, only to exhaust their budget three months in. The cause is architectural. The AI system is consuming far more tokens than necessary because the data access layer has not been designed to filter and scope what reaches the model. The model is then forced to spend its reasoning capacity parsing irrelevant data before it can act on what actually matters.
“You are forcing it to shift between tons of irrelevant tokens, which are wasted spend and the actual information that it needs,” Naik explained. “But the model is having to spend a lot of intelligence, a lot of its chain of thought on parsing this data.”
This is why token economics have become a board-level conversation. The spend profile of enterprise AI at scale is not analogous to cloud infrastructure costs — it scales with usage patterns in ways that are harder to predict and control. The architectural decision to scope data access tightly is not just a quality decision; it is a financial one. Enterprises that treat it as a pure engineering concern, rather than a budget governance issue, are likely to find themselves in the position Naik describes, where they have accurate outputs that are too expensive to sustain.
Agentic AI Transforms Data Quality From an Output Problem to an Execution Risk
The risk profile of poor data connectivity changes fundamentally when AI systems move from generating text to taking actions. Conversational AI with bad grounding data produces wrong answers. Agentic AI with bad grounding data executes wrong transactions. The difference between a bad response and an erroneous procurement action, a corrupted HR record, or an unauthorized communication is the difference between a quality problem and a compliance and operational incident.
Naik was direct about the implications: agentic systems are interacting with heuristic models, not deterministic workflows. That autonomy is the source of their power, but it requires a corresponding increase in governance infrastructure. “Agentic systems have more autonomy, which is both their power and where things can go horribly wrong unless you have good governance policies in place.”
The governance challenge is structural, not procedural. It is not enough to review agent outputs after the fact. Enterprises need to define at deployment time what tools an agent can call, what permissions those tools carry, and what data each tool can access. They also need the ability to map an agent’s action back to the identity it was acting on behalf of, whether that is a human user or another agent in an A2A workflow. As agent-to-agent coordination becomes more common, this attribution chain becomes increasingly difficult to trace without deliberate infrastructure support
Open Source MCP Infrastructure Creates Compliance Exposure Most Teams Have Not Evaluated
One of the clearest risk signals in the conversation was Naik’s assessment of how enterprises are approaching MCP server infrastructure. The pattern he observes frequently is teams building their own MCP servers, using open source implementations of uncertain maintenance quality, or treating MCP connectivity as a lightweight integration task. None of these approaches holds up well under enterprise compliance scrutiny.
“Is your compliance or governance group going to really allow you to take some open source project and use it in your critical connectivity, governance, or context functions?” Naik asked. The question is pointed, and many enterprise teams have not yet formally asked it. The OWASP Top Ten vulnerabilities for MCP, including confused deputy attacks and malicious tool injection, represent genuine threat surface for any agent that has write access to business systems.
The risk is not theoretical. An agent operating with an overly broad permission set, acting on a compromised tool call, can become an attack vector into enterprise data. The consequences are more severe when that agent has write access to financial, HR, or procurement systems. Vibe-coded or community-maintained MCP infrastructure is unlikely to provide the audit trails, permission scoping, and governance controls that enterprise compliance teams will eventually require.
Real-Time Data Access Is an Architectural Requirement, Not a Performance Preference
A recurring theme in the conversation was the inadequacy of analytic data pipelines as the foundation for enterprise AI workloads. The typical enterprise pattern of processing operational data into an analytic layer and then exposing that layer to AI systems introduces latency that directly impairs output quality. AI systems working from data that is days or weeks old are not grounded in the current state of the business.
Naik framed this as a fundamental mismatch: “AI typically works best when it has real time data. Doesn’t work so well when it has analytic data. You don’t want answers that are stale from a few days ago, a few months ago, whenever your analytic workload processed.” For use cases involving customer data, inventory, financial state, or HR records, the gap between analytic latency and real-time access is the gap between a useful AI system and one that is confidently wrong.
This has infrastructure implications that many enterprises have not fully worked through. The data systems that support AI workloads need to provide live, scoped, authenticated access to operational data sources. That requirement pushes back toward the connectivity layer, which is precisely where CData has built its product position.
Analyst Take
Enterprise AI is entering a phase where model capability is no longer the primary differentiator. The current generation of models is sufficient for the majority of enterprise use cases. What separates successful AI deployments from failed ones is the quality, timeliness, and governance of the data infrastructure underneath the model. Organizations that have not yet made this architectural shift will continue to misdiagnose failures and misallocate investment.
The traditional approach of bolting AI onto existing analytic data pipelines, using broad data access to compensate for poor connectivity, and treating governance as a follow-on concern, is structurally insufficient for where the market is heading. As agentic systems gain write access to operational systems, the cost of these architectural shortcuts escalates from degraded output quality to operational and compliance risk. The governance debt compounds quickly.
What enterprises should recognize is that AI data connectivity is not a solved integration problem. It requires real-time access, deliberate context scoping, attribution-capable identity management, and governance controls that most current connectivity approaches do not provide. The gap between a vibe-coded MCP server and an enterprise-grade managed connectivity layer is not a feature gap; it is a compliance and security gap that will eventually surface under audit or incident pressure.
The next phase of this market will be defined less by which model an enterprise selects and more by how well the data layer connecting that model to business systems is designed, governed, and controlled. The enterprises that treat data connectivity as a first-class architectural concern will be the ones that sustain AI investment and deliver measurable business outcomes. The ones that do not will keep running out of budget and blaming the model.
