Enterprise AI strategies over the past two years have been dominated by a single obsession: compute. Organizations raced to secure GPU allocations, signed long-term infrastructure contracts, and built out machine learning platforms with significant capital investment. The underlying assumption was that once accelerator capacity was in place, AI workloads would scale. That assumption is now colliding with a more stubborn reality. The bottleneck was never exclusively compute; it was always data delivery, and the storage layer is now the constraint that determines whether AI investments actually pay off.
In this episode of AppDevANGLE, I spoke with Ronen Schwartz, VP of Products for Google Cloud Storage at Google, about the growing misalignment between AI compute capacity and the storage architectures enterprises use to feed it. Our conversation explored the structural reasons storage has been overlooked in AI infrastructure planning, how training, inference, and agentic workloads each impose fundamentally different storage demands, why checkpoint management during training represents a category of risk most enterprises have not yet priced in, how unstructured data handling is becoming a critical requirement for production multi-agent systems, and where storage architecture is heading as agent-scale deployments move from pilot to production.
The broader market signal is clear: storage is no longer a backend commodity in AI infrastructure. It has moved into the critical path, and enterprises that fail to architect for it will find their most expensive assets systematically underperforming.
GPU Utilization Is a Storage Problem Disguised as a Compute Problem
The conventional framing of enterprise AI infrastructure places compute at the center and storage at the periphery. That framing made sense when AI workloads were experimental and data volumes were modest. It becomes expensive as organizations move into sustained training runs, high-throughput inference pipelines, and increasingly complex agentic systems.
Schwartz described the pattern directly from customer conversations: “The team has spent a long time fighting to get their TPUs, their GPUs, to get their hands on accelerators. And once they got them, they’re actually not always able to get the output they needed. And very often it actually goes to not enough thought put into the data delivery path — what actually feeds the TPUs, the GPUs with data.”
The financial logic is straightforward. Accelerators represent the largest and most constrained line item in any serious AI infrastructure budget. When storage cannot deliver data at the rate required, every idle GPU cycle represents a direct cost with no productive return. Schwartz made the economic argument explicitly: “Accelerators are the enterprise’s most expensive line item. Every second they sit idle, waiting for data is basically a massive amount of money loss.”
When GPU underutilization is treated as a compute problem, organizations respond by seeking more accelerator capacity, which compounds the actual issue. When it is treated correctly as a data delivery problem, the solution is fundamentally different and, as Schwartz noted, often more tractable. The bottleneck is addressable if it is properly identified.
Training, Inference, and Agents Each Demand a Distinct Storage Architecture
One of the clearest analytical points from the conversation is that “AI workloads” is not a useful unit of analysis for storage planning. The data movement patterns across training, inference, and agentic pipelines are structurally different, and treating them as a single category creates architectural gaps that surface as operational failures at production scale.
Schwartz framed it this way: “AI is not one workload. Really, the movement of data looks completely different. It’s like different heartbeats. Each phase has its own unique heartbeat and its own unique requirements.”
Training workloads are dominated by massive sequential reads, but the less-discussed demand is checkpoint writes. Every fifteen minutes or so, a training cluster must pause and commit a multi-terabyte checkpoint to storage. Slow checkpoint writes freeze the entire cluster. Slow restores after a failure compound the cost further. The implications for total training economics are significant, particularly for organizations running multi-day or multi-week training jobs.
Inference introduces an entirely different profile: real-time, latency-sensitive, user-facing. The storage requirements shift from throughput-dominated to latency-dominated. Agentic systems add another layer: variable-duration agents, shared state across agent instances, dynamic context retrieval, and the need to persist and resume agent sessions efficiently. Each of these patterns requires deliberate architectural decisions. Enterprises that inherit a training-optimized storage architecture and extend it to inference or agents without modification will find the results disappointing.
Checkpoint Management Is an Underpriced Operational Risk
Checkpoint behavior during training is a specific failure mode that has not received adequate attention in enterprise AI planning conversations. When a training job fails without a recent checkpoint, the productive computation performed since the last successful write is lost. For large model training runs, that window can represent hours of GPU time and significant direct cost.
Schwartz quantified the performance gap in concrete terms, describing the ability to write checkpoints more than three times faster and restore them more than five times faster than standard object storage. The operational implication is not only faster recovery, it is also the difference between a training failure that costs minutes and one that costs hours, multiplied across the frequency of failures in long-running distributed training jobs.
This is the kind of infrastructure characteristic that only becomes visible after a painful production incident. Organizations that evaluate storage infrastructure primarily on cost-per-gigabyte or raw throughput numbers miss the operational resilience dimension entirely. Checkpoint performance belongs in the evaluation criteria alongside throughput and latency for any enterprise running sustained training workloads.
Zonal Colocation Inverts the Traditional Storage Reliability Assumption
Conventional enterprise storage design prioritizes geographic distribution. Multi-zone and multi-region configurations are standard practice for disaster recovery and resilience. That design philosophy, applied without modification to AI training workloads, can produce a counterproductive result: data is distributed across zones in ways that increase the physical distance between storage and accelerator clusters, adding latency that compounds at the scale required by large model training.
Schwartz described a deliberate architectural reversal: “When you’re actually looking into accelerator performance, you want the data to be as close as possible to the GPU or the TPU cluster. This is actually very important. And this is where we have been first to the market with zonal colocation, meaning we can ensure the data is very close to the CPU GPU, directly influencing the time of the byte to the accelerator.”
This represents a genuine architectural trade-off, not simply a product feature. Zonal colocation optimizes for accelerator utilization at the cost of some of the geographic redundancy that traditional enterprise storage design assumes. For AI training workloads where the cost of idle GPU time is high and the data itself is typically reproducible, this trade-off is often well-justified. Enterprises need to make this trade-off explicitly rather than inheriting a default storage configuration designed for different workload priorities.
Unstructured Data Is the Unsolved Problem for Production Agentic Systems
Multi-agent AI systems are moving from demonstration to production deployment across enterprise environments. The architectural requirements for that transition are more demanding than pilot deployments reveal. Agents in production need to retrieve context dynamically, share state across instances, and make decisions based on data that is predominantly unstructured, such as documents, images, video, sensor streams, and PDFs.
Traditional storage handles unstructured data as opaque binary objects. Agents cannot query meaning from raw bytes without additional processing, which introduces latency and complexity that degrades the real-time responsiveness agents are designed to provide. Schwartz described an architectural response: an object enrichment capability that automatically extracts semantic metadata from files at ingest time, enabling agents to reason about unstructured content without on-demand processing overhead. “When a file or an image lands in the bucket, we instantly attach relevant metadata about the content of that data to the object. Now an agent can basically interact with what used to be called unstructured data because there is actually a very good understanding.”
The broader implication is that storage for agentic systems is not merely a performance problem; it is a semantic problem. The storage layer must become capable of representing meaning, not just bytes, if agents are to operate reliably on the kinds of heterogeneous enterprise data that actually exist in production environments. This is a meaningfully different capability requirement from anything that traditional enterprise storage architectures were designed to address.
Developer Velocity and TCO Arguments Now Apply Directly to AI Storage Selection
Enterprise technology selection has historically treated storage as a secondary decision made after the primary infrastructure platform is chosen. ECI Research survey data suggests that this approach deserves reconsideration in AI contexts. According to our research, 47.2% of respondents identified “developer velocity and ease of integration” as the greatest weight in their final technical selection process, and 32.6% identified total cost of ownership, including consumption predictability, as the primary factor.
Both of these criteria connect directly to storage architecture for AI workloads. Developer velocity is affected by how easily teams can configure storage for specific workload types, manage checkpoint behavior, and integrate storage into training and inference pipelines. TCO is directly affected by how efficiently storage can feed accelerators, because GPU utilization is the primary driver of AI infrastructure cost. A storage architecture that delivers higher accelerator utilization at equivalent hardware cost improves AI TCO more effectively than further price optimization on the storage layer itself.
The implication for enterprise technology buyers is that storage evaluation criteria need to be rebuilt around AI workload economics, not legacy assumptions about capacity pricing and redundancy configurations.
Analyst Take
The enterprise AI infrastructure market is entering a maturation phase in which the returns on raw compute investment are increasingly determined by everything that surrounds the compute, e.g., data pipelines, storage architecture, retrieval efficiency, and state management. The organizations that will extract the most value from their GPU investments are not necessarily those with the most accelerator capacity, but those that have built storage foundations capable of feeding that capacity continuously and efficiently.
The traditional approach to storage — treating it as a back-end commodity evaluated primarily on price and capacity — is insufficient for AI workloads at production scale. Training, inference, and agentic pipelines each impose distinct requirements that must be addressed deliberately in architecture design. Defaulting to general-purpose storage configurations for AI workloads creates the same kind of invisible tax that legacy technical debt creates in application portfolios: costs that accumulate quietly until they become impossible to ignore.
The larger shift is this: storage has moved from infrastructure background to AI critical path, and the organizations that recognize this early will avoid the expensive pattern of discovering the bottleneck only after GPU clusters are already idle. The next phase of enterprise AI competition will be decided less by who has the most accelerator capacity and more by who has built the data infrastructure capable of using that capacity effectively. For organizations looking to get more from their AI investments, now is the time to take a closer look at the storage layer. Explore how Google Cloud Storage can support your training, inference, and agentic AI workloads, or reach out to Ronen Schwartz to discuss the right storage approach for your organization’s needs.
