Last year, theCUBE Research asserted that we are on the brink of a transformative shift toward intelligent data applications, set to revolutionize business operations. We introduced the concept of “Uber for All” as a metaphor, predicting that today’s linear value chains would evolve into dynamic digital representations of a business (i.e. digital twins), where intelligent data applications empower firms to orchestrate entire ecosystems in real-time.
This week, these research insights were highlighted on the cover of The Economist, in an article co-authored by George Gilbert of theCube Research with the senior editor of AI initiatives. The piece examines how digital twins are reshaping traditional business models. Our premise is that for the last twenty-five years, platform technology in business has been limited to consumer online services that match people to content or other people and functions that reserve resources, like Uber or Airbnb. However, the emergence of digital twins allows businesses to orchestrate complex operations in mainstream businesses from end to end. This innovation facilitates a dynamic assembly of resources and expertise across entire ecosystems to uniquely serve each customer request—similar to how Uber connects riders with drivers. The impact on mainstream business operations is significant, heralding a transformation in business models.
In this Breaking Analysis we share our latest insights on the future of intelligent data platforms. More specifically, we will dig into how we envision the transition from Web 2.0 applications to a new dynamic era supported by digital twin platforms. We’ll also share some spending data over the past ten years that will serve as a representation for how we see the market unfolding over the next decade.
The Economist Leans into theCUBE Research Concepts
First, we were thrilled to see the Economist collaborate with the co-author of this research and publish on the topic. We don’t control the headlines but we believe the impact is more profound than making companies more efficient. While that’s true we see a profound shift in business and operational models, as well as industry structures that we’ll dig into here. In addition there’s more to come with the economist, so stay tuned for longer form articles featuring many of the concepts we’ve put forth.
Uber’s Platform as an Organizing Metaphor for the Future of Business Operations
By way of review, as you may recall, theCUBE Research began working on this idea that today’s data and application platforms will evolve to become more Uber-like, where the people, places and things in a business would be orchestrated by a system that, like Uber, could integrate data across what used to be a linear value chain and harmonize those disparate data elements in near real time. And we used Uber as the metaphor where drivers, riders, ETAs, prices, destinations and the like were assembled together, on-demand, to satisfy a customer’s needs and deliver a monetizable set of actions, all orchestrated by a single platform.
We made Uber the organizing metaphor for the article because we believe it will be seen as a lighthouse example by business historians in the future. The key to the metaphor, and why we open the article with this concept, is that Uber can use its end-to-end orchestration to dynamically assemble the operational workflow to serve each customer. In this case, it’s the optimal path to match a rider with a driver, and then continually optimize the fastest or cheapest path to the destination.
This is the heart of the metaphor for mainstream business operations because orchestrating a digital twin means dynamically assembling an operational workflow to assemble and match each customer with the expertise and resources to fulfill their request. In the past, we could only do simple matching functions like consumer online services or reservation systems. We couldn’t do sophisticated operational workflows in a non-linear fashion. To achieve this vision, traditional business however needs to change, not just the technology, but the people and processes.
The Old Way…Web 2.0 Application Platforms
It’s worth reviewing today’s business platforms, which largely organize linear value chains digitally.
To provide historical context, business management was designed to be a machine. It was modeled in large part on the Prussian military and designed in the age of mass production a hundred years ago. The goal was to manage large fixed assets like assembly lines, refineries, railroads, etc. And the people, processes and technology of traditional business organizations were organized around these linear value chains i.e. (source, make, deliver). This approach was popularized by Michael Porter’s seminal research on strategy that’s been orthodoxy for forty plus years.
In this model, businesses owned and employed the people and resources that were in this value chain. One of the core management functions was planning and control that converted these inputs, the people and resources, into products and services. But the core limitation was that it was organized for efficiency, not agility. The scope and span of control was limited largely by what this machine could employ, what it could own, and what it could control internally.
True Integration Across Applications has Eluded Businesses
As we’ve stated many times, the holy grail of seamless integration has been aspirational and difficult to achieve without massive resources. Rather today’s application estates are essentially islands of automation with data generally locked inside the system.
We’ve posited in the past, the knowledge graph is a linchpin of the transition to our Uber-like era. What’s missing today is a data harmonization layer, what people sometimes call the semantic layer, but it goes beyond today’s metrics layer seen with the likes of dbt and AtScale. Rather, the knowledge graph leverages data from all applications with a common definition across the ecosystem’s application portfolio.
The difference between today’s Web 2.0 world and the future of platforms is profound in our view. Specifically, for decades, Web 2.0, built on Enterprise 2.0, was focused on integrating apps. However we’ve only succeeded in building islands of automation with data, business logic and processes locked inside of enterprise apps. With microservices, which was the Web 2.0 technical underpinning, we really built essentially droplets of functionality that each spoke its own language. The combination of these islands of automation and then microservices on top, were only really integrated later in data lakes and the modern data stack. This technology has limits in terms of the management span of control to these hardwired connections between departmental or functional apps.
Technology to harmonize the data and application estates is the first revolution that’s coming along in our view, which will usher significant benefit to organizations. We’ve been calling them digital twins and we extent the notion to entire software platforms. The technical implementation is achieved through knowledge graphs that convert the “Tower of Babel” shown in the lower right of the slide above, into an abstraction layer that harmonizes the data estate. We believe this will enable analytics, specifically AI, to orchestrate the people, places and things in a much greater span of control in real time.
How Digital Twin Platforms Enable Real-time Orchestration
Digitally Mapping Strings Databases Understand to People, Places and Things in the Physical World
Let’s dig deeper into how we see the orchestration of an ecosystem across an organization, its customers and suppliers, using Uber as the metaphor for this new era, enabled by digital twin platforms.
In the graphic below, we show a digital representation of the most important elements involved in an Uber transaction. A rider initiates a request (people), the app knows where that rider is (places) and where cars (things) are proximate. The system offers the consumer a number of options varying in value, like big car, small car, shared ride, etc. Machine intelligence figures out the time it will take to pick up the rider, the price based on the destination and overall demand, an ETA based on traffic and so forth.
The architecture of today’s systems is what’s limiting us from implementing this sort of business model, which is if you have to extract, in a batch mode, all the data from these different operational apps, centralize it in a data lake or the modern data stack, perform your analytics and then reverse ETL it back, there’s so much that gets lost.
There’s time, the data comes from applications that speak different languages, disparate metadata and governance models. What’s missing is a real-time abstraction that allows a higher intelligence, which we’ll get to as the next breakthrough, to see what’s going on across all these systems. As we described above, to match the rider to the driver, to find the route and to continually update the route…none of that can happen in real time if you’re extracting data in batch mode, mixing it up, doing analysis, and then in some process, sending it by reverse ETL back to the original applications.
What’s needed is a new layer, similar to what Uber built. But in the case of Uber it took many thousands of engineering hours and deep technical expertise. Rather we envision an emerging platform that is accessible to all organizations.
And that brings us to the second revolution where if you want to do this at scale with millions of riders and drivers every day, you can’t do it with a hierarchical management system and you can’t do it with traditional data architecture.
Applying the Concept to Organizations
Now let’s take a look at how we see this metaphor evolving to enable a business to take advantage of the concept. Below we show how an intelligent data app might dynamically assemble a workflow for each customer in real-time. The fundamental assumption is applications are integrated and the data supporting these apps is harmonized. The example we show here personalizes offerings for customers based on the availability of products, the time to deliver, product attributes, etc. And is able to dynamically price the offering based on these dimensions.
The harmonization layer provides the enabling technology for the second revolution, which is agent-based platforms. Agents are large language models that know how to plan, take action, invoke tools, adhere to rules and execute toward top down goals. We see large language models becoming large action models. But for them to work well, they need a common language that they’re navigating and traversing. They provide the productivity to orchestrate processes so human managers can supervise these millions of rider and driver matches and routes. This is the key so that the digital twin provides the world model by which agents can navigate. Then agents can be essentially the machine tools that make the humans far more productive supervisors.
It’s worth reminding folks that a hundred years ago AT&T thought it would eventually need to employ all working-age women, and yes, it was women at the time, in the country to be switchboard operators, but technology ultimately made it possible for the consumers to be their own operators by dialing their own calls. What we’re talking about here with this digital platform, with digital twins and corresponding agents is a similar revolution, but not in labor productivity like mass production or consumer technology like the telephone, this is a revolution in management technology.
And it’s not about agents that you hear solving math Olympiads, that’s nice and impressive. We’re talking about essentially microservices that can be given a goal and that can invoke some set of workflows and tools and call on other agents to accomplish that goal. The combination of humans supervising these armies of agents will remake how we organize work, turning traditional companies into platforms with previously unimaginable scope and control. Importantly, today’s microservices are hard-coded and as such subject to breaking. We see intelligent agents as more flexible and capable of adjusting to changing business conditions.
Time Series of Legacy Application and Modern Data Platforms
Let’s bring in ETR data to show some of the old and new players in this new era and we’ll comment on how the market will evolve.
This chart above shows a ten year history from ETR’s data set. The lines represent Net Score on the vertical axis, which is a measure of spending velocity on a platform. The horizontal axis is time back to January 2014. The red dotted line at 40% represents highly elevated spending momentum. The N in the latest July survey was over 1,700 and even back ten years ago the Ns were well over 1,000, often north of 1,800.
We chose SAP and Oracle Fusion as two examples of legacy systems that contain critical information, access to which is necessary for the vision we’ve laid out. As well, Salesforce is shown and we’ve added Databricks and Snowflake as representatives of firms that could vie for the future of intelligent data apps.
We’ve also superimposed the logos of Celonis, EnterpriseWeb, RelationalAI and Palantir as firms that are working on, or have delivered early versions of the harmonization layer we so often discuss. And you can see above that over the past decade, the representative systems of SAP and Oracle, while very large, have seen more maturity. Salesforce as a customer management system has seen pretty steady momentum although you can see it maturing and dipping below that red dotted line.
Meanwhile Databricks and Snowflake remain above that line with the former crossing over when the market shifted to having more of a cost focus. And the names we show at the top of the chart as logos, form at least part of the missing link. As we discussed last week there are connections to the back end, the harmonization layer, a governance system and an agent orchestrattion layer that can interpret and manage to top/down metrics and supervise a bottom up outcome.
The bottom line is we’re seeing a shift away from the silos to a state where platforms become more integrated and harmonized. We don’t have all the harmonization technology today. As we’ve laid out iun the logos, these are the four candidates right now we think are best positioned, today. The agent layer comes next. And as we talked about last week, we think we’ll hear that from Salesforce next month and from Microsoft two months after that at their big conferences.
Implications for How Businesses Operate
Let’s close by summarizing the implications to businesses and how we see this vision affecting business models, the programming and admin models and the industry structure.
We believe we are going through a profound transition. Web 2.0 emerged 20 plus years ago. And that was centered around the application programming model, the admin model, the business model and created the industry structure we’ve been living with for a couple decades.
One of the profound findings we see coming in this era of the enterprise digital twin is businesses become platforms. They essentially become ecosystems of digital twins. And the system becmes an interlocking collection of platforms. The platforms comprise the digital twins that an organization controls and those of its partners, suppliers and potentially customers.
The admin model has been, “you build it, you run it,” similar to the famous dictum of AWS’ Werner Vogel’sfrom some years back. That’s not going to work in the world we’re moving to. It’s got to be self running.
The business model is characterized by platforms over applications. Platforms are programmed and customized with data. The pricing model is not utility based so much as much closer to the customer outcomes. The real upside is matching pricing with value to a greater degree than can be accomplished today.
The last point is industry structure for platforms has always been winner take most. So we think we’re going to see a very different world. The world that people talk about with consumer online services and big tech, we may see that coming to the rest of the economy. Importantly, this is not just about platforms that are developed by technology companies, this is industry or companies, leaders within industry, within manufacturing, automobiles, energy, financial services, healthcare, etc., building out this capability for their ecosystems. Of course they’re going to rely on the technologies that we know and love and talk about all the time, the data platforms, those legacy applications, the connectors in between, the emerging capabilities, the AI capabilities, the LLMs and the small language models and the action models and all the surrounding tooling, et cetera. But those will be applied to dramatically transform industries and the structure of those industries, which will be dominated by existing and new players.
What do you think? How do you predict AI will transform businesses? Let us know.
