Wikibon’s 2018 True Private Cloud Forecast and Market Shares

With, Dr. Ralph Finos, David Floyer, Stu Miniman

Premise. The True Private Cloud (TPC) segment of the cloud market is a natural consequence of the global migration to cloud operating models. Data that can’t, or shouldn’t, be moved to public clouds will be computed on TPC platforms, which leverage and integrate with cloud technologies on-premises or near-premises. The TPC market will continue to accelerate, attracting all major cloud players, resulting in a highly dynamic market through the next decade.

Wikibon has been predicting the emergence of a distinct True Private Cloud (TPC) segment of the cloud market since 2015. Our reasoning was simple: As businesses adopted data-first business strategies – the basis for digital business transformation – they would begin organizing cloud-based business systems around natural patterns of data placement. Latency, bandwidth, IP protection, regulation, ingest/egest costs – all constrain cloud options. As we’ve stated, the long-term industry trend will not be to move all data to the public cloud, but to move the cloud experience to the data (see Figure 1 below).

For the last few years, this prediction generated disagreement among some cloud industry competitors. The reason? On the one hand, cloud service suppliers (CSP) argued that all data could move to public clouds. On the other hand, suppliers of on-premises systems continued to negate the cloud operating model, suggesting that customers really wanted to buy cloud-friendly technologies on traditional system types. The exception was Microsoft, which positioned Azure Stack as an on-premises extension of the Azure Cloud.

Recent True Private Cloud Events

That has all changed in 2018. Most major CSPs and systems companies have introduced products and services that give customers real choices regarding where to process data within a hybrid cloud operating model. While some of these approaches are nascent, all point to serious longer-term commitments to support TPC architectures. This is excellent for users, who shouldn’t have to make choices between adopting cloud operating models and exploiting data-first application architectures, like edge computing, AI, and others.

As a result, a robust and increasingly competitive TPC segment has emerged, is poised for significant growth, and will be a locus of technology innovation globally. As the technology foundation for TPC matures, the TPC market:

• Experienced strong 2017 growth. The TPC segment grew nearly 55% in 2017, bolstered by maturing TPC offerings and services. The VMware ecosystem, featuring the most robust technology base and strong partnerships with AWS, Google, and IBM public clouds, is leading TPC growth today.
• Offers real enterprise options. All large vendors of infrastructure products and services – including AWS, Google, and Azure – are moving to provide fully-functional TPC solutions. Enterprise demand for hybrid cloud capabilities means that only trumpeting public or on-premises cloud services is not viable long-term vendor option.
• Will be an epicenter of cloud growth and innovation. The cloud operating model is defined by public cloud service providers, but traditional systems vendors that make strong TPC commitments will help shape hybrid cloud technology stacks. Inter-cloud integration technologies that simplify operating hybrid cloud applications and supporting workloads (e.g., data protection) will be an especially keen area of invention.

TPC Experienced Strong 2017 Growth

In May of 2017, The Economist famously declared that data was the world’s most valuable resource; that “data is the new oil.” It’s a perspective that has been emerging for some time, but it fails to capture a central tenant of data-first, digital business transformations. Oil – and virtually any other non-data resources – is constrained by the economics of scarcity. Generally, oil can be applied to one use, in one place, at one time. The same oil cannot concurrently power your car and heat your home. That stands in stark contrast to data as a business resource. Data can be copied, combined, and shared at almost no cost; the same data can concurrently improve products and customer engagement. The economics of scarcity do not apply to data.

That doesn’t mean that data isn’t subject to rules. Among other things, data requires time to move and energy to sustain; latency and bandwidth dictate absolute constraints on what a business can do with data. Furthermore, privatizing data requires compliance with security, IP policy, and regulatory regimes. Taken together, these constraints ensure that all data won’t end up in one public cloud, but rather a multitude of distributed locations, each architected to compute local data according to the needs of local tasks.

But while data locality ensures a distribution of processing, enterprises increasingly demand a public cloud-like experience for all locations, both on-premises and edge. Essentially, this means that technology vendors must render products in cloud service terms: pay-as-you-consume; self-service console; software-defined resources services acquired via APIs; and advanced use of automation to keep administration complexity and costs at a minimum.

This combination of data realities and cloud experience ensures the health of the TPC market. As Wikibon has written, TPC provides the cloud experience where the data demands it. And the cloud-based, data needs of enterprises have been demanding. In 2017, the TPC market:

• Exceeded $20 billion worldwide. From a 2016 base of $13.1B, the 2017 TPC market grew to 20.3B, right in line with our expectations (see Figure 2). The TPC sector is roughly 40% the size of infrastructure-as-a-service (IaaS) sector and growing about 50% faster.
• Was strong in both on-premises and near-premises options. For the past few years, much of the TPC spending was on hosted private cloud services. These near-premises options continued to attract spending, growing 20% to $9.7B in 2017. On-premises TPC options, however, experienced hyper-growth in 2017, more than doubling to $10.6B.
• Sports strong brands at the top of the leaderboard. Dell EMC, HPE, and Cisco/NetApp hold the top three share positions in the on-premises TPC market (see Table 1), with Dell EMC enjoying a leadership 29% share. The hosted TPC sector is led by IBM, Rackspace, and DXC (see Table 2).
• Features healthy ecosystems. TPC solutions are most distinguished by their cloud software platform. The leading options are VMware, Nutanix, HPE, and Microsoft (see Table 3). Of these, VMware is the alpha dog, pulling through over $2.5B of software and hardware sales in the on-premises TPC market.

TPC Offers Real Enterprise Options

A variety of advanced, data-first application patterns will drive demand for TPC. These include internet of things (IoT), advanced analytics (especially use cases that use an enterprise’s sensitive information), social media, and applications that employ AI technologies to automate or augment operations and engagement. Together, using these applications, about 4 billion users are generating nearly 3 exabytes of data per day. An important portion of that data will move to public clouds, but the vast majority – we estimate more than 90% – will stay local or be discarded.

Recognizing this, every single global provider of infrastructure products and services is mobilizing to provide TPC options or support TPC options. In the last year, we’ve seen:

• Microsoft Azure, AWS, and Google Cloud are providing core and Edge platforms. Microsoft Azure was early to the TPC segment, but AWS and Google Cloud recently introduced native TPC platform options (beyond partnerships with VMware, for example). Microsoft had a clear business reason to embrace TPC: It’s huge installed base needed an on-premises cloud option. Now AWS and Google Cloud are entering the TPC fray, providing customers on-premises TPC platforms that can seamlessly integrate into their global public cloud offerings. AWS’s offering is limited – Snowball Edge effectively adds EC2 instances to their Snowball data transfer box – but Wikibon thinks it’s an important start. Google Cloud introduced GKE On-Prem and Knative to establish a TPC foothold. GKE – Kubernetes Engine – simplifies the process of setting up Kubernetes clusters on-premises, while Knative provides frameworks that simplify building, deploying, and managing Kubernetes container-based applications. Like AWS’s TPC entry, Google Cloud’s technology combo is a first important, if limited, TPC step.
• Systems vendors are introducing TPC technology and services. After years of pasting “private cloud” onto product literature, in 2017 and 2018 systems vendors actually started delivering products and services that can provide a true private cloud experience. Dell EMC use of VMware’s VMware Cloud Foundation (VCF) and IBM’s Softlayer have been strong TPC options for a while. More recently, Oracle’s “Cloud at Customer” and HPE “Greenlake” service initiatives have entered the market. Each of these options provides a true cloud experience, especially by providing a “pay-as-you-go” flexible consumption capability. Based on our conversations with users, we expect enterprises will embrace these options as part of their cloud strategies. Notably, OpenStack is not a widely adopted on-premises TPC platform but remains the basis for a number of successful hosted TPC services from CSPs.
• Serverless computing is moving into the development mainstream. Function-as-a-Service computing – colloquially called serverless – moves functions to data, presuming that for many application patterns it’s easier and cheaper than moving data to the function. An array of application frameworks for serverless computing are maturing. Moreover, storage vendors (and developers) are finding ways to add CRUD capabilities to otherwise stateless serverless frameworks. While not as robust and general as platform-oriented TPC solutions, serverless is an essential, lightweight approach for extending the cloud experience to the on-premises core and edge.
• Supporting technologies are being recast to support TPC. In addition to public cloud and TPC vendors, vendors in adjacent markets are adding product and service support for TPC. This includes security (e.g., Fortinet), data protection (e.g., Cohesity, Dell EMC, Veeam), and dynamic data integration (e.g., Informatica, Wandisco). As the TPC market – and especially the Edge – matures as a cloud platform, adding TPC capabilities to truly support hybrid cloud environments will be a requirement for all infrastructure-related vendors.

TPC Will Be An Epicenter of Cloud Growth and Innovation

As enterprises further embed complex, time-sensitive activities into operations and market engagement workflows, the demand for TPC will continue to grow. Wikibon projects the TPC worldwide market will experience a compound annual growth rate of 29.2%, reaching $262.4B by 2027 (see Figure 3). TPC growth will far outpace infrastructure-as-a-service (IaaS) growth of 15.2% – still very respectable! – over the same period. On-premises TPC is projected to be especially robust, growing at a 32.5% CAGR to nearly $176B in 2027.

As we stated in last year’s TPC report, this growth will be more than enough to sustain large systems companies, like IBM, HPE, Oracle, Cisco, and others, that have both excellent enterprise customer relationships and possess the means to transition their installed base to cloud solutions that truly are comparable to leading-edge public cloud services. The public cloud will not create a monopsony of just a few buyers capable of either vertically integrating hardware development and manufacturing or extracting business-threatening concessions from traditional infrastructure heavyweights.

Additionally, our prediction that enterprises would follow patterns for choosing TPC or public cloud options based on workload characteristics is unfolding. Workload will be the main determinant of public cloud and TPC choices (see Figure 5). Horizontal SaaS applications that don’t feature significant latency concerns, like most human-user, function-specific applications, will continue to grow in the public cloud, as will a range of big data and greenfield applications, so long as they don’t involve significant data movement costs. Industrial internet of things (IIoT), high-value legacy, and essential data (e.g., finance) applications will tend to move toward Edge TPC options. TPC also will attract greenfield applications that are likely to suffer onerous data communications costs unless sensible Edge data reduction policies are taken. Very importantly, as AI workloads become more prevalent, we believe training jobs will tend to run in public clouds, while inferencing applications will be placed on-premises or at the edge, often using TPC solutions.

Our expectation is that virtually all enterprise IT will be conducted utilizing a hybrid cloud model that will be dominated by SaaS and TPC, with IaaS playing an important role in engagement applications and big data services. This will lead to a new dynamic in the cloud business: how best to traverse cloud boundaries as applications increasingly are composed from networks of cloud services operating in different clouds. Cloud gateways will be a fix, but the pressure to choose cloud alternatives that can support applications end-to-end will be extreme. Microsoft and Oracle are best positioned to offer end-to-end SaaS and TPC today, but Alibaba, AWS, Google, Huawei, Tencent and others will strengthen their TPC offerings to keep pace across all points of enterprise presence.

One area where Wikibon will be especially focused in 2018/2019 is application migration. Migrating database-centric operational applications is extremely difficult. “Good enough” is failure. AWS and Google Cloud are touting services based on advanced ML and related technologies for simplifying migrations to public cloud-based database options. For Oracle (to Oracle’s application clouds) and SAP (to multiple public cloud options) customers, the transitions are made simpler by following paths specifically engineered by Oracle and SAP. Our advice? If an application, by virtue of its workload characteristics, can easily be re-platformed to the public cloud, it should eventually be re-platformed. Thus, users should let the large public cloud vendors assess migrations and offer options. At the least, you’ll gain important insight into what has to be done to modernize high-value operational applications.

Action Item

True Private Cloud (TPC) solutions are no longer aspirational. All major providers of infrastructure products and cloud services are mobilizing to provide TPC options to support workloads featuring data that doesn’t naturally fit in public clouds. CIOs should incorporate TPC into enterprise cloud strategies, selecting vendors that demonstrate strong TPC commitment, support for an advanced cloud operating model, and provide services to streamline transitions.

Appendix

True Private Cloud vs. Private Cloud

A “true private cloud” is distinguished from a “private cloud” by the completeness of the integration of all aspects of the offering, including performance characteristics such as price, agility, and service breadth.  Equally important is the nature of the relationship with the cloud supplier – namely a single point of purchase, support, maintenance, and upgrades (often referred to as a “single hand to shake, and a single throat to choke”).  The key benefit of true private cloud is that they provide solutions close to the cost and agility characteristics of public cloud in an on-premises deployment when business, security, and latency requirements dictate.

While aspects of the performance characteristics (i.e., virtualization) are broadly available today and characterized as “private cloud”, true private clouds are emerging as a distinct market sector – primarily from the hyperconverged infrastructure offerings (e.g., Cisco/NetApp Flexpod, Dell EMC Vblock & Vscale, HPE SimpliVity, HPE Synergy, Microsoft Partner ECI, Nutanix, Oracle Engineered Systems, etc).

While these products generally do not offer the totality of services today that we characterize as true private clouds, they are the vanguard of advanced building blocks along that path that Wikibon expects to rapidly evolve over the next few years.

Wikibon True Private Cloud Definition

As such, true private cloud should incorporate the following characteristics of public cloud:

• Significantly simplify the relationship between the user and the provider. That is, the user should have a relationship/transaction with a single provider, as is the case with public cloud.   This can be realized in one or more ways:
  • Through a foundation of hyperconverged systems (i.e.,Dell EMC VxRail, Nutanix, HPE SimpliVity, Pivot3, VMware vSAN, et alia) that are highly automated and managed as local pools of compute, storage, and network resources optimized to integrate the infrastructure as a single managed entity. We include cases where the system is converged but where Level 2 software support may reside with another provider.
  • Hosted managed private cloud (CenturyLink Private Cloud, Cisco Metapod, CSC BixCloud, Expedient Private Cloud, IBM BlueBox, Rackspace Private Cloud, et alia) would also be included in our definition. The chief characteristics of true private cloud in this case are it’s embrace of hyper-convergence as a service delivery vehicle and similar efficiencies for customers as provided by public clouds.
• Available on a self-service basis. Of course, the user can elect to pay for support from a provider if it useful, but there should be no requirement for permissions or provider intervention to use the service. Enterprises may impose rules of use of their own (say for access to hosted private cloud resources), but these would not affect how the user could interact with the true private cloud if they had license to.
• Allow users the flexibility to choose how IT resources are consumed – either “by the drink” or in a longer-term commitment. Whichever method they chose to pay their bills, the per-transaction cost of true private cloud should be transparent to the user.
• Designed to accommodate hybrid cloud application use cases.  We believe that effective hybrid clouds will require much greater commonality and convergence of hardware and software within the hybrid on-premises and cloud components.  As a result, a significant portion of the enterprise public cloud and on-premises true private cloud will be part of an integrated hybrid cloud. The most effective and highest function hybrid clouds will share common storage as well as hyperscale server and orchestration/automation layers between public clouds and true private clouds.  See the recent Wikibon report https://wikibon.com/hyperconverged-infrastructure-as-a-stepping-stone-to-true-hybrid-cloud/ for a more complete exploration of this emerging requirement.

Convergence & True Private Cloud

In terms of levels of convergence to qualify as true private cloud, Wikibon would expect to see automation and orchestration including:

• Cluster management
• Network automation and management
• VM/container automation and management
• Storage automation and management
• Application templates and deployment tooling
• Operations dashboard
• Workload analytics
• Capacity optimization
• Log management
• Root cause analysis
• Remediation tools
• Configuration monitoring and dynamic changing
• Proactive alerts
• Backup and replication services on premises or hybrid to other cloud services
• Snapshot management & catalog services

Managed/Hosted True Private Cloud

Managed/hosted true private cloud includes offerings such as CenturyLink Cloud Managed Services, Cisco MetaCloud, Dell Managed Cloud Services, IBM BlueBox, IBM Softlayer Private Cloud, Internap Dedicated Private Cloud, Platform9, Rackspace Private Cloud, Verizon Private Cloud, et alia.  These offerings, as far as the user is concerned, look and behave like a public cloud services except that under the covers, the resources are not shared across enterprises.

What Our Definition Does Not Include

Wikibon is taking a long term, disruptive perspective on the optimal end state that true private clouds should be reaching in the next decade to realize the full value of “cloud” if they elect to own their own cloud in their data center or via providers. As such the following are not included in our sizing or forecast of true private cloud.

• Converged systems with limited orchestration and automation, i.e. not meeting the true private cloud criteria above.
• Self-integrated private cloud or convergence involving numerous vendors. For example, Wikibon would regard the installation of VMware vSphere (or Microsoft HyperV) on new Dell EMC or HP servers and storage, and Cisco networking along with packaged orchestration and automation as “virtualization” but not true private cloud. Virtualization is a useful step, but ultimately falls short in terms of costs and supportability to the quantum gains that true private cloud can deliver. This extends to situations where users deploy VMware tools such as vSphere, vCloud Suite, etc. or Microsoft Cloud Platform System – but not as part of a single entity managed converged solution. Microsoft,  VMware, and other software providers, of course, participate in this market significantly as OEM providers to hosters of private clouds who take full responsibility for a true private cloud converged systems with infrastructure management and automation.
• Spending by service providers (AWS, Azure, Softlayer etc.) for their public cloud infrastructure whether self-built or acquired via a converged system. While these are advanced deployments and there is considerable activity in the Service Provider sector to build their own clouds to offer as a public cloud service, Wikibon’s perspective for this report is on what enterprises should be doing to duplicate Public Cloud experience, costs and efficiencies.  We cover this architecture in the report “Server SAN Readies for Enterprise and Cloud Domination“.
• Spending for cloud consulting, support and deployment, general data center outsourcing, co-location, and similar services – unless the service is to maintain and manage a private cloud for an enterprise.
• Virtual private cloud (Amazon VPC, Azure Virtual Network, CSC Virtual Private Cloud et alia).  These revenues are included in our public cloud figures now.