Hyperconverged Infrastructure as a Stepping Stone to True Hybrid Cloud

Wikibon Analyst

David Floyer

Premise

The migration from a traditional enterprise IT infrastructure architecture with separate servers, separate traditional SAN, and separate hyperscale public cloud to an integrated hyperconverged architecture including Server SAN and an integrated hybrid cloud is a profound change. Wikibon explores the premise that the migration is justified, and the optimum hybrid cloud strategy is to use the same architecture and software for both the on-premises True Private Cloud and the public cloud services. Wikibon uses the term “True Hybrid Cloud” to describe this approach.

Some of the terms in the premise may be unfamiliar. They are defined in the “Defining…” section below, with additional more detailed information in the footnotes.

Executive Summary

Wikibon tested the premise by defining a Hyperconverged reference engineered system, and talking to practitioners who had deployed it on-premises and as service providers. Both sets of users find the reference engineered systems easy to use, and confirm that it scaled well, and was suitable for service provider deployment.

Figure 1 shows the results of Wikibon’s detailed business case analysis. The traditional solution is 47% more expensive ($400,000) than the hyperconverged Server SAN approach. People costs are 600% higher.

Wikibon concludes that Senior IT executives should consider adopting an aggressive strategy for moving to a hyperconverged environment including Server SAN. Previous Wikibon research found that gateways between different on-premises and service provider deployments are expensive, and are an impediment to hybrid cloud functionality.

Wikibon recommends Senior IT executives adopt a True Hybrid Cloud strategy, and ensure where possible that the same hyperconverged infrastructure solution including Server SAN  can be run on premises and in the cloud, using the same hyperconverged technology and orchestration/automation software.

Defining Hyperconverged, True Private Cloud, Engineered Systems, Server SAN, Hyperscale and True Hybrid Cloud

Hyperconverged infrastructure is a system with a software-led architecture that tightly integrates virtualized compute, storage and networking resources with orchestration and automation. This  software is integrated on commodity hardware and supported by a single vendor, and is often the basis for private cloud deployments.

True Private Cloud is distinguished from private cloud by the completeness of the integration of all aspects of the offering,  including the performance characteristics (price, agility, service breadth). Most importantly it also includes a close relationship with the vendor, with a single point of purchase, support, maintenance, and upgrades (often referred to as a “single hand to shake, and a single throat to choke”). True Private Clouds are deployed on-premises when business, security, and latency requirements dictate.

Engineered System

Engineered systems are hyperconverged infrastructure offerings, and are deployed in True Private Clouds.  Examples of Engineered systems include Dell EMC Vblock & Vscale, Dell VxRack with ScaleIO, HPE SimpliVity, Nutanix, Oracle Engineered Systems,  etc). Engineered systems integrate server, storage, and networking, include automation, orchestration and management software, and provide users with solutions close to the cost and agility characteristics of public cloud. Engineered systems also includes a close relationship with the vendor, with a single point of purchase, support, maintenance, and upgrades, a “single hand to shake, and a single throat to choke”. The key benefit of engineered systems is to move undifferentiated IT work from the enterprise to the vendor, and reducing the cost and complexity of managing enterprise IT.

Hyperscale architecture can scale appropriately as increased demand is applied to the system, and enables the seamless addition and provisioning of servers, memory, networking, and storage resources. Hyperscale computing is required to build very large and scalable clouds. Hyperscale is differentiated from hyperconverged infrastructure as being built and deployed by a single large cloud or application service provider, e.g., Amazon, Facebook, Google, IBM and Microsoft. Smaller cloud vendors can use hyperconverged infrastructure to achieve similar economies of scale.

Server SAN is a storage component of hyperconverged or hyperscale infrastructure, and is implemented as a pooled compute and storage resource comprising more than one storage device directly attached to separate multiple servers. Storage is directly attached to the servers via a high speed interconnect, such as Ethernet, InfiniBand, or Converged Ethernet with RDMA (RoCE). Data coherency and data services are managed by software running directly on the infrastructure servers.

• Hyperconverged Server SAN is usually configured in enterprise applications to ensure high availability and rapid recoverability. Examples of Hyperconverged Server SAN vendors are listed in Wikibon Server SAN research, and include Nutanix, VMware VSAN, Dell EMC ScaleIO, HPE VSA, SimpliVity (now HPE), DataCore, Nexenta, Pivot3, Scale computing and others. Enterprises and other cloud service providers would be the major deployers. Wikibon expects that most cloud service providers will offer Hyperscale Server SAN services (e.g., VMware is planning to offer VMware and VSAN as a cloud offering within the AWS cloud).
• Hyperscale Server SAN as similar in architecture, with the differences being the focus on very large-scale deployments with very few staff. The major cloud service providers mentioned above and many smaller ones are deployers of hyperscale Server SAN, integrating and maintaining their own software, open source software and ISV software.

True Hybrid Cloud is a combination of on-premises True Private Cloud combined with a cloud service provider that is offering the same Hyperconverged software and hardware. Examples of this could be:

• VMware offering the same service in the AWS cloud as well as on-premises;
• Oracle offering the Oracle Cloud machine on premises with the same software and hardware as deployed in the cloud;
• Dell EMC offering Hyperconverged VxRack with ScaleIO to both a service provider and an enterprise customer.

Hyperscale and Hyperconverged Storage Infrastructure

One of the most entrenched components of traditional infrastructure is the storage SAN, built on proprietary code running mainly on storage controllers where the drawback is the high cost of replacement and migration.  Wikibon defined and began forecasting an alternative,  Server SAN, in April 2014 in a research publication called “The Rise of Server SAN”.   In that report we projected that traditional enterprise storage (SAN, NAS and DAS) would be largely superseded by Hyperscale Server SAN and Hyperconverged Server SAN.  Wikibon projects that Hyperscale Server SAN will be seen mainly in the public cloud sector of the enterprise storage market and Hyperconverged Server SAN in the enterprise on-premises sector. At the same time, Wikibon expects there to be many cases where service providers and SaaS cloud providers will use hyperconverged solutions  while some large enterprises will use on-premises hyperscale solutions.

Figure 1 below is derived from the Wikibon research and projections entitled “Server SAN Readies for Enterprise and Cloud Domination“.  It shows Wikibon’s 2016 projection of the Server SAN market from 2014 through to 2026.

Hyperconverged System Architectures

This section details the hyperconverged architectures and available technology components that enable these hyperscale architectures to span the public cloud and True Private Cloud components to create True Hybrid Clouds. Figure 3 describes the fundamental system architectures for servers with Traditional SAN, Hyperconverged systems with Server SAN, and upcoming Hyperconverged systems. This research focuses on illustrating the business case for infrastructure deploying hyperconverged Server SAN (e.g. Dell EMC VxRack with ScaleIO, middle of Figure 3) over systems with traditional SAN (e.g., traditional systems from Dell EMC, IBM, HPE, etc, left-hand side of Figure 2). Future Wikibon research will look in more detail at upcoming Server SAN hyperconverged and hyperscale architectures (right-hand side of Figure 3).

Traditional Servers, Network and SAN Architecture

The traditional SAN architecture in Figure 2 has the following components

• Server X86, separate from the storage layer which also uses X86 architecture in the SAN controllers.
• Server DRAM, which again is separate from the storage layer DRAM in the SAN controllers.
• 4/8/10 Gb/sec (mainly Fibre Channel) connections between the enterprise servers and SAN storage, with some 16/40 Gb/sec availability.
• Fibre Channel or iSCSI SAN separate from the server layer.
• FC or iSCSI switching between servers and storage SAN.
• Flash storage, which is offered primarily as a flash cache layer. Some progress has been made towards flash-only storage environments, often with a flash cache layer with a more durable flash technology.
• HDD, the fundamental mechanical storage technology. Traditional SAN was developed to mitigate the high latency and low bandwidth of HDD, and share storage between servers.
• SAN data services, which are mainly focused as proprietary software in the traditional SAN controllers.

In general, service providers and public clouds do not use this traditional separate server and SAN architecture because:

• Both equipment and software costs are too high, and often do not have features required by service providers.
• SAN storage hardware and software is separate from the server and has to be managed separately.
• Separate server, storage and network teams are inefficient, inflexible, and do not scale well.
• The architecture does not enable modern, very low latency applications.
• The architecture inhibits integration between on-premises clouds and public clouds/service providers.

Hyperconverged with Server SAN Architecture

The Server SAN architecture components in Figure 2 are based on a reference engineered system which, in Wikibon’s opinion, is able to effectively meet both enterprise public cloud and True Private Cloud requirements in a generalized application environment. The reference engineered system chosen for this Server SAN is based on Dell EMC’s ScaleIO Server SAN software integrated with Dell EMC VxRack hyperconverged technology.*

• Server X86 within the VxRack hypersconverged architecture, which runs the Server SAN, applications and orchestration/automation layers.
• Server DRAM, which services storage, network, and applications.
• 10Gb/sec interconnect fabric, which connects storage and servers.
• Proprietary IO Protocol running on the 10Gb interconnect to reduce traditional SCSI IO protocol overheads and limitations.
• CISCO interconnect to allow virtualization of the server layer and orchestration of updates to X86 server microcode updates and patches, as well as updates to Server SAN software.
• Commodity flash drives,  as cache and/or as a direct storage layer.
• Commodity HDD, with the ability to use server DRAM instead of SAN controller DRAM, and striping to manage latency and bandwidth for mechanical storage devices.
• Storage Data Services at the server level, rather than at the traditional SAN controller level.

This reference engineered system is used to compare Traditional SAN with Server SAN in the business case illustrated below.

*Other potential reference engineered systems candidates include Microsoft Spaces, Nutanix, Oracle, VMware and others.  The Microsoft Spaces solution based on the Microsoft Azure Stack is not yet generally available. Nutanix and VMware are software solutions at the moment.  A key value of a single vendor converged approach is a “single hand to shake and single throat to choke”, which is more difficult to evaluate at the moment for Nutanix and VMware. VMware’s partnership with AWS should allow VMware to offer such solutions in the future. Dell EMC and Oracle both offer identical on-premises and service provider solutions. Oracle’s focus is usually on high value compute application environments with very high software costs, whereas this research is focused on a more generalized application business case scenario, which may not be entirely applicable.

Upcoming Hyperconverged Architectures

Future Wikibon research will evaluate upcoming hyperconverged  architectures, an emerging and exciting architecture using:

• NVMe and NVMe over Fabric protocols together with high-speed 100Gb interconnect technologies and advanced offloaded protocols (e.g., Mellanox);
• High performance flash storage devices (<50μ);
• Very large DRAM
• FPGA and GPU support for storage, network and other services;
• New coherent memory protocols such as Open Foundation CAPI or Nvidia’s  NVLink.

This scenario will be of great interest in low-latency high application value environments.

In addition Wikibon will also be looking at lower-cost converged ARM technology-based solutions, which will be of particular interest in IoT deployments.

The Business Case for Hyperconverged

Business Case Executive Summary

This business case compares a traditional server and traditional SAN solution with a hyperconverged Server SAN solution, in a generalized application space in the context of a future requirement to support a hybrid cloud architecture. The Server SAN reference engineered system used is Dell EMC’s ScaleIO on their VxRack hyperconverged infrastructure.

Figure 4 below shows the 3-year cost benefits of the Server SAN reference engineered system vs. a traditional white-box racked servers together with a Traditional SAN based on a storage array.

The traditional solution is 47% more expensive than the hyperconverged Server SAN approach. Storage costs of the traditional approach are 100% higher, with the server, DAS and network costs being 20% lower.  More importantly, people costs were 600% higher. The time to value (e.g., to a working system) for Server SAN was 6 times faster, and the ease of incremental upgrades much easier to implement. Overall, the solution is $400K less expensive, while also preparing the way for enterprises to deploy True Hybrid Clouds with identical architectures in both an on-premises cloud and external service provider cloud.

Business Case Methodology and Detailed Results

Wikibon’s methodology for analyzing business case costs leverages our strong database of detailed benchmark values for the traditional white box and storage array side of the equation. This data has been developed over >10 years, and is based on thousands of interviews with IT users. The reference engineered system costs are developed based on interviews with enterprise IT and service providers using the Dell EMC VxRack engineered system. Some of these enterprise names were provided by Dell EMC, and some from Wikibon’s own contacts. The detailed results of the comparisons are shown in Table 2 and Table 3 in the footnotes below.

Figure 5 below shows these comparative costs in detail. The costs for the traditional white box and storage array environment are skewed towards storage costs, and server/network/DAS costs are higher in the hyperconverged Server SAN approach. The people costs in this generalized application environment are low in both scenarios, but costs are much lower in the hyperconverged Server SAN environments, an observation supported very strongly in our user interviews with those who deployed the reference engineered systems.

Figure 6 shows the elapsed time to create the infrastructure environment. The time to configure, install and test the traditional solution is 33 days, against 6 days for the Server SAN approach. The detailed breakdown of the days required and assumptions are shown Table 2 and Table 3 in the Footnotes below. Application development and application software costs were not included in this analysis.

Business Case Business Case

The 3-year business case is shown in Table 1 below. The initial investment cost is much lower for the Server SAN solution, so that the traditional ROI, IRR and breakeven metrics are not relevant. The net present value (assuming a 5% cost of money) of choosing the Server SAN approach is $400K.

The monthly cumulative cash flow analysis is shown in Figure 7 below. The red line on the top shows an initial investment in the first month for the traditional solution of $734K, with maintenance and operational costs increasing these costs to $1.3 million at the end of three years. The Server SAN solution shows a much lower initial investment cost in the first month of $405K, with incremental equipment cost increases at the end of year 1 and year 2, and lower maintenance and operational costs leading to a total 3-year cost of $865K. The end users we spoke with strongly confirmed the ease of upgrade of the solution.

True Hybrid Cloud

In our research, Wikibon interviewed to service providers who had installed the engineered systems  (VxRack and ScaleIO) defined earlier. The service providers found the same ease-of-use characteristics as the enterprise users. In addition, the service providers found that solution scaled very well indeed, and is competitive with hyperscale solutions.

In the Wikibon research report entitled “The Challenges of Building Hybrid Clouds“, Wikibon tested the premise that effective hybrid clouds will require much greater commonality and convergence of hardware and software within the hybrid on-premises and cloud components.   This research concluded that IT senior management should work with vendors that offer hybrid cloud solutions with as much commonality of architecture and hardware/software components as possible across the different cloud components.

Wikibon has also projected the size of the different cloud markets in recent research entitled “Cloud Vendor Revenue Projections 2015-2026“.  This report shows the emergence of both public SaaS and IaaS clouds, and on-premises True Private Clouds. Together these are projected to account for about 50% of the enterprise IT market in 2026. A key driver for this migration is the projected reduction in enterprise spend on operational IT staff support of 50% between 2016 and 2026.

Bottom Line: The overall conclusion of this research program is that a significant portion of the enterprise public cloud and on-premises true private cloud will be part of an integrated hybrid cloud. The most effective and highest function hybrid clouds will share common storage as well as hyperscale server and orchestration/automation layers between public clouds and True Private Clouds.

Research Limitations

The major limitation of this research is the fact that for most enterprise IT teams, hybrid cloud is more an aspiration than a concrete strategy and architecture. There are very few enterprises that are planning a hybrid cloud in detail, other than using the public cloud for basic services such as development, backup and recovery, and archiving. Wikibon’s hybrid cloud conclusions are based more on theoretical discussions and analysis of cost, rather than end-user input from those who have implemented a True Hybrid Cloud. We explored some of our assumptions and details in the aforementioned Wikibon report  “The Challenges of Building Hybrid Clouds“. While Wikibon believes the future benefits of hybrid cloud are important, these potential benefits have not been included in the financial analysis and business case above.

The other area not addressed is this research is  the potential of new Server SAN infrastructure to enable improved application value. The removal of the architectural layer improves IO latency and bandwidth. This can significantly benefit both end-user response time, and create additional application value by being able to handle application extensions that process much more data. The potential value to the lines of business is very real, but not included in the financial analysis and business case above.

The value to the IT operational team has been included in our business case, and is confirmed by the high degree of overall satisfaction reported with the engineered systems by IT staff.

Overall Conclusions and Recommendations

All but one of the practitioners interviewed by Wikibon stressed how easy the engineered system is to upgrade, how well it performed, and how easy it was to manage. All but one practitioner found the solution easy to install. Migrating to a hyperconverged Server SAN infrastructure is a no-brainer strategy, which:

• Offers lower equipment and operational costs.
• Offers a lower latency and faster bandwidth infrastructure, which will increase choices for improving application value for current application maintenance and updates.
• Puts the infrastructure on a path for modern new internal application or ISV packages with much larger data sizes and data flows that can tackle new operational, decision support, and AI and big data applications that will enable digital transformation.
• Ensures that, where possible, there is a “single hand to shake and a single throat to choke” for each distinct solution deployed. This may mean negotiating with vendors to include other smaller vendors as OEM partners within a total solution.

There may be some complex infrastructure and operational processes with high migration costs or long depreciation on IT assets, but these should be isolated and given minimum future investment.

For larger-scale deployments of infrastructure, Wikibon found the engineered system solution (Dell EMC ScaleIO in their VxRack infrastructure) to be a strong contender for both enterprise and cloud providers, and recommends it be included in any evaluation for on-premises or service provider solutions. The business case for deployment is strong both for overall savings and strategic positioning.

When architecting IT infrastructure to allow flexible hybrid cloud solutions, Wikibon believes it is preferable (where possible) that the same hyperconverged Server SAN solution be run on premises and in the public cloud, using the same storage technology and orchestration/automation software. This will minimize cost, provide much greater flexibility in data placement, and be a sound basis for future additional hybrid cloud functionality.

Wikibon expects many vendors will start to compete for hyperconverged Server-SAN based infrastructure in both on-premises and cloud service environments. These include current hardware vendors (including Dell EMC, HPE, Oracle and others), SaaS cloud vendors (Microsoft, Oracle & others), IaaS cloud vendors (including AWS and Microsoft Azure), and new vendors (such as Micron and others).

Wikibon also expects that Hyperscale Server SAN service providers will offer an on-premises version of their software and hardware, as a basis for developing more sophisticated True Hybrid Cloud functionality, availability, and flexibility. Microsoft has already announced the Microsoft Azure Stack, and Wikibon expects that AWS will follow suit.

Action Item

Wikibon strongly recommends that Senior IT executives adopt an aggressive strategy for moving to a hyperconverged Server SAN environment. Wikibon further recommends Senior IT executives adopt a True Hybrid Cloud strategy, and ensure where possible that the same hyperconverged Server SAN infrastructure solution can be run on premises and in the cloud, using the same hyperconverged technology and orchestration/automation software. 

Footnotes:

Table 2  below shows the detailed calculations within the business case in Table 1 above. Table 3 below shows the detailed assumptions behind the calculations in Table 1 and Table 2.

Footnote ¹ – Server SAN Definition

Wikibon defines:

1.  Hyperconverged Server SAN as pooled compute and storage resource comprising more than one storage device directly attached to separate multiple servers (more than one). Communications between the direct attached storage (DAS) occurs via a high speed interconnect (such as InfiniBand or RoCE on Low Latency Ethernet), where data coherency is managed by the software of the solution. Server SAN multi-protocol storage can utilize both spinning disk and flash storage. Enterprise Hyperconverged Server SAN is usually configured in enterprise applications to ensure high availability and rapid recoverability.
2. Hyperscale Server SAN as similar in architecture, with the differences being the focus on extremely large, low-cost, and lower performance deployments with very few staff. In many deployments of Hyperscale Server SAN, open source software is used or developed. IBM, Google, Facebook, and recently Microsoft have be contributing back open-source software improvements and projects, on the basis that adoption will be improved by future additional ideas and development: in addition a maintenance/development costs could be lower for the vendors.