Five Years Into the Flash Wave
The storage industry is famous for being rather slow to adopt change. The creation of data continues to grow at a staggering rate leaving IT organizations struggling to keep up. It has been about five years since the wave of flash storage started sweeping across the enterprise (Fusion-io started shipping in 2007 and EMC added enterprise-class flash to storage arrays in 2008); it is still early days in the adoption of flash technology. Over these last five years, the cost of flash has dropped, driving the creation of a variety of new architectures that are built to take advantage of flash. The pace of innovation and disruption to existing solutions will accelerate over the next five years.
Flash in the Storage Array
When flash storage was first introduced into storage arrays, the cost justification required for a few flash drives was rather complicated. While flash offered an order-of-magnitude higher IOPS and used significantly lower power, only certain use cases could justify the deployment. Configurations typically consisted of 3%-5% of overall capacity as flash, which would be used only as cache. Wikibon has been looking closely into how Hybrid Storage Poised to Disrupt Traditional Disk Arrays. These hybrid solutions use a higher percentage of flash (typically 20%-40%) and dense SATA (no SAS) for capacity.
While the hybrid solutions all differ from previous systems that tiered or added flash into existing arrays, they have varying designs. One of the key metrics for flash-based solutions is the hit rate (the percentage of reads that are retrieved from flash at very low latency instead of having to be retrieved from much slower disk). According to a published report, Nimble storage delivered 75% hit rate for a VDI deployment, which is even higher than the baseline for hybrids that David Floyer lists in his [of hybrid vs. traditional arrays]; it is also assumed that VMware environments have a negative impact on hit rate due to the inherent randomness of IO.
Tintri’s hybrid architecture was specifically designed for the VMware market, and according to Tintri architect Ed Lee it can get over 99% hit rate with its flash-first architecture. Tintri sends both reads and writes to flash, providing an experience similar to all-flash solutions at a lower price point. Overall, the superior response time delivered by hybrid arrays allow users to add more VMs/core and/or to purchase fewer cores for databases. As Floyer points out, large environments with high IO demands will need to consider all-flash arrays, and the high performance array market is shifting towards hybrid and all-flash arrays.
Server-based Flash Unbounded
While even the best external arrays measure latencies in milliseconds, server-based flash solutions deliver response times in microseconds. One of the top drawbacks to server-based options has been that just like DAS (direct-attached storage), storage is “captive” to a single server. A new generation of software-led solutions is coming to market that extend the reach of the PCIe and SSD storage in the server. The first announcement in this new class of solution was QLogic’s Mt Rainier that leverages the adapter’s driver to seamlessly make server-based flash part of the SAN.
PernixData has come out of stealth with its “Flash Virtualization Platform” (FVP), which embeds software in the VMware hypervisor that enables server side flash to be clustered and scale-out. PernixData’s founders include Satyam Vaghani, the creator of VMFS (VMware’s clustered file system), and Poojan Kumar, who led the development of Oracle Exadata and VMware vFabric Data Director. PernixData aims to not only increase the utility of server-based flash, but also to accelerate the performance of external storage without any modification. Since the technology is built into the hypervisor, it can leverage the full ecosystem of VMware HCL supported storage solutions (host-based and external arrays).
Howard Marks recently commented “In a virtual world, the VM has to be the unit we work in. Virsto (recently acquired by VMware), Tintri, and SimpliVity get this. Hopefully the rest of the industry will follow.” PernixData definitely does.
Finally, Virident has brought to market FlashMAX Connect Suite software, which extends the functionality of its PCIe server-side flash storage solutions across servers and with the SAN. All of these solutions are currently pre-GA, but this rapid series of announcements portends both the maturation and rapid expansion of an already crowded flash storage marketplace.
Action Item: Just as VMware looked to bring greater efficiency to servers, both hybrid and software-led server-based flash solutions hold the promise of delivering greater storage efficiency to data centers. CIOs should examine the numerous use cases where hybrid storage solutions are being deployed. The enhanced offerings for extending server-based solutions are not yet ready for production but should be on the radar for testing this year. Users should start research since the rate of innovation is faster than the average enterprise storage refresh cycle.
