Version ‘zero’ on a brand new proprietary OLTP means there’s a long way to go for mission-critical transactional workloads
by, Marc Staimer
June 2022
Wikibon Unistore Overview
On June 14, at the Snowflake Summit 2022 in Las Vegas, Nevada, USA, Snowflake announced Unistore (currently in private preview). Snowflake claims that “For decades, transactional and analytical data have remained separate, significantly limiting how fast organizations could evolve their businesses.[1]” Apparently Snowflake is willfully unaware that Oracle’s converged database has integrated both transactional and analytical data on-premises in a converged database for more than forty years and in the cloud since 2016. And more recently with their very popular MySQL HeatWave database cloud service.
Unistore is Snowflake’s first attempt at unifying online transaction processing (OLTP) with online analytical processing (OLAP) in its database cloud service. This enables Snowflake to generally support analytics on the transactional data created only in Unistore.
To do this Snowflake introduced something they call hybrid tables that enable relatively fast single row operations. Since OLAP generally processes data in columns, hybrid tables open Snowflake to potential OLTP applications. The focus of Snowflake is to allow developers to build transactional applications in Snowflake’s platform. Unistore’s target market appears to be current Snowflake virtual data warehouse users planning to develop new OLTP applications.
Since its inception, Snowflake has consistently pushed the idea that they are a revolutionary virtual data warehouse cloud service provider in that they abstract the data warehouse from the underlying hardware infrastructure—and only that. Their objective has been to keep it simple. However, times, technology, competition, and market requirements change.
Snowflake appears to have finally recognized that there’s a major problem with being just a data warehouse cloud service. That problem is that nearly all the data in the data warehouse originates somewhere else. And most of that data comes from OLTP applications such as eCommerce. Getting that data into the Snowflake data warehouse requires extraction, transform, and load—or ETL. This is the process of copying the data from the OLTP transaction, massaging it, storing it separately, and placing it properly in the data warehouse. This is a painstaking process that takes time, resources, and money. It also makes it impossible to analyze transactional data in real-time. Unistore is Snowflake’s first attempt at a solution to this long-standing problem. Hence, their “MEA CULPA.” But based on Snowflake’s technical documentation and marketing literature, it comes up appreciably short of meeting market requirements and competitive offerings.
Why is the Current Snowflake Unistore Not Enough?
Snowflake’s announcement is by no means revolutionary, innovative, or “ushering in a renaissance of building and deploying a new generation of applications”, as they claim. It’s actually pretty late to the multi-model or converged database cloud services party. Cloud service providers such as Oracle have had converged database cloud services since October 2016 and converged databases longer than that.
So why isn’t this new Snowflake Unistore enough? There are several reasons.
First of all, Snowflake Unistore is a proprietary OLTP. That means no one, other than the private preview users, are currently using it anywhere else. There are no statements, documentation, or assertions that Unistore is compatible with any other OLTP database. That means users of MySQL, PostgreSQL, MariaDB[2], PerconaDB2, Microsoft SQL Server, Azure SQL, GCP Cloud SQL, Oracle Database, Oracle MySQL HeatWave, Oracle Autonomous Database, and other OLTP databases or database cloud services cannot just transfer their databases to Unistore. If they want to use Unistore and/or Snowflake’s virtual data warehouse, their data will have to be duplicated and go through an ETL process—and ETLs aren’t free. Conversely, OLTP applications developed within the Snowflake Unistore environment can’t run anywhere else.
Unistore is not a mature OLTP. OLTP is both mission- and business-critical. A nascent version 1.0 Unistore OLTP database will need time to work all the bugs out. Mission and business-critical applications are not very tolerant of bugs. There’s an adage that illustrates this issue—no one wants someone else learning how to shave on their face.
Keep in mind that Unistore, as of June 2022, is only available in private preview. This is what the industry ordinarily calls ‘alpha testing.’ It’s also not currently available for private preview in all regions. Currently, potential users can sign up for public preview when as stated in the previously referenced Snowflake blog post. Not many organizations would approve the use of an alpha-mode product for their critical business transactions. And considering that Unistore is not a new version of an established OLTP database, it’s a brand new proprietary OLTP database. Unistore is comparatively so green, it makes other databases in beta test mode look more like industry-hardened solutions.
Then there’s performance. Snowflake has not published any benchmark performance results or performance numbers of any kind. All Snowflake has said is that Unistore will use the same performance engine as the Snowflake virtual data warehouse. This is the same performance engine that per TPCH benchmarks trails badly, i.e., is much slower than Amazon Redshift, Microsoft Azure Synapse, and Oracle MySQL HeatWave.
It appears that Snowflake is learning to shave on those on their customers’ faces.
There are several other significant Unistore shortcomings. Snowflake talks about Unistore in a vacuum. It’s as if they are the first and only provider to solve this problem. They’re not, as previously discussed. More importantly the definition of “simple” has changed. It cannot just mean abstracting the database from the underlying hardware. Simply put, today, according to database cloud service users, databases must address the significant amount of manual labor-intensive tasks that DBAs have to perform. Many of these tasks are administrative and negatively affect productivity. There is little-to-no automation built-in Unistore. This is especially important now that they’re entering the OLTP world. One of the biggest time sinks for DBAs is OLTP performance tuning. This is a constant effort and takes up much too much time. Unistore does not currently address this evolution of simplicity.
Machine learning (ML) has become increasingly important to many IT organizations as OLAP has proven to not be enough. ML enables better predictions and faster decision making. It has become an essential competitive tool, making customers more nimble, agile, and flexible. But ML requires significant expertise from data scientists to use effectively. This is why many data warehouse cloud services are integrating and incorporating ML into their databases. Snowflake does not integrate ML at this time. They acquired a ML application vendor (Streamlit) that enables applications to lay on top of ML, but it is not ML. Snowflake also has some ML partnerships with Anaconda and DataRobot. However, partnerships are neither part of Snowflake, nor tightly integrated with their virtual data warehouse, and, of course, add significant cost.
What about cost? Snowflake has not yet specifically articulated a separate pricing structure for Unistore. However, they have implied that it will fall under the same pricing metrics as their virtual data warehouse. Under those circumstances, Unistore is a separate on-demand service based on Snowflake’s rigid hardware shapes of S, M, L, XL, 2XL, 3XL, 4XL, 5XL[3], and 6XL3. Charged by second per use. When set to turn off when the database is idle, then there are no charges. There is a brief lag as the database spins up from idle. If the Snowflake virtual data warehouse is any indication, Unistore will be quite pricey. To put that in perspective, Wikibon researchdetermined that Snowflake virtual data warehouses were significantly more expensive than Oracle Autonomous Data Warehouse, Oracle MySQL HeatWave, Amazon Redshift, and Microsoft Azure Synapse.
Unistore currently as is, trails badly in solving the everyday problems that database users and DBAs are facing and their peers are solving. A good example is the Oracle Cloud Infrastructure (OCI) database cloud services.
It starts with the Oracle Autonomous Database (ADB) for transaction processing (ATP), ADB for analytics and data warehousing (ADW), and Autonomous JSON Database (AJD). These autonomous database cloud services running on Oracle Cloud Infrastructure are all variations of the same database with a performance emphasis on specific use cases. Autonomous Database is a complete multi-model, multi-workload converged database including OLTP, OLAP, JSON (document), Spatial, Graphic, Time Series, XML (object), Blockchain, Machine Learning (ML), and others. All part of the same service working off the same data with no ETLs or data movement. So OLAP on OLTP data happens in real-time. That’s just table stakes. The key is the autonomy that eliminates nearly all administrative manual labor, freeing DBAs for more strategic work.
Oracle Autonomous Database is all about comprehensive database automation with automated database ops. It deeply integrates more than four decades of expertise, best practices, and machine learning into the ADB database cloud services. Administration expertise is no longer required by the database administrator (DBA) since it’s all performed by the ADB cloud service without human intervention. It’s easy to see why:
- Auto-provisioning
- Auto-deploys fully secure, fault-tolerant, and highly available mission-critical databases.
- Auto-configuration
- Automatically configures databases for optimal workload performance.
- Auto-indexing—this one is huge
- Continuously monitors all workloads suggesting or implementing index updates that will improve performance.
- Auto-scaling
- Automatically scales compute resources elastically up to 3X in real-time based on workload demand.
- Automated Security
- Always-on intrusion detection and encryption for all databases, backups, in-flight, and at-rest.
- Auto-Backups
- Ensures databases are automatically backed up daily or on-demand.
- Automated Data Protection
- Database-sensitive and regulated data are automatically safeguarded through the unified management console.
- Auto-Patching
- Applies patches and upgrades automatically with zero downtime or disruption. Especially important for vulnerability patching.
- Automated Hardware Failure Detection and Resolution
- Hardware failures are automatically predicted using ML pattern recognition, remediation taken, all without disruption, and with little-to-no performance impact.
- Auto-Failover
- Zero-data loss automated failover to a standby database.
Another powerful example of a multi-model converged database cloud service that solves many more problems than Snowflake Unistore is Oracle MySQL HeatWave.
MySQL HeatWave is targeted at customers that do not necessarily need all of the capabilities of ADB or are using MySQL today. It’s 100% compatible with MySQL. What makes it different from other MySQL database cloud services is that it has a tightly integrated built-in, in-memory OLAP engine, ML-based automation, ML, and ML model recommendation with minimal sampling, ML decision enlightenment, and real-time elastic scaling.
Oracle MySQL HeatWave benchmarks, publicly available on GitHub, demonstrate 44X better price performance than Snowflake—without the additional cost of Unistore. Additional Wikibon research ranked Snowflake virtual data warehouses at the bottom (#9) of data warehouses in performance while it ranked Oracle ADW as number 1 and Oracle MySQL HeatWave as number 2.
Conclusion
There are no compelling reasons or rationale to move OLTP applications to Snowflake Unistore (if that’s even possible). Nor has Snowflake provided any persuasive rationale for IT organizations to develop new OLTP applications on Unistore. Mostly Snowflake customers deeply entrenched with virtual data warehouses will find Unistore appealing. This is likely why Unistore appears to be targeted at current Snowflake virtual data warehouse customers.
The bottom line is that Snowflake is on Version ’0’ on all these features—and they take many years to mature. The fact that they are launching Unistore is an admission that specialized single-model Data Warehouse-only database cloud services are not the future. The future is the multi-model converged database cloud service that handles all workloads, data types, and development styles. Snowflake is just starting to introduce this functionality and pretentiously asserting they are an innovation leader. But the undisputed facts reveal they are decades behind. This is analogous to a broadcast TV station suddenly asserting they are the leader in streaming media a decade after all their customers have adopted it from other suppliers.
Two things remain constant about all cloud databases—increasing degrees of automation are being added every few months and convergence of more data types, workloads and models. Few cloud database cloud service providers have a roadmap calling for increased manual administration of their services or further proliferation of single-model database cloud services. Of course, there are exceptions like AWS. (But that is fodder for a future blog.)
Snowflake is evolving with Unistore and that’s a good thing. However, they have a long way to go. IT organizations looking to develop multi-model, multi-workload applications today with increased automation, and they should be, are better served by taking a hard look at Oracle’s database cloud services—it’s all built in.
[1] Introducing Unistore, Snowflake’s New Workload for Transactional and Analytical Data
[2] MySQL fork
[3] In preview.
