ABSTRACT: This theCUBE Research Analyst Brief on the TCO benefits of moving from the Intel-based Oracle Exadata X7-2 to the new AMD-based Exadata X10M, reflects analysis from interviews with Oracle subject matter experts, a review of Oracle documentation and independent TCO modeling by theCUBE Research. We find that relative to the X10M, the previous generation X7-2 demonstrates between 72% and 85% higher TCO over a fiveyear period. These improvements result from the newer processor architecture with significantly more cores per database and storage servers and faster per-core processing based on 4th-generation AMD EPYC™ CPUs. This enhancement, enable more work to be done in a significantly smaller and energy consumption footprint. In addition, our research found substantial non-monetary benefits to be achieved which we will outline in this analysis.

Introduction

The Oracle Exadata X10M is the latest version of Oracle’s engineered system that is purpose-built to to optimally run Oracle Database. Released in 2023 and incorporating AMD EPYC processor throughout, it represents a meaningful step-up in performance, efficiency, and cost-effectiveness. This analysis looks at the total cost of ownership (TCO) of the Exadata X10M and compares it to an Exadata X7-2 purchased in 2017. The analysis process would be substantially the same when looking at the relative TCO for Exadata platforms that predate or follow the X7-2.  Factors we consider in this analysis include:

• Infrastructure costs – including server and storage
• Data center-related costs, most notably power and cooling
• Differences in staffing for administration and management 
• Costs of data migration
• The cost of operational interruptions

While a detailed comparison of the technical specifics between the Exadata X7-2 and the Exadata X10M is not the goal of this analysis, information on the advancements across the history of Exadata platforms is included in Appendix A. The Exadata platform has made increasing levels of performance and scale available to Exadata customers over the years, and new capabilities such as the addition of flash caches, XRMEM caches, and low-latency RDMA over Converged Ethernet (RoCE) should help organizations achieve lower data access latencies and higher throughput for both online transaction processing (OLTP) and analytics using data warehouses. Oracle’s recent adoption of AMD EPYC processors for both database servers and storage servers is a significant factor in the current phase of this evolution, and we believe it enables an improved level of performance for the broad range of Oracle Database workloads in a smaller hardware footprint.

Most organizations run a mixture of online transaction processing (OLTP) and analytics workloads. Over time, most organizations will look to increase their OLTP throughput to support larger and an increased number of workloads. Many organizations are looking to analyze larger data sets, to provide real-time business insights, and are seeing this as feeding new AI and ML initiatives.  As technology and expectations evolve of both internal and external customers, many organizations experience a need to refresh existing systems with newer technologies.  While it is impossible to quantify these benefits in a traditional TCO analysis, they are critical factors that organizations must consider.  Most organizations replace critical infrastructure IT infrastructure, such as Exadata, every five to seven years.  This report illustrates how Exadata X10M can reduce the complexity and cost of providing Oracle Database capabilities in an organization’s data centers. 

As illustrated in the following diagram, a 5-year TCO analysis of upgrading Exadata infrastructure from an X7-2 used for OLTP workloads shows that the older generation system is 85% more expensive over a five-year period. Underpinning the X10M’s TCO advantage is a system that supports OLTP workloads with 5.6 million SQL Read IOs per second, and more than 14 times better IO latency shows relative to the earlier generation Exadata systems. A similar analysis for X7-2 and X10M configurations that support roughly 250 terabytes of usable storage capacity shows slightly lower benefits but still shows the previous generation is 72% more expensive than the X10M.  

It’s important to note that these findings are based on standard business scenarios using the data and assumptions outlined in this report. When considering your specific requirements, you should decide based on your unique needs and circumstances. In most cases, upgrading provides excellent value for the business, but this may not be true in all cases.

Common enterprise use cases for Oracle Exadata

Exadata is optimized to support all Oracle Database workloads. It combines optimized database software with ideal database hardware and built-in automation. 

1. OLTP: Oracle Exadata is designed to provide low IO latency and high levels of parallelism, allowing it to excel in processing business-critical transactions at scale.  Recent optimizations in the Exadata architecture allow database servers to directly read data in storage server caches. According to Oracle, Exadata X10M can accomplish this in less than 17 microseconds, compared to the 250 microseconds required on X7-2 systems.
2. Analytics with data warehouses: Exadata’s architecture is also optimized for performance when analyzing large volumes of data stored in data warehouses. It accelerates data warehouse queries and analytics by offloading data and compute-intensive processing to storage servers where SQL queries are processed, returning only results to the database server. This architecture allows Exadata to analyze large amounts of data on the storage servers where it resides, eliminating the overhead of moving it to database servers.
3. Machine learning (ML): Similar to analytics, Exadata can offload many ML queries to storage servers, accelerating model building without overloading database server CPUs.
4. Business-critical applications: Workloads such as ERP and CRM often combine OLTP, analytics, and ML workloads. Organizations count on these applications to be continuously available. Exadata offers built-in fault tolerance that enables databases to continue to run when individual components or servers fail. Technologies in additional Oracle products, such as Active Data Guard and GoldenGate, can provide cross-instance and cross-site replication that enables robust business continuity and disaster recovery scenarios.
5. Database consolidation: Most enterprises run multiple Oracle Database instances, and those that use Exadata typically run tens, hundreds, or, in some cases, thousands of databases on a single platform. Organizations could acquire an Exadata system to support critical workloads and then find that they can save money by supporting additional workloads on the same platform.

Most enterprises support multiple of these workloads on the same Exadata, so each enterprise needs to evaluate its own mix. However, OLTP and analytics with data warehouses are the cornerstone workloads for many organizations so we have focused our TCO analysis on them.

Benefits of Moving to New Exadata Infrastructure

Exadata’s scale-out architecture enables organizations to combine the amount of compute and storage resources needed for their unique set of workloads. Each generation of Exadata infrastructure builds on this architecture with advancements in processors, storage, memory, and networking. These improvements enable organizations to do significantly more with the same number of database and storage servers, reduce the size of their systems, or a combination of both. 

Exadata X10M uses 4th-generation AMD EPYC processors with 96 cores each, which means that each database server has four times the number of cores than on an X7-2 and three times the number found in the previous-generation X9M systems. These servers also have up to twice the memory of previous Exadata database servers and significantly more memory bandwidth. The combination of more cores, memory, and memory bandwidth means that you need fewer database servers to support a given number of concurrent users and workloads.

In addition, the AMD EPYC processors in Exadata X10M storage servers have more than 3 times the number of cores found in X7-2 systems and twice that found in X9M generation systems. As a result, Exadata X10M storage servers can offload significantly more SQL queries from database servers to process those requests faster, enabling organizations to develop deeper insights into their business.

An added benefit to the new Exadata infrastructure that isn’t captured in the TCO analysis is that the AMD EPYC processors have extremely fast encryption and decryption coprocessors that enable encrypted data to be processed with virtually no overhead. These embedded capabilities help organizations can improve their security postures without impacting database performance.

Between the X7-2 and its earlier versions and the X10M, the network interconnect technology was upgraded from 40Gb Infiniband to 100Gb ROCE-based Ethernet with two active connections per server.  This high level of bandwidth reduces the time it takes to transfer data between database and storage servers, while the RDMA-based IO running over the Exadata X10M’s dedicated networking dramatically reduces SQL Read latency and reduces contention for system-wide locking of relational database data. This critical technology enables individual databases to use compute resources from multiple servers to increase performance and database availability.

While the Exadata X7-2 was a powerful and versatile database platform when it was introduced in 2016, Exadata X10M provides substantial upgrades that enable organizations to derive greater value from their business data while reducing costs.

TCO Comparison Model

Our TCO analysis compares the Oracle Exadata X7-2 and Oracle Exadata X10M. With the assistance of subject-matter experts from Oracle, we configured these systems to offer similar levels of capabilities that are key to success in two different scenarios. These platforms and configurations would be recommended if an expert from Oracle were tasked with meeting specific workload requirements. Of course, your requirements will be different, so it’s important that you work with an Oracle Exadata expert. Given your mix of workloads, you may see different levels of benefit when moving to Exadata X10M, and they could be much higher if you cannot just swap out your old Exadata for a new one but also consolidate more workloads from other platforms. 

A note on database licensing costs and why they are not included as part of our TCO model: Oracle Exadata software licenses and Oracle Database software licenses act like perpetual licenses. If the license acquisition cost had been added to the X7-2, only the maintenance would have been applied to the X10M configuration, resulting in increased savings for the X10M TCO as it would only include maintenance.

The particulars of the Exadata X10M

The Oracle Exadata X10M Database Machine is designed to meet modern data processing demands across the full spectrum of Oracle Database workloads. Due to their scale, these engineered systems handle higher OLTP transaction throughput and increase parallelized analytic workloads than previous-generation platforms.

Other features and capabilities include:

1. High-performance database servers
   1. Each database server comes with two 4^th generation AMD EPYC processors and a total of 192 processor cores. These processors have a base frequency of 2.6 GHz, which can boost up to 3.7 GHz, providing highperformance processing for core database operations. With database servers, you have a choice of memory sizes, with 1.5 TB, 2.25 TB, and 3TB per server.  The database servers use the XRMEM and flash capacity as additional low-latency caches for database data, accessing over a RoCE network with less than 17 microseconds of latency.
2. Flexible storage servers:
   1. High Capacity (HC) storage servers integrate 12 hard disk drives with 22TB of capacity each. Data is replicated across three storage servers that each provide 80TB of usable “High Redundancy” capacity per server. Each storage server incorporates 27.2TB of flash and 1.25 TB of extremely fast Exadata RDMA Memory, automatically moving data between disk, flash, and XRMEM to optimize performance. With Exadata X10M, each storage server has 64 cores worth of processing power for SQL processing.
   1. Extreme Flash (EF) Storage Servers implement an all-flash storage model with the 122.8TB of PCIe flash replacing the hard disk drives found in the HC storage servers. With triple redundancy in storage servers, each EF storage server adds roughly 37.2 TB of usable capacity. High-performance flash and XRMEM continue to be used to optimize storage performance. 64 cores are also available for SQL processing.
   1. Extended Storage (XT) Servers are primarily used for in-database archiving of data using Exadata Hybrid Columnar Compression. They feature that same disk drives as HC servers, but do not have the flash or XRMEM caching tiers and only have 32 cores worth of processing power.
   1. In the context of storage servers, it’s important to note XRMEM’s impact on performance. Incorporated in HC and EF Storage Servers, XRMEM boosts performance by delivering up to 2.8 million 8K OLTP Read IOPS per storage server—nearly a 5x increase compared to the Exadata X7-2 and a 21% increase over the previous generation X9M system.
3. Scalability
   1. Exadata X10M systems start out with two database servers and three storage servers. This architecture is designed to provide very high performance and availability.  Database servers and storage servers can be scaled independently to meet your aggregate workload requirements.
   1. Each Exadata X10M system can have dozens of database servers configured in scale-out clusters. With such configurations, you can grow compute infrastructure to have from 384 to more than 6,000 CPU cores and 100 TB of memory.
   1. Storage servers can be independently scaled relative to compute servers, enabling a single system to have more than 5PB of usable storage capacity.
   1. You probably won’t need anywhere near this level of scalability to meet your workload needs, but you should rest comfortable knowing that a configuration with several hundred cores and hundreds of terabytes of memory can easily be expanded if needed.

The particulars of the Exadata X7-2

The Exadata X7-2 Database Machine offers a similar architecture to the Exadata X10M with several notable differences.

• Exadata X7-2 Intel® Xeon® processors from 2017. Each compute server has 48 cores, and each database server has 20 cores. However, these processors provide only a small fraction of the performance of the processors in Exadata X10M.
• Exadata X7-2 uses 10 TB disks instead of the 22 TB disks used in Exadata X10M, requiring more than two times the number of storage servers to deliver the same usable capacity.
• Exadata X7-2 uses 40 Gbps InfiniBand networking, which requires database server IO requests to go through the entire IO and networking stack to access data on storage servers. This takes 250 microseconds, compared to the 17 microseconds available through Exadata X10M’s use of a 100Gbps RoCE fabric and bypassing of the IO and networking stack.
• Exadata X7-2 does not have the XRMEM capabilities of Exadata X10M, another factor contributing to its longer data access latency.

Here are some critical aspects of that version of Exadata X7-2

1. High-performance Database Servers
   1. Each Exadata X7-2 database server has two 24-core Intel® Xeon® processors and 384 GB of memory, which is expandable up to 1.5 TB. 
2. Flexible storage solutions: 
   1. High Capacity storage servers utilize twelve 10 TB disks and support them with 25.6 TB of flash cache. With “High Redundancy,” each storage server provides roughly 35.6TB of usable capacity. Two 10-core Intel Xeon processors are provided to offload SQL processing from database servers.
   1. Extreme Flash storage servers provide 51.2 TB of all-flash storage. Each storage server provides approximately 14.7 TB of usable capacity. 
3. Scalability:
   1. Scalability is the same as with Exadata X10M, starting with 2 databases and 3 storage servers and providing independent scalability to support dozens of each. 

Things we left out of the TCO analysis on purpose

You will see a number of factors explicitly shown in the TCO model, but some of the items we mentioned at the start of this analysis aren’t there. This warrants a short explanation.   

For the purpose of this TCO study, we assumed that the administrative costs of running your Oracle Databases and infrastructure do not change. Why would we do that?  The largest amount of administrative costs are for the management of the databases themselves.  Since the number of databases doesn’t change, the cost to administer them won’t change either. As you will see, the X10M systems have fewer database servers and storage servers than the X7-2 systems we’re comparing them against. However, the amount of automation that Oracle provides in this area makes the incremental amount of time small. There will be savings for large organizations with 10’s or 100’s of Exadata systems, but not for many organizations. 

The costs of data migrations are the second item we considered and left off the list. Data migrations are real, and they do have costs, but they are straightforward and automated when moving from one Exadata to another. In addition, the costs can vary widely from organization to organization based on their approach to moving data. When you do a TCO analysis for your organization, you should look into this, but it shouldn’t appreciably change the relative value of the Exadata X10M. 

The cost of operational interruptions is the final factor that we’ve not specifically quantified but for a different reason. The costs are real and are based on the lost revenue or employee productivity they entail, as well as the administrative costs to recover from them. It is well known that components in older systems do tend to have higher failure rates than newer systems, but we’ve excluded this factor because Oracle has designed the hardware and software in Exadata systems to be fault tolerant.  If a component or server fails, databases enabled with Oracle Real Application Clusters (RAC) will continue to run on other database and storage servers in the Exadata without losing data or in-flight transactions.  Your applications will run with slightly less performance which repairs are being made, but it’s incredibly difficult to quantify the negative economic impact of such an event.

Leaving these factors out of the TCO analysis should have little to no impact on the total amount of savings you achieve. By adding to both columns in the comparisons, they would increase the costs of each and reduce the percentage savings. However, we do not consider these factors to have a material impact in either direction.

Comparing Exadata X10M with Exadata X7-2 for OLTP workloads

When comparing different generation platforms for OLTP, two critical factors should be considered. The first is the number of SQL Read IOPS available to process requests; the second is the SQL Read Latency for those requests. OLTP processing is a “read-heavy” workload, so read performance is more important than write performance.

Exadata X10M has a built-in advantage on IO latency due to its use of RDMA, so we focused on equalizing the number of processing cores in the two systems. Given the relative core counts, it isn’t possible to have an exact match between the two systems.  Our analysis is based on the following configurations. The total number of database and storage servers includes those in the “quarter-rack” system that’s used as a building block.

For the X7-2, the number of database and storage servers required is driven by the SQL Read IO rates they enable. Organizations starting with a smaller X7-2 configuration could potentially consolidate workloads from multiple Exadata systems and non-Exadata platforms onto the Exadata X10M.
The comparative TCO models for these two configurations are given below. As you can see, the increased density afforded by Exadata X10M significantly reduces the purchase price, support costs (which are based on the purchase price), and power and cooling costs.

As explained above, we did not factor in the cost of Oracle Database licenses. That is because we assume that organizations would bring your licenses from your existing Exadata systems. The cost per core stays the same, but it should be noted that the AMD EPYC cores run faster than the older Intel Xeon cores, so you may not need as many licenses to support the same workloads.

In this configuration, the X10M delivered roughly 4% more SQL Read IOPS than the equivalent X7-2 configuration. It should also be noted that the Exadata X10M configuration does not need the extra storage server to provide the 5.6 million SQL Read IOPS level of performance. However, most organizations will look for similar storage capacity with their new system configurations for in-database archiving and other storage purposes. Removing the extra X10M storage server would reduce the 5-year TCO by more than $200,000, further increasing the X10M’s advantage.

Organizations may keep their systems for more or less than 5 years. Still, since the relative savings are about the same in each category, the TCO savings will remain approximately the same over shorter deployment lifetimes.
Comparing Exadata X10M with Exadata X7-2 for analytics with data warehouses
When comparing different generation platforms for analytics with data warehouses, a key criterion to evaluate is how much storage capacity both systems deliver. However, analytics isn’t just about the amount of data stored but also about how it is processed on both database servers and storage servers. So, the total number of cores you have in each is essential, particularly if you want to work with encrypted data.
Exadata X10M’s use of 22TB disks and larger flash caches per storage server enables it to provide similar capacity with many fewer storage servers. In addition, its use of AMD EPYC processors with more cores per both database and storage server means that it can process data more quickly.

Normalizing on capacity, database server core count, and storage server core count gives us the following configurations for our TCO analysis. Again, the total number of database and storage servers includes those in the “quarter-rack” system that’s used as a building block.

In this case, we were also able to get the various factors to line up closely. Everything is pretty much in line except that the total size of the flash on the X7-2 is larger than that on the X10M due to the higher number of storage servers in the X7-2. However, the flash throughput per storage server on the X10M is more than twice that of the X7-2, so we find this acceptable. Furthermore, the increase in the number and speed of the storage server compute cores and the higher memory bandwidth on those systems will allow each of them to process data substantially faster than their X7-2 equivalents. It should be noted that in this configuration the Exadata X10M provides 68% more SQL Read IOPS than the X7-2 configuration so customers looking to run mixed workloads or consolidate OLTP and Analytics together, which is most people, would see a significant improvement for their transaction-oriented workloads.
The comparative TCO models for these two configurations are given below. As you can see, the increased density afforded by Exadata X10M once again significantly reduces the purchase price, support, and power and cooling costs. As with the OLTP example, we did not factor in the cost of Oracle Database licenses since organizations will bring existing licenses from their existing systems.

The TCO savings in this scenario are slightly less since it’s driven more by the storage capacity and less by enhancements such as RoCE networking and XRMEM that assisted the X10M with OLTP. For X10M, this is the minimum configuration, and we know of people running data warehouses on Exadata that far exceed these sizes. In this scenario, the additional performance available with the AMD EPYC processors should more than offset reductions in flash capacity.

Other considerations not directly captured in the TCO analysis

You should consider several other things when evaluating whether to upgrade to the X10M system
RoCE Networking and XRMEM: Exadata X10M uses these technologies to dramatically reduce SQL Read latency to 17 microseconds compared to 250 microseconds on X7-2. This allows the X10M to complete complex transactions much quicker than the X7-2. Lower latency can lead to faster transaction completion which can have positive effects for users, but we did not include it in the TCO analysis.

Exadata Virtualization: In 2019, Exadata transitioned from OVM/Xen to the more advanced Oracle Linux KVM with the X8M2 generation. The change to KVM was tied to the move to RoCE networks and was not backported to earlier Exadata generations. KVM allows for larger memory configurations and has lower overhead than OVM/Xen, allowing IT organizations to pack in more databases on the same number of servers. This significantly improves database consolidation on the X10M, which Oracle reports most organizations take advantage of.

Moving to the cloud: This TCO analysis covers the on-premises Exadata X10M versus the on-premises Exadata X7-2. Oracle also offers Exadata Cloud@Customer X10M, which lets organizations run Oracle Exadata Database Service and Oracle Autonomous Database in their data center. Oracle manages the infrastructure and also the database when using Autonomous Database, which reduces administration expenses, which were considered equal for the on-premises scenario. That, combined with variable consumption costs, can provide significant additional benefits and TCO savings for organizations. With Oracle, organizations can also use Exadata platforms in the Oracle Public Cloud and in Microsoft Azure. If your plans are to move some workloads to the public cloud and run others in your data center, then you should examine Oracle’s Cloud offerings.

Carbon footprint: A recent study from Lancaster University pegged the global carbon footprint of information and communications technology at between 2.1% and 3.9% of worldwide emissions. While not a huge part of the global carbon footprint, it’s much higher than that for companies in data-intensive industries. Although the carbon footprint of energy consumption varies from location to location and company to company, it is ultimately correlated to energy consumption. Reducing energy consumption per unit of work by 46% to 50% will be seen by many as a step in the right direction.

Support lifetime: Oracle typically supports Exadata systems for up to 7 years after the date of last purchase. The Exadata X7-2 was last sold as a new system in 2019, so it will be supported through 2026. Exadata X10M will probably be sold through at least 2025, so you can get support on it up through 2032.

Our Perspective

Several factors must be considered when deciding when to update to newer generations of technologies. In some cases, employing a new product version does not result in a significant TCO, leading enterprises to choose not to do so.

However, this was not the case with Exadata when comparing X10M with previous product generations, including the X7-2.

Our main findings indicate that not moving off older generation Exadata X7-2 systems and taking advantage of a streamlined and simplified X10M architecture will result in customers absorbing between a 70% to 85% higher TCO. This TCO analysis focused on two of the most common use cases, so if your database usage differs, you will likely fall somewhere in the middle. However, it’s essential to do your own analysis based on your workloads and growth projections. Oracle has several tools that can help you do this, either on your own or with their help.

The choice of whether an upgrade is viable or not for you is dependent on your specific circumstances.  However, an upgrade or net new usage of Exadata X10M should provide you with significant TCO advantages.   This is a solid business case and should be followed up on by those using older Exadata releases. The time to start thinking about upgrading is now.

This theCUBE Research Analyst Brief was commissioned by Oracle and is distributed under license from theCUBE Research.