The rise of Kubernetes came about through a combination of forces that were in hindsight, quite a long shot. AWS’ dominance created momentum for cloud native application development and the need for simpler experiences beyond easily spinning up compute as a service. This wave crashed into innovations from a startup named Docker and a reluctant open source benefactor in Google that needed a way to change the game on Amazon in the cloud. Factor in Red Hat, which needed a path beyond Linux and was just about to opt for an alternative to Kubernetes to power OpenShift. Finally, figure out a governance structure to herd all the cats in the ecosystem so you can win out over other competition and create a de facto standard. Make all that happen and you get the remarkable ascendancy of Kubernetes.
Such was the story unveiled recently in a new two-part documentary series from Honeypot simply titled “Kubernetes.” In this Breaking Analysis we tap the back stories of this documentary, which explains the improbable events leading to the creation of Kubernetes. We’ll share commentary from early Kubernetes committers and key players who came on theCUBE to piece together how it all happened. Finally, we’ll present new survey data from ETR on containers.
The Story of Kubernetes
Kubernetes Part 1 explains the dynamics of the technology industry in the early part of last decade, the impact that AWS had on the DevOps movement, the importance of Docker, which blazed a new trail for developers and how Google ultimately landed on the decision to open source Kubernetes. Part 2 of the series introduces the contribution of Red Hat and how the ecosystem evolved to create a new standard for software development.
The documentary is sponsored by Google, Red Hat and the CNCF. It includes commentary from many of the players who were early committers or key individuals who influenced the development of Kubernetes. Many of these innovators came on theCUBE at various events during the formative years of mainstream containers including Tim Hockin, Kelsey Hightower, Craig McLuckie, Joe Beda, Brian Grant, Solomon Hykes, Jerry Chen and others. John Furrier and Stu Miniman were at the many shows theCUBE covered and unpacked what was happening at the time. We’ll share the commentary from the guests they interviewed in this post.
Developer Defined Infrastructure
Over the years we’ve used several phrases to describe the programmability of infrastructure. Jerry Chen is a partner at Greylock Ventures. Years ago, he coined the term developer defined infrastructure (DDI). Chen was head of VMware’s cloud product group and left the company in 2013 to become a full time investor. His first investment was Docker. He explained on theCUBE what happens when infrastructure is defined in software. How breaking down infrastructure into components that each have an API allows developers to control infrastructure through code and make applications portable. And how DDI completely disrupted the entire application development process.
[Listen to Jerry Chen from 2015 explain DDI and how it changed development in IT.]
The concept of DDI or what Wikibon’s David Floyer called software-led infrastructure in 2012, was spawned from the cloud and the obvious opportunity for automation. This ushered in the DevOps movement, which was initially seen as something for startup developers but over time, as Kelsey Hightower explains eventually caught on with the ops crowd.
[Listen to Kelsey Hightower at KubeCon 2017 explain the dynamic between dev and ops.]
Abstractions on Abstractions
What Jerry was explaining is a new abstraction layer that was being built to essentially make infrastructure invisible to developers. Here’s some ETR data that quantifies that trend and visually shows you where we are today.
The chart above shows Net Score or spending momentum on the vertical axis and Market Share, which represents the pervasiveness in the survey. As Jerry Chen and the innovators who created Docker saw, the cloud was becoming prominent and you can see it still has spending velocity that is elevated above that 40% red line – kind of a magic mark of momentum. And of course it’s prominent on the X axis. Cloud is what got this movement going.
Below the 40% line you see the low level infrastructure including virtualization. Virtualization is an abstraction layer above servers, storage and networking. Back in 2011, in a VMware briefing with Chad Sakac, we first heard the stated goal the company had to make infrastructure “invisible.” Containers are yet another abstraction that live above virtualization.
And ironically the spending chart above shows this. The two red arrows pointing to containers – container orchestration and container platforms – which are the abstraction layers and services above the underlying VMs and hardware.
And you can see the spending momentum containers have in the market along with cloud, AI/ML and RPA.
How the Stars Aligned to Create Kubernetes
You had the forces that Jerry Chen and Kelsey Hightower described taking shape. The Kubernetes documentary does an excellent job explaining how they came together in the words of the insiders at Google, Red Hat and CNCF. The picture below identifies the key elements and we’ll describe their improbable interaction which formed Kubernetes.
In the upper left you see AWS. We inserted a picture from a post we did right after the first re:Invent in 2012. It was obvious Amazon was the cloud gorilla and had all the momentum in IT. We saw it as an unstoppable wave as did the protagonists in the documentary.
Solomon Hykes, the founder of Docker (upper right) saw the need to simplify the packaging of applications for cloud developers. Containers had been around for more than 30 years prior to Docker, initially developed as part of the Unix operating system to isolate certain processes from other application components. But the vast majority of programmers didn’t know or care about containers. They were a low level construct and hidden from the mainstream.
Docker changed all that. Here’s how Docker founder Solomon Hykes described containers in 2014 on theCUBE with John Furrier at Red Hat Summit.
[Listen to Solomon Hykes answer the question – “What is a container?”]
Docker at the time was a 30 person company and had just changed its name from dotcloud. It’s astounding to think that this tiny company changed the history of computing.
Google’s Need to Change the Cloud Game Tips the Scales
Imagine you’re at Google in 2013. You’ve built the world’s largest and most impressive computer network with millions of servers around the world. You have great tech and along comes Amazon, an online commerce company. Its entry into the cloud captures the minds and wallets of developers and organizations everywhere. Amazon has all the momentum in IT and your business is built on appropriating consumer data to sell ads. And your motto at the time is “Don’t be Evil.”
If you’re an engineer at Google you want to change the world. You want to develop something meaningful. You don’t want to just follow Amazon – that won’t get you far. So what’s the play? Hence in the diagram above the question mark in the Google cloud logo. You see Docker emerge and it creates an opportunity. Docker is great because it simplified containers and took them mainstream. But Google sees that you need more. Heck, Docker saw that you needed more, so did Mesosphere and others.
Why do you Need Something Other Than Docker?
So why exactly would you need more than what Docker had created? Craig McLuckie who was a product manager at Google in 2014, explained to John Furrier the need for another management layer. He shares that what Docker did was create the capability to run a binary anywhere. But there needs to be a way to manage that with a framework that can run anywhere.
So McLuckie explained why the industry needed something like Kubernetes but let’s go further back in time. Here’s Google with this amazing cloud infrastructure but no commercial cloud business to compete with AWS – at least not one that was taken seriously at the time. It had compute as a service and could spin up VMs easily but it really wasn’t that exciting in Google’s mind and AWS had nailed that use case. So Google needed a way to increase its relevance.
Google Never Embraced Virtualization to Solve its Internal Challenges
Google had this thing called Google Borg, which is a container management system and scheduler. And Google looked at the problem in cloud, which was not just how to I spin up a virtual machine, but rather how do I run and manage my code.
Joe Beda who was with Google at the time explains to John Furrier and Stu Miniman their mindset and how they saw the opportunity.
And so Google looked at the market and said what can we do to make Google relevant in cloud? Here’s Eric Brewer from Google talking on theCUBE about Google’s thought process at the time. Google started the same year as VMware. So it never used virtualization for its own infrastructure. Rather it used containers and low level processes to build its cloud network. So it came at the problem with a different mindset.
Now Eric Brewer gave us the short version of the story. What he reveals in the documentary is that initially his posture was an instant negative reaction to open sourcing Kubernetes. Google management was skeptical and asked “why would we open source our secret sauce to help our competitors?” But over time and through a series of meetings and discussions, Google’s engineers were able to convince management that open sourcing Kubernetes was not only good for the industry but also good for Google.
Google Management was Reluctant to Open Source Kubernetes
Tim Hockin and Brian Grant were on the original Kubernetes team and pressed management to convince them to bless open sourcing Kubernetes. Hockin explained to John Furrier on theCUBE, the narrative inside of Google at the time.
The open source strategy became more compelling as they studied the problem because it gave Google a way to neutralize AWS’ advantage. Because with containers you could develop on AWS for example and then run the applications anywhere. Like Google’s cloud. So it not only gave developers a path off of AWS, if Google could develop a strong service on GCP it could monetize the play over time.
Linux Needed a Path to the Cloud
Now back to the diagram, which shows a smiling Alex Polvi from CoreOS. His company was acquired by RedHat in 2018. And he saw the need to bring Linux to the cloud. After all, Linux was powering the Internet. It was the OS for enterprise apps and to Polvi, it made sense to extend its path. Linux became the operating system for the data center. It did so by becoming an open standard with consistent APIs. And while there are still other OSes out there, Linux is the dominant production grade operating system for data center applications.
At an OpenStack event in 2015, Polvi described how he thought a similar standard would emerge in an abstraction layer that treats the data center as a computer, rather than addressing individual machines. And he stated at the time his belief that layer would standardize around Kubernetes.
[Listen to Alex Polvi describe the parallels between the standardization of Linux and Kubernetes].
Red Hat’s Future Depended on a New Path to the Cloud
Red Hat has been around since 1993. So it needed a future that took the company to cloud. OpenShift was that future and it had to leverage a container management framework. According to the Kubernetes documentary, Red Hat connected to Google through their respective board members which led to a discussion amongst engineers about containers. Google was non-committal but did expose that they were thinking about doing something and maybe open sourcing this new project.
But Google’s internal debates and arm twisting were taking too long so Red Hat couldn’t wait. They had to move forward with an open source alternative to they decided to do what Apple, AirBnB and Heroku were doing and build on an alternative. They were ready to commit to Mesos, which was more sophisticated and much more mature than Kubernetes. But at the last minute, Google came back and said “let’s do it.”
Clayton Coleman was the Red Hat architect that started leaning in right away and was one of the first committers outside of Google to participate in the project.
Decision Time for Kubernetes – Go for Scale or Simplicity?
At the time you had competing forces in the market and internally between do we go with simplicity or do we go for system scale. Google clearly had the expertise to build large, complex, scalable systems. Platforms like Docker Swarm and Mesos were tackling more complex use cases and yet the Kubernetes team bet on simplicity first.
Chen Goldberg from Google talked about why they focused first on simplicity and getting that right before tackling the larger problems. Perhaps is seems obvious today but with external market forces and internal engineers advocating for different directions it was an important decision. Goldberg explains rather than competing right away with say Mesos or Docker Swarm, which were far more baked, they made the bet to keep it simple and go for ubiquitous adoption. Which obviously turned out to be the right choice.
[Listen to Google’s Chen Goldberg explain the rationale of focusing on simplicity first].
Herding the Cats
The last piece of the complex puzzle was governance. According to commentary in the documentaries, Google had promised to open source Kubernetes but when they started to open up to contributors outside of Google, the code was still controlled by Google. Developers had to sign Google paper that basically said Google could still do whatever it wanted. We’ve seen this before when a large company tries to elbow its way into an open source community and yet still maintain control. Google had to pass the baton to an independent entity and that’s how CNCF was started. Kubernetes was its first project.
Chris Aniszczyk of the CNCF explained the genesis of the neutral organization to John Furrier on theCUBE.
Quite an amazing path to get to this point. CNCF now has over 725 members and has become a premier organization hosting more than 100 projects beyond Kubernetes.
Kubernetes’ Dominant Position in the Market
This ETR data above shows container orchestration offerings – the same XY graph as shown before and you can see where Kubernetes lands. Notwithstanding that Kubernetes is not a company but respondents know they’re doing Kubernetes. They may not know which platform they’re using but they know it’s Kubernetes.
The ETR taxonomy has two container categories, container platforms and container orchestration offerings. It’s a challenging topic to survey within the ETR sample because Kubernetes is increasingly becoming embedded into cloud platforms and IT pros may not even know which one they’re specifically using.
In the diagram above we link Red Hat OpenShift and Kubernetes because we really think OpenShift is a dominant revenue player in the space as an increasingly popular PaaS layer. Yes you can download Kubernetes and do what you want with it but if you’re building enterprise apps you’re going to need support and that’s where OpenShift comes in.
Tracking Container Revenue
There’s not much market data on container software revenue but we did find this data below from Omdia. It shows revenue share for container software players. It’s not clear how that’s defined but Red Hat has a nearly 50% revenue share with Mirantis (Docker Swarm), VMware and SUSE (Rancher) in the top four. We found some other data from Informa but it’s dated. It also had Red Hat as the largest player by revenue, which we believe is accurate. Especially because we know the muscle of IBM is behind OpenShift, which is a linchpin of IBM’s application modernization services strategy. IBM can bundle OpenShift into its application modernization services and that gives Red Hat a captive market. We’ve always said that was a key ROI enabler for the Red Hat acquisition and gives IBM an edge in the battle for hybrid cloud.
Kubernetes Under the Hood
The CNCF recently released its annual survey. A data table from that publication is shown below.
This past CNCF annual survey had 1,800 respondents, as shown above. Note: The data above only includes managed services only. It indicates the following:
- 79% of respondents use certified Kubernetes hosted platforms;
- Amazon Elastic Container Service for Kubernetes was the most prominent at 39%;
- Azure Kubernetes Service followed at 23%; and
- Azure AKS Engine comes in at 17%, with Google Kubernetes Engine falling behind those.
Also note that OpenShift shows up on the right side of the chart. Remember, this represents just managed services, which is a relatively new offering for OpenShift. The majority of Red Hat’s business is self-service, either in the data center or in the cloud. Moreover, many of the cloud service providers offer their Kubernetes services for no charge. This explains why Red Hat is the revenue leader (shown earlier) but may not show as strong in some of the surveys.
Which makes you think back to Google management’s initial concern – i.e. why open source such a key technology? The premise was that it would level the playing field in cloud. And for sure it has.
But one has to ask…has that driven the monetization Google was after? You’d have to say probably not. But where would Google be if it hadn’t open sourced Kubernetes? How relevant would it be in cloud? And despite its distant third position behind AWS and Microsoft, without Kubernetes Google probably wouldn’t be in the conversation.
What’s Ahead for Kubernetes?
Let’s wrap with some comments on the state of Kubernetes and thoughts about where the community is headed.
The State of Kubernetes Today
No new insight on this first one but Kubernetes, for all its improbable beginnings, has gone mainstream. In the past year or so we’re seeing much more maturity and support for stateful workloads and big ecosystem support in this respect with better security and continued simplification. But Kubernetes is still pretty complex. It’s getting better but it’s not VMware level of maturity for example. Of course not.
Adoption has always been strong for cloud native companies who start with containers on day one. But we’re seeing many more IT organizations adopting Kubernetes as it matures, confirming Kelsey Hightower’s earlier commentary.
It’s interesting that Docker set out to be the OS of the cloud and Kubernetes has really become that piece of the puzzle. Docker Desktop is where Docker is thriving. It sold off Swarm to Mirantis and has made some tweaks to its licensing model to be able to continue to evolve its business model.
As we said in our 2020 predictions post, we expected then for Kubernetes to become less visible and more embedded into other platforms; and that’s exactly what is happening. But it still has some complexity challenges to overcome. Remember in the early and mid cycle of VMware adoption? Understanding application performance required people in lab coats to remediate problems and scale the system. And in some ways you’re seeing that dynamic repeat with Kubernetes. To secure, scale, get the system to perform and recover quickly when something goes wrong require lots of expertise. This is all made more difficult by the rapid pace at which the ecosystem is evolving Kubernetes. But it’s definitely headed in the right direction.
And it presents opportunities for the ecosystem.
More Abstractions, Multi-cluster, Automation, New Workloads
So what’s ahead for Kubernetes?
We would expect further simplification and more abstractions. As Kubernetes improves support for multi-cluster it will begin to treat those clusters as a unified group and enable clusters to be managed together. And this will create a lot of ecosystem focus on scaling globally and lowering latency, keeping pace with security and recovery. And that will require new services to share resources across clusters. So look for that in the near- to mid-term.
Expect more automation. This will be driven to a large degree by host cloud providers, but also others in the ecosystem. As Kubernetes supports more stateful applications and begins to extend its cluster management, cloud providers will inject as much automation as possible into the system. And ecosystem startups will bring innovations to better manage application resources.
And finally, as these capabilities mature, we would expect to see better support for data-intensive workloads such as AI, machine learning and inference. Scheduling with these workloads becomes harder because they are so resource intensive. And performance management becomes more complex so that will have to evolve. Frankly many of the things the Kubernetes team back burnered early on that were seen for example in Docker Swarm or Mesos, will start to enter the Kubernetes scene.
What’s After Kubernetes?
The last thing we’ll ask you to think about is what’s next beyond Kubernetes? You know this isn’t it, right? With serverless and IoT and the edge and new data-heavy workloads there’s something that will disrupt Kubernetes. In that CNCF survey, nearly 40% of respondents were using serverless and that’s going to keep growing. How will that change the development model? Andy Jassy famously said that if they had to start over with Amazon retail they’d start with serverless. So let’s keep and eye on the horizon to see what’s coming next.
One last thought. A prediction of sorts. The cloud native community likes to meet. Many of the innovations we’re seeing today were spawned at small meetups or other significant events. This crowd will be some of the first to return to in-person meetings to keep the ideas flowing.
Don’t miss it!
Keep in Touch
Thanks to Stephanie Chan who researched several topics for this episode. Remember we publish each week on Wikibon and SiliconANGLE. These episodes are all available as podcasts wherever you listen.
Email david.vellante@siliconangle.com | DM @dvellante on Twitter | Comment on our LinkedIn posts.
Also, check out this ETR Tutorial we created, which explains the spending methodology in more detail.
Watch the full video analysis:
Note: ETR is a separate company from Wikibon and SiliconANGLE. If you would like to cite or republish any of the company’s data, or inquire about its services, please contact ETR at legal@etr.ai.
All statements made regarding companies or securities are strictly beliefs, points of view and opinions held by SiliconANGLE media, Enterprise Technology Research, other guests on theCUBE and guest writers. Such statements are not recommendations by these individuals to buy, sell or hold any security. The content presented does not constitute investment advice and should not be used as the basis for any investment decision. You and only you are responsible for your investment decisions.
