ABSTRACT: AWS re:Invent 2024 highlights the expanding role of generative AI (GenAI) while unveiling major advances in compute, storage, networking, and application modernization. Key releases include Trainium2-powered EC2 instances and the upcoming Trainium3 chips, both optimized for cost-effective AI workloads, plus multi-agent capabilities in Bedrock to orchestrate complex workflows. SageMaker Lakehouse upgrades, featuring zero-ETL, further streamline machine learning pipelines. Serverless enhancements in Lambda, ECS, and Fargate boost developer agility by abstracting infrastructure, while EKS Auto Mode and Hybrid Nodes simplify Kubernetes deployments both on-premises and in the cloud. New S3 capabilities like S3 Tables deliver near real-time analytics, collectively driving cost efficiency, scale, and broader GenAI adoption.
Going into this week and comparing it to the AWS re:Invent 2024 highlights, we see that the cloud and AI landscape are poised for a significant shift. The key themes that emerged before the keynotes on Monday and Tuesday set the stage for the week to focus on the evolution of generative AI (GenAI), infrastructure innovation, and application modernization. These are at the forefront, offering a preview of the industry’s direction.
What we expected to hear
- The AI Infrastructure Cloud Is Taking Shape: We expected Garman to highlight a critical point that aligns with broader industry observations: GenAI isn’t being built for its own sake—it’s a foundation for better, more intuitive, and feature-rich applications. These next-gen applications will leverage collections of agents, driving the rise of agentic workflows that deliver real business value. However, underpinning this evolution is the next AI infrastructure cloud, which must balance the complexity of cutting-edge workloads with the need for enterprise-grade reliability, scalability, and cost control.
- Infrastructure Complexity: A Necessary Growing Pain: Building this next generation of applications doesn’t mean simplifying the underlying cloud infrastructure—at least not yet. The cloud is becoming more complex before it gets more uncomplicated as AWS and other providers innovate across layers like compute, networking, storage, and serverless technologies. Garman’s remarks on supply constraints for high-end chips point to a pressing question: Who needs which chips for which workloads? These decisions will directly impact performance, cost, and time-to-value for enterprises adopting GenAI at scale.
- From Proof of Concept to Production: The era of isolated GenAI proof-of-concepts is fading. Enterprises are now focused on scaling their AI capabilities into real-world, production-grade applications. This shift requires thoughtful integration into existing workflows, data pipelines, and security models. As Garman noted, customers are eager to take the hundreds of pilots they’ve run and transform them into a few high-impact solutions that deliver measurable enterprise value.
- Cost and Observability Will Be Make-or-Break Factors: With the cost of inference and AI applications under scrutiny, observability tools and finOps frameworks will play a crucial role. Enterprises need visibility into workloads and costs to ensure they’re optimizing resources effectively. Garman’s emphasis on AWS’s efforts to drive down inference costs—through innovations in hardware, instance types, and serverless models—shows a clear commitment to enabling cost-effective AI adoption.
- The Building Blocks: Compute, Storage, Networking, and Serverless: The foundational elements of the cloud remain critical to scaling GenAI and its applications. Whether through serverless AI models, new silicon for inference, or optimized networking solutions, re:Invent 2024 will spotlight advancements across these domains. These innovations will empower developers to abstract away complexity while still leveraging powerful underlying technologies.
AWS re:Invent 2024 Highlights – What we heard
AWS re:Invent 2024 has unveiled a host of innovations that signal a transformative shift in cloud computing and artificial intelligence (AI). These announcements emphasize AWS’s commitment to advancing infrastructure, simplifying generative AI (GenAI) adoption, and enabling enterprises, developers, and partners to innovate at scale. From Bedrock’s enhancements in multi-agent collaboration to SageMaker’s zero-ETL capabilities, the updates represent AWS’s vision for building a more connected, data-driven, and efficient future.
Key Innovations Across AWS Services
AWS has announced the general availability of Trainium2-powered EC2 Trn2 instances, offering up to 40% better price-performance compared to GPU-based EC2 instances like P5e. These instances are purpose-built for the most demanding generative AI workloads, featuring 16 Trainium2 chips interconnected via NeuronLink for a staggering 20.8 peak petaflops of compute power. Additionally, AWS unveiled the Trn2 UltraServers, which scale performance across 64 interconnected Trainium2 chips, enabling up to 83.2 petaflops of compute for training and inference tasks on trillion-parameter models. These advancements cater to enterprises building state-of-the-art AI models, reducing training times while optimizing costs.
Looking ahead, AWS pre-announced Trainium3, its next-generation AI chip set to launch in 2025. Built on a 3-nanometer process, Trainium3 promises to deliver 4x the performance of its predecessor, setting a new benchmark for high-performance AI training and real-time inference. These innovations solidify AWS’s leadership in AI infrastructure, providing unparalleled tools for scaling generative AI workloads efficiently and cost-effectively.
Another standout update at re:Invent 2024 is the next generation of SageMaker, a platform that bridges analytics, machine learning, and AI into a single, cohesive experience. The newly introduced SageMaker Unified Studio offers a centralized environment for data exploration and AI development, simplifying workflows for developers and data scientists. With SageMaker Lakehouse, enterprises can now unify data from lakes, warehouses, and SaaS applications, enabling seamless cross-silo analytics without complex data transfers. This is not yet another lakehouse but a federation capability. By supporting Apache Iceberg and offering integrations with platforms like Databricks and Snowflake, SageMaker positions itself as a formidable alternative in the data-first platform market.
Other parts we will go deeper into another time include SageMaker HyperPod, HyperPod Task governance, where Automated reasoning fits, and Multi-agent in Q.
Amazon Bedrock also received a significant boost with features designed to address critical challenges in generative AI. Automated Reasoning Checks safeguard against model hallucinations, ensuring factual response accuracy, a crucial feature for industries such as healthcare and finance. Multi-agent collaboration enables the orchestration of agentic workflows, making it easier to manage complex tasks with supervisor agents that dynamically route and coordinate workloads. Meanwhile, Model Distillation offers a cost-effective way to optimize models, delivering smaller, faster alternatives that retain accuracy while reducing latency and costs.
One interesting concept that was discussed in a briefing but didn’t make the main stage, was LLM-as-a-Judge. This is one we will go deeper into in the coming months as it ties in with some of the other features of bringing together multiple agents and figuring out which one does the work.
Beyond AI, AWS introduced revolutionary updates to its database offerings. Amazon Aurora DSQL, the fastest distributed SQL database, combines multi-region strong consistency, low latency, and zero infrastructure management. Similarly, DynamoDB enhancements now support multi-region strong consistency, catering to globally distributed applications where precision is paramount.
In developer enablement, Amazon Q Developer unveiled agents for automated unit testing, documentation, and operational troubleshooting. These capabilities simplify software development while accelerating modernization efforts for legacy systems like VMware and COBOL-based mainframes. On the business intelligence front, Amazon Q Business added over 50 new actions to automate workflows across applications such as Asana and Zoom, powered by a canonical enterprise data index.
Amazon S3 continues to push the boundaries of cloud storage with its newly announced S3 Tables and S3 Metadata features, tailored to streamline data management and accelerate analytics. S3 Tables bring fully managed support for Apache Iceberg, a popular open table format, enabling up to 3x faster query performance and 10x higher transactions per second than standard S3 buckets. These table buckets are optimized for analytics workloads, automating maintenance tasks like compaction and snapshot management while providing advanced features such as row-level transactions and schema evolution.
Complementing this, S3 Metadata introduces near real-time, queryable metadata generation, simplifying data discovery and organization. S3 Metadata eliminates the need for complex, manual metadata systems by automatically capturing system and custom metadata. This innovation allows organizations to quickly search and organize their data using SQL queries, making preparing data for business analytics, machine learning workflows, and retrieval-augmented generation (RAG) models easier.
We also had a chance to go deeper with the AWS Serverless team, Lambda and ECS. AWS is celebrating the 10-year anniversary of AWS Lambda, marking a decade of pioneering serverless computing. In a discussion with Nick Coult, Director of Product for Serverless Compute, highlights how AWS’s “serverless operating model” removes the complexity of infrastructure management to help developers move faster. By abstracting away the need to manage servers, developers can focus on application code rather than cluster maintenance, scaling, or patching. Offerings like Lambda, ECS, and Fargate enable cost-effective, event-driven workloads, reduce cold starts with SnapStart (now for Java, Python, and .NET), and simplify the developer experience through integrations with popular IDEs like VS Code and built-in AWS tools such as CloudWatch, Kinesis, and Amazon SQS.
Coult emphasizes that even highly regulated industries are now embracing serverless because it shifts the burden of security and compliance from customers to AWS, and allows organizations to modernize applications at their own pace. Whether teams are starting with consumer-facing web apps or back-end systems, the ability to quickly spin up or tear down environments without paying for idle capacity drives both agility and cost efficiency. Looking ahead, AWS sees serverless continuing to evolve—especially in light of AI and generative workloads—but believes the core principles of simplicity, rapid scaling, and lower operational overhead will remain the foundation of serverless innovation.
It wouldn’t be a cloud conversation if we didn’t dig further into some news around Kubernetes. AWS is marking Kubernetes’ 10th anniversary by emphasizing its continued commitment to the open-source ecosystem and its fully managed service, Amazon EKS. In a conversation with Barry Cooks, VP of AWS Kubernetes, he highlighted how EKS caters primarily to operators and platform engineers who need to manage large-scale containerized workloads while freeing developers to focus on shipping code. He stresses that EKS, which supports both Linux and Windows containers and now includes new features like EKS Auto Mode and EKS Hybrid Nodes, aims to make life easier for customers—whether they’re new to Kubernetes or have deep in-house expertise. Auto Mode offloads tasks such as CVE patching, networking, and storage management to AWS, greatly reducing complexity. At the same time, Hybrid Nodes allow users to run on-premises hardware under a single EKS control plane, enabling a gradual migration to the cloud.
Cooks also notes that AWS sees a huge range of workloads on EKS, from high-scale GenAI training and low-latency gaming workloads to more conventional web services and financial applications. Recognizing that Kubernetes upgrades can be challenging—especially for enterprises on regulated or extended-release cycles—AWS now offers extended support for older Kubernetes versions, giving customers the flexibility to upgrade on their schedule. Altogether, the key themes in AWS’s Kubernetes strategy are choice, with full open-source compatibility and no strict mandates on third-party tooling, simplicity through managed offerings that handle cluster patching and scaling, and scalability supporting everything from one pod per node to tens of thousands of pods and beyond.
Increasingly, AWS is designing EKS features with platform engineers in mind—those responsible for bridging developers’ fast-paced innovation needs with the operational demands of production systems. By providing operators and platform teams with automated patching, flexible scaling, and extensive policy and security controls, like the newly introduced EKS Auto Mode and extended support offerings, EKS empowers these teams to establish reliable, secure, high-performing infrastructure. This frees developers to focus on writing and deploying application code, accelerating time to market while maintaining governance and compliance at scale.
Takeaways for Enterprises, Developers, Partners, and the Industry
For enterprises and GSIs, AWS’s announcements emphasize the importance of modernizing legacy systems and migrating workloads to the cloud. Tools like Amazon Q Developer make transitioning mainframe and VMware systems easier into cloud-native environments, reducing costs and timelines. To fully leverage GenAI, enterprises must identify business-critical applications where AI can deliver measurable ROI. Investments in observability and cost management will be crucial as organizations scale their AI initiatives while maintaining financial oversight.
Developers face an evolving landscape driven by the rise of agentic workflows and serverless AI models. Platforms like Bedrock and SageMaker offer the tools to design and deploy next-generation applications. Features such as multi-agent collaboration, automated reasoning, and zero-ETL integrations allow developers to focus on building scalable, intuitive solutions while AWS handles much of the underlying complexity.
AWS’s advancements create immense opportunities for partners and independent software vendors (ISVs). As GenAI becomes integral to enterprise workflows, partners with expertise in data integration, infrastructure optimization, and security will be in high demand. AWS’s tools enable partners to help businesses navigate the complexities of integrating AI into legacy systems while ensuring compliance with regulatory standards.
From an industry perspective, the innovations unveiled at re:Invent show that infrastructure advancements are currently outpacing application simplicity. However, these developments lay the groundwork for future abstractions to make advanced AI technologies accessible to a broader audience. Tools like Aurora DSQL and Bedrock’s automated reasoning checks address pressing industry needs while paving the way for a more democratized approach to AI adoption.
Our ANGLE
AWS re:Invent 2024 highlights the company’s ability to innovate across the stack while maintaining a sharp focus on practical applications. Going into AWS re:Invent 2024, we fully expected GenAI to remain at the forefront of conversation, especially around the emergence of next-generation, “agentic” applications. We anticipated AWS would highlight how AI infrastructure is evolving to support enterprise-grade workloads, touching on advanced chip options, new instance types, and how customers can best align the right chips for the right tasks. This played out largely as expected, with Amazon unveiling details of Trainium2- and Trainium3-powered EC2 instances and a strong emphasis on cost optimization for large-scale AI training and inference. Yet, some nuances around how enterprises will adopt multi-agent workflows, optimize prompt engineering at scale, and solve the thorniest real-world data challenges were introduced but not deeply explored or mentioned in briefings that never reached the main stage. We suspect more details will emerge in the coming months, especially on the operational complexities of agentic systems and how they integrate with broader AWS services.
On the developer front, we had been looking for deeper, session-level detail around how AWS’s new or updated services—particularly SageMaker Lakehouse, Bedrock’s multi-agent collaboration, and Amazon Q Developer—tackle the day-to-day pains of getting AI models into production. We got partial confirmations: SageMaker Lakehouse indeed emphasizes cross-silo analytics, Bedrock introduced new agent orchestration capabilities, and Amazon Q Developer looks promising for automated tasks like unit testing or documentation. Bedrock’s advancements in multi-agent systems position it ahead of competitors, particularly with features like Automated Reasoning and Model Distillation. However, gaps remain in process modeling tools, critical for the broader adoption of agentic workflows, such as features provided by companies like Celonis. Similarly, SageMaker’s zero-ETL capabilities make it a strong competitor to platforms like Databricks and Snowflake, reducing the friction of data movement while supporting robust analytics and AI development.
However, specific roadmaps for evolving “agentic” AI into practical workflows were somewhat high-level, and some intriguing ideas (like “LLM-as-a-Judge”) appeared off-stage in smaller briefings. We expect AWS to continue refining these features into more robust offerings, especially as customers move beyond pilot projects into enterprise-wide AI adoption. Meanwhile, database innovations like Aurora DSQL and DynamoDB’s multi-region strong consistency provide the scalability and reliability needed for today’s globally distributed applications. We also anticipated more discussion on observability and finOps for AI, beyond broad references to cost. While AWS did stress inference cost reductions and new serverless models, the practical tools to manage complex AI pipelines, from cost tracking to performance tuning, weren’t fully fleshed out. We see this as a gap where further real-world guidance on scaling from proof-of-concept to production will be essential.
The S3 announcements are exciting and needed. Many on-premises storage providers have similar capabilities around metadata, with some going beyond that, such as VAST Data, NetApp, Dell Technologies, and HPE Storage. The S3 Tables buckets are a great feature addition for platform engineers tasked with or inherited DBMS function maintenance. The biggest “miss” was not hearing more detailed, prescriptive guidance around bridging the AI infrastructure and data pipeline chasm. We believe the guidance will surface in the coming months, particularly for regulated industries and enterprises needing to integrate legacy environments into these modern cloud-native patterns.
On the serverless front, the breadth of newly announced features in (Lambda/ECS) and Kubernetes (EKS) corroborates our prediction that infrastructure modernization, while gaining complexity, will remain a critical on-ramp for GenAI. AWS is clearly focused on bridging the gap between simplified developer experiences and the inevitable demands of large-scale AI. EKS Auto Mode, extended Kubernetes support, and Hybrid Nodes were all major developments we partly expected but did not foresee the hybrid aspect being so prominent. Having Barry Cooks say, “We start with the notion that Kubernetes is bigger than us,” was a very positive and accurate statement backed up by the 60k plus contributions to the CNCF this year so far. The emphasis on platform engineers and operators also resonates with our view that internal cloud platforms will be foundational for scaling AI and microservices architectures.
AWS re:Invent 2024 highlights are a testament to the company’s vision for the future of cloud and AI. The announcements demonstrate AWS’s clear roadmap for enterprises, developers, and partners looking to transform their operations and unlock the full potential of AI and cloud technologies. As the industry continues to evolve, AWS’s innovations will undoubtedly play a central role in shaping its trajectory. One point to understand is that there is overlap among many services; navigating that understanding will still take some expertise.
Stay tuned for deeper dives into these innovations on thecuberesearch.com and thecube.net as we continue to analyze their impact throughout the conference.
