Update February 6, 2025
Earlier this week at IBM’s Investor Day, Red Hat President and CEO, Matt Hicks, did a deeper dive on enterprise AI, touting a TAM approaching $1T by the end of this decade. We believe small language models (SLMs) that are secure and sovereign will play a key role in accelerating enterprise adoption. Moreover, the recent DeepSeek announcements validate IBM’s Granite strategy in our view.
Hicks in his presentation focused on three key pillars (shown below) that will accelerate enterprise AI adoption.
Cost. By lowering the denominator for AI deployments the resulting value automatically increases (i.e. Benefit / Cost yields a higher result when the costs are lowered).
Complexity. Making LLMs smaller and easier to deploy, while baking in security and governance will remove much of the heavy lifting off the shoulders of enterprise practitioners and engender trust.
Expertise. IBM’s head of consulting, Mohamad Ali, stresses embedding expertise into AI workflows. IBM has vast domain depth in virtually all key industries and regions around the globe.
The combination of these three factors are fundamental to IBM’s strategy and in our view position IBM for enterprise AI success, further validating its Granite strategy.
Below is a post from last October immediately after the Granite 3.0 announcement providing an overview, our take and a conversation with one of IBM’s key AI technical experts.
Introduction
IBM has recently unveiled its Granite 3.0 family of large language models (LLMs), marking a significant milestone in the enterprise AI landscape. Unlike some Internet giants and specialized LLM players like OpenAI and Anthropic, which focus on massive, general-purpose models, IBM’s strategy centers on developing smaller, efficient, and more cost-effective models tailored for specific business needs. This research note delves into IBM’s approach with Granite 3.0, the integration of these models into its consulting services, and the broader implications for the future of AI in business.
Below is our summary and analysis of IBM’s Granite 3.0 announcement and a video conversation with Sriram Raghavan, IBM’s VP of AI research.
IBM’s Strategic Approach to AI and LLMs
IBM is charting a course distinct from the mainstream AI trajectory dominated by colossal models like OpenAI’s GPT-4 and Meta’s Llama. IBM’s premise is that 99% of enterprise data remains untapped by internet-trained models. IBM aims to harness this proprietary data to deliver domain-specific solutions. By focusing on smaller models that are easier to train and deploy, IBM is positioning itself to offer customized AI solutions that drive customer differentiation and competitive advantage leveraging a client’s proprietary data sets.
Key to this strategy is IBM’s choice to open-source Granite. By releasing Granite 3.0 under the permissive Apache 2.0 license, IBM not only fosters transparency and community collaboration but also allows clients full commercial rights to adapt and integrate the models as needed. We see this approach as a means to lower development costs, accelerate adoption and close the gap with first movers.
Granite 3.0: Performant AI Models Built for Business
Overview of Granite 3.0 Models
IBM’s Granite 3.0 family includes a range of models the company says is designed to meet diverse enterprise needs:
- Granite 3.0 8B & 2B Models: Compact but functional, these models are optimized for tasks like Retrieval Augmented Generation (RAG), classification, summarization, entity extraction, and tool use.
- Granite Guardian 3.0 Models: Focused on safety and trustworthiness, these models incorporate comprehensive guardrails to mitigate risks like social bias, hate speech, toxicity, and hallucinations.
- Granite Mixture-of-Experts (MoE) Models: Smaller models that deliver efficient inference and low latency, suitable for CPU-based deployments and edge computing.
- Granite Time Series Models: Achieving state-of-the-art performance in zero/few-shot forecasting. According to IBM, these models outperform others that are ten times larger.
Performance and Benchmarks
Granite 3.0 models demonstrate competitive performance across various benchmarks:
- Academic and Industry Benchmarks: The Granite 3.0 8B Instruct model leads or matches similar-sized models from leading providers like Meta and Mistral on Hugging Face’s OpenLLM Leaderboard.
- Safety Benchmarks: On IBM’s own AttaQ safety benchmark, Granite 3.0 exhibits superior accuracy in harm detection compared to counterparts like Llama Guard.
- Enterprise Tasks: In core enterprise applications, Granite 3.0 shows leading performance in RAG, tool use, and cybersecurity tasks.
These results underscore IBM’s ability to deliver high-performing models that are both efficient and tailored for business use.
Open-Source Commitment
By releasing Granite 3.0 under Apache 2.0, IBM gives enterprises the flexibility to customize and deploy AI models without restrictive licensing. This approach encourages innovation and allows businesses to retain control over their data and AI solutions. Notably several customers have expressed concerns over foundation models such as Meta’s Llama which include certain terms that potentially restrict use for commercial purposes (i.e. if monetized at scale).
Granite Guardian 3.0: Advancing Safe and Trustworthy AI
Safety and trustworthiness are paramount in enterprise AI applications. According to IBM, Granite Guardian 3.0 models introduce advanced guardrail capabilities:
- Comprehensive Risk Detection: These models can detect a wide range of potential harms, including social bias, hate speech, profanity, and hallucinations.
- RAG-Specific Checks: Unique to enterprise needs, the models perform groundedness and context relevance checks in RAG applications.
- Superior Performance: Granite Guardian models outperform others in harm detection and hallucination detection, enhancing the reliability of AI deployments.
By integrating these guardrails, IBM addresses a top-of-mind concern for businesses adopting AI technologies: ensuring outputs are safe, accurate, and aligned with organizational values.
Integrating Granite into IBM Consulting Advantage
IBM Consulting Advantage Platform
IBM Consulting Advantage is an AI-powered delivery platform designed to enhance the capabilities of IBM’s 160,000 consultants worldwide. The platform includes:
- AI Agents and Applications: Pre-built tools that accelerate solution delivery and improve productivity.
- Methods and Best Practices: Frameworks reflecting IBM’s extensive experience in successful client projects.
- Multi-Modal and Multi-Cloud Support: Flexibility to handle text, video, and integrate with third-party tools across various cloud environments.
Granite as the Default Model
According to IBM, Granite 3.0 models are now the default AI engines powering IBM Consulting Advantage. IBM claims this integration offers several benefits:
- Cost Efficiency: Granite models can operate at up to 90% lower costs compared to larger models, making AI deployments more economically viable for clients.
- Performance: Despite their smaller size, Granite models deliver comparable results to larger models, ensuring high-quality outcomes.
- Customization: Clients can fine-tune models with proprietary data, enhancing relevance and effectiveness in specific domains.
By equipping consultants with these advanced AI tools, IBM aims to deliver enhanced client value at lower costs, accelerating time-to-value and maximizing ROI.
Cost Efficiency and Business Value
One of the most compelling aspects of IBM’s approach is the dramatic reduction in the cost of training and running LLMs:
- Economic Advantage: Granite models offer a cost reduction of up to 90% compared to larger models like GPT-4 and Llama 3.1.
- Performance Parity: This cost efficiency does not come at the expense of performance; Granite models deliver similar or better results in enterprise tasks.
- Scalability: For large-scale deployments, such as equipping thousands of consultants or processing vast amounts of data, the cost savings are significant.
By addressing the economic barriers to AI adoption, IBM is positioning itself as a practical and attractive option for enterprises seeking to leverage AI without prohibitive costs.
Agents and the Future Vision of AI
From Assistants to Agents
IBM envisions a progression from AI assistants to more autonomous AI agents capable of complex reasoning and multi-step problem-solving:
- AI Assistants: Tools like watsonx Assistant have already improved significantly, aiding in tasks such as customer service, HR inquiries, and code assistance.
- AI Agents: The next evolution involves AI systems that can self-direct, reflect, and perform tasks with minimal human intervention. These agents can interact with other agents and systems, orchestrating complex workflows.
Agentic Capabilities
Granite 3.0 models are designed with agentic capabilities in mind:
- Advanced Reasoning: Enhanced abilities to understand context and make decisions.
- Tool Use Workflows: Structured prompting and interfaces that allow models to utilize external tools and resources.
- Multi-Agent Orchestration: Coordination between multiple AI agents to accomplish sophisticated tasks.
Future Developments
IBM plans to continue advancing agent technologies:
- Pre-Built Agents: Development of domain-specific agents for various industries and use cases.
- Integration with Existing Tools: Low friction Interaction with platforms like Adobe Firefly, Salesforce, and others.
- AI Agent Chat Feature: Upcoming features in IBM watsonx Orchestrate to enhance productivity across teams.
By focusing on agentic AI, IBM is preparing for a future where AI systems can handle more complex and dynamic business environments, driving significant productivity gains for organizations.
IBM’s Ecosystem and Partnerships
IBM emohasizes the importance of collaboration and integration in the AI ecosystem:
- Open-Source Contributions: By releasing models under Apache 2.0, IBM contributes to the broader AI community, fostering innovation and transparency. This also allows IBM to close the gap on first movers that are well capitalized.
- Strategic Partnerships: Collaborations with companies like Adobe, Salesforce, AWS, Meta, and others expand the reach and capabilities of Granite models.
- Integration with Partner Platforms: Granite models are being integrated into offerings from partners, providing clients with greater choice and flexibility.
This ecosystem approach allows IBM to leverage strengths from various partners, enhancing the overall value proposition for clients.
IBM’s Internal Use of AI and Client Successes
IBM as Client Zero
IBM is “dogfooding” its own models by being the first adopter of its AI technologies and claims the following as of this date:
- Cost Savings: Achieved $1.6 billion in cost efficiencies, with a line of sight to $3 billion by 2025.
- Automation: Automated 75% of queries through self-service AI, improving response times and reducing workload.
- Operational Improvements: Applied AI in procurement, invoicing, HR tasks, and more to streamline operations.
Client Examples
According to IBM, its clients across various industries are tapping the benefits of IBM’s AI solutions:
- Vodafone: Implemented “Tobi,” an AI assistant that reduced problem resolution times from days to minutes with 99% accuracy.
- Global Retailer: Used AI agents for marketing analysis and competitive intelligence, achieving productivity increases from 25% to 4x.
- Healthcare Company: Answering 46% of customer inquiries through AI, aiming to reach 70% efficiency.
These examples underscore the benefits of IBM’s AI solutions in real-world applications and in our view give us greater confidence that IBM’s Granite models are substantive innovations.
Bottom Line
IBM’s introduction of Granite 3.0 represents an important and strategic approach to enterprise AI. By focusing on smaller, efficient, and customizable models, IBM addresses the unique needs of businesses seeking to leverage their proprietary data. The integration of Granite models into IBM Consulting Advantage amplifies the potential impact, equipping a vast network of consultants with advanced AI tools.
IBM’s commitment to open-source principles, cost efficiency, and a vision for the future of AI positions it squarely in the mix of AI innovators. IBM’s emphasis on agents and the development of agentic capabilities indicate IBM’s forward-thinking approach to AI’s future.
Overall, we believed IBM’s direction aligns with the needs of enterprises looking for practical, trustworthy, and efficient AI solutions. Moreover, it signals a lower cost future to AI deployment. Specifically by using smaller models that are open source, organizations can more cost effectively build AI systems leveraging their own proprietary data sets.
If IBM continues to align its diverse business units and leverages its research capabilities effectively, it stands poised to become a major force in the AI industry, driving innovation and delivering substantial value to clients.
