top of page

The Rise of Composition of Experts: A Transformative New Paradigm for Enterprise AI

3/9/24

Editorial team at Bits with Brains

An exciting new approach to LLMs is emerging that promises to make custom, impactful, and responsible AI implementation more accessible.

Large Language Models are rapidly scaling, with architectures and datasets growing ever larger. This exponential trajectory can seem daunting for enterprise leaders exploring how AI can strategically empower their organizations.


Now, an exciting new approach is available that promises to make custom, impactful, and responsible AI implementation more accessible – composable AI.


Specifically, composable AI refers to blending specialized AI models together into ensembles tuned for specific tasks. This builds on the broader concept of a "composition of experts", mixing complementary specialized experts rather than relying on a single massive generalist foundation.


Recently AI startup SambaNova Systems unveiled a new composable architecture and accompanying models that demonstrate the huge potential of this technique. Their new "Samba-1" offering comprises over 50 distinct AI models woven together to achieve an unprecedented scale of 1 trillion parameters. This novel composition points towards the future of enterprise AI: highly customizable solutions blending targeted expert components to deliver efficient, secure and controlled capabilities.


Understanding Composable AI (CAI) Architectures

To appreciate how composable AI differs from conventional approaches, it helps to understand some key properties of this methodology:

  • CAI combines specialized narrow models rather than a single generalist model - Blends complementary "expert" models each trained on specific data for defined tasks. This balances broad capacity with specificity.

  • Features flexible dynamic composition - Expert models interact fluidly, with changing combinations responding to prompts. This allows exploring diverse perspectives.

  • Facilitates tuning for efficiency – CAI engages only required capacity for a given query, avoiding activating unused parameters. This optimized approach can deliver orders-of-magnitude efficiency gains.

  • Highly customizable for specific use cases - Modular architecture allows swapping components to target precise enterprise needs. Models can be trained on proprietary data.

  • Features privacy and security by design - Models inherit restrictions of training data, enabling tight control over sensitive data. Isolated training preserves integrity.


This Composition of Experts approach should not be confused with the Mixture of Experts approach, as the former keeps each expert model separately trained on its own secure dataset, ensuring the security and privacy of the training data.


These principles come together in SambaNova's new Samba-1 model. This 1 trillion parameter ensemble weaves together over 50 distinct models specialized for enterprise challenges. Models were curated from SambaNova's own catalog and trusted open-source libraries like Hugging Face. The composition pools capacity from state-of-the-art foundations like LLaMA, Mistral, and more. Each model concentrates on excelling at a particular task, or narrow range of tasks, from conversational AI to code generation and beyond.


Samba-1 is available within the SambaNova Suite, which also includes the company's SN40L AI chip, aiming to compete against industry leaders like Nvidia with a highly efficient approach for training and inference. This full-stack solution offers a 10x reduction in footprint, resulting in lower costs, power consumption, and infrastructure requirements for inference.


This flexible blending enables appropriately sized models to dynamically assemble for specific prompts. Rather than overwhelm queries with a full trillion parameters, efficiency emerges from engaging only the most relevant experts. Early benchmarks already demonstrate 10-100x performance gains over monolithic alternatives.


Customizing AI for Enterprise Impact

One major advantage of the composable approach is empowering robust customization targeted for enterprise needs. Conventional wisdom often implies that scaling model size is the dominant path to improve performance. But model architecture and tuning can be just as critical, if not more so. Composable AI allows emphasizing the most useful strengths for each business challenge.


For example, a biopharmaceutical firm could train experts on proprietary drug discovery data to gain unique advantages. A dedicated 20 billion parameter model expertly trained on internal datasets will often vastly outperform a more general 1 trillion parameter alternative. These owned assets integrate seamlessly into broader compositions for customized solutions.


Beyond training, curation and mixing expertise are also immensely powerful. Carefully selecting the most relevant models for each vertical and use case yields optimized impact. The dynamic interaction between experts even allows examining challenges from diverse viewpoints. For complex business decisions, exploring perspectives informed by different datasets can surface invaluable nuances.


Responsible and Secure Implementation

For many enterprise leaders, concerns around responsible and ethical AI implementation may override pure performance considerations. Here composable architectures also shine, enabling fine-grained control over security, privacy and impartiality.


Isolating training processes preserves integrity and confidentiality. Experts bound by their training data’s restrictions securely contribute to collective capabilities. This protects sensitive customer or business data from leakage across domains. Furthermore, bias and toxicity can be curtailed by judiciously selecting and tuning safe expert models. Dynamic interaction allows examining issues from debiased perspectives, undoing embedded prejudices.


Overall composable AI allows enterprises to implement sophisticated models while retaining tight control over critical factors. This permits accessing cutting-edge AI advances while upholding the highest ethical standards.


The Future of Enterprise AI

SambaNova's Samba-1 release provides a compelling glimpse into the future of composable AI. But this paradigm is still in its early stages. Many open questions remain regarding optimal composition design, mixture tuning, efficiency improvements and more. There is no single formula or architecture that guarantees success. Enterprise leaders still need to apply informed strategies rooted in their unique requirements and objectives.


That said, the emergence of composable AI promises to empower dramatic leaps forward in accessible enterprise AI adoption. As specialized models evolve and architectures mature, solutions will rapidly scale in sophistication. Unique, secure and highly efficient ensembles will dynamically serve use cases ranging from finance to healthcare and beyond.


The path forward will involve nurturing a robust ecosystem that pools together specialized talent, proprietary data and advanced hardware. But the promise is that any organization will be able to leverage AI, large and small. Where customized solutions will allow small firms to compete with giants. And where every business has access to friendly AI assistance, multi-perspective advisors and customizable intelligence amplification.


Sources:

[1] https://venturebeat.com/ai/sambanova-debuts-1-trillion-parameter-composition-of-experts-model-for-enterprise-gen-ai/

[2] https://insidehpc.com/2024/02/sambanova-says-its-first-with-trillion-parameter-genai-model/

[3] https://www.nextplatform.com/2024/02/28/sambanova-pits-llm-collective-against-monolithic-ai-models/

[4] https://sambanova.ai/products/samba-1

[5] https://www.linkedin.com/posts/mmoalem_sambanova-is-first-to-market-with-secure-activity-7168631759234957312-FhOw

[6] https://www.linkedin.com/posts/dana-bennett-74a750_sambanova-debuts-1-trillion-parameter-composition-activity-7168616511027228672-7wPx

[7] https://sambanova.ai

[8] https://www.techradar.com/pro/iphone-of-ai-ai-startup-first-to-deliver-trillion-plus-parameter-ai-model-that-works-in-symbiosis-with-its-very-own-chip-sambanova-promises-90-savings-on-inference-costs-but-take-that-with-a-pinch-of-salt

Sources

© 2023 Analytical Outcomes LLC, All Rights Reserved

bottom of page