Meta’s Llama 3 Unleashed: Meta's AI Flexes Its Billion-Parameter Muscles

4/20/24

Editorial team at Bits with Brains

Llama 3 is Meta's much anticipated latest iteration of its large language model (LLM), following the open-source Llama 2.

Llama 3 has been released in two versions: one with 8 billion parameters and another with 70 billion parameters. This model represents a significant advancement in AI capabilities, particularly in terms of performance for its scale. The model has been trained on a dataset that is seven times larger than that used for the very popular Llama 2, which includes a substantial amount of code and multilingual text. This training has allowed Llama 3 to outperform other models in its class on various benchmarks.

Here's a summary of how Llama 3 stacks up against other models:

Benchmark Performance

On the MMLU benchmark, which is a common benchmark for AI models, Llama 3's 8 billion parameter model leads with 68.4%, outperforming Google’s Gemma 7B and Mistral 7B. The 70 billion parameter version of Llama 3 has an MMLU score of 82%, which narrowly surpasses Gemini Pro 1.5 and Sonnet’s scores.

Llama 3's 70 billion parameter model also outperforms high-profile models like OpenAI's GPT-3.5 and Google's Gemini on tasks including coding, creative writing, and summarization. In human evaluations, Llama 3 received higher marks compared to other models, including OpenAI's GPT-3.5.

Training Data and Scale

Llama 3 was trained on a dataset comprising 15 trillion tokens, which is about seven times the size of the dataset used for its predecessor Llama 2, contributing to its improved performance. Somewhat disappointingly, the model only supports an 8K context length. Nevertheless, this is double the capacity of Llama 2 and reportedly excels at language nuance, contextual understanding, and complex translation or dialogue generation tasks

Integration and Accessibility

Llama 3 models are already integrated into the Hugging Face ecosystem, making them readily available to developers. Meta also plans to make Llama 3 models available on major cloud platforms like AWS, Databricks, Google Cloud, and others, ensuring broad accessibility for developers.

Comparison with Proprietary Models

While OpenAI’s GPT-4 and Claude Opus are still the frontrunners in terms of state-of-the-art performance, Llama 3's advancements may be enough to shift enterprises’ approach to AI, with the open-source community potentially offering more innovation than any single organization.

Llama 3's performance on various benchmarks and human evaluations makes it a formidable competitor to proprietary models, and it debuted among the top 5 on the AI leaderboard, being the only non-proprietary model to do so, which is very significant given that proprietary models usually outperform open-source models. As we’ve said previously, that gap is narrowing quickly.

Other Features

One of the standout features of Llama 3 is its integration of real-time knowledge from Google and Bing directly into its responses. This integration allows the model to provide up-to-date information and a wider range of answers to user queries. By not relying solely on its training data, Llama 3 can access a broader spectrum of information, making it a more versatile and powerful tool for users seeking information on the web.

Llama 3 also introduces unique creation features, such as the ability to generate animations and high-quality images in real time. This feature enhances the interactive experience with the model, allowing for more creative and dynamic use cases as you can adjust the input in response to the output as it unfolds.

While both current releases of Llama 3 are impressive, there is significant anticipation for the upcoming 400 billion parameter model. This larger model is expected to have enhanced capabilities, including multimodality, the ability to converse in multiple languages, larger context windows, and stronger overall performance. It is poised to compete with current models like GPT-4 and Claude 3 Opus, and early benchmark scores suggest it will be a formidable contender.

The model is also expected to be available on Groq, an inference platform that speeds up the processing of large language models. This availability ensures that Llama 3 can be utilized efficiently and effectively across various platforms.

Llama 3 is currently available on Hugging Face, allowing users to interact with the model via an API. Additionally, Meta has launched a new website, Meta.ai, which provides a user-friendly interface for interacting with Llama 3. This website includes an AI image generator that can create images in real time as users type, further demonstrating the model's capabilities in content creation.

No one is surprised at the release of Llama 3 as Meta had been transparent about its development. However, it represents a significant step forward in gen AI, particularly for open source models. A quantized version of Llama 3 was already available within hours of its release by Meta.

Llama 3's performance and features are expected to significantly influence the industry, particularly as domain-specific fine-tuned versions start to become available.

Sources:

[1] https://youtu.be/ybI3Y2zsFOM?si=f6i81nzPDJgdpBaC

[2] https://www.youtube.com/watch?v=BHFaG4EMdaI

[3] https://www.reddit.com/r/LocalLLaMA/comments/1ar4gw1/is_llama3_going_to_have_a_model_with_more_than/

[4] https://www.reddit.com/r/singularity/comments/1c777f2/introducing_meta_llama_3_the_most_capable_openly/

[5] https://siliconangle.com/2024/04/18/meta-debuts-next-generation-llama-3-llm-series-new-chatbot-features/

[6] https://www.theregister.com/2024/04/19/meta_debuts_llama3_llm/

[7] https://huggingface.co/meta-llama/Meta-Llama-3-8B

[8] https://www.searchenginejournal.com/meta-integrates-google-bing-search-results-into-ai-assistant/514291/

[9] https://techcrunch.com/2024/04/18/meta-releases-llama-3-claims-its-among-the-best-open-models-available/

[10] https://www.infoworld.com/article/3715265/meta-eyes-llm-dominance-with-new-llama-3-models.html

[11] https://www.mediapost.com/publications/article/395398/meta-integrates-real-time-bing-and-google-search-w.html