Meta launches Llama 3 with versions of up to 405 billion parameters and challenges GPT-4

The new era of open-source models

In July 2024, the Meta made the release of Llama 3, its third generation of open-source large language models (LLM), offering three distinct versions: 8 billion, 70 billion and 405 billion parameters. The announcement represents an important milestone in the democratization of advanced AI, offering the community of developers and researchers free access to a technology that, in its most robust version, competes directly with proprietary models such as OpenAI GPT-4 and Anthropic Claude 3.5.

Verified capabilities and performance

According to official data from Meta, the Llama 3.1 of 405 billion parameters achieved impressive results in the industry's main benchmarks. In the MMLU (Massive Multitask Language Understanding), which assesses general knowledge, the model reached 88.6%, surpassing GPT-4 (86.5%) and Claude 3.5 Sonnet (88.3%). In mathematical reasoning tasks (MATH), Llama 3.1 405B obtained 73.8%, while GPT-4 reached 74.6%.

The smaller versions also demonstrate competitive performance: the Llama 3.1 70B rivals medium-sized commercial models, while the 8B stands out for efficiency in devices with limited resources. All models have been trained in more than 15 trillion tokens and support a context of up to 128 thousand tokens, allowing extensive documents to be processed.

Open source commitment

Meta's decision to make Llama 3.1 available free of charge to researchers and developers reinforces the company's commitment to open-source philosophy. This approach contrasts with the strategy of companies such as OpenAI and Google, which maintain their most advanced models under restricted and commercial access.

The model is available under the Llama 3.1 license Community License, which allows commercial use for companies with less than 700 million monthly users. This permissiveness enables smaller startups and companies to develop state-of-the-art AI-based products and services without the prohibitive costs of licensing or use of paid APIs.

Impact on the AI ecosystem

The launch of Llama 3.1 has the potential to significantly accelerate innovation in AI. Researchers can explore new applications, develop fine-tuning for specific use cases and contribute to the collective advancement of technology. Companies will have the opportunity to implement sophisticated AI solutions while maintaining full control over their data and infrastructure.

Meta highlighted cases of use as AI agents, code generation and long document processing tools. Furthermore, the availability of such a powerful open source model promotes greater transparency and public scrutiny of the functioning of AI systems, crucial aspects for security, bias and ethics issues in the development of artificial intelligence.

Infrastructure and availability

According to Meta, Llama 3.1 405B training used more than 16,000 NVIDIA H100 GPUs, representing one of the greatest computational efforts in open-source AI. The templates are available for download via the official Meta AI website and on platforms such as Hugging Face, and are accessible via API on cloud services such as AWS, Google Cloud and Microsoft Azure.

With Llama 3.1, Meta consolidates its position as an open-source AI leader and establishes a new standard for accessible language models, challenging the hegemony of proprietary systems and expanding the possibilities of democratic innovation in artificial intelligence.