Table of Contents
Full SOTA, but 8k window
Llama welcomes the official web version
One More Thing
Home Technology peripherals AI Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

Apr 19, 2024 pm 12:43 PM
python meta python program

Llama 3 is coming!

Just now, Meta’s official website was updated, officially announcing the Llama 3 8 billion and 70 billion parameter versions.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

And the launch is open source SOTA:

Meta official data shows that the Llama 3 8B and 70B versions surpass all opponents in their respective parameter scales.

8B model outperforms Gemma 7B and Mistral 7B Instruct on many benchmarks such as MMLU, GPQA, and HumanEval.

The 70B model has surpassed the popular closed-source Claude 3 Sonnet, and has gone back and forth with Google's Gemini Pro 1.5.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

As soon as the Huggingface link came out, the open source community became excited again.

The sharp-eyed blind students also immediately discovered Huadian:

Meta even hid a version of Llama 3 with 400 billion parameters, which is no less than the Claude 3 super large Opus!

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

The CEO of HyperWriteAI, an AI writing assistant startup, couldn’t help but sigh when he saw this:

We are entering a new world, a GPT -A world where level 4 models are open source and freely accessible.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

NVIDIA scientist Jim Fan believes that Llama 3 400B, which is still in training, will become a watershed for open source large models and change the development of many academic research and start-up companies. Way.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

Full SOTA, but 8k window

More technical details, Meta is given in the blog post.

At the architectural level, Llama 3 chose the classic decoder-only Transformer architecture, using a word segmenter containing a 128K token vocabulary.

Looking at the training data, the training data scale of Llama 3 has reached 15T tokens, all of which come from public information, of which 5% is non-English data, covering more than 30 languages.

Llama 3 has 7 times more training data than Llama 2, and has 4 times more code than Llama 2.

In addition, in order to improve the reasoning efficiency of the Llama 3 model, Meta AI also adopts the Group Query Attention (GQA) mechanism to train the model on a sequence of 8192 tokens, and uses a mask to ensure that self-attention is not Will cross document boundaries.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

As a result, whether it is the 8B or 70B version, Llama 3 has made a major leap forward compared to the previous generation Llama 2 of similar size.

Among the 8B and 70B parameter scale models so far, Llama 3 has become a new SOTA model.

In terms of language (MMLU), knowledge (GPQA), programming (HumanEval), mathematics (GSM-8K, MATH) and other capabilities, Llama 3 is almost completely ahead of other models of the same scale.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

In addition to these conventional data sets, Meta AI also evaluated the performance of Llama 3 in real-life scenarios and developed a high-quality test data set for this purpose.

This test set contains 1,800 pieces of data, covering 12 key use cases such as coding, reasoning, writing, and summary, and is confidential to the development team.

As a result, Llama 3 not only significantly surpassed Llama 2, but also defeated well-known models such as Claude 3 Sonnet, Mistral Medium and GPT-3.5.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

The performance of Llama 3 is also remarkable on higher-order and more difficult data sets such as AGIEval, BIG-Bench, and ARC-Challenge.

The 8B version surpassed Mistral and Gemma in these tasks, while the 70B version defeated Gemini Pro and Mixtral with MoE architecture, winning SOTAs of corresponding sizes respectively.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

However, the only drawback is that the context window of Llama 3 is only 8k. Compared with the current large models with dozens or millions of windows, it seems that it is still stuck in the previous generation. (Manual dog head).

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

But don’t worry too much. Matt Shumer is optimistic about this. He expressed his belief that with the efforts of the open source community, the window length will soon be expanded.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

Llama welcomes the official web version

Currently, the basic and Instruct versions of both parameters of Llama 3 are available on Hugging Face for download.

In addition, cloud service platforms such as Microsoft Azure, Google Cloud, Amazon AWS, and NVIDIA NIM will also launch Llama 3 one after another.

At the same time, Meta also said that Llama 3 will be supported by hardware platforms provided by Intel, Nvidia, AMD, Qualcomm and other manufacturers.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

It is worth mentioning that this time, together with the basic model, there is an official Web version based on Llama 3, called Meta AI.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

Currently, the platform has two major functions: dialogue and painting. If you only use dialogue, you don’t need to register and log in, and it can be used immediately. To use the painting function, you need to log in to your account first.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

However, the platform currently does not support Chinese, and functions such as text uploading have not yet been launched.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

In terms of code, the platform can also run some simple Python programs, but it seems that it can only output text, and tasks involving drawing cannot be run.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

# Overall, this web version is still relatively rudimentary, but you might as well look forward to a wave of subsequent updates.

One More Thing

A small incident is that in fact, a few hours before Meta’s official announcement, Microsoft’s Azure market had already stolen the news of the Llama 3 8B Instruct version.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

The Llama 3 price list on the open source model machine learning online platform Replicate was also immediately pulled out by netizens.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived

#But soon, these "little tidbits" were all 404'ed.

Fortunately, the mistake is over, and the official is not delaying it. Friends who care about open source large models can start to doge.

Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived


参考链接:
[1]https://ai.meta.com/blog/meta-llama-3/。
[2]https://about.fb.com/news/2024/04/meta-ai-assistant-built-with-llama-3/。
[3]https://huggingface.co/meta-llama/Meta-Llama-3-70B。

The above is the detailed content of Llama3 comes suddenly! The open source community is boiling again: the era of free access to GPT4-level models has arrived. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

How to run programs in terminal vscode How to run programs in terminal vscode Apr 15, 2025 pm 06:42 PM

In VS Code, you can run the program in the terminal through the following steps: Prepare the code and open the integrated terminal to ensure that the code directory is consistent with the terminal working directory. Select the run command according to the programming language (such as Python's python your_file_name.py) to check whether it runs successfully and resolve errors. Use the debugger to improve debugging efficiency.

Can visual studio code be used in python Can visual studio code be used in python Apr 15, 2025 pm 08:18 PM

VS Code can be used to write Python and provides many features that make it an ideal tool for developing Python applications. It allows users to: install Python extensions to get functions such as code completion, syntax highlighting, and debugging. Use the debugger to track code step by step, find and fix errors. Integrate Git for version control. Use code formatting tools to maintain code consistency. Use the Linting tool to spot potential problems ahead of time.

Is the vscode extension malicious? Is the vscode extension malicious? Apr 15, 2025 pm 07:57 PM

VS Code extensions pose malicious risks, such as hiding malicious code, exploiting vulnerabilities, and masturbating as legitimate extensions. Methods to identify malicious extensions include: checking publishers, reading comments, checking code, and installing with caution. Security measures also include: security awareness, good habits, regular updates and antivirus software.

Can vs code run in Windows 8 Can vs code run in Windows 8 Apr 15, 2025 pm 07:24 PM

VS Code can run on Windows 8, but the experience may not be great. First make sure the system has been updated to the latest patch, then download the VS Code installation package that matches the system architecture and install it as prompted. After installation, be aware that some extensions may be incompatible with Windows 8 and need to look for alternative extensions or use newer Windows systems in a virtual machine. Install the necessary extensions to check whether they work properly. Although VS Code is feasible on Windows 8, it is recommended to upgrade to a newer Windows system for a better development experience and security.

Python: Automation, Scripting, and Task Management Python: Automation, Scripting, and Task Management Apr 16, 2025 am 12:14 AM

Python excels in automation, scripting, and task management. 1) Automation: File backup is realized through standard libraries such as os and shutil. 2) Script writing: Use the psutil library to monitor system resources. 3) Task management: Use the schedule library to schedule tasks. Python's ease of use and rich library support makes it the preferred tool in these areas.

What is vscode What is vscode for? What is vscode What is vscode for? Apr 15, 2025 pm 06:45 PM

VS Code is the full name Visual Studio Code, which is a free and open source cross-platform code editor and development environment developed by Microsoft. It supports a wide range of programming languages ​​and provides syntax highlighting, code automatic completion, code snippets and smart prompts to improve development efficiency. Through a rich extension ecosystem, users can add extensions to specific needs and languages, such as debuggers, code formatting tools, and Git integrations. VS Code also includes an intuitive debugger that helps quickly find and resolve bugs in your code.

Can visual studio code run python Can visual studio code run python Apr 15, 2025 pm 08:00 PM

VS Code not only can run Python, but also provides powerful functions, including: automatically identifying Python files after installing Python extensions, providing functions such as code completion, syntax highlighting, and debugging. Relying on the installed Python environment, extensions act as bridge connection editing and Python environment. The debugging functions include setting breakpoints, step-by-step debugging, viewing variable values, and improving debugging efficiency. The integrated terminal supports running complex commands such as unit testing and package management. Supports extended configuration and enhances features such as code formatting, analysis and version control.

Python vs. JavaScript: The Learning Curve and Ease of Use Python vs. JavaScript: The Learning Curve and Ease of Use Apr 16, 2025 am 12:12 AM

Python is more suitable for beginners, with a smooth learning curve and concise syntax; JavaScript is suitable for front-end development, with a steep learning curve and flexible syntax. 1. Python syntax is intuitive and suitable for data science and back-end development. 2. JavaScript is flexible and widely used in front-end and server-side programming.

See all articles