Table of Contents
Nearly 90 models competed in 750,000 rounds
A large model test that everyone can participate in
△Schematic diagram, earlier version
One More Thing
Home Technology peripherals AI 750,000 rounds of one-on-one battle between large models, GPT-4 won the championship, and Llama 3 ranked fifth

750,000 rounds of one-on-one battle between large models, GPT-4 won the championship, and Llama 3 ranked fifth

Apr 23, 2024 pm 03:28 PM
bootstrap Version gpt-4 arrangement llama 3

Regarding Llama 3, another test result has been released——

The large model evaluation community LMSYS released a large model ranking list, Llama 3 ranked fifth, English single item and GPT -4 tied for first place.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

Different from other Benchmarks, this list is based on model one-on-one battles, and the evaluators from all over the network make their own propositions and scores.

In the end, Llama 3 ranked fifth on the list, followed by three different versions of GPT-4 and Claude 3 Super Cup Opus.

In the English single list, Llama 3 overtook Claude and tied with GPT-4.

Regarding this result, Meta's chief scientist LeCun was very happy, retweeted the tweet and left a "Nice".

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

Soumith Chintala, the father of PyTorch, also said excitedly that such results are incredible and he is proud of Meta .

The 400B version of Llama 3 has not yet come out, and it won the fifth place based on 70B parameters alone...
I still remember that when GPT-4 was released in March last year, it achieved the same level. Performance is almost impossible.
……
The popularization of AI now is truly incredible, and I am very proud of my colleagues at Meta AI for achieving such success.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

So, what specific results does this list show?

Nearly 90 models competed in 750,000 rounds

As of the release of the latest list, LMSYS has collected nearly 750,000 large model solo battle results, involving 89 models.

Among them, Llama 3 has participated 12,700 times, and GPT-4 has multiple different versions, with the most participating 68,000 times.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

The picture below shows the number of competitions and winning rates of some popular models. Neither of the two indicators in the picture counts the number of draws.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

In terms of the list, LMSYS is divided into a general list and multiple sub-lists. GPT-4-Turbo ranked first and tied with it. The ones are the earlier 1106 version and the Claude 3 extra large Opus.

Another version (0125) of GPT-4 ranks second, followed closely by Llama 3.

But what’s more interesting is that the newer version 0125 does not perform as well as the older version 1106.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

In the English single list, Llama 3's results directly tied with the two GPT-4s, and even surpassed 0125 Version.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

The first place in the Chinese proficiency ranking is shared by Claude 3 Opus and GPT-4-1106, while Llama 3 has been ranked 20 Outstanding name.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

In addition to language ability, the list also includes long text and code ability rankings, and Llama 3 is also among the best.

However, what are the specific "rules of the game" of LMSYS?

A large model test that everyone can participate in

This is a large model test that everyone can participate in. The questions and evaluation criteria are decided by the participants.

The specific "competition" process is divided into two modes: battle and side-by-side.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

In battle mode, after entering the question in the test interface, the system will randomly call the two models in the library, and the tester will not Knowing who the system has drawn, only "Model A" and "Model B" are displayed on the interface.

After the model outputs the answer, the evaluator needs to choose which one is better or a tie. Of course, if the model's performance does not meet expectations, there are corresponding options.

Only after a selection is made, the model's identity is revealed.

Side-by-side is where the user selects the specified model for PK. The rest of the test process is the same as the battle mode.

However, only the voting results in the anonymous mode of the battle will be counted, and in If the model accidentally reveals its identity during the conversation, the results will be invalid.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

According to the Win Rate of each model to other models, such an image can be drawn:

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

△Schematic diagram, earlier version

The final ranking is obtained by using Win Rate data and converting it into scores through the Elo evaluation system.

The Elo rating system is a method of calculating the relative skill level of players, designed by American physics professor Arpad Elo.

Specifically for LMSYS, under the initial conditions, the ratings (R) of all models are set to 1000, and then the expected winning rate (E) is calculated based on this formula.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

As the test continues, the score will be corrected based on the actual score (S). There are three types of S: 1, 0 and 0.5 The values ​​correspond to three situations: winning, losing and drawing.

The correction algorithm is shown in the following formula, where K is the coefficient, which needs to be adjusted by the tester according to the actual situation.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

After all valid data are finally included in the calculation, the Elo score of the model is obtained.

However, during the actual operation, the LMSYS team found that the stability of this algorithm was insufficient, so it used statistical methods to correct it.

They used the Bootstrap method for repeated sampling, obtained more stable results, and estimated confidence intervals.

The final revised Elo score became the basis for ranking in the list.

One More Thing

Llama 3 can already run on the large model inference platform Groq (not Musk’s Grok).

The biggest highlight of this platform is "fast". Previously, the Mixtral model has been used to run at a speed of nearly 500 tokens per second.

It is also quite fast when running Llama 3. According to the actual test, the 70B version can run about 300 Tokens per second, and the 8B version is close to 800.

大模型一对一战斗75万轮,GPT-4夺冠,Llama 3位列第五Picture

Reference link:
[1]https://lmsys.org/blog/2023-05-03- arena/
[2]https://chat.lmsys.org/?leaderboard
[3]https://twitter.com/lmsysorg/status/1782483699449332144

The above is the detailed content of 750,000 rounds of one-on-one battle between large models, GPT-4 won the championship, and Llama 3 ranked fifth. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

The world's most powerful open source MoE model is here, with Chinese capabilities comparable to GPT-4, and the price is only nearly one percent of GPT-4-Turbo The world's most powerful open source MoE model is here, with Chinese capabilities comparable to GPT-4, and the price is only nearly one percent of GPT-4-Turbo May 07, 2024 pm 04:13 PM

Imagine an artificial intelligence model that not only has the ability to surpass traditional computing, but also achieves more efficient performance at a lower cost. This is not science fiction, DeepSeek-V2[1], the world’s most powerful open source MoE model is here. DeepSeek-V2 is a powerful mixture of experts (MoE) language model with the characteristics of economical training and efficient inference. It consists of 236B parameters, 21B of which are used to activate each marker. Compared with DeepSeek67B, DeepSeek-V2 has stronger performance, while saving 42.5% of training costs, reducing KV cache by 93.3%, and increasing the maximum generation throughput to 5.76 times. DeepSeek is a company exploring general artificial intelligence

NVIDIA dialogue model ChatQA has evolved to version 2.0, with the context length mentioned at 128K NVIDIA dialogue model ChatQA has evolved to version 2.0, with the context length mentioned at 128K Jul 26, 2024 am 08:40 AM

The open LLM community is an era when a hundred flowers bloom and compete. You can see Llama-3-70B-Instruct, QWen2-72B-Instruct, Nemotron-4-340B-Instruct, Mixtral-8x22BInstruct-v0.1 and many other excellent performers. Model. However, compared with proprietary large models represented by GPT-4-Turbo, open models still have significant gaps in many fields. In addition to general models, some open models that specialize in key areas have been developed, such as DeepSeek-Coder-V2 for programming and mathematics, and InternVL for visual-language tasks.

How to check traffic on Apple mobile phone How to check traffic on Apple mobile phone May 09, 2024 pm 06:00 PM

How to check data usage on Apple 1. The specific steps to check data usage on Apple mobile phone are as follows: Open the settings of the phone. Click the Cellular button. Scroll down on the cellular network page to see the specific data usage of each application. Click Apply to also set allowed networks. 2. Turn on the phone, find the settings option on the phone desktop, and click to enter. In the settings interface, find "Cellular Network" in the taskbar below and click to enter. In the cellular network interface, find the "Usage" option on the page and click to enter. 3. Another way is to check the traffic by yourself through the mobile phone, but the mobile phone can only see the total usage and will not display the remaining traffic: turn on the iPhone, find the "Settings" option and open it. Select "Bee"

How to disable snapshot layout in Windows 11_ Tips for not using snapshot layout in win11 How to disable snapshot layout in Windows 11_ Tips for not using snapshot layout in win11 May 08, 2024 pm 06:46 PM

Win11 system announced the new [Snapshot Layout], which provides users with various window layout options through the [Maximize] button, so that users can choose from multiple layout templates to display two, three or four on the screen. open applications. This is an improvement over dragging multiple windows to the sides of the screen and then adjusting everything manually. [SnapGroups] will save the collection of apps the user is using and their layout, allowing the user to easily return to that setting when they have to stop and deal with other things. If someone is using a monitor that the user must unplug, when re-docking, the previously used snapshot layout will also be restored. To use snapshot layout, we can use the keyboard shortcut WindowsKey+Z to start

How to sort the list page alphabetically in vscode How to sort the list page alphabetically in vscode How to sort the list page alphabetically in vscode How to sort the list page alphabetically in vscode May 09, 2024 am 09:40 AM

1. First, after opening the vscode interface, click the settings icon button in the lower left corner of the page 2. Then, click the Settings option in the drop-down page column 3. Then, find the Explorer option in the jumped window 4. Finally, on the right side of the page Click the OpenEditorsnaming option, select the alphabetical button from the drop-down page and save the settings to complete the alphabetical sorting

The operation process of creating bottle type text layout effect with AI The operation process of creating bottle type text layout effect with AI May 06, 2024 pm 02:28 PM

1. Open the material picture of a bottle in AI and type the text content that needs to be produced on the side. 2. Cancel the fill color of the bottle and only stroke it to form a hollow closed path. 3. Adjust the font size, font and line spacing of the text, and arrange the bottle layers to the top. 4. Select the text and the bottle at the same time, click Object-Envelope Distortion-Create with top-level object, and you will get a bottle-shaped text group. 5. Double-click the text to enter the isolation mode, and you can modify the text content and change the color. After the modification, the bottle shape will not be affected when exiting isolation mode. The final effect is as follows:

How to set ring text in ai - specific method of setting ring text in ai How to set ring text in ai - specific method of setting ring text in ai May 06, 2024 pm 05:58 PM

1. First, after opening the interface, click the Ellipse tool to draw a perfect circle 2. Click the Path Text tool button on the left and enter text along the circular frame 3. Select the letter with the mouse, open the character panel, and set the font size to 20.7 pt4. Select the circle, click 3D options in the effect menu, and select the rotation button 5. In the opened 3D rotation option settings, set the position option to custom rotation effect, modify the parameters and click OK to save 6. Finally, it is a ring Just add a red fill effect to the text

Can I plug in a wireless network card when assembling a computer? Can I plug in a wireless network card when assembling a computer? May 08, 2024 am 09:13 AM

Can I plug in a wireless network card when assembling a computer? First of all, the wireless network card you are talking about here should be a 2G/3G/4G wireless network card, that is, a wireless network card, right? My answer is yes. However, you also need an AP that supports USB wireless network cards, such as: (only for Jiuli use, not a recommended product) Can I use a wireless network card to access the Internet by assembling a desktop computer? Network cards are essential for modern computers. Without a network card, you cannot access the Internet, whether it is an onboard network card, an independent network card, or a wireless network card. When assembling a computer, a separate network card is generally not installed, because the current motherboards have integrated network cards, so there is no need to buy another one. However, the computers assembled now cannot use wireless Internet access like notebooks, because there is no wireless network card installed. Players can According to your own needs

See all articles