Home > Technology peripherals > AI > body text

The open source model wins GPT-4 for the first time! Arena's latest battle report has sparked heated debate, Karpathy: This is the only list I trust

王林
Release: 2024-04-10 15:16:14
forward
704 people have browsed it

An open source model that can beat GPT-4 has appeared!

The latest battle report of the large model arena:

The 104 billion parameter open source model Command R climbed to 6th place, tying with GPT-4-0314 and surpassing GPT-4-0613.

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

This is also the first open-weight model to beat GPT-4 in the large model arena.

The large model arena is one of the only test benchmarks that the master Karpathy trusts.

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

Command R from AI unicorn Cohere. The co-founder and CEO of this large model startup is none other than Aidan Gomez, the youngest author of Transformer (referred to as the wheat reaper).

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

As soon as this battle report came out, it set off another wave of heated discussions in the large model community.

The reason why everyone is excited is simple: the basic large model has been rolled out for a whole year. Unexpectedly, the pattern will continue to develop and change in 2024.

HuggingFace co-founder Thomas Wolf said:

The situation in the large model arena has changed dramatically recently:

Anthropic’s Claude 3 opus is in the closed source model Take the lead among them.

Cohere's Command R has become the strongest among the open source models.

Unexpectedly, in 2024, the artificial intelligence team will develop so fast on both open source and closed source routes.

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

In addition, Cohere Machine Learning Director Nils Reimers also pointed out something worthy of attention:

The biggest feature of Command R is The built-in RAG (Retrieval Enhanced Generation) has been fully optimized, but in the large model arena, plug-in capabilities such as RAG were not included in the test.

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

RAG optimization model ascends to the throne of open source

In Cohere’s official positioning, Command R is a “RAG optimization model” .

That is to say, this large model with 104 billion parameters has been deeply optimized for retrieval enhancement generation technology to reduce the generation of hallucinations and is more suitable for enterprise-level workloads.

Like the previously launched Command R, the context window length of Command R is 128k.

In addition, Command R also has the following features:

  • Covers 10 languages, including English, Chinese, French, German, etc.;
  • Can use tools to complete complex tasks Automation of business processes

Judging from the test results, Command R has reached the level of GPT-4 turbo in the three dimensions of multi-language, RAG and tool usage.

But in terms of input cost, the price of Command R is only 1/3 of GPT-4 turbo.

In terms of output cost, Command R is 1/2 of GPT-4 turbo.

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

It is this that has attracted the attention of many netizens:

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

However, despite its eye-catching performance in human subjective evaluations such as the large model arena, some netizens still expressed some different opinions.

On HumanEval, Command R’s coding ability has not even surpassed GPT-3.5, ranking 32nd and 33rd respectively in the two sets of tests.

The latest version of GPT-4 turbo won the first place without any suspense.

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

In addition, we also briefly tested the Chinese ability of Command R on the Mentally Retarded Benchmark, which has recently been listed in serious papers.

The open source model wins GPT-4 for the first time! Arenas latest battle report has sparked heated debate, Karpathy: This is the only list I trustPicture

How would you rate it?

It should be noted that the open source of Command R is only for academic research and is not free for commercial use.

One More Thing

Finally, let’s talk more about the wheat-cutting guy.

Aidan Gomez, the youngest of the Transformer Knights of the Round Table, was just an undergraduate when he joined the research team——

However, he joined the Hinton experiment when he was a junior at the University of Toronto. The kind of room.

In 2018, Kaomaizi was admitted to Oxford University and began studying for a PhD in CS like his thesis partners.

But in 2019, with the founding of Cohere, he finally chose to drop out of school and join the wave of AI entrepreneurship.

Cohere mainly provides large model solutions for enterprises, and its current valuation has reached US$2.2 billion.

Reference link:
[1]https://www.php.cn/link/3be14122a3c78d9070cae09a16adcbb1[2]https://www.php.cn/ link/93fc5aed8c051ce4538e052cfe9f8692

The above is the detailed content of The open source model wins GPT-4 for the first time! Arena's latest battle report has sparked heated debate, Karpathy: This is the only list I trust. For more information, please follow other related articles on the PHP Chinese website!

Related labels:
source:51cto.com
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template
About us Disclaimer Sitemap
php.cn:Public welfare online PHP training,Help PHP learners grow quickly!