Grok-2 is here, it can generate images and recognize images, and its performance is comparable to GPT-4o. Musk: It is developing like a rocket

GPT-5 is not out yet, Grok has caught up.

On the same day that Google and OpenAI were grabbing news from each other, Musk’s xAI was not idle either.

On Wednesday afternoon Beijing time, xAI officially released the new generation Grok 2 large model.

Chatbot Arena, a third-party large model benchmark organization, also immediately updated the results list of the LMSYS list. The early model of Grok 2 (sus-column-r) can be ranked fourth just behind GPT-4o (version 0513), outperforming Claude 3.5 Sonnet and GPT-4-Turbo.

It excels at coding, complex problems and math.

Musk couldn't help but boast, "Grok's propulsion speed is like a rocket."

Note that this is only the score of the early version. Chatbot Arena said that it will test the official version in the future.

Musk said that Grok-2 is an advanced language model with the most advanced reasoning capabilities. The new generation includes two versions: Grok-2 and Grok-2 mini. Both models are now released on the X platform for Grok users. Currently, X Premium and Premium+ users can already experience the Grok-2 and Grok-2 mini models.

Compared with the previous Grok-1.5, the early preview version of Grok-2 has achieved significant progress, demonstrating leading capabilities in chat, reasoning, coding, etc. Grok-2 and Grok-2 mini are currently in beta on X and will be available via an enterprise API later this month, xAI said.

Less than half an hour after the new model was released, a netizen was already showing off the results. He used Grok 2 mini to generate an image of "Me and Musk eating hot dogs."

Try other methods to generate a portrait of Washington.

Some people also tried Grok 2 mini to generate a flying cat.

Someone else built a Tesla Model Y, does it look similar?

Grok-2 performance PK

With xAI putting the early version of Grok-2 "sus-column-r" into Chatbot Arena, we see it competing with other popular switches Performance comparison of source models.

In terms of overall Elo score, Grok-2 performs better than Claude series models and most versions of GPT-4. Of course, the first one on the list is GPT-4o (version August 8), which OpenAI just released these days.

The picture below shows the Win Rate comparison between Grok-2 and other popular models.

The picture below shows a fact-based win rate comparison between the two versions of Grok 1.5 and Grok 2.

xAI adopts this process to evaluate the Grok 2 model, using AI Tutors to truly interact with the model in various tasks. During each interaction, Grok 2 provides two responses to the AI ​​Tutors and then selects the best response based on specific criteria listed in the guide.

xAI focuses on evaluating model performance in two key areas, namely instruction following and providing accurate, authentic information. The results show significant improvements in Grok 2's ability to reason from retrieved content and use tools such as correctly identifying missing information, reasoning through sequences of events, discarding irrelevant posts, etc.

Benchmark Scores

xAI evaluated the Grok-2 model across a range of academic benchmarks including Reasoning, Reading Comprehension, Mathematics, Science, and Coding.

Both the Grok-2 and Grok-2 mini are significant improvements over the previous Grok-1.5 model. Performance is comparable to other cutting-edge models in areas such as graduate-level science knowledge (GPQA), general knowledge (MMLU, MMLU-Pro), and mathematics competition problems (MATH).

In addition, Grok-2 also performs well on vision-based tasks, with remarkable performance in visual mathematical reasoning (MathVista) and document-based question answering (DocVQA).

Grok 2 interface and functionality "big makeover"

In the past few months, xAI has been continuously improving the Grok experience on the x platform. Now, with the launch of the next generation Grok 2, xAI has redesigned the interface, as shown below.

Of course, xAI provides some new features, such as a simple implementation of Conway's "Game of Life".

Another example is multi-modal understanding ability (looking at pictures and talking).

Among them, Grok-2 is xAI’s most advanced AI assistant, with text and visual understanding capabilities and integrated real-time information from the X platform, accessible through the Grok tab in the X application.

The Grok-2 mini is a small but powerful model that strikes a good balance between speed and answer quality.

Compared to its predecessor, Grok-2 is more intuitive, more controllable and more flexible, suitable for a variety of tasks, whether you are looking for answers, collaborative writing or solving coding tasks.

Additionally, xAI is partnering with startup Black Forest Labs to experiment with their FLUX.1 model to extend Grok’s capabilities on X.

Later this month, xAI will also release Grok-2 and Grok-2 mini to developers via a new enterprise API platform. The upcoming API is built on a new custom technology stack that allows for multi-region inference deployments for global low-latency access.

Of course, xAI also offers some enhanced security features, such as mandatory multi-factor authentication (e.g. using Yubikey, Apple TouchID or TOTP).

As you can see, since the launch of Grok-1 in November 2023, xAI has been advancing this series of models at an alarming rate. Soon, they will release a preview version with multi-modal understanding. The focus after xAI will be to improve the core reasoning capabilities of the model through new computing clusters.

^{Blog address: https://x.ai/blog/grok-2}

The above is the detailed content of Grok-2 is here, it can generate images and recognize images, and its performance is comparable to GPT-4o. Musk: It is developing like a rocket. For more information, please follow other related articles on the PHP Chinese website!