Table of Contents
Function Call Capability
Home Technology peripherals AI Tsinghua University and Zhipu AI open source GLM-4: launching a new revolution in natural language processing

Tsinghua University and Zhipu AI open source GLM-4: launching a new revolution in natural language processing

Jun 12, 2024 pm 08:38 PM
ai code glm-4

Since the launch of ChatGLM-6B on March 14, 2023, the GLM series models have received widespread attention and recognition. Especially after ChatGLM3-6B was open sourced, developers are full of expectations for the fourth-generation model launched by Zhipu AI. This expectation has finally been fully satisfied with the release of GLM-4-9B.

The birth of GLM-4-9B

In order to give small models (10B and below) more powerful capabilities, the GLM technical team spent nearly half a year exploring , launched this new fourth-generation GLM series open source model: GLM-4-9B. This model greatly compresses the model size while ensuring accuracy, and has faster inference speed and higher efficiency. There is no end to the exploration of the GLM technical team, and we will continue to work hard to launch more competitive open source

Innovative pre-training technology

In the pre-training process, we introduce large The language model performed data screening and finally obtained 10T of high-quality multilingual data. This amount of data is more than three times that of the ChatGLM3-6B model. In addition, we use FP8 technology for efficient pre-training, which improves training efficiency by 3.5 times compared to the third-generation model. Taking into account the user's storage needs, the parameter size of GLM-4-9B has been increased from 6B to 9B. Ultimately, we increased the pre-training computation by 5 times to maximize performance capabilities under limited storage conditions.

Excellent performance display

GLM-4-9B is a comprehensive technology upgrade tool with more powerful inference performance and better It has the advantages of context processing capabilities, multi-language support, multi-modal processing, and full tool set All Tools calling. These upgrades provide users with more stable, more reliable, and more accurate technical support, and improve users' work efficiency and quality.

The GLM-4-9B series includes multiple versions:

  • Basic version: GLM-4-9B (8K)
  • Conversational version: GLM -4-9B-Chat (128K)
  • Extra-long context version: GLM-4-9B-Chat-1M (1M)
  • Multi-modal version: GLM-4V-9B-Chat (8K)

GLM-4-9B’s powerful abilities

Basic abilities

GLM-4- Based on strong pre-training, 9B’s comprehensive ability in Chinese and English has improved by 40% compared to ChatGLM3-6B. In particular, significant improvements have been achieved in the Chinese alignment capability AlignBench, the instruction compliance capability IFeval, and the engineering code processing capability Natural Code Bench. Even when comparing the Llama 3 8B model with more training volume, GLM-4-9B is not inferior at all and leads in English performance. In the field of Chinese subjects, GLM-4-9B has improved by up to 50% [Performance Evaluation chart].

Long text processing capability

清华大学与智谱AI重磅开源 GLM-4:掀起自然语言处理新革命Pictures

The context length of the GLM-4-9B+ model is expanded from 128K At 1M tokens, it means that it can process input of up to 2 million words at the same time, which is equivalent to the length of two "Dream of Red Mansions" or 125 academic papers. The GLM-4-9B-Chat-1M model successfully demonstrated its excellent ability to non-destructively process long text input in the "needle in the haystack" experiment [illustration of long text experiment].

The following are two demo video cases showing long text processing capabilities:

  1. GLM-4-9B-Chat model: Input 5 PDF files with a total length of about 128K, and give a prompt to write a detailed research report on the development of large models in China. The model can quickly generate high-quality research reports (the video is not accelerated).
  2. GLM-4-9B-Chat-1M Model: Input about 900,000 words of the complete collection of "The Three-Body Problem" and ask the model to write a sequel outline for the novel. The model is reasonably planned and provides a continuation framework (video accelerated 10 times).

Multi-language support

GLM-4-9B+ supports up to 26 languages, including Chinese, English, Russian, etc. We expanded the tokenizer vocabulary size from 65K to 150K, improving coding efficiency by 30%. In multi-language understanding and generation tasks, GLM-4-9B-Chat outperforms Llama-3-8B-Instruct [Multi-language performance comparison chart].

Function Call Capability

The function calling capability of GLM-4-9B has been improved by 40% compared to the previous generation. On the Berkeley Function-Calling Leaderboard, its Function Call The capabilities are comparable to GPT-4 [Function call performance comparison chart].

All Tools full tool call

The "All Tools" capability means that the model can understand and use various external tools (such as code execution, network browsing, and drawing) etc.) to assist in completing the task. At the Zhipu DevDay on January 16, the GLM-4 model was fully upgraded with All Tools capabilities, which can intelligently call web browsers, code interpreters, CogView and other tools to complete complex requests [All Tools task icon].

Multimodal processing

GLM-4V-9B As an open source multimodal model of the GLM-4 base, capable of processing high-resolution input, By directly mixing visual and text data for training, it demonstrates significant multi-modal processing effects and is comparable to GPT-4V in performance. It performs very well when identifying and processing complex multi-modal tasks [Multi-modal application example diagram].

清华大学与智谱AI重磅开源 GLM-4:掀起自然语言处理新革命Picture

Future Outlook

GLM-4-9B has demonstrated its powerful performance in a variety of tasks, It is a major breakthrough in the field of natural language processing. Whether it is academic research or industrial applications, the GLM-4-9B will be your best choice.

We sincerely invite you to join the ranks of GLM-4 users and explore the possibilities brought by this excellent model:

  • GitHub repository
  • Hugging Face model page
  • 魔达Community

The above is the detailed content of Tsinghua University and Zhipu AI open source GLM-4: launching a new revolution in natural language processing. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

WorldCoin (WLD) price forecast 2025-2031: Will WLD reach USD 4 by 2031? WorldCoin (WLD) price forecast 2025-2031: Will WLD reach USD 4 by 2031? Apr 21, 2025 pm 02:42 PM

WorldCoin (WLD) stands out in the cryptocurrency market with its unique biometric verification and privacy protection mechanisms, attracting the attention of many investors. WLD has performed outstandingly among altcoins with its innovative technologies, especially in combination with OpenAI artificial intelligence technology. But how will the digital assets behave in the next few years? Let's predict the future price of WLD together. The 2025 WLD price forecast is expected to achieve significant growth in WLD in 2025. Market analysis shows that the average WLD price may reach $1.31, with a maximum of $1.36. However, in a bear market, the price may fall to around $0.55. This growth expectation is mainly due to WorldCoin2.

What does cross-chain transaction mean? What are the cross-chain transactions? What does cross-chain transaction mean? What are the cross-chain transactions? Apr 21, 2025 pm 11:39 PM

Exchanges that support cross-chain transactions: 1. Binance, 2. Uniswap, 3. SushiSwap, 4. Curve Finance, 5. Thorchain, 6. 1inch Exchange, 7. DLN Trade, these platforms support multi-chain asset transactions through various technologies.

Why is the rise or fall of virtual currency prices? Why is the rise or fall of virtual currency prices? Why is the rise or fall of virtual currency prices? Why is the rise or fall of virtual currency prices? Apr 21, 2025 am 08:57 AM

Factors of rising virtual currency prices include: 1. Increased market demand, 2. Decreased supply, 3. Stimulated positive news, 4. Optimistic market sentiment, 5. Macroeconomic environment; Decline factors include: 1. Decreased market demand, 2. Increased supply, 3. Strike of negative news, 4. Pessimistic market sentiment, 5. Macroeconomic environment.

What are the hybrid blockchain trading platforms? What are the hybrid blockchain trading platforms? Apr 21, 2025 pm 11:36 PM

Suggestions for choosing a cryptocurrency exchange: 1. For liquidity requirements, priority is Binance, Gate.io or OKX, because of its order depth and strong volatility resistance. 2. Compliance and security, Coinbase, Kraken and Gemini have strict regulatory endorsement. 3. Innovative functions, KuCoin's soft staking and Bybit's derivative design are suitable for advanced users.

How to win KERNEL airdrop rewards on Binance Full process strategy How to win KERNEL airdrop rewards on Binance Full process strategy Apr 21, 2025 pm 01:03 PM

In the bustling world of cryptocurrencies, new opportunities always emerge. At present, KernelDAO (KERNEL) airdrop activity is attracting much attention and attracting the attention of many investors. So, what is the origin of this project? What benefits can BNB Holder get from it? Don't worry, the following will reveal it one by one for you.

Aavenomics is a recommendation to modify the AAVE protocol token and introduce token repurchase, which has reached the quorum number of people. Aavenomics is a recommendation to modify the AAVE protocol token and introduce token repurchase, which has reached the quorum number of people. Apr 21, 2025 pm 06:24 PM

Aavenomics is a proposal to modify the AAVE protocol token and introduce token repos, which has implemented a quorum for AAVEDAO. Marc Zeller, founder of the AAVE Project Chain (ACI), announced this on X, noting that it marks a new era for the agreement. Marc Zeller, founder of the AAVE Chain Initiative (ACI), announced on X that the Aavenomics proposal includes modifying the AAVE protocol token and introducing token repos, has achieved a quorum for AAVEDAO. According to Zeller, this marks a new era for the agreement. AaveDao members voted overwhelmingly to support the proposal, which was 100 per week on Wednesday

What is the analysis chart of Bitcoin finished product structure? How to draw? What is the analysis chart of Bitcoin finished product structure? How to draw? Apr 21, 2025 pm 07:42 PM

The steps to draw a Bitcoin structure analysis chart include: 1. Determine the purpose and audience of the drawing, 2. Select the right tool, 3. Design the framework and fill in the core components, 4. Refer to the existing template. Complete steps ensure that the chart is accurate and easy to understand.

The top ten free platform recommendations for real-time data on currency circle markets are released The top ten free platform recommendations for real-time data on currency circle markets are released Apr 22, 2025 am 08:12 AM

Cryptocurrency data platforms suitable for beginners include CoinMarketCap and non-small trumpet. 1. CoinMarketCap provides global real-time price, market value, and trading volume rankings for novice and basic analysis needs. 2. The non-small quotation provides a Chinese-friendly interface, suitable for Chinese users to quickly screen low-risk potential projects.

See all articles