MediaTek Dimensity 9300: Leading the industry, supporting the largest 33 billion parameter AI large language model

The latest version of MediaTek’s 5G generative AI mobile chip Dimensity 9300 has been released. Its novel full-core architecture design and new generation AI processor APU as well as MediaTek’s unique cutting-edge technology provide powerful capabilities for generative AI applications. performance support, allowing users to enjoy a colorful and rich generative AI experience. In addition, MediaTek has also strengthened cooperation with a large number of AI industry companies to create a rich AI ecosystem on the mobile side.

The new seventh-generation AI processor APU 790, born for generative AI

As users’ demand for generative artificial intelligence applications continues to grow, the convenience and security of end-side generative artificial intelligence have also emerged. Of course, to deploy a large-scale artificial intelligence language model on the client side, it requires the support of powerful artificial intelligence computing capabilities

Dimensity 9300 is equipped with MediaTek’s seventh-generation AI processor APU 790, which is designed for generative AI. It has a hardware-level generative AI engine, which can achieve faster and safer edge AI calculations, and is deeply adapted to the Transformer model. Operator acceleration is 8 times faster than the previous generation.

At the same time, the performance and energy efficiency of APU 790 have been significantly improved. The integer operation and floating point operation capabilities have been increased to 2 times that of the previous generation. Zurich ETHZv5.1 AI-Benchmark Mobile Soc scored 2109 points. The AI ​​performance successfully dominated the list. Consumption has been reduced by 45%. With the support of powerful AI performance, pictures can be generated in less than 1 second. Dimensity 9300's powerful AI computing power, innovative full-core CPU architecture and Immortalis-G720 GPU have laid a solid performance foundation for running generative AI on the device.

At the same time, based on the characteristics of large language models with billions of parameters, MediaTek has developed mixed-precision INT4 quantization technology, combined with MediaTek’s unique memory hardware compression technology NeuroPilot Compression, which can more efficiently utilize memory bandwidth and significantly reduce the terminal occupation of large AI models. Memory breaks through the memory limitations of mobile phones for running AI large language models on the device side, and helps larger parameter models to be implemented on the device side.

Based on the above, Dimensity 9300 is the first to launch a 7 billion parameter AI large language model on the vivo flagship mobile phone, with a processing speed of up to 20 Tokens per second. Not only that, MediaTek has broken through the limits of the industry and has successfully run a large language model with 13 billion parameters on the device side with vivo. Even Dimensity 9300 has taken the lead in successfully running an AI large language model with 33 billion parameters on a mobile chip, leading the industry.

Dimensity 9300 also supports multi-modal generative AI large models, creating rich and interesting end-side experiences such as "Wen Sheng Poems", "Wen Sheng Pictures" and "Wen Sheng Interesting Pictures".

It can be seen that Dimensity 9300’s AI computing power and end-side generative AI capabilities have led the industry, which is enough to allow users’ AI creativity to soar anytime and anywhere.

The end-side skills of the generative AI model have been expanded, bringing a comprehensive and rich end-side generative AI experience

Unlike cloud-based generative AI solutions, due to differences in hardware environments, deploying device-side generative AI also requires consideration of factors such as mobile phone memory, storage capacity, and load limit. Therefore, MediaTek took the lead in proposing advanced solutions

APU 790 supports the generative AI model end-side skill expansion technology NeuroPilot Fusion. This technology can continuously perform Low-Rank Adaptation (LoRA) fusion on the device side. Based on the basic large model, through cloud training, it can achieve the fusion of N functions, thereby giving the basic large model more comprehensive and richer generation. Type AI application capabilities

For example, through the "Tusheng GIF animation" function developed based on AI model end-side skill expansion technology, users can change different styles and expressions based on a photo to create a unique personalized emoticon package, which instantly becomes an emoticon Bao Daren

AI development platform NeuroPilot accelerates the end-side generative AI ecological layout

Dimensity 9300’s APU 790 uses powerful AI computing power and advanced memory hardware compression technology, as well as AI model end-side skill expansion and other technologies to elevate the speed and breadth of end-side generative AI to a whole new level. At the same time, MediaTek has built a rich AI ecosystem with its AI development platform NeuroPilot, from underlying hardware to tool chains, model centers and development ecosystems, helping the ecosystem quickly and efficiently deploy end-side generative AI applications and accelerate their deployment on the end-side. and popularity

NeuroPilot is an AI development platform that can support leading AI large models such as Android, Meta LIama 2, Baidu Wenxin Yiyan large model, and Baichuan Intelligent Baichuan large model

Another important advantage of NeuroPilot is its advanced tool chain, which includes NeuroPilot Compression low-rank adaptive fusion, Speculative Decoding speculative decoding acceleration, and model optimization and transformation technology, which are all very complete

MediaTek’s Dimensity Developer Center also provides one-stop developer resources for end-side generative AI implementation and shares end-side model deployment cases to improve development efficiency. At present, more than 20 generative AI partners have joined the ecological co-construction

MediaTek also works with industry contract partners to create wonderful generative AI application experiences. ArcSoft's generative AI super-resolution technology is based on the edge computing capabilities of Dimensity 9300 APU, which can improve performance by 30% compared to the previous generation. When shooting at 25x magnification, generative AI super-resolution technology can be used to capture images with more realistic details.

Jigan Technology’s generative AI semantic search technology is also based on the edge computing capabilities of Dimensity 9300 APU. Compared with the previous generation, the performance can be improved by 260%. For example, if you search for photos in the photo album of your mobile phone and describe the content of the photo, you can accurately find the corresponding photo within milliseconds. Moreover, you can search even when the internet is disconnected, and your privacy will not be leaked.

Morpho’s real-time digital avatar generation technology for video calls utilizes the edge computing capabilities of the Dimensity 9300 APU, improving performance by 26% compared to the previous generation. General virtual portrait generators require manual selection of appearance styles, which is time-consuming. However, based on the real-time digital avatar generation technology for video calls, users can operate easily. They only need to turn on the camera and take a frame of photos to instantly generate a digital avatar

Huili’s generative AI anti-glare technology can improve performance by 60% with the support of edge computing based on Dimensity 9300 APU. When using this technology, you only need to slightly dim the light to eliminate glare interference when shooting indoors or outdoors

It can be seen that under the trend of AI end-to-cloud integration, Dimensity 9300 has demonstrated comprehensive advantages in AI computing power, generative AI user experience and ecology, establishing a new generation of flagship end-side generative AI experience. To set a new benchmark, powerful generative AI must use Dimensity.

At the same time, generative AI pioneers led by MediaTek are vigorously promoting the development of hybrid AI computing through continuous technological innovation and ecological layout, launching a unique and efficient path for end-side generative AI deployment, and fully Popularize generative AI on the device side to enable more users to enjoy the personalized experience of device-side AI, create a new all-scenario intelligent experience, and fully benefit the public with the advantages of technology.

The above is the detailed content of MediaTek Dimensity 9300: Leading the industry, supporting the largest 33 billion parameter AI large language model. For more information, please follow other related articles on the PHP Chinese website!