Baidu CTO Wang Haifeng: The number of Flying Paddle developers has reached 8 million, and large language models bring the dawn of general artificial intelligence

Beijing will host the 2023 WAVE SUMMIT Deep Learning Developer Conference hosted by the National Engineering Research Center for Deep Learning Technology and Applications

Wang Haifeng, chief technology officer of Baidu and director of the National Engineering Research Center for Deep Learning Technology and Applications, said for the first time in his keynote speech that large-scale language models possess the core basic capabilities of artificial intelligence, including understanding, generation, logic and memory, and provide General artificial intelligence brings new hope

8 million developers are using Flying Paddle and more than 800,000 models have been created

In April 2019, the WAVE SUMMIT Deep Learning Developer Conference was held for the first time. At the conference, Wang Haifeng emphasized the versatility of deep learning and its features of standardization, automation and modularity for industrial mass production, which promoted artificial intelligence into the stage of industrial mass production. After four years of development, the progress of deep learning technology and applications has fully verified this point of view. Deep learning technology is becoming more and more versatile, and the standardization, automation, and modular features of deep learning platforms are becoming more and more obvious. At the same time, the rise of pre-trained large models has further expanded the depth and breadth of artificial intelligence applications. Therefore, it can be said that artificial intelligence has entered the stage of industrial mass production

In terms of standardization, we jointly optimize frameworks and models and uniformly adapt to a variety of hardware to make the application model more concise and efficient, thereby significantly lowering the threshold for artificial intelligence applications. In terms of automation, we provide full-process artificial intelligence R&D solutions, including training, adaptation and inference deployment to improve efficiency. In terms of modularity, we provide a rich industrial-level model library to support the convenient application of artificial intelligence in various scenarios

Fei Paddle industrial-level deep learning open source open platform and Wenxin large model promote each other, making Fei Paddle ecology flourish, attracting 8 million developers, providing services to 220,000 enterprises and institutions, and creating 80 products based on Fei Paddle Thousands of models. Wang Haifeng explained the profound meaning of the Chinese name of the Flying Paddle developer community AI Studio "Galaxy Community": "Wenxin and Flying Paddle blend together and sail to the galaxy together." With the joint promotion of Feipiao and Wenxin, we will work with all developers to build the Galaxy community and explore the infinite possibilities of general artificial intelligence

Large-scale language models bring new hope for general artificial intelligence

Wang Haifeng believes that artificial intelligence has a variety of typical abilities, among which the core basic abilities include understanding, generation, logic and memory. The stronger these four abilities are, the closer the artificial intelligence is to the level of general artificial intelligence. The large language model has these four capabilities, bringing hope to the development of general artificial intelligence

Specifically, the typical capabilities of artificial intelligence, such as creation, programming, problem solving, planning, etc., are based on core basic capabilities, such as understanding, generation, logic, memory, etc., although the degree of dependence varies to different degrees. . Taking problem solving as an example, from understanding the question, solving the question to finally writing the answer, you need to comprehensively use understanding, memory, logic and generative abilities

How to obtain these abilities? Taking Wen Xinyiyan as an example, we first train a pre-trained large model by fusion learning trillions of data and hundreds of billions of knowledge. We then use techniques such as supervised fine-tuning, reinforcement learning with human feedback, and prompts to further improve model performance. In addition, we also have technical advantages such as knowledge enhancement, retrieval enhancement and dialogue enhancement

Optimize data sources and data distribution through multiple strategies, build basic models for long-text modeling, perform multi-type and multi-stage supervised fine-tuning and multi-task adaptive supervised fine-tuning, and implement multi-level and multi-granularity reward models and other technologies Innovate and comprehensively improve basic general capabilities. In terms of enhanced retrieval and knowledge, the ability to master and apply world knowledge is improved through knowledge point enhancement; logical capabilities are improved through large-scale logical data construction, logical knowledge modeling, multi-granular semantic knowledge combination, and symbolic neural networks; through the construction of comprehensive Secure data, content, model and system security system to ensure the security of large models

Through flying paddle end-to-end adaptive hybrid parallel training technology and collaborative optimization of compression, inference, and service deployment, the training speed of the Wenxin large model has been increased by 3 times, and the inference speed has been increased by more than 30 times, thereby improving efficiency.

Through data-driven, prompt construction and plug-in enhanced applications, we have launched five plug-ins, including Wen Xin Yi Yan, Baidu Search, Browsing Documents, E Yan Yi Tu, Shuo Tu Jie Hua and Yijing Liuying . These plug-ins enable our model to generate real-time accurate information, long text summaries and Q&A, data insights and chart production, image-based creation and Q&A, and Vincent videos. Through the plug-in mechanism, we expand the capability boundaries of large models to better adapt to the needs of various scenarios. Wang Haifeng said that in the future we will work with developers to build a plug-in ecosystem and share technological innovation results

Artificial intelligence represented by large language models is penetrating into thousands of industries, accelerating industrial upgrading and economic growth. In this process, technological innovation and application implementation form a virtuous cycle. Capabilities such as understanding, generation, logic, and memory continue to improve. The breadth and depth of industrial applications continue to expand. Large-scale language models bring new hope for general artificial intelligence.

The above is the detailed content of Baidu CTO Wang Haifeng: The number of Flying Paddle developers has reached 8 million, and large language models bring the dawn of general artificial intelligence. For more information, please follow other related articles on the PHP Chinese website!