Baidu CTO Wang Haifeng: Large language models bring the dawn of general artificial intelligence

On August 16, 2023, Beijing time, the WAVE SUMMIT Deep Learning Developer Conference was hosted by the National Engineering Research Center for Deep Learning Technology and Applications. At this meeting, Baidu Chief Technology Officer and National Engineering Research Center for Deep Learning Technology and Applications Center Director Wang Haifeng delivered a keynote speech. Wang Haifeng publicly stated for the first time that large language models already possess the core basic capabilities of artificial intelligence such as understanding, generation, logic, and memory, bringing new hope to the development of general artificial intelligence

8 million developers have used Flying Paddle and created more than 800,000 models

WAVE SUMMIT Deep Learning Developer Conference has been held since April 2019. At the first conference, Wang Haifeng pointed out that deep learning has broad applicability and has the characteristics of standardization, automation and modularization of industrial production, which has promoted artificial intelligence to enter the industrialization stage. After four years of development, the progress of deep learning technology and applications has fully verified this point of view. The scope of application of deep learning technology is getting wider and wider, the standardization, automation and modularity features of deep learning platforms are becoming more and more obvious, and the rise of pre-trained large models has further expanded the depth and breadth of artificial intelligence applications. Therefore, artificial intelligence has entered the stage of industrial production

In terms of standardization, we have jointly optimized frameworks and models to uniformly adapt to a variety of hardware and simplify application models, thereby greatly lowering the threshold for artificial intelligence applications; in terms of automation, we have improved the efficiency of the entire artificial intelligence research and development process. Everything from training, adaptation to inference deployment has been automated; in terms of modularity, we provide a rich industrial-level model library to facilitate the rapid application of artificial intelligence in various scenarios

Fei Paddle industrial-level deep learning open source open platform and Wenxin large model promote each other, making Fei Paddle ecology more prosperous, attracting 8 million developers, providing services to 220,000 enterprises and institutions, and creating 800,000 based on Model of a flying oar. Wang Haifeng explained the profound meaning of the Chinese name of the Flying Paddle developer community AI Studio "Galaxy Community", "Wenxin and Flying Paddle combine to enter the galaxy together." Under the guidance of Fei Piao and Wen Xin, we work with all developers to build the Galaxy community and jointly explore the endless possibilities of general artificial intelligence

Large-scale language models bring new hope for general artificial intelligence

Wang Haifeng believes that the core basic capabilities of general artificial intelligence include understanding, generation, logic and memory, and the large language model has these four capabilities, bringing hope to the realization of general artificial intelligence

Specifically, the typical abilities of artificial intelligence, such as creation, programming, problem solving and planning, are based on core basic abilities, including understanding, generation, logic and memory, although they may differ in the degree of dependence. For example, the problem-solving process requires the comprehensive use of understanding, memory, logic and generative abilities, from reading the question, solving the question to finally writing the answer

How to obtain these abilities? Taking Wen Xinyiyan as an example, we first train a large pre-trained model through fusion learning of trillions of data and hundreds of billions of knowledge. It is then further optimized using techniques such as supervised fine-tuning, reinforcement learning with human feedback, and prompts. In addition, the model also has technical advantages such as knowledge enhancement, retrieval enhancement and dialogue enhancement

Optimize data sources and data distribution through multiple strategies, long-text modeling of basic models, multi-type and multi-stage supervised fine-tuning, multi-task adaptive supervised fine-tuning, and multi-level and multi-granularity reward models and other technologies Innovate and comprehensively improve basic general capabilities. On the basis of retrieval enhancement and knowledge enhancement, the ability to master and apply world knowledge is improved through knowledge point enhancement; logical capabilities are improved by building large-scale logical data, logical knowledge modeling, multi-granular semantic knowledge combination and symbolic neural network; Ensure the security of large models by building a comprehensive security system that includes data, content, model and system security

Through Fei Paddle’s end-to-end adaptive hybrid parallel training technology and collaboratively optimized compression, inference and service deployment, the training speed of the Wenxin large model has been increased by 3 times, and the inference speed has been increased by more than 30 times

Through data-driven, prompt construction and plug-in enhancement, we have carried out scene adaptation and collaborative optimization in applications. We have launched five plug-ins: Wen Xin Yi Yan, Baidu Search, Browsing Documents, E Yan Yi Tu, Shuo Tu Jie Hua and Yijing Liuying. These plug-ins enable our models to generate real-time accurate information, long text summaries and Q&A, data insights and chart production, image-based creation and Q&A, and Vincent videos. The introduction of the plug-in mechanism expands the functional boundaries of large models and better meets the needs of different scenarios. Wang Haifeng said that in the future, Baidu will work with developers to build a plug-in ecosystem and share technological innovation results

Artificial intelligence represented by large language models is penetrating into thousands of industries, accelerating industrial upgrading and economic growth. In this process, technological innovation and application implementation form a virtuous cycle. Capabilities such as understanding, generation, logic, and memory continue to improve. The breadth and depth of industrial applications continue to expand. Large-scale language models bring new hope for general artificial intelligence.

The above is the detailed content of Baidu CTO Wang Haifeng: Large language models bring the dawn of general artificial intelligence. For more information, please follow other related articles on the PHP Chinese website!