Alibaba Cloud launches PilotScope technology to accelerate AI4DB application innovation

On December 20, VLDB2024, the top international database conference, announced a set of new papers, and Alibaba Cloud’s new technology PilotScope was successfully selected. This platform technology can realize "one-click deployment" of AI algorithms in databases, greatly reducing the application threshold of AI algorithms in databases and opening up a new space for database intelligence. On the same day, Alibaba Cloud announced that all PilotScope technologies will be open sourced for free. Basic software technology for people’s lives. The update of database technology has a major impact on all walks of life in the digital era. One of the frontier areas is database intelligence (AI4DB, AI for Database). The current database system is very complex and has extremely high stability requirements. Even matching and debugging a single AI algorithm with a single database requires engineers from both parties to work closely together for weeks or even months, which is inefficient and ineffective. This has resulted in the industry's delay in applying rapidly developing AI algorithms to databases

Please see: Illustration of the Alibaba Cloud PilotScope architecture diagram

For this To solve this problem, Alibaba Cloud proposed a new solution: by abstracting and generalizing module and interface definitions at the level of database and artificial intelligence systems, it developed a new middleware system platform PilotScope, which enables artificial intelligence algorithms to Achieve "one-click deployment" in the database in hours or even minutes. The VLDB review believes that PilotScope's innovative system design based on application scenarios will open up a new direction of database intelligence

It is understood that PilotScope is targeted at mainstream database tasks such as parameter tuning, index recommendation, cardinality estimation, and query optimization. It has developed more than 10 AI algorithms and completed adaptation proofing of two mainstream open source databases such as PostgreSQL and Spark. Experimental data shows that using PilotScope to embed AI algorithms into the database can speed up tasks such as query optimization by 1 to 2 times compared with the traditional "hard implant" method, and the additional cost of deployment caused by PilotScope itself is basically negligible, and the performance is outstanding.

Rewritten content: PilotScope rendering analysis

"PilotScope is a database AI 'super administrator'. Through this platform, AI engineers only need to focus on designing general algorithms. It can realize the deployment and application of different databases; and database users can use AI as conveniently and efficiently as calling APIs." Zhu Rong, the person in charge of the project, said that PilotScope has a 'zero intrusion' design for the database, and also designed Mechanisms such as intelligent detection, rollback, and isolation have been implemented to reduce the risk of AI hallucinations and achieve intelligent improvement while ensuring database stability

Under the current circumstances, PilotScope has begun to be implemented within Alibaba Cloud Pilot applications, related technologies are also open source for free through GitHub and Modelscope communities

VLDB is one of the three top international conferences on databases. Every year, it only includes new results that have had an important impact in academia and industrial practice. It is the authoritative weather vane of database technology. The 50th VLDB Conference is expected to be held in Guangzhou, China in August 2024

Attached is the open source address:

https://github.com/alibaba/pilotscope

The above is the detailed content of Alibaba Cloud launches PilotScope technology to accelerate AI4DB application innovation. For more information, please follow other related articles on the PHP Chinese website!