Home Technology peripherals AI How to give AI a lot of physics knowledge? The EIT and Peking University teams proposed the concept of 'importance of rules'

How to give AI a lot of physics knowledge? The EIT and Peking University teams proposed the concept of 'importance of rules'

Mar 14, 2024 pm 04:49 PM
theory

How to give AI a lot of physics knowledge? The EIT and Peking University teams proposed the concept of importance of rules

Editor | ScienceAI

Deep learning models have had a profound impact in the field of scientific research due to their ability to learn latent relationships from massive amounts of data. However, models that rely purely on data gradually reveal their limitations, including over-reliance on data, limitations in generalization capabilities, and consistency issues with the real physical world. These issues drive researchers to explore more interpretable and explainable models to make up for the shortcomings of data-driven models. Therefore, combining domain knowledge and data-driven methods to build models with more interpretability and generalization capabilities has become an important direction in current scientific research. This

For example, the text-to-video model Sora developed by the American company OpenAI is highly praised for its excellent image generation capabilities and is considered an important progress in the field of artificial intelligence. Despite being able to generate realistic images and videos, Sora still has some challenges in dealing with the laws of physics, such as gravity and object fragmentation. While Sora has made significant progress in simulating real-life scenarios, there is still room for improvement in understanding and accurately simulating physical laws. The development of AI technology still requires continuous efforts to improve the comprehensiveness and accuracy of models to better adapt to various real-world situations.

One potential way to solve this problem is to incorporate human knowledge into deep learning models. By combining prior knowledge and data, the generalization ability of the model can be enhanced, resulting in an "informed machine learning" model that can understand physical laws. This approach is expected to improve the performance and accuracy of the model, making it better able to cope with complex problems in the real world. By integrating the experience and insights of human experts into machine learning algorithms, we can build more intelligent and efficient systems, thereby promoting the development and application of artificial intelligence technology.

Currently, there is still a lack of in-depth exploration of the exact value of knowledge in deep learning. There is an urgent problem in determining what prior knowledge can be effectively integrated into the model for "pre-learning". At the same time, blindly integrating multiple rules may lead to model failure, which also requires attention. These limitations bring challenges to the in-depth exploration of the relationship between data and knowledge.

In response to this problem, the research team from Eastern Institute of Technology (EIT) and Peking University proposed the concept of "rule importance" and developed a framework that can accurately calculate the impact of each rule on the prediction accuracy of the model. contribute. This framework not only reveals the complex interaction between data and knowledge and provides theoretical guidance for knowledge embedding, but also helps balance the influence of knowledge and data during the training process. In addition, this method can also be used to identify inappropriate a priori rules, providing broad prospects for research and applications in interdisciplinary fields.

This research, titled "Prior Knowledge's Impact on Deep Learning", was published on March 8, 2024 in the interdisciplinary journal "Nexus" under Cell Press. The research received attention from AAAS (American Association for the Advancement of Science) and EurekAlert!

How to give AI a lot of physics knowledge? The EIT and Peking University teams proposed the concept of importance of rules

When teaching children puzzles, you can either let them find out the answers through trial and error, or you can guide them with some basic rules and techniques. Likewise, incorporating rules and techniques—such as the laws of physics—into AI training can make them more realistic and more efficient. However, how to evaluate the value of these rules in artificial intelligence has always been a difficult problem for researchers.

Given the rich diversity of prior knowledge, integrating prior knowledge into deep learning models is a complex multi-objective optimization task. The research team innovatively proposes a framework to quantify the role of different prior knowledge in improving deep learning models. They view this process as a game full of cooperation and competition, and define the importance of rules by evaluating their marginal contribution to model predictions. First, all possible rule combinations (i.e., "coalitions") are generated, a model is built for each combination, and the mean square error is calculated.

In order to reduce the computational cost, they adopted an efficient algorithm based on perturbation: first train a completely data-based neural network as a baseline model, then add each rule combination one by one for additional training, and finally on the test data Evaluate model performance. By comparing the model's performance across all coalitions with and without a rule, the marginal contribution of that rule can be calculated, and thus its importance.

How to give AI a lot of physics knowledge? The EIT and Peking University teams proposed the concept of importance of rules

Illustration: Calculation process of rule importance (source: paper)

Through fluid mechanics examples, researchers explored the complex relationship between data and rules. They found that data and prior rules played completely different roles in different tasks. When the distribution of test data and training data is similar (ie, in-distribution), the increase in data volume will weaken the effect of the rules.

However, when the distribution similarity between the test data and the training data is low (ie Out-of-distribution), the importance of global rules is highlighted, while the influence of local rules is weakened. The difference between these two types of rules is that global rules (such as governing equations) affect the entire domain, while local rules (such as boundary conditions) only act on specific areas.

How to give AI a lot of physics knowledge? The EIT and Peking University teams proposed the concept of importance of rules

Illustration: The relationship between the importance of rules and the amount of data (source: paper)

The research team found through numerical experiments that in knowledge embedding , there are three types of interaction effects between rules: dependence effect, synergy effect and substitution effect.

Dependency effect means that some rules need to rely on other rules to be effective; synergy effect shows that the effect of multiple rules working together exceeds the sum of their independent effects; substitution effect shows that the function of a rule may be affected by data or other Rule substitution.

These three effects exist at the same time and are affected by the amount of data. By calculating rule importance, these effects can be clearly demonstrated, providing important guidance for knowledge embedding.

At the application level, the research team tried to solve a core issue in the knowledge embedding process: how to balance the role of data and rules to improve embedding efficiency and screen out inappropriate prior knowledge. During the training process of the model, the team proposed a strategy to dynamically adjust the weight of the rules.

Specifically, as the training iteration steps increase, the weight of positive importance rules gradually increases, while the weight of negative importance rules decreases. This strategy can adjust the model's attention to different rules in real time according to the needs of the optimization process, thereby achieving more efficient and accurate knowledge embedding.

In addition, teaching the laws of physics to AI models can make them “more relevant to the real world and thus play a greater role in science and engineering.” Therefore, this framework has a wide range of practical applications in engineering, physics, and chemistry. The researchers not only optimized the machine learning model to solve multivariate equations, but also accurately identified rules that improve the performance of the prediction model for thin-layer chromatography analysis.

The experimental results show that by incorporating these effective rules, the performance of the model is significantly improved, and the mean square error on the test data set is reduced from 0.052 to 0.036 (a reduction of 30.8%). This means that the framework can transform empirical insights into structured knowledge, thereby significantly improving model performance.

Overall, accurately assessing the value of knowledge helps build AI models that are more realistic and improve safety and reliability, which is of great significance to the development of deep learning.

How to give AI a lot of physics knowledge? The EIT and Peking University teams proposed the concept of importance of rules

Illustration: Rule Importance to Identify Effective Rules (Source: Paper)

Next, the research team plans to develop their framework into Plug-in tools for artificial intelligence developers. Their ultimate goal is to develop models that can extract knowledge and rules directly from data and then improve themselves, thereby creating a closed-loop system from knowledge discovery to knowledge embedding, making the model a true artificial intelligence scientist.

Paper link: https://www.cell.com/nexus/fulltext/S2950-1601(24)00001-9

AAAS Report link: https://www.eurekalert. org/news-releases/1036117

The above is the detailed content of How to give AI a lot of physics knowledge? The EIT and Peking University teams proposed the concept of 'importance of rules'. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Breaking through the boundaries of traditional defect detection, 'Defect Spectrum' achieves ultra-high-precision and rich semantic industrial defect detection for the first time. Breaking through the boundaries of traditional defect detection, 'Defect Spectrum' achieves ultra-high-precision and rich semantic industrial defect detection for the first time. Jul 26, 2024 pm 05:38 PM

In modern manufacturing, accurate defect detection is not only the key to ensuring product quality, but also the core of improving production efficiency. However, existing defect detection datasets often lack the accuracy and semantic richness required for practical applications, resulting in models unable to identify specific defect categories or locations. In order to solve this problem, a top research team composed of Hong Kong University of Science and Technology Guangzhou and Simou Technology innovatively developed the "DefectSpectrum" data set, which provides detailed and semantically rich large-scale annotation of industrial defects. As shown in Table 1, compared with other industrial data sets, the "DefectSpectrum" data set provides the most defect annotations (5438 defect samples) and the most detailed defect classification (125 defect categories

NVIDIA dialogue model ChatQA has evolved to version 2.0, with the context length mentioned at 128K NVIDIA dialogue model ChatQA has evolved to version 2.0, with the context length mentioned at 128K Jul 26, 2024 am 08:40 AM

The open LLM community is an era when a hundred flowers bloom and compete. You can see Llama-3-70B-Instruct, QWen2-72B-Instruct, Nemotron-4-340B-Instruct, Mixtral-8x22BInstruct-v0.1 and many other excellent performers. Model. However, compared with proprietary large models represented by GPT-4-Turbo, open models still have significant gaps in many fields. In addition to general models, some open models that specialize in key areas have been developed, such as DeepSeek-Coder-V2 for programming and mathematics, and InternVL for visual-language tasks.

Training with millions of crystal data to solve the crystallographic phase problem, the deep learning method PhAI is published in Science Training with millions of crystal data to solve the crystallographic phase problem, the deep learning method PhAI is published in Science Aug 08, 2024 pm 09:22 PM

Editor |KX To this day, the structural detail and precision determined by crystallography, from simple metals to large membrane proteins, are unmatched by any other method. However, the biggest challenge, the so-called phase problem, remains retrieving phase information from experimentally determined amplitudes. Researchers at the University of Copenhagen in Denmark have developed a deep learning method called PhAI to solve crystal phase problems. A deep learning neural network trained using millions of artificial crystal structures and their corresponding synthetic diffraction data can generate accurate electron density maps. The study shows that this deep learning-based ab initio structural solution method can solve the phase problem at a resolution of only 2 Angstroms, which is equivalent to only 10% to 20% of the data available at atomic resolution, while traditional ab initio Calculation

Google AI won the IMO Mathematical Olympiad silver medal, the mathematical reasoning model AlphaProof was launched, and reinforcement learning is so back Google AI won the IMO Mathematical Olympiad silver medal, the mathematical reasoning model AlphaProof was launched, and reinforcement learning is so back Jul 26, 2024 pm 02:40 PM

For AI, Mathematical Olympiad is no longer a problem. On Thursday, Google DeepMind's artificial intelligence completed a feat: using AI to solve the real question of this year's International Mathematical Olympiad IMO, and it was just one step away from winning the gold medal. The IMO competition that just ended last week had six questions involving algebra, combinatorics, geometry and number theory. The hybrid AI system proposed by Google got four questions right and scored 28 points, reaching the silver medal level. Earlier this month, UCLA tenured professor Terence Tao had just promoted the AI ​​Mathematical Olympiad (AIMO Progress Award) with a million-dollar prize. Unexpectedly, the level of AI problem solving had improved to this level before July. Do the questions simultaneously on IMO. The most difficult thing to do correctly is IMO, which has the longest history, the largest scale, and the most negative

Nature's point of view: The testing of artificial intelligence in medicine is in chaos. What should be done? Nature's point of view: The testing of artificial intelligence in medicine is in chaos. What should be done? Aug 22, 2024 pm 04:37 PM

Editor | ScienceAI Based on limited clinical data, hundreds of medical algorithms have been approved. Scientists are debating who should test the tools and how best to do so. Devin Singh witnessed a pediatric patient in the emergency room suffer cardiac arrest while waiting for treatment for a long time, which prompted him to explore the application of AI to shorten wait times. Using triage data from SickKids emergency rooms, Singh and colleagues built a series of AI models that provide potential diagnoses and recommend tests. One study showed that these models can speed up doctor visits by 22.3%, speeding up the processing of results by nearly 3 hours per patient requiring a medical test. However, the success of artificial intelligence algorithms in research only verifies this

To provide a new scientific and complex question answering benchmark and evaluation system for large models, UNSW, Argonne, University of Chicago and other institutions jointly launched the SciQAG framework To provide a new scientific and complex question answering benchmark and evaluation system for large models, UNSW, Argonne, University of Chicago and other institutions jointly launched the SciQAG framework Jul 25, 2024 am 06:42 AM

Editor |ScienceAI Question Answering (QA) data set plays a vital role in promoting natural language processing (NLP) research. High-quality QA data sets can not only be used to fine-tune models, but also effectively evaluate the capabilities of large language models (LLM), especially the ability to understand and reason about scientific knowledge. Although there are currently many scientific QA data sets covering medicine, chemistry, biology and other fields, these data sets still have some shortcomings. First, the data form is relatively simple, most of which are multiple-choice questions. They are easy to evaluate, but limit the model's answer selection range and cannot fully test the model's ability to answer scientific questions. In contrast, open-ended Q&A

PRO | Why are large models based on MoE more worthy of attention? PRO | Why are large models based on MoE more worthy of attention? Aug 07, 2024 pm 07:08 PM

In 2023, almost every field of AI is evolving at an unprecedented speed. At the same time, AI is constantly pushing the technological boundaries of key tracks such as embodied intelligence and autonomous driving. Under the multi-modal trend, will the situation of Transformer as the mainstream architecture of AI large models be shaken? Why has exploring large models based on MoE (Mixed of Experts) architecture become a new trend in the industry? Can Large Vision Models (LVM) become a new breakthrough in general vision? ...From the 2023 PRO member newsletter of this site released in the past six months, we have selected 10 special interpretations that provide in-depth analysis of technological trends and industrial changes in the above fields to help you achieve your goals in the new year. be prepared. This interpretation comes from Week50 2023

The accuracy rate reaches 60.8%. Zhejiang University's chemical retrosynthesis prediction model based on Transformer was published in the Nature sub-journal The accuracy rate reaches 60.8%. Zhejiang University's chemical retrosynthesis prediction model based on Transformer was published in the Nature sub-journal Aug 06, 2024 pm 07:34 PM

Editor | KX Retrosynthesis is a critical task in drug discovery and organic synthesis, and AI is increasingly used to speed up the process. Existing AI methods have unsatisfactory performance and limited diversity. In practice, chemical reactions often cause local molecular changes, with considerable overlap between reactants and products. Inspired by this, Hou Tingjun's team at Zhejiang University proposed to redefine single-step retrosynthetic prediction as a molecular string editing task, iteratively refining the target molecular string to generate precursor compounds. And an editing-based retrosynthetic model EditRetro is proposed, which can achieve high-quality and diverse predictions. Extensive experiments show that the model achieves excellent performance on the standard benchmark data set USPTO-50 K, with a top-1 accuracy of 60.8%.

See all articles