Table of Contents
Principles and Methods
Summary and Outlook
Home Technology peripherals AI Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

Apr 09, 2023 pm 11:11 PM
AI Research

In recent years, CNN has been favored by researchers in various fields such as computer vision and natural language processing due to its excellent performance. However, CNN is a "black box" model, that is, the learning content and decision-making process of the model are difficult to extract and express in a way that humans can understand, which limits its prediction credibility and practical application. Therefore, the interpretability of CNN has received more and more attention. Researchers have tried to use feature visualization, network diagnosis and network architecture adjustment to assist in explaining the learning mechanism of CNN, thereby making this "black box" transparent. Make it easier for humans to understand, detect and improve their decision-making processes.

Recently, Research teams from Peking University, Eastern Institute of Technology, Southern University of Science and Technology, Pengcheng Laboratory and other institutions have proposed a semantic interpretable artificial intelligence (semantic The research framework of explainable AI (S-XAI) explains the learning mechanism of CNN from the semantic level, and takes the cat and dog binary classification problem as an example to vividly reveal how the model learns cats in the category sense. The concept of "what is a cat".

This research focuses on the common features learned by CNN from samples of the same category, and extracts human-understandable semantic concepts, providing semantics for CNN level of explanation. Based on this, the study first proposed the concept of "semantic probability" to characterize the occurrence probability of semantic elements in the sample. Experiments show that S-XAI can successfully extract common features and abstract ultra-realistic but identifiable semantic concepts in both binary and multi-classification tasks. It has broad application prospects in credibility assessment and semantic sample search.

The study was titled "Semantic interpretation for convolutional neural networks: What makes a cat a cat?" and was published in "Advanced Science" on October 10, 2022.

Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

##Paper link: https://onlinelibrary.wiley.com/doi/10.1002/advs.202204723

Code link: https://github.com/woshixuhao/semantic-explainable-AI

Model effect

Different from previous single sample visualization research, S-XAI can extract and visualize the common features of group samples, thereby Get global interpretability. Based on the further abstracted semantic space and calculated semantic probabilities, S-XAI can automatically generate human-understandable semantic explanations for the decision logic of the CNN and evaluate the credibility of the decision from a semantic level.

As shown in Figure 1, in the cat and dog classification problem, for pictures of the same cat from three angles, S-XAI automatically generates the corresponding semantic probability radar map and Explain the statement. Although the neural networks all identified these pictures as cats with a probability of more than 90%, S-XAI provided more interpretation information from the semantic probability, reflecting the differences between these pictures. For example, for the frontal image, S-XAI's explanation is "I am convinced that it is a cat, mainly because it has vivid eyes and nose, which are obviously cat's eyes and nose. At the same time, it has lifelike legs, which is a bit Like a cat's legs." This explanation shows a high degree of credibility. For the image from the side angle, S-XAI's explanation is "It's probably a cat, mainly because it has eyes, maybe cat eyes, but its legs are a little confusing." For the image from the back of the cat, None of the semantic probabilities are obvious, and S-XAI's interpretation is "It might be a cat, but I'm not sure." Meanwhile, for a picture of a dog, S-XAI's interpretation is: "I'm sure it's a cat." A dog, mainly because it has vivid eyes and a nose that are clearly those of a dog. Although its legs are a bit confusing."

In fact, if The dog's upper body is covered and only the legs are visible, making it difficult even for humans to tell whether it is a cat or a dog. It can be seen that the semantic explanation provided by S-XAI is more accurate and consistent with human cognition, allowing humans to better understand the category recognition logic of neural networks from the semantic level.

Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

Figure 1. Semantic probability radar chart and explanation statements automatically generated by S-XAI

At the same time, S-XAI also has broad application prospects in semantic sample search. As shown in Figure 2, when people need to filter out pictures with certain semantic features from a large number of pictures, S-XAI provides a fast and accurate way to filter through semantic probability. Considering that calculating semantic probabilities only involves the forward operation (i.e., prediction) of the neural network, the process is very fast.

Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

Figure 2. Semantic sample search example

In the study, the researchers also proved that S-XAI has good scalability in multi-classification tasks. As shown in Figure 3, taking the Mini-ImageNet data set (containing 100 animal categories) as an example, S-XAI can still extract clearly identifiable images from different categories of data (such as birds, snakes, crabs, fish, etc.) Common features and semantic space, and generate corresponding semantic explanations.

Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

Figure 3. Performance of S-XAI in multi-classification tasks.

Principles and Methods

Currently, common ideas for improving model interpretability are mainly divided into two categories: visualization and model intervention. Visualization methods visualize the feature maps, filters or heat maps inside the CNN to understand the features that the network pays attention to when facing a given sample. The limitation of this method is that it can only extract individual features from a single sample to obtain local interpretability, and cannot help people understand the overall decision-making logic of the model when facing the same type of data. The model intervention method integrates some existing highly interpretable models (such as tree models, etc.) into the architecture of the neural network to improve the interpretability of the model. Although this type of method has the advantage of global interpretability, it often requires retraining the model, resulting in high interpretation costs, which is not conducive to generalization and application.

Inspired by human cognitive models, in S-XAI, researchers adopted a new explanation strategy to explain CNN from the semantic level Category learning mechanism (Figure 4). In nature, objects of the same type often have certain similar common characteristics, which form an important basis for category cognition. For example, although cats have different shapes, they all share some common features (such as whiskers, noses, and eye-related features), which allows humans to quickly identify them as cats. In experiments, researchers found that CNN’s category learning mechanism is similar to humans.

Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

Figure 4. Semantic Interpretable Artificial Intelligence Research Framework

A technology called

row-centered sample compression was used in the study to extract samples from the same category from CNN learned common features. Different from traditional principal component analysis, row center sample compression reduces the dimensionality of the feature maps obtained by a large number of samples in the CNN in the sample space, thereby extracting a small number of principal components as common features learned by the CNN. In order to make the extracted common features clearer, the sample found the optimal superpixel combination through superpixel segmentation and genetic algorithm to reduce interference. The extracted common features are displayed visually (Figure 5).

Figure 5. Extraction path of common features

Taking the cat and dog classification problem on the VGG-19 network architecture as an example, for cats The different principal components extracted from the category data of dog and dog are shown in Figure 6. It can be clearly seen from the figure that different principal components exhibit identifiable features at different levels. It is obvious that the first principal component shows complete facial features, the second principal component shows scattered semantic concepts, such as beard, eyes and nose, etc., and the third principal component mainly shows the characteristics of fur. It is worth mentioning that the characteristics exhibited by these principal components are supernatural, that is, they do not belong to any sample, but reflect the common characteristics of all samples of the same category.

Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

Figure 6. Visualization results of different principal components extracted from cat and dog category data

Based on the extracted common features, the researchers masked the semantic information in the samples and compared the changes in the principal components to further separate the mixed semantic concepts. to extract the semantic vector corresponding to each semantic concept and abstract the semantic space. Here, the researchers used human-understood semantic concepts such as eyes, nose, etc., and visualized the abstracted semantic space. After successfully extracting the semantic space, the researchers defined the concept of "semantic probability" to characterize the occurrence probability of semantic elements in the sample, thus providing a quantitative analysis method for the semantic level explanation of CNN.

As shown in Figure 7, clearly identifiable semantic concepts (bright eyes, small nose) appear in the semantic space, which indicates that the semantic space is successfully extracted from the CNN comes out, showing the semantic information learned by CNN from categorical data. At the same time, researchers have found that CNN’s understanding of semantics is somewhat different from that of humans. The “semantics” it learns are not necessarily the “semantics” agreed upon by humans. It may even be that the semantics of neural networks are more efficient. For example, researchers found that for cats, CNN often treats the cat's nose and whiskers as a whole semantics, which may be more effective. At the same time, CNN has learned some connections between semantics. For example, a cat’s eyes and nose often appear at the same time. This aspect deserves further in-depth research.

Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level

Figure 7. Semantic vectors extracted from CNN and visualized semantic space (top: cat eye space; bottom: cat Nose space)

Summary and Outlook

In summary, the semantically explainable artificial intelligence (S-XAI) proposed in the study extracts common features and Semantic space provides an explanation for the category recognition mechanism of CNN from a semantic level. This research framework can obtain certain global explanation capabilities without changing the CNN architecture. Since it does not involve network retraining, S-XAI has the advantage of faster response speed and has considerable applications in credibility assessment and semantic sample search. potential.

In essence, S-XAI is similar to knowledge discovery. Knowledge discovery aims to find function terms that reflect common physical laws from neural networks, while S-XAI aims to find semantic spaces that reflect common characteristics of samples from CNN. The core idea of ​​both is to find commonalities and represent them, so as to Possible to make human understandable.

The above is the detailed content of Why a cat? Explainable AI understands the recognition mechanism of CNN from a semantic level. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Bytedance Cutting launches SVIP super membership: 499 yuan for continuous annual subscription, providing a variety of AI functions Bytedance Cutting launches SVIP super membership: 499 yuan for continuous annual subscription, providing a variety of AI functions Jun 28, 2024 am 03:51 AM

This site reported on June 27 that Jianying is a video editing software developed by FaceMeng Technology, a subsidiary of ByteDance. It relies on the Douyin platform and basically produces short video content for users of the platform. It is compatible with iOS, Android, and Windows. , MacOS and other operating systems. Jianying officially announced the upgrade of its membership system and launched a new SVIP, which includes a variety of AI black technologies, such as intelligent translation, intelligent highlighting, intelligent packaging, digital human synthesis, etc. In terms of price, the monthly fee for clipping SVIP is 79 yuan, the annual fee is 599 yuan (note on this site: equivalent to 49.9 yuan per month), the continuous monthly subscription is 59 yuan per month, and the continuous annual subscription is 499 yuan per year (equivalent to 41.6 yuan per month) . In addition, the cut official also stated that in order to improve the user experience, those who have subscribed to the original VIP

Context-augmented AI coding assistant using Rag and Sem-Rag Context-augmented AI coding assistant using Rag and Sem-Rag Jun 10, 2024 am 11:08 AM

Improve developer productivity, efficiency, and accuracy by incorporating retrieval-enhanced generation and semantic memory into AI coding assistants. Translated from EnhancingAICodingAssistantswithContextUsingRAGandSEM-RAG, author JanakiramMSV. While basic AI programming assistants are naturally helpful, they often fail to provide the most relevant and correct code suggestions because they rely on a general understanding of the software language and the most common patterns of writing software. The code generated by these coding assistants is suitable for solving the problems they are responsible for solving, but often does not conform to the coding standards, conventions and styles of the individual teams. This often results in suggestions that need to be modified or refined in order for the code to be accepted into the application

Can fine-tuning really allow LLM to learn new things: introducing new knowledge may make the model produce more hallucinations Can fine-tuning really allow LLM to learn new things: introducing new knowledge may make the model produce more hallucinations Jun 11, 2024 pm 03:57 PM

Large Language Models (LLMs) are trained on huge text databases, where they acquire large amounts of real-world knowledge. This knowledge is embedded into their parameters and can then be used when needed. The knowledge of these models is "reified" at the end of training. At the end of pre-training, the model actually stops learning. Align or fine-tune the model to learn how to leverage this knowledge and respond more naturally to user questions. But sometimes model knowledge is not enough, and although the model can access external content through RAG, it is considered beneficial to adapt the model to new domains through fine-tuning. This fine-tuning is performed using input from human annotators or other LLM creations, where the model encounters additional real-world knowledge and integrates it

Seven Cool GenAI & LLM Technical Interview Questions Seven Cool GenAI & LLM Technical Interview Questions Jun 07, 2024 am 10:06 AM

To learn more about AIGC, please visit: 51CTOAI.x Community https://www.51cto.com/aigc/Translator|Jingyan Reviewer|Chonglou is different from the traditional question bank that can be seen everywhere on the Internet. These questions It requires thinking outside the box. Large Language Models (LLMs) are increasingly important in the fields of data science, generative artificial intelligence (GenAI), and artificial intelligence. These complex algorithms enhance human skills and drive efficiency and innovation in many industries, becoming the key for companies to remain competitive. LLM has a wide range of applications. It can be used in fields such as natural language processing, text generation, speech recognition and recommendation systems. By learning from large amounts of data, LLM is able to generate text

Five schools of machine learning you don't know about Five schools of machine learning you don't know about Jun 05, 2024 pm 08:51 PM

Machine learning is an important branch of artificial intelligence that gives computers the ability to learn from data and improve their capabilities without being explicitly programmed. Machine learning has a wide range of applications in various fields, from image recognition and natural language processing to recommendation systems and fraud detection, and it is changing the way we live. There are many different methods and theories in the field of machine learning, among which the five most influential methods are called the "Five Schools of Machine Learning". The five major schools are the symbolic school, the connectionist school, the evolutionary school, the Bayesian school and the analogy school. 1. Symbolism, also known as symbolism, emphasizes the use of symbols for logical reasoning and expression of knowledge. This school of thought believes that learning is a process of reverse deduction, through existing

To provide a new scientific and complex question answering benchmark and evaluation system for large models, UNSW, Argonne, University of Chicago and other institutions jointly launched the SciQAG framework To provide a new scientific and complex question answering benchmark and evaluation system for large models, UNSW, Argonne, University of Chicago and other institutions jointly launched the SciQAG framework Jul 25, 2024 am 06:42 AM

Editor |ScienceAI Question Answering (QA) data set plays a vital role in promoting natural language processing (NLP) research. High-quality QA data sets can not only be used to fine-tune models, but also effectively evaluate the capabilities of large language models (LLM), especially the ability to understand and reason about scientific knowledge. Although there are currently many scientific QA data sets covering medicine, chemistry, biology and other fields, these data sets still have some shortcomings. First, the data form is relatively simple, most of which are multiple-choice questions. They are easy to evaluate, but limit the model's answer selection range and cannot fully test the model's ability to answer scientific questions. In contrast, open-ended Q&A

SOTA performance, Xiamen multi-modal protein-ligand affinity prediction AI method, combines molecular surface information for the first time SOTA performance, Xiamen multi-modal protein-ligand affinity prediction AI method, combines molecular surface information for the first time Jul 17, 2024 pm 06:37 PM

Editor | KX In the field of drug research and development, accurately and effectively predicting the binding affinity of proteins and ligands is crucial for drug screening and optimization. However, current studies do not take into account the important role of molecular surface information in protein-ligand interactions. Based on this, researchers from Xiamen University proposed a novel multi-modal feature extraction (MFE) framework, which for the first time combines information on protein surface, 3D structure and sequence, and uses a cross-attention mechanism to compare different modalities. feature alignment. Experimental results demonstrate that this method achieves state-of-the-art performance in predicting protein-ligand binding affinities. Furthermore, ablation studies demonstrate the effectiveness and necessity of protein surface information and multimodal feature alignment within this framework. Related research begins with "S

SK Hynix will display new AI-related products on August 6: 12-layer HBM3E, 321-high NAND, etc. SK Hynix will display new AI-related products on August 6: 12-layer HBM3E, 321-high NAND, etc. Aug 01, 2024 pm 09:40 PM

According to news from this site on August 1, SK Hynix released a blog post today (August 1), announcing that it will attend the Global Semiconductor Memory Summit FMS2024 to be held in Santa Clara, California, USA from August 6 to 8, showcasing many new technologies. generation product. Introduction to the Future Memory and Storage Summit (FutureMemoryandStorage), formerly the Flash Memory Summit (FlashMemorySummit) mainly for NAND suppliers, in the context of increasing attention to artificial intelligence technology, this year was renamed the Future Memory and Storage Summit (FutureMemoryandStorage) to invite DRAM and storage vendors and many more players. New product SK hynix launched last year

See all articles