Simulating brain functions, this AI model truly achieves continuous learning like a human being

As breakthroughs in various AI fields such as image processing, smart healthcare, self-driving cars, and smart cities demonstrate, deep learning is undoubtedly experiencing a golden age. Over the next decade or so, AI and computer systems will finally have human-like learning and thinking capabilities to process a constant flow of information and interact with the real world.

However, current AI models suffer performance losses when continuously trained on new information. This is because whenever new data is generated, it is written on top of the existing data, erasing the previous information. This effect is called "catastrophic forgetting." The stability-plasticity dilemma results in the need for AI models to update their memory to continuously adapt to new information while maintaining the stability of current knowledge. This issue prevents SOTA AI models from continuously learning from real-world information.

Meanwhile, edge computing systems allow computing to be moved from cloud storage and data centers closer to the original source, such as devices connected to the Internet of Things. Efficiently applying continuous learning on resource-limited edge computing devices remains a challenge, although many continuous learning models have been proposed in the field to solve this problem. Traditional models require high computing power and large storage capacity.

Recently, a research team from the Singapore University of Technology and Design (SUTD) designed a new model that can achieve a highly energy-efficient continuous learning system, studying "Continual Learning Electrical Conduction in Resistive-Switching-Memory Materials" ” was published in the journal Advanced Theory and Simulation.

Paper address: https://onlinelibrary.wiley.com/doi/abs/10.1002/adts.202200226

The The team proposed a model inspired by the brain, Brain-Inspired Replay (BIR), which can naturally carry out continuous learning. The BIR model is based on artificial neural networks and variational autoencoders to simulate the functions of the human brain and performs well in incremental learning-like scenarios without stored data. Researchers have also used BIR models to represent the growth of conductive filaments using electric current in digital memory systems.

The upper left of the figure below shows the generative playback settings, the upper right shows the training of artificial neural network with generative playback; the lower left and lower right show the normalized current accuracy of the conventional and BIR models respectively.

Assistant Professor Loke, one of the corresponding authors of the paper, explained, “In BIR, knowledge is saved in the trained model, thereby introducing additional tasks. The performance loss is minimized at the same time, and there is no need to refer to data from previous work. Therefore, this approach can save a lot of energy."

He also added, "In the absence of stored data, Achieved 89% SOTA accuracy on the compliance challenge of current learning tasks, which is approximately two times higher than traditional continuous learning models, and achieves high energy efficiency."

In addition, in order To enable the BIR model to independently process real-world field information, the team plans to expand its tunability in the next phase of research.

Loke said the research is based on a small-scale demonstration and is still in its early stages. However, taking this approach promises to allow edge AI systems to develop independently without human control.

The above is the detailed content of Simulating brain functions, this AI model truly achieves continuous learning like a human being. For more information, please follow other related articles on the PHP Chinese website!