Table of Contents
From thinking "step by step" to thinking "little by little"
Has AI learned to hide its thoughts?
Home Technology peripherals AI AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black box

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black box

May 06, 2024 pm 12:00 PM
ai Model tc0

When AI does math problems, the real thinking is actually "mental arithmetic" secretly?

New research by a New York University team found that even if AI is not allowed to write steps and is replaced with meaningless "...", its performance on some complex tasks can be greatly improved!

First author Jacab Pfau said: As long as you spend computing power to generate additional tokens, you can bring advantages. It doesn’t matter what token you choose.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

For example, let Llama 34M answer a simple question: How many of the first 6 digits of the natural constant e are greater than 5 ?

The AI's direct answer is equivalent to making trouble. It only counts the first 6 digits and actually counts 7.

Let AI write out the steps to verify each number, and you can get the correct answer.

Let AI hide the steps and replace them with a lot of "...", and you can still get the correct answer!

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

This paper sparked a lot of discussion as soon as it was released, and was evaluated as "the most metaphysical AI paper I have ever seen."

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

So, young people like to say more meaningless words such as "um...", "like...", is it okay? Strengthen reasoning skills?

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

From thinking "step by step" to thinking "little by little"

In fact, the New York University team The research starts from the Chain-of-Thought (CoT).

That’s the famous prompt “Let’s think step by step”.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

In the past, it was found that using CoT inference can significantly improve the performance of large models on various benchmarks.

It’s unclear whether this performance improvement comes from imitating humans by breaking tasks into easier-to-solve steps, or whether it is a by-product of the extra calculations.

In order to verify this problem, the team designed two special tasks and corresponding synthetic data sets: 3SUM and 2SUM-Transform.

3SUM requires finding three numbers from a given set of number sequences so that the sum of the three numbers satisfies certain conditions, such as dividing by 10 with a remainder of 0.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

The computational complexity of this task is O(n3), and the standard Transformer uses the input of the upper layer and the activation of the next layer Only secondary dependencies can occur between them.

That is to say, when n is large enough and the sequence is long enough, the 3SUM task exceeds the expression ability of Transformer.

In the training data set, "..." with the same length as the human reasoning steps is filled between the question and the answer. That is, the AI ​​has not seen how humans disassemble the problem during training.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

In the experiment, the performance of Llama 34M that does not output the padding token "..." decreases as the sequence length increases, while the output When filling the token, 100% accuracy can be guaranteed until the length is 14.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

2SUM-Transform only needs to determine whether the sum of two numbers meets the requirements, which is within the expressive capabilities of Transformer.

But at the end of the question, a step is added to "randomly replace each number of the input sequence" to prevent the model from directly calculating on the input token.

The results show that using padding tokens can increase the accuracy from 78.7% to 93.6%.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxpicture

In addition to the final accuracy, the author also studied the hidden layer representation of the filled token. Experiments show that by freezing the parameters of the previous layers and only fine-tuning the last Attention layer, the prediction accuracy increases as the number of available filling tokens increases.

This confirms that the hidden layer representation of the populated token does contain implicit computation related to downstream tasks.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

Has AI learned to hide its thoughts?

Some netizens wonder, is this paper saying that the "thinking chain" method is actually fake? The prompt word project that I have been studying for so long has been in vain.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

The team stated that theoretically the role of filling tokens is limited to the scope of TC0 complexity problems.

TC0 is a computational problem that can be solved by a fixed-depth circuit, in which each layer of the circuit can be processed in parallel and can be quickly solved by a few layers of logic gates (such as AND, OR and NOT gates) , which is also the upper limit of computational complexity that Transformer can handle in single forward propagation.

And a long enough thinking chain can extend the expression ability of Transformer beyond TC0.

And it is not easy for a large model to learn to use padding tokens, and specific intensive supervision needs to be provided to converge.

That said, existing large models are unlikely to benefit directly from the padding token method.

But this is not an inherent limitation of current architectures; if provided with sufficient demonstrations in the training data, they should be able to obtain similar benefits from padding symbols.

This research also raises a worrying issue: large models have the ability to perform secret calculations that cannot be monitored, posing new challenges to the explainability and controllability of AI.

In other words, AI can reason on its own in a form invisible to people without relying on human experience.

This is both exciting and terrifying.

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

Finally, some netizens jokingly suggested that Llama 3 first generate 1 quadrillion dots, so that the weight of AGI can be obtained (dog head) .

AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black boxPicture

Paper:https://www.php.cn/link/36157dc9be261fec78aeee1a94158c26

Reference Link:
[1]https://www.php.cn/link/e350113047e82ceecb455c33c21ef32a[2]https://www.php.cn/link/872de53a900f3250ae5649ea19e5c381

The above is the detailed content of AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black box. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Centos shutdown command line Centos shutdown command line Apr 14, 2025 pm 09:12 PM

The CentOS shutdown command is shutdown, and the syntax is shutdown [Options] Time [Information]. Options include: -h Stop the system immediately; -P Turn off the power after shutdown; -r restart; -t Waiting time. Times can be specified as immediate (now), minutes ( minutes), or a specific time (hh:mm). Added information can be displayed in system messages.

How to check CentOS HDFS configuration How to check CentOS HDFS configuration Apr 14, 2025 pm 07:21 PM

Complete Guide to Checking HDFS Configuration in CentOS Systems This article will guide you how to effectively check the configuration and running status of HDFS on CentOS systems. The following steps will help you fully understand the setup and operation of HDFS. Verify Hadoop environment variable: First, make sure the Hadoop environment variable is set correctly. In the terminal, execute the following command to verify that Hadoop is installed and configured correctly: hadoopversion Check HDFS configuration file: The core configuration file of HDFS is located in the /etc/hadoop/conf/ directory, where core-site.xml and hdfs-site.xml are crucial. use

What are the backup methods for GitLab on CentOS What are the backup methods for GitLab on CentOS Apr 14, 2025 pm 05:33 PM

Backup and Recovery Policy of GitLab under CentOS System In order to ensure data security and recoverability, GitLab on CentOS provides a variety of backup methods. This article will introduce several common backup methods, configuration parameters and recovery processes in detail to help you establish a complete GitLab backup and recovery strategy. 1. Manual backup Use the gitlab-rakegitlab:backup:create command to execute manual backup. This command backs up key information such as GitLab repository, database, users, user groups, keys, and permissions. The default backup file is stored in the /var/opt/gitlab/backups directory. You can modify /etc/gitlab

How is the GPU support for PyTorch on CentOS How is the GPU support for PyTorch on CentOS Apr 14, 2025 pm 06:48 PM

Enable PyTorch GPU acceleration on CentOS system requires the installation of CUDA, cuDNN and GPU versions of PyTorch. The following steps will guide you through the process: CUDA and cuDNN installation determine CUDA version compatibility: Use the nvidia-smi command to view the CUDA version supported by your NVIDIA graphics card. For example, your MX450 graphics card may support CUDA11.1 or higher. Download and install CUDAToolkit: Visit the official website of NVIDIACUDAToolkit and download and install the corresponding version according to the highest CUDA version supported by your graphics card. Install cuDNN library:

Detailed explanation of docker principle Detailed explanation of docker principle Apr 14, 2025 pm 11:57 PM

Docker uses Linux kernel features to provide an efficient and isolated application running environment. Its working principle is as follows: 1. The mirror is used as a read-only template, which contains everything you need to run the application; 2. The Union File System (UnionFS) stacks multiple file systems, only storing the differences, saving space and speeding up; 3. The daemon manages the mirrors and containers, and the client uses them for interaction; 4. Namespaces and cgroups implement container isolation and resource limitations; 5. Multiple network modes support container interconnection. Only by understanding these core concepts can you better utilize Docker.

Centos install mysql Centos install mysql Apr 14, 2025 pm 08:09 PM

Installing MySQL on CentOS involves the following steps: Adding the appropriate MySQL yum source. Execute the yum install mysql-server command to install the MySQL server. Use the mysql_secure_installation command to make security settings, such as setting the root user password. Customize the MySQL configuration file as needed. Tune MySQL parameters and optimize databases for performance.

How to view GitLab logs under CentOS How to view GitLab logs under CentOS Apr 14, 2025 pm 06:18 PM

A complete guide to viewing GitLab logs under CentOS system This article will guide you how to view various GitLab logs in CentOS system, including main logs, exception logs, and other related logs. Please note that the log file path may vary depending on the GitLab version and installation method. If the following path does not exist, please check the GitLab installation directory and configuration files. 1. View the main GitLab log Use the following command to view the main log file of the GitLabRails application: Command: sudocat/var/log/gitlab/gitlab-rails/production.log This command will display product

How to operate distributed training of PyTorch on CentOS How to operate distributed training of PyTorch on CentOS Apr 14, 2025 pm 06:36 PM

PyTorch distributed training on CentOS system requires the following steps: PyTorch installation: The premise is that Python and pip are installed in CentOS system. Depending on your CUDA version, get the appropriate installation command from the PyTorch official website. For CPU-only training, you can use the following command: pipinstalltorchtorchvisiontorchaudio If you need GPU support, make sure that the corresponding version of CUDA and cuDNN are installed and use the corresponding PyTorch version for installation. Distributed environment configuration: Distributed training usually requires multiple machines or single-machine multiple GPUs. Place

See all articles