Li Feifei reveals the entrepreneurial direction of 'spatial intelligence': visualization turns into insight, seeing becomes understanding, and understanding leads to action

After Stanford Li Feifei started his business, he revealed the new concept "spatial intelligence" for the first time.

This is not only her entrepreneurial direction, but also the "North Star" that guides her. She considers it to be "the key puzzle to solve the artificial intelligence problem."

Visualization becomes insight; seeing becomes understanding; understanding leads to action.

Based on the complete disclosure of Li Feifei’s 15-minute TED speech, from the origin of the evolution of life hundreds of millions of years ago to how humans We are not satisfied with the development of artificial intelligence given by nature until the next step is to build spatial intelligence.

Nine years ago, Li Feifei introduced the newly born ImageNet to the world on the same stage - one of the starting points for this round of deep learning explosion.

She herself also expressed her confidence to netizens: If watches both videos, you will be able to understand computer vision and space in the past 10 years. Intelligence and AI have a good understanding.

Below, we will sort out the content of Li Feifei’s speech without changing its original meaning.

Spatial intelligence, allowing AI to understand the real world

The evolution of biological vision

Let me show you something, exactly, I will show you "Nothing".

This is the world 540 million years ago. Pure, endless darkness. It's not dark because of lack of light. It is dark because of the lack of vision.

Although sunlight can penetrate 1,000 meters below the surface of the ocean, and light from hydrothermal vents reaches the bottom of the sea, teeming with life, there is not a single eye to be found in these ancient waters.

No retina, no cornea, no lens. So all this light, all this life, remains unseen.

There was a time when the concept of "seeing" did not exist. It had never been realized until it was.

For reasons we are only beginning to understand, the first organisms that could sense light appeared—trilobites. They are the first creatures capable of sensing the reality we take for granted. They were the first creatures to discover that there was something other than themselves.

For the first time, the world is filled with many "selves".

Visual abilities are thought to have triggered the Cambrian explosion, a period when animal species entered the fossil record in large numbers. What begins as a passive experience, the simple act of letting in light, soon becomes more active and the nervous system begins to evolve.

Vision becomes insight. Seeing becomes understanding. Understanding leads to action.

All these give rise to intelligence.

The rise of computer vision

Today, we are no longer satisfied with the visual capabilities given by nature. Curiosity drives us to create machines that can see at least as well as we do, if not better.

Nine years ago, on this stage, I submitted an early progress report on computer vision.

At that time, three powerful forces came together for the first time:

• A class of algorithms called Neural Networks
• Fast, specialized hardware, called a graphics processing unit, or GPU
• plus Big Data, such as the 15 million images my lab spent several years sorting , called ImageNet.

Together they ushered in the modern era of artificial intelligence.

We have come a long way from then to now.

In the beginning, just labeling images was a major breakthrough, but the speed and accuracy of the algorithm quickly improved.

This progress is measured in the annual ImageNet Challenge hosted by my lab. In this chart, you can see the improvement in model capabilities each year, and some of the milestone models.

We went a step further and created algorithms that can segment visual objects or predict dynamic relationships between them, work done by my students and collaborators.

there are more.

Recall the first computer vision algorithm I showed in my last speech. AI can describe a photo using human natural language. That's what I did with my brilliant student Andrej Karpathy.

At that time, I boldly said: "Andrej, can we make the computer do the opposite?" Andrej smiled and said: "Haha, that's impossible. ."

Well, as you can see today, the impossible has become possible.

This is thanks to a series of diffusion models that power today’s generative AI algorithms, which can transform human prompt words into photos and videos to create entirely new things.

Many of you have seen OpenAI’s Sora achieve impressive results recently. However, a few months ago, without a lot of GPUs, my students and collaborators developed an AI video generation model called Walt.

△Walt Published in December 2023

There is room for improvement here, look at that cat’s eyes, It never got wet under the waves, what a disaster~(cat-astrophe).

(Homophonous stalks deduct money!)

Spatial Intelligence: Just looking is not enough

The past is a prologue, we will learn from these mistakes Learn and create the future we imagine. In this future, we want AI to do everything it can to do things for us, or help us do things.

For years I have been saying that taking pictures is not the same thing as seeing and understanding. Today, I would like to add one more point: just looking is not enough.

Look, for action and learning.

When we act in 3D space and time, we learn, we learn to see better and do things better. Nature creates a virtuous cycle of seeing and acting through "spatial intelligence."

To demonstrate what spatial intelligence is, take a look at this photo. If you feel the urge to do something, raise your hand.

In a split second, your brain observes the geometry of the cup, its position in 3D space, its relationship to the table, the cat, and all other objects, and You can predict what will happen next.

The urge to act is inherent in all creatures with spatial intelligence, which links perception to action.

If we want AI to go beyond current capabilities, we not only want AI that can see and speak, we want AI that can act.

In fact, we are making exciting progress.

The latest milestone in spatial intelligence is teaching computers to see, learn, act, and learn to see and act better.

And it’s not easy.

Nature has spent millions of years evolving spatial intelligence. The eye captures light and projects a 2D image onto the retina, and the brain converts this data into 3D information.

Until recently, a group of researchers from Google developed an algorithm to convert a set of photos into a 3D space.

My students and collaborators went a step further and created an algorithm that turns a single image into a 3D shape.

#A team of researchers at the University of Michigan has found a way to convert sentences into 3D room layouts.

My colleague at Stanford University and his students have developed an algorithm that can generate an infinite space of possibilities from a single image for viewers to explore.

These are prototypes of future possibilities. Within this possibility, humans can transform our entire world into digital form and simulate its richness and subtlety.

What nature does implicitly in each of our minds, spatial intelligence technology promises to do the same thing for our collective consciousness.

#With the accelerated progress of spatial intelligence, a new era is unfolding before our eyes in this virtuous cycle. This cycle is catalyzing robot learning, a key component of any embodied intelligence system that needs to understand and interact with the 3D world.

Ten years ago, my lab’s ImageNet enabled a database of millions of high-quality photos to help train computer vision.

Today we are doing something similar,

training computers and robots how to act in a 3D world

. This time instead of collecting static images, we develop simulation environments driven by 3D spatial models so that the computer can learn the infinite possibilities of actions.

What you just saw is a small sample of the robots that teach us, a project led by my lab called Behavior.

We are also making exciting progress in robotic language intelligence.

Using input based on large language models, my students and collaborators are one of the first teams to demonstrate that a robotic arm can perform a variety of tasks based on verbal instructions.

Like opening this drawer or unplugging the phone cord. Or make a sandwich using bread, lettuce, tomatoes, or even place a napkin for the user. Normally I'd like a sandwich that's a little more substantial, but this is a good place to start.

The application prospects of spatial intelligence

In the primitive oceans of ancient times, the ability to see and sense the environment triggered the desire to interact with other life forms Cambrian explosion.

Today, that light is reaching digital thinking.

Spatial intelligence allows machines to interact not only with each other, but also with humans, and the real or virtual 3D world.

As this future takes shape, it will have a profound impact on many lives.

Let’s take healthcare as an example. Over the past decade, my lab has been conducting initial efforts to apply AI to challenges that impact patient outcomes and healthcare staff fatigue.

Together with collaborators from Stanford School of Medicine and other partner hospitals, we are piloting smart sensors that can detect if a clinician enters a patient room without properly washing their hands. Or tracking surgical instruments, or alerting care teams when a patient is at physical risk, such as a fall.

We think of these technologies as a form of ambient intelligence, like

extra eyes.

But I would prefer more interactive assistance for our patients, clinicians and caregivers who desperately need an extra pair of hands.

Imagine an autonomous robot transporting medical supplies while caregivers focus on the patient, or using augmented reality to guide surgeons through safer, faster, less invasive procedures.

Also imagine that severely paralyzed patients could control robots with their thoughts. That’s right, using brain waves to perform the everyday tasks you and I take for granted.

This is a recent pilot study conducted in my lab. In this video, a robotic arm, controlled solely by electrical signals from the brain, is cooking a Japanese sukiyaki meal. where signals are collected non-invasively through an EEG cap.

Five hundred million years ago, the emergence of vision overturned the dark world and triggered the most profound evolutionary process: the development of intelligence in the animal world.

The progress of AI in the past decade has been equally amazing. But I believe that the full potential of this Digital Cambrian Explosion will not be fully realized until we empower computers and robots with spatial intelligence, just as nature has done for all of us.

It’s an exciting time to teach our digital companions how to reason and interact with this beautiful 3D space we call home, while also creating more new worlds we can explore.

Achieving this future won’t be easy, it requires all of us to think deeply and develop technology that always puts people at the center.

But if we do it right, computers and robots powered by spatial intelligence will become not only useful tools, but also trusted partners, making us more productive and empowering while respecting the dignity of the individual. humanity and enhance our collective prosperity.

The future I am most excited about is one in which AI becomes more sentient, insightful, and spatially aware, and joins us in our pursuit of creating a better world. method.

（Full text ends）

Video playback: https://www.ted.com/talks/fei_fei_li_with_spatial_intelligence_ai_will_understand_the_real_world/transcript

The above is the detailed content of Li Feifei reveals the entrepreneurial direction of 'spatial intelligence': visualization turns into insight, seeing becomes understanding, and understanding leads to action. For more information, please follow other related articles on the PHP Chinese website!