Home web3.0 Pixel Transformers (PiTs) Challenge the Need for Locality Bias in Vision Models

Pixel Transformers (PiTs) Challenge the Need for Locality Bias in Vision Models

Jun 15, 2024 am 09:31 AM

A latest research by Meta AI and the University of Amsterdam have shown that transformers, a popular neural network architecture, can operate directly on individual pixels of an image without relying on the locality inductive bias present in most modern computer vision models.

Pixel Transformers (PiTs) Challenge the Need for Locality Bias in Vision Models

Meta AI and researchers from the University of Amsterdam have demonstrated that transformers, a popular neural network architecture, can operate directly on individual pixels of an image, without relying on the locality inductive bias present in most modern computer vision models.

Their study, titled "Transformers on Individual Pixels," challenges the long-held belief that locality – the notion that neighboring pixels are more related than distant ones – is a fundamental requirement for vision tasks.

Traditionally, computer vision architectures like Convolutional Neural Networks (ConvNets) and Vision Transformers (ViTs) have incorporated locality bias through techniques such as convolutional kernels, pooling operations, and patchification, assuming neighboring pixels are more related.

In contrast, the researchers introduced Pixel Transformers (PiTs), which treat each pixel as an individual token, removing any assumptions about the 2D grid structure of images. Surprisingly, PiTs achieved highly performant results across various tasks.

For instance, when PiTs were applied to image generation tasks using latent token spaces from VQGAN, they outperformed their locality-biased counterparts on quality metrics like Fréchet Inception Distance (FID) and Inception Score (IS).

While PiTs, operating on the lines of Perceiver IO Transformers, can be computationally expensive due to longer sequences, they challenge the need for locality bias in vision models. As advances in handling large sequence lengths are made, PiTs may become more practical.

The study ultimately highlights the potential benefits of reducing inductive biases in neural architectures, which could lead to more versatile and capable systems for diverse vision tasks and data modalities.

News source:https://www.kdj.com/cryptocurrencies-news/articles/pixel-transformers-pits-challenge-locality-bias-vision-models.html

The above is the detailed content of Pixel Transformers (PiTs) Challenge the Need for Locality Bias in Vision Models. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial
1662
14
PHP Tutorial
1261
29
C# Tutorial
1234
24
Nasdaq Files to List VanEck Avalanche (AVAX) Trust ETF Nasdaq Files to List VanEck Avalanche (AVAX) Trust ETF Apr 11, 2025 am 11:04 AM

This new financial instrument would track the token's market price, with a third-party custodian holding the underlying AVAX

OM Mantra Cryptocurrency Crashes 90%, Team Allegedly Dumps 90% of Token Supply OM Mantra Cryptocurrency Crashes 90%, Team Allegedly Dumps 90% of Token Supply Apr 14, 2025 am 11:26 AM

In a devastating blow to investors, the OM Mantra cryptocurrency has collapsed by approximately 90% in the past 24 hours, with the price plummeting to $0.58.

Zcash (ZEC) Reaches a High of $35.69 as a Record Amount of Tokens Move Out of Circulation Zcash (ZEC) Reaches a High of $35.69 as a Record Amount of Tokens Move Out of Circulation Apr 09, 2025 am 10:36 AM

Zcash was one of the top gainers during the latest market rally, reaching a high of $35.69 as traders moved a record amount of tokens out of circulation.

Can BRICS Win from Trump's Tariffs? Can BRICS Win from Trump's Tariffs? Apr 07, 2025 am 11:14 AM

The global economic landscape is continuously shifting, and one of the latest disruptions comes from former U.S. President Donald Trump's imposition of tariffs

Is Wall Street Quietly Backing Solana? $42 Million Bet Says Yes Is Wall Street Quietly Backing Solana? $42 Million Bet Says Yes Apr 10, 2025 pm 12:43 PM

A group of former Kraken executives acquired U.S.-listed company Janover, which secured $42 million in venture capital funding to begin building a Solana (SOL) treasury.

Dogecoin (DOGE) Price Plummets 17% Dogecoin (DOGE) Price Plummets 17% Apr 08, 2025 am 11:20 AM

The Dogecoin price plummeted 17% in the last 24 hours to trade at $0.1365 as of 4.30 a.m. EST on trading volume that skyrocketed 271% to $2.24 billion.

TrollerCat ($TCAT) Stands Out as a Dominant Force in the Meme Coin Market TrollerCat ($TCAT) Stands Out as a Dominant Force in the Meme Coin Market Apr 14, 2025 am 10:24 AM

Have you noticed the meteoric rise of meme coins in the cryptocurrency world? What started as an online joke has quickly evolved into a lucrative investment opportunity

As Fear Drives Selling, BlockDAG (BDAG) Stands Out from the Crowd As Fear Drives Selling, BlockDAG (BDAG) Stands Out from the Crowd Apr 13, 2025 am 11:48 AM

As fear drives selling in the crypto market, major coins like Cardano and Solana face tough times.