July's most popular AI research list is released, with Ma Yi's latest 'Standard Model” ranked ninth

The list of the most popular AI research in July is out!

This list compiled by Reddit netizen @bycloudai is ranked among the top ten AI research in July 2022 based on Twitter likes, retweets and Github stars, including DeepMind, Google, Well-known institutions such as MIT CSAIL.

Let’s take a look at who is on the list~

TOP1: Formal Algorithms for Transformers

Author: Mary Phuong, Marcus Hutter

Institution: DeepMind

Abstract: This paper is intended to be a stand-alone, mathematically accurate overview of the Transformer architecture and algorithm. It covers what Transformers are, how they are trained, their uses, their key architectural components and a preview of the most prominent models.

Top2: Confident Adaptive Language Modeling

Authors: Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q Tran, Yi Tay, Donald Metzler

Institutions: Google, MIT CSAIL

Abstract: Recent advances in Transformer-based large language models (LLMs) have driven significant performance improvements on many tasks. However, while performance improves, model size also increases dramatically, which may lead to complex inference processes and increased costs. In practice, however, large language models produce a series of iterations consisting of varying degrees of difficulty.

In this work, we introduce Confident Adaptive Language Model-ing (CALM), a framework for dynamically allocating varying amounts of computer input and generation duration.

Early exit decoding involves several issues we address here, such as: (1) what confidence measure to use; (2) linking sequence-level constraints to exit decisions for local tokens; (3) backtracking Hidden representation lost due to early exit of the previous token. Through theoretical analysis and experiments on three different text generation tasks, we demonstrate the efficacy of our framework in reducing computation – potentially speeding up up to 3x while maintaining high performance.

Top3：Language Models (Mostly) Know What They Know

Author: Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, etc.

Organization: Anthropic

Abstract: This paper investigates whether language models can assess the validity of their own claims and predict which questions they will be able to answer correctly. We first show that when larger models are provided in the correct format, they calibrate well to a variety of multiple choice and true/false questions. Therefore, we can self-evaluate the open sampling task by asking the model to first propose an answer and then evaluate the probability P(True) that its answer is correct.

We find P(True) exciting in its performance, calibration, and scaling across a variety of tasks. The performance of the self-assessment improves further when we allow the model to consider many of its own samples before predicting the validity of a particular possibility. Next, we investigate whether we can train a model to predict P(IK), the probability of "I know the answer to the question", without reference to any specific suggested answer.

Top4：YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time objectdetectors

Authors: Chien-Yao Wang, Alexey Bochkovskiy, Hong-Yuan Mark Liao

Institution: Institute of Information Science, Academia Sinica

Top5: Language Model Cascades

Author: David Dohan, Winnie Xu, Aitor Lewkowycz et al

Institution: Google

Top6: Collaborative Neural Rendering using AnimeCharacter Sheets

Author: Zuzeng Lin, Ailin Huang, Zhewei Huang et al

Institution: Wuhan University, Megvii Technology

Top7: Neural Networks and the Chomsky Hierarchy

Author: Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya, Tim Genewein, etc.

Institution: DeepMind

Top8: Language modeling with Pixels​

Author: Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, etc.

Institution: University of Copenhagen, Johns Hopkins University, Uppsala University

##Top9: On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence

Authors: Ma Yi, Cao Ying, Shen Xiangyang

Institution: University of California, Berkeley, Guangdong-Hong Kong-Macao Greater Bay Area Digital Economy Research Institute

This paper is a research review on the emergence and development of artificial intelligence published by Professor Ma Yi, computer scientist Dr. Shen Xiangyang, and neuroscientist Professor Cao Ying. It can be called an outline of the development of AI in the past 70 years. do.

Top10：Scaling Laws vs Model Architectures:How does Inductive Bias Influence Scaling

Author: Yi Tay, Mostafa Dehghani, Samira Abnar

Institution: Google, DeepMind

After reading the papers of the Top 10 experts, let’s talk about some interesting details of this list.

As we all know, Twitter likes can be generated by robots. The author’s use of the number of likes as a key indicator for the list is indeed open to question.

In addition, the previously highly popular "Infinite Visual Generation Model NUWA-Infinity" only ranked 12th in terms of number of likes on Twitter, but the number of Github stars has exceeded 2.4k.

Since NUWA Infinity released its first version as early as November 2021, this list only counts the number of likes for the second version after that. Therefore it only ranks 12th.

The above is the detailed content of July's most popular AI research list is released, with Ma Yi's latest 'Standard Model” ranked ninth. For more information, please follow other related articles on the PHP Chinese website!