Google lost the LLM battle! More and more top researchers are switching jobs at OpenAI

A few days ago, Google almost encountered a public relations crisis. Jacob Devlin, a former employee of Bert who had quit OpenAI, revealed that Bard was actually trained using ChatGPT data.

Subsequently, Google quickly denied it.

And this controversy has also led to a big discussion: Why are more and more top Google researchers switching jobs to OpenAI? Can it still win this LLM battle?

Reply from a friend

A doctor from Rice University and a friend said that the gap between Google and OpenAI is actually Data gaps.

"OpenAI has a strong obsession with LLM, which is completely unmatched by companies like Google. Of course, the gap between people is only one aspect, the gap in data and the attitude towards data are the keys to its success. . People may be able to spend money to dig it back, and technology may be able to make up for it, but the data gap will not be caught up by Google in a short time.”

##https:/ /arxiv.org/abs/2303.10158

Obviously, a crucial part of the success of ChatGPT is high-quality annotated data.

"A Pile of Waste Paper" introduces that OpenAI attaches great importance to the quality of data and tags. It is extremely strict in the selection of annotators (there are exams), and it will even issue questionnaires in the end. It is this obsession that has created the success of the GPT model, which Google cannot match at all.

As hundreds of millions of users continue to provide OpenAI with new data, the gap between Google and OpenAI will only grow wider.

The concept of Data-centric AI has laid the foundation for OpenAI’s success.

"Zhang Junlin", an excellent respondent on natural language processing and deep learning topics, said that OpenAI has left everyone behind.

Including Google, in fact, the understanding of LLM development concept is obviously one step behind OpenAI. The reality is that OpenAI has performed too well and has left everyone behind, not just domestically. I think that OpenAI is ahead of Google and DeepMind abroad for about half a year to a year in terms of concepts and related technologies for LLM, and it is about two years ahead of China.

Zhihu respondent "Zhou Daodao" said that comparing Google's recent overturn with OpenAI must have given these top researchers a huge shock.

In addition, it is said that OpenAI will give researchers more resources and broader requirements than Google. After all, OpenAI is more like a research institution, while Google is more like a department that treats AI as a product.

The answer given by the respondent "Chen Dabao" is very heartbreaking.

Future business history will record two classic and ironic cases:

1. Kodak invented the digital camera

2. Google invented the transformer

Photo source: "Chen Dabao"

An anonymous person with interests also appeared in the comment area.

Another anonymous person hit the mark.

# A friend of mine pointed out the "innovator's dilemma" of Google as a large company.

Another anonymous person with an interest appears.

Zhizhi "Zhou Xingnan (Bill)" concluded that large companies like Google are going to talent-based. Based on policy, everyone is a screw.

And Insider summarized a long article for us, taking stock of the top talents that have been lost from Google’s artificial intelligence team over the years.

Where have all the talents gone?

Google has contributed some very important research to the field of artificial intelligence. However, the company has been slow to turn its innovations into products.

As a result, top artificial intelligence researchers are leaving for startups that can generate greater value and impact, such as OpenAI, Character.AI, DeepMind, Cohere, Inceptive.

Google is already in a defensive position and will lose its leading position in the field of artificial intelligence at any time. The departure of top researchers has exacerbated the problem.

Cohere co-founding team Ivan Zhang, Aidan Gomez, and Nick Frosst

While Google may be in defensive mode, it doesn’t need to be. The company created much of the underlying technology itself, powering products like ChatGPT. It also makes its research available as open source, which somewhat ironizes OpenAI's meteoric rise.

Google has long been hesitant to release chatbots similar to ChatGPT out of concern that the technology would cause reputational damage to its business.

Daniel De Freitas and Noam Shazeer, two researchers behind Google's large language model LaMDA, have left the company, frustrated by its delay in releasing a chatbot similar to ChatGPT.

Other former Google researchers also believe that in such an exciting era of artificial intelligence, startups will provide researchers with ownership of their results and exert more value and impact themselves.

Here are some of the most high-profile papers in artificial intelligence by researchers who have left Google for other companies.

Ilya Sutskever

"Sequence-to-Sequence Learning with Neural Networks" was published in 2014. This sequence-to-sequence paper discusses training language models and transforming a field into Convert a sequence of words into a sequence in another domain. For example, convert an English sentence to a French sentence.

Ilya Sutskever led the research for this paper. He left Google in 2015 after nearly three years as a research scientist. Sutskever is a co-founder of OpenAI and continues to work there as its chief scientist.

Attention is All You Need

This Transformer’s blockbuster paper has now been cited more than 70,000 times Second-rate. Transformer is considered a breakthrough in natural language processing. It helps AI understand meaning by gleaning contextual nuances by looking at each word in a sentence simultaneously and weighing the importance of each word.

The "T" in ChatGPT represents Transformer, which shows the importance of this paper.

However, the eight authors of this article, except Llion Jones, have all left Google.

Ashish Vaswani left Google Brain (Google’s deep learning artificial intelligence research team) after five years to start Adept, a company that recently raised $350 million. Build generative AI tools to help people use productivity software more effectively. He recently left Adept for a stealthy startup.

Noam Shazeer is now the CEO of Character.AI.

Niki Parmar left Google Brain after five years to serve as co-founder and chief technology officer of Adept, but like Vaswani, she also recently left , went to a secretive startup company.

Jakob Uszkoreit has worked at Google for 13 years, working on neural networks and deep learning. He is now co-founder of Inceptive, a startup using deep learning to design new treatments.

Aidan Gomez is co-founder and CEO of Cohere, a company that has raised approximately $160 million to help developers bring generative artificial intelligence to life Incorporate into their apps and websites. He spent a year and a half as a researcher at Google Brain. And his co-founder at Cohere, Nick Frosst, spent four years as a researcher at Google Brain.

Lukasz Kaiser left Google Brain after working at Google Brain for more than 7 years and joined OpenAI in 2021. Kaiser was recently cited in OpenAI’s GPT-4 white paper as a core contributor to its long-context capabilities, allowing chatbots to have longer conversations before forgetting the context of the discussion.

Illia Polosukhin has been working on deep learning and natural language understanding at Google Brain for three years. He left in 2017 and founded Pagoda, a Web3 startup platform.

Building a human-like open domain chatbot

This paper introduces Google’s original chatbot Meena, Explored how chatbots can learn to talk about topics by studying data scraped from public social media conversations. It also describes a test created by Google to evaluate the speaking performance of chatbots.

This paper is another important milestone in large language modeling. The authors believe that they can make a large language model that can produce human-like responses to problems without hard-coded training.

One of the authors, Daniel De Freitas, served as co-founder and president of Character.AI after five years as a researcher at Google Brain.

De Freitas’ Character.AI colleague Romal Thoppilan also contributed to this article.

On the left is Romal Thoppilan; on the right is Daniel De Freitas

LaMDA: Language model for conversational applications

LaMDA is the abbreviation of Language Model for Dialogue Applications and is the basis of the chatbot Bard. It was first demoed in 2020 as Meena, but Google never released Meena to the public. Former employees of Google's artificial intelligence research department explained that this would be a public relations nightmare because Google was worried that the bots would make harmful comments.

# Several key researchers behind LaMDA have left Google Brain.

Character.AI, a company founded last year by Daniel De Freitas and Noam Shazeer, recently raised about $200 million to create chatbots that speak in the form of a variety of characters, from Musk to therapists to life coaches. kind of role.

Romal Thoppilan serves as a founding researcher at Character.AI after working at Google Brain for 7 years.

Alicia Jin will join Character.AI near the end of 2022 as a research engineer. She worked at Google Brain for three years.

BERT

BERT (Bidirectional Encoder Representations from Transformers) is built on the Transformer model of natural language processing. After pre-training, it can complete two tasks well: Masked language modeling and next sentence prediction. In other words, BERT attempts to predict hidden or "masked" words, forcing the algorithm to work hard to learn more about the surrounding text and better predict hidden words.

If you type "Can you buy medicine for someone else's pharmacy?" it will understand that "other people" is an important part of the query.

Google began to incorporate BERT into its search engine as early as 2019. This is one of the biggest improvements in search accuracy since the incorporation of another machine learning algorithm, RankBrain, in 2015.

Jacob Devlin is the lead author of this paper, and he joined OpenAI shortly before the launch of ChatGPT.

T5

#The official name of the T5 paper is "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer". It Built on the basis of BERT, it is very suitable for tasks such as translation and summarization.

Colin Raffel, who led the paper, served as a research scientist at Google Brain for about five years before leaving in 2021. Currently an assistant professor at UNC-Chapel Hill, he spends one day a week as a researcher at Hugging Face. Hugging Face recently announced that it raised $100 million in May 2022, valuing the company at $2 billion. Users can share large language models and data sets on Hugging Face.

Sharan Narang, another contributor to the T5 paper, left Google Brain in 2022 after four years there. He is now an artificial intelligence researcher at Meta Corporation.

A pattern placement method for rapid chip design

Led by Google scientists Azalia Mirhoseini and Anna Goldie The paper found that artificial intelligence can complete the chip design process faster than human experts.

Another paper led by the duo, "Chip Layout Using Deep Reinforcement Learning," provides a way to use artificial intelligence in chip design to maximize Improve performance while minimizing area and power usage.

#These findings helped Google design TPU chips specifically for machine learning tasks.

Mirhoseini and Goldie both left Google in 2022 to join Anthropic, an OpenAI competitor that is developing its own large-scale language model and a chatbot called Claude.

DeepMind

Mustafa Suleyman is the co-founder of DeepMind and serves as the company’s chief product officer. This is an artificial intelligence laboratory that was acquired by Google in 2014. The lab developed AlphaGo, the machine learning program that beat world champion professionals at the game of Go.

Google’s parent company Alphabet recently announced in its fourth-quarter earnings report that DeepMind’s financial results will be separated from “other investments,” marking the rise of artificial intelligence. importance in Google's future strategy. Typically, "other investments" is a general term for the company's nascent projects that have not yet reached profitability.

Suleyman has been a vocal advocate for ensuring the safety of new artificial intelligence products. While working at DeepMind, he established a research department called DeepMind Ethics and Society to study the real-life impacts of artificial intelligence. In 2019, he was placed on leave by DeepMind after being accused of bullying employees. He moved back to Google as a vice president while the investigation was ongoing.

Suleyman has been cited in many research papers related to machine learning. In February 2022, he co-founded the artificial intelligence startup Inflection with LinkedIn creator Reid Hoffman.

Many of the key contributors to Google’s most groundbreaking artificial intelligence papers have left, joining competitors such as OpenAI or starting their own companies.

When OpenAI’s ChatGPT came out at the end of 2022, Google CEO Sundar Pichai announced a “red alert” internally, calling on the company to respond.

At the time, Pichai said that Google would continue to hire top researchers in artificial intelligence without question.

But after losing so many capable people, can Google still achieve greatness again?

References:

https://www.businessinsider.com/google-ai-teams-brain-drain-researchers-leave-2023-3

https: //www.zhihu.com/question/592975340/answer/2963265672

https://zhuanlan.zhihu.com/p/597586623

https://www.zhihu.com/ question/592975340/answer/2964598555

The above is the detailed content of Google lost the LLM battle! More and more top researchers are switching jobs at OpenAI. For more information, please follow other related articles on the PHP Chinese website!