Since the advent of ChatGPT, large models and AI technology have attracted widespread attention around the world. On the one hand, people marvel at the emerging capabilities of large models; on the other hand, they are concerned about the controllability and future development of artificial intelligence. This year, many experts in the AI field, including Turing Award winners Geoffrey Hinton and Yoshua Bengio, have jointly warned that large AI models will cause a series of risks, and some have even called for a halt to the development of subsequent large AI models for GPT-4. .
OpenAI, as the company behind large models such as ChatGPT and GPT-4, has undoubtedly been pushed to the forefront. OpenAI CEO Sam Altman is currently on a global speaking tour to dispel people's "fear" about artificial intelligence and listen to the opinions of developers and users of OpenAI products.
According to "Fortune" report, in May Sam Altman met behind closed doors with some developers and startup founders, And talked about OpenAI’s roadmap and challenges. One of the participants in this closed-door meeting, Raza Habib, co-founder and CEO of Humanloop, recently mentioned OpenAI’s product planning and development bottlenecks in a blog.
The original blog has been deleted, but some netizens have uploaded a snapshot (copy) of the blog. Let’s take a look at the specific content of the blog:
Currently OpenAI faces very severe GPU limitations, which also delays the implementation of some of their short-term plans. The most common customer complaints these days are about the reliability and speed of APIs. Sam acknowledged the problem, explaining that most of the issues customers complained about were due to GPU shortages.
When it comes to processing text, the longer 32k context is not yet available to more people. Now OpenAI has not completely overcome the O (n^2) expansion problem of the attention mechanism. Although OpenAI seems to be able to achieve 100k-1M token context window (within this year) text processing soon, larger text processing windows require further progress. research breakthroughs.
Not only that, but currently, the fine-tuning API is also limited by GPU supply. OpenAI does not yet use efficient fine-tuning methods like Adapters or LoRa, so fine-tuning is very computationally intensive to run and manage. Sam revealed that better fine-tuning technology will be introduced in the future, and they may even provide a community dedicated to research models.
Additionally, dedicated capacity provision is limited by GPU supply. OpenAI also offers dedicated capacity, giving customers a private copy of their model. To use the service, customers must be willing to commit $100,000 up front.
During the conversation, Sam shared the near-term roadmap of OpenAI API, which is mainly divided into two stages:
Road to 2023:
Many developers are interested in accessing the ChatGPT plugin through the API, but Sam said He doesn't think these plugins will be released anytime soon. Use of plugins other than browsing suggests they don't have PMF yet. Sam pointed out that many people want their applications to be within ChatGPT, but what they really want is ChatGPT within their applications.
Many developers say that when OpenAI releases new products, they are nervous about applications built using the OpenAI API because OpenAI may eventually release a competing product. Sam said that OpenAI will not release more products besides ChatGPT. He said there are a lot of great platform companies that have a killer app, and ChatGPT will allow them to make their APIs better by becoming customers of their own products. The vision for ChatGPT is to be a super-intelligent work assistant, but there are many other GPT use cases that OpenAI won’t be getting into.
Although Sam advocates for regulation of future models, he does not believe that existing models are dangerous, And think regulating or banning them would be a huge mistake. He once again emphasized the importance of open source and said that OpenAI is considering open source GPT-3. Part of the reason why OpenAI has been slow to open source is because they feel that not many people and companies have the ability to properly manage such large language models.
Many recent articles have claimed that "the era of giant artificial intelligence models is over." Sam said that didn't convey exactly what he meant.
OpenAI’s internal data shows that the law of scaling still holds, and increasing the size of the model will continue to improve performance. However, model size cannot always increase at the same scale, as OpenAI has already increased model size millions of times in just a few years, and continuing to do so will be unsustainable. But that doesn’t mean OpenAI will stop trying to make its models bigger, but it does mean they might double or triple in size every year instead of growing by orders of magnitude.
The fact that the extended model is still valid has important implications for the development of AGI. The idea of scaling is that we probably already have most of the elements needed to build an AGI, and most of the remaining work will be taking existing methods and scaling them to larger models and larger data sets. If the era of model extensions is over, it will be even longer before we reach AGI. The fact that the law of scaling still applies implies that we will achieve AGI in less time.
The above is the detailed content of Sam Altman talks about OpenAI: Facing GPU shortage panic, GPT-3 may be open source. For more information, please follow other related articles on the PHP Chinese website!
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
