Ray, the open source AI framework behind ChatGPT, is now worth $1 billion

Text-generating artificial intelligence has taken the internet by storm recently: ChatGPT is popular for its ability to provide highly detailed, near-lifelike answers to almost any question one can think of. The emergence of large model applications has made people full of confidence in AI technology breakthroughs, but few people know that behind it, a distributed machine learning framework is powering this generative AI revolution.

Distributed computing framework Ray from A16z-backed startup Anyscale is key to enabling OpenAI to power up its training models like ChatGPT. Ray is behind all of OpenAI's recent large-scale language models — and it may also be the framework behind OpenAI's much-anticipated GPT-4. With the continuous implementation of large-scale model technology, industry insiders believe that an industry worth billions of dollars is being formed by generating content that is close to humans.

In this field, Ray is the most influential framework. Before its advent, OpenAI used a custom collection of tools to develop large models. But OpenAI president Greg Brockman said at the Ray Summit earlier this year that the company had turned to Ray as the challenges it faced increased.

Lukas Biewald, CEO of software company Weights & Biases, believes that Ray is already a hot rising star in the AI ​​world. "Because of new tools, you can run the same code on a laptop and on a large distributed server. That's a huge change, and it's going to increase in importance as the models get bigger," Biewald said.

A billion-dollar bet

As the technology matures, Ray has attracted the attention of the capital market. Anyscale's equity has become a scarce commodity, with Business Insider reporting that its latest funding round, an extension of its Series C round that valued it at more than $1 billion, closed within days, according to people familiar with the matter.

Some investors have described Anyscale as Horowitz’s hopeful “next Databricks” — a description that seems reasonable, given that the startup’s co-founder, Ion Stoica He is the co-founder of Databricks, a data giant with a market capitalization of $31 billion.

“Artificial intelligence is developing at an incredible pace and people are trying new approaches all the time,” said Robert Nishihara, CEO of Anyscale. "ChatGPT combines a lot of previous work on large language models. On top of this, you need to have an infrastructure that enables flexibility, rapid innovation, and expansion of different algorithms and methods."

With ever-larger models behind hot new tools like ChatGPT, tech companies are having to rethink the way they develop AI from the ground up. Ray was born to make it easier to train these massive models and can contain hundreds of billions of data points, giving each response a quasi-lifelike feel.

How Ray becomes the tool of choice for machine learning

Ray is a distributed computing framework based on memory sharing, suitable for fine-grained parallel computing and heterogeneous computing. It provides an underlying infrastructure for managing the complex task of distributing the work of training machine learning models.

In 2017, UC Berkeley researchers submitted Ray's paper "Ray: A Distributed Framework for Emerging AI Applications" for the first time:

• Paper link: https://arxiv.org/abs/1712.05889
• GitHub: https:// github.com/ray-project/ray

#In this work, the researchers predict what the next generation of AI applications will look like: one with continuous interactions with the environment , and learn from interactive actions. These applications must increasingly complete tasks in dynamic environments, react to changes in the environment, and perform a series of actions to achieve long-term goals. These characteristics have put forward new and demanding system requirements for the performance and flexibility of the operating environment, so researchers have proposed a distributed-based Ray framework.

Ray implements a unified interface that can express task parallelism and actor-based computation, supported by a single dynamic execution engine. To meet performance requirements, Ray uses a distributed scheduler and distributed fault-tolerant storage to manage the system's control state. It is the first distributed computing framework that unifies training, simulation and services. It unifies role parallel (actor) and task parallel (task) calculations based on a dynamic task execution engine, and ensures the high scalability and high performance of the framework. Fault tolerance.

Ray's architecture.

Based on this work, in December 2019, Robert Nishihara, Philipp Moritz and Ion Stoica of UC Berkeley and Berkeley Professor Michael I. Jordan founded Anyscale. The company has raised $260 million so far.

Machine learning practitioners can often run small models using limited data sets on their laptops, such as simple models that predict what products users will buy. . However, laptops are not feasible for very large models like ChatGPT, which require massive servers to train.

Training a model using a large number of devices faces an important challenge - coordinating training on different hardware. Ray just solves this problem. It provides practitioners with a mechanism to manage different hardware as a unit to determine what data goes where, handle failures, etc. The hardware types span Google Cloud, AWS and other A portfolio of products that address the same problem. In addition, Ray also extended "actor", a key programming concept in other languages, to Python, which is known to be the language of choice for machine learning programs.

As a distributed computing framework, Ray has two key advantages, namely location-aware (Locality-aware) and task placement (task placement) ). As shown in the figure below, Ray is able to scale out the system to support high-throughput fine-grained tasks while maintaining fault tolerance and low-latency task scheduling.

Ray removes significant complexity from training large models for OpenAI, freeing up the company to focus on the model’s critical capabilities .

The next generation of AI requires new development tools, and Ray is just one of a rapidly emerging set of next-generation machine learning tools that are rapidly disrupting the way AI is developed. For example, Google's JAX framework has also received huge attention. JAX is expected to become the backbone of Google's core machine learning tools and has been widely adopted in DeepMind and Google Brain.

Similarly, Coiled, a startup backed by FirstMark Capital and Bessemer Venture Partners, has developed a parallel computing framework called Dask.

Large-scale language models are unlocking more potential recently, and these new machine learning tools will build more powerful language models for technology giants and startups in the industry.

The above is the detailed content of Ray, the open source AI framework behind ChatGPT, is now worth $1 billion. For more information, please follow other related articles on the PHP Chinese website!