30,000 A100s are too expensive, and 300 people at Microsoft secretly developed their own AI chips for 5 years! TSMC 5nm, ChatGPT saves about 30%

In order to train ChatGPT, Microsoft has spent enough money.

The total computing power consumed by ChatGPT is about 3640PF-days. If it is calculated one quadrillion times per second, it will take 3640 days to calculate.

Microsoft connected more than 30,000 NVIDIA A100 chips to build a supercomputer tailored for OpenAI at a cost of hundreds of millions of dollars.

If the consumption continues like this, Microsoft's pockets may not be able to bear it.

In fact, the plan to replace Nvidia chips has been secretly carried out within the company since 2019.

The Information reported that about five years ago, Microsoft began to develop a chip, internally code-named "Athena", and was jointly developed by 300 people.

The original plan was to use TSMC’s 5nm process.

Self-developed Athena, challenging NVIDIA

Needless to say, "Athena" is specially designed for training large language models (LLM) ) and designed.

Now, if Microsoft wants to put all GPT-4 capabilities into FamilyMart buckets including Bing, Microsoft 365 and GitHub, it will definitely require a lot of computing power support.

Shockingly, the cost of developing a chip like Athena could be about $100 million per year.

When training models, researchers can use the capabilities of "Athena" to process new data and perform inference at the same time.

In this way, the previous situation of having to use dedicated computers to process artificial intelligence software has been alleviated. After all, computers specifically used to run AI have never been enough.

Because only NVIDIA previously produced this kind of chip, and the lack of supply has made the entire technology industry feel this shortage.

Because of this, Microsoft has to provide dedicated chips for some internal teams.

Previously, research company SemiAnalysis estimated that ChatGPT burned US$700,000 per day, with each query costing 0.36 cents.

If "Athena" is competitive, it can reduce the cost of each chip by 1/3.

Tracy Woo, senior cloud analyst at Forrester Research, said that the explosion of AI has caused major companies to rush into it, and the lack of chips not only puts pressure on suppliers, but also Putting pressure on AI technology companies.

Large technology companies like Google and Amazon have enough capital to design and develop their own chips, and other companies have to work hard to keep up.

Let’s talk about Microsoft’s “Athena”.

In fact, the timing of the release of Microsoft's AI chip project was purely accidental and not planned.

At the beginning of this year, OpenAI and Microsoft made a qualitative leap in LLM training. People familiar with the matter said that the release of ChatGPT detonated the Internet, and because of this, Microsoft hastily accelerated the launch of "Athena".

You know, ChatGPT has more than 100 million users.

It is predicted that Microsoft may apply "Athena" on a large scale as early as next year, including within Microsoft and OpenAI. What they are still hesitating about is whether to provide "Athena" to customers of Azure cloud computing services.

Microsoft stated that most customers do not need to train their own LLM, so they do not need chip blessing.

However, if you really want to give it, Microsoft will have to do better than Nvidia. After all, NVIDIA's chips have been hard-working for fifteen years and are best-selling to all developers.

Dylan Patel, chief analyst at research firm SemiAnalysis, said that the operating cost of ChatGPT is about US$700,000 per day, which is about 0.36 cents per query. The main source of these costs is the server, which is too expensive.

But if OpenAI uses its own developed "Athena" and can beat Nvidia, then the cost can be reduced by one-third.

He further stated that Microsoft also hopes to use LLM in all its applications in the future, such as Bing, Office 365, GitHub, etc. If you continue to use current hardware for deployment, it will cost tens of billions of dollars every year.

#In addition, Microsoft’s self-developed chips have another data support.

Microsoft executives predict how many graphics processing units will be needed to support AI work. One of the people familiar with the matter said that relying entirely on Nvidia chips would be prohibitively expensive.

Although Microsoft and NVIDIA just reached a cooperation project to build supercomputer last year, in terms of chips, "Athena" still competes with NVIDIA's products.

However, Nvidia declined to comment on this sensitive topic.

The above is the detailed content of 30,000 A100s are too expensive, and 300 people at Microsoft secretly developed their own AI chips for 5 years! TSMC 5nm, ChatGPT saves about 30%. For more information, please follow other related articles on the PHP Chinese website!