Can AI design chips surpass humans? Google's 'Nature' paper was 'slapped in the face' by the University of California research team

A research paper published by Google employees in the journal Nature states that the company develops artificial intelligence (AI) software that can be faster and better than humans design the chip. This conclusion has recently been questioned and criticized by researchers at the University of California, San Diego (UCSD).

As early as June 2021, Google claimed to have developed an AI chip design system based on reinforcement learning, which attracted widespread attention. At the time, the company claimed that this system could automatically generate an optimized microchip layout and had been used in the design of TPU chips independently developed by Google and achieved excellent performance.

Is AI designing chips faster and better than manual work? Google's "Nature" magazine paper was questioned

The layout of the chip is very important because it directly determines performance. Designers need to carefully arrange the circuit blocks in the chip so that signals and data travel between these areas at the desired rate. Engineers often spend weeks or months refining their designs, trying to find the best configuration to develop more powerful, energy-efficient, smaller chips.

Previously, chip layout was usually completed by a combination of manual and automated tools. Google's chip team is trying to prove that its AI systems can do things better and faster than human engineers.

Google employees wrote in the "Nature" magazine paper: "Despite 50 years of research, chip layout still cannot be automated, and physical design engineers require months of painstaking efforts to make a manufacturable device. Layout... And in less than 6 hours, the chip layout automatically generated by our AI system was better than or comparable to the design drawings drawn by humans in all key indicators."

This article. The paper gained attention from the electronic design automation community, who began integrating machine learning algorithms into their software suites. But a UCSD research team has cast doubt on Google's assertion that AI models can outperform humans in chip layout.

Led by UCSD computer science and engineering professor Andrew Kahn (Kahn served as a reviewer for Nature during the Google paper's peer review process), the team spent several months Google reverse-engineered the floorplan layout described in Nature. They ultimately found that through a re-creation of Google's original code (referred to in their study as circuit training), Google's approach actually performed worse than human engineers using traditional industrial methods and tools.

What causes this difference? The team noted that Google used Synopsys' EDA suite to create a starting layout of the chip's logic gates, which was then optimized by Google's reinforcement learning system.

Google points out in the paper that after the model generates the layout, it uses industry-standard software tools and manual adjustments, mainly to ensure that the processor can work as expected and ultimately complete manufacturing. Google believes that this is a necessary step regardless of whether the floor plan is created by a machine learning algorithm or by a human engineer using standard tools, so this AI model deserves credit for optimizing the final product.

However, the UCSD research team stated that the paper in Nature did not mention that EDA tools were used to prepare the model layout in advance for improvement. In other words, these tools from Synopsys may have given the AI ​​models a good enough start that the true capabilities of the AI ​​systems have been called into question.

The university team wrote of using Synopsys' suite to build the layout for the model, "This was not evident during the paper review process and was not mentioned in Nature. The experiments we conducted It shows that having initial position information can significantly improve the results of circuit training (CT).” Some academics have since urged Nature to review the Google paper based on the UCSD study. In their email to the journal, the researchers highlighted the concerns raised by Professor Kahng and his colleagues and questioned whether Google's paper was misleading.

Bill Swartz, a senior lecturer in electrical engineering at the University of Texas at Dallas, said that the Nature paper left many researchers in the dark because its findings used Google’s proprietary TPU and therefore could not verify. He said, "The collaboration using Synopsy's software to optimize Google's software needs to be investigated. We all just want to know the actual algorithm so we can replicate it. If Google's claims are correct , then we hope that if Google’s conclusion is scientific and objective, then its results must be true and effective.”

"Nature" magazine stated that it is investigating Google's paper. A spokesman for the journal said: "For confidentiality reasons we cannot comment on the details of individual cases. However, when concerns are raised about any paper published in the journal, we investigate carefully in accordance with the established process. This process includes working with The authors consult and, where appropriate, seek advice from reviewers and other external experts. Once we have sufficient information, we develop the most appropriate response to provide readers with a clear understanding of our findings."

Information shows that this is not the magazine’s first investigation into this Google research paper. In March 2022, the paper corrected the author and added links to some open source CT codes from Google for those trying to follow the research method.

Google Azalia Mirhoseini and Anna Goldie, lead authors of the paper, said the UCSD research team's experiments did not accurately implement their method. They pointed out that the results obtained by Professor Kahng's team were not ideal because they did not pre-train their model on any data at all.

The duo said in a statement, “Learning-based methods will of course perform worse without learning from previous experience. We pre-trained with 20 circuit blocks before testing the cases ”

They also pointed out that Professor Kahng’s research team also did not use the same computing power as Google to train their system, which may also have weakened the performance of its model.

Mirhosini and Goldie also said that they did not explicitly describe the use of EDA tools in their Nature paper because it was irrelevant to the overall situation and not worth mentioning. They said, "Our study focuses on the transition from physical synthesis to initial placement of clustered circuit blocks. Physical synthesis must be performed before using any placement method, which is standard practice in chip design."

However, UCSD The research team said they did not pre-train their model because they did not have access to Google's proprietary data. At the same time, they claim that the software they developed for this has been verified by two other engineers at Google, who are also co-authors of the Nature paper.

The Google paper caused internal controversy, and the fired employees said that it was to win the contract

It is worth mentioning that this article published by Google in the magazine "Nature" The paper also sparked controversy within the company.

In May last year, Google AI researcher Satrajit Chatterjee claimed to have been fired by the company without reason because he criticized the research paper and questioned its conclusions. Before his firing, Chatterjee was told by Google not to publish articles critical of the paper.

Some Google employees have rebuked him, claiming he went too far with his criticism, such as when he described the paper's seriousness as a "train wreck." He was also investigated by Google's human resources department for this critical behavior.

Chatterjee later sued Google in Santa Clara Superior Court in California, claiming he was unlawfully fired. After Chatterjee was fired, Mirhoseini and Goldie also left in mid-2022.

Chatterjee last month amended his complaint against Google, and his lawyers claimed that Google was considering commercializing its AI-based floor plan generation software with "S Company" and was negotiating a deal with S Company Cloud deal said to be worth $120 million. Chatterjee claimed that Google supported the paper primarily to help convince Company S to sign this important commercial agreement.

Chatterjee wrote in an email to Google executives: "This paper is, to a certain extent, the first step taken by Google to achieve cooperation with Company S. Since this research is in "This was done in the context of a potentially large cloud deal, and when our testing showed otherwise, it showed that it was unethical for Google to own this revolutionary technology," the email was disclosed as part of the lawsuit.

He accused Google in court documents of exaggerating its research results and deliberately concealing important information from Company S to induce it to sign a cloud deal. In fact, it used this problematic technology to attract Company S to cooperate.

S ​​Company is described in court documents as an "electronic design automation company." People familiar with the matter said Company S actually refers to Synopsys. But Synopsys and Google declined to comment.

The above is the detailed content of Can AI design chips surpass humans? Google's 'Nature' paper was 'slapped in the face' by the University of California research team. For more information, please follow other related articles on the PHP Chinese website!