Three OpenAI leaders personally wrote this article: How should we govern superintelligence?

AI has never affected human life so extensively and brought so many worries and troubles to human beings as it does today.

Like all other major technological innovations in the past, the development of AI also has two sides, one facing good and the other facing evil. This is also the beginning of regulatory agencies around the world. One of the important reasons for active intervention.

In any case, the development of AI technology seems to be a general trend and unstoppable. How to make AI comply with human values ​​and ethics? More secure development and deployment is an issue that all current AI practitioners need to seriously consider.

Today, OpenAI’s three co-founders—CEO Sam Altman, President Greg Brockman, and Chief Scientist Ilya Sutskever Co-authored an article discussing how to govern superintelligence. They argue that now is a good time to start thinking about superintelligent governance—future artificial intelligence systems that are even more capable than AGI.

Based on what we have seen so far, it is conceivable that within the next ten years, artificial intelligence (AI) will surpass humans in most fields Expert skill level and can carry out as many production activities as one of the largest companies today.

#Regardless of the potential benefits and drawbacks, superintelligence will be more powerful than other technologies that humans have had to face in the past. We can have an incredibly prosperous future; but we must manage risks to get there. Considering the possibility of existential risk, we cannot just respond passively. Nuclear power is one example of a technology with this property; another is synthetic biology.

Likewise, we must mitigate the risks of current AI technologies, but superintelligence will require special handling and coordination. A Starting Point

There are many important ideas in our opportunity to successfully lead this development; here we Initial reflections were given on three of these ideas.

First, we need some coordination among leading development efforts to ensure that superintelligence is developed in a way that both keeps us safe and can help these systems integrate smoothly with society. There are many ways to achieve this; the world's major governments could come together to establish a project that many current efforts would be part of, or we could do so through "collective consent" (with A new organizational support force, as suggested below), would limit the growth of cutting-edge AI capabilities to a certain rate each year.

Of course, individual companies should also be held to extremely high standards to develop responsibly.

Secondly, we may eventually need something similar to the International Atomic Energy Agency (IAEA) to regulate the development of superintelligence; Any effort that exceeds a certain threshold of capabilities (or computing and other resources) needs to be overseen by this international body, which can inspect the system, require audits, test for compliance with security standards, impose limits on the extent of deployment and security levels, etc. Tracking compute and energy usage would be of great help and give us some hope that this idea is actually achievable. As a first step, companies could voluntarily agree to begin implementing elements that such a body might one day require, while as a second step, individual countries could implement it. Importantly, such an agency should focus on reducing “existential risks” rather than issues that should be left to individual countries, such as defining what AI should be allowed to say.

Third, We need to have enough technical capabilities to make super intelligence safe. This is an open research question into which we and others are investing considerable effort.

Out of scope

We believe it is important to allow companies and open source projects to develop models below significant capability thresholds , without requiring the kind of regulation we describe here (including cumbersome mechanisms such as licensing or auditing).

Today’s systems will create tremendous value for the world, and while they do have risks, those risk levels appear to be comparable to other Internet technologies, and society’s response The approach seemed appropriate.

By contrast, the systems we are focusing on will have power beyond that of any technology to date, and we should be careful not to overwhelm our efforts by applying similar standards to far lower technology at this threshold to downplay attention to them.

Public Participation and Potential

But for,the governance of the most robust systems, and the issues surrounding their deployment decision, there must be strong public oversight. We believe that people around the world should democratically determine the boundaries and defaults of AI systems. We don't yet know how to design such a mechanism, but we plan to do some experiments. We continue to believe that, within these broad boundaries, individual users should have significant control over how they behave with AI.

#Given the risks and difficulties, it’s worth thinking about why we’re building this technology.

#At OpenAI, we have two basic reasons. First, we believe it will lead to a better world than we can imagine today (we are already seeing early examples of this in areas such as education, creative work, and personal productivity). The world faces a lot of problems, and we need more help solving them; this technology can improve our society, and we're sure to be surprised by everyone's creativity using these new tools. The economic growth and improvement in quality of life will be staggering.

Second, we believe that stopping building superintelligence would be a decision that is unintuitively risky and fraught with difficulty. Due to its huge advantages, the cost of building superintelligence is getting lower every year, the number of builders is increasing rapidly, and it is essentially part of the technological path we are on, stopping it will require something like A global surveillance regime exists, and even such a regime is no guarantee of success. Therefore, we must get it right.

The above is the detailed content of Three OpenAI leaders personally wrote this article: How should we govern superintelligence?. For more information, please follow other related articles on the PHP Chinese website!