Have you ever wanted to tell a robot what to do with your own words, like you would a human?
For example, just tell your home assistant robot "Please heat up my lunch" and it will find the microwave on its own. Amazing, right?
Although language is the most intuitive way for humans to express their intentions, for a long time, people still relied heavily on handwritten code to implement robots. control. However, when ChatGPT appears, this situation will change.
In a recent study, a Microsoft team is exploring how to use OpenAI’s new AI language model ChatGPT to make natural human-computer interaction possible.
Paper link: https://www.microsoft.com/en-us/research/uploads/prod/ 2023/02/ChatGPT___Robotics.pdf
ChatGPT is a language model trained on a large corpus of text and human interactions, so it can generate coherent and grammatical responses to a wide range of prompts and questions. Correct response. The goal of this research is to see if ChatGPT can think beyond text and reason about the real world to help robots complete tasks. Researchers hope this will make it easier for people to interact with robots without having to learn complex programming languages or the details of robotic systems.
The key challenge of the research is to teach ChatGPT how to solve problems by taking into account the laws of physics, the operating environment, and the way the robot uses body movements to change its surroundings.
It turns out that ChatGPT can do a lot on its own, but it still needs some help. In the paper, the team describes a series of design principles that can be used to guide language models in solving robotic tasks, including (but not limited to) ad hoc prompt structures, high-level APIs, and feedback via human text. The researchers believe this work is just the beginning of a transformation in developing robotic systems and hope this study will inspire other researchers to join this interesting research area.
The current robot operation process is from Starting with an engineer or technical user, they are required to translate task requirements into system code. Engineers will be in the loop of the workflow, constantly writing new code and specifications to correct the robot's behavior. Overall, the process is slow (users need to write low-level code), expensive (requires highly skilled users with in-depth knowledge of robotics), and inefficient (requires multiple interactions to function properly).
But ChatGPT opens up a new bot paradigm and allows for potentially non-technical Users participate in the loop, monitoring the robot's performance while providing high-level feedback to the large language model (LLM). By following the researched design principles, ChatGPT can generate code for robotic scenarios. Without any fine-tuning, the study exploits the knowledge of LLM to control different robot shapes for various tasks. In their work, the researchers demonstrated multiple examples of ChatGPT solving robotic challenges, as well as complex robot deployments in the operational, aerial, and navigation domains.
Prompting LLM is a highly empirical science. The research established a set of methods and design principles for writing prompts for robot tasks through trial and error:
If the user is satisfied with the solution, the code can finally be deployed to the robot.
Let’s look at a few examples, you can find more case studies in the code repository.
Researchers let ChatGPT control real drones, and it proved to be the most effective among non-technical users and robots A very intuitive language-based interface. When user instructions are ambiguous, ChatGPT asks clarifying questions and writes complex code structures for the drone to visually inspect the architecture, such as zigzag patterns. It even learned to take selfies!
Researchers ChatGPT was also used in a simulated industrial inspection scenario using the Microsoft AirSim simulator. The model is able to effectively parse the user's high-level intentions and geometric cues to accurately control the drone.
ChatGPT-AirSim interface
The release of these technologies is something worth celebrating, Because this will expand the audience for robotics. Microsoft researchers believe that language-based robot control will lay the foundation for bringing robots from scientific laboratories into the lives of everyday users.
This article would like to emphasize that the output of ChatGPT is not meant to be deployed directly on a robot without careful analysis. The researchers encourage users to harness the power of simulation in order to evaluate these algorithms before potential real-life deployment and always take necessary safety precautions. The work described in this article represents only a small portion of what is possible at the intersection of large-scale language models operating in the field of robotics, and hopefully provides inspiration for more research.
Original link: https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/
The above is the detailed content of While I was still chatting with ChatGPT, someone had already started using it to control the robot's work.. For more information, please follow other related articles on the PHP Chinese website!