


Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM
Multi-task robot learning is of great significance in dealing with diverse and complex scenarios. However, current methods are limited by performance issues and difficulties in collecting training datasets.
This paper proposes GeRM (General Robot Model), where researchers use offline reinforcement learning to optimize data utilization strategies, learning from demonstrations and sub-optimal data, thereby surpassing human demonstrations limitations.
Authors: Song Wenxuan, Zhao Han, Ding Pengxiang, Cui Can, Lu Shangke, Fan Yaning, Wang Donglin
Unit: West Lake University, Zhejiang University
Paper address: https://arxiv.org/abs/2403.13358
Project address: https://songwxuan.github.io/GeRM/
Then a Transformer-based vision-language-action model is used to process multi-modal input and output actions.
By introducing an expert hybrid structure, GeRM achieves faster inference speed and higher overall model capacity, thus solving the problem of limited reinforcement learning parameters and improving multi-task performance. Model performance during learning while controlling computational cost.
Through a series of experiments, it is proven that GeRM outperforms other methods in all tasks, while verifying its efficiency in the training and inference processes.
In addition, the researchers also provided the QUARD-Auto data set to support training. The construction of this data set follows the new paradigm of data automation collection proposed in the article. This method can reduce the number of collection robots. The cost of data drives progress in the multi-task learning community.
Main contributions:
#1. Proposed a hybrid expert model for four-legged reinforcement learning for the first time. Train on mixed-quality data with the potential to learn optimal policies.
2. Compared with existing methods, GeRM shows a higher success rate when only activating 1/2 of its own parameters, activating the emergence ability, and at the same time during the training process A better data utilization strategy is demonstrated in .
3. Proposed a paradigm for fully automatic robot data set collection, and collected a large-scale open source data set.
Method
The GeRM network structure is shown in Figure 1. The visual-linguistic input including demonstration data and failure data is input to 8 after passing through the encoder and tokenizer respectively. The decoder uses a layer of mixed expert structure to generate action tokens, which are eventually converted into discrete robot action data and deployed to the robot through the underlying strategy. In addition, we use reinforcement learning for training.
Figure 1 GeRM network structure diagram
GeRM Decoder is an architecture model including Transformer Decoder, in which A feedforward network (FFN) was selected from a set of 8 different expert networks.
At each layer, for each token, the gating network selects two experts to process the token and combine their outputs in a weighted manner.
Different experts are good at different tasks/different action dimensions to solve problems in different scenarios, thereby learning a common model across multiple tasks. This architecture expands the amount of network parameters while keeping the computational cost essentially unchanged.
Figure 2 Decoder structure diagram
We propose an automatic paradigm to collect robot multi-mode status data. In this way, we constructed QUARD-Auto, a large-scale robotics dataset containing a combination of demonstration and suboptimal data. It includes 5 tasks and 99 subtasks, with a total of 257k trajectories. We will open source to promote the development of the robotics community.
Table 1 Introduction to the data set
Figure 3 Data Volume statistics
Experiments
#We conducted a comprehensive and robust series of experiments covering all 99 subtasks, each of which was carefully tested on 400 trajectories.
As shown in Table 1, GeRM has the highest success rate among all tasks. Compared with RT-1 and other variants of GeRM, it effectively learns from mixed-quality data, outperforms other methods, and exhibits superior capabilities in multiple tasks. At the same time, the MoE module balances computational cost and performance by activating some parameters during inference.
Table 2 Multi-task comparison experiment
GeRM shows commendable training efficiency. Compared with other methods, GeRM achieves extremely low loss and high success rate with only a few batches, highlighting GeRM's ability to optimize data utilization strategies.
Figure 4 Success rate/Loss change curve
GeRM demonstrates dynamic adaptive path planning emergent ability. As shown in the video, the quadruped robot has a limited field of view in the initial position, making it difficult to determine the direction of movement. To avoid the obstacle, it randomly chooses to turn left.
Subsequently, after encountering erroneous visual input, the robot performed a substantial reorientation to align with the correct target outside the original field of view. It then continues toward its destination, ultimately completing its mission.
It is worth noting that such trajectories do not belong to the distribution of our training data set. This demonstrates GeRM's emergent capabilities for dynamic adaptive path planning in the context of a scene, i.e., its ability to make decisions based on visual perception, plan future paths, and change next steps as needed.
Figure 5 Emergent Capability
The above is the detailed content of Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM. For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics



When converting strings to objects in Vue.js, JSON.parse() is preferred for standard JSON strings. For non-standard JSON strings, the string can be processed by using regular expressions and reduce methods according to the format or decoded URL-encoded. Select the appropriate method according to the string format and pay attention to security and encoding issues to avoid bugs.

Pagination is a technology that splits large data sets into small pages to improve performance and user experience. In Vue, you can use the following built-in method to paging: Calculate the total number of pages: totalPages() traversal page number: v-for directive to set the current page: currentPage Get the current page data: currentPageData()

The article introduces the operation of MySQL database. First, you need to install a MySQL client, such as MySQLWorkbench or command line client. 1. Use the mysql-uroot-p command to connect to the server and log in with the root account password; 2. Use CREATEDATABASE to create a database, and USE select a database; 3. Use CREATETABLE to create a table, define fields and data types; 4. Use INSERTINTO to insert data, query data, update data by UPDATE, and delete data by DELETE. Only by mastering these steps, learning to deal with common problems and optimizing database performance can you use MySQL efficiently.

There are many reasons why MySQL startup fails, and it can be diagnosed by checking the error log. Common causes include port conflicts (check port occupancy and modify configuration), permission issues (check service running user permissions), configuration file errors (check parameter settings), data directory corruption (restore data or rebuild table space), InnoDB table space issues (check ibdata1 files), plug-in loading failure (check error log). When solving problems, you should analyze them based on the error log, find the root cause of the problem, and develop the habit of backing up data regularly to prevent and solve problems.

Git and GitHub are not the same thing. Git is a version control system, and GitHub is a Git-based code hosting platform. Git is used to manage code versions, and GitHub provides an online collaboration environment.

Efficiently process 7 million records and create interactive maps with geospatial technology. This article explores how to efficiently process over 7 million records using Laravel and MySQL and convert them into interactive map visualizations. Initial challenge project requirements: Extract valuable insights using 7 million records in MySQL database. Many people first consider programming languages, but ignore the database itself: Can it meet the needs? Is data migration or structural adjustment required? Can MySQL withstand such a large data load? Preliminary analysis: Key filters and properties need to be identified. After analysis, it was found that only a few attributes were related to the solution. We verified the feasibility of the filter and set some restrictions to optimize the search. Map search based on city

In order to set the timeout for Vue Axios, we can create an Axios instance and specify the timeout option: In global settings: Vue.prototype.$axios = axios.create({ timeout: 5000 }); in a single request: this.$axios.get('/api/users', { timeout: 10000 }).

MySQL performance optimization needs to start from three aspects: installation configuration, indexing and query optimization, monitoring and tuning. 1. After installation, you need to adjust the my.cnf file according to the server configuration, such as the innodb_buffer_pool_size parameter, and close query_cache_size; 2. Create a suitable index to avoid excessive indexes, and optimize query statements, such as using the EXPLAIN command to analyze the execution plan; 3. Use MySQL's own monitoring tool (SHOWPROCESSLIST, SHOWSTATUS) to monitor the database health, and regularly back up and organize the database. Only by continuously optimizing these steps can the performance of MySQL database be improved.
