Table of Contents
Method
Home Technology peripherals AI Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Apr 17, 2024 pm 11:40 PM
git ai Model

Multi-task robot learning is of great significance in dealing with diverse and complex scenarios. However, current methods are limited by performance issues and difficulties in collecting training datasets.

This paper proposes GeRM (General Robot Model), where researchers use offline reinforcement learning to optimize data utilization strategies, learning from demonstrations and sub-optimal data, thereby surpassing human demonstrations limitations.

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Authors: Song Wenxuan, Zhao Han, Ding Pengxiang, Cui Can, Lu Shangke, Fan Yaning, Wang Donglin

Unit: West Lake University, Zhejiang University

Paper address: https://arxiv.org/abs/2403.13358

Project address: https://songwxuan.github.io/GeRM/

Then a Transformer-based vision-language-action model is used to process multi-modal input and output actions.

By introducing an expert hybrid structure, GeRM achieves faster inference speed and higher overall model capacity, thus solving the problem of limited reinforcement learning parameters and improving multi-task performance. Model performance during learning while controlling computational cost.

Through a series of experiments, it is proven that GeRM outperforms other methods in all tasks, while verifying its efficiency in the training and inference processes.

In addition, the researchers also provided the QUARD-Auto data set to support training. The construction of this data set follows the new paradigm of data automation collection proposed in the article. This method can reduce the number of collection robots. The cost of data drives progress in the multi-task learning community.

Main contributions:

#1. Proposed a hybrid expert model for four-legged reinforcement learning for the first time. Train on mixed-quality data with the potential to learn optimal policies.

2. Compared with existing methods, GeRM shows a higher success rate when only activating 1/2 of its own parameters, activating the emergence ability, and at the same time during the training process A better data utilization strategy is demonstrated in .

3. Proposed a paradigm for fully automatic robot data set collection, and collected a large-scale open source data set.

Method

The GeRM network structure is shown in Figure 1. The visual-linguistic input including demonstration data and failure data is input to 8 after passing through the encoder and tokenizer respectively. The decoder uses a layer of mixed expert structure to generate action tokens, which are eventually converted into discrete robot action data and deployed to the robot through the underlying strategy. In addition, we use reinforcement learning for training.

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Figure 1 GeRM network structure diagram

GeRM Decoder is an architecture model including Transformer Decoder, in which A feedforward network (FFN) was selected from a set of 8 different expert networks.

At each layer, for each token, the gating network selects two experts to process the token and combine their outputs in a weighted manner.

Different experts are good at different tasks/different action dimensions to solve problems in different scenarios, thereby learning a common model across multiple tasks. This architecture expands the amount of network parameters while keeping the computational cost essentially unchanged.

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Figure 2 Decoder structure diagram

We propose an automatic paradigm to collect robot multi-mode status data. In this way, we constructed QUARD-Auto, a large-scale robotics dataset containing a combination of demonstration and suboptimal data. It includes 5 tasks and 99 subtasks, with a total of 257k trajectories. We will open source to promote the development of the robotics community.

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Table 1 Introduction to the data set

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Figure 3 Data Volume statistics

Experiments

#We conducted a comprehensive and robust series of experiments covering all 99 subtasks, each of which was carefully tested on 400 trajectories.

As shown in Table 1, GeRM has the highest success rate among all tasks. Compared with RT-1 and other variants of GeRM, it effectively learns from mixed-quality data, outperforms other methods, and exhibits superior capabilities in multiple tasks. At the same time, the MoE module balances computational cost and performance by activating some parameters during inference.

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Table 2 Multi-task comparison experiment

GeRM shows commendable training efficiency. Compared with other methods, GeRM achieves extremely low loss and high success rate with only a few batches, highlighting GeRM's ability to optimize data utilization strategies.

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Figure 4 Success rate/Loss change curve

GeRM demonstrates dynamic adaptive path planning emergent ability. As shown in the video, the quadruped robot has a limited field of view in the initial position, making it difficult to determine the direction of movement. To avoid the obstacle, it randomly chooses to turn left.

Subsequently, after encountering erroneous visual input, the robot performed a substantial reorientation to align with the correct target outside the original field of view. It then continues toward its destination, ultimately completing its mission.

It is worth noting that such trajectories do not belong to the distribution of our training data set. This demonstrates GeRM's emergent capabilities for dynamic adaptive path planning in the context of a scene, i.e., its ability to make decisions based on visual perception, plan future paths, and change next steps as needed.

Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM

Figure 5 Emergent Capability

The above is the detailed content of Sweep 99 sub-missions with MoE! Zhejiang University and others proposed a new general robot strategy GeRM. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

What method is used to convert strings into objects in Vue.js? What method is used to convert strings into objects in Vue.js? Apr 07, 2025 pm 09:39 PM

When converting strings to objects in Vue.js, JSON.parse() is preferred for standard JSON strings. For non-standard JSON strings, the string can be processed by using regular expressions and reduce methods according to the format or decoded URL-encoded. Select the appropriate method according to the string format and pay attention to security and encoding issues to avoid bugs.

How to use vue pagination How to use vue pagination Apr 08, 2025 am 06:45 AM

Pagination is a technology that splits large data sets into small pages to improve performance and user experience. In Vue, you can use the following built-in method to paging: Calculate the total number of pages: totalPages() traversal page number: v-for directive to set the current page: currentPage Get the current page data: currentPageData()

How to use mysql after installation How to use mysql after installation Apr 08, 2025 am 11:48 AM

The article introduces the operation of MySQL database. First, you need to install a MySQL client, such as MySQLWorkbench or command line client. 1. Use the mysql-uroot-p command to connect to the server and log in with the root account password; 2. Use CREATEDATABASE to create a database, and USE select a database; 3. Use CREATETABLE to create a table, define fields and data types; 4. Use INSERTINTO to insert data, query data, update data by UPDATE, and delete data by DELETE. Only by mastering these steps, learning to deal with common problems and optimizing database performance can you use MySQL efficiently.

How to solve mysql cannot be started How to solve mysql cannot be started Apr 08, 2025 pm 02:21 PM

There are many reasons why MySQL startup fails, and it can be diagnosed by checking the error log. Common causes include port conflicts (check port occupancy and modify configuration), permission issues (check service running user permissions), configuration file errors (check parameter settings), data directory corruption (restore data or rebuild table space), InnoDB table space issues (check ibdata1 files), plug-in loading failure (check error log). When solving problems, you should analyze them based on the error log, find the root cause of the problem, and develop the habit of backing up data regularly to prevent and solve problems.

Is Git the same as GitHub? Is Git the same as GitHub? Apr 08, 2025 am 12:13 AM

Git and GitHub are not the same thing. Git is a version control system, and GitHub is a Git-based code hosting platform. Git is used to manage code versions, and GitHub provides an online collaboration environment.

Laravel's geospatial: Optimization of interactive maps and large amounts of data Laravel's geospatial: Optimization of interactive maps and large amounts of data Apr 08, 2025 pm 12:24 PM

Efficiently process 7 million records and create interactive maps with geospatial technology. This article explores how to efficiently process over 7 million records using Laravel and MySQL and convert them into interactive map visualizations. Initial challenge project requirements: Extract valuable insights using 7 million records in MySQL database. Many people first consider programming languages, but ignore the database itself: Can it meet the needs? Is data migration or structural adjustment required? Can MySQL withstand such a large data load? Preliminary analysis: Key filters and properties need to be identified. After analysis, it was found that only a few attributes were related to the solution. We verified the feasibility of the filter and set some restrictions to optimize the search. Map search based on city

How to set the timeout of Vue Axios How to set the timeout of Vue Axios Apr 07, 2025 pm 10:03 PM

In order to set the timeout for Vue Axios, we can create an Axios instance and specify the timeout option: In global settings: Vue.prototype.$axios = axios.create({ timeout: 5000 }); in a single request: this.$axios.get('/api/users', { timeout: 10000 }).

How to optimize database performance after mysql installation How to optimize database performance after mysql installation Apr 08, 2025 am 11:36 AM

MySQL performance optimization needs to start from three aspects: installation configuration, indexing and query optimization, monitoring and tuning. 1. After installation, you need to adjust the my.cnf file according to the server configuration, such as the innodb_buffer_pool_size parameter, and close query_cache_size; 2. Create a suitable index to avoid excessive indexes, and optimize query statements, such as using the EXPLAIN command to analyze the execution plan; 3. Use MySQL's own monitoring tool (SHOWPROCESSLIST, SHOWSTATUS) to monitor the database health, and regularly back up and organize the database. Only by continuously optimizing these steps can the performance of MySQL database be improved.

See all articles