Table of Contents
The realization of the universality of the humanoid double-arm system
Multifunctional cooperative operation control subframe
△ Highly intersecting work spaces Stacked humanoid dual-arm robot
△Sampling algorithm based on active learning
Cooperative grasping and self-collision avoidance are all excellent
△Comparative analysis of smoothness
Summary:
Home Technology peripherals AI Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

Mar 26, 2024 pm 05:41 PM
data Model

New breakthrough in domestic robots:

Two independent robotic arms can already cooperate smoothly!

If you don’t believe it, look at thisTwist the bottle cap with both hands:

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

After twisting, pick up the cup and pour water

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people is as vivid as a real person.

Hey, you can also help the programmer

install the display

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real peopleEven took over The

big box in the hands of the "colleague":


It can be said that objects with various geometric and physical properties can be stable Hold on.

(What else can I do next, I dare not think about it)

This is

Tencent Robotics X Laboratory The latest achievement: Universal Double-arm cooperative dexterous operation framework. It has been published in the ## "IEEE Transactions on Robotics", the top magazine in the field of robotics.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

Since it is work, Anti-interferenceThe ability must be:

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

Since it is done with both hands, the two arms cannot be "stirred" together:

The following "360° chair flip operation" that is not clear is not obvious, it shows its fully autonomous " The ability to avoid self-collision with both arms.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

Finally, if it occasionally encounters naughty humans, it can easily get it, complete intent recognition and dynamic interaction:

(Give me, give me, hey, forget it, I’ll stop)

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

The specific technical details are revealed~

The realization of the universality of the humanoid double-arm system

Universal humanoid robots are undoubtedly the latest breakthrough between robots and AI in recent years. Research hotspots in cross-cutting fields.

The humanoid dual-arm system directly carries the execution capabilities of this type of robot operation tasks.

Randomly grabbing and manipulating any object with various geometric and physical properties is the technical embodiment of the generalization of this type of system.

Most of the existing research work focuses on solving problems at a specific level, such as environment-object perception, reasoning and strategy generation, and planning or operation control of robot systems.

And the solution is usually strongly related to the specific operated object or task, making it difficult to transfer and generalize.

Tencent Robotics

First

, how to achieve optimal grasping of unknown objects with two arms based on visual perception.

Secondly, how to ensure the system safety of dual-arm robots with highly overlapping work spaces during dynamic collaboration.

Finally

, how to transfer rich human skills to robotic systems with only limited tracking interface types.

Here, Tencent Robotics Sub-framework uses an end-to-end evaluation network and probabilistic modeling of robot accessibility to achieve optimal collaborative grasping of unmodelled objects.

2. Optimization-based Multi-functional control sub-framework adopts a hierarchical multi-priority optimization framework and embeds the lightweight distance surrogate function generated based on learning and the Riemannian manifold speed level tracking control technology.

At the same time, it achieves high-precision self-collision avoidance of both arms and highly realistic double-arm operation ellipsoid tracking, ensuring the intrinsic safety of the dual-arm system and opening up the operation ellipsoid tracking interface. In addition, this research also systematically solved the dual-arm collaborative optimal grasping of unknown objects based on visual perception for the first time, achieving the intrinsic safety of a dual-arm robot with highly overlapping work spaces and opening up human-machine skills. Transfer learning ellipsoid tracking interface for dual-arm cooperative operation.

In the end, the system provides rich interfaces in terms of perception layer, two-hand grasping, collaborative operation planning and underlying control, and has high versatility, scalability and compatibility. Technical details are as follows:

Dexterous reachable sensing subframe

Revolving around the smart reachable sensing subframework, in order to realize the two-arm collaborative grasping of unknown objects, it is necessary to The dual-arm system autonomously generates executable dexterous optimal grasping pairs.

Starting from three levels: data set generation, grasping quality assessment network and probabilistic modeling of dual-arm robot reachability, the researchers systematically designed a dexterous-reachable sensing dual-arm grasping sub-framework to realize the vision system End-to-end mapping from perceived object 3D point clouds to dexterous-optimal two-arm 6D grasping pose pairs.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

1. Two-arm dexterity sensing and grasping data set

In order to establish an accurate input point cloud and output two-arm grasping To map the relationship between masses and effectively train the grasp evaluation network, the researchers proposed the industry's first large-scale dexterity-aware dual-arm grasp synthetic dataset (DA2 Dataset).

This data set contains Mesh of 6327 objects and more than 9 million groups of grasping pairs. Each group of grasping is measured using three indicators: grasping singularity - grasping stability - grasping force compatibility. marked.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

2. Double-arm cooperative grasping quality assessment network

In actual scenarios, it is generally Technologies such as heuristic sampling can be used to generate candidate grasp pairs on the perceived 3D point cloud of the object, in order to evaluate the quality of these candidate grasp pairs online in real time and output the optimal solution.

The researchers designed the quality assessment network Dual-PointNetGPD for dual-arm cooperative grasping and used the DA2 data set for training.

Compared with traditional analysis methods based on simplified contact force models and relying on precise object geometric features, the learning-based Dual-PointNetGPD can directly process noisy 3D point clouds and construct candidate double-arm grasping structures. The dexterity and reliability of the model are evaluated.

Thus it has higher robustness to widespread visual perception noise.

3. Dual-arm reachability modeling and optimal grasp allocation

Dual-PointNetGPD outputs the grasp pair quality evaluation structure from the perspective of visual perception, aiming at For a specific dual-arm robot system, it is also necessary to consider the accessibility of candidate grasping pairs in the robot's operating space and the distribution of grasping pairs between the arms from the perspective of the robot body.

The researchers used the unit quaternion Gaussian mixture model modeling technology to model the 6-degree-of-freedom workspace of both arms and designed a probability maximization allocation strategy to achieve the reachability and grasping of both arms. Take the integration and unification of dexterity assessment.

Multifunctional cooperative operation control subframe

For the multifunctional cooperative operation control subframe, having the ability to cooperatively grasp unknown objects is the first step to achieve universalization of both arms, and having body safety awareness Cooperative operation motion planning is the second step to connect universal grasping and achieve universal operations.

Since the working spaces of the two manipulator arms of the humanoid dual-arm robot system are often highly overlapping, For the robot body that performs dual-arm dynamic tasks, self-collision avoidance needs to be placed at the highest level in motion planning. Priority , while improving the accuracy of collision avoidance, it is also necessary to reduce the online calculation cost to ensure real-time collision avoidance.

In addition, as one of the important operating performance indicators and robot skill representations, the dual-arm collaborative operation ellipsoid needs to be continuously controlled to achieve real-time optimization of the robotic arm configuration, thereby improving the operation of the dual-arm system Dexterity to avoid mission failures caused by operational singularities.

The researchers designed a new multi-functional collaborative operation control sub-framework for this purpose.

This framework is based on a hierarchical multi-objective multi-priority optimization framework, which embeds the learned lightweight two-arm minimum distance surrogate function and Riemannian manifold velocity tracking task into different priorities in the form of inequality constraints. middle.

This achieves the intrinsic safety of dual-arm operation motion planning and the high fidelity of dual-arm configuration optimization in human-machine skill transfer learning.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

1. Avoid self-collision with both arms with high precision

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people
△ Highly intersecting work spaces Stacked humanoid dual-arm robot

Obtaining an accurate and reliable minimum distance function describing the collision boundary of both arms is a key step to embed collision avoidance constraints into the multi-priority optimization framework and achieve self-collision avoidance for both arms.

Due to the high dimensionality of the dual-arm configuration space and the overlapping work spaces, the distribution of the safety subspace and collision subspace is seriously unbalanced and difficult to separate.

Researchers propose a novel active learning-based sampling method to generate large-scale, balanced, and information-rich dual-arm collision data sets.

Then use machine learning to fit the mapping relationship between the minimum distance of the two arms and the configuration of the two arms, thereby efficiently obtaining a high-precision minimum distance surrogate function.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people
△Sampling algorithm based on active learning

2. Ellipsoid tracking of two-arm collaborative operation based on Riemannian geometry

In addition to self-collision avoidance, operating ellipsoid tracking is also a key issue that needs to be addressed in dual-arm system control because it is strongly related to operating skills and operating configurations.

The dual-arm operation degree ellipsoid is described by a symmetric positive definite matrix (SPD). The researchers realized the dual-arm operation degree ellipsoid by designing the velocity feedforward term in the manifold space and combining it with the error control principle. Real-time tracking and global exponential convergence of tracking error, and complete theoretical analysis and proof are given.

The dual-arm operation degree ellipsoid tracking task is embedded in the multi-priority optimization framework in the form of equality constraints and presented in the form of a dual-arm operation degree ellipsoid tracking interface.

3. Multi-priority optimization and controller design

The researchers use collaborative task space variables (absolute movement of both arms and relative movement of both arms) to describe all Dual-arm collaborative-non-cooperative/synchronous-asynchronous tasks, and a system framework for universal collaborative operation control of both arms is designed based on the multi-objective and multi-priority framework.

This framework organically integrates the above-mentioned multi-tasks such as self-collision avoidance of both arms and ellipsoid tracking of both arms operation, thereby clearing the way for universal use of both arms at the level of motion planning and control.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

Cooperative grasping and self-collision avoidance are all excellent

To verify the optimal collaborative grasping of both arms, self-collision avoidance and self-collision avoidance of both arms and double-arm coordination To operate the effectiveness of ellipsoid tracking, the researchers conducted a quantitative analysis on the designed Dual-PointNetGPD network, the learned-generated minimum distance surrogate function for both arms, and the SPD manifold tracking error.

1, Optimal cooperative grasping of both arms Researchers carried out large-scale ablation experiments in the physical simulation engine, and the results showed that the designed and trained Dual-PointNetGPD network was effective Improved the success rate of two-arm cooperative grasping in large-size unmodeled object grasping tasks.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

#2. Self-collision avoidance with both arms
Mainly generated from the two-dimensional collision data set of both arms and the minimum distance proxy function of both arms. Analysis, considering that the double-arm configuration space is 14 dimensions, low-precision sampling is performed with a sampling step of 20-30°, and each dimension is divided into 10 equal parts, the required data set size also reaches 1014.

The researchers used sampling technology based on active learning to generate a double-arm collision data set. With a data scale of 2 million, a high prediction accuracy of more than 95% can be achieved. Compared with the current learning in unbalanced data sets, It has higher data efficiency and prediction accuracy than technologies such as b-SMOTE that are widely used in China.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

△Analysis of average prediction error and prediction accuracy

In addition, the minimum distance between both arms proxy function generated based on learning trainingcalculates the minimum distance between both arms once The time is about 0.072ms, which is significantly lower than the 5.36ms required by the widely used FCL algorithm library, and the variance is smaller.

This greatly improves the real-time performance of the method in actual deployment and ensures the stable and predictable motion planning of the entire two-arm collaborative movement.

In addition, the surrogate function exhibits excellent continuity and differentiability, which is very important for embedding self-collision avoidance into the smooth multi-objective optimization framework in the form of inequality constraints.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

△Comparative analysis of table calculation time

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people
△Comparative analysis of smoothness

3. Ellipsoid tracking for coordinated operation of both arms

The ablation experiment verified that the operation ellipsoid tracking constraints designed and embedded in the multi-objective optimization framework effectively reduced the actual configuration of both arms. The difference from the expected configuration verifies the effectiveness of the SPD manifold tracking interface.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

4. Double-arm cooperative flipping experiment:

In order to verify the ability to autonomously avoid self-collision, the researchers designed a device that required extreme torsion in a collaborative flipping experiment.

In this experiment, the two-arm system autonomously completed the intertwining and collision-free trajectory motion planning, and realized the 360° flipping operation of the chair without changing the grip configuration.

The absolute motion of the object always maintains high-precision tracking in the rotation direction, and dynamically and autonomously relaxes in other degrees of freedom. The relative motion error of the arms is less than 0.003 (m or rad), and the collision avoidance safety distance is Efficiently compressed to 2.5 cm.

At the same time, the typical calculation time of the entire multi-functional cooperative operation control sub-frame using the high-precision grid model of the robotic arm is 8.318 ms, which solves the dilemma of high precision and high real-time performance not being achieved at the same time.

The above results also show that tasks at all levels are executed strictly according to the given priorities.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

In the long-horizon rearrangement experiment, the two-arm robot system continuously grasped and manipulated three unmodeled objects, namely desktop bookshelf, storage basket and monitor stand, which are common in human living environments.

The experimental results show that the two-arm robot can collaboratively grasp large-sized objects under the designed universal framework and can fully autonomously avoid self-collision with high precision during the execution of complex collaborative operation tasks.

Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people

Summary:

It can be seen that this research systematically solved the dual-arm collaborative optimal grasping of unknown objects based on visual perception for the first time, achieving The dual-arm robot with highly overlapping workspaces is intrinsically safe and opens an ellipsoid tracking interface for dual-arm collaborative operation that can be used for human-machine skill transfer learning.

This top-down study of embodied intelligence and bottom-up study of robot operation planning and control will help bridge the technical gap between AI and Robotics and improve operation strategies based on AI generation. The stability and reliability of execution in the physical world accelerate the arrival of fully autonomous generalization of dual-arm systems.

Paper address: https://ieeexplore.ieee.org/document/10449470

The above is the detailed content of Tencent Robot Research Tops the Issue! It can help programmers install monitors and work together like real people. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Open source! Beyond ZoeDepth! DepthFM: Fast and accurate monocular depth estimation! Open source! Beyond ZoeDepth! DepthFM: Fast and accurate monocular depth estimation! Apr 03, 2024 pm 12:04 PM

0.What does this article do? We propose DepthFM: a versatile and fast state-of-the-art generative monocular depth estimation model. In addition to traditional depth estimation tasks, DepthFM also demonstrates state-of-the-art capabilities in downstream tasks such as depth inpainting. DepthFM is efficient and can synthesize depth maps within a few inference steps. Let’s read about this work together ~ 1. Paper information title: DepthFM: FastMonocularDepthEstimationwithFlowMatching Author: MingGui, JohannesS.Fischer, UlrichPrestel, PingchuanMa, Dmytr

The world's most powerful open source MoE model is here, with Chinese capabilities comparable to GPT-4, and the price is only nearly one percent of GPT-4-Turbo The world's most powerful open source MoE model is here, with Chinese capabilities comparable to GPT-4, and the price is only nearly one percent of GPT-4-Turbo May 07, 2024 pm 04:13 PM

Imagine an artificial intelligence model that not only has the ability to surpass traditional computing, but also achieves more efficient performance at a lower cost. This is not science fiction, DeepSeek-V2[1], the world’s most powerful open source MoE model is here. DeepSeek-V2 is a powerful mixture of experts (MoE) language model with the characteristics of economical training and efficient inference. It consists of 236B parameters, 21B of which are used to activate each marker. Compared with DeepSeek67B, DeepSeek-V2 has stronger performance, while saving 42.5% of training costs, reducing KV cache by 93.3%, and increasing the maximum generation throughput to 5.76 times. DeepSeek is a company exploring general artificial intelligence

KAN, which replaces MLP, has been extended to convolution by open source projects KAN, which replaces MLP, has been extended to convolution by open source projects Jun 01, 2024 pm 10:03 PM

Earlier this month, researchers from MIT and other institutions proposed a very promising alternative to MLP - KAN. KAN outperforms MLP in terms of accuracy and interpretability. And it can outperform MLP running with a larger number of parameters with a very small number of parameters. For example, the authors stated that they used KAN to reproduce DeepMind's results with a smaller network and a higher degree of automation. Specifically, DeepMind's MLP has about 300,000 parameters, while KAN only has about 200 parameters. KAN has a strong mathematical foundation like MLP. MLP is based on the universal approximation theorem, while KAN is based on the Kolmogorov-Arnold representation theorem. As shown in the figure below, KAN has

Hello, electric Atlas! Boston Dynamics robot comes back to life, 180-degree weird moves scare Musk Hello, electric Atlas! Boston Dynamics robot comes back to life, 180-degree weird moves scare Musk Apr 18, 2024 pm 07:58 PM

Boston Dynamics Atlas officially enters the era of electric robots! Yesterday, the hydraulic Atlas just "tearfully" withdrew from the stage of history. Today, Boston Dynamics announced that the electric Atlas is on the job. It seems that in the field of commercial humanoid robots, Boston Dynamics is determined to compete with Tesla. After the new video was released, it had already been viewed by more than one million people in just ten hours. The old people leave and new roles appear. This is a historical necessity. There is no doubt that this year is the explosive year of humanoid robots. Netizens commented: The advancement of robots has made this year's opening ceremony look like a human, and the degree of freedom is far greater than that of humans. But is this really not a horror movie? At the beginning of the video, Atlas is lying calmly on the ground, seemingly on his back. What follows is jaw-dropping

The vitality of super intelligence awakens! But with the arrival of self-updating AI, mothers no longer have to worry about data bottlenecks The vitality of super intelligence awakens! But with the arrival of self-updating AI, mothers no longer have to worry about data bottlenecks Apr 29, 2024 pm 06:55 PM

I cry to death. The world is madly building big models. The data on the Internet is not enough. It is not enough at all. The training model looks like "The Hunger Games", and AI researchers around the world are worrying about how to feed these data voracious eaters. This problem is particularly prominent in multi-modal tasks. At a time when nothing could be done, a start-up team from the Department of Renmin University of China used its own new model to become the first in China to make "model-generated data feed itself" a reality. Moreover, it is a two-pronged approach on the understanding side and the generation side. Both sides can generate high-quality, multi-modal new data and provide data feedback to the model itself. What is a model? Awaker 1.0, a large multi-modal model that just appeared on the Zhongguancun Forum. Who is the team? Sophon engine. Founded by Gao Yizhao, a doctoral student at Renmin University’s Hillhouse School of Artificial Intelligence.

AI subverts mathematical research! Fields Medal winner and Chinese-American mathematician led 11 top-ranked papers | Liked by Terence Tao AI subverts mathematical research! Fields Medal winner and Chinese-American mathematician led 11 top-ranked papers | Liked by Terence Tao Apr 09, 2024 am 11:52 AM

AI is indeed changing mathematics. Recently, Tao Zhexuan, who has been paying close attention to this issue, forwarded the latest issue of "Bulletin of the American Mathematical Society" (Bulletin of the American Mathematical Society). Focusing on the topic "Will machines change mathematics?", many mathematicians expressed their opinions. The whole process was full of sparks, hardcore and exciting. The author has a strong lineup, including Fields Medal winner Akshay Venkatesh, Chinese mathematician Zheng Lejun, NYU computer scientist Ernest Davis and many other well-known scholars in the industry. The world of AI has changed dramatically. You know, many of these articles were submitted a year ago.

Slow Cellular Data Internet Speeds on iPhone: Fixes Slow Cellular Data Internet Speeds on iPhone: Fixes May 03, 2024 pm 09:01 PM

Facing lag, slow mobile data connection on iPhone? Typically, the strength of cellular internet on your phone depends on several factors such as region, cellular network type, roaming type, etc. There are some things you can do to get a faster, more reliable cellular Internet connection. Fix 1 – Force Restart iPhone Sometimes, force restarting your device just resets a lot of things, including the cellular connection. Step 1 – Just press the volume up key once and release. Next, press the Volume Down key and release it again. Step 2 – The next part of the process is to hold the button on the right side. Let the iPhone finish restarting. Enable cellular data and check network speed. Check again Fix 2 – Change data mode While 5G offers better network speeds, it works better when the signal is weaker

The U.S. Air Force showcases its first AI fighter jet with high profile! The minister personally conducted the test drive without interfering during the whole process, and 100,000 lines of code were tested for 21 times. The U.S. Air Force showcases its first AI fighter jet with high profile! The minister personally conducted the test drive without interfering during the whole process, and 100,000 lines of code were tested for 21 times. May 07, 2024 pm 05:00 PM

Recently, the military circle has been overwhelmed by the news: US military fighter jets can now complete fully automatic air combat using AI. Yes, just recently, the US military’s AI fighter jet was made public for the first time and the mystery was unveiled. The full name of this fighter is the Variable Stability Simulator Test Aircraft (VISTA). It was personally flown by the Secretary of the US Air Force to simulate a one-on-one air battle. On May 2, U.S. Air Force Secretary Frank Kendall took off in an X-62AVISTA at Edwards Air Force Base. Note that during the one-hour flight, all flight actions were completed autonomously by AI! Kendall said - "For the past few decades, we have been thinking about the unlimited potential of autonomous air-to-air combat, but it has always seemed out of reach." However now,

See all articles