Table of Contents
Experimental results:
Home Technology peripherals AI How to closely integrate online mapping and trajectory prediction?

How to closely integrate online mapping and trajectory prediction?

Apr 03, 2024 pm 12:00 PM
Autopilot trajectory

Original title: Producing and Leveraging Online Map Uncertainty in Trajectory Prediction

Paper link: https://arxiv.org/pdf/2403.16439.pdf

Code link: https ://github.com/alfredgu001324/MapUncertaintyPrediction

Author affiliation: Vector Institute NVIDIA Research, University of Toronto, Stanford University

How to closely integrate online mapping and trajectory prediction?

##Thesis idea:

High-definition (HD) maps have played an integral role in the development of the modern autonomous vehicle (AV) technology stack, despite the high annotation and maintenance costs associated with this. Therefore, many recent works propose methods for online estimation of HD maps from sensor data, complicating integration in autonomous driving technology stacks. In particular, they do not generate uncertainty or confidence estimates. This paper extends multiple state-of-the-art online map estimation methods to enable additional estimates of uncertainty and improve predictive power by up to 15% on the real-world nuScenes driving dataset. In the process, we find that incorporating uncertainty improves training convergence by up to 50% and improves predictive power by up to 15% on the real-world nuScenes driving dataset.

Main contributions:

This paper introduces a general vectorized map uncertainty description and extends many state-of-the-art online map estimation methods to make them additional Output uncertainty estimates without degrading pure mapping performance.

This paper empirically analyzes the potential sources of map uncertainty, confirms the lack of confidence in current map estimation methods, and provides information for future research directions.

This article will discuss recent online map estimation models combined with multiple state-of-the-art trajectory prediction methods, and show how the performance and training characteristics of downstream prediction models can be significantly improved by incorporating online mapping uncertainty. Accelerate training convergence by up to 50% and improve online prediction accuracy by up to 15%.

Network Design:

A key component of autonomous driving is understanding the static environment, such as the road layout and traffic surrounding autonomous vehicles (AVs). Therefore, high-precision (HD) maps have been developed to capture and provide such information, containing semantic information such as road boundaries, lane dividers, and centimeter-level road markings. In recent years, HD maps have proven to be indispensable for the development and deployment of autonomous vehicles and are widely used today [35]. However, HD maps are expensive to annotate and maintain over time, and they can only be used in geofenced areas, which limits the scalability of autonomous vehicles. ”.

To address these issues, many recent studies turn to online estimation of high-precision (HD) maps from sensor data. Broadly speaking, they aim to predict the locations and categories of map elements, usually in the form of polygons or polylines form, all derived from camera images and lidar (LiDAR) scans. However, current online map estimation methods do not produce any relevant uncertainty or confidence information. This is problematic because it leads to downstream users (consumers) implicitly assume that the inferred map components are deterministic, and any mapping errors (e.g., incorrect movement or placement of map elements) may lead to erroneous downstream behavior. To this end, this paper proposes to reveal the differences in online map estimation methods. of map uncertainty and incorporate it into downstream modules. Specifically, this paper incorporates map uncertainty into trajectory prediction and finds that mapper-predictor systems that incorporate map uncertainty (Figure 1) are better than those that do not. Compared with the system with map uncertainty, the performance is significantly improved.

How to closely integrate online mapping and trajectory prediction?

Figure 1. Uncertainty is generated from the online high-precision (HD) map estimation method and the Its incorporation into downstream modules brings various benefits. Left: Real HD map and agent location. Middle: HiVT[41] prediction using MapTR[22] output map. Right: Using MapTR[22] output The map and the HiVT[41] prediction with added point uncertainty (due to the large uncertainty due to the left road boundary being blocked by parked vehicles).

How to closely integrate online mapping and trajectory prediction?

Fig. 2. Many online high-precision vector map estimation methods operate by encoding multi-camera images, transforming them into a common bird's-eye view (BEV) feature space, and regressing the vertices of map elements. Our work does this by adding a probabilistic regression head This common output structure is enhanced by modeling each map vertex as a Laplacian distribution. To evaluate the resulting downstream effects, this paper further extends the downstream prediction model to encode map uncertainty, enhancing graph neural Network (GNN) and Transformer-based map encoder.

Experimental results:

How to closely integrate online mapping and trajectory prediction?

Figure 3. The uncertainty representation proposed in this article can capture the differences between the camera of the autonomous vehicle (AV) and the surrounding environment. Uncertainty due to occlusion between map elements. Left: Images from the front and front right cameras. Right: HD map generated by the online high-precision map model enhanced in this article. The ellipse represents the standard deviation of the distribution. Colors represent road boundaries, lane dividers, crosswalks, and lane centerlines.

How to closely integrate online mapping and trajectory prediction?

# Figure 4. In a dense parking lot, many models fail to produce accurate maps. Left: rear and rear left camera images. Right: HD map generated by the online high-precision map model enhanced in this article. The ellipse shows the standard deviation of the distribution. Colors represent road boundaries, lane dividers, crosswalks, and lane centerlines.

How to closely integrate online mapping and trajectory prediction?

How to closely integrate online mapping and trajectory prediction?

How to closely integrate online mapping and trajectory prediction?

How to closely integrate online mapping and trajectory prediction?

How to closely integrate online mapping and trajectory prediction?

#Summary:

This paper proposes a general vectorized map uncertainty formula and extends a variety of the latest online map estimation methods, including MapTR [22], MapTRv2 [23 ] and StreamMapNet [38], enabling them to additionally output uncertainty. We systematically analyze the resulting uncertainty and find that our approach captures many sources of uncertainty (occlusions, distance from the camera, time of day, and weather). Finally, this paper combines these online map estimation models with state-of-the-art trajectory prediction methods (DenseTNT [13] and HiVT [41]) and shows that incorporating online map uncertainty significantly improves the performance and training characteristics of the prediction models, respectively. Up to 15% and 50%. An exciting future research direction is to use these uncertainty outputs to measure the calibration of map models (similar to [16]). However, this task is complicated by the need for fuzzy point set matching, which is a challenging problem in itself.

Citation:

Gu X, Song G, Gilitschenski I, et al. Producing and Leveraging Online Map Uncertainty in Trajectory Prediction[J]. arXiv preprint arXiv:2403.16439 , 2024.

The above is the detailed content of How to closely integrate online mapping and trajectory prediction?. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
Two Point Museum: All Exhibits And Where To Find Them
1 months ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Why is Gaussian Splatting so popular in autonomous driving that NeRF is starting to be abandoned? Why is Gaussian Splatting so popular in autonomous driving that NeRF is starting to be abandoned? Jan 17, 2024 pm 02:57 PM

Written above & the author’s personal understanding Three-dimensional Gaussiansplatting (3DGS) is a transformative technology that has emerged in the fields of explicit radiation fields and computer graphics in recent years. This innovative method is characterized by the use of millions of 3D Gaussians, which is very different from the neural radiation field (NeRF) method, which mainly uses an implicit coordinate-based model to map spatial coordinates to pixel values. With its explicit scene representation and differentiable rendering algorithms, 3DGS not only guarantees real-time rendering capabilities, but also introduces an unprecedented level of control and scene editing. This positions 3DGS as a potential game-changer for next-generation 3D reconstruction and representation. To this end, we provide a systematic overview of the latest developments and concerns in the field of 3DGS for the first time.

How to solve the long tail problem in autonomous driving scenarios? How to solve the long tail problem in autonomous driving scenarios? Jun 02, 2024 pm 02:44 PM

Yesterday during the interview, I was asked whether I had done any long-tail related questions, so I thought I would give a brief summary. The long-tail problem of autonomous driving refers to edge cases in autonomous vehicles, that is, possible scenarios with a low probability of occurrence. The perceived long-tail problem is one of the main reasons currently limiting the operational design domain of single-vehicle intelligent autonomous vehicles. The underlying architecture and most technical issues of autonomous driving have been solved, and the remaining 5% of long-tail problems have gradually become the key to restricting the development of autonomous driving. These problems include a variety of fragmented scenarios, extreme situations, and unpredictable human behavior. The "long tail" of edge scenarios in autonomous driving refers to edge cases in autonomous vehicles (AVs). Edge cases are possible scenarios with a low probability of occurrence. these rare events

Choose camera or lidar? A recent review on achieving robust 3D object detection Choose camera or lidar? A recent review on achieving robust 3D object detection Jan 26, 2024 am 11:18 AM

0.Written in front&& Personal understanding that autonomous driving systems rely on advanced perception, decision-making and control technologies, by using various sensors (such as cameras, lidar, radar, etc.) to perceive the surrounding environment, and using algorithms and models for real-time analysis and decision-making. This enables vehicles to recognize road signs, detect and track other vehicles, predict pedestrian behavior, etc., thereby safely operating and adapting to complex traffic environments. This technology is currently attracting widespread attention and is considered an important development area in the future of transportation. one. But what makes autonomous driving difficult is figuring out how to make the car understand what's going on around it. This requires that the three-dimensional object detection algorithm in the autonomous driving system can accurately perceive and describe objects in the surrounding environment, including their locations,

This article is enough for you to read about autonomous driving and trajectory prediction! This article is enough for you to read about autonomous driving and trajectory prediction! Feb 28, 2024 pm 07:20 PM

Trajectory prediction plays an important role in autonomous driving. Autonomous driving trajectory prediction refers to predicting the future driving trajectory of the vehicle by analyzing various data during the vehicle's driving process. As the core module of autonomous driving, the quality of trajectory prediction is crucial to downstream planning control. The trajectory prediction task has a rich technology stack and requires familiarity with autonomous driving dynamic/static perception, high-precision maps, lane lines, neural network architecture (CNN&GNN&Transformer) skills, etc. It is very difficult to get started! Many fans hope to get started with trajectory prediction as soon as possible and avoid pitfalls. Today I will take stock of some common problems and introductory learning methods for trajectory prediction! Introductory related knowledge 1. Are the preview papers in order? A: Look at the survey first, p

Have you really mastered coordinate system conversion? Multi-sensor issues that are inseparable from autonomous driving Have you really mastered coordinate system conversion? Multi-sensor issues that are inseparable from autonomous driving Oct 12, 2023 am 11:21 AM

The first pilot and key article mainly introduces several commonly used coordinate systems in autonomous driving technology, and how to complete the correlation and conversion between them, and finally build a unified environment model. The focus here is to understand the conversion from vehicle to camera rigid body (external parameters), camera to image conversion (internal parameters), and image to pixel unit conversion. The conversion from 3D to 2D will have corresponding distortion, translation, etc. Key points: The vehicle coordinate system and the camera body coordinate system need to be rewritten: the plane coordinate system and the pixel coordinate system. Difficulty: image distortion must be considered. Both de-distortion and distortion addition are compensated on the image plane. 2. Introduction There are four vision systems in total. Coordinate system: pixel plane coordinate system (u, v), image coordinate system (x, y), camera coordinate system () and world coordinate system (). There is a relationship between each coordinate system,

SIMPL: A simple and efficient multi-agent motion prediction benchmark for autonomous driving SIMPL: A simple and efficient multi-agent motion prediction benchmark for autonomous driving Feb 20, 2024 am 11:48 AM

Original title: SIMPL: ASimpleandEfficientMulti-agentMotionPredictionBaselineforAutonomousDriving Paper link: https://arxiv.org/pdf/2402.02519.pdf Code link: https://github.com/HKUST-Aerial-Robotics/SIMPL Author unit: Hong Kong University of Science and Technology DJI Paper idea: This paper proposes a simple and efficient motion prediction baseline (SIMPL) for autonomous vehicles. Compared with traditional agent-cent

nuScenes' latest SOTA | SparseAD: Sparse query helps efficient end-to-end autonomous driving! nuScenes' latest SOTA | SparseAD: Sparse query helps efficient end-to-end autonomous driving! Apr 17, 2024 pm 06:22 PM

Written in front & starting point The end-to-end paradigm uses a unified framework to achieve multi-tasking in autonomous driving systems. Despite the simplicity and clarity of this paradigm, the performance of end-to-end autonomous driving methods on subtasks still lags far behind single-task methods. At the same time, the dense bird's-eye view (BEV) features widely used in previous end-to-end methods make it difficult to scale to more modalities or tasks. A sparse search-centric end-to-end autonomous driving paradigm (SparseAD) is proposed here, in which sparse search fully represents the entire driving scenario, including space, time, and tasks, without any dense BEV representation. Specifically, a unified sparse architecture is designed for task awareness including detection, tracking, and online mapping. In addition, heavy

FisheyeDetNet: the first target detection algorithm based on fisheye camera FisheyeDetNet: the first target detection algorithm based on fisheye camera Apr 26, 2024 am 11:37 AM

Target detection is a relatively mature problem in autonomous driving systems, among which pedestrian detection is one of the earliest algorithms to be deployed. Very comprehensive research has been carried out in most papers. However, distance perception using fisheye cameras for surround view is relatively less studied. Due to large radial distortion, standard bounding box representation is difficult to implement in fisheye cameras. To alleviate the above description, we explore extended bounding box, ellipse, and general polygon designs into polar/angular representations and define an instance segmentation mIOU metric to analyze these representations. The proposed model fisheyeDetNet with polygonal shape outperforms other models and simultaneously achieves 49.5% mAP on the Valeo fisheye camera dataset for autonomous driving

See all articles