SuMa++: Efficient LiDAR-based Semantic SLAM

Authors: Xieyuanli Chen, Andres Milioto, Emanuele Palazzolo, Philippe Giguère, Jens Behley, Cyrill Stachniss · IROS 2019

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

SuMa++ adds per-point semantic labels from RangeNet++ to surfel-based LiDAR SLAM and uses them to remove moving objects from the map and to weight ICP residuals.

Why it matters

The method needs only a LiDAR, no camera, and uses semantics both to clean the map and to make registration more robust to outliers. On KITTI it also showed that removing all movable classes can make SLAM diverge in urban scenes, a concrete warning for anyone filtering dynamic objects by class alone.

Problem

LiDAR SLAM that relies on geometry alone can lock onto consistently moving objects, which corrupts the map and the pose estimate; highways with few static structures and many moving cars are the hardest case. Removing every potentially movable class is a blunt fix, because it also discards parked cars that are useful features for scan registration.

Proposed solution

SuMa++ runs RangeNet++ on a spherical projection of each scan to label every point, reduces back-projection label artifacts with an erosion followed by a depth-consistent flood-fill, and stores a label and its probability in each surfel. A surfel whose label disagrees with the new observation gets a penalty in its stability log-odds and is removed after several observations, and the frame-to-model ICP weights each residual by a Huber term, the semantic compatibility of scan and map labels, and surfel stability.

Key insights

Filtering by semantic inconsistency between the scan and the map removes objects that actually moved while keeping parked cars, which are useful registration features.
Removing all movable classes (SuMa nomovable) made the surfel map sparser and caused divergence in urban KITTI sequences, partly because wrong labels removed static surfels.
Erosion followed by depth-consistent flood-fill reduces the label errors that arise when 2D range-image predictions are projected back to 3D points.
Semantic weights in ICP down-weight residuals where scan and map labels disagree, for example on moving cars already filtered from the map.

Main results

KITTI road sequences 30–41 (segments of 5–400 m, rotational error in °/100 m / translational error in %): SuMa 1.35/6.13, SuMa nomovable 1.17/1.46, SuMa++ 1.04/1.46.
Highway sequence 35: translational error fell from 26.8% (SuMa) to 1.11% (SuMa++); sequence 40 from 18.0% to 1.95%; sequence 41 from 15.6% to 1.67%.
KITTI odometry training sequences 00–10 (100–800 m segments): SuMa++ 0.29/0.70 against SuMa 0.36/0.83; IMLS-SLAM reports 0.55% and LOAM 0.84% translational error.
KITTI odometry test set (server-side evaluation): translational error 1.06% against 1.39% for SuMa, with rotational error 0.0032 deg/m for both.
Timing on an Intel Xeon W-2123 with an Nvidia Quadro P4000: on average 75 ms per scan for RangeNet++ labels and 48 ms for surfel mapping, up to 190 ms when integrating loop closures.

Limitations

Dynamic objects cannot be filtered in the first observation; with many moving objects in the first scan the method fails, so all movable classes are removed during an initialisation period (authors' stated limitation).
Only movable classes are penalised; vegetation, whose appearance changes with viewpoint, is kept on purpose.
Segmentation errors remain after flood-fill and can remove surfels that are actually static.
On the road sequences SuMa++ performs close to the remove-all-movables baseline, and on several sequences (33, 36, 37, 39, 40, 41) its translational error is higher than at least one baseline.
Evaluation uses KITTI data only, from one sensor (Velodyne HDL-64E S2); semantics are not used for loop closure, which the authors list as future work.

Field impact

Cited 584 times (figure as given). The authors release the source code at https://github.com/PRBonn/semantic_suma/.

Abstract

Reliable and accurate localization and mapping are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the mapping step or derail localization. In this paper, we propose an extension of a recently published surfel-based mapping approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the mapping process. The semantic information is efficiently extracted by a fully convolutional neural network and rendered on a spherical projection of the laser range data. This computed semantic segmentation results in point-wise labels for the whole scan, allowing us to build a semantically-enriched map with labeled surfels. This semantic map enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints. Our experimental evaluation on challenging highways sequences from KITTI dataset with very few static structures and a large amount of moving cars shows the advantage of our semantic SLAM approach in comparison to a purely geometric, state-of-the-art approach.

Open Source Robotics