LONER: LiDAR Only Neural Representations for Real-Time SLAM

Authors: Seth Isaacson, Pou-Chun Kung, Mani Ramanagopal, Ram Vasudevan, Katherine A. Skinner · RA-L 2023

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

Real-time LiDAR-only SLAM with a neural implicit map: ICP tracking, joint pose and map optimization over keyframes, and a JS-divergence dynamic-margin loss for faster, sharper online learning.

Why it matters

It estimates the sensor trajectory and a dense map together in real time from LiDAR alone, and the map can be rendered offline into depth images from novel viewpoints or meshes. The paper tests the dynamic-margin loss against other losses used in depth-supervised neural implicit frameworks and finds it converges faster and reconstructs geometry more accurately.

Problem

Neural implicit maps give a continuous scene model that can be queried for occupancy anywhere, but existing LiDAR implicit-mapping methods either need ground-truth poses or run slower than real time, and real-time neural SLAM systems use RGB or RGB-D cameras and work mainly indoors. During online training different map regions are learned to different degrees, so a single margin for all rays, as in the LOS loss, makes the network forget learned geometry and converge slowly.

Proposed solution

LONER runs parallel tracking and mapping threads: tracking decimates scans to 5 Hz and estimates odometry with point-to-plane ICP, without an IMU; mapping adds a keyframe every 3 s and jointly optimizes an MLP with a hierarchical feature grid and the poses of 8 keyframes (the current one plus random past ones). The loss combines a Jensen-Shannon dynamic-margin loss, which gives each ray a larger margin where the map is poorly learned and a smaller one where it is well learned, with a down-weighted depth loss and a sky loss found from holes in the LiDAR returns.

Key insights

Setting each ray's margin from the JS divergence between the goal and predicted ray-termination distributions lets new regions learn quickly while keeping already-learned geometry sharp.
Depth loss alone gives blurry geometry under limited training time but helps hole-filling, so it is kept at a small weight (λ₁ = 5 × 10⁻⁶).
Using ICP instead of inverse-NeRF for tracking saves GPU time for mapping, keeping the system real-time; the mapping step then refines the ICP poses.
Sky rays are found without a camera, from holes in the LiDAR depth image that point above the horizon, assuming the sensor is roughly level at initialization.
Selecting past keyframes at random for the optimization window generally works better than using only recent keyframes or a half-recent, half-random mix.

Main results

Trajectory RMS absolute pose error (m), LONER vs LeGO-LOAM: MCR 0.029 vs 0.052, Canteen 0.064 vs 0.129, Garden 0.056 vs 0.161, Quad 0.130 vs 0.126.
NICE-SLAM, run in an offline configuration with more samples and iterations, tracked only the small MCR sequence (0.248 m) and failed on Canteen and Garden.
Map quality on the indoor MCR sequence, LONER vs SHINE Mapping (which uses ground-truth poses): accuracy 0.186 vs 0.164 m, completion 0.069 vs 0.075 m, precision 0.473 vs 0.624, recall 0.932 vs 0.757; on Quad, accuracy 0.380 vs 0.301 m, completion 0.373 vs 0.148 m, precision 0.327 vs 0.453, recall 0.809 vs 0.717.
Runtime on an AMD Ryzen 5950X CPU and NVIDIA A6000 GPU: 14 ms per tracking step; 50 mapping iterations per keyframe take 2.79 s (about 56 ms each, about 18 Hz), within the 3 s keyframe interval.
Loss ablation (Table V), Quad sequence: full system L1 depth 0.880 and pose error 0.130 m, against 2.265 and 0.779 for depth loss only, 1.747 and 0.570 for depth plus fixed-decay LOS loss, and 2.262 and 0.154 for dynamic-margin LOS without depth loss.
Trained on a single CARLA simulator scan, the JS loss reached low depth error faster than LOS losses with slow, medium and fast margin decay and than depth loss alone.

Limitations

Dynamic objects are not handled; the test sequences were chosen to contain few dynamic objects, and the authors say more work is needed for highly dynamic environments.
No inertial data is used; the authors note it could help tracking under rapid rotation and in feature-sparse scenes where LiDAR is less informative.
City-scale operation may need other input feature embeddings and ray-selection heuristics (stated by the authors).
The map is geometry only; adding RGB without losing real-time performance is future work (stated by the authors).
On the outdoor Quad sequence, SHINE Mapping (with ground-truth poses) produced more complete geometry, capturing pillars that LONER omits, and LeGO-LOAM's trajectory error was slightly lower.

Field impact

Winner of the RA-L 2024 Best Paper Award and cited 56 times. The authors post the complete evaluation data on the project page, which also links the code repository.

Abstract

This paper proposes LONER, the first real-time LiDAR SLAM algorithm that uses a neural implicit scene representation. Existing implicit mapping methods for LiDAR show promising results in large-scale reconstruction, but either require groundtruth poses or run slower than real-time. In contrast, LONER uses LiDAR data to train an MLP to estimate a dense map in real-time, while simultaneously estimating the trajectory of the sensor. To achieve real-time performance, this paper proposes a novel information-theoretic loss function that accounts for the fact that different regions of the map may be learned to varying degrees throughout online training. The proposed method is evaluated qualitatively and quantitatively on two open-source datasets. This evaluation illustrates that the proposed loss function converges faster and leads to more accurate geometry reconstruction than other loss functions used in depth-supervised neural implicit frameworks. Finally, this paper shows that LONER estimates trajectories competitively with state-of-the-art LiDAR SLAM methods, while also producing dense maps competitive with existing real-time implicit mapping methods that use groundtruth poses.

Open Source Robotics