NGEL-SLAM: Neural Implicit Representation-based Global Consistent Low-Latency SLAM System

Authors: Yunxuan Mao, Xuan Yu, Kai Wang, Yue Wang, Rong Xiong, Yiyi Liao · ICRA 2024

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

NGEL-SLAM pairs ORB-SLAM3 tracking and loop closure with multiple octree-based neural sub-maps, so the dense map is re-aligned quickly after loop closure instead of retrained.

Why it matters

Robots that build dense maps of large indoor areas need those maps to stay consistent after loop closure and to absorb new keyframes quickly. Keeping tracking in a feature-based system and splitting the neural map into anchored sub-maps gives both without retraining the whole map.

Problem

Neural implicit SLAM systems such as iMAP and NICE-SLAM give dense geometry but track through the neural map, which has no loop closure and loses global consistency in large scenes. Replacing the tracker with a traditional one does not solve this, because retraining a single scene map after loop-closure pose updates is slow, and slow convergence of the mapping network also prevents low-latency mapping.

Proposed solution

ORB-SLAM3 handles tracking, local bundle adjustment and loop closure, while the scene is stored as several local neural implicit sub-maps, each anchored to a keyframe. After global bundle adjustment the sub-maps are first moved with their anchor poses in real time and then fine-tuned; each sub-map stores multi-level features in a sparse octree with small occupancy and color MLP decoders for fast convergence. Images are rendered by choosing, per pixel, the sub-map with the lowest occupancy-variance uncertainty.

Key insights

Representing the scene as anchored sub-maps lets loop-closure corrections be applied by updating sub-map poses, which removes most map error without retraining; fine-tuning then removes the remaining small errors.
Feature-based tracking with loop closure (ORB-SLAM3) was more accurate on TUM RGB-D than tracking through the neural map.
A sparse octree grows only where space is occupied, so it saves memory compared with dense voxel grids while keeping fast convergence; mapping only keyframes further cuts mapping frames and data I/O.
Rendered occupancy variance correlates positively with depth error and negatively with PSNR, so it works as a criterion for choosing which sub-map renders each pixel.

Main results

Replica (8 scenes), rendering with estimated poses: depth L1 0.65 cm, PSNR 30.44, SSIM 0.876, LPIPS 0.151, versus 2.28 cm, 29.25, 0.855 and 0.190 for ESLAM; with ground-truth poses: 1.28 cm, 29.53, 0.864 and 0.156 versus 2.81 cm, 27.33, 0.836 and 0.204 for ESLAM.
TUM RGB-D tracking (ATE RMSE): 1.5 cm on fr1/desk, 0.5 cm on fr2/xyz and 1.0 cm on fr3/office, versus 2.5, 1.1 and 2.4 cm for ESLAM.
ScanNet tracking (6 scenes): average ATE RMSE 7.44 cm versus 7.50 cm for ESLAM, 9.63 cm for NICE-SLAM and 36.67 cm for iMAP; lowest error on 3 of the 6 scenes.
Mapping time per iteration: 14 ms versus 19 ms (ESLAM), 130 ms (NICE-SLAM) and 448 ms (iMAP), using 120 mapping frames instead of 400; the Fig. 1 caption describes this as running 10x faster than NICE-SLAM.
Memory: 5.32 MB versus 6.79 MB (ESLAM), 12.02 MB (NICE-SLAM) and 1.04 MB (iMAP).
Map fine-tuning improved Replica depth L1 from 1.34 to 1.28 cm and PSNR from 28.39 to 29.53.

Limitations

The conclusion scopes the system to indoor scenes, and it requires an RGB-D input stream.
Evaluation uses recorded datasets only (Replica, ScanNet, TUM RGB-D and the NICE-SLAM apartment sequence); no robot or live-sensor deployment is reported.
On ScanNet it does not have the lowest error on three of six scenes (0106, 0181, 0207), and its average margin over ESLAM is 0.06 cm.
iMAP uses less memory (1.04 MB vs 5.32 MB) because it stores the scene in a single MLP.
The second stage of loop-closure handling, sub-map fine-tuning, runs below real time.

Field impact

Winner of the IEEE ICRA 2024 Best Paper Award in Robot Vision, with 40 citations. The paper does not give a code or project link.

Abstract

Neural implicit representations have emerged as a promising solution for providing dense geometry in Simultaneous Localization and Mapping (SLAM). However, existing methods in this direction fall short in terms of global consistency and low latency. This paper presents NGEL-SLAM to tackle the above challenges. To ensure global consistency, our system leverages a traditional feature-based tracking module that incorporates loop closure. Additionally, we maintain a global consistent map by representing the scene using multiple neural implicit fields, enabling quick adjustment to the loop closure. Moreover, our system allows for fast convergence through the use of octree-based implicit representations. The combination of rapid response to loop closure and fast convergence makes our system a truly low-latency system that achieves global consistency. Our system enables rendering high-fidelity RGB-D images, along with extracting dense and complete surfaces. Experiments on both synthetic and real-world datasets suggest that our system achieves state-of-the-art tracking and mapping accuracy while maintaining low latency.

Open Source Robotics