Range Image

• Also called: range view, spherical range image
• Category: Sensing › Perception outputs
• Tags: lidar, 3d-perception, semantic-segmentation, point-cloud, data-representation

A 2D image of a 3D range scan in which each pixel stores the measured distance along one beam direction, giving the scan an image grid for fast processing.

Definition

A range image stores a 3D scan as a 2D array in which each pixel corresponds to a viewing direction and holds the distance measured along it. For a rotating multi-beam lidar, rows usually correspond to the laser beams, or elevation angles, and columns to azimuth angles around the rotation, so each point is placed by a spherical projection of its direction. Extra channels often hold the x, y and z coordinates, the intensity of the return and a validity mask.

The image layout makes neighborhood relations cheap. Points that are close in the image are usually close in space, so normals, edges, ground segmentation and clustering can be computed with simple image operations instead of nearest-neighbor searches in 3D. It also lets standard 2D convolutional networks process lidar data: networks such as SqueezeSeg and RangeNet++ label each pixel, and the labels are then projected back to the 3D points. SuMa and SuMa++ use range images for a different reason, rendering the surfel map into the same image layout so that new scans can be associated with the map by pixel position.

The projection has costs. When several points fall into one pixel, only one, usually the nearest, is kept, so some points must be labeled afterwards from their neighbors. Pixels with no return, from sky, glass or absorbing surfaces, need masking. At object boundaries, a pixel's neighbors may lie on surfaces far apart in depth, which causes labels to bleed between foreground and background; RangeNet++ adds a k-nearest-neighbor clean-up in 3D to reduce this. A range image differs from a camera depth image in projection and value: a depth camera uses a pinhole projection and usually stores distance along the optical axis, while a lidar range image uses angular coordinates and stores the distance along each beam.

Why it matters

Range images turn sparse, unordered lidar point clouds into a dense grid that fast image algorithms and convolutional networks can process. Many real-time lidar segmentation, odometry and compression pipelines rely on them.

Key applications

• Semantic segmentation of lidar scans with 2D networks
• Projective data association in lidar SLAM
• Ground removal and obstacle clustering
• Normal and edge estimation for lidar odometry features
• Compression and storage of lidar data

In practice

An autonomous driving team needs to label each lidar point as road, car, pedestrian or vegetation in real time. They project every scan to a range image with one row per laser and a fixed number of columns, storing range, x, y, z and intensity in five channels. A 2D segmentation network predicts a class per pixel, the labels are projected back to the points, and a nearest-neighbor vote in 3D fixes points that were dropped by the projection or mislabeled at object edges. The same range image is reused downstream for ground removal and for clustering obstacles by connected pixel regions.

Common misconceptions

• A range image is the same as a camera depth image — the projection is angular rather than pinhole, and pixels store distance along the beam rather than along the optical axis
• The projection keeps every point — points that share a pixel are discarded, so labels must be propagated back to them
• Neighboring pixels are always neighboring surfaces — at depth discontinuities adjacent pixels can lie far apart, which causes label bleeding
• Range images only suit spinning lidars — any sensor with a regular grid of beam directions, including some solid-state lidars, can be stored this way

Who should know this

• Lidar perception engineers
• Autonomous driving segmentation teams
• SLAM and odometry developers
• Machine learning engineers working with 3D data

Related technologies

• RangeNet++
• SqueezeSeg
• Spherical Projection
• Bird's-Eye-View Projection
• Semantic KITTI

Related terms

• LIDAR
• Point Cloud
• Depth Camera
• Surfel
• LiDAR Odometry
• Semantic Map

Related projects

• Open3D: Modern library for 3D point cloud processing, mesh operations, and 3D machine learning
• oomwoo: Open-source DIY robot vacuum: 3D-printed, ROS2 + 2D LiDAR, fully local.
• PCL: Industry-standard C++ library for point cloud processing in robotics perception
• Cartographer: Google's real-time 2D and 3D SLAM system for multi-platform robotics
• OpenDroneMap: Open-source photogrammetry toolkit that turns drone images into maps and 3D models
• FAST-LIVO2: Fast, direct LiDAR-inertial-visual odometry from HKU-MARS, published in IEEE T-RO