SPIN Road Mapper: Extracting Roads from Aerial Images via Spatial and Interaction Space Graph Reasoning for Autonomous Driving

Authors: Wele Gedara Chaminda Bandara, Jeya Maria Jose Valanarasu, Vishal M. Patel · ICRA 2022

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

Aerial-image road mapping, with no robot involved: graph reasoning over spatial and projected interaction spaces, at three scales, links occluded road segments and raised APLS connectivity by about 1 point on two benchmarks.

Why it matters

Road connectivity, not just pixel accuracy, decides whether an extracted map is usable for route planning, and SPIN's gains are largest on the connectivity metric. The module adds only 0.03M parameters and can be dropped into an existing ConvNet after a convolutional block. Faster convergence shortens training on the large high-resolution datasets that mapping for autonomous vehicles needs.

Problem

Road maps extracted from aerial images help autonomous vehicles find drivable areas and constrain motion planning. Roads vary in width, branch across the image and are often hidden by trees, buildings or clouds, and plain ConvNets capture long-range dependencies poorly, which breaks road connectivity. Transformers capture such dependencies but need high compute and long training on large high-resolution images.

Proposed solution

The SPIN module builds two graphs from a feature map: one over spatial positions, using a data-dependent similarity matrix to link distant road regions, and one in a projected interaction space, using a learned similarity matrix to separate roads from buildings, trees and other features. Their outputs are added to the input features, and a SPIN pyramid applies this at scales 1, 1/2 and 1/4. The full network uses residual blocks, two stacked hourglass modules, a road-orientation branch and multi-scale SoftIoU and orientation losses.

Key insights

Spatial-space graph reasoning links distant road segments, while interaction-space reasoning separates roads from other land cover; combining both beat either alone
A single graph convolution captures long-range dependencies that would otherwise need many stacked convolution layers
Running graph reasoning at three scales enlarges the receptive field without a transformer's compute cost
The largest gains were on APLS, a graph-based connectivity metric, consistent with recovering roads under trees and buildings
Adding SPIN made training converge faster; the authors state it increases the convergence rate by half

Main results

Massachusetts Roads: F1 84.47, relaxed IoU 73.12, accurate IoU 65.24 and APLS 72.49, the best of all compared methods; next best Batra et al. at 83.97, 72.37, 64.44 and 71.34
DeepGlobe: F1 84.32, relaxed IoU 72.89 and APLS 74.14, the best compared values; accurate IoU 67.02 was slightly below Batra et al. (67.21)
APLS gains over the next best method of +1.15 (Massachusetts) and +1.02 (DeepGlobe)
Ablation (Table II): ConvNet only 73.01 APLS, spatial reasoning 73.59, interaction reasoning 73.52, full SPIN 74.14. Table II labels its other two columns IoU_a and F1, but the full-model row (84.32, 67.02) matches DeepGlobe F1 and IoU_a in Table I, so those two labels appear swapped
The SPIN module adds 0.03M parameters to the ConvNet

Limitations

The work is aerial-image mapping: it extracts road maps from satellite and aerial images for autonomous driving, and no robot or vehicle is used in its experiments.
The paper states no limitations section; on DeepGlobe its accurate IoU (67.02) was slightly lower than Batra et al. (67.21)
Partially occluded images were removed from the Massachusetts training set because they hurt performance
Gains over the strongest baseline are small on pixel metrics (0.35–0.5 F1 points); the main improvement is in APLS
The faster-convergence claim rests on a single training curve (Fig. 6) with no reported wall-clock times

Field impact

Finalist for the ICRA 2022 Outstanding Automation Paper award, with 43 citations. The authors release their code on GitHub.

Abstract

Road extraction is an essential step in building autonomous navigation systems. Detecting road segments is challenging as they are of varying widths, bifurcated throughout the image, and are often occluded by terrain, cloud, or other weather conditions. Using just convolution neural networks (ConvNets) for this problem is not effective as it is inefficient at capturing distant dependencies between road segments in the image which is essential to extract road connectivity. To this end, we propose a Spatial and Interaction Space Graph Reasoning (SPIN) module which when plugged into a ConvNet performs reasoning over graphs constructed on spatial and interaction spaces projected from the feature maps. Reasoning over spatial space extracts dependencies between different spatial regions and other contextual information. Reasoning over a projected interaction space helps in appropriate delineation of roads from other topographies present in the image. Thus, SPIN extracts long-range dependencies between road segments and effectively delineates roads from other semantics. We also introduce a SPIN pyramid which performs SPIN graph reasoning across multiple scales to extract multi-scale features. We propose a network based on stacked hourglass modules and SPIN pyramid for road segmentation which achieves better performance compared to existing methods. Moreover, our method is computationally efficient and significantly boosts the convergence speed during training, making it feasible for applying on large-scale high-resolution aerial images. Code available at: https://github.com/wgcban/SPIN_RoadMapper.git.

Open Source Robotics