NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration

Authors: Ajay Sridhar, Dhruv Shah, Catherine Glossop, Sergey Levine · ICRA 2024

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

A single diffusion policy with an optional, maskable goal image handles both exploration and goal reaching; on a LoCoBot it beat image-subgoal diffusion with a model about 15× smaller.

Why it matters

One compact policy replaces a goal-conditioned policy plus a separate image-generation model, cutting model size from 335M to 19M parameters, which the authors note suits onboard computers with less compute. The diffusion head kept multiple collision-free options open at junctions and still narrowed to the right path when given a goal image. It was trained on public multi-robot navigation datasets (GNM and SACSoN) and tested on a real robot rather than only in simulation.

Problem

A robot searching an unfamiliar environment for a goal shown in an image needs two behaviours: task-agnostic exploration to find the goal and goal-directed navigation once it is found. Prior systems split these into separate models, for example a goal-conditioned policy plus a 300M-parameter image diffusion model that proposes subgoal images. Exploration also requires representing multimodal action distributions, such as left or right at a junction, which simpler policy classes average out.

Proposed solution

NoMaD extends the ViNT Transformer backbone (EfficientNet-B0 encoders, 4 layers, 4 heads) with a binary goal mask that blocks attention to the goal token, sampled with probability 0.5 during training so one policy learns both modes. The observation context conditions a 1D conditional U-Net diffusion model that denoises a sequence of 8 future actions in 10 steps. The policy is paired with a topological graph memory and frontier-based exploration, following the ViKiNG setup.

Key insights

Masking the goal token during training lets a single network learn both goal-conditioned and undirected behaviour without losing performance in either mode.
Modelling actions directly with diffusion, instead of generating subgoal images, captured multimodal exploration choices with a much smaller model.
Autoregressive prediction over discretised actions could in principle represent several modes, but its predictions were largely unimodal and the deployed policy was jerky and slow to react to moving obstacles.
The choice of visual encoder mattered: the ViNT encoder with attention-based goal masking trained well with diffusion, while a ViT encoder did not.

Main results

Exploration in unseen environments: NoMaD 98% success with 0.2 collisions per experiment, against Subgoal Diffusion 77% / 1.7 (335M parameters), Autoregressive 90% / 2.0, Random Subgoals 70% / 2.7, Masked ViNT 50% / 1.0 and VIB 30% / 4.0.
Navigation in known environments with a topological graph: NoMaD 90% success, the same as Subgoal Diffusion and Random Subgoals.
Unified versus dedicated policies: NoMaD (19M) reached 98% undirected and 92% goal-conditioned success, matching Diffusion Policy (15M, 98% undirected) and the ViNT policy (16M, 92% goal-conditioned).
Encoder ablation: success / collisions were 52% / 3.2 for a late-fusion CNN, 68% / 1.5 for an early-fusion CNN, 32% / 2.5 for a ViT and 98% / 0.2 for NoMaD.
NoMaD uses 19M parameters against 335M for Subgoal Diffusion, which the paper describes as 15× fewer.
The paper gives two counts for the test environments: 6 in the evaluation set-up (Section V) and 5 in the discussion of Table I.

Limitations

Goals are specified only as images, which the authors note are not always the most natural modality for users; language and spatial coordinates are left to future work.
High-level planning uses a standard frontier-based exploration strategy; semantic or prior-knowledge-based selection of regions to explore is not used.
In exploration NoMaD succeeded in all but the hardest test environment.
Real-world evaluation used one robot platform, the LoCoBot.

Field impact

NoMaD won the ICRA 2024 Best Conference Paper Award and was a finalist for the ICRA 2024 Best Paper Award in Cognitive Robotics and the Best Student Paper Award. The authors released code and pre-trained model checkpoints through the project page.

Abstract

Robotic learning for navigation in unfamiliar environments needs to provide policies for both task-oriented navigation (i.e., reaching a goal that the robot has located), and task-agnostic exploration (i.e., searching for a goal in a novel setting). Typically, these roles are handled by separate models, for example by using subgoal proposals, planning, or separate navigation strategies. In this paper, we describe how we can train a single unified diffusion policy to handle both goal-directed navigation and goal-agnostic exploration, with the latter providing the ability to search novel environments, and the former providing the ability to reach a user-specified goal once it has been located. We show that this unified policy results in better overall performance when navigating to visually indicated goals in novel environments, as compared to approaches that use subgoal proposals from generative models, or prior methods based on latent variable models. We instantiate our method by using a large-scale Transformer-based policy trained on data from multiple ground robots, with a diffusion model decoder to flexibly handle both goal-conditioned and goal-agnostic navigation. Our experiments, conducted on a real-world mobile robot platform, show effective navigation in unseen environments in comparison with five alternative methods, and demonstrate significant improvements in performance and lower collision rates, despite utilizing smaller models than state-of-the-art approaches. For more videos, code, and pre-trained model checkpoints, see https://general-navigation-models.github.io/nomad/

Open Source Robotics