Success Weighted by Path Length

• Also called: SPL
• Category: State estimation & navigation › Navigation
• Tags: navigation, evaluation-metrics, embodied-ai, benchmarking, simulation

A navigation metric that averages, over episodes, success multiplied by the ratio of the shortest-path distance to the length of the path the agent actually took.

Definition

Success weighted by path length (SPL) is an evaluation metric for goal-directed navigation agents, proposed by Anderson and colleagues in the 2018 report On Evaluation of Embodied Navigation Agents. It combines whether the agent reached the goal with how efficiently it did so. For N test episodes, SPL = (1/N) Σ Sᵢ · ℓᵢ / max(pᵢ, ℓᵢ), where Sᵢ is 1 if episode i succeeded and 0 otherwise, ℓᵢ is the shortest-path distance from the start to the goal, and pᵢ is the length of the path the agent actually traveled.

The shortest-path distance ℓᵢ is the geodesic distance through the navigable space, computed on the environment's map or navigation mesh, not the straight-line distance through walls. An agent that reaches the goal along the shortest path scores 1 for that episode, an agent that reaches it after traveling twice as far scores 0.5, and a failed episode scores 0 regardless of how close the agent came. Success itself is defined by the benchmark, typically stopping within a set distance of the goal, and many benchmarks require the agent to issue an explicit stop action. SPL is the standard headline metric for PointGoal and ObjectGoal navigation in embodied AI simulators such as Habitat, and variants exist for object-goal tasks, where ℓᵢ is the distance to the nearest valid goal instance.

SPL has known weaknesses. Because failure scores zero, it is binary at the episode level and can hide agents that consistently get very close. It measures only path length, not time, collisions, energy or the number of turns. In unseen environments where an agent must explore to find the goal, scores well below 1 are expected even for good agents, since the shortest path assumes a map the agent does not have. Variants such as SoftSPL, which replaces binary success with progress toward the goal, and metrics based on time or actions were proposed to address some of these issues.

Why it matters

SPL gave embodied navigation research a shared way to compare agents that rewards both reaching the goal and not wandering, and it is reported in most papers on simulated navigation benchmarks. Knowing what it measures, and what it ignores, is needed to read those results and to judge whether a high score will carry over to a real robot.

Key applications

• Benchmarking PointGoal and ObjectGoal navigation agents
• Embodied AI challenges in simulators such as Habitat
• Comparing learned and classical navigation stacks
• Evaluating vision-and-language navigation agents
• Tracking sim-to-real navigation performance on physical robots

In practice

A team evaluates two navigation policies on 1,000 episodes in held-out simulated apartments. For each episode, the simulator computes the geodesic shortest path from start to goal on the navigation mesh, and the evaluation script logs whether the agent stopped within the success radius and how far it traveled. Policy A succeeds more often but takes long detours; policy B succeeds less often but takes nearly optimal routes when it does. The team reports success rate and SPL side by side, plus collision counts, because SPL alone would not show that one policy bumps into furniture much more than the other.

Common misconceptions

• SPL uses straight-line distance to the goal — it uses the geodesic shortest-path distance through navigable space
• SPL rewards getting close to the goal — a failed episode scores zero however close the agent stopped; SoftSPL was proposed to give partial credit
• An SPL near 1 should be expected from a good agent — in unknown environments an agent must explore, so even strong agents score well below the shortest-path ideal
• SPL captures overall navigation quality — it ignores time, collisions, smoothness and energy, which usually need to be reported separately

Who should know this

• Embodied AI and navigation researchers
• Engineers benchmarking mobile robot navigation stacks
• Reviewers and readers comparing navigation papers
• Simulation and benchmark designers

Related technologies

• Habitat Simulator
• Navigation Meshes
• SoftSPL
• PointGoal Navigation
• ObjectGoal Navigation

Related terms

• Embodied AI
• Obstacle Avoidance
• Topological Map
• Semantic Map
• Frontier-Based Exploration
• Sim2Real

Related projects

• GENESIS: World's fastest physics engine: 43M FPS for embodied AI learning
• LeRobot: HuggingFace ML library for real-world robotics in PyTorch
• PyBullet: Python physics simulator for robotics and RL research with fast CPU-based simulation
• XLeRobot: Practical dual-arm mobile home robot for $660 with <4hrs assembly
• RLinf: Scalable infrastructure for post-training foundation models via RL
• Nav2: ROS 2 navigation stack with path planning and obstacle avoidance for mobile robots