Neural Implicit Representation
- Also called: neural field, implicit neural representation, INR, coordinate-based network
- Category: Sensing › 3D maps & reconstruction
- Tags: 3d-representation, neural-fields, mapping, deep-learning, slam
A way of storing a shape, scene or map as a neural network that maps spatial coordinates to values such as occupancy, signed distance, density or color, so it can be queried at any point.
Definition
Instead of storing geometry explicitly as points, voxels or meshes, a neural implicit representation stores a function. A network, often a multilayer perceptron (MLP), takes a 3D coordinate, sometimes with a viewing direction or a latent code, and returns a value at that point: an occupancy probability (Occupancy Networks), a signed distance (DeepSDF), or a volume density and color (NeRF). The surface is implicit: it is defined as a level set of the function, such as the zero level of a signed distance, and can be rendered by volume rendering or sphere tracing, or extracted as a mesh with marching cubes.
The representation is continuous, so it can be queried at any resolution; its memory depends on the size of the model rather than directly on scene volume; and it is differentiable, so it can be trained from depth, lidar or images through rendering losses. Plain MLPs struggle with fine detail and are slow to train and query. Positional encodings help with detail, and hybrid representations store learned features in a grid, such as a dense or sparse voxel grid or the multi-resolution hash grid of Instant-NGP, and decode them with a small network, trading memory for speed and detail.
In robotics, neural implicit maps are used in SLAM systems such as iMAP and NICE-SLAM for RGB-D input and LONER for lidar, which optimize the map network online while tracking the sensor pose. They are also used as object models for grasping and as distance queries for planning. The costs are real: each scene must be trained, incremental training can forget earlier regions unless past keyframes are replayed, and evaluating many points is slower than a table lookup. Explicit alternatives such as 3D Gaussian splatting give up some continuity in exchange for much faster rendering.
Why it matters
Neural implicit representations give compact, continuous and differentiable maps that fuse many observations and fill small gaps. They let learning-based mapping, rendering and planning share one model of the world, which is why they appear across SLAM, manipulation and simulation work.
Key applications
- Dense SLAM and mapping from lidar or RGB-D sensors
- Object shape models for grasp planning
- Novel view synthesis for simulation and data generation
- Distance and collision queries for motion planning
- Shape completion from partial observations
In practice
A lidar SLAM system represents the map as a feature grid with a small MLP decoder that predicts density. Each new scan provides rays; the system samples points along a subset of those rays and trains the map so that the depth rendered from the predicted density matches the measured range, while estimating the scan pose. A set of past keyframes is replayed during training so that older parts of the map are not overwritten. After mapping, the field is evaluated on a grid and meshed with marching cubes for inspection, or queried directly for distances during planning.
Common misconceptions
- Neural implicit maps have no explicit structure — most real-time systems combine a feature grid or hash table with a small network, so memory still grows with scene size
- An implicit surface cannot be turned into a mesh — it is a level set and can be extracted with marching cubes at any chosen resolution
- A neural implicit representation is the same as NeRF — NeRF is one instance that predicts density and color; occupancy and signed distance networks are others
- Continuous means unlimited detail — detail is limited by model capacity, input encoding and the training data
Who should know this
- SLAM and mapping engineers
- 3D vision researchers
- Manipulation researchers using learned object models
- Simulation engineers building scene reconstructions
Related technologies
- NeRF
- DeepSDF
- Occupancy Networks
- Instant-NGP
- NICE-SLAM
- 3D Gaussian Splatting
Related terms
Related projects
- LingBot-Map: Streaming 3D reconstruction from one camera: pose and depth for every frame, live
- oomwoo: Open-source DIY robot vacuum: 3D-printed, ROS2 + 2D LiDAR, fully local.
- Cartographer: Google's real-time 2D and 3D SLAM system for multi-platform robotics
- OpenDroneMap: Open-source photogrammetry toolkit that turns drone images into maps and 3D models
- FAST-LIVO2: Fast, direct LiDAR-inertial-visual odometry from HKU-MARS, published in IEEE T-RO
- DimOS: Python robotics runtime with agents, SLAM and teleop, no ROS required
Videos
- Computer Vision - Lecture 9.1 (Coordinate-based Networks: Implicit Neural Representations): Tübingen Machine Learning