Keyframe

• Also called: key frame, KF
• Category: State estimation & navigation › Localisation
• Tags: slam, visual-odometry, mapping, state-estimation, bundle-adjustment

A selected frame that a SLAM or mapping system keeps, with its pose and observations, for later optimization, while most other frames are used briefly and discarded.

Definition

A keyframe is one of a small set of camera images or sensor scans that a SLAM, odometry or mapping system decides to keep. Every incoming frame is used to track the sensor's motion, but only keyframes are stored with their estimated pose and their observations, such as image features, depth or lidar points. The map and the trajectory are then refined using these keyframes rather than every frame. The approach was established for real-time visual SLAM by PTAM, from Georg Klein and David Murray in 2007, which split tracking and mapping into separate threads, and it was developed further in systems such as ORB-SLAM.

The rule for creating a new keyframe is a central design choice. Common criteria include time or distance since the last keyframe, enough parallax to triangulate new points, and a drop in how much of the current view overlaps with the last keyframe, measured for example by shared features or by shared map voxels. Systems also cull keyframes that have become redundant. Too few keyframes and tracking is lost or the map is sparse; too many and optimization slows down without adding information.

Keyframes are what make optimization-based SLAM affordable. Bundle adjustment and pose graph optimization run over keyframes, often only those in a local window, and loop closure searches compare new frames against stored keyframes. In neural implicit and dense mapping systems, a keyframe set is replayed during training of the map so that regions observed earlier are not forgotten as the network or feature grid is updated with new frames.

Why it matters

Sensors produce far more frames than a robot can optimize over in real time. Keyframe selection decides which information is kept, so it directly affects map quality, tracking robustness, memory use and the computational budget of every SLAM system built on it.

Key applications

• Local bundle adjustment in visual and visual-inertial SLAM
• Loop closure detection against a database of stored keyframes
• Replay of past observations in neural implicit and dense mapping
• Relocalization after tracking loss
• Building lightweight maps for long-term localization

In practice

A team running a visual-inertial SLAM system on a handheld rig at 30 frames per second sees that only a few keyframes per second are created when walking and almost none when standing still. When they move into a feature-poor corridor, tracking becomes fragile; lowering the overlap threshold so keyframes are inserted sooner helps keep enough triangulated points. For a neural lidar mapping system, they set the keyframe rule so that a new scan is kept when its overlap with the last keyframe falls below a threshold, and each optimization step samples rays from the current frame and a selection of stored keyframes. They watch the keyframe count in long runs, since an unbounded set will eventually slow optimization.

Common misconceptions

• Every frame becomes part of the map — most frames are used only for tracking; only selected keyframes are stored and optimized
• More keyframes always give a better map — redundant keyframes add computation with little new information, which is why systems cull them
• A keyframe in SLAM is the same as a keyframe in animation — in SLAM it is a stored sensor observation with a pose, not a hand-set pose used for interpolation

Who should know this

• SLAM and visual odometry engineers
• Developers of neural and dense 3D mapping systems
• Drone and mobile robot navigation teams
• Computer vision students studying structure from motion

Related technologies

• ORB-SLAM
• PTAM
• Bundle Adjustment
• Visual-Inertial Odometry
• Bag of Visual Words

Related terms

• SLAM
• Visual Odometry
• Pose Graph Optimization
• Neural Implicit Representation
• Neural Radiance Field
• Loop Closure

Related projects

• LingBot-Map: Streaming 3D reconstruction from one camera: pose and depth for every frame, live
• oomwoo: Open-source DIY robot vacuum: 3D-printed, ROS2 + 2D LiDAR, fully local.
• Cartographer: Google's real-time 2D and 3D SLAM system for multi-platform robotics
• OpenDroneMap: Open-source photogrammetry toolkit that turns drone images into maps and 3D models
• FAST-LIVO2: Fast, direct LiDAR-inertial-visual odometry from HKU-MARS, published in IEEE T-RO
• DimOS: Python robotics runtime with agents, SLAM and teleop, no ROS required