image_pipeline

The standard ROS camera stack: calibration, rectification, stereo and depth

License

BSD-3-Clause

Overview

image_pipeline is the canonical camera-processing stack of ROS, maintained by ros-perception. Its packages take a robot's cameras from raw driver output to calibrated, rectified images and 3D data: camera_calibration (interactive checkerboard calibration), image_proc (debayering and rectification), stereo_image_proc (disparity and point clouds), depth_image_proc (depth-to-cloud conversions and registration) and image_view. Practically every ROS robot with a camera runs some part of this pipeline.

The stack is organized as composable ROS 2 nodes connected by image_transport topics and camera_info calibration messages. camera_calibration provides the classic interactive GUI that computes monocular or stereo intrinsics from checkerboard views and writes them to the camera driver. image_proc consumes raw images plus camera_info and emits mono/color, rectified variants, handling Bayer demosaicing and distortion removal; crop/resize/decimate nodes reshape streams cheaply. stereo_image_proc runs block-matching disparity between rectified pairs and produces stereo point clouds, while depth_image_proc converts depth images from RGB-D sensors into point clouds, registers depth to color frames and converts metric/quantized encodings. Everything supports run-time composition for zero-copy intra-process pipelines on ROS 2, and the conventions it established — camera_info semantics, image_raw/image_rect topic naming — are the de-facto contract between camera drivers and perception code across the ROS ecosystem.

Why it matters

image_pipeline defines how cameras work in ROS: its calibration GUI is where most roboticists calibrated their first camera, and its topic and camera_info conventions are the contract that lets any ROS camera driver talk to any ROS perception stack.

Problem it solves

Raw camera output is uncalibrated, distorted, Bayer-patterned and 2D. image_pipeline turns it into calibrated, rectified images, disparity maps and point clouds with standard, composable nodes — so perception code never reimplements camera plumbing.

Key takeaways

• Interactive checkerboard calibration GUI for monocular and stereo cameras
• image_proc: debayering, distortion removal and rectification driven by camera_info
• stereo_image_proc: block-matching disparity and stereo point clouds
• depth_image_proc: depth-to-point-cloud conversion and depth-to-color registration
• Composable ROS 2 nodes supporting zero-copy intra-process pipelines

Who it is for

• ROS developers integrating cameras
• Perception engineers building stereo/RGB-D pipelines
• Robotics students calibrating their first camera
• Camera driver authors following ROS conventions

Limitations

• Classic block-matching stereo is dated versus learned or SGM-based matchers
• Calibration GUI workflow is manual — no automated target-free calibration
• Performance tuning (QoS, composition) is left to the integrator