Bird's-Eye View

• Also called: BEV, top-down view
• Category: Sensing › Perception outputs
• Tags: 3d-perception, object-detection, lidar, sensor-fusion, autonomous-driving

A top-down grid representation of the area around a robot or vehicle, built from lidar, cameras or both, used for 3D detection, mapping and planning.

Definition

A bird's-eye view (BEV) represents a scene as seen from directly above, as a 2D grid over the ground plane centered on the robot or vehicle. Each cell holds features rather than a single value: occupancy, maximum or mean point height, point density, lidar intensity, semantic class or learned feature vectors. Because objects on the ground keep their real-world size and do not overlap in this view, a BEV grid is a convenient place to detect vehicles and pedestrians, estimate free space and pass information to a planner that already works in a top-down map frame.

From lidar, a BEV is built by binning points into cells. Hand-designed versions encode each cell with several height-slice channels plus density and sometimes intensity, as in the AVOD and MV3D detectors; learned versions such as PointPillars use a small network to encode the points in each vertical column into a feature vector, producing a pseudo-image that a standard 2D convolutional network can process. From cameras the view has to be constructed, because images are perspective projections. Early methods used inverse perspective mapping, which assumes a flat ground plane. Later learned methods estimate depth or use attention to lift image features into 3D and splat them onto the BEV grid, which also makes it straightforward to fuse several cameras and lidar into one shared representation.

A BEV grid differs from an elevation map. An elevation map stores terrain height for legged or off-road locomotion; a BEV is a general top-down feature grid for perception, usually built on roads or floors where the ground is close to flat. The flattening that makes BEV convenient also loses information: vertical structure is reduced to a few channels, overhanging objects are merged with what lies below them, and small or distant objects can cover very few cells at the chosen resolution.

Why it matters

BEV has become the common working space for perception in autonomous driving and outdoor mobile robots. It lets detection, tracking, mapping and planning share one metric frame and makes fusing several sensors into a single representation much simpler.

Key applications

• 3D object detection for autonomous vehicles
• Multi-camera and camera-lidar fusion
• Drivable area and lane segmentation
• Occupancy and free-space prediction for planning
• Cooperative perception sharing maps between vehicles

In practice

A delivery robot fuses a roof lidar and four cameras for sidewalk navigation. Lidar points within 40 m are binned into a grid with 0.1 m cells, with channels for several height slices and point density. Camera features are lifted onto the same grid using predicted depth distributions, and the two feature maps are concatenated before a convolutional head predicts drivable area, pedestrian boxes and their headings. The planner reads the same grid directly. When the team adds a ramp to the test route, they find that false obstacles appear where the ground rises, because the height slices were defined relative to a flat ground plane, and they add a ground-height estimate per cell to fix it.

Common misconceptions

• A BEV image is just an overhead camera photo — it is a constructed metric grid, usually built from lidar or by lifting perspective camera images into 3D
• Camera BEV is as precise as lidar BEV — camera methods must estimate depth, so position errors grow with distance
• BEV keeps the full 3D scene — height is compressed into a few channels, so overhangs and stacked objects are merged
• Inverse perspective mapping works everywhere — it assumes flat ground and distorts anything above the ground or on slopes

Who should know this

• Autonomous driving perception engineers
• Mobile robot and outdoor navigation developers
• Computer vision researchers working on 3D detection
• Sensor fusion engineers

Related technologies

• PointPillars
• Lift-Splat-Shoot
• Inverse Perspective Mapping
• Occupancy Grid
• Voxelization

Related terms

• LIDAR
• Point Cloud
• Range Image
• Elevation Map
• Semantic Map
• Cooperative Perception

Related projects

• oomwoo: Open-source DIY robot vacuum: 3D-printed, ROS2 + 2D LiDAR, fully local.
• Cartographer: Google's real-time 2D and 3D SLAM system for multi-platform robotics
• FAST-LIVO2: Fast, direct LiDAR-inertial-visual odometry from HKU-MARS, published in IEEE T-RO
• RTAB-Map: Versatile graph-based SLAM supporting RGB-D, stereo, and LiDAR with real-time operation
• GTSAM: Factor-graph optimization library powering many modern SLAM backends
• LeGO-LOAM: Classic lightweight, ground-optimized lidar odometry and mapping baseline (IROS 2018)