Event Camera

• Also called: dynamic vision sensor, DVS, neuromorphic camera, silicon retina
• Category: Sensing › Exteroceptive
• Tags: computer-vision, sensors, neuromorphic, high-speed-perception, drones

A vision sensor whose pixels independently report changes in brightness as timestamped events instead of full frames, giving very fine timing, high dynamic range and no motion blur.

Definition

An event camera does not capture images at a fixed frame rate. Each pixel works independently and fires an event when the logarithm of the light intensity it sees has changed by more than a set threshold since its last event. An event carries the pixel coordinates, a timestamp with microsecond-level resolution, and a polarity that says whether the brightness rose or fell. The output is an asynchronous stream whose rate depends on how much the scene changes: a static scene in front of a static camera produces almost nothing, while fast motion produces a very high event rate.

The first widely used sensor of this kind was the dynamic vision sensor (DVS) of Lichtsteiner, Posch and Delbruck (2008), inspired by the way the retina responds to change. Later designs such as DAVIS also read out conventional intensity frames from the same pixel array, and ATIS-type sensors add per-pixel intensity measurements. Because each pixel responds to log intensity relative to its own recent past, event cameras handle scenes with very bright and very dark regions together, and because there is no shared exposure time, fast motion does not blur the output. Latency and power use are also low compared with frame cameras running at high rates.

The cost is that standard computer vision algorithms do not apply directly. Methods either process events one at a time, for example in event-by-event filters, or accumulate them over short windows into event frames, time surfaces or voxel grids that can feed conventional or learned pipelines. Event cameras are noisy, sensitive to flicker from mains-powered lights, typically offer lower resolution than frame cameras, and respond mainly to edges moving relative to the sensor, so textureless regions and motion along an edge give little signal. They are used for high-speed tracking, visual odometry under fast rotation, and low-light or high-contrast scenes, often alongside a frame camera and an IMU.

Why it matters

Many perception failures on fast robots come from motion blur, saturated or underexposed images and latency in frame cameras. Event cameras remove those failure modes, which makes it possible to keep tracking features during spins and rapid maneuvers or in scenes that mix bright light and shade.

Key applications

• Visual odometry and SLAM during fast motion
• Low-latency obstacle detection and avoidance for drones
• High-speed object and feature tracking
• Vision in high-dynamic-range and low-light scenes
• Optical flow estimation

In practice

A quadrotor that loses a propeller can stay controllable by spinning fast about its vertical axis, but at that rotation rate a frame camera's images blur, especially in dim light, and feature tracking breaks down. Work on this problem compares a frame-based pipeline with one that uses an event camera: events are accumulated into short windows, features are detected and tracked in that representation, and the tracks are passed with IMU data to a visual-inertial odometry back end. Setting up the sensor involves calibrating intrinsics with a blinking or moving pattern (a static checkerboard produces no events), synchronizing timestamps with the IMU, and tuning the contrast thresholds and bias settings to balance sensitivity against noise. Flickering indoor lights generate spurious events that must be filtered before tracking.

Common misconceptions

• An event camera is just a very fast frame camera — it has no frames; each pixel reports changes asynchronously, so a static scene produces almost no output
• Event cameras see everything in the scene — they respond to brightness changes, mostly at moving edges, so textureless surfaces and motion along an edge give little signal
• Events are clean data — thresholds vary between pixels, background noise and light flicker produce spurious events, and filtering is part of every practical pipeline

Who should know this

• Perception engineers working on fast-moving robots
• Drone autonomy researchers
• Computer vision researchers in neuromorphic sensing
• State estimation engineers

Related technologies

• DAVIS Sensor
• Time Surfaces
• Spiking Neural Networks
• Neuromorphic Computing
• Event-Based Optical Flow

Related terms

• RGB Camera
• Visual-Inertial Odometry
• Visual Odometry
• IMU
• UAV

Related projects

• ArduPilot: World's most trusted open-source autopilot for drones, planes, rovers, and submarines
• PX4 Autopilot: Open-source flight control software for drones and other unmanned vehicles
• OpenDroneMap: Open-source photogrammetry toolkit that turns drone images into maps and 3D models
• QGroundControl: Cross-platform ground control station for mission planning and drone flight monitoring
• DimOS: Python robotics runtime with agents, SLAM and teleop, no ROS required
• pySLAM: Hybrid Python/C++ visual SLAM pipeline with classic and deep-learning front-ends

Videos

• Event Cameras: a New Way of Sensing - Davide Scaramuzza - ICCP 2024 Keynote: UZH Robotics and Perception Group