Gaussian process regression

• Also called: GPR, GP regression, kriging
• Category: Sensing › Perception outputs
• Tags: machine-learning, uncertainty, bayesian-methods, mapping, regression

A Bayesian, nonparametric regression method that models an unknown function as a Gaussian process, giving both a predicted mean and an uncertainty at every query point.

Definition

Gaussian process regression (GPR) places a prior over functions: any finite set of function values is assumed to be jointly Gaussian, with a mean function, often zero, and a covariance given by a kernel k(x, x') that encodes how strongly the outputs at two inputs are correlated. Conditioning this prior on noisy observations gives a posterior that is again a Gaussian process, with a closed-form predictive mean and variance at any new input. The variance is low near the data and grows away from it, which is what makes GPR useful when a robot needs to know where its model is unreliable. The standard reference is Rasmussen and Williams's Gaussian Processes for Machine Learning (2006); in geostatistics the same method is known as kriging.

The kernel determines what kind of functions the model expects. The squared exponential (RBF) kernel produces very smooth functions, while the Matérn family allows rougher ones. Kernel hyperparameters such as length scales, signal variance and noise variance are usually fitted by maximizing the marginal likelihood of the data. Standard kernels are stationary: they assume the same smoothness everywhere. Real fields such as terrain elevation, water temperature or pollutant concentration often change sharply in some places and slowly in others, which motivates nonstationary kernels such as the attentive kernel in the AK paper.

The main practical limit is cost. Exact GPR solves a linear system with an n × n covariance matrix, which takes O(n³) time and O(n²) memory for n data points. Sparse approximations based on inducing points, local GP models and structured kernels make it practical for larger datasets and on board robots. Another caveat is that, for fixed hyperparameters, the predictive variance depends only on where the data were taken, not on the measured values, so a poorly fitted kernel gives misleading uncertainty. Gaussian noise assumptions also fit heavy-tailed sensor errors badly.

Why it matters

GPR provides predictions with uncertainty from small datasets. Robots use that uncertainty to decide where to sample next, how far to trust a learned dynamics model, and how to interpolate sparse measurements into continuous maps.

Key applications

• Mapping continuous spatial fields such as temperature, salinity or gas concentration
• Learning dynamics models with uncertainty for model-based control
• Bayesian optimization of controller gains and other parameters
• Safe learning, where uncertainty bounds keep exploration within constraints
• Terrain modeling from sparse range data

In practice

An environmental monitoring team runs an autonomous surface vessel that measures water temperature on a lake. Each measurement, tagged with its GPS position, is added to a GP model, and the kernel hyperparameters are re-fitted periodically as data accumulate. The posterior mean gives the temperature map, and the posterior variance shows which areas are still poorly known. The planner sends the vessel toward locations with high variance, or another information measure, within its remaining battery budget. As the dataset grows, the team switches to a sparse GP with inducing points so that each update stays fast enough for online replanning.

Common misconceptions

• GP uncertainty is always reliable — it is only as good as the kernel and its hyperparameters; a wrong length scale gives overconfident or overly cautious variance
• GPR cannot scale — exact inference is cubic in the number of points, but sparse and local approximations handle large datasets
• Nonparametric means there are no parameters — GPs still have kernel hyperparameters; nonparametric means the model's complexity grows with the data instead of being fixed in advance

Who should know this

• Researchers working on informative path planning and environmental monitoring
• Engineers building learned dynamics models
• Control engineers using Bayesian optimization for tuning
• Machine learning practitioners who need uncertainty estimates

Related technologies

• GPyTorch
• GPflow
• scikit-learn
• Sparse Variational Gaussian Processes
• Bayesian Optimization

Related terms

• Conformal Prediction
• Robotic information gathering
• Forward Dynamics Model
• System Identification
• Elevation Map

Related projects

• LeRobot: HuggingFace ML library for real-world robotics in PyTorch
• LingBot-Map: Streaming 3D reconstruction from one camera: pose and depth for every frame, live
• Open3D: Modern library for 3D point cloud processing, mesh operations, and 3D machine learning
• Cartographer: Google's real-time 2D and 3D SLAM system for multi-platform robotics
• OpenVLA: 7B-parameter Vision-Language-Action model for robotic manipulation
• OpenDroneMap: Open-source photogrammetry toolkit that turns drone images into maps and 3D models

Videos

• Gaussian Processes: Mutual Information
• Machine Learning Lecture 26 "Gaussian Processes" -Cornell CS4780 SP17: Kilian Weinberger

Wikipedia

• Kriging