Affordance
- Also called: affordances, affordance detection, action possibilities
- Category: Planning & manipulation › Manipulation
- Tags: perception, manipulation, grasping, robot-learning, cognitive-robotics
An action that an object or environment makes possible for a particular agent, such as a handle that can be grasped or a surface that can support a placed object.
Definition
Affordances are possibilities for action that the environment offers to a particular agent. The term was introduced by the psychologist James J. Gibson, who argued that animals perceive the world partly in terms of what they can do with it: a chair affords sitting, a handle affords grasping, a flat surface affords placing. Affordances are relational; they depend on both the object and the body and abilities of the agent.
In robotics, affordance reasoning means predicting what actions are possible and where. Affordance detection labels parts of an image or point cloud with actions such as grasp, cut, pour, push or contain. Affordance maps assign a score to each pixel or location, for example how likely a grasp at that point is to succeed. These representations link perception directly to action rather than stopping at object names.
The relational nature matters for robots. A cup handle affords grasping to a hand that fits through it but not to a large parallel-jaw gripper; a step affords climbing to a legged robot but not to a wheeled one. Good affordance models therefore account for the robot's embodiment, not just the object category.
With large language and vision models, affordances also serve as a grounding signal. A language model can propose steps for a task, but it does not know which of those steps the robot can actually perform in the current scene. Systems such as SayCan combined language model suggestions with learned estimates of whether each skill could succeed from the current state, an affordance in Gibson's sense.
Why it matters
Robots need to know not only what objects are but what can be done with them. Affordances connect perception to action, which is why they appear throughout grasping, tool use and language-guided manipulation.
Key applications
- Grasp and placement prediction from images or point clouds
- Tool-use and part-level manipulation, such as handles, lids and spouts
- Grounding language instructions in actions the robot can actually perform
- Navigation and locomotion over terrain that does or does not support the robot
In practice
A team building a household manipulation system might train a network that predicts, for each pixel of an RGB-D image, where the robot's gripper can grasp and where objects can be placed. The task planner then queries these maps: to open a drawer it looks for a graspable handle region, and to set a cup down it looks for a placeable surface. Training data often comes from the robot's own grasp attempts, labeled by whether they succeeded, so that the predictions reflect that specific gripper.
Common misconceptions
- An affordance is a property of the object alone — it is a relation between the object and the agent's body and abilities
- Affordance detection is the same as object recognition — recognition names an object, while affordance detection predicts what actions it supports and where
- Affordances learned for one robot transfer directly to another — a different gripper or body can change which actions are possible
Who should know this
- Manipulation and grasping researchers
- Perception engineers building action-oriented vision models
- Researchers connecting language models to robot skills
Related technologies
- Grasp Planning
- Semantic Segmentation
- Vision-Language Models
- Task Planning
- Point Cloud Processing
Related terms
Related projects
- LeRobot: HuggingFace ML library for real-world robotics in PyTorch
- OpenPI: Physical Intelligence VLA foundation models for general robot control
- Open3D: Modern library for 3D point cloud processing, mesh operations, and 3D machine learning
- PCL: Industry-standard C++ library for point cloud processing in robotics perception
- Isaac Lab: NVIDIA robot learning framework: one task API across Newton, PhysX and Isaac Sim
- NVIDIA Isaac GR00T: NVIDIA open foundation model for humanoid robot reasoning and skills