Learning to Share Autonomy across Repeated Interaction

Authors: Ananth Jonnavittula, Dylan P. Losey · IROS 2021

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

Shared autonomy without a predefined goal list: the robot learns tasks from repeated teleoperation, recognises them from joystick input, imitates past runs, and returns control when unsure.

Why it matters

Users of assistive arms repeat the same daily tasks, so an arm that learns from those repetitions can reduce teleoperation effort without anyone programming a goal list. The discriminator-based hand-back avoids the deadlock seen with goal-inference methods, where the robot resists a user heading for an unknown target.

Problem

Shared-autonomy methods for assistive arms need prior knowledge, such as the set of goals the user might reach or the reward the user optimises. When the user attempts a task outside that set, such as opening a fridge the robot was never told about, the robot cannot help and may even push toward the wrong known goal.

Proposed solution

The robot stores every teleoperated interaction and trains an encoder that maps the user's current states and joystick commands to a latent task, and a decoder policy that outputs assistive joint velocities replicating the user's past behaviour for that task. A separate discriminator, trained to tell seen interactions from noisy deformations of them, scores how familiar the current interaction is, and that score sets the blend β between robot and human actions. The models are retrained between interactions so new tasks are learned continually.

Key insights

Repetition in daily life is what makes learning to assist possible; the robot never needs to be told what task the user wants.
Encoding the interaction into a latent task before choosing an action improved goal reaching over behaviour cloning (DAgger) conditioned directly on the interaction.
A separate discriminator for familiarity gave steadier confidence on a known task than DropoutDAgger's self-assessed uncertainty, which returned control on minor deviations.
Using only the human's commands, not the blended action, as input to the encoder avoids feedback loops where the robot convinces itself of the goal with its own actions.
Bayesian goal inference over a fixed goal set can deadlock on an unknown task, as the appendix shows when the user opens a drawer and the robot pulls toward a nearby notepad.

Main results

In the reaching ablation with three goals and 3 or 5 training demonstrations, where the user guided only the first 0.5 s, the encoder-based approach reached the goal with lower final state error than DAgger given the same data.
With five prior repetitions of a glass-lifting skill, the approach stayed confident throughout, while DropoutDAgger rarely provided assistance.
Trained on five repetitions of a drawer skill, the approach reduced human effort on the drawer task across all tested noise levels and needed no more effort than No Assist on an unseen can-reaching task.
In a study with 10 participants, human effort for reaching two known goals did not differ significantly between the approach and a Bayesian shared-autonomy baseline.
When participants performed two new tasks 9 times each with retraining every three trials, the robot recognised the drawer skill and provided assistance after training on 6 demonstrations; a model trained on all tasks assisted the original goals about as well as task-specific models.
The authors state the method learns from a practical number of interactions (fewer than 10).

Limitations

The approach does not explicitly model co-adaptation: as users trust the robot they may change how they teleoperate, for example by stopping joystick input altogether.
The user study had 10 participants from a university community, not users living with physical disabilities.
Experiments used a fixed 7-DoF arm in a simplified household setting, not a wheelchair-mounted arm.
Simulation experiments use simulated operators with Gaussian noise on their commands rather than real users.

Field impact

The paper was a finalist for the IROS 2021 Best RoboCup Paper Award and has 16 citations. It links a video of the experiments; it does not state a code or data release.

Abstract

Wheelchair-mounted robotic arms (and other assistive robots) should help their users perform everyday tasks. One way robots can provide this assistance is shared autonomy. Within shared autonomy, both the human and robot maintain control over the robot’s motion: as the robot becomes confident it understands what the human wants, it increasingly intervenes to automate the task. But how does the robot know what tasks the human may want to perform in the first place? Today’s shared autonomy approaches often rely on prior knowledge: for example, the robot must know the set of possible human goals a priori. In the long-term, however, this prior knowledge will inevitably break down — sooner or later the human will reach for a goal that the robot did not expect. In this paper we propose a learning approach to shared autonomy that takes advantage of repeated interactions. Learning to assist humans would be impossible if they performed completely different tasks at every interaction: but our insight is that users living with physical disabilities repeat important tasks on a daily basis (e.g., opening the fridge, making coffee, and having dinner). We introduce an algorithm that exploits these repeated interactions to recognize the human’s task, replicate similar demonstrations, and return control when unsure. As the human repeatedly works with this robot, our approach continually learns to assist tasks that were never specified beforehand: these tasks include both discrete goals (e.g., reaching a cup) and continuous skills (e.g., opening a drawer). Across simulations and an in-person user study, we demonstrate that robots leveraging our approach match existing shared autonomy methods for known goals, and outperform imitation learning baselines on new tasks. See videos here: https://youtu.be/NazeLVbQ2og

Open Source Robotics