Robots learn better by asking smarter questions
A new method helps robots understand tasks by focusing on what they do not know.
Robots can learn tasks by asking humans for help. This new method helps robots ask better questions to understand underspecified rewards.
Reward learning usually assumes human input covers everything about a task. In practice, humans can only attend to some features better than others.
Preference queries are a low-effort way to resolve uncertainty. An informed prior targets these queries at features that need resolving.
The robot provides an explanation with each query. This explanation focuses human attention on specific features.
In simulation, the informed prior beats the standard active preference learning baseline by up to . This happens in the low-query regime where reducing human effort matters most.
A user study showed that the informed prior and explanations improve reward recovery. These two methods amplify each other.
Key facts
- The informed prior beats the standard active preference learning baseline by up to in the low-query regime.
- Explanations accompany queries to focus human attention on specific features.
- The method improves reward recovery in a user study.