Inner product regularization for imitation learning
A new method called InneR helps robots respond consistently to changes in their environment.
Robots learning from demonstrations must handle state perturbations like sensor noise or scene variation. These include compounding errors.
Inner product regularization, or InneR, explores how policy output shifts when given the same perturbation at different states. The robot should respond to state perturbations in a consistent manner locally.
The policy Jacobian is the Fisher information for stochastic policies. The null space of the inner product governs which state perturbations have a first-order effect on the policy.
InneR clusters states with aligned inner products. It averages these inner products and minimizes the smallest eigenvalues of that average. This shapes the policy so nearby states share a common null space.
In simulated experiments, InneR makes learned policies more robust to irrelevant state dimensions and shifts between training and test distributions. Norm penalties often perform worse than behavior cloning.
On a real robot arm performing three manipulation tasks, InneR raises the average success rate of a diffusion policy from 18% to 57% without additional data.
Key facts
- InneR raises the average success rate of a diffusion policy from 18% to 57% on a real robot arm.
- The method works for deterministic or stochastic policies.
- InneR enforces consistent responses to perturbations without limiting policy expressiveness.