MOMASPLAT augments robot demonstrations for mobile manipulation

This framework synthesizes full manipulation trajectories from unseen robot base poses including dynamic object interactions.

MOMASPLAT is a novel-view demonstration augmentation framework for mobile manipulation (MoMa). It synthesizes full manipulation trajectories from unseen robot base poses. This includes dynamic object interactions.

Visuomotor policies trained through imitation learning are highly sensitive to changes in camera viewpoint. This is problematic for MoMa. Robots rarely execute manipulation from the exact same base pose used during demonstration collection.

The framework combines 3D Gaussian Splatting, feed-forward 3D foundation models, and kinematics-based action retargeting. 3D Gaussian Splatting is used to reconstruct the scene and manipulated object. Kinematics-based action retargeting helps generate geometrically consistent synthetic demonstrations paired with transformed actions.

MOMASPLAT was evaluated on five real-world MoMa tasks involving rigid and articulated objects across three scenes. It improves robustness to unseen base poses. It more than doubles the average success rate of the strongest baseline.

The method maintains non-trivial success at large base displacements where all baselines fail. It reduces the need for large-scale real-world data collection.

Key facts

  • MOMASPLAT synthesizes full manipulation trajectories from unseen robot base poses.
  • The framework more than doubles the average success rate of the strongest baseline.
  • MOMASPLAT was evaluated on five real-world tasks across three scenes.

All sections

Edition