A new model helps robots handle complex physical tasks

ForceLDM uses force and motion data to help robots manage contact-rich manipulation.

Researchers created ForceLDM, a force-aware latent dynamics model. This model helps robots with contact-rich manipulation. These tasks require fine-grained control and complex physical interactions.

Vision-only policies struggle because contact states are hard to see. Existing methods use force or tactile sensing as auxiliary observations. ForceLDM learns future contact and motion dynamics to guide action generation.

A teacher network uses privileged future force/torque and optical flow signals. It extracts future dynamic features. These features are distilled into a student network. The student network uses only current observations.

A curriculum-based progressive noise injection strategy improves robustness. This strategy helps mitigate over-reliance on future features. The model learns a compact representation of future dynamics through knowledge distillation.

Experiments on five real-world contact-rich manipulation tasks show the model works. It outperforms state-of-the-art baselines. It generalizes to novel objects and remains robust to environmental perturbations.

Key facts

  • ForceLDM learns future contact and motion dynamics to guide action generation.
  • The model uses a teacher network to distill features into a student network.
  • Experiments were conducted on five real-world contact-rich manipulation tasks.

All sections

Edition