A new way for humanoid robots to move objects with their whole body

A new framework allows robots to use their arms and torso to grasp and move bulky items.

Researchers introduced Whole-Body Dexterity (WBD). This framework uses a robot's body and appendages as contact surfaces to grasp and manipulate objects.

Current humanoid manipulation is mostly limited to end-effectors and simple grasps. It lacks the human ability to brace and slide bulky objects against the arms and torso.

The process uses a sim-to-real pipeline. This pipeline first generates a dataset of feasible whole-body grasps. It then trains reinforcement-learning experts on an any-to-any repositioning objective.

These experts are distilled into a single policy. This policy uses proprioception and object geometry. The controller is steered by coarse motion references from a human teleoperator.

The policy transferred zero-shot to a real Unitree G1. The robot repositioned a basketball and a cardboard box around the torso. It also stowed a box under one arm and carried it from a shelf to a table.

The distilled controller completed 90% of trials. This is higher than the 18% for direct retargeting and 4% for a generalist whole-body controller.

Key facts

  • The Whole-Body Dexterity (WBD) framework uses the robot's body and appendages as contact surfaces.
  • The distilled controller completed 90% of real-world whole-body manipulation tasks.
  • The Unitree G1 robot successfully moved a basketball and a cardboard box using this method.

All sections

Edition