A new way to control robots with unknown movements
Researchers developed a method to help robots learn how to move and balance using a mathematical approach.
Researchers proposed a Koopman model-based reinforcement learning (K-MBRL) algorithm. This method controls nonlinear robotic systems that have unknown dynamics.
The algorithm uses a stochastic Koopman operator-based learning method. This method approximates unknown nonlinear dynamics in a linear representation. A critic function estimates the remaining future cost.
The linear Koopman dynamics simplify the policy gradient computation to a constant matrix product. This allows for efficient policy improvement. One-step model propagation reduces prediction errors from long-horizon rollouts.
The learned policy allows for real-time control through feedforward evaluation. This happens without online optimization. The method shows strong sample efficiency in HalfCheetah and quadruped four-legged locomotion tasks.
A physical quadruped robot used the method to perform maneuvers. The robot rose to an upright posture on two rounded rear feet. It maintained balance during bipedal walking and returned safely to the ground.
Key facts
- The K-MBRL algorithm uses a critic function to estimate future costs.
- The method was tested on HalfCheetah and quadruped four-legged locomotion tasks.
- A physical quadruped robot performed bipedal walking and upright posture maneuvers.