Disney Research Studios and ETH Zurich presented RobotKeyframing, a control framework that lets legged robots follow sequences of poses specified at particular times. Instead of prescribing every movement, an operator can provide partial or complete target poses, with the learned controller generating the locomotion needed to reach them. The work was presented at the Conference on Robot Learning.
The system combines a transformer encoder, which accepts a variable number of future targets, with reinforcement learning using multiple critics to handle both continuous movement rewards and sparse target-reaching rewards. In simulation and hardware experiments, robots followed the requested keyframe sequences at the specified times. The researchers reported that anticipating upcoming targets improved performance, while the multi-critic approach reduced the hyperparameter tuning needed compared with a single-critic alternative.
