Human-Inspired Modular Reinforcement Learning of Movement and Posture for Musculoskeletal Control

Controlling upper-body movement while maintaining bipedal balance is a long-standing challenge in high-dimensional musculoskeletal models. Although recent reinforcement learning (RL) algorithms are capable of handling large action spaces, they still struggle here because balancing requires the rest of the body to continuously compensate for arm motion. In human physiology, upright posture is maintained not by a single rigid block but by sets of muscles that link body modules from the feet to the head, counteracting internal perturbations and keeping the center of mass within a narrow base of support. Inspired by human motor control, we propose a modular RL framework that coordinates movement, posture, and equilibrium across body segments. Specifically, we divide the body into arm, torso, and leg modules and train them jointly, with posture imposed as a constraint on the whole-body controller rather than as a reward. This lets the whole-body policy maximize task reward while staying within a stable postural region. In experiments on MyoSuite and MS-Human-700, with control expanded from the arm to the whole body, the resulting policy performs standing and manipulation tasks that demand both stability and mobility. These findings suggest that structuring control along biomechanical lines makes learning in neuromuscular models substantially more tractable.