Learning Reusable Motion Primitives by Separating Identity from Execution

Complex skills can be viewed as organized compositions of reusable motion primitives. Learning such primitives therefore requires an explicit notion of equivalence: when do different motion trajectories instantiate the same primitive? Recent action-representation research spans action chunking, tokenization, and latent-action learning, with increasing attention to shaping latent-action semantics; yet these advances do not by themselves specify what defines reusable primitive identity. To make this equivalence explicit, we define primitive identity as the ordered structure of canonical basic motions, while execution captures how that structure is realized through local transformations, relative rhythm, and overall temporal scale. We instantiate this separation in a hierarchical model that canonicalizes local motion geometry, encodes the ordered structure of the corresponding basic motions, and estimates execution factors separately. We evaluate the resulting representation by testing cross-execution reuse, preservation of the complete identity-defining structure, and identity–execution recombination, and further test whether the model can detect and identify primitive occurrences within longer, unsegmented trajectories. On a controlled benchmark of variable-length primitives, the model recovers both primitive structure and execution from isolated recordings and recovers primitive boundaries and identities within longer trajectories composed from a known set of primitives. Two comparison settings reveal distinct gaps. A fixed-duration chunking interface is poorly aligned with the variable temporal support of primitive instances. Even with oracle primitive support, reconstruction-based models with primitive-level semantic shaping—including a factorized extension with supervised execution—do not satisfy the required representation responsibilities: cross-execution reuse, complete-structure preservation, and recombination. Together, these results show that defining primitive identity separately from execution yields a learnable representation with a directly testable criterion for recognizing the same primitive across execution variations and across occurrences within longer trajectories.