HALO: LEARNING MOTION REPRESENTATIONS AS CONTROLLED DEPARTURES FROM SIGNAL STATISTICS

Wearable sensors can record the same activity differently across people, body locations, and devices, making a model trained in one setting difficult to reuse in another. Handcrafted signal statistics often remain useful under such changes, but they are limited to a fixed set of measurements. Neural encoders are more flexible, yet they usually learn a separate feature space instead of building on this prior structure. We introduce HALO (Handcrafted Anchors, Limited Offsets), a 0.16M-parameter encoder that starts from a statistical representation and learns a bounded departure from it. HALO treats the statistics as an anchor and predicts a zero-initialized, input-conditioned offset whose distance from the anchor is limited by an interpretable radius. Varying this radius produces a family of representations from a single forward pass. To separate motion from acquisition effects, we pretrain HALO in HAR-CAUKER, a synthetic motion world with physically coupled inertial rendering and supervision from simulator-only factors, then freeze the encoder for real-data evaluation. Across 14 public datasets, HALO ranks first in aggregate under cross-subject, cross-position, and cross-dataset protocols, exceeding the strongest external comparator by 0.6, 11.0, and 9.0 macro-F1 points. It ranks first in all six cross-position directions and on 9/13 held-out datasets. Without a fitted classifier, HALO also improves Recall@1 over MantisV2 by 9.7/8.6 points across positions/datasets; shuffling its learned directions removes 15.9/20.0 points, showing that the learned departure carries transferable, input-matched structure. These results support learning bounded departures from signal statistics as a practical alternative to relearning representations for each deployment.