RIM-Net: Learning from Rotation-Invariant Motion States for Wearable Human Activity Recognition

Even when the human activity remains unchanged, uncontrolled variations in sensor orientation can substantially alter the inertial measurements along the sensor's three local axes, posing a major challenge to robust wearable human activity recognition. We explicitly transform these axis-dependent readings into an intrinsic motion state jointly characterized by the magnitudes of dynamic acceleration and angular velocity and their geometric alignment. This motion state preserves motion intensity and the geometric relationship between the two inertial modalities while remaining unchanged under rotations of the sensor coordinate frame, thereby mapping the same physical motion observed under different sensor orientations to a consistent and stable input representation. Building on this motion state, we propose RIM-Net with two core components: Adaptive Quadrature Harmonic Encoding (AQHE) and Cross-Part Phase-Relation Learning (CPRL). To address the limited ability of standard convolutions to explicitly capture motion rhythms at different temporal frequencies, AQHE analyzes the motion-state sequence across multiple frequency bands and adaptively combines the resulting quadrature responses to extract discriminative local motion patterns. Moreover, local frequency and amplitude features alone do not describe whether movements at different body locations are synchronized or exhibit frequency-specific lead–lag relationships. CPRL explicitly models these cross-part temporal relations and uses them to guide feature aggregation. Experiments on five public human activity recognition benchmarks under subject-independent protocols demonstrate that RIM-Net consistently achieves strong recognition performance while maintaining a compact parameter footprint. These results support our central hypothesis that rotation-invariant motion states constructed from inertial measurements provide a reliable foundation for robust wearable activity recognition.