acceptodds
Withdrawn from ICLR 2027

RIM-Net: Learning from Rotation-Invariant Motion States for Wearable Human Activity Recognition

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Even when the human activity remains unchanged, uncontrolled variations in sensor orientation can substantially alter the inertial measurements along the sensor's three local axes, posing a major challenge to robust wearable human activity recognition. We explicitly transform these axis-dependent readings into an intrinsic motion state jointly characterized by the magnitudes of dynamic acceleration and angular velocity and their geometric alignment. This motion state preserves motion intensity and the geometric relationship between the two inertial modalities while remaining unchanged under rotations of the sensor coordinate frame, thereby mapping the same physical motion observed under different sensor orientations to a consistent and stable input representation. Building on this motion state, we propose RIM-Net with two core components: Adaptive Quadrature Harmonic Encoding (AQHE) and Cross-Part Phase-Relation Learning (CPRL). To address the limited ability of standard convolutions to explicitly capture motion rhythms at different temporal frequencies, AQHE analyzes the motion-state sequence across multiple frequency bands and adaptively combines the resulting quadrature responses to extract discriminative local motion patterns. Moreover, local frequency and amplitude features alone do not describe whether movements at different body locations are synchronized or exhibit frequency-specific lead–lag relationships. CPRL explicitly models these cross-part temporal relations and uses them to guide feature aggregation. Experiments on five public human activity recognition benchmarks under subject-independent protocols demonstrate that RIM-Net consistently achieves strong recognition performance while maintaining a compact parameter footprint. These results support our central hypothesis that rotation-invariant motion states constructed from inertial measurements provide a reliable foundation for robust wearable activity recognition.

Primary area:
learning on time series and dynamical systems
Keywords:
Wearable human activity recognition, inertial sensing, rotation invariance, adaptive harmonic encoding, cross-part phase modeling

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_Yt3BOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces RIM-Net, a rotation-robust method for wearable human activity recognition. It constructs rotation-invariant motion states using dynamic acceleration and angular velocity magnitudes, combined with reliability-weighted geometric alignment. The architecture features an Adaptive Quadrature Harmonic Encoder (AQHE) to extract local motion patterns via state-conditioned multi-frequency quadrature filters, and Cross-Part Phase-Relation Learning (CPRL) to capture frequency-specific temporal coordination across sensor locations. Evaluated on five benchmarks under subject-independent protocols, RIM-Net achieves state-of-the-art weighted F1 scores on four datasets with only 58K–104K parameters, while ablation studies confirm the effectiveness of both AQHE and CPRL.

Critical strengths
  1. The motion-state construction has a clear geometric interpretation and an explicit invariance guarantee under fixed sensor-frame rotations. This is a useful alternative to relying exclusively on data augmentation to learn orientation robustness.

  2. The method performs well across several HAR datasets. Importantly, the authors investigate both the contributions of AQHE and CPRL and the mechanisms within these modules, providing more convincing evidence than an end-to-end comparison alone.

Critical weaknesses
  1. The proposed state discards the direction of gravity and retains only the magnitudes and alignment of dynamic acceleration and angular velocity. For an ideal stationary window with constant acceleration and zero angular velocity, the dynamic acceleration residual becomes zero. Consequently, different static postures can become indistinguishable to the model.

  2. The synthetic rotation evaluation applies a fixed coordinate rotation shared by the accelerometer and gyroscope at each location. This is precisely the setting for which the representation is analytically invariant. Consequently, these results do not establish robustness to practical sensor reattachment, location displacement, time-varying orientation, or imperfect sensor alignment. Furthermore, the rotation robustness experiments do not include a direct comparison with the rotation-invariant TRI-HAR baseline under the same perturbations. The exact subject partitions and activity-label protocols also need to be specified more clearly, particularly for Opportunity.

Questions for authors
  1. Can the authors provide per-class F1 scores and confusion matrices, especially for static activities? How does the model retain information necessary to distinguish stationary postures after gravity-related directional information has been removed?

  2. Can the authors evaluate time-varying orientation perturbations or realistic sensor reattachment, and include TRI-HAR under the same rotation-robustness protocol?

  3. What are the exact subject assignments, activity-label definitions, and retained input channels for each benchmark? Can the authors clarify the input-channel descriptions in Section 4.1 relative to the configurations in Appendix Table 4?

  4. Since the magnitude of the complex phase relation is deterministically recoverable from its real and imaginary components, what explains the large performance drop when the explicit reliability channel is removed?

1 comment

Dear authors, we would appreciate your response to the questions raised in our review. We remain open to discussion and would be happy to clarify any of our points. Thank you.

Reviewer_wrcbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RIM-Net for wearable human activity recognition that is robust to sensor-coordinate variations. From the acceleration and angular velocity at each body location, it constructs a rotation-invariant motion state composed of the magnitudes of dynamic acceleration and angular velocity and their reliability-weighted geometric alignment. Adaptive Quadrature Harmonic Encoding (AQHE) uses this state to adaptively combine multi-frequency quadrature responses, and Cross-Part Phase-Relation Learning (CPRL) uses frequency-specific phase relations between motion-magnitude signals at different body locations as a bias in cross-part attention. Under subject-independent evaluation on five public benchmarks, the paper reports an average weighted F1-score of 93.96% and the best result among the compared methods on four datasets, with 58K–104K trainable parameters. In controlled experiments where each body location is rotated independently by a fixed angle, performance remains largely stable without rotation augmentation.

Critical strengths
  1. The paper obtains rotation invariance structurally in the input representation rather than through rotation augmentation, and proves the invariance of the motion state under explicitly stated assumptions: the co-located accelerometer and gyroscope undergo the same rotation, the rotation remains constant over the support of the low-pass filter, and the preceding preprocessing is rotation-equivariant. The controlled rotation experiments, in which different body locations are rotated independently, match the conditions of this analysis, so the scope of the claim is clear.
  2. The paper reports results over five random seeds on five datasets and performs a parameter-matched 2×2 component-level ablation. Comparisons with Static-QHE, content-only attention, and RIM-Net-Mag examine the individual contributions of the state-conditioned offsets, the phase bias, and the geometric alignment component.
Critical weaknesses
  1. Removing the low-pass acceleration component discards the gravity direction, so static-posture information is no longer directly available. Without class-level results, it is not possible to judge whether this explains the lower performance on UCI-HAR, where RIM-Net trails TinyHAR and TinierHAR.
  2. It is unclear whether the normalization (in particular, any per-axis scaling) preserves the rotation-equivariance assumption in Appendix A.1, and whether the number of frequency bands K was selected on validation data. Protocol details such as the subject partitions and baseline tuning are also insufficiently reported.
Questions for authors
  1. Is normalization applied to the raw-axis signals or to the motion state? If it is applied to the raw signals, how is the rotation-equivariance assumption in Appendix A.1 preserved?
  2. What are the train/validation/test subject partitions for each dataset, and what do the five random seeds vary? How are the label set and the Null class handled for Opportunity?
  3. Were the number of frequency bands K and the center-frequency placement in Appendix A.6 selected based on validation performance or on test results?
  4. Since “CPRL w/o reliability” retains Re(C_ij,k) and Im(C_ij,k), the magnitude |C_ij,k| can still be recovered from them. This comparison therefore seems to test whether the consistency feature is provided explicitly, rather than whether the reliability information is removed. What is the scope of the claim this ablation is intended to support?
Reviewer_zUkROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RIM-Net for wearable human activity recognition (HAR) under sensor-coordinate rotations. It constructs a rotation-invariant motion state and introduces Adaptive Quadrature Harmonic Encoding (AQHE) for local frequency-aware feature extraction and Cross-Part Phase-Relation Learning (CPRL) for modeling frequency-specific relations across body locations. Experiments on five public datasets show strong performance, with the best results on four datasets, alongside ablation studies and controlled rotation tests. However, the methodological novelty and advantages over existing rotation-invariant methods remain insufficiently established.

Critical strengths
  1. Rotation-invariant representation: The motion state is constructed from vector norms and inner products, with an analytical justification of its invariance under stated assumptions.

  2. Extensive evaluation: The paper evaluates five datasets, uses subject-independent splits, reports results over five seeds, and provides comprehensive ablations.

Critical weaknesses
  1. Limited methodological novelty: AQHE and CPRL combine established quadrature filtering, phase-relation features, and attention mechanisms. The paper does not sufficiently demonstrate that these combinations constitute a substantial advance beyond existing methods.

  2. Incomplete robustness comparisons: The controlled rotation experiments compare RIM-Net mainly against raw-input and rotation-augmented baselines, but do not include TRI-HAR under the same conditions. This weakens the evidence for superiority over existing rotation-invariant approaches.

Questions for authors
  1. Could the authors evaluate TRI-HAR under the same controlled rotation settings to establish a fair comparison with existing rotation-invariant methods?

  2. How does CPRL differ fundamentally from conventional phase-consistency features combined with learned relation modeling? What benefit does explicitly including the phase-consistency magnitude provide?

  3. Could the authors clarify the discrepancy between the input-channel counts reported in Section 4.1 and Appendix Table 4, and provide exact parameter counts for each dataset?

1 comment
Reviewer_zUkROpenReview ↗

Review

Dear authors, I would appreciate your responses to the questions raised in my review. I am happy to discuss any of the points further if needed. Thank you.

Reviewer_8A68OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes tackling the problem with human activity recognition models being sensitive to changes in sensor orientation, devices may be attached at different orientations, so it is desirable to have orientation invariant activity recognition methods. To address this, the authors construct a rotation-invariant motion state (dynamic acceleration magnitude, angular velocity magnitude, geometric alignment between acceleration and angular velocity). Building on this representation, the paper introduces two components: 1) AQHE to capture local motion rhythms through multi-frequency quadrature filters whose responses are adaptively modulated using the rotation-invariant motion state, and 2) CPRL to model frequency-specific temporal coordination between body locations. The proposed method is evaluated using weighted F1-score on five HAR datasets and across 5 models.

Critical strengths
  • Sensor orientation variability is a real and longstanding challenge in wearable HAR.
  • The design is interpretable and based on physics, rather than being purely learned.
  • Combination of, what I would argue, are existing invariance ideas working well together is interesting.
  • The results on the chosen metric are positive for the method
  • Authors include some meaningful ablation studies
Critical weaknesses
  • There is limited novelty in the method as a vast body of prior work use acceleration magnitude, gyroscope magnitude or cosine similarity as standard rotational invariants. So this makes the study a somewhat straightforward engineering design. -Invariance proof relies on strong assumptions. From what I gather, the theoretical invariance assumes: constant rotation within filter support, shared accelerometer/gyroscope rotation, rotation-equivariant preprocessing. But in real-world wearable applications, sensors often experience continuously changing orientation. So it is unclear how well the theoretical guarantees extend meaningfully beyond the fixed-orientation assumption
  • Comparison to state-of-the-art is limited in terms of large-scale datasets and models. Recent HAR architectures are missing in related works and for a field with significant number of publications, it is strange to see such a short section. No strong reasons are provided for the limited evaluations and benchmarks.
  • Results. Reports only weighted F1 and results do not support claims of "general" superiority. Given imbalanced class distributions, report mean F1 and not weight it to the majority class predictions. Did not consider other measures such as misalignment. Then UCI-HAR, which is probably the easiest seems to show that the tiny models outperform the proposed. In addition, how are computational efficiency claims substantiated? A comparison of performance vs. cost of training/inference in FLOPS or latency, memory consumption are missing and the trade-off is not seen in the paper. Further, sub analysis uses a very old study ConvLSTM that has been superseded by many other works.
  • Reference are massively inadequate and large number of state of the art are missing (too many to list here but for comparisions, HARMamba, HARTransformer, CrossHAR)
  • A claim is that CPRL captures meaningful phase-based coordination between body parts. However, there is little qualitative evidence of demonstrating this claim. It is unclear if CPRL truly learns interpretable coordination or just serves as another attention mechanism.
Questions for authors
  • How was the null class treated in the oppo. dataset and results.
  • Can the authors report mean F1 for Table 1?
  • Can you show that CPRL captures meaningful phase-based coordination?
  • The invariance proof assumes the rotation matrix remains constant over the support of the low-pass filter from my reading. How does performance change when sensor orientation varies continuously within a window, which is common in practical wearables?
  • By reducing tri-axial signals into magnitudes and alignment statistics, directional information is lost. So, are there activities that become indistinguishable under this representation? What are the failure cases from this information loss?
  • How does RIM-Net compare with more recent Transformer- and State-Space-based HAR models (e.g., HARMamba) and recent orientation-robust HAR methods such as Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts?
  • Why does RIM-Net underperform TinyHAR and TinierHAR on UCI-HAR? Is this suggestive of a limitation of the harmonic encoding?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.