- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes WAFER, a method to reduce the annotation effort for IMU-based human activity recognition. It extends the visual clustering approach of Bock et al. by adding features from a pretrained wearable model. Video, optical-flow, and imu features are standardized, combined through weighted concatenation, and clustered separately for each subject. One representative from each cluster is labeled, and that label is propagated to the remaining samples. These annotations are then used to train HAR models using only IMU data.
The method is evaluated on WEAR, ActionSense, and Wetlab through annotation accuracy and downstream performance with TinyHAR and DeepConvLSTM under leave-one-subject-out evaluation. WAFER improves annotation accuracy over the selected CLIP+flow baseline on all three datasets, with the clearest downstream gains on WEAR. The paper also examines modality weights, visual backbones, and the tradeoff between annotation effort and performance.
Critical strengths
- The results on WEAR show a clear benefit from adding wearable features. Annotation accuracy improves from 83.96% to 90.64% in the main comparison. The modality ablation also supports this: within the weight search in Table 4, the best CLIP+flow configuration reaches 85.14%, while using all three modalities reaches 90.64%. IMU-only features perform much worse. This suggests that the modalities provide complementary information.
- its interesting that the evaluation checks whether better annotations lead to better downstream recognition. The paper evaluates two HAR architectures under leave-one-subject-out evaluation and reports the annotation effort for different cluster counts. On WEAR, DeepConvLSTM with Weak-CE improves from 73.38% to 80.12% accuracy and from 63.23 to 73.77 macro F1. Matching the percentage of retained data for the thresholded comparisons is also reasonable, since the feature scales differ between the methods.
Critical weaknesses
- My main concern is that the comparisons do not establish an improvement over prior work as broadly as the paper suggests. The main technical change is adding pretrained wearable embeddings to the clustering and label-propagation pipeline of Bock et al. (2024b). This is a useful extension, but its value needs to be established through clear comparisons.
- The main annotation comparison only uses the earlier paper’s CLIP+flow configuration. However, Table 2 of Bock et al. reports 72.70% with CLIP alone on Wetlab, which is above WAFER’s 71.60%, and 60.25% with DINOv2 alone on ActionSense, which is close to WAFER’s 60.84%. Comparing against the fixed CLIP+flow pipeline is reasonable, but it does not establish an improvement over the strongest configurations reported in that paper. Bock et al., Table 2
- The downstream results are also mixed. In Table 3, the best weak-training DeepConvLSTM accuracy on ActionSense is 16.26% for WAFER, compared with 17.35% for the baseline. I find the WEAR results promising, but the evidence for a broader improvement is less convincing.
- Another concern is the model-selection protocol. Section 4.4 selects the fusion weights using annotation accuracy on WEAR, and Appendix A.4 uses the same dataset to select the visual backbone. I could not find a description of independent validation subjects or an accounting of the ground-truth labels used for these choices. This matters because reducing annotation requirements is the main motivation of the paper. It is unclear whether the reported WEAR performance is independent of these decisions.
- The baseline comparison also needs clarification. It is not clear which results were taken from the earlier paper and which were rerun under the current setup. For example, Random-CE differs between the baseline and WAFER columns despite using the same stated annotation budget. This could reflect differences in sampling or training randomness, but it needs to be explained. Appendix A.6 mentions three-seed averaging under GMM clustering, without making clear whether this also applies to downstream training. Performance variability is not reported, making the smaller gains difficult to assess.
Questions for authors
- Which subjects and ground-truth labels were used to select the modality weights, wrist weights, and visual backbone? Were the reported WEAR results evaluated independently of these choices, including within each downstream leave-one-subject-out fold? How are labels used for model selection counted in the annotation budget?
- Which results in Tables 2, 3, and A3 were taken from Bock et al., and which were rerun? Were the baseline and WAFER downstream models evaluated using identical sensor inputs, preprocessing, training windows, losses, training schedules, folds, and seeds? Why does Random-CE differ between the two columns?
- Why is the main annotation comparison limited to CLIP+flow, given the stronger visual-only results reported by Bock et al. on Wetlab and ActionSense? Are those results directly comparable under your protocol, and how do they affect the claim of improvement across all three datasets?
Dear reviewer,
Thank you for taking the time to assess our work and marking it clear in presentation and fixable in terms of soundness. Below we'd like to answer to your questions: