- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces WristHARBench, a common participant-disjoint evaluation protocol for wrist activity recognition, and WISP, a configurable framework combining several time-series representations, standard classifiers, and optional Viterbi decoding. The study examines representation combinations, temporal decoding, and configuration search across nine primary datasets.
Critical strengths
Wrist-only recognition is a useful practical setting, and a common evaluation protocol makes the results easier to compare. I also appreciate the controlled experiments: all representation combinations are evaluated with and without decoding, and random and evolutionary search are compared at equal candidate budgets. In particular, combining R and S improves direct prediction over either component alone on all nine datasets. This is meaningful evidence that the feature blocks complement each other, independently of Viterbi decoding.
Critical weaknesses
-
The novelty is mostly in the particular feature combination and empirical study. The underlying components (e.g., random convolutions, interval statistics, spectral and symbolic descriptors, standard classifiers, and Viterbi decoding) are established. WISP offers a useful wrist-HAR recipe, but I am less convinced that it establishes a substantially new representation idea. The R/S results are the strongest contribution here. However, S bundles spectral, autocorrelation, state-transition, and cross-axis features, without separating them.
-
WISP searches 329 configurations, while strong baselines such as RDST and MR-Hydra use mostly fixed settings. Giving these baselines similar tuning and decoder choices would make the comparison more convincing. Also, Select5 always use Viterbi, while full search can turn it off. Most of the search gain over Select5 comes from HARMES, where the selected models disable Viterbi. A simple selector that also chooses whether to decode would help show what the broader search adds.
Questions for authors
-
How do the strongest external baselines perform with the same direct/Viterbi selection and a comparable tuning budget?
-
How much of the full-search advantage remains when the fixed-recipe selector can also choose the decoder, and when decoding respects recording boundaries and gaps?
4 comments
Response to Reviewer pwBJ (2/2): search as representation tuning, 4-metric Pareto frontier, Select5 HMM choice, and open-source Cookbook (W2, Q1, Q2)
For W2 and Q1
HMM-smoothing temporal decoding runs Viterbi smoothing over window scores with zero extra training in WISP. Across 9 primary cohorts (3 seeds), we tune wide TSC baselines over a 4-setting validation grid (Tune Time) and refit them (Training Time), alongside standalone and tuned WISP (including WISP-CIS-Funnel, subsampled search on WISP-CIS). CPU times report standardized single-core hours on one Intel Xeon 8-series CPU for fair comparison (running on multiple CPU cores in parallel finishes much faster in wall-clock time), and we convert NVIDIA A100 GPU hours to CPU core-equivalent hours by rated power (1 A100 GPU h = 64 Xeon 8570 core-h, 400 W vs. 6.25 W/core):
| Method (9 Cohorts, 3-Seed ) | Macro-F1 | Acc. | Bal. Acc. | Worst F1 | Tune Time | Training Time | Peak RAM |
|---|---|---|---|---|---|---|---|
WISP-Random (329-config tuning) | 70.20 | 77.28 | 70.88 | 28.25 | 232.4 h | 0.31 h | 68.5 GiB |
Select5 + switch (5-recipe menu) | 69.60 | 76.63 | 70.30 | 26.94 | 1.25 h | 0.51 h | 9.0 GiB |
WISP-Evolution (329-config tuning) | 69.59 | 76.84 | 70.17 | 27.96 | 424.0 h | 0.71 h | 79.2 GiB |
WISP-CIS-Funnel (subsampled CIS search) | 69.22 | 76.11 | 70.01 | 26.27 | 7.66 h tot | incl. | 10.6 GiB |
WISP-CIS (+ HMM-smoothing decoding) | 66.14 | 74.07 | 66.04 | 24.78 | 0 h | 0.35 h | 9.0 GiB |
HIVE-COTE 2.0 (120-min, 32 cores) | 61.68 | 72.13 | 61.91 | 18.04 | 0 h | 448.0 h | 295.0 GiB |
RDST (tuned) | 61.44 | 69.91 | 63.80 | 21.16 | 92.5 h | 34.15 h | 124.4 GiB |
MR-Hydra (tuned) | 60.21 | 68.62 | 61.38 | 20.53 | 119.6 h | 45.52 h | 240.4 GiB |
MiniRocket (tuned) | 59.40 | 69.52 | 60.00 | 16.13 | 35.5 h | 15.53 h | 49.5 GiB |
MultiRocket (tuned) | 59.39 | 68.11 | 60.40 | 19.72 | 201.3 h | 84.63 h | 150.2 GiB |
H-InceptionTime (deep) | 57.63 | 66.77 | 59.83 | 19.13 | 0 h | 102.4 h | A100 GPU |
HARNet (fine-tuned SSL) | 54.78 | 61.69 | 59.36 | 21.30 | 34.5 h | 16.89 h | A100 GPU |
Macro-F1 ^ Pareto Frontier: Accuracy vs. Tune + Training CPU Time & RAM (9 Cohorts)
70.2% | * WISP-Random (70.20 | 232.4h + 0.31h, 68.5 GiB)
69.6% | * Select5 (69.60 | 1.25h + 0.51h, 9.0 GiB)
69.2% | * WISP-CIS-Funnel (subsampled search on WISP-CIS, 69.22 | 7.66h, 10.6 GiB)
66.1% | * WISP-CIS (+ HMM-smoothing decoding, 66.14 | 0h + 0.35h, 9.0 GiB)
61.7% | o HIVE-COTE 2.0 (61.68 | 448h, 295 GiB)
61.4% | o RDST tuned (61.44 | 92.5h + 34.15h, 124.4 GiB)
60.2% | o MR-Hydra tuned (60.21 | 119.6h + 45.52h, 240.4 GiB)
+-+--+--+----+---+--+--> Tune + Training CPU core-h (log scale)
0.3h 2h 8h 127h 233h 448h
For W2 and Q2, following your suggestion, we ran a new experiment where Select5 chooses on validation whether to apply HMM-smoothing temporal decoding. We close nearly the whole gap to full search and match evolutionary search.
| Method | 9 Primary Datasets | HARMES | Other 8 Datasets | Configs Fit |
|---|---|---|---|---|
| WISP-Random (329-config tuning) | 70.20 | 44.84 | 73.37 | 329 |
| Select5 (val. recipe + decoding on/off) | 69.60 | 44.43 | 72.74 | 5 |
| WISP-Evolution (evolutionary) | 69.59 | 43.86 | 72.81 | 329 |
| Select5 (always HMM-smoothing) | 67.10 | 23.51 | 72.55 | 5 |
When we let Select5 turn off HMM-smoothing temporal decoding on validation, we fix HARMES and leave only a small, non-significant gap to 329-config search.
Cookbook insights and open-source package: We will release WristHARBench and WISP, an open-source representation framework that is memory-efficient, fast, highly accurate and extendable architecture to outperform powerful HAR baselines and a large number of methods ported from TSC..
We will revise all the points mentioned above to improve the readability of the article, enhance the explanation of the design rationale and differences in the Methods section, and improve the organisation of the experimental results. If these explanations and the new experimental results have addressed your concerns and you feel they are valuable to the HAR community, we would greatly appreciate it if you would consider raising your rating.
I appreciate the authors' rebuttal during the discussion period. The new analysis and ablations help clarify and strengthen the presentation. However, taking into account the overall novelty and contribution, I feel my initial rating already represents the upper bound for this submission. Therefore, I prefer to maintain my score.
Thank you for your feedback. I would appreciate it if you could consider raise the rating from the perspective of the value it brings in terms of new insights and performance efficiency. For Q1, We have further developed the HMM smooth decoding into a component that can be enabled or disabled based on validation results, and have also adapted it for several of the strongest baselines. Our work organizes time-series classification and HAR representations into one systematic cookbook and answers key questions on large-scale wrist data with new insight.
1. Symmetric Q1 Controls: Always-On HMM-Smoothing (0 Tuning) and Validation HMM On/Off Check
HMM-smoothing temporal decoding itself has zero tuned hyperparameters (transitions come from training label counts). Across 9 cohorts (3 seeds), we report both (a) 0-tuning always-on HMM-smoothing (Tune Time = 0 h, no validation check) and (b) your exact Q1 setting where 4-config tuned baselines and WISP share the same validation binary HMM on/off check (direct vs. HMM-smoothing, single-core hours on one Intel Xeon 8-series CPU):
| Method (9 Cohorts, Symmetric Decoding) | Macro-F1 | Acc | Bal-Acc | Worst-F1 | Tune | Train | RAM |
|---|---|---|---|---|---|---|---|
WISP-CIS (0 tune, always HMM) | 66.14 | 74.07 | 66.04 | 24.78 | 0 h | 0.35 h | 9.0 GiB |
HIVE-COTE 2.0 (0 tune, always HMM) | 65.47 | 78.26 | 65.37 | 23.91 | 0 h | 448 h | 295 GiB |
MR-Hydra (default, always HMM) | 62.67 | 74.18 | 62.18 | 24.79 | 0 h | 91.01 h | 240.4 GiB |
MultiRocket (default, always HMM) | 62.48 | 73.95 | 61.93 | 24.61 | 0 h | 74.55 h | 183.0 GiB |
RDST (default, always HMM) | 60.03 | 73.16 | 59.81 | 21.74 | 0 h | 33.15 h | 131.2 GiB |
WISP-Random (329-config repr. tuning) | 70.20 | 77.28 | 70.88 | 28.25 | 232.4 h | 0.31 h | 68.5 GiB |
Select5 (5-menu + val. HMM on/off) | 69.60 | 76.63 | 70.30 | 26.94 | 1.25 h | 0.51 h | 9.0 GiB |
WISP-CIS-Funnel (subsampled CIS search) | 69.22 | 76.11 | 70.01 | 26.27 | 7.66 h tot | incl. | 10.6 GiB |
WISP-CIS (fixed + val. HMM on/off) | 68.54 | 75.41 | 69.37 | 26.78 | 0.31 h | 0.35 h | 9.0 GiB |
RDST (tuned + val. HMM on/off) | 67.07 | 74.93 | 68.21 | 25.64 | 92.5 h | 34.15 h | 124.4 GiB |
MR-Hydra (tuned + val. HMM on/off) | 66.57 | 74.75 | 67.11 | 27.82 | 119.6 h | 45.52 h | 240.4 GiB |
MultiRocket (tuned + val. HMM on/off) | 66.52 | 74.45 | 66.06 | 26.95 | 201.3 h | 84.63 h | 150.2 GiB |
MiniRocket (tuned + val. HMM on/off) | 64.03 | 73.59 | 64.08 | 22.73 | 35.5 h | 15.53 h | 49.5 GiB |
At 0 h tuning, WISP-CIS already beats every baseline with always-on HMM-smoothing. When both sides also use your Q1 validation HMM on/off check, external transforms jump substantially and WISP still leads at far lower compute and RAM.
2. Representation Novelty Beyond SAX and catch22 (W1)
Our earlier label ("Adapted From SAX, catch22") understated what is new in S and early linear conditioning, tested under one shared Ridge head with no decoding :
Direct Window Test (Shared Ridge, No Decoding) | Macro-F1 | Comparison |
|---|---|---|
R + I + standard SAX word-count histogram | 56.59 | -1.05 vs. R + I (57.64) |
R + I + catch22-style 3-state transitions | 59.13 | +1.49 vs. R + I |
R + I + WISP 6-state asymmetric transitions | 60.07 | +2.43 vs. R + I; beats SAX on 9/9 |
R / I / S validation-weighted 3-head late fusion | 59.90 | +2.26 vs. R + I |
R + I + S single-Ridge early fusion (WISP-CIS) | 61.12 | +3.48 vs. R + I; beats late fusion on 8/9 |
MiniRocket (60.26) -> MiniRocket + I + S | 62.13 | +1.87; improves MiniRocket on 9/9 |
Standard SAX word counts hurt accuracy because bag-of-words histograms drop forward-versus-reverse posture order across words. Our asymmetric transition matrix beats SAX on all nine cohorts, and our representation also improve MiniRocket and fine-tuned HARNet on all nine cohorts.
3. Efficiency Contribution
National wrist cohorts such as NHANES, UK Biobank (over 100,000 participants, 7 days: over 4,000x CAPTURE-24 in participant-days), and All of Us reach tens of TB. Even on CAPTURE-24 (151 participants, 1 day), WHAR Arena excludes it and strong baselines OOM on 200 GiB nodes (MR-Hydra: 240.4 GiB; HIVE-COTE 2.0: 295 GiB). Scaling them to national cohorts requires multi-terabyte to tens-of-terabytes HPC systems that standard scientific workstations cannot run directly. With much lower memory requirements (9.0 GiB RAM), fast execution (within minutes on 16 CPU cores), and flexible, modular components that extend easily (plus open-source WristHARBench), WISP scales smoothly to much larger wrist cohorts.
Response to Reviewer pwBJ (1/2): Cookbook positioning, cross-domain MTSC adaptation, and new S sub-component ablations (W1)
Thank you for appreciating our wrist-only benchmark (
WristHARBench) and ourR+Sresults across all nine datasets, and for flagging presentation issues in our submission. Because we compressed both a benchmark and a method study into nine pages, we under-explained our motivation and positioning. Below we clarify our positioning and present new controlled experiments directly (Part 1 coversW1; Part 2 coversW2,Q1, andQ2).For W1, our work is a representation cookbook for wrist HAR. We brought modern time-series classification (TSC) methods into wrist HAR, adapted them so that they run fast in small memory on large cohorts, and built standalone
WISP-CIS.WristHARBench: Most multi-dataset HAR benchmarks mix sensors across the body, and benchmarks likeWHAR Arena(Burzer et al., 2026) explicitly exclude large cohorts such asCAPTURE-24because of dataset size and GPU cost. They also test mostly neural networks or random forests, and rarely compare modern TSC methods. We define single-wrist HAR as a multichannel TSC problem, which lets us keepCAPTURE-24in our nine primary wrist cohorts and bring modern TSC families (ROCKET,MultiRocket,MR-Hydra,RDST,CIF,QUANT,HIVE-COTE 2.0,TimesNet,PatchTST) into one participant-disjoint benchmark.WISPandWISP-CIS: Existing strong TSC methods that use ten thousand wide features or four heavy ensembles. On large wrist cohorts, they demand extreme memory: in our runs,HIVE-COTE 2.0and tunedMR-Hydrastill ran out of memory at200 GiBRAM and reached240.4-295 GiBpeak RAM. So we adapted each method to short three-axis wrist windows and kept all blocks and variants as a cookbook so readers can see what works and why. For main comparisons, we useWISP-CISas our standalone method, which concatenatesR,I, andSbefore one balanced linearRidgeclassifier with HMM-smoothing temporal decoding:R)ROCKET,MultiRocketI)CIF,QUANTS)SAX,catch22,MTFRidge.S)HIVE-COTE 2.0R,I, andSbefore one balanced linearRidgehead, which conditions cross-block features in one fast pass.CAPTURE-24(RF+HMM)Ridgescores in milliseconds.Following your comment on
S, we ran new controlled experiments that separate the four parts insideSunder direct prediction (sharedRidgehead, three seeds, no decoding), with time-shuffling and equal-width controls:Ridge, No Decoding)RaloneR)R+ quantile-state transitionsR+ spectral bandsR+ multi-lag autocorrelationR+ cross-axis correlationR+ time-shuffled state transitionsRS(fullS)RSwithout state transitionsRS)RI(57.64) + equal-width extraRfeaturesRIRI(57.64) + equal-width MiniRocket featuresRIRI(57.64) + equal-widthS(WISP-CIS)RIWithin-window state transitions give the largest gain and beat equal-width random convolutions. When we shuffle time order inside each window, state transitions lose about three-quarters of their gain.