acceptodds
Under review as a conference paper at ICLR 2027

WISP: Composable Representations for Wrist-Worn Activity Recognition

Abstract

Smartwatches and fitness bands are among the most common wearable devices, and large national health studies record movement at the wrist. Yet existing human activity recognition benchmarks mix sensors worn on different parts of the body, so they do not show how well activities can be recognised from the wrist alone. We introduce WristHARBench, a benchmark that standardises participant-disjoint evaluation across nine wrist-acceleration datasets and two additional multichannel inertial datasets. We hypothesise that different human activities leave complementary patterns in wrist motion, such as local shape, rhythm, temporal order and cross-axis relationships, and propose WISP (Wrist-Informed Space of Predictive Representations), a composable representation space that combines representations of these patterns with compatible classifiers. Across the nine primary datasets, WISP with random search achieves 70.20% mean Macro-F1, compared with 59.62% for the strongest of 18 external baselines. Controlled ablations and baseline comparisons indicate that recognition improves mainly when a representation adds motion patterns that the others lack, rather than when it represents the same pattern in a different way.

Primary area:
learning on time series and dynamical systems
Keywords:
human activity recognition, wearable sensing, wrist accelerometry, time-series classification, benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_pwBJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces WristHARBench, a common participant-disjoint evaluation protocol for wrist activity recognition, and WISP, a configurable framework combining several time-series representations, standard classifiers, and optional Viterbi decoding. The study examines representation combinations, temporal decoding, and configuration search across nine primary datasets.

Critical strengths

Wrist-only recognition is a useful practical setting, and a common evaluation protocol makes the results easier to compare. I also appreciate the controlled experiments: all representation combinations are evaluated with and without decoding, and random and evolutionary search are compared at equal candidate budgets. In particular, combining R and S improves direct prediction over either component alone on all nine datasets. This is meaningful evidence that the feature blocks complement each other, independently of Viterbi decoding.

Critical weaknesses
  1. The novelty is mostly in the particular feature combination and empirical study. The underlying components (e.g., random convolutions, interval statistics, spectral and symbolic descriptors, standard classifiers, and Viterbi decoding) are established. WISP offers a useful wrist-HAR recipe, but I am less convinced that it establishes a substantially new representation idea. The R/S results are the strongest contribution here. However, S bundles spectral, autocorrelation, state-transition, and cross-axis features, without separating them.

  2. WISP searches 329 configurations, while strong baselines such as RDST and MR-Hydra use mostly fixed settings. Giving these baselines similar tuning and decoder choices would make the comparison more convincing. Also, Select5 always use Viterbi, while full search can turn it off. Most of the search gain over Select5 comes from HARMES, where the selected models disable Viterbi. A simple selector that also chooses whether to decode would help show what the broader search adds.

Questions for authors
  1. How do the strongest external baselines perform with the same direct/Viterbi selection and a comparable tuning budget?

  2. How much of the full-search advantage remains when the fixed-recipe selector can also choose the decoder, and when decoding respects recording boundaries and gaps?

4 comments
AuthorsOpenReview ↗

Response to Reviewer pwBJ (1/2): Cookbook positioning, cross-domain MTSC adaptation, and new S sub-component ablations (W1)

Thank you for appreciating our wrist-only benchmark (WristHARBench) and our R+S results across all nine datasets, and for flagging presentation issues in our submission. Because we compressed both a benchmark and a method study into nine pages, we under-explained our motivation and positioning. Below we clarify our positioning and present new controlled experiments directly (Part 1 covers W1; Part 2 covers W2, Q1, and Q2).

For W1, our work is a representation cookbook for wrist HAR. We brought modern time-series classification (TSC) methods into wrist HAR, adapted them so that they run fast in small memory on large cohorts, and built standalone WISP-CIS.

  1. How to read WristHARBench: Most multi-dataset HAR benchmarks mix sensors across the body, and benchmarks like WHAR Arena (Burzer et al., 2026) explicitly exclude large cohorts such as CAPTURE-24 because of dataset size and GPU cost. They also test mostly neural networks or random forests, and rarely compare modern TSC methods. We define single-wrist HAR as a multichannel TSC problem, which lets us keep CAPTURE-24 in our nine primary wrist cohorts and bring modern TSC families (ROCKET, MultiRocket, MR-Hydra, RDST, CIF, QUANT, HIVE-COTE 2.0, TimesNet, PatchTST) into one participant-disjoint benchmark.
  2. How to read WISP and WISP-CIS: Existing strong TSC methods that use ten thousand wide features or four heavy ensembles. On large wrist cohorts, they demand extreme memory: in our runs, HIVE-COTE 2.0 and tuned MR-Hydra still ran out of memory at 200 GiB RAM and reached 240.4-295 GiB peak RAM. So we adapted each method to short three-axis wrist windows and kept all blocks and variants as a cookbook so readers can see what works and why. For main comparisons, we use WISP-CIS as our standalone method, which concatenates R, I, and S before one balanced linear Ridge classifier with HMM-smoothing temporal decoding:
Cookbook BlockAdapted FromHow We Adapted It for Wrist Signals
Local shape (R)ROCKET, MultiRocketWe use compact multi-axis dilated kernels with four summary stats, which keeps the feature matrix small and cuts memory and training time.
Interval stats (I)CIF, QUANTWe compute moment, trend, and quantile stats on sub-intervals so that we capture posture directly in linear time.
Directed states (S)SAX, catch22, MTFWe map eight views into six quantile states and feed the full asymmetric transition matrix + entropies to Ridge.
Rhythm & axes (S)Inertial featuresWe extract spectral-band energy/entropy/frequency, multi-lag autocorrelation, and cross-axis correlation.
Early linear fusionHIVE-COTE 2.0We concatenate R, I, and S before one balanced linear Ridge head, which conditions cross-block features in one fast pass.
Smoothing decodingCAPTURE-24 (RF+HMM)We add optional HMM-smoothing temporal decoding directly over continuous Ridge scores in milliseconds.

Following your comment on S, we ran new controlled experiments that separate the four parts inside S under direct prediction (shared Ridge head, three seeds, no decoding), with time-shuffling and equal-width controls:

New Direct-Prediction Ablation (Shared Ridge, No Decoding)Macro-F1Diff. vs. Ref.
Base R alone57.28Ref. (R)
R + quantile-state transitions59.76+2.48
R + spectral bands58.73+1.45
R + multi-lag autocorrelation58.26+0.98
R + cross-axis correlation57.35+0.07
R + time-shuffled state transitions57.93+0.65 (-1.83 vs. ordered)
RS (full S)60.87+3.59
RS without state transitions59.09+1.81 (-1.78 vs. RS)
RI (57.64) + equal-width extra R features58.30+0.66 vs. RI
RI (57.64) + equal-width MiniRocket features59.45+1.81 vs. RI
RI (57.64) + equal-width S (WISP-CIS)61.12+3.48 vs. RI

Within-window state transitions give the largest gain and beat equal-width random convolutions. When we shuffle time order inside each window, state transitions lose about three-quarters of their gain.

AuthorsOpenReview ↗

Response to Reviewer pwBJ (2/2): search as representation tuning, 4-metric Pareto frontier, Select5 HMM choice, and open-source Cookbook (W2, Q1, Q2)

For W2 and Q1

HMM-smoothing temporal decoding runs Viterbi smoothing over window scores with zero extra training in WISP. Across 9 primary cohorts (3 seeds), we tune wide TSC baselines over a 4-setting validation grid (Tune Time) and refit them (Training Time), alongside standalone and tuned WISP (including WISP-CIS-Funnel, subsampled search on WISP-CIS). CPU times report standardized single-core hours on one Intel Xeon 8-series CPU for fair comparison (running on multiple CPU cores in parallel finishes much faster in wall-clock time), and we convert NVIDIA A100 GPU hours to CPU core-equivalent hours by rated power (1 A100 GPU h = 64 Xeon 8570 core-h, 400 W vs. 6.25 W/core):

Method (9 Cohorts, 3-Seed )Macro-F1Acc.Bal. Acc.Worst F1Tune TimeTraining TimePeak RAM
WISP-Random (329-config tuning)70.2077.2870.8828.25232.4 h0.31 h68.5 GiB
Select5 + switch (5-recipe menu)69.6076.6370.3026.941.25 h0.51 h9.0 GiB
WISP-Evolution (329-config tuning)69.5976.8470.1727.96424.0 h0.71 h79.2 GiB
WISP-CIS-Funnel (subsampled CIS search)69.2276.1170.0126.277.66 h totincl.10.6 GiB
WISP-CIS (+ HMM-smoothing decoding)66.1474.0766.0424.780 h0.35 h9.0 GiB
HIVE-COTE 2.0 (120-min, 32 cores)61.6872.1361.9118.040 h448.0 h295.0 GiB
RDST (tuned)61.4469.9163.8021.1692.5 h34.15 h124.4 GiB
MR-Hydra (tuned)60.2168.6261.3820.53119.6 h45.52 h240.4 GiB
MiniRocket (tuned)59.4069.5260.0016.1335.5 h15.53 h49.5 GiB
MultiRocket (tuned)59.3968.1160.4019.72201.3 h84.63 h150.2 GiB
H-InceptionTime (deep)57.6366.7759.8319.130 h102.4 hA100 GPU
HARNet (fine-tuned SSL)54.7861.6959.3621.3034.5 h16.89 hA100 GPU
Macro-F1 ^ Pareto Frontier: Accuracy vs. Tune + Training CPU Time & RAM (9 Cohorts)
  70.2% |                * WISP-Random (70.20 | 232.4h + 0.31h, 68.5 GiB)
  69.6% |    * Select5 (69.60 | 1.25h + 0.51h, 9.0 GiB)
  69.2% |       * WISP-CIS-Funnel (subsampled search on WISP-CIS, 69.22 | 7.66h, 10.6 GiB)
  66.1% | * WISP-CIS (+ HMM-smoothing decoding, 66.14 | 0h + 0.35h, 9.0 GiB)
  61.7% |                   o HIVE-COTE 2.0 (61.68 | 448h, 295 GiB)
  61.4% |            o RDST tuned (61.44 | 92.5h + 34.15h, 124.4 GiB)
  60.2% |              o MR-Hydra tuned (60.21 | 119.6h + 45.52h, 240.4 GiB)
        +-+--+--+----+---+--+--> Tune + Training CPU core-h (log scale)
         0.3h 2h 8h 127h 233h 448h

For W2 and Q2, following your suggestion, we ran a new experiment where Select5 chooses on validation whether to apply HMM-smoothing temporal decoding. We close nearly the whole gap to full search and match evolutionary search.

Method9 Primary DatasetsHARMESOther 8 DatasetsConfigs Fit
WISP-Random (329-config tuning)70.2044.8473.37329
Select5 (val. recipe + decoding on/off)69.6044.4372.745
WISP-Evolution (evolutionary)69.5943.8672.81329
Select5 (always HMM-smoothing)67.1023.5172.555

When we let Select5 turn off HMM-smoothing temporal decoding on validation, we fix HARMES and leave only a small, non-significant gap to 329-config search.

Cookbook insights and open-source package: We will release WristHARBench and WISP, an open-source representation framework that is memory-efficient, fast, highly accurate and extendable architecture to outperform powerful HAR baselines and a large number of methods ported from TSC..

We will revise all the points mentioned above to improve the readability of the article, enhance the explanation of the design rationale and differences in the Methods section, and improve the organisation of the experimental results. If these explanations and the new experimental results have addressed your concerns and you feel they are valuable to the HAR community, we would greatly appreciate it if you would consider raising your rating.

I appreciate the authors' rebuttal during the discussion period. The new analysis and ablations help clarify and strengthen the presentation. However, taking into account the overall novelty and contribution, I feel my initial rating already represents the upper bound for this submission. Therefore, I prefer to maintain my score.

Thank you for your feedback. I would appreciate it if you could consider raise the rating from the perspective of the value it brings in terms of new insights and performance efficiency. For Q1, We have further developed the HMM smooth decoding into a component that can be enabled or disabled based on validation results, and have also adapted it for several of the strongest baselines. Our work organizes time-series classification and HAR representations into one systematic cookbook and answers key questions on large-scale wrist data with new insight.

1. Symmetric Q1 Controls: Always-On HMM-Smoothing (0 Tuning) and Validation HMM On/Off Check

HMM-smoothing temporal decoding itself has zero tuned hyperparameters (transitions come from training label counts). Across 9 cohorts (3 seeds), we report both (a) 0-tuning always-on HMM-smoothing (Tune Time = 0 h, no validation check) and (b) your exact Q1 setting where 4-config tuned baselines and WISP share the same validation binary HMM on/off check (direct vs. HMM-smoothing, single-core hours on one Intel Xeon 8-series CPU):

Method (9 Cohorts, Symmetric Decoding)Macro-F1AccBal-AccWorst-F1TuneTrainRAM
WISP-CIS (0 tune, always HMM)66.1474.0766.0424.780 h0.35 h9.0 GiB
HIVE-COTE 2.0 (0 tune, always HMM)65.4778.2665.3723.910 h448 h295 GiB
MR-Hydra (default, always HMM)62.6774.1862.1824.790 h91.01 h240.4 GiB
MultiRocket (default, always HMM)62.4873.9561.9324.610 h74.55 h183.0 GiB
RDST (default, always HMM)60.0373.1659.8121.740 h33.15 h131.2 GiB
WISP-Random (329-config repr. tuning)70.2077.2870.8828.25232.4 h0.31 h68.5 GiB
Select5 (5-menu + val. HMM on/off)69.6076.6370.3026.941.25 h0.51 h9.0 GiB
WISP-CIS-Funnel (subsampled CIS search)69.2276.1170.0126.277.66 h totincl.10.6 GiB
WISP-CIS (fixed + val. HMM on/off)68.5475.4169.3726.780.31 h0.35 h9.0 GiB
RDST (tuned + val. HMM on/off)67.0774.9368.2125.6492.5 h34.15 h124.4 GiB
MR-Hydra (tuned + val. HMM on/off)66.5774.7567.1127.82119.6 h45.52 h240.4 GiB
MultiRocket (tuned + val. HMM on/off)66.5274.4566.0626.95201.3 h84.63 h150.2 GiB
MiniRocket (tuned + val. HMM on/off)64.0373.5964.0822.7335.5 h15.53 h49.5 GiB

At 0 h tuning, WISP-CIS already beats every baseline with always-on HMM-smoothing. When both sides also use your Q1 validation HMM on/off check, external transforms jump substantially and WISP still leads at far lower compute and RAM.

2. Representation Novelty Beyond SAX and catch22 (W1)

Our earlier label ("Adapted From SAX, catch22") understated what is new in S and early linear conditioning, tested under one shared Ridge head with no decoding :

Direct Window Test (Shared Ridge, No Decoding)Macro-F1Comparison
R + I + standard SAX word-count histogram56.59-1.05 vs. R + I (57.64)
R + I + catch22-style 3-state transitions59.13+1.49 vs. R + I
R + I + WISP 6-state asymmetric transitions60.07+2.43 vs. R + I; beats SAX on 9/9
R / I / S validation-weighted 3-head late fusion59.90+2.26 vs. R + I
R + I + S single-Ridge early fusion (WISP-CIS)61.12+3.48 vs. R + I; beats late fusion on 8/9
MiniRocket (60.26) -> MiniRocket + I + S62.13+1.87; improves MiniRocket on 9/9

Standard SAX word counts hurt accuracy because bag-of-words histograms drop forward-versus-reverse posture order across words. Our asymmetric transition matrix beats SAX on all nine cohorts, and our representation also improve MiniRocket and fine-tuned HARNet on all nine cohorts.

3. Efficiency Contribution

National wrist cohorts such as NHANES, UK Biobank (over 100,000 participants, 7 days: over 4,000x CAPTURE-24 in participant-days), and All of Us reach tens of TB. Even on CAPTURE-24 (151 participants, 1 day), WHAR Arena excludes it and strong baselines OOM on 200 GiB nodes (MR-Hydra: 240.4 GiB; HIVE-COTE 2.0: 295 GiB). Scaling them to national cohorts requires multi-terabyte to tens-of-terabytes HPC systems that standard scientific workstations cannot run directly. With much lower memory requirements (9.0 GiB RAM), fast execution (within minutes on 16 CPU cores), and flexible, modular components that extend easily (plus open-source WristHARBench), WISP scales smoothly to much larger wrist cohorts.

Reviewer_QXwmOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper proposes WristHARBench, a unified evaluation benchmark for human activity recognition (HAR) using wrist-worn accelerometers, and WISP, a framework that combines multiple feature representations. WISP extracts random convolution, interval statistics, spectral, temporal, and cross-axis features, and performs activity recognition using conventional machine learning classifiers such as Ridge and ExtraTrees. The framework also incorporates Viterbi-based temporal post-processing and configuration selection based on validation data. Experiments on nine primary datasets compare the proposed approach against 18 existing methods, reporting a mean Macro-F1 of 70.20% for WISP-Random. The paper further investigates the complementarity of different feature representations through ablation studies.

Critical strengths

S1. Unified evaluation benchmark: The paper focuses on wrist-worn accelerometers and establishes a unified participant-disjoint evaluation protocol across nine datasets. This provides a useful benchmark for evaluating existing methods under consistent experimental conditions.

S2. Extensive comparisons and ablation studies: The paper compares the proposed approach with 18 existing methods, including deep learning approaches, and provides detailed analyses of feature combinations and temporal post-processing. In particular, the investigation of the complementarity between random convolution and spectral, temporal, and cross-axis features across multiple datasets is valuable.

Critical weaknesses

W1. Limited methodological novelty and scientific contribution: WISP primarily combines existing feature extraction techniques with conventional machine learning classifiers, without introducing a new representation learning algorithm or classification principle. Combining heterogeneous feature representations has already been extensively investigated in prior work, such as HIVE-COTE 2.0. Similarly, unified participant-disjoint HAR benchmarks have been studied in prior work, including WHAR Arena. Although the wrist-specific evaluation is valuable, the methodological and scientific contributions beyond existing research are not sufficiently clear.

W2. The performance comparison in Table 3 does not sufficiently establish the superiority of the proposed feature representations: Although WISP-Random outperforms the strongest existing baseline by 10.58 percentage points in Table 3, this result includes the effects of Viterbi-based temporal post-processing and validation-based configuration search. In fact, fixed WISP-CIS with direct prediction achieves a Macro-F1 of 61.12%, only 1.50 percentage points higher than the strongest existing baseline (59.62%). Moreover, most external baselines are not provided with comparable temporal post-processing or configuration search. Therefore, it is difficult to attribute the substantial improvements in Table 3 to the proposed feature representations themselves, raising concerns about the fairness of the comparison.

W3. Concerns about the configurations of some deep learning baselines: For example, HARNet uses a frozen pretrained encoder and zero-pads 5-second inputs to fit a model designed for 10-second recordings. Similarly, TF-C uses only one of the three accelerometer channels. It is unclear whether these configurations adequately reflect the capabilities of the existing methods. Further evaluation is needed to determine whether the proposed approach maintains its advantages under more appropriate fine-tuning and input configurations.

W4. Insufficient analysis of the novelty of the empirical findings and the reasons for the reported performance improvements: The paper presents several useful empirical findings, including the complementarity between random convolution and spectral, temporal, and cross-axis features, as well as the condition-dependent effectiveness of temporal post-processing. However, the complementarity of heterogeneous features is already well established in prior research, and the significance of the new scientific insights specific to wrist-based HAR is not sufficiently clarified. In particular, the individual contributions of spectral features, autocorrelation, state transitions, and cross-axis relationships are not separately analyzed, leaving the underlying reasons for their complementarity unclear. Furthermore, the paper provides insufficient analysis of why relatively simple feature extraction techniques combined with conventional machine learning classifiers outperform many deep learning approaches, particularly in relation to dataset characteristics and differences from existing methods. A more thorough discussion is needed to clarify how these findings advance our understanding beyond prior work on feature fusion.

Questions for authors

Q1. Novelty and differences from prior work: What are the fundamental methodological contributions of WISP compared with existing approaches that combine heterogeneous feature representations, such as HIVE-COTE? In particular, what new insights into feature complementarity for wrist-based HAR does this study provide that could not be obtained from prior research?

Q2. Fairness of performance comparisons: Would the performance advantages reported in Table 3 remain if comparable Viterbi-based temporal post-processing and validation-based model selection were also applied to RDST, MR-Hydra, and major deep learning baselines? In particular, how can the improvements attributable to the feature representations themselves be distinguished from those arising from post-processing and configuration search?

Q3. Deep learning baseline configurations: Why was HARNet evaluated without fine-tuning the entire encoder or using a model appropriate for the input sequence length? Similarly, why was only a single accelerometer channel provided to TF-C? Would the main conclusions remain valid under more appropriate configurations for these baselines?

Q4. New scientific insights and factors underlying performance improvements: The paper demonstrates the complementarity between R and S representations. How do these findings extend existing knowledge from prior research on heterogeneous feature fusion? Furthermore, considering the individual contributions of the components within S, feature dimensionality, and dataset characteristics, to what extent can the current experimental results explain why relatively simple feature extraction techniques combined with conventional machine learning classifiers outperform existing deep learning approaches?

6 comments
AuthorsOpenReview ↗

Response to Reviewer QXwm (1/3): wrist-customized representations vs. HIVE-COTE 2.0, WHAR Arena, MTSC transfer, and open-source cookbook (W1, Q1)

We thank you for your careful review, for noting the value of WristHARBench (S1) and our complementarity study (S2), and for pointing out where our presentation was unclear. Because we squeezed both a benchmark and a method study into 9 pages, we compressed our motivation and did not highlight our gains in speed, memory, and all four metrics. We clarify our two main contributions and present our new experiments directly below (Part 1: positioning, W1, Q1; Part 2: W2, Q2; Part 3: W3, Q3, W4, Q4):

  1. How to understand WristHARBench: Existing multi-dataset HAR benchmarks mix sensors across the body, and WHAR Arena (Burzer et al., 2026, Sec. 4.2) explicitly excludes large free-living cohorts like CAPTURE-24 because of dataset size and GPU budget. Prior HAR benchmarks also focus on neural models or random forests and rarely test many modern time-series classification (TSC) methods. We frame single-wrist HAR as multichannel TSC, which lets us bring modern TSC families into one wrist-only benchmark (retaining CAPTURE-24) alongside deep and self-supervised baselines.
  2. **How to understand WISP and primary WISP-CIS: On large, imbalanced wrist cohorts, these baselines hit out-of-memory crashes even at 200 GiB RAM, and require 240.4 GiB (MR-Hydra) to 295.0 GiB (HIVE-COTE 2.0) of server memory plus hundreds of CPU hours. We therefore built WISP as a modular representation cookbook for wrist signals. We adapt each block to short three-axis wrist windows and concatenate them before one class-balanced linear classifier to form WISP-CIS:
Cookbook BlockPrior Method We AdaptedHow We Adapted It for Wrist Signals
Local shape (R)ROCKET, MultiRocketWe use compact multi-axis dilated kernels with four summary stats, which cuts memory and solver time sharply on large cohorts.
Interval stats (I)CIF, QUANTWe compute trend, moment, and quantile stats on multi-scale sub-intervals so that we capture posture shifts directly in linear time.
State & rhythm (S)SAX, catch22, MTF, inertialWe map signals into six normalized states to track forward vs. reverse motion order (sit-to-stand vs. stand-to-sit), rhythm spectra, and axis stats.
Early linear fusionHIVE-COTE 2.0 (4 ensembles)We concatenate R, I, and S before one balanced Ridge linear head, which keeps training fast and allows clean ablations.
Sequence & tuningCAPTURE-24 (RF+HMM), HPOWe add optional HMM-smoothing temporal decoding on continuous Ridge window scores and lightweight validation tuning.

**For W1 and Q1, Unlike HIVE-COTE 2.0 averages four heavy ensembles 4 models (STC, DrCIF, Arsenal, TDE), and WISP-CIS puts compact wrist representations into one linear model ,WISP-Random (329-config tuning) &WISP-Evolution (329-config tuning) can be regard as a kinds of representation tuning, just search the final representations fusion then input one model not ensemble. The resulst list below.

AuthorsOpenReview ↗

Response to Reviewer QXwm (2/3): search as representation tuning, matched-baseline comparisons, and speed-memory-accuracy Pareto frontier (W2, Q2)

For W2 and Q2, WISP-CIS without HMM decoding is close to RDST because we brought strong TSC baselines into wrist HAR that already beat standard HAR deep models.

1. (Four Metrics, Tune Time, Training Time & Peak RAM)

Method ( 9 Cohorts)Macro-F1AccBal-AccWorst-F1Tune TimeTraining TimePeak RAM
WISP-CIS (+ HMM-smoothing decoding)66.1474.0766.0424.780 h0.35 h9.0 GiB
Select5 + switch (5-menu)69.6076.6370.3026.941.25 h0.51 h9.0 GiB
WISP-CIS-Funnel (subsampled CIS search)69.2276.1170.0126.277.66 h totincl.10.6 GiB
WISP-Random (repr. tuning)70.2077.2870.8828.25232.4 h0.31 h68.5 GiB
WISP-Evolution (329-config tuning)69.5976.8470.1727.96424.0 h0.71 h79.2 GiB
HIVE-COTE 2.0 (120-min, 4 ensembles)61.6872.1361.9118.040 h448.0 h295.0 GiB
RDST (tuned)61.4469.9163.8021.1692.5 h34.15 h124.4 GiB
MR-Hydra (tuned)60.2168.6261.3820.53119.6 h45.52 h240.4 GiB
MiniRocket (tuned)59.4069.5260.0016.1335.5 h15.53 h49.5 GiB
MultiRocket (tuned)59.3968.1160.4019.72201.3 h84.63 h150.2 GiB
H-InceptionTime (deep)57.6366.7759.8319.130 h102.4 hA100 GPU
Attend-Disc. (deep)56.8664.5763.0023.480 h489.4 hA100 GPU
HARNet (fine-tuned SSL)54.7861.6959.3621.3034.5 h16.89 hA100 GPU
TF-C (3-channel SSL)47.2155.2151.1415.310 h92.8 hA100 GPU
MethodMacro-F1Acc.Bal. Acc.Worst F1Tune TimeTraining TimePeak RAM
Macro-F1 ^ Pareto Frontier: Accuracy vs. Tune + Training CPU Time & RAM
  70.2% |                   * WISP-Random (70.20 | 232.7h, 68.5 GiB)
  69.6% |     * Select5 + switch (69.60 | 1.76h, 9.0 GiB)
  69.2% |        * WISP-CIS-Funnel (subsampled search on WISP-CIS, 69.22 | 7.66h, 10.6 GiB)
  66.1% | * WISP-CIS (+ HMM-smoothing decoding, 66.14 | 0.35h, 9.0 GiB)
  61.7% |                       o HIVE-COTE 2.0 (61.68 | 448.0h, 295 GiB)
  61.4% |              o RDST tuned (61.44 | 92.5h tune + 34.2h train, 124 GiB)
  60.2% |                o MR-Hydra tuned (60.21 | 119.6h tune + 45.5h train, 240 GiB)
        +-+---+--+-----+----+---+--> Tune + Training CPU core-h (log scale)
         0.3h 2h 8h   130h 230h 450h

I understand the reviewers’ concerns regarding fairness. Firstly, regarding TFC, we used the recommended parameters at first; furthermore, as requested, we have fine-tuned several strong baselines and utilised the corresponding parameters, and we have also observed instances where the TFC input has been modified to a three-channel format and fine-tuned. One point worth noting is that HMM smooth decoding is a component of the WISP framework; consequently, the WISP series can naturally utilise it with ease. However, for many end-to-end deep learning models or powerful TSC baselines, we need to adapt and modify the method. This raises the question: does the inclusion of the HMM decoding module still constitute the original method? This is indeed an issue, and the backbones of deep learning models and many TSC baselines are not consistent. However, to better address the reviewers’ questions regarding the effectiveness of HMM smooth decoding , we have designed rigorous ablation experiments within WISP ; the results of these experiments will be presented in our next reply. In our submission, we ran TF-C on one channel and used frozen probing for HARNet. When we give TF-C all three axes and fine-tune HARNet end-to-end with native 5s/10s checkpoints (picking early-stopping epochs on validation), both improve across all 9 of 9 cohorts.

AuthorsOpenReview ↗

Response to Reviewer QXwm (3/3):

Following your suggestion, we ran a new experiment where Select5 chooses on validation whether to apply HMM-smoothing temporal decoding. We close nearly the whole gap to full search and match evolutionary search.

Method9 Primary DatasetsHARMESOther 8 DatasetsConfigs Fit
WISP-Random (329-config tuning)70.2044.8473.37329
Select5 (val. recipe + decoding on/off)69.6044.4372.745
WISP-Evolution (evolutionary)69.5943.8672.81329
Select5 (always HMM-smoothing)67.1023.5172.555

When we let Select5 turn off HMM-smoothing temporal decoding on validation, we fix HARMES and leave only a small, non-significant gap to 329-config search. For W4 and Q4, we ran new direct-prediction ablations, sample-order shuffling tests, and equal-width controls 3 seeds. They show that forward-vs.-reverse state transitions inside each window drive S, and that WISP-CIS also combines well with fine-tuned HARNet.

Below we separate all four parts inside S and test sample-order shuffling and equal-width controls using one shared Ridge head without HMM-smoothing temporal decoding:

Direct Sub-Block Ablation (No Decoding)Macro-F1Gain vs. RWidth & Sample-Order ControlsMacro-F1Diff.
Base convolution R57.28Ref.Doubled-width R58.74+1.46 vs. R
R + quantile-state transitions59.76+2.48R + time-shuffled transitions57.93-1.83 vs. ordered
R + spectral rhythm bands58.73+1.45R + I baseline57.64Ref.
R + multi-lag autocorrelation58.26+0.98R + I + equal-width extra R58.30-2.82 vs. CIS
R + cross-axis correlation57.35+0.07R + I + equal-width MiniRocket59.45-1.67 vs. CIS
Full R + S60.87+3.59R + I + equal-width S (WISP-CIS)61.12Ref. (CIS)

These controlled tests give three practical insights for wrist HAR:

  1. Motion order inside a window matters: Convolution pooling averages across a window, so it loses motion direction (such as sitting-to-standing vs. standing-to-sitting -- a within-window signal separate from across-window HMM-smoothing temporal decoding). When we shuffle sample order inside each window, roughly three quarters of the state gain disappears across all 9 cohorts. Adding S also beats equal-width extra R or MiniRocket features (and beats catch22 and SAX state counts), while adding I + S to MiniRocket improves all 9 cohorts.
  2. When compact linear features beat deep models: WISP-CIS has a bigger lead over deep models on higher-sampling-rate datasets (which pretrained models downsample to 30 Hz) and smaller cohorts, while its lead over wide ROCKET features grows on larger cohorts.
  3. Pretrained models plug right into our cookbook: When we combine WISP-CIS with end-to-end fine-tuned HARNet using validation weights, the fused model beats both standalone models on all 9 of 9 cohorts.

Cookbook insights and open-source package: We will release WristHARBench and WISP, an open-source representation framework that is memory-efficient, fast, highly accurate and extendable architecture to outperform powerful HAR baselines and a large number of methods ported from TSC.. We will revise all the points mentioned above to improve the readability of the article, enhance the explanation of the design rationale and differences in the Methods section, and improve the organisation of the experimental results. If these explanations and the new experimental results have addressed your concerns and you feel they are valuable to the HAR community, we would greatly appreciate it if you would consider raising your rating.

Reviewer_NsdWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Authors argue existing human activity recognition benchmarks do not provide evaluations using data from wrist worn inertial sensors alone and results use a mix of sensors worn on different parts of the body but wrist worn devices are more common. So the authors: 1) consider providing an activity recognition benchmark based on considering data only from wrist worn sensors and introduce WristHARBench for evaluations with wrist-worn inertial sensing. Then: 2) introduce WISP (Wrist-Informed Space of Predictive Representations), a compositional framework that combines multiple hand-crafted and time-series representations designed to capture complementary motion characteristics. The hypothesis is that wrist activities exhibit multiple complementary motion patterns and that recognition performance improves when they are combined in a complimentary way with relevant classifiers. The author use a mix of engineered features and learned features an evaluate with a mix of different ensemble methods to achieve good performance on each dataset.

Critical strengths
  • It is interesting to see that recognition performance improves with appropriate selection of parameters in ensembles specific to each dataset.
  • identifying complementary motion representations for wrist worn sensor HAR and organizing them into a composable search space.
Critical weaknesses
  • Novelty is low.
    • Then the study is largely a composition of existing representation families in other studies. - The finding that complementarity improves recognition in this domain is not new.
    • General HAR from wrist worn devices alone, without priors, like context (I am going to exercise or run) is notoriously bad, especially when some level of granularity is attempted. While this is not a critical weakness, it does reduce the impact of the work, although the motivation in the first paragraph against a backdrop of data sources is somewhat appealing to elevate the study's significance at first.
    • The main hypothesis, to me, has already been answered, albeit in different studies, in the past. For example, different human activities leave complementary patterns in wrist motion is exploited in older feature engineering papers for activity recognition, then other studies use, including cross-axial or even cross sensor. The presented questions are not unanswered or lead to new areas for scientific exploration. Older feature engineering papers have considered which features and how generalisable these are across sensors and persons in various studies. See for example a very early work from Najafi (Physical Activity Monitoring and Risk of Falling Evaluation in Elderly People (Thesis No. 2672), EPFL)
    • The datasets do include multiple locations because it is very hard to discriminate activities from wrist alone. For example, if you take opportunity, the evaluators can select which sensor streams to use. Then participant disjoint splits for datasets exist. As far as I can see, there are not new datasets contributed here. But the disjoint evaluation is claimed as a contribution (See 1).
    • There are many studies, along with the arrival of smart watches, that consider HAR with wrist -only sensing, some cited here and some not (AccNet24, OpenWatch, Hang-Time HAR, UMAHand, Opportunity (you can use only wrist), too many to list here).
    • it is like an AuoML type work for HAR with wrist worn sensors.
  • Cost are substantial, 50-100 hours for Random and Evolution Search, ideally these should be reported in the main paper.
  • The Random search performs the best. The strongest reported result (70.2%) comes from WISP-Random (random search and dataset adaptations), not the fixed WISP recipe. So the search seems to contribute more than any model level innovation or contribution.
  • The study shows that combining handcrafted representations helps. But it is not clear, why self-supervised representations transfer poorly, whether the representations composed captures information absent from, say HARNet embedding.
Questions for authors
  • Can the authors confirm simply, if ensembles are tuned for each dataset using the validation set to find the right mix or recipe?
  • Could the better results reflect suboptimal transfer foundational models to the benchmark datasets rather than a fundamental advantage of compositional features?
  • Can the authors comment of if the performance gains are due to a better family of representations/parameters, or simply as a result of a larger model-selection budget than available to baseline methods? Could the representations be combined with pretrained representations?
3 comments
AuthorsOpenReview ↗

Response to Reviewer NsdW (1/3): benchmark and cookbook positioning, bringing TSC into wrist HAR, related work, and wrist adaptations (W1)

Thank you for your review, paper pointers, and noting "identifying complementary motion representations for wrist worn sensor HAR and organizing them into a composable search space." Because we packed both a benchmark and a method into 9 pages, we did not explain our design motivation clearly enough -- thank you for flagging this presentation issue. Below we clarify our positioning and answer each point directly (Part 1: W1; Part 2: Q1, W2, W3; Part 3: W4, Q2, Q3).

W1 (Part A): Wrist-Only HAR Without User Context at Cohort Scale

While multi-sensor setups and user prompts ("I am going to run") are useful directions, passive single-wrist accelerometry is the realistic 24/7 signal that large health cohorts collect across daily life and sleep.

We agree that wrist-only HAR without user priors is hard at fine granularity. Still, in large cohorts (UK Biobank, NHANES, All of Us) and wearable pretraining (RelCon, Xu et al., ICLR 2025), wearers cannot log prompts or wear chest, thigh, ankle, and phone sensors all day. Requiring multiple sensors shrinks usable data to a small intersection and fails during sleep, whereas wrist devices stay on 24/7. We also report Worst-Class F1 (4 metrics) so fine-grained limits stay clear.

W1 (Part B): Bringing Time-Series Classification Into Wrist HAR & Related Work

We do not claim new raw recordings or inventing participant-disjoint splits; we built WristHARBench to keep large free-living cohorts like CAPTURE-24 (which WHAR Arena excludes) and to bring modern non-neural time-series classification (TSC) methods into wrist HAR.

We built WristHARBench to fix two gaps in prior benchmarks: first, multi-sensor suites like WHAR Arena (Burzer et al., 2026, Sec. 4.2) mix body locations and explicitly exclude the large free-living CAPTURE-24 cohort (and ExtraSensory) because its size inflates GPU training time, whereas we retain CAPTURE-24 in our nine-cohort primary wrist benchmark (plus two auxiliary IMU tasks); second, prior HAR benchmarks mainly test neural networks, self-supervised encoders, or simple forests. We formulate wrist HAR as multichannel TSC so we can bring strong non-neural TSC families into wrist HAR. We also address your literature pointers directly:

  • HAR studies: Along with Hang-Time HAR and OPPORTUNITY (selectable wrist streams), we cite AccNet24 (it has no public code, and we benchmark its forest smoothing, Attend-and-Discriminate, and self-supervised lines).
  • Hand-gesture tasks: We cite OpenWatch and UMAHand and clarify that finger/hand gestures sit outside whole-body HAR.
  • Early feature work (Najafi, EPFL Thesis 2672): We credit classic multi-feature/cross-axial HAR and do not claim complementarity is new; Part 3 shows classic cross-axis correlation adds almost nothing over convolutions while directed state transitions add clear signal.

W1 (Part C): How to Understand WISP as a Wrist Representation Cookbook

WISP is a modular representation cookbook that adapts TSC blocks to compact short wrist windows with optional HMM-smoothing temporal decoding.

Wide some TSC methods (MultiRocket, MR-Hydra, RDST, HIVE-COTE 2.0) target small datasets with tens of thousands of features or four heavy ensembles. On large wrist cohorts, these wide baselines hit out-of-memory crashes even at 200 GiB RAM, and require 240.4 GiB (MR-Hydra) to 295.0 GiB (HIVE-COTE 2.0) of server memory plus hundreds of CPU hours, while pooled kernels miss rhythm and forward-vs.-reverse motion order inside short windows. We adapt each idea to wrist motion:

Cookbook StageTSC SourceHow We Adapt It to Wrist Data & Why
1. Local shape (R)ROCKET, MultiRocketWe use compact multi-axis dilated kernels with four pooled stats, which cuts RAM and solver time sharply on large cohorts.
2. Interval stats (I)CIF, QUANTWe compute moment, slope, and quantile stats on multi-scale sub-intervals, which captures posture and trend directly in linear time.
3. State & rhythm (S)SAX, catch22, MTF, inertialWe use a six-state window-quantile transition matrix (handles wearer amplitude shifts; separates forward vs. reverse motion) + rhythm and axis stats.
4. Early linear fusionHIVE-COTE 2.0 (4 ensembles)Unlike this method, which involves an ensemble of multiple classifier, Wisp-CIS concatenate all three representation before one classifier, which keeps training fast and allows clean ablations.
5. Sequence & tuningUK Biobank RF+HMMWe add optional HMM-smoothing temporal decoding across windows + optional validation tuning of representation weights and decoding.
AuthorsOpenReview ↗

Response to Reviewer NsdW (2/3): validation protocol (Q1), search as representation tuning, and four-metric speed-memory Pareto frontier (W2, W3)

Q1: Do We Tune Ensembles or Recipes Per Dataset on Validation Data?

Yes for Select5, WISP-Random, and WISP-Evolution, where we pick configurations strictly on validation participants and never touch test participants until final scoring; no for standalone WISP-CIS, where we run the exact same fixed recipe on every dataset with zero tuning.

W2 & W3: Search as Representation Tuning, Compute Costs, and Four-Metric Pareto Lead

For W2, the cumulative search hours sum all 329 candidate fits on large cohorts.

For W3, almost the whole gap between Select5 and WISP-Random came from HARMES, where short clips break continuity. We tune wide TSC baselines across a 4-setting validation grid and refit them, alongside Select5 + switch and WISP-CIS-Funnel (subsampled search on WISP-CIS). CPU times report standardized single-core hours on one Intel Xeon 8-series CPU for fair comparison (running on multiple CPU cores in parallel finishes much faster in wall-clock time), and we convert NVIDIA A100 GPU hours to CPU core-equivalent hours by rated power (1 A100 GPU h = 64 Xeon 8570 core-h, 400 W vs. 6.25 W/core):

Method (9 Cohorts, 3-Seed )Macro-F1Acc.Bal. Acc.Worst F1Tune TimeTraining TimePeak RAM
WISP-Random (329-config tuning)70.2077.2870.8828.25232.4 h0.31 h68.5 GiB
Select5 + switch (5-recipe menu)69.6076.6370.3026.941.25 h0.51 h9.0 GiB
WISP-Evolution (329-config tuning)69.5976.8470.1727.96424.0 h0.71 h79.2 GiB
WISP-CIS-Funnel (subsampled CIS search)69.2276.1170.0126.277.66 h totincl.10.6 GiB
WISP-CIS (+ HMM-smoothing decoding)66.1474.0766.0424.780 h0.35 h9.0 GiB
HIVE-COTE 2.0 (120-min, 32 cores)61.6872.1361.9118.040 h448.0 h295.0 GiB
RDST (tuned)61.4469.9163.8021.1692.5 h34.15 h124.4 GiB
MR-Hydra (tuned)60.2168.6261.3820.53119.6 h45.52 h240.4 GiB
MiniRocket (tuned)59.4069.5260.0016.1335.5 h15.53 h49.5 GiB
MultiRocket (tuned)59.3968.1160.4019.72201.3 h84.63 h150.2 GiB
H-InceptionTime (deep)57.6366.7759.8319.130 h102.4 hA100 GPU
HARNet (fine-tuned SSL)54.7861.6959.3621.3034.5 h16.89 hA100 GPU
Macro-F1 ^ Pareto Frontier: Macro-F1 vs. Tune + Training CPU Time & RAM (9 Datasets)
  70.2% |                          * WISP-Random (70.20 | 232.7h tot, 0.31h train)
  69.6% |     * Select5 + switch (69.60 | 1.76h, 9.0 GiB)
  69.2% |          * WISP-CIS-Funnel (subsampled search on WISP-CIS, 69.22 | 7.66h, 10.6 GiB)
  66.1% | * WISP-CIS (+ HMM-smoothing decoding, 66.14 | 0.35h, 9.0 GiB)
  61.7% |                               o HIVE-COTE 2.0 (61.68 | 448h, 295.0 GiB)
  61.4% |                o RDST tuned (61.44 | 126.7h, 124.4 GiB)
  60.2% |                     o MR-Hydra tuned (60.21 | 165.1h, 240.4 GiB)
        +-+---+----+-----+----+----+----+--> Tune + Training CPU core-h (log)
         0.3h 1.8h 8h   127h 165h 233h 450h
  1. Our wrist representations drive the main gain: Letting Select5 toggle HMM-smoothing temporal decoding on validation data nearly matches WISP-Random in under two core-hours (difference not significant), and WISP-Evolution matches WISP-Random at equal budgets.
AuthorsOpenReview ↗

Response to Reviewer NsdW (3/3): zero-search representation attribution, fine-tuned HARNet fusion, and extensible open-source cookbook (W4, Q2, Q3)

W4, Q2, & Q3 (Part A): Zero-Search Attribution vs. Fine-Tuned Pretrained Models

We pair every representation with the same class-balanced linear Ridge head (pure window classification, zero selection budget, no HMM-smoothing temporal decoding) so we can test representation quality directly; our new ablations and end-to-end HARNet fine-tuning show that our gains come directly from wrist-specific motion cues.

First, using one shared linear head without any decoding, we tested each sub-block of S, shuffled sample order inside each window, and ran equal-width convolution controls:

Window-Level Ablation (Shared Linear Head, No Decoding)Macro-F1What This Shows
Base convolutions R alone / doubled-width R57.28 / 58.74Adding more random kernels gives a modest gain
R + directed quantile-state transitions59.76Gives the largest gain among S sub-blocks
R + time-shuffled quantile-state transitions57.93Shuffling sample order erases ~75% of the gain (drops on 9/9 cohorts)
R + spectral bands / + multi-lag autocorrelation58.73 / 58.26Rhythm and periodicity add helpful signal
R + classic cross-axis correlation57.35Cross-axis correlation alone adds almost nothing
Full R + S (vs. 59.09 without state transitions)60.87Beats R alone and S alone on all 9/9 cohorts
R + I (57.64) + equal-width extra R columns58.30Equal-width random-convolution control
R + I + equal-width S = Standalone WISP-CIS61.12Beats equal-width extra R on all 9/9 cohorts (p = .020)
MiniRocket -> MiniRocket + I + S (shared head)60.26 -> 62.13Our I and S blocks also improve MiniRocket on 9/9 cohorts

Second, to check whether our baseline gap came from suboptimal transfer (Q2), we ran new 3-seed experiments on NVIDIA A100 GPUs where we fine-tuned the full HARNet encoder end-to-end with dataset-matched checkpoints (5s and 10s backbones, as its authors recommend) and expanded TF-C to all three accelerometer axes (converting GPU hours to single-core CPU equivalent hours by rated power at 1 A100 GPU h = 64 Xeon 8570 core-h, 400 W vs. 6.25 W/core):

Direct Window Classification (No Decoding, 3 Seeds)Hardware & Total CostMacro-F1Acc.Bal. Acc.Worst F1
Standalone WISP-CIS (0 tuning, 1 linear head)Xeon CPU (0.35 h, 9.0 GiB)61.1268.8166.4222.17
H-InceptionTime (train+val refit)NVIDIA A100 (102.4 core-eq h)57.6366.7759.8319.13
Attend-and-Discriminate (train+val refit)NVIDIA A100 (489.4 core-eq h)56.8664.5763.0023.48
HARNet (end-to-end fine-tuned, vs. 47.90 probe)NVIDIA A100 (51.4 core-eq h)54.7861.6959.3621.30
TF-C (all 3 axes, vs. 34.66 single-axis)NVIDIA A100 (92.8 core-eq h)47.2155.2151.1415.31

Full fine-tuning improves HARNet on all nine cohorts and three-axis input improves TF-C on all nine cohorts, and standalone WISP-CIS still leads on our participant-disjoint wrist benchmark at a tiny fraction of the compute (this result is specific to standardized three-axis windows and class imbalance in our benchmark, while pretraining remains valuable on broader multimodal tasks).

W4 & Q3 (Part B): Combining With HARNet (9/9 Wins) & Open-Source Package

We can easily combine WISP with pretrained models -- fusing standalone WISP-CIS with fine-tuned HARNet beats both standalone models on all nine primary datasets -- and we will release WristHARBench and WISP as an open-source Python package.

Matched Train-Only Window Classification (No Decoding)Macro-F1Cohort Wins
Standalone WISP-CIS60.75--
End-to-end fine-tuned HARNet54.52--
Validation-weighted fusion (WISP-CIS + fine-tuned HARNet)63.37Beats both on 9/9 cohorts (p = .004)

Cookbook insights and open-source package: We will release WristHARBench and WISP, an open-source representation framework that is memory-efficient, fast, highly accurate and extendable architecture to outperform powerful HAR baselines and a large number of methods ported from TSC.. We will revise all the points mentioned above to improve the readability of the article, enhance the explanation of the design rationale and differences in the Methods section, and improve the organisation of the experimental results. If these explanations and the new experimental results have addressed your concerns and you feel they are valuable to the HAR community, we would greatly appreciate it if you would consider raising your rating.

Reviewer_ZuUBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors proposed Wrist-Informed Space of Predictive representations (WISP) and WristHARBench to measure how well current models perform on data collected mainly from wrist sensors. The motivation of the study is sound since wrist sensors are widely used and deployed and deserve special attention.

Critical strengths
  • The work is well motivated. Wrist devices are among the most common wearables and are used in large cohorts. Also, the benchmark is well organised.
  • The comparison is extensive, covering 18 baselines
Critical weaknesses
  • There are many mentions of the appendix in places where it is necessary to actually explain the methods in the paper.
  • Results are hard to follow. For example, Figure 2 packs five panels under one caption: ranks, margins, block gains, ablations, and decoder gains. Several numbers in Sec. 4.2 can only be derived from Appendix Table 14. I suggest that you split Figure 2 into an external comparison and an ablation table, and name one primary method.
  • Decoder results are not accessible. I do not understand why we need a decoding algorithm for this research problem. Isn’t it just a classification task? I think this needs to be explained well, rather than jumping to the results and evaluations.
Questions for authors

My main concerns about the study are the presentation of the results. Unfortunately, I could not evaluate the paper properly once I reached the results section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ZuUB (1/3): representation cookbook positioning, core highlights and differences, and self-contained rationale (W1)

Thank you for pointing out the presentation issues that made our paper hard to follow: missing upfront method rationale (W1), packing five panels into one composite figure (W2), and introducing sequence decoding without clear intuition (W3). Because we squeezed both a benchmark and a method into 9 pages, we did not explain our design motivation clearly. Below we explain our positioning, method rationale, and results directly (Part 1 covers W1; Part 2 covers W3 and our question-driven experimental design; Part 3 gives the standalone W2 tables).

W1: Accurate Cookbook Positioning, Core Differences, and Method Rationale

We wrote this paper as a representation Cookbook for wrist HAR: we bring classic and modern time-series classification (TSC) methods into a wrist-only benchmark that retains large cohorts like CAPTURE-24, we adapt compact wrist blocks before one linear classifier to form standalone WISP-CIS, and we keep controlled variants so practitioners can see what actually works on wrist data.

  1. Why we built WristHARBench: Existing multi-dataset HAR benchmarks mix sensors across the chest, thigh, phone, and wrist, and WHAR Arena (Burzer et al., 2026, Sec. 4.2) explicitly excludes the large free-living CAPTURE-24 cohort because its size inflates GPU training time. Prior HAR benchmarks also focus on neural networks, pretrained encoders, or simple forests, rarely testing many modern non-neural TSC methods. Because smartwatches and wristbands are the passive sensors used in large health studies (UK Biobank, NHANES, All of Us), we isolate wrist-only data (retaining CAPTURE-24) and bring modern TSC families into wrist HAR alongside deep and self-supervised models.
  2. Why we built WISP and standalone WISP-CIS: On large, imbalanced wrist cohorts, these baselines hit out-of-memory crashes even at 200 GiB RAM, and require 240.4 GiB (MR-Hydra) to 295.0 GiB (HIVE-COTE 2.0) of server memory plus hundreds of CPU hours. We therefore built WISP as a modular representation cookbook for wrist signals. We adapt each block to short three-axis wrist windows and concatenate them before one class-balanced linear classifier to form WISP-CIS:
Cookbook StageSource IdeaHow We Adapt It to Wrist Windows & Why
1. Local shape (R)ROCKET, MultiRocketWe use 384 compact multi-axis dilated kernels with four pooled stats, which captures short wrist waveforms with low RAM and fast solver time.
2. Interval stats (I)CIF, QUANTWe compute moment, trend, and quantile stats on multi-scale sub-intervals, which captures coarse posture and trend distributions directly.
3. State & rhythm (S)SAX, catch22, MTF, inertialWe build a six-state within-window quantile transition matrix that handles wearer amplitude shifts and separates forward vs. reverse motion, plus rhythm and axis stats.
4. Early linear fusionHIVE-COTE 2.0 (4 ensembles)We concatenate all three blocks before one balanced linear Ridge head, which fits in one pass and supports clean one-by-one ablations.
5. Sequence & tuningUKB RF + HMM, HPOWe add optional HMM-smoothing temporal decoding (Part 2) and optional validation tuning of block weights and decoding (Select5, WISP-CIS-Funnel, WISP-Random).
AuthorsOpenReview ↗

Response to Reviewer ZuUB (2/3): CAPTURE-24 inspiration for HMM-smoothing temporal decoding (W3), new experiments, and question-driven organization

W3: Why We Include Optional HMM-Smoothing Temporal Decoding (Inspired by CAPTURE-24)

For W3, inspired by Oxford's CAPTURE-24 study (RF + HMM), we found HMM-smoothing temporal decoding really effective for wrist HAR, and we also designed paired controlled experiments to test when this component is useful.

  • How it works in WISP: Our linear Ridge classifier first predicts class scores independently for every 5-10s window. Because daily activities like walking, sleeping, or cleaning last for minutes while one short wrist window is noisy, optional HMM-smoothing temporal decoding combines training transition counts directly with continuous Ridge scores in milliseconds to fix isolated one-window flips.
  • Controlled experiments on when it helps: In our cookbook, we run controlled tests of WISP-CIS both under pure window classification (Direct, no decoding) and with HMM-smoothing temporal decoding, while Select5 + switch uses a validation switch to turn smoothing off on short-clip datasets (HARMES) where clips lack continuous time order.

Question-Driven Experimental Design and Key Findings

We anchor every comparison on standalone WISP-CIS and organize our original and new experiments around three concrete research questions:

Research QuestionWhat We Test (Changing One Factor at a Time)Direct Finding in This Rebuttal
Q1. External baselines, speed & RAMHow does standalone WISP-CIS compare with all 18 baselines (including HIVE-COTE 2.0, end-to-end fine-tuned HARNet, and 3-axis TF-C) across all four metrics, Tune Time, Training Time, and Peak RAM?Standalone WISP-CIS beats all 18 baselines across all four metrics in 0.35 h and 9.0 GiB RAM, while wide baselines hit 200 GiB OOM and reach 240.4-295.0 GiB RAM (Part 3, Table 1).
Q2. Cookbook ingredients & decodingDo shape (R), interval (I), and state/rhythm (S) blocks complement each other under one shared linear head, does within-window sample order matter, and when does HMM-smoothing temporal decoding help?R + S beats R and S alone on 9/9 cohorts, shuffling sample order erases ~75% of state gains (Part 3, Table 2), and Select5 + switch turns smoothing off on short-clip HARMES (Part 3, Table 1).
Q3. Validation selection & search budgetHow much does full 329-candidate tuning add over zero-tuning WISP-CIS, a 5-recipe menu (Select5), or WISP-CIS-Funnel (subsampled search on WISP-CIS), and does evolutionary search beat random sampling?Select5 + switch (69.60, 1.76 h) and WISP-CIS-Funnel (69.22, 7.66 h) nearly match 329-config WISP-Random (70.20), and evolutionary search matches random sampling (69.59; Part 3, Table 1).

Part 3 below presents the standalone comparison and ablation tables directly.

AuthorsOpenReview ↗

ResponseReviewer ZuUB (3/3): unified train+val four-metric, Tune Time, Training Time, and RAM tables (W2) and open-source cookbook

For W2 and Questions, we named WISP-CIS as our primary method and present standalone comparison and ablation tables directly below.

We run our 3-seed setup on all 9 cohorts, where wide TSC baselines use 4-setting validation tuning (Tune Time) and refits (Training Time). CPU times report standardized single-core hours on one Intel Xeon 8-series CPU for fair comparison (running on multiple CPU cores in parallel finishes much faster in wall-clock time), and we convert NVIDIA A100 GPU hours to CPU core-equivalent hours by rated power (set 1 A100 GPU h = 64 Xeon 8570 core-h, 400 W vs. 6.25 W/core):

1. (Four Metrics, Tune Time, Training Time & Peak RAM)

Method ( 9 Cohorts)Macro-F1AccBal-AccWorst-F1Tune TimeTraining TimePeak RAM
WISP-CIS (+ HMM-smoothing decoding)66.1474.0766.0424.780 h0.35 h9.0 GiB
Select5 + switch (5-menu)69.6076.6370.3026.941.25 h0.51 h9.0 GiB
WISP-CIS-Funnel (subsampled CIS search)69.2276.1170.0126.277.66 h totincl.10.6 GiB
WISP-Random (repr. tuning)70.2077.2870.8828.25232.4 h0.31 h68.5 GiB
WISP-Evolution (329-config tuning)69.5976.8470.1727.96424.0 h0.71 h79.2 GiB
HIVE-COTE 2.0 (120-min, 4 ensembles)61.6872.1361.9118.040 h448.0 h295.0 GiB
RDST (tuned)61.4469.9163.8021.1692.5 h34.15 h124.4 GiB
MR-Hydra (tuned)60.2168.6261.3820.53119.6 h45.52 h240.4 GiB
MiniRocket (tuned)59.4069.5260.0016.1335.5 h15.53 h49.5 GiB
MultiRocket (tuned)59.3968.1160.4019.72201.3 h84.63 h150.2 GiB
H-InceptionTime (deep)57.6366.7759.8319.130 h102.4 hA100 GPU
Attend-Disc. (deep)56.8664.5763.0023.480 h489.4 hA100 GPU
HARNet (fine-tuned SSL)54.7861.6959.3621.3034.5 h16.89 hA100 GPU
TF-C (3-channel SSL)47.2155.2151.1415.310 h92.8 hA100 GPU
MethodMacro-F1Acc.Bal. Acc.Worst F1Tune TimeTraining TimePeak RAM
Macro-F1 ^ Pareto Frontier: Accuracy vs. Tune + Training CPU Time & RAM
  70.2% |                   * WISP-Random (70.20 | 232.7h, 68.5 GiB)
  69.6% |     * Select5 + switch (69.60 | 1.76h, 9.0 GiB)
  69.2% |        * WISP-CIS-Funnel (subsampled search on WISP-CIS, 69.22 | 7.66h, 10.6 GiB)
  66.1% | * WISP-CIS (+ HMM-smoothing decoding, 66.14 | 0.35h, 9.0 GiB)
  61.7% |                       o HIVE-COTE 2.0 (61.68 | 448.0h, 295 GiB)
  61.4% |              o RDST tuned (61.44 | 92.5h tune + 34.2h train, 124 GiB)
  60.2% |                o MR-Hydra tuned (60.21 | 119.6h tune + 45.5h train, 240 GiB)
        +-+---+--+-----+----+---+--> Tune + Training CPU core-h (log scale)
         0.3h 2h 8h   130h 230h 450h

2. Unified Controlled Ablations of WISP-CIS (No Decoding)

We have added numerous new ablation experiments to enrich our representation cookbook, in the hope of providing the HAR community with further valuable insights into method design.

Cookbook QuestionControlled Contrast (Shared Ridge)Macro-F1 (%)Wins
Do local shape (R) and state/rhythm (S) complement?R + S vs. R alone / S alone60.87 vs. 57.28 / 55.309/9
New motion pattern or just extra width?WISP-CIS vs. R + I + equal-width extra R61.12 vs. 58.309/9
Does motion order inside a window matter?R + quantile states: true vs. shuffled order59.76 vs. 57.939/9
Why fuse early before one linear head?One linear head vs. 3-model val. late fusion61.12 vs. 59.908/9
Do I + S also help wider backbones?MiniRocket + I + S vs. MiniRocket alone62.13 vs. 60.269/9
  • Cookbook insights and open-source package: Our ablations show within-window transitions capture motion order missed by convolutions, and WISP-CIS + HARNet beats both standalone models on all 9 cohorts. We will release WristHARBench and WISP, an open-source representation framework that is memory-efficient, fast, highly accurate and extendable architecture to outperform powerful HAR baselines and a large number of methods ported from TSC..

We will revise all the points mentioned above to improve the readability of the article, enhance the explanation of the design rationale and differences in the Methods section, and improve the organisation of the experimental results. If these explanations and the new experimental results have addressed your concerns and you feel they are valuable to the HAR community, we would greatly appreciate it if you would consider raising your rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.