acceptodds
Under review as a conference paper at ICLR 2027

STaRC: Spectral–Temporal Cross-Prediction and Bayesian Relational Calibration for Cross-Dataset Few-Shot Activity Recognition

Abstract

Cross-dataset few-shot human activity recognition (HAR) requires learning transferable motion representations across heterogeneous sensing conditions while adapting to target activities from few labeled examples. Existing approaches often treat cross-dataset representation learning and few-shot adaptation separately, leaving uncertainty in transferring heterogeneous base knowledge under scarce target supervision insufficiently modeled. We propose STaRC, a two-stage framework that integrates Spectral–Temporal Cross-Prediction (STCP) with Bayesian Relational Calibration (BRC). We harmonize ten public IMU datasets into a common input format for self-supervised pretraining without requiring aligned activity labels. The resulting harmonized multi-dataset collection will be publicly released. STCP learns transferable representations through bidirectional cross-prediction between temporal and spectral views, capturing complementary temporal dynamics and spectral regularities beyond direct cross-view alignment. During few-shot adaptation, BRC probabilistically models relations between target features and base-class prototypes, samples multiple relational graphs to calibrate target representations, and integrates predictions from multiple base-knowledge experts through task-conditioned routing. This component transfers heterogeneous base knowledge while accounting for relational uncertainty induced by cross-dataset shift and limited supervision. Experiments across target datasets and shot settings demonstrate the effectiveness of both STCP and BRC. Under the same downstream calibration protocol, comparisons with representative self-supervised methods show the benefits of STCP pretraining. Comparisons with competitive few-shot baselines further support the effectiveness of BRC in transferring base knowledge to target episodes, particularly in low-shot settings. Ablation studies examine the effects of pretraining data scale, temporal and spectral learning objectives, and Bayesian calibration compared with point-estimate or no calibration. These results suggest that predictive representation learning and uncertainty-aware relational calibration provide a complementary approach to data-efficient cross-dataset HAR, with potential for personalized wearable sensing.

Primary area:
learning on time series and dynamical systems
Keywords:
Few-Shot Learning, Bayesian Inference, Human Activity Recognition

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_x4yMOpenReview ↗
Rating
3
Confidence
1(−2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to solve cross-dataset few-shot human activity recognition under heterogeneous sensing conditions and scarce target labels. Specifically, the authors propose Spectral–Temporal Cross-Prediction (STCP) to learn transferable representations through bidirectional prediction between augmented temporal and spectral views, and Bayesian Relational Calibration (BRC) to model uncertain target–base relations and combine calibrated predictions through task-conditioned expert routing. Using ten harmonized public IMU datasets for pretraining, experiments on PAMAP2 and UCI HAR report improved few-shot classification accuracy over the evaluated baselines across the tested shot settings.

Critical strengths
  1. Transferring wearable activity recognition models across sensing environments with limited annotation is practically important.

  2. STCP and BRC address distinct stages of the transfer process.

  3. The study evaluates several support-set sizes and includes both self-supervised and few-shot baselines.

Critical weaknesses
  1. The single-data UCI HAR experiment does evaluate cross-dataset transfer, but its comparison with full-data training changes both pretraining sources and the episodic-training domain.

  2. Accuracy improvements alone do not establish that inferred relation variances capture transfer uncertainty.

  3. The authors do not separately isolate expert routing, expert subsets, the two KL regularizers, or the prototype prediction branch.

  4. Classification results cover only two target datasets; routing statistics on additional datasets do not establish recognition performance there.

Questions for authors

see above

Reviewer_AeMrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

STaRC combines spectral–temporal cross-prediction (STCP) for self-supervised IMU pretraining with Bayesian relational calibration (BRC) for few-shot classification. BRC samples target–base relations, calibrates features using source-class prototypes, and combines expert predictions through an episode-conditioned router. The paper pretrains on ten datasets and evaluates classification on PAMAP2 and UCI HAR, with ablations of pretraining data, prediction objectives, and relational calibration.

Critical strengths

The reported PAMAP2 improvements are substantial and consistent. In the full-data setting, STaRC exceeds the strongest reported baseline by 2.93–5.15 percentage points across all five shot settings.

The cross-prediction ablation provides useful evidence for STCP. Removing cross-prediction reduces accuracy by 2.03–5.92 percentage points across all five PAMAP2 settings. This supports a contribution from the predictive objective within the proposed pipeline.

Critical weaknesses

The described training procedure does not match the supplied implementation. Equation 4 and Appendix B.4 minimize a positively weighted neighbor contrastive loss, whereas starc/stcp.py subtracts this loss. The directional averaging also differs. There are additional discrepancies in the preprocessing code discussed below.

The evidence does not clearly isolate BRC’s uncertainty modeling. The “None” variant in Table 2c uses Meta-ProtoNet, while full BRC includes ridge classification, expert routing, and a prototype mixture. The components retained in the point-estimate variant are unspecified. Consequently, the ablation does not establish how much of the improvement comes specifically from Bayesian relations.

Questions for authors

Which objective produced Tables 1–2? Equation 4 adds the neighbor loss, but starc/stcp.py, line 130, subtracts standard NT-Xent terms. Lines 125 and 129 sum the two directions, whereas Equations 3 and 29 average them. Please confirm the sign, normalization, and code version used for the reported experiments.

How was overlap between HAPT and the UCI HAR test data checked? HAPT is an updated release of UCI HAR containing raw recordings, rather than an independent source dataset. The supplied HAPT reader processes the raw label list without an explicit exclusion of UCI HAR test recordings. Please identify the overlap checks or exclusions applied before pretraining.

In preprocessing/hapt.py, lines 37–57, the first labeled segment of each recording is not appended, and the final recording is not flushed. In preprocessing/msm.py, lines 29–30, every second sample is retained; for the official 50 Hz MotionSense data, this would yield approximately 25 Hz, contrary to Appendix B.2.

What is held fixed between point-estimate and Bayesian calibration in Table 2c? In particular, do they use the same relation networks, expert subsets, router, ridge classifier, and prototype mixture?

Reviewer_yQBoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

STaRC combines self-supervised spectral–temporal pretraining with supervised episodic calibration using source-class prototypes, sampled relations, and multiple experts. It reports few-shot activity-recognition results on PAMAP2 and UCI HAR.

Critical strengths

The reported PAMAP2 gains are substantial: full-data 1-shot accuracy is 62.29%, compared with 58.42% for TS-TCC and 57.01% for UNEM. The component comparisons and implementation details help separate representation learning from adaptation.

Critical weaknesses
  1. The UCI HAR scaling comparison changes target-label access. Full-data training adds labeled UCI HAR validation episodes, whereas single-data training uses PAMAP2. Its gain cannot be attributed to broader pretraining alone. This concern applies to the UCI single-to-full comparison, not the matched comparisons within each setting.

  2. The Bayesian contribution remains uncertain. Gains over the point-estimate control range from 0.56 to 2.16 percentage points, without training-seed variability. Evaluating 1,000 test episodes does not measure this variability. The paper should also confirm that experts, support views, routing, and fusion are matched.

  3. The Gaussian bound does not validate the uncertainty approximation. Appendix B.1 correctly compares two Gaussians, but does not bound the gap to the original binomial process or establish better transfer. This acknowledged limitation leaves the inference benefit to empirical validation.

  4. Data provenance and reproduction remain unresolved. The archive lacks a complete runner and split/prototype manifests. HAPT, used for pretraining, is an updated UCI HAR dataset; recording-level exclusion against UCI HAR test data needs documentation. This is an unresolved overlap risk, not demonstrated leakage.

Questions for authors
  1. Can the UCI HAR scaling comparison keep episodic supervision fixed and explicitly list target-data access at each stage?
  2. Which settings are shared by Table 2(c)'s Bayesian and point-estimate variants, and does their difference survive repeated training seeds?
  3. Can you provide the split/prototype manifests, HAPT–UCI deduplication procedure, and commands for the reported runs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.