acceptodds
Under review as a conference paper at ICLR 2027

Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series

Abstract

Multimodal wearable systems must remain reliable when sensor streams become noisy or unavailable. Existing multimodal test-time adaptation (TTA) methods often assess reliability online, but cross-modal agreement can be misleading when sensors measure different physical processes, and evaluating alternative modality configurations adds inference cost. We propose CARAT, which decouples model reliance from runtime corruption detection to guide omission or attenuation, amortizing reliance estimation through source training. An asymmetric modality-dropout curriculum prepares a missingness-resilient backbone for omission and derives a frozen, backbone-specific reliance proxy from windowed input-projection gradient norms. At deployment, a lightweight one-class detector flags suspect streams, and the proxy guides a joint choice between replacing the suspect set with the backbone’s trained missingness symbol and attenuating its representations before fusion, without candidate-subset evaluation. Across four wearable datasets, five corruption types, three backbones, and eight TTA baselines, CARAT achieves the highest overall macro-F1 and best mean rank (2.42), exceeding EATA, the strongest baseline, by 1.58 F1 points across 12 equally weighted dataset–backbone settings. Across five profiled configurations, CARAT uses 9.49% fewer GFLOPs and updates 47.82% fewer parameters than EATA. A pattern also emerges across sensing regimes: multimodal TTA methods such as PTA are competitive on IMU-dominated homogeneous datasets, whereas unimodal TTA methods like TENT and EATA match or exceed it on heterogeneous datasets. These results position CARAT as a practical default to wearable TTA, offering competitive robustness with modest computational requirements and benefits that vary across backbones and dataset regimes.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Multimodal time series, efficient test-time adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wvLsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CARAT, a test-time adaptation method for multimodal wearable time series under sensor corruption. During source training, an asymmetric modality-dropout curriculum adjusts each modality's dropout probability using windowed input-projection gradient norms; the terminal dropout rates are then frozen as a backbone-specific reliance profile ρj\rho_j. At test time, a one-class detector flags suspect modalities using reconstruction and spectral-distance features relative to clean source statistics. If the unflagged modalities retain at least a fraction ν=0.8\nu=0.8 of total reliance mass, suspect streams are replaced with the backbone's trained missingness symbol. Otherwise, their latent representations are attenuated before fusion. Adaptation updates only normalization and attenuation parameters using a weighted entropy objective. The evaluation covers four wearable datasets, three missingness-resilient backbones, five synthetic corruption types, and eight TTA baselines. All TTA methods use the same source checkpoint in the main comparison. CARAT achieves the best overall macro-F1 and mean rank across the 12 dataset backbone settings, with the largest improvements occurring on ShaSpec. The paper also includes ablations on detector quality, routing choices, reliance-prior perturbations, alternative source-training regimes, and computational cost.

Critical strengths
  • The paper addresses a useful deployment problem that identifying a corrupted modality is not sufficient to determine whether it should be discarded. CARAT separates current sensor abnormality from the model's dependence on that sensor and uses the two signals to choose between omission and attenuation. The unconditional-drop experiments support this motivation; for example, on PAMAP2--MAESTRO, dropping every flagged modality reduces F1 from about 0.920.92 to 0.680.68, while CARAT maintains approximately 0.920.92.

  • The benchmark covers multiple sensing regimes, backbones, corruption families, and TTA baselines, while evaluating all methods on a common source checkpoint and corrupted test batches. The appendices also provide condition-level results and explicitly qualify the dependence structure of the statistical comparisons rather than treating all corruption conditions as independent replications.

  • The paper includes useful component analyses. It evaluates always-drop and always-attenuate policies, detector removal/noise, frozen attenuation, alternative source-training curricula, prior perturbations, and sensitivity to the routing threshold. These experiments help separate the contributions of source preparation, detection, routing, and online adaptation.

Critical weaknesses
  • The incremental benefit of the source-learned reliance prior is not clearly established. On the main DaliaHAR--ShaSpec ablation, always attenuation matches CARAT on pooled Div. F1 (0.630.63), and a random prior also matches CARAT on Div. and reversing the prior reduces performance to 0.610.61. These results suggest that the coarse routing policy matters, but do not yet show that the particular learned reliance profile is a major source of the overall gains. The key prior ablations are also concentrated on the setting where CARAT performs best.

  • The headline improvement is strongly backbone-dependent. CARAT improves over EATA by 4.394.39 F1 points on average for ShaSpec, but only 0.150.15 on MAESTRO and 0.190.19 on Flex-MoE. Several condition-level comparisons on MAESTRO and Flex-MoE favor other methods. This makes the broader practical default framing stronger than the current evidence supports.

  • The statistical analysis does not capture source-training uncertainty in the main TTA comparison. The condition-level intervals reuse the same checkpoint and overlapping examples, and the five-corruption-redraw experiment also keeps the source checkpoint fixed. This is particularly relevant for the small margins on MAESTRO and Flex-MoE.

  • The evaluation is limited to synthetic corruption families. The detector uses reconstruction and spectral features that are well suited to several of these corruptions, but it remains unclear how well the detector and frozen reliance prior transfer to natural sensor failures, long-term drift, or device/subject shifts. Although the paper itself notes that the prior may become stale and lists natural failures as future work.

Questions for authors
  • Can the authors report the always-drop, always-attenuate, random-prior, and reversed-prior ablations across all 12 dataset--backbone settings? If always attenuation or a random prior performs similarly to CARAT beyond DaliaHAR--ShaSpec, what incremental benefit comes from the source-learned reliance profile itself?

  • Can the authors report the main TTA results across multiple independently trained source checkpoints, with uncertainty at the dataset--backbone level? The current condition-level intervals and corruption-redraw experiment keep the source checkpoint fixed, which does not capture training-seed variability.

  • Can the authors compare against a simpler control that gives TENT or EATA access to the same suspect-modality detector, for example by masking or down-weighting flagged streams? This would help separate the value of the detector from the value of reliance-guided routing between omission and attenuation.

4 comments
AuthorsOpenReview ↗

Response to Reviewer wvLs (1/3)

Thank you for your time and constructive feedback. We appreciate your recognition that our work identifies a useful gap in multimodal TTA and offers an intuitive method, supported by extensive evaluations and comprehensive ablations. Below, we address your concerns and questions point by point.

Q1 / W1 — Across all 12 dataset–backbone settings, how does CARAT compare with always-drop, always-attenuate, random-prior, and reversed-prior controls, and what incremental value does the learned profile add?

Based on your suggestion we have added a new policy/prior experiment across all 12 settings (Appendix S.1, Table 44). Always drop/attenuate force the intervention on flagged streams while retaining CARAT’s other components as given below :

SettingCARATAlways dropAlways attenuateRandom priorReversed prior
DSADS–MAESTRO0.720.670.710.710.71
DSADS–ShaSpec0.520.530.520.490.53
DSADS–Flex-MoE0.550.550.540.540.55
DaliaHAR–MAESTRO0.760.570.760.730.75
DaliaHAR–ShaSpec0.660.590.630.630.61
DaliaHAR–Flex-MoE0.680.520.680.670.60
PAMAP2–MAESTRO0.920.210.920.920.92
PAMAP2–ShaSpec0.680.070.630.630.63
PAMAP2–Flex-MoE0.770.090.770.770.77
WESAD–MAESTRO0.630.620.640.630.62
WESAD–ShaSpec0.550.370.550.530.55
WESAD–Flex-MoE0.540.560.600.600.60

CARAT is higher/tied/lower than the random prior in 8/3/1 settings and the reversed prior in 6/4/2. On DaliaHAR–ShaSpec, CARAT scores 0.66 and the reversed prior scores 0.61. Always attenuating is higher than CARAT in 2/12 settings and tied in 6/12. We acknowledge that the learned profile has setting-dependent benefits, and always attenuating could be a strong alternative. However, in the existing ablations (Tables 2 and 4) and a new sensor-pair omission experiment (Appendix S.2, Table 45). we observe that removing corrupted EDA + temperature improves F1 from 71.21 to 73.62, whereas removing corrupted wrist + chest acceleration reduces it from 48.14 to 22.95; CARAT achieves 55.86 on the latter. The acceleration pair also receives more prior mass (51.9% vs. 22.2%). This supports the motivation for reliance-guided interventions.

AuthorsOpenReview ↗

Response to Reviewer wvLs (2/3)

Q2 / W3 — Can you report TTA results across independently trained source checkpoints, with uncertainty at the dataset–backbone level?

As requested, we have added a new source-checkpoint experiment: on DaliaHAR, we evaluate three independently trained checkpoints per backbone (source seeds 2711, 2712, and 2713), each with three TTA seeds (2711, 2712, and 2713), across all three backbones and ten methods (Appendix Q, Table 40). Every method uses the same source checkpoints and matched corrupted inputs within each evaluation. We first average the 50 conditions within each TTA run, then the three TTA runs within each checkpoint, and report mean ± sample SD across the three source-checkpoint means (n=3).

The closest competing methods are shown below; Table 40 gives all ten methods.

MethodMAESTROShaSpecFlex-MoEMean F1Mean rank ↓
TENT0.76 ± 0.02810.64 ± 0.00740.68 ± 0.00230.69 ± 0.00612.33
EATA0.76 ± 0.02870.64 ± 0.00650.68 ± 0.00250.69 ± 0.00662.00
CARAT0.76 ± 0.03350.66 ± 0.00630.68 ± 0.01570.70 ± 0.00381.67

Approximately, for each dataset it takes ~11 H100 or ~15 GB200 hours per checkpoint and one TTA seed, leading to ~300 or ~405 GPU-hours, respectively, for the full 3 × 3 study across three backbones, excluding training and setup. Given this cost, we currently report DaliaHAR (Table 40), with additional source checkpoint variations on request during rebuttal and full coverage planned for the final revision. Separately, Tables 13–24 now report three TTA evaluations at fixed checkpoints across all 12 settings, directly quantifying adaptation variability and complementing source-checkpoint uncertainty, consistent with general benchmarking suggestions (Bouthillier et al., MLSys 2021).

Q3 — Can TENT/EATA use the same suspect-modality detector with simple masking or downweighting to separate detector benefit from reliance-guided routing?

Yes as suggested, we have added a new detector-matched controls (Appendix R, Tables 41–43) equip TENT and EATA with CARAT’s detector on DaliaHAR across all three backbones and all 50 corruption conditions. Within each backbone, the controller variants share the source checkpoint, corrupted inputs, and detector flags. They either drop every flagged stream or attenuate it by α=0.5, retaining each baseline’s adaptation objective and update scope as given below :

MethodControllerMAESTROShaSpecFlex-MoE
TENToriginal72.0060.0067.00
Always drop flagged57.18159.06951.460
Always attenuate, α=0.572.86063.04368.246
EATAoriginal72.0060.0067.00
Always drop flagged57.24758.78651.550
Always attenuate, α=0.572.86862.91268.246
CARAToriginal76.0066.0068.00

Always dropping performs substantially worse than fixed attenuation for both baselines across all three backbones. CARAT’s reported reference is higher on MAESTRO and ShaSpec, while fixed attenuation is slightly higher on Flex-MoE. Thus, simple detector-assisted attenuation is competitive, and intervention choice matters. These comparisons assess the combined detector, intervention, and adaptation design; Tables 2 and 4; and the new policy/prior experiment across all 12 settings (Appendix S.1, Table 44) discussed in Q1, more directly examine the overall reliance guided routing within CARAT.

W2 — How should backbone-dependent gains and condition-level losses qualify the headline improvement and “practical default” claim?

The evidence supports CARAT as a practical TTA option with backbone-dependent benefits. Updated Table 1 (based on 3 re-runs of each TTA method acros 50 scenarios) reports a mean gain of 2.43 F1 points over EATA and mean rank 2.33. While the gains remain uneven: 5.60 points on ShaSpec, 1.21 on MAESTRO, and 0.48 on Flex-MoE, as reported in Section 4.1 (p. 7) using means before rounding, CARAT ranks first/ties in 10/12 settings, including 6/8 non-ShaSpec settings. Further our new headroom and corruption-family analyses (Appendix P, Table 39; Figure 14). The frozen-model diagnostic across all 12 settings finds greater clean-to-corrupted loss for ShaSpec on DSADS (0.33, versus 0.17/0.21 for MAESTRO/Flex-MoE) and DaliaHAR (0.14, versus 0.05/0.13). These separate diagnostic cohorts suggest more opportunity for recovery. Within DSADS–ShaSpec, the two families with the largest losses also yield the largest CARAT gains.

The updated manuscript uses “practical TTA option” in the abstract (p. 1) and Section 4.1 (p. 8), where the claim is conditional on adaptation FLOPs and updated-parameter count being the primary constraints. Section 5 (p. 10) describes CARAT as “a practical option in the evaluated setting.”

AuthorsOpenReview ↗

Response to Reviewer wvLs (3/3)

W4 — How well do the detector and frozen reliance prior transfer to natural failures, long-term drift, and device/subject shifts, including when the prior becomes stale?

Our 720-condition benchmark uses real wearable recordings and five synthetic profiles motivated by observation loss, waveform artifacts, saturation/compression, frequency-response degradation, and decreasing within-window SNR. In a new unmodified-input and false-positive sensitivity experiment (Appendix D.1, Table 8) on DaliaHAR–ShaSpec windows, using one fixed checkpoint and three paired evaluation repeats, native CARAT remains 0.88 F1 points above Static. Native detector false positives incur a 0.22% relative F1 decrease compared with CARAT with all detector flags suppressed, which still adapts; independently injecting false positives at a 40% probability per modality–window pair yields a 1.19% relative decrease.

Our test sets already use non-overlapping subject splits (Appendix B) and we explicitly acknowledge the stale-prior limitation in Section 5, which states that “The frozen, parameterization-dependent prior may become stale” and identifies slow-timescale prior refresh and “validation under natural failures across broader tasks and architectures” as future work.

I thank the authors for their detailed response and additional experiments as well as the effort in addressing several of my concerns. I particularly appreciate the new ablations and source checkpoint analyses. While I do still have some reservations about the incremental benefit of the learned reliance prior over simpler alternatives, I still appreciate the authors’ efforts and therefore I will maintain my original positive recommendation.

Reviewer_jW39OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CARAT, which learns a modality-reliance profile during source training and combines it with a test-time corruption detector to choose between dropping suspicious modalities and attenuating their latent representations. The method is motivated by the observation that cross-modal disagreement is not necessarily a reliable corruption signal for heterogeneous wearable sensors, while exhaustive evaluation of modality subsets is computationally expensive.

Critical strengths
  • Separating input abnormality from model-specific modality reliance is well motivated, particularly for heterogeneous wearable sensing where different modalities may naturally disagree.
  • The method avoids enumerating modality subsets at test time and adapts only a relatively small set of parameters, which makes the approach attractive from a deployment perspective.
  • The experiments cover multiple wearable datasets, several missingness-resilient backbones, and a broad set of TTA baselines, with reasonably extensive ablations on the detector, routing policy, prior, and source-training strategy.
  • The overall performance is competitive, and CARAT reaches or remains close to the best-performing method in most dataset–backbone settings.
Critical weaknesses
  • The methodological novelty appears somewhat incremental. READ (ICLR 2024) already studies multimodal reliability bias, while TSA (ICML 2025) uses modality-specific adapters and routing for selective multimodal TTA. Adaptive treatment of modality imbalance and reliability during training has also been explored in works such as SleepMG (ACM MM 2024) and SleepSMC (ICLR 2025). These works do not solve exactly the same test-time corruption problem, but they suggest that modality reliability, imbalance, and selective intervention are already well-established directions. The most distinctive component of CARAT is therefore the combination of a source-learned reliance prior with a test-time detector to choose between dropping and attenuation, and this distinction could be articulated more clearly.
  • Gradient norm is used as a proxy for modality reliance, but it is also affected by parameterization, projection width, and optimization dynamics. Its relationship to actual predictive contribution is therefore indirect, and the theoretical justification remains heuristic.
  • The improvement is sensitive to the backbone: the largest gains are concentrated on a subset of backbone configurations, while improvements are relatively modest on others. This weakens the claim that the method is a broadly applicable default TTA strategy.
  • The evaluated deployment shifts are primarily synthetic corruptions. Validation under natural sensor failures, device degradation, or long-term deployment drift would make the practical robustness claims considerably stronger.
Questions for authors

Please see Weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer jW39 [1/2]

Thank you for your time and constructive feedback. We appreciate your recognition of the revealing observation about modality heterogeneity in multimodal TTA, the attractive deployment solution, the broad experiments, and the competitive performance. Below, we address your concerns and questions point by point.

W1 — Could CARAT’s methodological contribution be distinguished more clearly from READ, TSA, SleepMG, and SleepSMC, particularly its combination of a learned reliance prior and detector to select dropping or attenuation?

Our existing discussion in Section 2 compares CARAT with READ’s reliability-aware fusion and TSA’s modality-specific adapters and routing. We have now expanded this discussion and added SleepMG & SleepSMC to our references, acknowledging their contributions to training-time modality imbalance and reliability. CARAT repurposes the backbone’s trained missing-input operation as a conditional test-time intervention, using the reliance mass retained by unflagged modalities to select omission or attenuation without online candidate-subset evaluation. The revised text positions CARAT more precisely around its specific contribution: combining a frozen, checkpoint-specific reliance proxy with runtime abnormality flags to choose joint omission or attenuation of suspect modalities. We hope this clarification addresses your concern and would welcome any further suggestions for making the distinction clearer.

W2 — Gradient norm's reliance relationship to actual predictive contribution is indirect, and the theoretical justification remains heuristic.

We use the gradient-based profile as a checkpoint-specific routing prior, and test its value through interventions on the deployed prior (Tables 4 and 44). The gradient-flow identity motivates this choice but is not meant to establish formal predictive importance. Section 3.2.1 explicitly states that “its connection to predictive dependence is assessed empirically rather than implied by the identity.” The existing Table 4 provides an initial controlled test with the DaliaHAR–ShaSpec. In a new prior controls across all 12 settings (Appendix S.1, Table 44), we extend the comparison to four datasets and three backbones, retaining the deployed source checkpoint and full corruption grid within each setting.

SettingCARATRandom priorReversed prior
DSADS–MAESTRO0.720.710.71
DSADS–ShaSpec0.520.490.53
DSADS–Flex-MoE0.550.540.55
DaliaHAR–MAESTRO0.760.730.75
DaliaHAR–ShaSpec0.660.630.61
DaliaHAR–Flex-MoE0.680.670.60
PAMAP2–MAESTRO0.920.920.92
PAMAP2–ShaSpec0.680.630.63
PAMAP2–Flex-MoE0.770.770.77
WESAD–MAESTRO0.630.630.62
WESAD–ShaSpec0.550.530.55
WESAD–Flex-MoE0.540.600.60

CARAT is higher/tied/lower than the random prior in 8/3/1 settings and the reversed prior in 6/4/2, which helps establishe the utility of our proposed grad-norm based source reliance as a test-time guidance for adaptation.

AuthorsOpenReview ↗

Response to Reviewer jW39 [2/2]

W3 — Why are improvements backbone-dependent, and how should this qualify the claim that CARAT is a broadly applicable default TTA strategy?

We agree that CARAT’s gains can be backbone-dependent and already acknowledge this in Section 4.1 (p. 7): “CARAT’s margin over EATA is largest on ShaSpec in each of the four datasets.” We now describe CARAT as a practical TTA option with setting-dependent benefits. Table 1, updated with three TTA evaluation runs per dataset–backbone setting at fixed source checkpoints, reports mean gains over EATA of 5.60 F1 points on ShaSpec, 1.21 on MAESTRO, and 0.48 on Flex-MoE. Benefits also extend beyond ShaSpec: at the displayed precision, CARAT ranks first or ties for first in 10/12 settings, including 6/8 non-ShaSpec settings.

Our new headroom experiment (Appendix P, Table 39) suggests that larger absolute gains may partly reflect greater corruption-induced loss and more opportunity for recovery. The table below summarizes clean-to-Static macro-F1 differences on the 0–1 scale from this separate frozen-model diagnostic.

DatasetMAESTROShaSpecFlex-MoE
DSADS0.170.330.21
DaliaHAR0.050.140.13
PAMAP20.020.140.14
WESAD0.020.030.03

ShaSpec has the largest headroom on DSADS and DaliaHAR, while its headroom is comparable to Flex-MoE on PAMAP2 and WESAD. These observations suggest that available headroom may be one contributing explanation. In another new within-backbone analysis on DSADS–ShaSpec (Figure 14), the two corruption families with the largest clean-to-Static losses also yield the largest CARAT gains.

We deliberately chose distinct multimodal backbones to evaluate the TTA methods. Fusion design may also influence the benefits: ShaSpec’s classifier jointly normalizes composed modality blocks, which may make multimodal interventions more useful than in its attention-based and MoE-based counterparts. Accordingly, Section 5 (p. 10) now describes CARAT as “a practical option in the evaluated setting.”

W4 — The evaluated deployment shifts are primarily synthetic corruptions.

Our 720-condition benchmark uses real wearable sensor recordings, with five synthetic corruption profiles inspired by practical sensor disturbances: observation loss, waveform artifacts, saturation/compression, frequency-response degradation, and decreasing within-window SNR (Section 4, Benchmark; Appendix A). We complement these controlled perturbations with a new experiment on DaliaHAR–ShaSpec (Appendix D.1, Table 8). Native CARAT remains 0.88 F1 points above Static, while native detector false positives incur a 0.22% relative F1 decrease compared with a control that suppresses all detector flags while retaining adaptation. Validation under independently annotated natural failures, device aging, and longitudinal drift are left as future work and our existing Section 5 explicitly identifies “validation under natural failures across broader tasks and architectures” as future work.

I thank the authors for their detailed response and additional experiments. However, the updated results W3 seem to further highlight CARAT's strong dependence on the ShaSpec backbone. While the improvement over EATA is substantial on ShaSpec (+5.60 F1 points), the gains on MAESTRO (+1.21) and Flex-MoE (+0.48) remain marginal. These findings reinforce my concern about the method's backbone dependence and leave its generalizability across different architectures insufficiently demonstrated. Therefore, I maintain my previous concerns and will keep my original rating unchanged.

AuthorsOpenReview ↗

Clarifying CARAT’s Competitiveness Across Backbones

Dear Reviewer jW39,

Thank you for your response. We would like to clarify two points. (1) The strongest TTA baseline changes across the backbones you cite, yet CARAT remains competitive. The leading TTA baseline is (averaged over the four datasets) are TENT on MAESTRO, SoTTA on ShaSpec, and EATA on Flex-MoE and CARAT exceeds their respective averages by 1.18, 5.45, and 0.48 F1 points and achieves the highest mean macro-F1 on each of the three backbones (Table 1). (2) Our new headroom diagnostic was intended to help contextualize the larger absolute recovery. It compares clean and corrupted frozen-model performance (Appendix P, Table 39), with ShaSpec showing the largest corruption-induced loss on DSADS and DaliaHAR. This leaves more lost performance to recover. It offers a possible explanation for larger absolute gains, but not indicative of backbone dependence. CARAT does not have any backbone specific intervention component.

Finally, we would like to highlight that our practical-option claim is based on CARAT's competitiveness across the evaluated architectures, while acknowledging that the gains are not uniform. This is why we describe CARAT as “a practical option in the evaluated setting” (Section 5).

We would welcome your thoughts on whether this clarifies your concern and would be happy to discuss any remaining questions while the discussion period is open. Thank you for your time.

Reviewer_ks7QOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CARAT, a test time adaptation method for multimodal wearable models when some sensor streams become corrupted. Its core idea is to separate whether a sensor is currently corrupted from how much the model relies on that sensor. During source training CARAT uses gradient magnitudes to learn a fixed modality reliance profile while an asymmetric modality dropout curriculum trains the model to tolerate missing sensors. At test time, an anomaly detector flags suspect sensors. CARAT then uses the learned reliance profile to decide whether to drop those sensors if the remaining healthy modalities are sufficient or attenuate/ downweight them if they are important to the model. Across four wearable datasets, three multimodal backbones, five synthetic corruption types and eight TTA baselines, CARAT achieves the best overall mean rank and modestly improves average macro F1 over the strongest baseline

Critical strengths
  • The observation that cross modal agreement is not always a reliable way to detect corruption is relevant for wearables because different sensors can measure very different physical processes. I also appreciated how this is reflected in the empirical results where the multimodal PTA baseline behaves differently from TENT / EATA across the homogeneous and heterogeneous sensing settings
  • The paper makes a useful distinction between detecting that a sensor is corrupted and deciding how much the model actually relies on that sensor. Using both pieces of information to decide whether to drop or downweight a sensor a pretty intuitive design choice (especially when simply removing a corrupted but important modality could hurt performance)
  • The authors test CARAT on four wearable datasets, three different multimodal backbones, five types of corruption and eight TTA baselines. The paper also evaluates computational efficiency rather than just focusing on predictive performance
  • I liked how the datasets are not just repetitions of the same setup: some are mostly IMU based while others combine heterogeneous signals such as motion, heart rate, temperature, EDA ,BVP etc; this is relevant because the motivation for CARAT is that different modalities may carry very different amounts of information
  • The paper includes several targeted ablations (for example the prior perturbations, unconditional drop comparison, white noise control and projection width experiment) which directly probe assumptions behind the method. The authors are transparent about the limitations of the gradient based reliance proxy and the observational nature of some of the empirical findings
Critical weaknesses
  • TARAT's improvements seem to come mainly from one backbone: ShaSpec. Although it improves over EATA by 1.58 F1 points on average, the gain is much larger on ShaSpec (+4.39) than on MAESTRO (+0.15) or Flex MoE (+0.19). In some settings, such as DaliaHAR-MAESTRO and WESAD-Flex MoE, CARAT actually performs worse than EATA. I would have liked to see more discussion of why the method helps substantially on ShaSpec but provides relatively little improvement on the other two backbones.. this also makes the "practical default" framing feel somewhat stronger than what the results currently support
  • I am not fully convinced that CARAT's reliance guided routing provides much benefit over simpler approaches. In Table 2, always downweighting flagged sensors (R2) performs almost identically to CARAT while a random reliance prior also matches CARAT in the Div. setting in Table 4. Most baselines also do not have access to the corruption detector so its unclear whether CARAT's improvements come from identifying corrupted sensors or from its proposed routing mechanism. It would be useful to compare against TENT/ EATA with the same detector and a simple drop or downweighting strategy across all backbones. Figure 12 also shows that the learned attenuation gates barely change from their initial values, suggesting that fixed downweighting might work just as well. Overall I would like stronger evidence that the reliance guided routing itself contributes meaningfully to performance
  • It's unclear how well CARAT's gradient based reliance scores reflect the actual importance of each sensor because the strongest validation is limited to DaliaHAR-ShaSpec and the projection width experiment shows that these scores can change substantially even when predictive performance remains similar. There’s also a possible feedback effect since the scores influence which sensors are dropped during training which could potentially change the reliance they aim to measure. Also several sensors receive the same maximum score which makes it difficult to distinguish their relative importance. Stronger validation across datasets and backbones would help establish how reliable these scores are.
  • I noticed several numerical and statistical reporting issues that need clarification. For example, the reliance scores for DaliaHAR-MAESTRO differ between Tables 9 and 36, and the DaliaHAR-ShaSpec results in Table 2 are inconsistent with the sensitivity figures. The fractional trainable parameter counts are also difficult to interpret. More importantly, the main results use a single source checkpoint per setting, while some ablation differences are as small as 0.01-0.02. Reporting results across multiple training seeds would help determine whether these differences are meaningful. I also noticed that the confidence interval for PAMAP2-MAESTRO in Table 29 excludes zero, despite a reported p-value of 0.14?
  • The evaluation only considers synthetic sensor corruptions so it’s unclear how CARAT would perform under real world conditions or when sensors are working normally. In particular the corruption detector may incorrectly flag healthy sensors but the impact of these false alarms on prediction accuracy isn’t evaluated. Since each sensor uses a threshold based on the 95th percentile of clean data, the chance of incorrectly flagging at least one sensor could increase when more sensors are used. Testing CARAT on clean data, examining the effect of false alarms and including naturally occurring sensor failures would make the practical claims more convincing
  • I found the efficiency results to be somewhat mixed. Although CARAT uses fewer computations (GFLOPs) and updates fewer parameters than EATA, its implementation has 44% higher latency and consumes 37% more CPU energy? Given these costs and the small performance gains on two of the three backbones, I again think the claim that CARAT is a "practical default" needs stronger justification
Questions for authors

See above.

5 comments
AuthorsOpenReview ↗

Response to Reviewer ks7Q [1/4]

Thank you for your time and constructive feedback. We appreciate your recognition of our observation that cross-modal agreement should not be the default criterion, the intuitive design, the diverse datasets and evaluation setup exposing multimodal TTA vulnerabilities, and the thorough ablations. Below, we address each of your concerns and questions point by point.

W1 — Why are gains concentrated on ShaSpec, with small gains on MAESTRO and Flex-MoE?

We agree that CARAT’s gains can be backbone-dependent and already acknowledge this in Section 4.1 (p. 7): “CARAT’s margin over EATA is largest on ShaSpec in each of the four datasets.” Table 1, updated with three TTA evaluation runs per dataset–backbone setting at fixed source checkpoints (the submitted version had one TTA run), reports mean gains over EATA of 5.60 F1 points on ShaSpec, 1.21 on MAESTRO, and 0.48 on Flex-MoE. Benefits also extend beyond ShaSpec: at the displayed precision, CARAT ranks first or ties for first in 10/12 settings, including 6/8 non-ShaSpec settings.

Our new headroom experiment across all 12 settings (Appendix P, Table 39) suggests that larger absolute gains may partly reflect greater corruption-induced loss and more opportunity for recovery. We measure headroom H = C − U, where C and U denote clean and corrupted Static macro-F1 from the same frozen checkpoint. The table below reports these differences on the 0–1 scale from this separate frozen-model diagnostic.

DatasetMAESTROShaSpecFlex-MoE
DSADS0.170.330.21
DaliaHAR0.050.140.13
PAMAP20.020.140.14
WESAD0.020.030.03

ShaSpec has the largest headroom on DSADS and DaliaHAR, while its headroom is comparable to Flex-MoE on PAMAP2 and WESAD. These observations suggest that available headroom may be one contributing explanation. In another new within-backbone analysis on DSADS–ShaSpec (Figure 14), the two corruption families with the largest clean-to-Static losses also yield the largest CARAT gains (descriptively, Pearson r = 0.97 and Spearman rho = 0.90 across five families).

We deliberately chose distinct multimodal backbones to evaluate the TTA methods. Fusion design may also influence the benefits: ShaSpec’s classifier jointly normalizes composed modality blocks, which may make multimodal interventions more useful than in its attention-based and MoE-based counterparts. This remains an architectural hypothesis; these diagnostics do not isolate causation. Accordingly, Section 5 (p. 10) now describes CARAT as “a practical option in the evaluated setting,” reflecting its setting-dependent benefits.

AuthorsOpenReview ↗

Response to Reviewer ks7Q [2/4]

W2 — a) What evidence shows that reliance-guided routing adds value beyond detector access and simple dropping/downweighting? Please include detector-matched TENT/EATA comparisons across backbones. b) Is the learnable gates useful?

a) We agree that simple unconditional policies can match CARAT in some settings: always attenuating ties CARAT on wrist acceleration and Div. in Table 2, and the random prior ties on Div. in Table 4. The evidence supports condition-dependent benefits from intervention choice. Figure 5 already shows that unconditional dropping can cause substantial harm: on PAMAP2–MAESTRO, F1 falls from 0.92 to 0.68, while CARAT retains 0.92. Just identifying suspect sensors therefore does not establish that removing them is beneficial.

In a new detector-control experiment (Appendix R, Tables 41–43). We equip TENT and EATA with CARAT’s detector and apply unconditional dropping or fixed attenuation with α=0.5 across all three backbones on DaliaHAR as shown below :

MethodControllerMAESTROShaSpecFlex-MoE
TENToriginal72.0060.0067.00
Always drop flagged57.18159.06951.460
Always attenuate, α=0.572.86063.04368.246
EATAoriginal72.0060.0067.00
Always drop flagged57.24758.78651.550
Always attenuate, α=0.572.86862.91268.246
CARAToriginal76.0066.0068.00

CARAT’s benchmark reference exceeds the dropping summaries across all three backbones and the fixed-attenuation summaries on MAESTRO and ShaSpec; fixed attenuation is slightly higher on Flex-MoE. These descriptive comparisons assess CARAT’s combined intervention and adaptation design beyond detector access. In W3 below, we further examine the reliance profile itself, connecting its reliance profile to routing decision .

b) In anotrher new gate-learning control (Appendix M, Figure 13) study, we observe that gates sometimes remain near initialization. Freezing them at their native initialization while retaining detection, routing, normalization updates, and fallback weights isolates gate learning. In selected ShaSpec cases, learned gates move on DSADS and WESAD and improve macro-F1 over this control by 11.27% and 0.75% relative, respectively; DaliaHAR remains near initialization with +0.06% relative change. The cases use one evaluation seed and were selected by gate movement, independently of F1. This supports gate learning as a complementary, case-dependent benefit. Table 2’s R8 instead disables both normalization and attenuation updates and measures their joint contribution.

W6 — How do the reported 44% higher latency and 37% higher CPU energy than EATA, despite lower GFLOPs and fewer updated parameters, affect the efficiency and “practical default” claims?

CARAT’s efficiency claim is resource-specific: across the five reference profiles, it uses 9.49% fewer GFLOPs and updates 47.82% fewer parameters than EATA, while incurring 44.23% higher latency and 37.37% higher CPU energy (median within-configuration ratios; Tables 25–26). Section 4.1 explicitly reports this tradeoff and qualifies the practical claim with “when adaptation FLOPs and the number of updated parameters are primary constraints” (p. 8, lines 405–406).

To address the implementation overhead, we conducted a new matched CPU-only study on DSADS across all three backbones (Appendix L.2, Table 28). Reusing FITS reconstructions across modalities and batching spectral computations reduces total detection–adaptation–prediction latency by 10.1–17.0% and CPU energy by 8.7–15.6% relative to reference CARAT, preserving suspect masks and class predictions on the evaluated inputs.

BackboneEATA latency (ms)Optimized CARAT latency (ms)EATA CPU energy (J)Optimized CARAT CPU energy (J)
MAESTRO2386.7 ± 61.42313.2 ± 53.6491.2 ± 15.7479.1 ± 19.4
ShaSpec5149.4 ± 197.04024.8 ± 348.21115.9 ± 49.8864.1 ± 71.6
Flex-MoE2714.9 ± 60.52367.7 ± 162.8578.2 ± 19.2505.2 ± 39.6

Values are mean ± sample SD per batch over three timing repeats, with detection, adaptation, and prediction entirely on CPU. These additional results strengthen the practical case for CARAT: detector optimization preserves the evaluated predictions while achieving lower observed mean latency and CPU energy than EATA across all three DSADS backbones. The original five-profile measurements remain separately reported. For the final revision, we will re-profile all five configurations using the optimized implementation and update all efficiency metrics. Section 5 accordingly describes CARAT as “a practical option in the evaluated setting,” retaining the explicit resource qualifications and runtime–energy tradeoffs.

AuthorsOpenReview ↗

Response to Reviewer ks7Q [3/4]

W3 — What evidence can you provide beyond DaliaHAR–ShaSpec supports the reliability of CARAT’s reliance scores across datasets and backbones?

As you note, Figure 3 provides motivating evidence from DaliaHAR–ShaSpec. We interpret the scores as model-dependent reliance, rather than an absolute or universal ranking of sensors. Section 3.2.1 explicitly describes the profile as “a frozen, backbone-specific prior for adaptation” (p. 5, lines 220–223). Tables 10–12 already show different profiles across datasets and backbones: for example, the mean chest-ECG score on WESAD is 0.80 for ShaSpec, 0.58 for Flex-MoE, and 0.00 for MAESTRO (Table 12). These differences illustrate model dependence; the omission and intervention diagnostics below assess the profile’s usefulness.

New sensor-pair omission experiment (Appendix S.2, Table 45). In DaliaHAR activity recognition, the omission diagnostic shows stronger dependence on acceleration than on wrist temperature and EDA. We compare two corrupted sensor pairs using the same ShaSpec checkpoint under discrete missingness, averaging two evaluation seeds. Clean macro-F1 is 74.09; scores below use the 0–100 scale.

| Corrupted streams | Share of deployed prior (%) | Pass corrupted inputs | Remove known-corrupted streams | CARAT | |---|---:|---:|---:|---:|---:| | EDA + temperature | 22.2% | 71.21 | 73.62 | 73.52 | | Wrist + chest acceleration | 51.9% | 48.14 | 22.95 | 55.86 |

Removing EDA and temperature improves F1 from 71.21 to 73.62, only 0.47 points below clean. Removing both acceleration streams instead reduces F1 from 48.14 to 22.95, or 51.14 points below clean; CARAT achieves 55.86. The acceleration pair’s larger prior share (51.9% vs. 22.2%) is consistent with its greater omission cost. Thus, different corrupted streams call for different interventions, even within one checkpoint. Known-corrupted removal uses ShaSpec’s native missing-stream handling; CARAT additionally uses detection and adaptation, so this comparison does not isolate routing alone.

Source training supplies this profile offline, amortizing reliance estimation across deployment. At test time, CARAT combines it with current detector flags to select an intervention without evaluating alternative modality subsets (Section 3.2.2, p. 5, lines 267–269).

New validation across all 12 dataset–backbone settings (Appendix S.1, Table 44). We extend the policy/prior controls to all four datasets and three backbones. Always drop/attenuate force the action on flagged streams while retaining CARAT’s other components; random/reversed priors replace all applicable runtime uses of the profile. Reversal preserves the clipped values while changing their sensor assignments.

Scores average 50/50/45/95 conditions on DSADS/DaliaHAR/PAMAP2/WESAD. The CARAT column reproduces Table 1; control scores retain the reported policy/prior results. All scores use the 0–1 scale with two decimals. Comparisons below use displayed precision and are descriptive rather than paired evaluation-seed effects.

SettingCARATAlways dropAlways attenuateRandom priorReversed prior
DSADS–MAESTRO0.720.670.710.710.71
DSADS–ShaSpec0.520.530.520.490.53
DSADS–Flex-MoE0.550.550.540.540.55
DaliaHAR–MAESTRO0.760.570.760.730.75
DaliaHAR–ShaSpec0.660.590.630.630.61
DaliaHAR–Flex-MoE0.680.520.680.670.60
PAMAP2–MAESTRO0.920.210.920.920.92
PAMAP2–ShaSpec0.680.070.630.630.63
PAMAP2–Flex-MoE0.770.090.770.770.77
WESAD–MAESTRO0.630.620.640.630.62
WESAD–ShaSpec0.550.370.550.530.55
WESAD–Flex-MoE0.540.560.600.600.60

CARAT is higher/tied/lower than the random prior in 8/3/1 settings and the reversed prior in 6/4/2. On DaliaHAR–ShaSpec, CARAT scores 0.66, compared with 0.61 for the reversed prior. Always attenuation remains competitive, and some prior replacements match or outperform CARAT. Together, these diagnostics support source-learned reliance as a practical, checkpoint-specific routing prior, without establishing universal sensor importance.

Comments on the paper

1 comment

Dear (Senior) Area Chair and Reviewers,

Multimodal backbones are often trained to handle missing inputs. CARAT reuses this ability to respond to corrupted inputs at test time. When the detector flags streams, CARAT omits them jointly if the unflagged streams retain sufficient mass under a frozen, source-learned, checkpoint-specific reliance proxy, and otherwise attenuates the suspect representations. It thus decides between omission and attenuation without evaluating candidate modality subsets at test-time.

Across four datasets and three backbones, CARAT improves overall mean macro-F1 by 2.43 points over EATA, the strongest overall baseline. It has the highest four-dataset mean on each backbone, with margins over the strongest TTA baseline of 1.18 (TENT) on MAESTRO, 5.45 (SoTTA) on ShaSpec, and 0.48 (EATA) on Flex-MoE.

In response to the reviewers, we added experiments addressing three questions:

  1. Are the gains robust? (ks7Q, wvLs) We reran the full comparison across all 12 settings (45–95 corruption conditions each), including all eight TTA baselines, with three evaluation seeds at fixed source checkpoints (updated Table 1; per-condition means and SDs in Tables 13–24). On DaliaHAR, we also evaluated three independently trained source checkpoints per backbone, each with three TTA seeds; Table 40 (App. Q) reports source-level mean+/-SD. In this source-checkpoint study, CARAT retains the highest cross-backbone mean F1 and best mean rank.

  2. Where do the gains come from? (ks7Q, wvLs) The policy controls show the risk of unconditional removal: on PAMAP2, always dropping flagged streams scores 0.07–0.21 macro-F1 across the three backbones, compared with 0.68–0.92 for CARAT (App. S.1). On DaliaHAR, TENT and EATA with CARAT’s detector and fixed attenuation have higher reported means than their original benchmark references. CARAT’s benchmark reference is higher on MAESTRO and ShaSpec, while fixed attenuation is slightly ahead on Flex-MoE (App. R). Compared with always-attenuate, which retains CARAT’s other components, CARAT is higher/tied/lower in 4/6/2 settings; compared with random and reversed priors, the counts are 8/3/1 and 6/4/2, respectively, at displayed precision (App. S.1). These comparisons are descriptive across separately reported evaluations.

  3. Is it practical? (ks7Q) On unmodified DaliaHAR–ShaSpec inputs, CARAT remains 0.88 F1 points above the frozen source model, and detector alarms incur only a 0.22% relative F1 decrease compared with CARAT with all flags suppressed while retaining adaptation (App. D.1). Our efficiency claim concerns settings where adaptation GFLOPs and the number of updated parameters are the main constraints, rather than a general latency or energy advantage. A new matched study on across three backbones shows that optimized CARAT also achieves 3.1–21.8% lower mean end-to-end latency and 2.5–22.6% lower mean CPU energy than EATA, while preserving reference CARAT’s masks and predictions (App. L.2). These implementation gains complement the compute-budget motivation and do not detract from the main design contribution.

Regarding Reviewer jW39’s backbone concern, uneven gains are already acknowledged in the paper. Our new analyses suggest that gain magnitude may partly reflect available headroom: ShaSpec has the largest clean-to-corrupted loss on DSADS and DaliaHAR (App. P, Table 39), and within DSADS–ShaSpec, the corruption families with the largest losses also yield the largest CARAT gains (Fig. 14). In Table 1, CARAT ranks first or ties in 10/12 settings. CARAT is also the most consistent method: its largest shortfall from the best method in any setting is 2 F1 points, while every alternative falls at least 8 points behind in some setting (at least 4 excluding ShaSpec). To address jW39’s novelty concern, Section 2 now clarifies comparisons with READ, TSA, SleepMG, and SleepSMC.

We also narrowed the stated scope from "practical default" to "a practical option in the evaluated setting," and validation on natural failures remains open. Reviewer wvLs appreciated the source-checkpoint analyses and new ablations and maintained a positive recommendation, while retaining reservations about the prior’s incremental benefit.

We hope that our additional experiments, manuscript revisions, and clarified scope will be taken into account during the reviewer–AC discussion and in reaching the final decision.

Thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.