acceptodds
Withdrawn from ICLR 2027

TRUST: Trustworthy Reliability-guided Update Selection for Multimodal Test-time Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal test-time adaptation aims to improve model robustness under distribution shifts without source data or target labels, yet real-world deployment often involves dynamic modality missingness, where incomplete or unreliable modalities can degrade predictions and contaminate online adaptation signals. To address this problem, we propose TRUST, a source-free online adaptation framework that selects trustworthy unimodal and fused views for model updates rather than relying only on prediction confidence. TRUST constructs a dynamic anchor from confident and temporally stable views, estimates update reliability using confidence gain, anchor agreement, and stream-level stability, and applies an adaptive update budget for efficient single-backward adaptation. Experiments on egocentric audio-visual recognition and multimodal sentiment analysis show that TRUST consistently improves robustness across missing rates, dynamic missing patterns, and pretrained backbones, while reliability analysis confirms its ability to stabilize adaptation under severe modality missingness.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Test-time adaptation, Multimodal Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kcqBOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work focuses on how to avoid missing or noisy modalities from influencing the gradient update in multimodal test-time adaptation. It proposes TRUST, a framework that uses a dynamic anchor from stable views as a reference to calculate the reliability of modalities and adaptively update the gradient during backward propagation. The authors conduct extensive experiments to prove the effectiveness of TRUST.

Critical strengths
  • This work investigates an interesting problem: the missing modality could not only influence the feed-forward prediction but also harm the gradient update of multimodal test-time adaptation.
  • The experiments are comprehensive, and the performance of TRUST is significant.
  • The paper writing is easy to follow.
Critical weaknesses
  • "Replace these values with measured results" are contained in Tables 7, 8, and 12, which makes me strongly concerned about the reliability, reproducibility, and trustworthiness.
  • There are many inconsistencies and conflicts between the paper and the appendix. For example, authors claims the τa=0.5,μ=0.9\tau_a=0.5, \mu=0.9 in Section 5.1 but also claims τa=1.0,μ=0.95\tau_a=1.0, \mu=0.95 in the appendix. And when deriving Proposition 2, it refers to Equation (1818) as Equation (1616). Meanwhile, it changes ζtv\zeta_t^v to ρtv\rho_t^v in the deduction, which are absolute and relative reliability, respectively, making Proposition 2 unreliable.
  • Novelty is moderate which combines several losses into a framework. Existing works already considered the reliability and its influence on model optimization in multimodal test-time adaptation [1].

[1] “Test-time Adaptation against Multi-modal Reliability Bias”, ICLR 2024

Questions for authors
  • Why is the performance of the unimodal baseline in Table 1 much higher than the performance reported under a 100% primary modality drop? What exactly does the uniform baseline refer to?
  • What are the exact and true values for experimental results and hyperparameters?
  • Will these losses both work properly without gradient conflicts and training fluctuations?
Reviewer_ns95OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TRUST, a source-free online test-time adaptation framework addressing multimodal models under dynamic modality missingness. The core mechanism involves constructing a dynamic anchor from confident and temporally stable views, and subsequently estimating update reliability through confidence gain, anchor agreement, and stream-level stability. Furthermore, the framework applies an adaptive update budget to facilitate computationally efficient single-backward adaptation.

Critical strengths
  1. Practical Real-World Motivation: The paper tackles a highly relevant and practical challenge in multimodal test-time adaptation (TTA)—dynamic missing modalities. This is a commendable exploration toward deploying multimodal systems in unpredictable real-world environments.
  2. Computational Efficiency: The proposed method is elegantly designed to operate with a single-backward-pass. By decoupling reliability estimation and deploying detached view-level gates prior to backpropagation, the framework avoids heavy computational overhead, making it highly efficient for online streaming scenarios.
Critical weaknesses
  1. Vulnerability to Simultaneous Corruption: While the framework successfully handles missing modalities and includes a brief stress test on singular corruption in the appendix, the core reliability-guided update heavily relies on the assumption that the remaining available modalities (and their constructed views) are relatively clean. In realistic edge cases where both candidate views suffer from severe, concurrent corruption, the anchor construction and the reliability-guided update mechanism are highly susceptible to failure, as the reliability metrics might confidently converge on degraded features.
  2. Reliance on Synthetic Missingness: The experimental validation relies exclusively on artificially constructing missing modalities (i.e., randomly dropping streams from complete datasets) rather than utilizing datasets with naturally occurring, authentic missing modalities. Simulated masking does not fully capture the complex, correlated sensor failures or physical occlusions seen in real-world deployments, which limits the empirical validation of the method's true robustness.
  3. Limited Algorithmic Novelty: The core technical components exhibit limited innovation. Specifically, the "Dynamic Anchor Construction" and "Reliability-Guided Update Selection" depend almost entirely on standard, off-the-shelf formulations like Jensen-Shannon (JS) Divergence and Softmax normalization. While combining them for this application is logically sound, the lack of deeper architectural or algorithmic novelty diminishes the overall technical contribution.
Questions for authors

Please comment on the points raised in the weaknesses.

Reviewer_ydV1OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TRUST, a source-free online test-time adaptation framework for multimodal models under dynamic modality missingness. It uses a dynamic anchor, view-level reliability estimation, and an adaptive update budget to reduce unreliable updates, with experiments on egocentric audio-visual recognition and multimodal sentiment analysis. The reported results show consistent improvements over existing methods, although concerns about the validity of several supplementary results make the empirical claims difficult to assess reliably.

Critical strengths
  1. The paper addresses a practical problem: unreliable modalities can affect not only predictions but also subsequent online updates.
  1. The method is clearly structured, and the evaluation covers multiple datasets and missing-modality patterns.
Critical weaknesses
  1. Appendix Tables 7, 8, 11, and 12 still contain the instruction “Replace these values with measured results,” despite reporting specific numbers. The reliability of these results is therefore seriously questionable.
  1. Sec. 5.1 and Appendix Table 15 report inconsistent hyperparameter settings, making it unclear which configurations were actually used.
  1. I also noticed that the proof of Proposition 2 appears inconsistent with the update-budget definition in Eq. (18), leaving the theoretical claim insufficiently justified.
  1. More concerning, the paper claims that all results were verified against experimental records, yet the placeholder instructions remain in four tables. This discrepancy raises serious research integrity concerns.

Given these issues, I do not think the experimental results provide a sufficiently reliable basis for supporting the paper's main claims.

Questions for authors
  1. Were the results in Appendix Tables 7, 8, 11, and 12 obtained from actual experiments? Why do these tables still contain placeholder instructions?
  1. Which hyperparameter settings were actually used in the main experiments, given the inconsistencies between Sec. 5.1 and Appendix Table 15?
  1. Could the authors clarify how the proof of Proposition 2 follows from the update-budget definition in Eq. (18)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.