- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work focuses on how to avoid missing or noisy modalities from influencing the gradient update in multimodal test-time adaptation. It proposes TRUST, a framework that uses a dynamic anchor from stable views as a reference to calculate the reliability of modalities and adaptively update the gradient during backward propagation. The authors conduct extensive experiments to prove the effectiveness of TRUST.
Critical strengths
- This work investigates an interesting problem: the missing modality could not only influence the feed-forward prediction but also harm the gradient update of multimodal test-time adaptation.
- The experiments are comprehensive, and the performance of TRUST is significant.
- The paper writing is easy to follow.
Critical weaknesses
- "Replace these values with measured results" are contained in Tables 7, 8, and 12, which makes me strongly concerned about the reliability, reproducibility, and trustworthiness.
- There are many inconsistencies and conflicts between the paper and the appendix. For example, authors claims the in Section 5.1 but also claims in the appendix. And when deriving Proposition 2, it refers to Equation () as Equation (). Meanwhile, it changes to in the deduction, which are absolute and relative reliability, respectively, making Proposition 2 unreliable.
- Novelty is moderate which combines several losses into a framework. Existing works already considered the reliability and its influence on model optimization in multimodal test-time adaptation [1].
[1] “Test-time Adaptation against Multi-modal Reliability Bias”, ICLR 2024
Questions for authors
- Why is the performance of the unimodal baseline in Table 1 much higher than the performance reported under a 100% primary modality drop? What exactly does the uniform baseline refer to?
- What are the exact and true values for experimental results and hyperparameters?
- Will these losses both work properly without gradient conflicts and training fluctuations?