- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper introduces the Fourier Representation Editor (FIRE), a novel approach to multimodal test-time adaptation (TTA). Moving beyond existing methods that primarily adjust fused features, FIRE intervenes from a representation engineering perspective by directly editing semantically rich intermediate representations.
Critical strengths
-
Algorithmic Innovation and Performance: The paper successfully applies Fourier representation editing to multimodal TTA for the first time, achieving results that consistently outperform current state-of-the-art methods.
-
Comprehensive Evaluation: The authors conducted extensive experiments evaluating the framework against a wide variety of corruption types (e.g., on Kinetics50-C and VGGSound-C), rigorously validating the strong empirical performance of FIRE.
Critical weaknesses
1.The proposed method is not a strictly source-free TTA approach, as it explicitly requires the pre-computation and caching of source data statistics (means and standard deviations)[cite: 6, 7]. This dependency is conceptually similar to Domain Adaptation methods like DUA (https://ieeexplore.ieee.org/abstract/document/9879821) and diminishes its practical applicability. In real-world blind deployments, caching source distributions in advance is often unrealistic and introduces additional computational overhead.
- FIRE applies its representation editing equally across all intermediate layers. This uniform treatment can harm overall computational efficiency and potentially lead to the accumulation of errors through the network. The framework would benefit significantly from an adaptive layer-selection mechanism. For instance, employing eXplainable AI (XAI) techniques, such as Grad-CAM, to identify and selectively edit only the most feature-relevant layers could optimize both computational efficiency and adaptation accuracy.
Questions for authors
see issues raised in weaknesses
3 comments
I thank the authors for their responses and supplementary experiments, and I will retain my score.
Dear Reviewer 2NCj,
Thank you for engaging with our response and for the positive feedback. We appreciate your time and support.
We clarified FIRE’s source-free TTA setting and provided additional experiments on computational efficiency and adaptive layer selection.
Thank you very much for your thorough review of our work and your insightful suggestions.
W.1. Source-Free TTA and Cached Source-Domain Statistics
We would like to clarify the distinction between accessing raw source-domain data at test time and using precomputed source-domain statistics in TTA. Before deployment, FIRE computes the mean and standard deviation of layer-wise representations offline using a minibatch of source samples. No raw source samples are accessed during test-time adaptation. Thus, FIRE remains consistent with the source-free TTA setting adopted by methods such as FOA. These statistics are computed only once before deployment, introducing minimal additional overhead. Although DUA also leverages statistical information, it differs from FIRE in the parameters being updated, how the statistics are utilized, and the specific representation calibration mechanism.
W.2. Computational Efficiency and Adaptation Strategies
Thanks to the low-rank design of the representation editors, FIRE maintains high computational efficiency even when applied to all intermediate layers. As shown in Table A, we additionally report the runtime of various TTA methods for a single run on VGGSound-C. The results demonstrate that FIRE requires fewer updated parameters. Despite the additional computational overhead, FIRE achieves substantial performance improvements.
We agree that adaptive layer selection could further improve efficiency, as also discussed in the Limitations section. Moreover, the results in Table 5 confirm that different layers contribute differently. However, approaches such as Grad-CAM may introduce additional computational overhead. We are currently exploring this direction. We experimented with dynamically selecting the Top-K layers with the largest distribution shifts in each modality for representation editing. As shown in Table B, while this improves efficiency, it leads to a performance drop. Further investigation is needed to better understand this trade-off.
Table A. Computational overhead.
Table B. Adaptive layer selection.
The relevant discussion and experimental results on computational efficiency have been added to Appendix E.2.