- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether representation-based test-time adaptation (TTA) can restore MLLM performance under common image corruptions. Testing "Gaussian-AcT" across three MLLMs and 15 corruption types from the RealWorldQA dataset, the authors find that it recovers only a small fraction of the performance loss. Through subsequent experiments—including offline fitting, full-covariance alignment, paired ridge/MLP regression, and activation patching—the authors conclude that this failure stems not primarily from estimation errors, mapping expressivity, or the choice of intervention location, but rather from the fact that corruption-induced shifts in MLLMs are highly token-specific; these shifts are approximately orthogonal across samples and exhibit high effective rank. In contrast, pooled representations exhibit more pronounced shared shifts, explaining why traditional feature alignment is more effective for classification tasks. Consequently, the paper’s primary contribution is an empirical, diagnostic negative result rather than the proposal of a new TTA method.
Critical strengths
I believe the paper's primary strength lies in the fact that it moves beyond the mere observation that ''a certain simple TTA method performs poorly.'' Instead, through a series of oracle and ablation experiments, the authors attempt to rule out alternative explanations—such as the estimator, mapping expressivity, supervision, and the intervention site—and further analyze the structure of the corruption shift. This lends the paper diagnostic value within the scope of sample-blind correction.
Additionally, the paper presents a clear core narrative: the authors contrast token-level MLLM outputs with pooled classification representations, employing shift decomposition, cosine similarity, and effective rank to explain the divergent results. The overall writing and experimental organization are easy to follow.
Critical weaknesses
The paper lacks a solution—or at least a constructive validation—that naturally derives from the core analysis. The paper primarily demonstrates the limited effectiveness of existing ''sample-blind representation correction'' methods in the face of MLLM corruption. It further suggests that effective remediation might require sample-specific or token-specific approaches, or even the utilization of contextual information from other visual tokens within the same image. While this diagnostic insight is valuable, the paper essentially stops there without verifying the feasibility of such directions. In particular, the authors could have easily incorporated a relatively simple contextual (or input-dependent) "oracle corrector" capable of accessing the full visual-token sequence to predict token-level corrections. Even if such a method were not practically deployable for TTA, it would have answered a crucial question: does the paper reveal a fundamental limitation of representation-based adaptation itself, or merely a limitation of shared/pointwise correction strategies? In my view, the absence of this experiment makes the work feel more like an analysis of failure cases rather than a fully scientific story.
Questions for authors
There are minor cross-reference formatting inconsistencies: ''Figure 1'' and ''Figure 4'' in the Introduction do not appear as hyperlinks, unlike the surrounding figure/section references.
Figure 1 is not fully self-contained. The abbreviation “MCQ” is used before it is defined, and the figure/caption does not clearly explain what the “classification” task refers to. Since Figure 1 presents the paper’s central contrast, these quantities should be defined directly in the caption or surrounding text.