acceptodds
Under review as a conference paper at ICLR 2027

Do Language Models Dream of Answers before They Speak?

Abstract

At the core of language model intelligence lies the formation of task-relevant representations, internal encodings of the information underlying the model’s inferential outcomes. However, the representational dynamics governing the transformation of latent states into model outputs remain insufficiently characterized. We study the evolution dynamics of task-relevant representations during inference and observe task-relevant direction supporting highly accurate readout that generalizes across datasets emerges abruptly at an intermediate model layer. To characterize the geometry underlying this readout, we develop and empirically validate a theoretical framework grounded in real-world language models, accounting for interactions among components and deriving empirically verifiable criteria for the direction’s spectral dominance. Furthermore, we substantiate the readout’s effectiveness and semantic alignment with the model’s native answer preferences through visual evidence reranking experiments for retrieval-augmented generation. By connecting the internal organization of task-relevant representations to model outputs, our work provides a principled basis for interpreting representational dynamics and applying internal model judgments in real-world systems.

Primary area:
interpretability and explainable AI
Keywords:
Interpreteability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wjMQOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how task-relevant representations become organized within vision-language models before answer generation. The authors report the abrupt emergence of a spectrally dominant “decision axis” at an intermediate layer, develop a spectral framework to characterize its geometry, and apply the resulting readout to visual evidence reranking.

Critical strengths
  1. The paper formulates decision formation as a problem of internal geometric organization and derives sufficient conditions for spectral dominance and alignment with a reference decision axis. Its theoretical analysis explicitly accounts for value, routing, and interaction contributions while retaining cross-block covariance terms.
  2. The experiments cover five vision-language models and provide evidence of cross-dataset transfer by applying directions estimated on CAC to Visual-RAG while keeping them fixed. Layerwise and component-level analyses, together with question-level bootstrap intervals, strengthen the empirical evidence.
Critical weaknesses

The paper shows that task-relevant information becomes easier to read out at an intermediate layer. However, further analysis is needed to show whether this improvement coincides with a sudden concentration of that information along a dominant direction. Tracking the relevant geometric measures across layers and comparing different readout methods would make this interpretation more convincing.

Questions for authors
  1. Before the transition layer, can linear probes, mean-difference directions, or multiple principal components already recover task-relevant information? Do the eigengap, directional alignment, and concentration of task-relevant variance also change abruptly at this layer?

  2. How would removing, amplifying, or replacing activations along the identified decision axis affect the model’s final Yes/No probabilities and answers? How do these effects compare with interventions along random directions or other principal components?

  3. Under the same models and data splits, how does the decision-axis readout compare with mean-difference probes, linear probes, tuned lens, and the model’s native P(Yes)P(\mathrm{Yes})? Could the authors report the full reranking results for the native-probability baseline described in the appendix?

  4. CAC labels images from the target class as relevant, but these images may not show the attribute being asked about. How do the authors distinguish whether the identified direction captures evidence sufficiency or simply category matching?

5 comments
AuthorsOpenReview ↗

Question 1 and Weakness 1 (Part 1)


Weakness 1 and Question 1: Layerwise Dynamics of Task-Relevant Information Before the Transition Layer

We further examine pre-transition readout performance and find that task-relevant information develops gradually but becomes abruptly concentrated along a spectrally dominant direction at the transition layer. Specifically, we compare readout methods at layer 0 and halfway to the transition layer in the CAC datasets (Exp. 1), with improvements in linear-probe and mean-difference performance suggesting that task-relevant information becomes progressively recoverable before the transition. Moreover, we examine PC1’s spectral and task-relevance properties, which reveal its abrupt emergence as a dominant task-relevant direction at the transition layer. Together, these results suggest that task-relevant information accumulates gradually, with its abrupt concentration along a dominant direction coinciding with the readout transition.

Exp. 1: Recoverability of Task-Relevant Information before the Transition Layer

We report AUROC on CAC for representation-level readouts at layers 0 and ⌊k/2⌋\lfloor k/2\rfloor in Table. 1 and Table. 2, respectively. Specifically, our readout projects standardized attention outputs onto PC1, with labels determining only its sign, whereas mean-difference readout uses the relevant–irrelevant mean contrast. Additionally, multiple-PC readout fits logistic regression on the leading m PC scores, with m1,2,4,8 selected by cross-validation within the fit split, while full-vector readout applies L2-regularized logistic regression to all standardized features.

Table. 1: Readout Performance at Layer 0 across Five VLMs

methodQwen3-VL-8B (0 / 36)Qwen2.5-VL-7B (0 / 28)InternVL2.5-8B-MPO (0 / 32)MiniCPM-V-4.5 (0 / 36)Ovis2.5-9B (0 / 36)mean of the five modelsmean relative to PC1 (ours)
Ours0.4990.4980.4970.5000.5180.5021.000
multiple PCs0.5490.5290.5300.5140.5180.5281.051
mean difference0.5400.5240.5360.5270.5440.5341.063
linear probe0.5840.5580.5520.5390.5550.5581.110

Table. 2: Readout Performance at Layer k/2 across Five VLMs

methodQwen3-VL-8B (9 / 36)Qwen2.5-VL-7B (8 / 28)InternVL2.5-8B-MPO (7 / 32)MiniCPM-V-4.5 (9 / 36)Ovis2.5-9B (10 / 36)mean of the five modelsmean relative to PC1 (ours)
Ours0.4850.4900.5100.5100.5190.5031.000
multiple PCs0.5210.5720.5060.5520.5110.5331.060
mean difference0.6160.7380.5550.6320.5380.6161.225
linear probe0.7470.8130.6700.7490.7270.7411.475

As shown in Table. 1, readout performance at layer 0 remains close to chance, with modest differences across methods, suggesting limited recoverable task-relevant information. While in Table. 2, full-dimensional supervised readouts, namely mean-difference and linear probe, exhibit greater task discrimination, whereas our method and the multiple-PC readout remain near chance. These results indicate that task-relevant information accumulates progressively before the transition layer but remains weakly expressed along the leading principal directions, becoming recoverable through supervised readouts.

AuthorsOpenReview ↗

Question 1 and Weakness 1 (Part 2)

Exp.2 :Spectral and Task-Relevance Properties before the Transition Layer

We further report spectral and task-relevance properties at layers 0 and ⌊k/2⌋\lfloor k/2\rfloor to characterize geometric changes preceding the readout transition. Specifically, at layers 0 and ⌊k/2⌋\lfloor k/2\rfloor, we compute the normalized test-covariance eigengap (1-2)/1, the squared cosine similarity between test PC1 and the fit-derived task axis, and the fraction of squared test-set class-mean separation captured by fit-derived PC1, as reported in Table. 3 and Table. 4, respectively.

Table. 3: Spectral Geometry at Layer 0 across Five VLMs.

quantityQwen3-VL-8B (0 / 36)Qwen2.5-VL-7B (0 / 28)InternVL2.5-8B-MPO (0 / 32)MiniCPM-V-4.5 (0 / 36)Ovis2.5-9B (0 / 36)mean of the five models
eigengap0.4800.4900.2410.3030.6570.434
directional alignment0.1470.1260.0000.1260.2970.139
concentration of task-relevant variance0.0050.0190.0210.0960.0240.033

Table. 4: Spectral Geometry at Layer k/2 across Five VLMs

quantityQwen3-VL-8B (9 / 36)Qwen2.5-VL-7B (8 / 28)InternVL2.5-8B-MPO (7 / 32)MiniCPM-V-4.5 (9 / 36)Ovis2.5-9B (10 / 36)mean of the five models
eigengap0.3240.5070.4040.3040.4190.392
directional alignment0.0010.0070.0180.0200.0370.017
concentration of task-relevant variance0.0140.0060.0180.0020.1230.033

As shown in Table. 3 and Table. 4, appreciable eigengaps coexist with low task alignment and task-relevant variance concentration across layers, suggesting that PC1’s spectral dominance precedes its emergence as a decision-relevant direction. These results suggest that PC1’s task relevance emerges with a change in alignment at the transition layer, rather than from spectral dominance.

AuthorsOpenReview ↗

Question 2


Question 2: Causal Validation of Interventions along the Decision Axis

We conduct causal interventions by amplifying or decreasing activation projections along the decision-relevant direction, other principal components, and random orthogonal directions, matching perturbation magnitudes in standardized activation space and measuring changes in output performance relative to the unedited baseline. Specifically, we edit the attention output at the final prompt token of the emergence layer as Ak′=Ak+sd⊙(αs1d),A_k' = A_k + \mathrm{sd}\odot(\alpha s_1 d), where d is a unit intervention direction, s1 is the standard deviation along the decision-relevant direction, and sd contains the coordinate-wise standard deviations used for standardization. The edited activation is then propagated through the remaining network to assess changes in the final Yes/No prediction. Our results in Table. 5 demonstrate that perturbing representations along or against PC1 causally steers model predictions across intervention strengths, with substantially larger effects than magnitude-matched perturbations along other principal components or random directions.

Table. 5: Causal Effects of Representation Perturbations across Five VLMs.

modelalphano editpushed along PC1pushed along other PCspushed along a random directionpushed against PC1
Qwen3-VL-8B0.50.605 (0.000)0.615 (+0.010)0.604 (-0.001)0.604 (-0.001)0.589 (-0.016)
Qwen2.5-VL-7B0.50.594 (0.000)0.602 (+0.008)0.591 (-0.003)0.592 (-0.002)0.580 (-0.014)
InternVL2.5-8B-MPO0.50.618 (0.000)0.638 (+0.020)0.615 (-0.003)0.619 (+0.001)0.591 (-0.027)
MiniCPM-V-4.50.50.652 (0.000)0.671 (+0.019)0.650 (-0.002)0.652 (0.000)0.642 (-0.010)
Ovis2.5-9B0.50.676 (0.000)0.684 (+0.008)0.678 (+0.002)0.677 (+0.001)0.673 (-0.003)
Mean0.50.629 (0.0000)0.642 (+0.0130)0.628 (-0.0014)0.629 (-0.0002)0.615 (-0.0140)
Qwen3-VL-8B10.605 (0.000)0.627 (+0.022)0.605 (0.000)0.603 (-0.002)0.570 (-0.035)
Qwen2.5-VL-7B10.594 (0.000)0.607 (+0.013)0.588 (-0.006)0.592 (-0.002)0.570 (-0.024)
InternVL2.5-8B-MPO10.618 (0.000)0.650 (+0.032)0.610 (-0.008)0.621 (+0.003)0.559 (-0.059)
MiniCPM-V-4.510.652 (0.000)0.680 (+0.028)0.649 (-0.003)0.651 (-0.001)0.633 (-0.019)
Ovis2.5-9B10.676 (0.000)0.690 (+0.014)0.678 (+0.002)0.676 (0.000)0.665 (-0.011)
Mean10.629 (0.0000)0.651 (+0.0218)0.626 (-0.0030)0.629 (-0.0004)0.599 (-0.0296)
Qwen3-VL-8B20.605 (0.000)0.640 (+0.035)0.603 (-0.002)0.602 (-0.003)0.539 (-0.066)
Qwen2.5-VL-7B20.594 (0.000)0.617 (+0.023)0.582 (-0.012)0.590 (-0.004)0.549 (-0.045)
InternVL2.5-8B-MPO20.618 (0.000)0.688 (+0.070)0.597 (-0.021)0.622 (+0.004)0.528 (-0.090)
MiniCPM-V-4.520.652 (0.000)0.685 (+0.033)0.647 (-0.005)0.650 (-0.002)0.603 (-0.049)
Ovis2.5-9B20.676 (0.000)0.696 (+0.020)0.676 (0.000)0.676 (0.000)0.654 (-0.022)
Mean20.629 (0.0000)0.665 (+0.0362)0.621 (-0.0080)0.628 (-0.0010)0.575 (-0.0544)

As shown in Table 5, perturbations along PC1 consistently improve accuracy, whereas opposite perturbations reduce it, demonstrating causal steering of model predictions. Across models and intervention strengths, the mean absolute accuracy change under bidirectional PC1 interventions is approximately 6.0 and 16.3 times that of the averaged other-PC and random-direction controls, respectively, supporting PC1’s privileged causal role in answer selection. These results distinguish the identified direction from the tested alternatives through its stronger causal influence on model predictions, supporting its privileged role and providing a functional basis for early decision readout from intermediate representations. We have added the causal intervention experiments and the corresponding discussion in **Section 5.5 **of the revised manuscript to further establish the causal significance of the identified decision-relevant directions.

Reviewer_39fnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper tries to answer: whether a task-relevant direction emerges inside language model's representation across layers.

Their core idea is: if such direction exists and prevails, the first principal component of the (attention-output) activations should align with the difference between the mean activations of positive and negative samples. In other words, this direction dominates the variation in activation space. This direction is the task-relevant direction, and it should be generalizable to unseen data. This can be referred to as PCA-based readout.

The experiments show that: across five VLMs, PCA-based readout performance increases sharply at an intermediate transformer layer. PCA directions estimated using the CAC dataset generalize to Visual-RAG benchmarks outperforms baseline methods and logit lens. The authors also justify the ideas with theoretical derivations in section 3.

Critical strengths

It is insightful to link the task-relevant direction with the PCA of the activation. And the paper demonstrate its effecctiveness in different ways: section 3, together with table 3 shows that the alignment indeed exists. Table 1 shows that this PC1 direction has a strong generalization ability, and table 2 shows the emergence across layers. This experimental framework is insightful and rigorous.

Critical weaknesses

There are several outstanding weakness in this paper, however, the most major ones (weakness 1 and 2) are fixable, and I will happily increase my score if they are fixed by the end of the discussion period.

  1. Direct validation of direction alignment. The section 3 (theoretical derivations) gives us formula 3.5 and 3.11 which is a lower bound for alignment score. Table 3 gives us the lower bound of alignment score using the formula 3.5. I agree that this is a valid way to show that alignment score between PC1 and task-relevant direction (difference between the mean activations of positive and negative samples) is high. However, to fully ensure that the chain of evidence is complete and closed, you should also report the actual measured alignment score, shows that this is indeed higher than what your theoretical boundary suggests. This serves as a final check that your theory in section 3 is correct, as well as illustrating how tight your lower bound is. This should be an easy fix in the paper if my understanding is correct, so please update your result during the discussion period.

  2. Task diversity Throughout the paper, only one task (whether the image provides enough information to solve the question) is involved. The paper's title is "Do Language Models Dream of Answers before They Speak?", so another text-based (meaning that there is no image) task should be evaluated in the paper. For this weakness, I recommend that you pick a text-based binary classification task, choose 2 language models and reproduce table 1, table 2 and weakness 1 (actual alignment score).

  3. Casual validation The paper is operating on a representation level, and it looks like no causal experiments are shown. It would be nice if you can demonstrate that if we add/subtract the PC1 to representation, it will steer the prediction of models.

For now, I recommend weak reject based on the original paper. However, if weakness 1 and 2 are properly resolved, I will raise the score to at least 3 (weak accept).

Update: the weaknesses are properly resolved, and I raise my score to 3 (weak accept).

Questions for authors
  1. How would your theory generalize to tasks beyond binary classification? In that case, the meaning of task-relevant direction is no longer obvious.
7 comments
AuthorsOpenReview ↗

Weakness 1


Weakness 1: Direct validation of direction alignment

We thank the reviewer for this suggestion. To address this concern, we conducted an additional experiment to directly measure the alignment between the decision-relevant direction and the reference decision axis and assess the validity and tightness of our theoretical lower bound. As shown in Exp. 1, the estimated lower bounds closely approximate the measured alignments from below across all five models, providing direct empirical support for our theoretical guarantees. We have updated Table 3 and the corresponding discussion in Section 5.4 of the revised manuscript to incorporate the directly measured alignment scores and bound tightness analysis.

Exp. 1: Validation of the Alignment Bound and Its Tightness

Following the protocol in manuscript, we compute the squared cosine similarity between the decision-relevant direction estimated from the CAC test split and the reference decision axis estimated from the fit split as actual alignment score. We compare the directly measured alignment against the theoretical lower bound and report their ratio to assess bound tightness across all five models.

Table.1: Alignment Scores Across Five Models.

modelAlign (actual)Align_RA (estimated)Ratio (Align_RA / Align)
Qwen3-VL-8B0.9740.9630.989
Qwen2.5-VL-7B0.9580.9330.973
InternVL2.5-8B-MPO0.9140.8840.968
MiniCPM-V-4.50.9760.9590.983
Ovis2.5-9B0.9250.9030.976
Mean0.9490.9280.978

As shown in Table 1, the directly measured alignments consistently exceed their theoretical lower bounds across all five models. The lower bounds capture 96.8%–98.9% of the measured alignment, with an average ratio of 97.8%. These results provide direct empirical validation of our theoretical alignment guarantees and demonstrate their tightness across different model architectures. They also strengthen the connection between the spectral characterization in Section 3 and the observed geometry of model representations.

AuthorsOpenReview ↗

Weakness 2 (Part 1)


Weakness 2: Task Diversity

We thank the reviewer for this constructive suggestion. Following the reviewer's recommendation, we extend our experiments beyond the original visual evidence sufficiency task to two text-based binary classification tasks using two language models, Mixtronix-8B and Mistral-NeMo-8B.

  1. Task 1. Knowledge-Based Question Answering: Predicting binary answers (Yes/No) to knowledge-based questions.
  2. Task 2. Passage Relevance Classification: Determining the relevance of retrieved passages to a given question for evidence reranking.

For each task, we reproduce the three key evaluations requested by the reviewer from the original manuscript.

In Exp. 2, we extend the functional validation (Table 1 in the original manuscript) to two text-based binary classification tasks. We evaluate the predictive performance of decision-relevant readouts on knowledge-based question answering (Exp. 2, Table 2) and passage relevance classification (Exp. 2, Table 3) through a reranking evaluation against established baselines.

In Exp. 3, we reproduce the architectural analysis (Table 2 in the original manuscript) to examine the emergence of decision-relevant readouts across transformer components. We evaluate this emergence separately for knowledge based question answering (Exp. 3, Table 4) and passage relevance classification (Exp. 3, Table 5).

In Exp. 4, we validate the theoretical alignment guarantees in the text-only setting by directly measuring the actual alignment scores between the decision-relevant directions and their reference decision axes. We compare these measurements against the theoretical lower bounds and validate the spectral dominance conditions for each task and language model (Exp. 4, Table 6).

AuthorsOpenReview ↗

Weakness 2 (Part 2)

Exp. 2: Generalization to Diverse Text-Only Tasks

To evaluate task generalizability, we extend our framework beyond the original visual evidence sufficiency task to two additional text-based tasks, knowledge-based question answering and passage relevance classification, using StrategyQA and HotpotQA. Specifically, StrategyQA serves as the counterpart to CAC, with fit, development, and test splits for readout estimation, head selection, and held-out theoretical validation, respectively. HotpotQA serves as the counterpart to Visual-RAG for downstream evaluation. Its native yes/no questions are used to evaluate knowledge-based question answering (Table 2), while its question–passage pairs and supporting evidence annotations are used to evaluate passage relevance classification through reranking (Table 3). These experiments assess the predictive effectiveness and cross-task generalization of our framework.

Table. 2: Knowledge-Based Questions Answering performance on HotpotQA

modelprediction (ours): accuracyprediction (ours): AUROCdirect answer: accuracydirect answer: AUROCaccuracy: ours / directAUROC: ours / direct
Mistral-NeMo-8B0.8600.9320.8970.9710.9590.960
Mixtronix-8B0.8470.9330.9060.9710.9350.960

Table. 3: Passage Reranking Performance on HotpotQA

methodmAP [95% CI]MRRnDCGhit@1R@2
BM250.642[0.615,0.671]0.8270.7800.7330.532
CLIP ViT-L/14-3360.535[0.506,0.564]0.6430.6890.4570.398
DPR-multiset0.663[0.634,0.692]0.7810.7840.6500.525
logit lens on g13, Mixtronix-8B0.667[0.638,0.695]0.7700.7840.6400.517
UniME-V2, Qwen2-VL-2B0.686[0.654,0.718]0.7800.7950.6600.565
Contriever-MS MARCO0.687[0.659,0.716]0.8250.8050.7230.562
VLM2Vec-Full, Phi-3.5-V0.689[0.659,0.718]0.7830.7980.6500.562
SPLADE++0.702[0.676,0.730]0.8590.8190.7730.587
UniME-V2, Qwen2-VL-7B0.714[0.684,0.742]0.8210.8190.7030.592
logit lens on g13, Minitron-8B0.740[0.713,0.768]0.8370.8360.7200.615
MS MARCO MiniLM-L120.750[0.726,0.777]0.9030.8540.8370.633
E5-large-v20.763[0.736,0.790]0.8640.8520.7700.653
GME-Qwen2-VL-2B0.772[0.745,0.799]0.8670.8580.7700.677
BGE-large-en-v1.50.797[0.770,0.823]0.8880.8750.8100.688
monoT5-3B0.799[0.773,0.824]0.8970.8770.8300.690
E5-Mistral-7B-instruct0.802[0.776,0.829]0.8700.8740.7830.703
GME-Qwen2-VL-7B0.817[0.794,0.841]0.9020.8880.8270.720
g13 Mixtronix-8B0.834[0.810,0.858]0.9090.8980.8470.728
g13, Mistral-NeMo-8B0.833[0.809,0.858]0.9030.8970.8330.732

As shown in Table 2, our decision-relevant readouts achieve performance comparable to the LLMs' native predictions on knowledge-based question answering, retaining approximately 96% of their AUROC. This demonstrates that the emerging internal representations capture information predictive of the models' final answers. In Table 3, the readouts outperform established retrieval and reranking baselines on passage relevance classification, achieving over 0.833 mAP in downstream reranking and demonstrating their practical effectiveness in a distinct decision task. These results extend our findings beyond visual evidence sufficiency, providing further evidence that decision-relevant representations support accurate predictions and effective downstream decisions across diverse text-based tasks.

Reviewer_53CvOpenReview ↗
Rating
2(+1)
Confidence
3(+1)
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The authors try to establish a solid theoretical characterization for the latent representation dynamics of vision-language models (VLMs) inference, and deliver some theorems that relate covariance structures of the representation space to the VLMs' answer decision state. They conclude that the decision state is the dominant direction of variation, which can be traced from attention outputs. They show their theoretical statements/conditions hold in practice and demonstrate the proposed interpretable scoring tool's usefulness in a visual rag application scenario with some open-source VLMs.

Critical strengths
  • Understanding the underlying mechanism of VLMs' inference through theoretical characterization of representation dynamics is very important and timely problem.
  • The authors built some theoretical statements on this line.
Critical weaknesses
  • Remarkably bad writing quality (especially Section 3 and 4) makes it hard to precisely evaluate the quality of content: I was really focusing about 2~3 hours to read the paper end-to-end. I could follow Section 1 and 2 but from Section 3, it was really hard to grasp what are the core messages the authors want to convey, what's their core contribution/originality. Dear authors, could you please revise Section 3 significantly? I believe the desired structure is to provide a key summary of what you want to claim and the implications of each theory (See Oh et al. 2025 and Kiyani et al. 2026a;b for example).
  • Mismatch between theoretical framework and practical implementation: the theory builds justification for attention output, but the practical algorithm the authors validated doesn't use the attention output representation directly.
  • Poor sustainability/extensibility to image domain / prompt shifts: the proposed method requires re-fitting PCA every time when facing new prompts or image category/domains.
  • Narrow application by design of the framework itself: they add “is this image sufficient to answer the question above correctly? answer with only yes or no” on top of the original prompt. ⇒ The quantity they tested is computed over 'whether the image would let it answer' rather than the original question, which is what we are ultimately interested in (See Long et al. 2026, for example).
  • Their demonstrations on downstream applications are too limited: the authors only show VisualRAG with closed-form QA, far from realistic multimodal LLM usages such as open-ended visual QA or GUI agents / embodied agent tasks; it is also unclear whether this interpretable readout can be used for intervention rather than limited to post-hoc analysis
  • Thin empirical evidence: In their visual RAG experiments, they do not provide a reasonable baseline -- using the P(YES) from the model directly as a relevant score. It is unclear how effective their proposed score is compared to a much simpler and intuitive approach.
  • Thin contextualization/positioning: the authors do not discuss how does their statements/observations are connected or different to the prior approaches and observation (Jiang et al. 2025, Long et al. 2026)
  • Title-content mismatch: the title pretends that this work investigates language models, but actually they only focus on VLMs' visual question answering scenario.
  • (minor? or significant?): the authors use a non-standard text font (not the ICLR formatting instruction default).

Reference

  1. Oh et al. 2025, Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach
  2. Kiyani et al. 2026a, When to Trust the Cheap Check: Weak and Strong Verification for Reasoning
  3. Kiyani et al. 2026b, Strategic Decision Support for AI Agents
  4. Long et al. 2026, UNDERSTANDING LANGUAGE PRIOR OF LVLMS BYCONTRASTING CHAIN-OF-EMBEDDING
  5. Jiang et al. 2025, Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens
Questions for authors

Check the weaknesses above

13 comments
AuthorsOpenReview ↗

Weakness 1 & 2


Weakness 1: Readness

We thank the reviewer for the feedback and the time devoted to carefully reading our manuscript. We have substantially rewritten Sections 3 and 4, with major revisions to the theoretical organization, explanations of key results, and connections to empirical validation.

  1. Restructured Section 3 around two central theoretical contributions. Section 3.1 establishes spectral dominance and alignment guarantees (Theorem 3.2), together with readout stability (Corollary 3.3). Section 3.2 connects global decision geometry to attention-level value, routing, and interaction contributions (Proposition 3.4 and Corollary 3.5).

  2. Reorganized Section 4 to clarify the practical implications of our theory. We explicitly connect decision-axis identification (Eq. 12), representation readout construction (Eq. 13), and downstream applications (Eqs. 14–15), establishing a clearer progression from theoretical characterization to practical implementation.

  3. Strengthened the connections between theoretical claims and empirical evidence. We revised the experimental discussions to clarify the role of downstream reranking (Table 1 in original paper), attention-level readout emergence (Table 2 in original paper), and spectral assumption and alignment validation (Table 3 in original paper).

Throughout these revisions, we have added section-level summaries, clarified the claims and implications of individual theoretical results, streamlined mathematical explanations, and moved supporting technical details to the appendix. We believe these substantial changes make the central contributions and their empirical significance considerably clearer.


Weakness 2: Relationship between theoretical framework and main results.

We would like to clarify that attention-output representations are directly used in our main experiments, including downstream reranking (Table 1 in original paper), decision-readout emergence (Table 2 in original paper), and theoretical assumption and alignment validation (Table 3 in original paper). Specifically, Table 1 in original paper directly applies the attention-output readout to downstream reranking, while Table 2 in original paper in original paper identifies attention outputs as the stage where decision-relevant information becomes strongly readable. Table 3 in original paper further evaluates the spectral conditions and alignment guarantees established for attention-output representations. The block-output readout in Eq. (15) is additionally used to estimate native model response probabilities, while the component-level comparisons in Table 2 in original paper trace the propagation of decision-relevant information.

To make this connection more explicit, we have substantially revised Sections 3 and 4 and clarified their correspondence with the empirical evaluations.

  1. Theoretical characterization (Section 3 Table 3 in original paper). We establish spectral dominance and alignment guarantees for attention-output representations, with Table 3 in original paper directly evaluating the corresponding theoretical conditions and measured alignments.

  2. Decision-axis identification (Section 4.1 Table 2 in original paper). We explicitly define decision-axis estimation and representation readouts in Eqs. (12–13). Table 2 in original paper traces their emergence across transformer components, identifying attention outputs as the key stage where decision-relevant information becomes strongly readable.

  3. Downstream application (Section 4.2 Table 1 in original paper). We explicitly formulate attention-output-based reranking in Eq. (14), which is directly evaluated on Visual-RAG in Table 1 in original paper.

We have also refined the experimental discussions to make these connections clearer and streamlined the appendix to retain methodological details directly relevant to the main text.

AuthorsOpenReview ↗

Weakness 3


Weakness 3: Sustainability/Extensibility to Image domain / Prompt shifts

We would like to clarify that our framework does not require refitting for each new image domain or prompt template. In the original manuscript, the decision-relevant readouts are estimated on CAC and directly transferred to Visual-RAG without refitting, demonstrating their generalization across image domains. To further address the concern about prompt shifts, we conduct an additional experiment to evaluate the transferability of fixed decision-relevant readouts under a different prompt template, without any re-fitting procedure.

Exp. 1: Sustainability/Extensibility Across Tasks and Prompt Templates Without Refitting

Following the original manuscript, we retain the decision-relevant directions and associated readout parameters estimated on CAC using the original reranking prompt. We then evaluate these fixed readouts on Visual-RAG validation and test questions paired with gold images, using a distinct question-answering prompt that instructs the model to answer the original question rather than assess image relevance. Importantly, no readout parameters are re-estimated or updated for the new prompt. We compare the resulting readout predictions against the models' native answering performance in Table 1.

Table. 1: Readout vs. Native Answering under Prompt Shifts without Refitting

modelprediction (ours): accuracyprediction (ours): AUROCdirect answer: accuracydirect answer: AUROC
Qwen3-VL-8B0.6470.6880.3810.599
Qwen2.5-VL-7B0.4120.5460.4850.650
InternVL2.5-8B-MPO0.6740.6940.5680.634
MiniCPM-V-4.50.3480.7470.6650.739
Ovis2.5-9B0.7110.7370.5350.630
Mean prediction / direct1.0591.050——

As shown in Table 1, the fixed decision-relevant readouts retain predictive effectiveness under a different question-answering prompt, achieving aggregate accuracy and AUROC comparable to the models' native answering performance without any refitting. Together with the cross-domain transfer from CAC to Visual-RAG demonstrated in the original manuscript, these findings provide evidence that the identified decision-relevant representations can be reused across image domains and prompt templates without repeated direction estimation, supporting the transferability of our framework beyond its original evaluation setting.

AuthorsOpenReview ↗

Weakness 4


Weakness 4: Application of the framework

In our original manuscript, visual evidence reranking serves as an application showcase to demonstrate the practical utility of our framework, with evidence sufficiency judgments guiding evidence selection in retrieval-augmented generation (RAG). This application does not limit our framework to evidence sufficiency judgments. Building on the findings in Exp. 1, we conduct further experiments on direct question answering, preserving the original questions and answer targets without reformulating the task, to demonstrate the broader applicability of our framework.

First, as demonstrated in Table 1, we evaluate the fixed decision-relevant readouts on Visual-RAG using a direct question-answering prompt rather than the original evidence sufficiency instruction. The readouts achieve aggregate accuracy and AUROC comparable to native answering across five VLMs, demonstrating that the identified representations retain predictive utility when the models directly address the original questions.

Exp. 2: Broader Applicability of the Proposed Framework in Direct Answer Prediction.

We further evaluate our framework on Visual CounterFact, where correct answers require image-specific visual evidence. Under this setting, we extend our decision-relevant readout to multiple-choice question answering, enabling direct prediction of the original answers without reformulating the questions or modifying their answer options.

Table. 2: Readout versus Direct Answering on Visual CounterFact Across Five VLMs

modelprediction (ours): accuracyprediction (ours): AUROCdirect answer: accuracydirect answer: AUROC
Qwen3-VL-8B0.8830.9590.9660.998
Qwen2.5-VL-7B0.7400.9050.9650.998
InternVL2.5-8B-MPO0.8920.9640.9640.998
MiniCPM-V-4.50.7450.9100.9640.998
Ovis2.5-9B0.8620.9610.9710.999
Mean prediction / direct0.8530.942——

As shown in Table. 2, our readout retains most of the models’ native answering performance, which is the model-specific reference ceiling. These results demonstrate that our framework remains effective for multi-alternative question answering, supporting its adaptability to different task formulations beyond reranking. As shown in Table 2, our decision-relevant readouts retain approximately 94% of AUROC across five VLMs. Importantly, the readouts predict the original multiple-choice answers rather than binary evidence sufficiency judgments. The counterfactual setting further requires predictions to reflect image-specific attributes, as changes in visual attributes alter the correct answers while preserving object categories. These results demonstrate that the identified decision-relevant representations capture information predictive of the original answers, extending the applicability of our framework beyond the evidence reranking task considered in the original manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.