- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper looks at LLMs answering Yes/No questions, investigating to what extent the models have an early preference (while reading the first part of the question) for an answer, and how such early preference correlates with final answer and accuracy. They measure this answer preference score (PS) by aborting the question part way and looking at the Yes/No logits following an answer prompt. In the cleanest setting, for NP-BoolQ with pairs of questions sharing at least 80% common prefix, but opposite answers, they show a substantial gap in accuracy between gold answers that aligns with that early preference vs the unaligned ones (e.g., 96.1% vs 82.4% for Qwen3-8B).
The paper then shows a correlation between swapping certain hidden states in the question text (given different paraphrased passages in CONDAQA) and their preference scores as well as final answer - using this to conclude (?) a causal relationship between such provisional answer preference (encoded in hidden states) and final answer. There are further experiments learning an activation steering vector indicating the answer preference, and showing the expected effect of applying this vector to early hidden state translating to final answer shifts. Finally, they do circuit analysis using position-aware edge attribution patching, arriving at similar conclusions about the causality of early answer preference.
Critical strengths
Provides interesting, direct evidence for the intuitive notion that LLMs are biased by answer preferences that get formed while reading the question (and background passage).
Considers a reasonably diverse set of models (albeit of limited size) and datasets (limited by the requirement to naturally involve some contrastive cases), including complementary approaches in support of hypothesis.
Critical weaknesses
The activation swapping in Section 5.1 seems to establish correlation more than causation, although it feels like the later sections makes a stronger case.
It is not clear what the takeaways would be for future model training or usage, in terms of overcoming such biases.
Questions for authors
Did you measure the total probability mass on the provisional answers? I imagine if the prefix is sufficiently ambiguous, like "Is it legal to? Please answer yes or no.", models might want to push back rather than following the yes/no instruction. It might still be directionally valid of course, as indicated in your Fig 2, but could be interesting to see to what extent this is what the model would naturally output (probably varies a lot between models).
3 comments
Thank you for the careful responses! My overall assessment of the work stays the same.
Official Comment by Authors
Thank you for your careful reading and follow-up. Your questions helped us clarify how activation swapping contributes to the causal argument, examine whether Yes/No outputs remain likely for incomplete questions, and articulate the implications for evaluating models’ ability to revise early preferences. We appreciate your constructive feedback and the opportunity to make the evidence and its implications clearer.
Official Comment by Authors
We thank the reviewer for the thoughtful questions which helped strengthen our analysis.
Weakness 1: Causal interpretation of activation swapping (Section 5.1)
We thank the reviewer for encouraging us to clarify what activation swapping establishes and how it contributes to the overall causal argument. To highlight differences in provisional preference as clearly as possible, we paired inputs with the same question and GT answer, using meaning-preserving passage paraphrases to hold answer-relevant semantic content fixed.
Activation swapping shows that question-prefix hidden states carry provisional answer preference and causally influence both provisional PS and the final-answer margin. The changes in both provisional preference score and the final-answer margin correlate with the pre-swap preference difference between the paired inputs. These correlations connect the causal effects of the representations to provisional preference.
The swapping results also provide the basis for the steering analysis in Section 5.2. We select a layer with strong preference transfer and fit a steering vector at that layer. Steering shifts both provisional preference scores and final-answer margins, providing a more targeted test of the causal role of provisional preference.
We will revise the last paragraph of Section 5.1 to clarify what activation swapping establishes:
Weakness 2: Takeaways for model training and usage
We thank the reviewer for encouraging us to clarify the practical implications of our findings. Our results highlight the importance of revising an early answer preference when later question words require the opposite answer. The lower accuracy on conflicting questions identifies this as a capability that current models struggle with.
These findings highlight the need for datasets that explicitly target preference revision. Shared-prefix pairs with opposite correct answers, as in our NP-BoolQ shared-prefix setting, provide a suitable format. A model cannot answer both questions correctly by following the same prefix preference. Such datasets could provide training supervision that encourages models to use later question information to revise an initial preference.
For usage, these pairs can serve as targeted evaluations of models’ ability to handle answer-changing completions. High accuracy on aligned questions can keep overall accuracy high even when accuracy on conflicting questions is much lower. Reporting aligned and conflicting accuracy separately helps users identify which models are better at revising an incorrect initial preference.. Future work could also examine whether provisional preferences bias final decisions in vision-language models. Shared-prefix pairs with a fixed image could extend this dataset design to multimodal question answering.
Question 1: Probability mass on Yes and No at the provisional readout
We thank the reviewer for highlighting the importance of checking whether Yes/No outputs remain likely for incomplete questions. Table R4 reports the mean total probability mass on the Yes and No token sets used for PS (Appendix A.5).
Table R4: Mean probability mass on the Yes/No token sets. Results use the 1,048 CONDAQA validation questions. Columns indicate word-based question progress; 0% includes the passage but no question words.
At the 20% and 40% checkpoints, where prefixes may still be highly ambiguous, the first three models assign at least 0.95 mean probability mass to Yes/No. Phi-4 assigns 0.788 and 0.942, respectively, so Yes/No still receives most of its probability mass on average. The clearest exception is Phi-4 at 0%, with a mean mass of 0.122. These results indicate that Yes/No outputs generally receive substantial probability mass under our prompt even when much of the question is still missing.