- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors propose a new probe that attend only on specific parts of the input. They apply it on several settings and get some results. I could not really understand most of the paper, if the probe or results are interesting, how the prove is built etc.
Critical strengths
- good coverage of tested models
- the proposed probe seem to do something consistently positive compared to the baselines, but I find it hard to understand what it is
- There seem to be interesting understandings from the probe, but it is very hard for me to understand them, whether they are really surprising (e.g. "Which reading is right is decidable only after the answer exists" seems trivial).
Critical weaknesses
-
Why using MRQA as the benchmark?
-
It was hard for me to read the paper. Examples: (a) The intro is not clear; (b) The method section conflates between analysis (not clear on which benchmark) and modeling, and it is in general hard to read (all sub sections); (c) answer-row is not defined; (d) I am not sure what do the D4/5/2 markers mean; (e) "for i ∈ T" - what is T? etc.
-
"δencS" in 20% - why is this surprising? ℓ¬S (y | x) encodes unconsistenly - the question with S, the answer without S but over an S-encoded question, which may include internal state aspects that cannot be recovered any more.
-
"the first-token log-probability ranks the better reading higher in only 55% of pairs" - could it be the the first answer token is quite generic and not related to S?, e.g. "the answer is" or "the capital of is" - in both, "the" is a generic starting point and most likely have no strong relatedness to S.
-
In Eq 5, why not also have a way to have only log pRθ(v) - testing directly only the flow in R? This seems a more direct measure than β=1 for the flow via R.
Questions for authors
-
See comments above
-
I strongly encourage the authors to rewrite and resubmit.
5 comments
Response to Reviewer vgGr (2/3): is the main finding trivial?
Is "which reading is right is decidable only after the answer exists" trivial? Put in words, it can sound obvious. Existing decoders, however, are built on the opposite assumption, and our measurements contradict several views held in prior work. That is what makes the findings informative.
1. Prior decoders decide per token; the decision needs the complete answer. CAD, COIECD, DAGCD, DoLa and SLED all fix one adjustment before generating and apply it at every token. They implicitly assume that the next-token distribution already carries the signal of which reading is right. We tested this assumption directly (all 62,316 questions; pairs of candidate answers whose F1 differs by at least 0.5):
| How the better of two answers is picked | Accuracy |
|---|---|
| first-token probability (what a per-token decoder sees) | 55% |
| complete answer's likelihood | 65% |
| complete answer contained in the passage | 66% |
| both together (our rule) | 75% |
The two answers already differ at the first token in 83% of pairs, so a per-token decoder commits at step one, yet its signal there is close to a coin flip. Acting on the complete answer instead is worth +3.0 to +4.2 F1 on five models, and the same step lifts CAD from 65.96 to 70.02 and COIECD from 67.20 to 69.92. It also explains why two of our four readings, each worse than greedy decoding on its own, still add accuracy: a reading only needs to be right where the others are wrong, and only a decision made after the answers exist can use that.
2. Findings that differ from prior views.
| Prior view | What we find |
|---|---|
| CAD (Shi et al., 2024): comparing the model with and without the context isolates what the context contributes. | Deleting the passage also changes how the question is encoded, and this part of CAD's score ranks the better answer first in only 20% of pairs, below the 50% of chance. The part that reflects the passage itself (ι_S) reaches 87%. |
| DAGCD (Huang et al., 2025) and Lookback Lens (Chuang et al., 2024): context-faithfulness errors come from insufficient attention to, or use of, the context. | Hallucinated answers rely on the passage as much as correct ones: blocking it lowers their log-likelihood by 3.42 vs 3.76 nats per token. What differs is the route: wrong answers depend more on the first four layers (0.87 vs 0.31) and on punctuation tokens (0.34 vs 0.10). |
| OPERA (Huang et al., 2024): over-attending to a few summary tokens causes hallucination, so it should be penalised. | Punctuation tokens do take 21–35% of the answer's attention from layer 5 on, yet blocking them barely changes a correct answer. They are a high-traffic, low-content route that helps on some questions and misleads on others. This is why we use both directions as readings (anchor contrast and anchor removal) instead of penalising one. |
3. Implications. Interventions on how a model reads should be judged at the answer level, not per token. Input deletion should not be used as the reference when the goal is to score how much an answer relies on its source. And hallucination can be addressed by changing which route the model reads through, not only how much it attends to the context. Blocking what the answer may attend to is a cheap and exact tool for such analyses: it leaves the prompt's representation untouched, and several interventions share one pass over the prompt.
Response to Reviewer vgGr (3/3): δ^enc, first tokens, and log p_R
Why is δ^enc at 20% surprising? The reviewer's reading of the mechanism is correct. Here is why the number is still surprising. δ^enc compares two computations in which the answer cannot see the passage S, and which differ only in whether the question was encoded with S or without it. That difference says nothing about whether a candidate answer is supported by S. A term that carries no information about which answer is better should rank the better one first about half the time, i.e. 50%. It does so in only 20%: it systematically favours the wrong answer. The reason is visible in the data. A question encoded without its passage evokes the model's most typical answer, and contrasting against that penalises every answer the model also knows, often the correct one. This term sits inside every input-deletion score, which is why CAD ranks at chance on TriviaQA and SearchQA (Finding 1). An intervention that keeps the question's encoding fixed removes it by construction (Section 4.1). The revision now states this expectation explicitly (Section 3 and Finding 1).
Are first tokens generic? This is a good alternative explanation, so we tested it. Most first tokens are content-bearing: 86% are content tokens, and only 3.5% are function words such as "the". On the 12,247 pairs where both first tokens are content words, first-token accuracy is still 55.5%, against 72.9% for complete answers. On pairs whose first tokens differ, it is 56.4% against 75.3%.
What this means. The low first-token accuracy is not an artefact of generic tokens. Even when the first token is a content word, such as the start of a name or a number, it does not reveal which reading is right. The information appears only once the answer is complete and can be checked against the passage. This is why the decision has to be deferred to complete answers (the second principle), and why a token-level decoder cannot exploit readings that are right on different questions (Finding 2; Appendix F).
"Only log p_R" in Eq. 5. As a reminder, Eq. 5 builds a reading from two distributions over the next token: the unmodified model p_θ, and p_θ^¬R, the same model with its answer positions blocked from attending to a region R (e.g. the passage). The reading is log p^(R,β) = (1+β) log p_θ − β log p_θ^¬R. With β = +1, it amplifies what flows through R; with β = −1, it removes R. The reviewer proposes the complement: a distribution p_R in which the answer positions attend only to R.
We implemented it. Letting the answer positions see only the passage would also hide the question, so this variant lets them see the passage and the question. Used alone it scores 49.45 F1, and its β = 1 contrast 63.60, against 67.43 for our passage contrast (fresh 7k panel, Llama-2). Ranking candidates by the passage-blocked likelihood alone is worse still (13–33 F1).
Why. Allowing only R removes everything else the answer needs: the instruction, the answer format, and the anchors outside the passage. The result measures a crippled model rather than the flow through R. Blocking R changes one thing and keeps everything else, so the difference between p_θ and p_θ^¬R isolates exactly the contribution of R. This is why Eq. 5 contrasts with p_θ^¬R (Appendix F).
We took this advice seriously and rewrote the paper within the discussion period. In short, it contributes:
- A diagnosis of existing decoders. An exact decomposition shows that deleting the passage, as CAD does, mixes the evidence with a re-encoding of the question that systematically favours wrong answers (20% vs 50% chance). We also show that the right reading cannot be identified at the first token (55%) but can be once the answer is complete (75%).
- A method. Readings built by blocking only what the answer may attend to, which keeps the question's encoding intact and costs one shared pass over the prompt, plus a decision among complete answers by the unmodified model's belief and the source's support. It is training-free and has no tuned hyperparameters. It gains +3.0 to +4.2 F1 on five models, is the only method that improves every domain on every model, and lifts every prior decoder it wraps.
- Findings on how hallucinated answers read. They read the passage as much as correct answers do, but through shallow layers and punctuation "summaries", a high-traffic, low-content channel.
We would be grateful if the reviewer could take another look at the revised version.
Response to Reviewer vgGr (1/3): the method, MRQA, readability
We thank the reviewer for the frank assessment, which made us restructure the paper (revised PDF, changes in blue). We answer each point below.
What the method is. AnswerProbe is a decoding method: it changes how a model generates an answer so that the answer is correct more often. It is not an analysis probe. "Probe" in the title refers to its last step, in which each candidate answer is checked against the source it should come from.
The problem. When an LLM answers from a passage, many wrong answers are errors of reading, not of knowledge. If the same model reads each MRQA prompt in four slightly different ways and the best of its four answers is kept, F1 rises from 67.0 to 74.7. What fails is how the passage is read, and which reading is trusted.
How existing decoders work, and where they fall short. Most context-faithful decoders follow one template, contrastive decoding: compare the model with a weakened copy of itself that lacks some evidence, and boost the tokens whose probability the evidence raises. They differ only in how the weakened copy, the reference, is built:
All of them build one such reading and apply it to every token of the answer. This has two consequences, which the paper calls blind spots.
Our method, in the same terms. AnswerProbe keeps the contrastive template but changes three things.
Prior decoders are special cases of this framework: one reading, one strength, decided per token (Appendix Table 6). Conversely, giving them several strengths and our decision rule lifts every one of them, e.g. CAD from 65.96 to 70.02 F1 (Table 3).
Why MRQA? MRQA is the standard benchmark of context-faithful decoding. CAD, COIECD and DAGCD all report on it, and following the public DAGCD protocol makes their published numbers directly comparable to ours. It spans seven domains, including knowledge conflict (NQ-Swap). The revision adds MRQA's six out-of-domain test sets (+2.9 F1, positive on all), multi-document HotpotQA, abstractive NarrativeQA, and closed-book TruthfulQA.
Readability. Thank you; we agree and revised the paper accordingly: