- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies whether LLMs that answer two-hop questions correctly actually chain the two facts given in the context. A correct answer could also come from pretraining or by picking an answer passage without using the first fact. To test this, the authors edit one fact at a time and check whether the model follows the edit and gives the edited fact. With Llama-3.1-8B and Qwen3-4B on MuSiQue, 2WikiMultiHopQA, and a synthetic task, many correct answers do not survive these edits. This is more common when entities have real names. The paper then patches hidden states between prompts. It argues that middle layers carry the intermediate entity, while later layers carry the final answer. It also points to a few attention heads that recur across tasks. Last, it shows that probes on these middle layers can predict, from the original prompt alone, whether the model will pass the edits.
Critical strengths
- The intuition is well-motivated, and the edit-based test is a simple way to check whether a correct answer actually uses both facts.
- The paper shows that a simple classifier (probe) on the model's hidden states can flag such answers from the original question alone.
Critical weaknesses
- The task setting is too simple to support general claims about multi-hop reasoning. Questions have only two hops, answers are picked from four candidates based on their first token, and models cannot reason step by step.
- Some questions may involve well-known people, places, or works that the model already knows from pretraining. For these, the edit contradicts what the model knows and replaces the answer with a made-up name. The model may then reject the edit rather than fail to chain facts. The paper gives no evidence to rule this out,
- Only two small models are evaluated, and key setup details are missing.
- The claim that middle-layer states carry the intermediate entity is weakly supported on natural-language questions. On fictionalized MuSiQue, bridge re-execution peaks at only 17–23% of items, and Fig. 2 reports no confidence intervals. The intermediate cannot be directly decoded from the state.
Questions for authors
Please refer to the weakness.
4 comments
We are very grateful for your insightful feedback (2/2)
Q3. “Only two small models are evaluated, and key setup details are missing.”
A3. Thank you, the revision addresses both points.
Models. We added Qwen3-14B, Qwen3-32B and the closed model gpt-6-luna. The first-hop gap on MuSiQue is 53.0, 22.2 and 16.5 points for Qwen3-4B, 14B and 32B, and every interval excludes zero. gpt-6-luna follows 89.5% of second-hop edits but only 66.7% of first-hop edits (Sec. 4.2, Table A.3).
Setup details. Full setup details are now in Appendix B.1, and the supplementary material holds the code and a step-by-step reproduction guide.
Q4. “The claim that middle-layer states carry the intermediate entity is weakly supported on natural-language questions. On fictionalized MuSiQue, bridge re-execution peaks at only 17–23% of items, and Fig. 2 reports no confidence intervals. The intermediate cannot be directly decoded from the state.”
A4. Thank you for the careful reading. The peak counts only items where the top answer switches to the bridge answer, which is why it is lower in text. The transplant also moves answers it does not switch, and a new matched test shows directly that the bridge steers the answer in text.
Peaks in text. Both peaks are well above the controls. The table gives rates (%) on MuSiQue at the peak layer. Bridge re-execution means the answer switches to the bridge answer, answer transport means it switches to the source's finished answer, and the control copies a state from a random position.
Transplant rates on MuSiQue at the peak layer (% of items)
| Rate (%) | Llama-3.1-8B | Qwen3-4B |
|---|---|---|
| Bridge re-execution | 16.7 | 22.9 |
| 95% CI | 11.6 to 23.4 | 14.6 to 34.0 |
| Answer transport | 1.3 | 7.1 |
| Random-position control | 1.0 | 1.2 |
The peak counts only items whose top answer switches to the bridge answer. In 92% to 100% of the other items, the transplant still makes the bridge answer more likely (details in our answer to Reviewer MPEV, Q3). To test this effect directly, we use a matched pair. Two source prompts are identical except for which bridge the first fact points to. The matching source pushes the answer toward its bridge answer by +2.83 logits (1.90 to 3.75), and 120 of 176 items move in the expected direction (p < 10⁻⁵; Table A.4).
Two further checks support the claim. On Symbolic chains, shuffling the order of facts in the source barely changes the bridge-answer rate (81% before and 76% after, for Llama), while renaming the bridge entity drops it to 24%. So the state carries the entity itself, not its place in the list. Corrupting the six heads our transplants select leaves 13 of 30 Llama and 19 of 30 Qwen items correct, against 30 of 30 and 29 of 30 for matched random heads.
Confidence intervals. Figure 2 now draws 95% intervals, and the table above gives them at the peak.
Decoding. Reading the middle-layer state through the output head gives the intermediate at most 0.5% probability, yet later layers use the same state to produce the bridge answer. Ye et al. (2025) likewise find reusable intermediate states by patching rather than decoding, so we test use with transplants.
Thanks again for your feedback, which made the paper stronger. Please feel free to let us know if anything is still unclear.
References
- Ye et al. (2025). How Do Transformers Learn Implicit Reasoning? NeurIPS.
Thank you for the response. The 95% intervals, added controls and larger models are helpful. However, bridge re-execution still holds for only a few MuSiQue items, and the transplants and probes remain limited to two small models. The four-hop results also cover only Symbolic chains. In addition, the revision adds new contributions, namely edit training and the HoH and τ-bench routing results, which go beyond the original submission. I will keep my score.
Thank you for your follow-up
Thank you very much for reading our response and for your follow-up. We are glad that the intervals, controls and larger models were helpful. We would like to kindly point to three results in our responses that may address your remaining concerns.
-
Bridge re-execution on MuSiQue. The 17–23% counts only items whose top answer switches. In 92% to 100% of the other items, the bridge answer still becomes more likely, and copying eight positions instead of one raises the Llama rate from 11% to 42%. Please see "The remaining items" and the matched-pair table in our answer to Reviewer MPEV, Q3 (5/5).
-
Probes beyond two small models. The probe also runs on Qwen3-14B and Qwen3-32B (AUROC 0.782 and 0.795, above every baseline). Please see the table "Predicting a wrong CoT answer before any reasoning" in our answer to Reviewer MPEV (3/5). Probes on the open models also predict gpt-6-luna's errors, as shown in our answer to Reviewer oi9P, Q3 (2/2).
-
Edit training, HoH and τ-bench. We added these in response to Reviewer MPEV's Q2 on downstream use (2/5 to 4/5). They apply the paper's own edit labels and probe, and the paper's claims remain unchanged.
We sincerely appreciate your time and careful review. Please let us know if anything remains unclear.
We are very grateful for your insightful feedback (1/2)
Dear Reviewer 4Njt,
We are very grateful for your insightful feedback. We are glad that you found the intuition well motivated and the edit test a simple way to check whether a correct answer uses both facts. In light of your suggestions, we have extended the setting, added controls against knowledge conflict, and strengthened the evidence in text. Please see our point-by-point responses below.
Q1. “The task setting is too simple to support general claims about multi-hop reasoning. Questions have only two hops, answers are picked from four candidates based on their first token, and models cannot reason step by step.”
A1. Thank you for raising this. We added each missing setting, and the first-hop weakness holds in all of them.
More than two hops. On four-hop Symbolic chains, the first hop is again the weakest. The table gives the share (%) of correct chains that follow each edit (Figure A.1).
Four-hop Symbolic chains following each edit (% of correct chains)
Free-text answers. We removed the answer list, so models answer in free text. The first-hop gap stays above zero, with every 95% interval above zero (Table 2a). The table gives the gap in points.
Step-by-step reasoning. The same table includes CoT and thinking. Reasoning raises accuracy but does not remove the gap.
First-hop gap in free text (points)
We also fixed written plans for four-hop chains, where the model's answer to each step is fed into the next step. The table gives the share (%) of chains passing the original question and all four edits (Table 2b). Reasoning works only when each written step is a single hop.
Four-hop chains passing every edit (% of chains)
Beyond these settings, we applied the test to real Wikipedia updates. There, models are 87% and 91% accurate, yet 67% and 77% of these correct answers stay the same when the two dates are swapped (Sec. 6.3).
Q2. “Some questions may involve well-known people, places, or works that the model already knows from pretraining. For these, the edit contradicts what the model knows and replaces the answer with a made-up name. The model may then reject the edit rather than fail to chain facts. The paper gives no evidence to rule this out.”
A2. Thank you for this important point. We agree that this needs to be ruled out, and four results show that rejecting edits is not the main cause of the gap (Sec. 4.2).
First, in the fictionalized setting every name on the path is generated, so there is no stored fact to defend, and the gap remains (54.9 points for Qwen and 22.2 for Llama).
Second, second-hop edits contradict the clean world just as much, yet models follow them about 90% of the time (89.8% for Llama and 86.3% for Qwen).
Third, replacing every remaining real-looking name in the context leaves the gap for all three models. The table gives the first-hop gap in points (Table A.2a). The default column is rerun alongside the control, so Llama's value differs slightly from Sec. 4.2 (22.2).
First-hop gap with every name replaced (points)
Fourth, Symbolic chains contain no world knowledge, yet the first hop is again the weakest at three and four hops (Figure A.1).
An independent LLM reader also screens every edit family and keeps only those whose variants it answers correctly, so retained edits are solvable from the context (Sec. 4.1).