acceptodds
Under review as a conference paper at ICLR 2027

More Than Correct Answers: Testing, Tracing, and Predicting Multi-Hop Execution in Transformers

Abstract

A correct answer to a multi-hop question does not establish that the model executed every required hop. The same answer may result from composing the supplied relations, recalling a stored association, or exploiting a prompt shortcut. We study multi-hop execution at three levels. First, we define execution by requiring predictions to remain correct under entity renaming and to follow counterfactual changes to every relation on the queried path. Tests on natural-language and Symbolic tasks reveal substantial failures hidden by high accuracy, including interference from familiar entity names and shortcuts that persist after renaming. Second, we trace execution within a single forward pass. State transplants show that middle-layer states carry an intermediate that later layers use to apply the next relation, while later states carry the completed answer. Component transplants identify recurring attention heads across tasks and benchmarks. Third, we predict execution from one unchanged prompt. Probes over layers identified by the transplant experiments outperform matched random layer ranges and generally outperform confidence- and correctness-based baselines, including under cross-benchmark transfer. Together, these methods provide a unified account of how to test multi-hop execution, trace it across depth, and predict it from a single forward pass.

Primary area:
interpretability and explainable AI
Keywords:
multi-hop reasoning, counterfactual evaluation, interpretability, activation patching, mechanistic interpretability, hidden-state probing, shortcut learning, systematic generalization, language models, transformers

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4NjtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether LLMs that answer two-hop questions correctly actually chain the two facts given in the context. A correct answer could also come from pretraining or by picking an answer passage without using the first fact. To test this, the authors edit one fact at a time and check whether the model follows the edit and gives the edited fact. With Llama-3.1-8B and Qwen3-4B on MuSiQue, 2WikiMultiHopQA, and a synthetic task, many correct answers do not survive these edits. This is more common when entities have real names. The paper then patches hidden states between prompts. It argues that middle layers carry the intermediate entity, while later layers carry the final answer. It also points to a few attention heads that recur across tasks. Last, it shows that probes on these middle layers can predict, from the original prompt alone, whether the model will pass the edits.

Critical strengths
  • The intuition is well-motivated, and the edit-based test is a simple way to check whether a correct answer actually uses both facts.
  • The paper shows that a simple classifier (probe) on the model's hidden states can flag such answers from the original question alone.
Critical weaknesses
  • The task setting is too simple to support general claims about multi-hop reasoning. Questions have only two hops, answers are picked from four candidates based on their first token, and models cannot reason step by step.
  • Some questions may involve well-known people, places, or works that the model already knows from pretraining. For these, the edit contradicts what the model knows and replaces the answer with a made-up name. The model may then reject the edit rather than fail to chain facts. The paper gives no evidence to rule this out,
  • Only two small models are evaluated, and key setup details are missing.
  • The claim that middle-layer states carry the intermediate entity is weakly supported on natural-language questions. On fictionalized MuSiQue, bridge re-execution peaks at only 17–23% of items, and Fig. 2 reports no confidence intervals. The intermediate cannot be directly decoded from the state.
Questions for authors

Please refer to the weakness.

4 comments
AuthorsOpenReview ↗

We are very grateful for your insightful feedback (1/2)

Dear Reviewer 4Njt,

We are very grateful for your insightful feedback. We are glad that you found the intuition well motivated and the edit test a simple way to check whether a correct answer uses both facts. In light of your suggestions, we have extended the setting, added controls against knowledge conflict, and strengthened the evidence in text. Please see our point-by-point responses below.

Q1. “The task setting is too simple to support general claims about multi-hop reasoning. Questions have only two hops, answers are picked from four candidates based on their first token, and models cannot reason step by step.”

A1. Thank you for raising this. We added each missing setting, and the first-hop weakness holds in all of them.

More than two hops. On four-hop Symbolic chains, the first hop is again the weakest. The table gives the share (%) of correct chains that follow each edit (Figure A.1).

Four-hop Symbolic chains following each edit (% of correct chains)

EditQwen3-14BQwen3-32B
Edit to the first hop2859
Edit to the last hop7078

Free-text answers. We removed the answer list, so models answer in free text. The first-hop gap stays above zero, with every 95% interval above zero (Table 2a). The table gives the gap in points.

Step-by-step reasoning. The same table includes CoT and thinking. Reasoning raises accuracy but does not remove the gap.

First-hop gap in free text (points)

ModeQwen3-14BQwen3-32B
One pass, free text19.610.4
CoT, free text8.58.5
Thinking, free text12.77.3

We also fixed written plans for four-hop chains, where the model's answer to each step is fed into the next step. The table gives the share (%) of chains passing the original question and all four edits (Table 2b). Reasoning works only when each written step is a single hop.

Four-hop chains passing every edit (% of chains)

PlanQwen3-14BQwen3-32B
One hop per step97.590.5
Best plan with a multi-hop step5.522.0

Beyond these settings, we applied the test to real Wikipedia updates. There, models are 87% and 91% accurate, yet 67% and 77% of these correct answers stay the same when the two dates are swapped (Sec. 6.3).

Q2. “Some questions may involve well-known people, places, or works that the model already knows from pretraining. For these, the edit contradicts what the model knows and replaces the answer with a made-up name. The model may then reject the edit rather than fail to chain facts. The paper gives no evidence to rule this out.”

A2. Thank you for this important point. We agree that this needs to be ruled out, and four results show that rejecting edits is not the main cause of the gap (Sec. 4.2).

First, in the fictionalized setting every name on the path is generated, so there is no stored fact to defend, and the gap remains (54.9 points for Qwen and 22.2 for Llama).

Second, second-hop edits contradict the clean world just as much, yet models follow them about 90% of the time (89.8% for Llama and 86.3% for Qwen).

Third, replacing every remaining real-looking name in the context leaves the gap for all three models. The table gives the first-hop gap in points (Table A.2a). The default column is rerun alongside the control, so Llama's value differs slightly from Sec. 4.2 (22.2).

First-hop gap with every name replaced (points)

ModelDefaultAll names replaced
gpt-6-luna22.824.4
Llama-3.1-8B20.625.9
Qwen3-4B54.950.8

Fourth, Symbolic chains contain no world knowledge, yet the first hop is again the weakest at three and four hops (Figure A.1).

An independent LLM reader also screens every edit family and keeps only those whose variants it answers correctly, so retained edits are solvable from the context (Sec. 4.1).

AuthorsOpenReview ↗

We are very grateful for your insightful feedback (2/2)

Q3. “Only two small models are evaluated, and key setup details are missing.”

A3. Thank you, the revision addresses both points.

Models. We added Qwen3-14B, Qwen3-32B and the closed model gpt-6-luna. The first-hop gap on MuSiQue is 53.0, 22.2 and 16.5 points for Qwen3-4B, 14B and 32B, and every interval excludes zero. gpt-6-luna follows 89.5% of second-hop edits but only 66.7% of first-hop edits (Sec. 4.2, Table A.3).

Setup details. Full setup details are now in Appendix B.1, and the supplementary material holds the code and a step-by-step reproduction guide.

Q4. “The claim that middle-layer states carry the intermediate entity is weakly supported on natural-language questions. On fictionalized MuSiQue, bridge re-execution peaks at only 17–23% of items, and Fig. 2 reports no confidence intervals. The intermediate cannot be directly decoded from the state.”

A4. Thank you for the careful reading. The peak counts only items where the top answer switches to the bridge answer, which is why it is lower in text. The transplant also moves answers it does not switch, and a new matched test shows directly that the bridge steers the answer in text.

Peaks in text. Both peaks are well above the controls. The table gives rates (%) on MuSiQue at the peak layer. Bridge re-execution means the answer switches to the bridge answer, answer transport means it switches to the source's finished answer, and the control copies a state from a random position.

Transplant rates on MuSiQue at the peak layer (% of items)

Rate (%)Llama-3.1-8BQwen3-4B
Bridge re-execution16.722.9
95% CI11.6 to 23.414.6 to 34.0
Answer transport1.37.1
Random-position control1.01.2

The peak counts only items whose top answer switches to the bridge answer. In 92% to 100% of the other items, the transplant still makes the bridge answer more likely (details in our answer to Reviewer MPEV, Q3). To test this effect directly, we use a matched pair. Two source prompts are identical except for which bridge the first fact points to. The matching source pushes the answer toward its bridge answer by +2.83 logits (1.90 to 3.75), and 120 of 176 items move in the expected direction (p < 10⁻⁵; Table A.4).

Two further checks support the claim. On Symbolic chains, shuffling the order of facts in the source barely changes the bridge-answer rate (81% before and 76% after, for Llama), while renaming the bridge entity drops it to 24%. So the state carries the entity itself, not its place in the list. Corrupting the six heads our transplants select leaves 13 of 30 Llama and 19 of 30 Qwen items correct, against 30 of 30 and 29 of 30 for matched random heads.

Confidence intervals. Figure 2 now draws 95% intervals, and the table above gives them at the peak.

Decoding. Reading the middle-layer state through the output head gives the intermediate at most 0.5% probability, yet later layers use the same state to produce the bridge answer. Ye et al. (2025) likewise find reusable intermediate states by patching rather than decoding, so we test use with transplants.

Thanks again for your feedback, which made the paper stronger. Please feel free to let us know if anything is still unclear.

References

  • Ye et al. (2025). How Do Transformers Learn Implicit Reasoning? NeurIPS.

Thank you for the response. The 95% intervals, added controls and larger models are helpful. However, bridge re-execution still holds for only a few MuSiQue items, and the transplants and probes remain limited to two small models. The four-hop results also cover only Symbolic chains. In addition, the revision adds new contributions, namely edit training and the HoH and τ-bench routing results, which go beyond the original submission. I will keep my score.

AuthorsOpenReview ↗

Thank you for your follow-up

Thank you very much for reading our response and for your follow-up. We are glad that the intervals, controls and larger models were helpful. We would like to kindly point to three results in our responses that may address your remaining concerns.

  1. Bridge re-execution on MuSiQue. The 17–23% counts only items whose top answer switches. In 92% to 100% of the other items, the bridge answer still becomes more likely, and copying eight positions instead of one raises the Llama rate from 11% to 42%. Please see "The remaining items" and the matched-pair table in our answer to Reviewer MPEV, Q3 (5/5).

  2. Probes beyond two small models. The probe also runs on Qwen3-14B and Qwen3-32B (AUROC 0.782 and 0.795, above every baseline). Please see the table "Predicting a wrong CoT answer before any reasoning" in our answer to Reviewer MPEV (3/5). Probes on the open models also predict gpt-6-luna's errors, as shown in our answer to Reviewer oi9P, Q3 (2/2).

  3. Edit training, HoH and τ-bench. We added these in response to Reviewer MPEV's Q2 on downstream use (2/5 to 4/5). They apply the paper's own edit labels and probe, and the paper's claims remain unchanged.

We sincerely appreciate your time and careful review. Please let us know if anything remains unclear.

Reviewer_oi9POpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors argue that a correct answer to a two-hop reasoning question does not show that the model can actually use both relations. The same answer could come from recalling a stored shortcut link or from a cue in the prompt. To study this problem, the authors propose a unified framework that separates multi-hop execution from final-answer correctness and studies it at three levels. First, the paper tests it behaviorally with entity renaming, which checks that the model generalizes to unfamiliar names, and hop-specific counterfactual edits, which check whether each relation controls the answer. Furthermore, this study traces execution inside the model with matched hidden-state transplants. These distinguish an intermediate entity that later layers consume from a completed answer carried to the output. They also identify recurring attention heads for each stage. Finally, this study predicts execution from a single unchanged prompt, using probes on the layers the transplants identified. These probes perform better than random layer choices and most confidence baselines. Together, these turn multi-hop reasoning from an endpoint-accuracy question into a test of whether the model applies the supplied relations.

Critical strengths

First, this study presented a well-controlled measurement study. Each counterfactual replaces a single entity span. All other passages, facts, positions and descriptions stay fixed, so every variant is a complete two-hop world with a unique answer. Raw and fictionalized versions share the same branches, edits and sentence forms. Therefore, only the names on the clean path differ. Parallel passages with fresh entities rule out simply finding an answer-shaped sentence. All these settings make the conclusion statistically convincing.

Furthermore, the authors copy a hidden state from one prompt into another and look at what the model then answers. They design the prompts so that each possible explanation leads to a different answer. If the copied state holds the intermediate entity, the model produces one answer. If it already holds the final answer, the model produces another answer. This test setting makes the result easy to interpret. They run this test only on questions the model answers correctly under every edit. They also compare against copying a state from a random position, which has almost no effect. Similarly, the authors test individual attention heads the same way. Disrupting the heads they identified breaks many answers. Finally, they choose the layer ranges from these experiments on the Symbolic task before training any predictor on the natural-language data. Predictors built on these layers beat predictors built on randomly chosen layers in every comparison. The three parts of the test in the paper support each other. The internal findings are not just descriptions. They lead to a better predictor on separate data, which is good evidence that these layers really matter for the computation.

Critical weaknesses

The scope of this study is narrow. The paper tests two small models (8B and 4B parameters). The test involves two-hop questions only and uses four-option forced choices rather than free-form answers. Larger models may be able to make better use of the relations between hops than the smaller models, but this needs further empirical observations to confirm. In parallel, would the conclusion in this study also hold for reasoning problems with more hops?

As stated by the paper, the most significant results can be observed on the Symbolic task, bridge re-execution peaks at 80% for Llama at layer 15 and 43% for Qwen at layer 27. On fictionalized MuSiQue, Llama bridge re-execution peaks at 16.7% at layer 18 and Qwen peaks at 22.9% at layer 27. What could be the reason that the re-execution peaks for the two LLMs vary greatly from the Symbolic task to MuSiQue?

Questions for authors

I understand it is costly to test larger models. But it would be interesting to check whether the conclusions in this study still hold using at least one commercialized model over the same two-hop reasoning problems. If yes, then the claim of the study can be consolidated.

As I mentioned in the weakness, what could be the reason that the re-execution peaks for the two LLMs vary greatly from the Symbolic task to MuSiQue?

3 comments
AuthorsOpenReview ↗

We truly appreciate your thoughtful and constructive comments (1/2)

Dear Reviewer oi9P,

We truly appreciate your thoughtful and constructive comments. We are glad that you found the study well controlled and that its three parts support each other. In light of your suggestions, we have added larger models, a closed model, free-text answers, reasoning and longer chains. Please see our point-by-point responses below.

Q1. “The scope of this study is narrow. The paper tests two small models (8B and 4B parameters). The test involves two-hop questions only and uses four-option forced choices rather than free-form answers. Larger models may be able to make better use of the relations between hops than the smaller models, but this needs further empirical observations to confirm. In parallel, would the conclusion in this study also hold for reasoning problems with more hops?”

A1. Thank you for the great suggestion. The conclusion holds in every extension we tried. Larger models narrow the first-hop gap, as you expected, but do not close it.

Larger models. We added Qwen3-14B and Qwen3-32B, and also gpt-6-luna (see Q3). The first-hop gap shrinks with size but stays above zero on MuSiQue (Table A.3). On 2Wiki, where the gap was already small at 4B, larger models follow both edits. The first two columns are the share of MuSiQue edits followed, and gaps are in points. Qwen3-4B is rerun here with the larger models, so its row differs slightly from Sec. 4.2 (gap 54.9).

First-hop gap by model size (edits followed in %, gaps in points)

ModelFirst-hop (%)Second-hop (%)Gap, MuSiQue (95% CI)Gap, 2Wiki (95% CI)
Qwen3-4B34.087.053.0 (46.7, 60.0)9.3 (5.0, 13.9)
Qwen3-14B74.997.122.2 (17.1, 27.3)−1.3 (−3.0, 0.2)
Qwen3-32B81.097.516.5 (11.7, 21.3)−0.9 (−2.4, 0.4)

Free-form answers. We removed the answer list, so models answer in free text. CoT and thinking raise accuracy, yet the gap stays above zero in all six cells (Table 2a, 316 families).

Free-text answers with reasoning (tokens per answer, gap in points)

ModeTokens, 14BGap, 14B (95% CI)Tokens, 32BGap, 32B (95% CI)
One pass719.6 (14.2, 25.0)710.4 (5.1, 16.1)
CoT688.5 (3.8, 13.6)788.5 (3.8, 13.6)
Thinking53612.7 (8.2, 17.4)4377.3 (3.2, 11.4)

More hops. Yes, the conclusion also holds here. On four-hop Symbolic chains, the first hop is again the weakest. For 14B and 32B, 28% and 59% of correct chains follow an edit to the first hop, against 70% and 78% for the last hop (Figure A.1).

We also tested written plans. We split each four-hop chain into written steps in a fixed way. For example, 2+2 means two steps of two hops each. The model answers each step, and its answer is fed into the next step. A chain passes only if the original question and all four edited versions are answered correctly. The table gives the share (%) of 200 chains that pass (Table 2b).

Four-hop chains passing every edit, by written plan (% of 200 chains)

Model4 (one pass)3+11+32+21+1+1+1
Qwen3-14B1.05.50.53.097.5
Qwen3-32B12.021.515.022.090.5

Written reasoning works well only when each written step is a single hop. This fits our account: each written step is computed in one forward pass, which has the same first-hop weakness.

AuthorsOpenReview ↗

We truly appreciate your thoughtful and constructive comments (2/2)

Q2. “What could be the reason that the re-execution peaks for the two LLMs vary greatly from the Symbolic task to MuSiQue?”

A2. Thank you for the question. The peak rate counts only items where the top answer switches to the bridge answer. In text, the transplant also moves most answers it does not switch: in 92% to 100% of those items, the bridge answer becomes more likely. Each reason below is now tested (Sec. 5.3, Appendix B.4).

Counting the rate. We copy a middle-layer state from a source prompt into a target prompt. Afterwards, the target gives one of three answers. The bridge answer means the copied state carried the source's bridge entity and later layers used it. The source answer means the state carried the finished answer instead (answer transport). The clean answer means the target kept its original answer. The peak rate counts only items whose top answer becomes the bridge answer.

Transplant outcomes and accuracy (% of items)

Rate (%)Llama-3.1-8BQwen3-4B
Symbolic peak8043
MuSiQue peak16.722.9
MuSiQue items keeping the clean answer8270
Symbolic accuracy9646

Three things explain the difference.

  1. The rate counts only switches. In 92% to 100% of the text items that keep their original answer, the transplant still makes the bridge answer more likely. These items are not counted.
  2. The transplant is conservative. In text, the bridge entity spans several tokens, and we copy only the last one. Copying the last eight nearly quadruples the Llama MuSiQue rate on the same items (11% to 42%).
  3. The peak can only count items the model solves. Qwen solves 46% of Symbolic items, against 96% for Llama, so its Symbolic peak is lower too.

A matched-pair test confirms that the bridge steers the answer in text. Two source prompts differ only in which bridge the first fact points to, and the matching source pushes the answer toward its bridge answer by +2.83 logits (1.90 to 3.75) over 176 items (Table A.4). The revision also gives the layer of Qwen's Symbolic peak (layer 26).

Q3. “I understand it is costly to test larger models. But it would be interesting to check whether the conclusions in this study still hold using at least one commercialized model over the same two-hop reasoning problems.”

A3. Thank you for the suggestion. Yes, gpt-6-luna shows the same first-hop gap on the same fictionalized families. On MuSiQue it follows 89.5% of second-hop edits but only 66.7% of first-hop edits. On 2Wiki it follows both (92.4% and 92.9%), as the open models do. The gap also stays under two context controls (Table A.2a). In the default data, the correct second-hop passage is the original Wikipedia passage (about 560 characters), while the passage that a first-hop edit points to is a single sentence (about 150 characters). We tested whether a model prefers the longer passage for that reason alone. Shortened passage cuts the original passage down to its one answer sentence, so the two passages look alike. All names replaced swaps every remaining real-looking name in the context for a made-up one. The table gives the first-hop gap in points on MuSiQue.

gpt-6-luna under context controls (first-hop gap in points)

ModelDefaultShortened passageAll names replaced
gpt-6-luna22.88.224.4

Even when both passages are a single sentence, gpt-6-luna follows 75.0% of first-hop edits against 83.2% of second-hop edits, so the gap remains (8.2 points). This control was a separate run, where the default gives 66.8% and 89.6%.

Probes built on the open models also predict its errors (Appendix B.5). The table uses probes trained only on Llama and Qwen, with no gpt-6-luna labels, and gives AUROC for predicting whether gpt-6-luna uses both facts.

Predicting gpt-6-luna without its labels (AUROC)

PredictorMuSiQue2Wiki
Llama and Qwen probes, combined0.6770.667
Surface features0.5640.553

A probe that reads open-model features but is trained on gpt-6-luna's own labels reaches AUROC 0.82. Flagging 10%, 20% or 30% of prompts catches about 40%, 60% or 73% of its errors.

Thanks again for your time and the helpful suggestions. Please feel free to let us know if anything is still unclear.

Thanks, authors, for your detailed response. I appreciate your efforts in preparing the additional tests and clarifications. I think they addressed my previous concerns. In this sense, I will keep my positive recommendation for this paper. Good luck!

Reviewer_MPEVOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether LLMs that answer two-hop questions correctly actually execute both hops. It tests execution behaviorally by renaming entities and applying hop-specific counterfactual edits on MuSiQue, 2Wiki and Symbolic chains with Llama-3.1-8B and Qwen3-4B. It finds that high clean accuracy often hides failures to follow edited relations. It then uses matched hidden-state transplants to show that middle-layer states carry the intermediate entity, which later layers use for the second hop. Finally, it trains probes on the identified layers to predict, from the unchanged prompt, whether the model executed both hops.

Critical strengths
  • Clear writing: The paper is clearly written and easy to understand. Also, the problem formulation is easy to follow.
  • Cover implementation details: The paper provides sufficient detail in the appendix, including concrete case studies that illustrate the different shortcut behaviors.
Critical weaknesses
  • Limited Novelty: I think the main claim, which is correct multi-hop answers can come from shortcuts rather than step-by-step reasoning has been studied before. There are many papers from 2024 and earlier on this topic. For example, EUREQA (Li et al., NAACL 2024), which is not cited, and also some work on context–memory knowledge conflicts (Longpre et al., 2021).
  • Unclear downstream application or usage: The probe requires execution labels obtained by running counterfactual prompts. Its AUROC is modest (mostly around 0.6–0.75) and it is only tested on two-hop questions with four answer candidates. It is unclear in what realistic scenario one would deploy such a probe instead of directly running the counterfactual tests or using existing correctness predictors. Also, focusing on multi-hop questions narrows its scope.
  • Some unexplainable results: Bridge re-execution peaks at 80% for Llama on Symbolic items, but only about 17–23% on fictionalized MuSiQue (and 43% for Qwen on Symbolic). The paper does not explain this large difference. It also does not report what happens in the remaining natural-language transplants.
Questions for authors

See the weakness section

6 comments
AuthorsOpenReview ↗

We sincerely appreciate your time and thoughtful feedback (1/5)

Dear Reviewer MPEV,

We sincerely appreciate your time and thoughtful feedback. We are glad that you found the paper clearly written and the case studies in the appendix useful. In light of your suggestions, we have added new experiments and discussion to the revised paper. Please find our point-by-point responses below.

Q1. “I think the main claim, which is correct multi-hop answers can come from shortcuts rather than step-by-step reasoning has been studied before. There are many papers from 2024 and earlier on this topic. For example, EUREQA (Li et al., NAACL 2024), which is not cited, and also some work on context–memory knowledge conflicts (Longpre et al., 2021).”

A1. Thank you for pointing us to these works. Both papers are now cited (Sec. 1 and 2). They show that shortcuts exist, and our paper builds on that to show which hop the model skips, where the bridge is used inside the model, and how to catch the failure at deployment. EUREQA rewrites reasoning chains with misleading cues, and Longpre et al. set a supplied fact against memory. Both show that a model can answer without reasoning, but neither shows which fact in the context the model skips.

Our test keeps the context fixed and edits one fact at a time, so each hop is tested on its own for the same question. This shows a specific and consistent failure. After an edit to the second fact, models switch to the new correct answer. After an edit to the first fact, they often keep the old answer. The table gives the share (%) of edited questions where the model gives the new correct answer.

Edits followed on fictionalized MuSiQue (%, gap in points)

ModelFirst-hop edit (%)Second-hop edit (%)Gap (points)
Llama-3.1-8B67.689.822.2
Qwen3-4B31.486.354.9
gpt-6-luna66.789.522.8
Qwen3-14B74.997.122.2
Qwen3-32B81.097.516.5

All three paired sign tests in the first three rows have p < 10⁻⁹ (315 fictionalized MuSiQue families). The 14B and 32B rows come from the size comparison in Table A.3.

Our edits also differ from knowledge conflict on real documents. On HoH, the date swap changes only the two dates, with both values already in the context, so it contradicts nothing the model has stored. Yet 67% and 77% of correct answers ignore it, because the models follow document position. Only the value edit, which replaces the value in the newer document, could touch memory, and it rarely does: 94% to 99% of value-edit errors take the outdated document's value, not a remembered one, so document position drives these errors too.

Two further parts are new. First, our transplant shows that later layers take the bridge entity from middle layers and apply a fact from the context to it. Prior causal studies trace facts stored in the weights, not facts given in the context. Second, the edits give a training signal for deployment, which we call edit training. Their labels are known by construction and come from inputs where the shortcut fails, so a probe learns the failure before any user meets it (Sec. 6.3, and our answer to Q2).

References

  • Li et al. (2024). Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination? NAACL.
  • Longpre et al. (2021). Entity-Based Knowledge Conflicts in Question Answering. EMNLP.
AuthorsOpenReview ↗

We sincerely appreciate your time and thoughtful feedback (2/5)

Q2. “The probe requires execution labels obtained by running counterfactual prompts. Its AUROC is modest (mostly around 0.6–0.75) and it is only tested on two-hop questions with four answer candidates. It is unclear in what realistic scenario one would deploy such a probe instead of directly running the counterfactual tests or using existing correctness predictors. Also, focusing on multi-hop questions narrows its scope.”

A2. Thank you for pressing on this point. It led to the largest addition to the paper (Sec. 6.3), and it sharpened the motivation for edit training. A deployed model's answers can be checked in three usual ways, and each falls short once inputs change. Confidence misses confident errors. A probe trained on correct and wrong answers learns the shortcut, so it fails on new layouts. A hand-written checker of the policy is exact only while nothing changes: once an API renames its fields or customers write requests in their own words, it cannot read the input and falls to chance (Appendix B.8, Table A.9). Edit training avoids all three. The edits are needed only once, to train the probe. On edited copies the shortcut gives the wrong answer, so the probe learns the failure itself rather than cues that happen to go with correct answers. At test time it reads the one pass the model already runs, with no extra call and no code to update. As a result, it is strongest exactly where the alternatives fail, and it is the best one-pass router on a real deployment problem. We go through each part below.

Labels from edited prompts. Yes, these are needed, but only for training. A team builds the edits offline from its own questions. At test time, the probe only reads the model's normal forward pass. It needs no edited prompt, no gold answer and no gold bridge.

Modest AUROC. The 0.6 to 0.75 range is the AUROC over all answers. The probe is built for the cases where confidence fails, and in all three of them it is much stronger. Here confidence is the model's probability for its top answer, and all scores are tested on question families held out from training.

  1. Confident errors (MuSiQue). We keep the half of answers with the highest confidence. Each score then tries to separate the wrong answers from the right ones (AUROC).
  2. Reliable correct answers (MuSiQue). Among correct answers, each score keeps the quarter it ranks highest. We report the share of these that stay correct under both edits (%). Without ranking, 27.0% (Qwen) and 58.4% (Llama) of correct answers do.
  3. Outdated documents (HoH). Each question comes with an old and a new document. Each score tries to separate the wrong answers from the right ones, to decide which ones gpt-6-luna should check (AUROC). Our answer on a realistic scenario below gives the setup.

Probe and confidence where confidence fails (AUROC; reliable answers in %)

CaseProbe (ours), QwenProbe (ours), LlamaConfidence, QwenConfidence, Llama
1. Confident errors (AUROC)0.950.820.520.64
2. Reliable correct answers (%)57.987.727.163.8
3. Outdated documents (AUROC)0.8120.9020.6310.630
AuthorsOpenReview ↗

We sincerely appreciate your time and thoughtful feedback (3/5)

Beyond two-hop questions with four candidates. Because it learns the failure rather than one layout, the probe now carries over well beyond the setting it was trained on. We tested five kinds of shift. First, before the model writes any reasoning, it predicts whether the CoT answer will be wrong, and it beats every score that reads the reasoning (AUROC, Table A.6).

Predicting a wrong CoT answer before any reasoning (AUROC)

ScoreQwen3-14BQwen3-32B
Probe on one pass (ours)0.7820.795
One-pass confidence0.6920.674
Reasoning length0.6710.696
Reasoning log-probability0.6780.674
One-pass and CoT disagree0.5400.707

Second, probes trained on two documents are tested on three-document inputs from new questions. With one pass, the edit-trained probe beats both one-pass baselines and matches the two-pass rerun for Llama (AUROC).

Transfer from two to three documents (AUROC)

RouterQwen3-4BLlama-3.1-8B
Confidence0.640.66
Accuracy-trained probe0.700.64
Edit-trained probe (ours)0.730.84
Rerun, documents swapped (2×)0.780.83

Third, a probe trained only on MuSiQue transfers to 2Wiki. Among correct 2Wiki answers, the quarter it rates highest passes both edits 84.7% (Qwen) and 90.4% (Llama) of the time, against 60.3% and 79.9% for confidence (Table A.5). Fourth, probes trained only on Llama and Qwen predict the errors of the closed gpt-6-luna with no gpt-6-luna labels, at AUROC 0.677 on MuSiQue and 0.667 on 2Wiki, against 0.564 and 0.553 for surface features (Appendix B.5).

Fifth, a new agent setting, where a hand-written checker breaks once inputs drift. We built 901 customer-support decisions from τ-bench retail: the model reads the store policy, the customer's request and the tool results, then picks the next action, such as refund, refuse or exchange. We trained the probe on copies with one tool field edited (order status, payment method, gift-card balance or stock). We then tested it under four changes it never saw: renamed fields, a policy update, an injected instruction and an extra unrelated tool result. It flags wrong decisions better than every baseline, including gpt-6-luna as a judge, which needs one extra call per answer (AUROC, same 400 test prompts; Appendix B.8).

Flagging wrong support-agent decisions under unseen changes (AUROC)

ScoreQwen3-4BLlama-3.1-8B
Edit-trained probe (ours)0.8550.904
Accuracy-trained probe0.7740.848
gpt-6-luna as judge (1 extra call)0.7290.717
Confidence0.6000.660

The gain repeats in τ-bench's airline domain, with its own policy, tools and data: trained on airline edits, the probe leads the accuracy-trained probe under unseen changes by 0.025 (Qwen) and 0.030 (Llama), with both intervals above zero (Appendix B.8).

The edit test itself now also covers free text, CoT, thinking and four-hop chains (see our answer to Reviewer oi9P, Q1).

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers

General response to all reviewers

Dear Reviewers, AC, and SAC,

We sincerely thank all reviewers for their time and careful reviews. Reviewers found the study “well-controlled” with parts that “support each other” (oi9P), the edit test “a simple way to check whether a correct answer actually uses both facts” (4Njt), and the paper “clearly written and easy to understand” (MPEV). The main concerns were novelty against shortcut and knowledge-conflict work, a narrow scope, whether models reject edits that contradict what they know, the lower transplant peaks in text, and the downstream use of the probe. We addressed each one with new experiments. All changes are marked in blue in the revised PDF, and Appendix B is new.

The revised paper shows three things:

  1. Test. We edit one fact in the context at a time and check whether the answer changes with it. Llama-3.1-8B, Qwen3-4B and gpt-6-luna follow about 90% of edits to the second hop but only 31% to 68% of edits to the first hop. The gap shrinks with size but stays at 32B. Written reasoning has the same weakness, because the model writes each step in one forward pass.
  2. Trace. Inside one pass, middle layers hold the bridge entity (the middle entity of the chain), and later layers use it to look up the next fact.
  3. Predict. A probe on those layers predicts, from the normal forward pass, whether the model really used both facts. We train it on copies of real questions with one fact edited, which mimic the changes a deployed model will see. In each copy, the correct answer is known by construction and the shortcut gives the wrong one, so the probe cannot rely on the shortcut and learns whether the model used each fact. We call this edit training. It wins in two applications: on real Wikipedia updates it is the best one-pass router, and on customer-support agent decisions it beats a gpt-6-luna judge. In both, it keeps working under changes it never saw, where accuracy training falls behind and hand-written rules break.

New experiments:

  1. Qwen3-14B, Qwen3-32B and gpt-6-luna (Sec. 4.2, Table A.3).
  2. Free-text answers, CoT and thinking, four-hop chains and fixed written plans (Table 2, Figure A.1).
  3. Context controls against knowledge conflict, and an LLM screen of every edit (Sec. 4, Table A.2).
  4. 95% intervals for the transplants and a matched-pair test in text (Figure 2, Table A.4).
  5. Edit training on real outdated documents from HoH (Sec. 6.3, Table 3, Figures A.3 and A.4), and on support-agent decisions in τ-bench retail and airline (Sec. 6.3, Appendix B.8).
  6. Full setup details, code and a reproduction guide (Appendix B.1, supplementary material).

Headline result (Table 3). Each router sends its riskiest answers to gpt-6-luna for review. Tokens is the output cost relative to one pass. Accuracy (%) is the final accuracy when half of the answers are reviewed. Reviews (%) is the share of answers that must be reviewed to reach 95% accuracy. The rerun router asks the model again with the documents in the other order, and the last router thinks on every query and reviews answers where thinking and one pass disagree.

Routing on outdated documents (accuracy and reviews in %, tokens relative to one pass)

RouterTokensAccuracy, QwenAccuracy, LlamaReviews, QwenReviews, Llama
Confidence1×87.987.79475
Accuracy-trained probe1×87.684.56985
Edit-trained probe (ours)1×94.295.65444
Rerun, documents swapped2×92.392.65963
Think on all, then review71× / 24×97.991.81665

Among routers that cost one pass, the edit-trained probe is the best, and it beats the probe trained on accuracy by 6.6 and 11.1 points (bootstrap SE 1.3 and 1.1). It also beats thinking on every query for Llama (95.6% against 91.8%), which costs 24 times the output tokens.

Thank you again for your careful consideration of our work. Please let us know if you have any questions.

Best regards,

Authors of Submission 33885

AuthorsOpenReview ↗

A kind reminder before the discussion closes

Dear Reviewers,

Thank you again for your time and careful reviews. The discussion closes on Nov 18 (AoE), so we would like to kindly follow up.

We thank Reviewer 4Njt for the follow-up, and we have replied to the remaining points. We have not yet heard from Reviewers MPEV and oi9P, and we would be grateful to hear whether our responses address your concerns.

In brief, the revision adds:

  • Scope. Qwen3-14B, Qwen3-32B and gpt-6-luna, with free text, CoT, thinking and four-hop chains. The first-hop gap shrinks with size but stays.
  • Controls. Four results rule out edit rejection, and the text transplants now have 95% intervals and a matched-pair test (+2.83 logits).
  • Use. Edit training is not a new method. The original submission already trains the probe on labels from the edits and reads only the unchanged prompt (Sec. 6). In response to Reviewer MPEV's question on downstream use, we applied the same recipe to real outdated documents, where it is the best one-pass router, and to support-agent decisions, where it beats a gpt-6-luna judge.

Please let us know if anything remains unclear, and we will be glad to clarify.

Best regards,

Authors of Submission 33885

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.