acceptodds
Under review as a conference paper at ICLR 2027

One Token Flips the Verdict: Hypergraph Propagation and State Readout Dissociate in Selective Belief Revision

Abstract

Selective belief revision requires propagating evidence changes while preserving judgments whose justification remains unchanged. We show that evidence propagation and state readout dissociate in two 8B language models. We introduce Evidence–Intervention–Claim (EIC) to measure this separation by intervening on evidence and resolving each claim’s supporting and opposing coverage before and after the change. Updating intermediate hypergraph sources improves multihop revision over matched StaticSource training by 9.01 points on FLDx2-formal and 34.34 on EntailmentBank across three Qwen seeds. Yet pairing a graph-recoverable partial endpoint with itself reduces Qwen’s joint accuracy from 94.94% to 11.24%, despite unchanged endpoint evidence and a bitwise-identical graph residual. Changing only the preceding state token from partial to full raises second-state partial accuracy from 12.92% to 82.58%. Crossed interventions separate token, companion-evidence, and generated-history effects; a pre-specified prospective source-disjoint replication supports the same pattern. The dissociation also persists, though attenuated, when oracle graphs are replaced at inference by cross-fitted predicted graphs in an additional Qwen control. Training on the missing partial-preservation composition helps, but heavy augmentation causes severe state redistribution, whereas lower-dose augmentation improves preservation with substantially less collateral redistribution. Endpoint-factorized generation substantially restores partial preservation across all three Qwen seeds. Together, these results identify evidence propagation and state readout as separable computational problems and motivate isolating generation histories for endpoint-wise judgments that are semantically independent.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Mechanistic Interpretability, Causal Intervention, Belief Revision, Evidence-Based Reasoning, Autoregressive Generation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_DuCFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies selective belief revision: updating judgments affected by evidence changes while preserving unaffected ones. It introduces the Evidence–Intervention–Claim (EIC) framework and a controlled hypergraph architecture to examine evidence propagation and autoregressive state readout. The experiments show that propagation benefits can coexist with unstable judgments: changing a preceding state token can substantially alter a subsequent prediction despite unchanged target evidence and graph contributions. The paper also explores training augmentation and endpoint-factorized generation to mitigate this failure.

Critical strengths
  • The sensitivity of evidence-based judgments to previously generated state tokens highlights a failure mode that aggregate task accuracy can obscure.
  • Matched source-update controls and interventions on preceding tokens, companion evidence, and earlier generated history provide useful evidence about the computational pathways influencing predictions.
Critical weaknesses
  • The findings are demonstrated in two model families, both approximately 8B and trained under the proposed protocol. How robust are these findings across other model scales and backbones?
  • The introduction moves too quickly to quantitative results before clearly defining the task, establishing its motivation, and explaining the experimental setting. In particular, since endpoint-factorized generation (§6.7) allows the two endpoint states to be assessed separately, what motivates the joint autoregressive formulation? In which practical settings is joint generation preferable, and to what extent does the observed failure generalize beyond this formulation?
Questions for authors

Could the authors apply the single-token intervention from §6.5 to the augmented models in §6.6?

2 comments
AuthorsOpenReview ↗

Response to Reviewer DuCF (1/2)

Thank you for the suggestion. Applying the Section 6.5 intervention to all four Section 6.6 augmentation models reveals that augmentation can change the direction and context of label dependence even as preservation improves. Target evidence, graph residual and the remaining prefix are fixed. On the primary diagnostic (133 claims/97 sources):

ModelDiagonal JointForced partialForced fullFull-minus-partial effect
Full104.64%6.19%48.45%+42.27 pp
Full16.70%9.79%81.96%+72.16 pp
Partial1093.30%100.00%20.10%-79.90 pp
Partial171.65%79.90%75.26%-4.64 pp

Joint scores both freely generated states. Forced-prefix columns score the second state conditionally. Effects use unrounded scores.

The original token effect is +67.01 pp. Both Full controls retain large positive effects. Partial10 reverses the effect: a partial prefix now favors partial. Partial1's diagonal effect is not distinguishable from zero, while the same checkpoint retains +53.61 pp with full companion/history. Its improved diagonal robustness is therefore context-dependent.

The original-task comparison shows the corresponding trade-offs (%). A/P/M denote affected, preserved and multihop-affected Joint:

TrainingAPMAll-partial diagonalCode-0 preservation
Original95.7899.0694.399.7796.89
Full1091.7798.9681.639.3998.05
Full195.9298.9891.337.3897.67
Partial1091.0398.7984.6979.127.20
Partial195.7298.6190.8244.3595.72

All-partial/code-0 evaluations contain 276 endpoint observations/174 sources and 410/257, distinct from the 133-claim diagnostic. Ten-percent and one-percent controls replace the same 321 and 32 rows, respectively, out of 3,209 training pairs.

Partial10 emits partial/partial for 389/410 no-coverage endpoints. Its M loss is not specific to the reversed token effect, since Full10 also lowers M. Partial1 preserves far more code-0 and affected-claim accuracy. The intervention distinguishes improved preservation from elimination of label dependence. The dose comparison identifies a less disruptive augmentation regime. We have incorporated these results in Section 6.6 and Appendix F (Tables 24 and 34).

AuthorsOpenReview ↗

Response to Reviewer DuCF (2/2)

W1 — Additional model scales and backbones. On the primary diagnostic (133 claims/97 sources), reference checkpoints give:

ModelOriginal partialDiagonal first / secondDiagonal JointToken effect
Qwen3-8B94.3382.47 / 14.9511.34+67.01 pp
Llama-3.1-8B91.7585.57 / 13.408.25+63.40 pp
Qwen3-14B-Base91.2484.54 / 6.703.61+69.59 pp
Qwen3-30B-A3B-Base91.2482.99 / 18.5613.40+59.79 pp

Accuracy columns are percentages. Token effects are forced-full minus forced-partial second-state accuracy at a fixed remaining prefix. Diagonal scores are free-running. The 30B configuration is an NF4 MoE with about 3B active parameters. Its FLD M-Joint is 90.82% versus 77.04% for matched StaticSource.

Across three seeds each for Qwen-8B, Llama-8B and Qwen-14B, plus this MoE checkpoint, all ten exhibit partial-preservation failure and positive token effects. Separately trained NoGraph models reproduce the failure (0% partial-diagonal Joint in both 8B families). The pattern extends across these tested scales and persists without graph-head injection.

W2 — Why evaluate joint generation? When evidence changes, some claims gain or lose justification while others should retain their warranted state. EIC evaluates these two requirements together: both before/after labels must be correct, so agreement between wrong labels is not successful preservation.

A literature assistant could report claim states before and after a supporting study is corrected. A technical-document assistant could summarize which requirements remain satisfied after a test record changes. Both motivate several evidence-grounded judgments in one structured report. They are illustrative applications, not deployment evaluations. We have introduced the task and motivation before the quantitative example in Section 1.

Joint generation produces this report in one decoding call. Separately generated judgments can be assembled into the same report, so the choice is an interface trade-off rather than a task requirement. Across three Qwen3-8B seeds:

InterfaceN1 Joint (%)Original A / P / M (%)Pair median latency (s)
Cached Joint15.0995.74 / 98.81 / 92.182.491
Factorized59.7993.82 / 98.13 / 89.464.132
Shared, recomputation8.3795.76 / 98.46 / 92.353.994
Masked, same implementation59.6793.73 / 98.36 / 88.443.985

Accuracy entries are three-seed source-macro means. Latency averages per-checkpoint medians for complete endpoint pairs, including both calls for Factorized. N1 changes which necessary premises are missing while both targets remain partial. These development tests edit existing training-held-out FLD evaluation sources. A/P/M retain the original task.

Cached Joint is fastest here. Factorized and Masked increase preservation with measured original-task costs.

Crucially, Shared/Masked retain training coverage, paired grammar, objective and main-decoder forward-token budget. Replacing only other-endpoint state codes gives +51.30 pp N1, at A/P/M costs of 2.03/0.10/3.91 pp. Preservation can therefore improve substantially within the joint format: the experiments identify generated-label visibility as a controllable source of error.

Code permutation, reversed output order and NoGraph controls extend the diagnosis beyond one state encoding, temporal order or graph branch. The evidence concerns controlled, supplied claims and views. The application examples motivate this evaluation rather than establish prevalence in unrestricted evidence revision.

Reviewer_Uw6MOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates selective belief revision in language models, focusing on whether a model can correctly update claims affected by evidence changes while preserving claims whose justification remains unchanged. The authors introduce the Evidence–Intervention–Claim (EIC) framework, which represents supporting and opposing evidence through registered hypergraph dependencies. A learned hypergraph propagation module computes claim-specific residuals, which are injected into the language model's vocabulary-head readout without modifying the autoregressive KV cache.

Critical strengths

The partial-to-partial diagonal experiment is compelling. The same endpoint is evaluated under different pairings, producing a dramatic change in second-position accuracy.

Critical weaknesses

The most important concern is the interpretation of the partial-diagonal collapse. The paper explicitly states that partial endpoints appear during training, but partial-to-partial transitions do not. Therefore, the dramatic reduction in accuracy occurs under a deliberately unseen output composition.

The authors then observe that the decoder's logit contribution changes while the graph residual remains bitwise identical. This is a valid and useful decomposition. However, the conclusion that evidence propagation and state readout are distinct computational problems is partly guaranteed by the chosen architecture. A fixed residual does not establish that the propagated representation is independently correct or sufficient. In fact, the paper reports that the residual itself favors the full state in the partial-diagonal setting, contributing negatively to the partial-minus-full margin.

Questions for authors

Does the partial-diagonal collapse persist when all state-transition combinations are adequately represented during training? Can the graph residual alone predict the correct endpoint state without the main decoder contribution? Does the phenomenon persist in a standard language-model architecture without head-local graph residual injection?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Uw6M (1/2)

Thank you for raising training coverage. We compare three coverage interventions, then change label visibility at fixed coverage.

Q1 — Complete transition coverage and remaining label dependence. EIC has nine endpoint states and 81 ordered before/after transitions. For Qwen3-8B-Base (True R4), cover81 replaces 1,152 of 3,209 training pairs with logically checked two-chain FLD composites: 64 for each of 9 unchanged-state transitions and 8 for each of 72 changing transitions. The other 2,057 pairs and the cXXXX=before,after grammar are unchanged. Training-token volume rises by 10.79%, so the comparison evaluates the augmented-coverage scheme.

On the original graph-recoverable partial cohort (121 claims/89 sources), the partial diagonal repeats the same partial evidence view at both endpoints:

Evaluationcover81 accuracy (%)
Partial endpoint in original pair87.08
Partial-diagonal first position82.58
Partial-diagonal second position50.00
Partial-diagonal Joint49.44

Free-generation Joint rises from 11.24% to 49.44%. At this checkpoint, forcing partial/full preceding codes gives 53.93%/75.28% conditional second-state accuracy, a +21.35 pp label effect. The training scheme mitigates preservation failure while label dependence remains. Transition presence is not evidence of equal frequency or complete mastery.

Withholding 3→3. Code 3 means partial support with no opposition. loo33 replaces cover81's 64 training pairs whose final claim (hypothesis) has transition 3→3 by other diagonal composites, 8 extra pairs for each of the other eight diagonal transitions. All-claim 3→3 counts fall from 102 to 0, including 38 non-hypothesis claims. Code 3 remains in other transitions. Both conditions retain 3,209 pairs, 6 epochs and 1,206 updates. Whole pairs are replaced, including intermediate labels.

Code-3 diagonal evaluationObservations / sourcescover81 Joint (%)loo33 Joint (%)
Original FLD, all eligible claims191 / 12726.642.76
Synthetic nine-state evaluation, all eligible claims618 / 10330.022.98

Each row compares the two models on the same eligible targets using hierarchical source-macro Joint, not correct counts divided by observations. Both subsets include intermediate claims and differ from the recoverable partial cohort above. Withholding 3→3 sharply reduces its preservation. Code 1 (no support, partial opposition) shows no comparable decline, although its diagonal training frequency also increases.

Adding original-format preservation pairs. Partial1/Partial10 replace 32/321 original training pairs with partial→partial pairs, approximately 1%/10%. Code 0 means no support or opposition:

TrainingAll-partial diagonal Joint (%)Code-0 preservation (%)
Original9.7796.89
Partial144.3595.72
Partial1079.127.20

Partial10 emits partial/partial for 389/410 no-coverage endpoints. Partial1 changes original affected/multihop Joint (A/M) by -0.07/-3.57 pp. Thus augmentation helps, with a dose-dependent trade-off. These scopes contain 276 partial observations/174 sources and 410 code-0 observations/257 sources.

Holding coverage fixed instead. Shared/Masked return to the original training data, matching paired grammar, targets, loss weights, updates and main-decoder forward-token budget. Masked replaces the other temporal endpoint's generated state codes with a fixed non-state token. Both evidence views and the current endpoint's history remain visible. Neither arm adds partial-to-partial training.

N1 uses new edits of training-held-out FLD sources (740 pairs/174 sources). The views lack different premises but both warrant partial. Across three Qwen seeds, Joint rises from 8.37% to 59.67%, with gains in every seed and affected/preserved/multihop Joint changes of -2.03/-0.10/-3.91 pp. At fixed Shared weights, inference-only replacement separately raises N1 from 4.92% to 54.64%, with 36.13% invalid development outputs and a 33.35 pp preserved-claim Joint loss. Trained masking is the remedy comparison.

In the conflict-state subset of the same synthetic evaluation set, incorrect prefixes reduce conditional second-state accuracy by 48–78 pp on diagonal inputs without a comparable free-generation collapse.

Missing transitions matter, while label visibility remains a controllable contributor at fixed coverage. We have added the coverage controls to Section 6.6 and Appendix F, with construction and aggregation in Appendices A–B. Part 2 tests whether the failure requires graph injection.

AuthorsOpenReview ↗

Response to Reviewer Uw6M (2/2)

Q2 — Can the residual alone predict the correct endpoint state? The residual is trained as an additive evidence contribution, rather than a standalone state classifier: native logits are WLM(h+δ)W_{\mathrm{LM}}(h + \delta), with δ=z(R)−z(0)\delta = z^{(R)} - z^{(0)}. Our correct-state classification results come from probes on the propagated representation zR, not from residual-only predictions. Here z0 and zR are the claim representations before and after graph propagation. We first examine the residual in native prediction, then separately test propagated information and the generated-label pathway.

On 121 graph-recoverable claims/89 sources, the mean partial-minus-full logit margins are:

ComponentPartial targets: Qwen / LlamaFull targets: Qwen / Llama
Fixed residual-5.90 / -9.52-17.56 / -18.35
Decoder, original pair13.78 / 16.73-0.05 / 3.15
Decoder, diagonal second position0.73 / 1.618.29 / 10.28

For partial targets, the combined original-pair margin is positive. At the diagonal second position it becomes negative. The residual's mean projection favors full for both target types, but varies with target state. The sign change comes from the decoder contribution while the residual stays fixed. These are average logit contributions, distinct from an endpoint classification score.

To test the score contribution by intervention, we zero the residual at fixed observed prefixes on the full 133-claim/97-source cohort:

ModelTargetWith residual (%)Residual zeroed (%)
QwenPartial15.4637.11
QwenFull98.970.00
LlamaPartial13.4049.48
LlamaFull98.450.00

Removal improves partial predictions but eliminates full-state accuracy in this intervention. The additive contribution must therefore be evaluated together with the decoder and across target states. These are conditional second-state scores rather than free-running Joint.

The additive contribution and the information in a representation are different questions. Three complementary tests assess whether propagation improves revision, makes correct states decodable and affects native predictions:

PropertyResult
Source-update utilityMatched Dynamic/StaticSource, three Qwen seeds: +34.34 pp EB and +9.01 pp FLD multihop Joint.
Decodable correct-state informationProbes fitted only on training endpoints, evaluated on each model's failed recoverable partial endpoints: z0→zR 60.1→78.5% Qwen, 40.1→69.8% Llama.
Native causal influenceQwen source-trajectory exchange at fixed weights and receiving decoder state: +7.94/+8.28 logits toward full/partial.

Probe subsets contain 102 claims/79 sources for Qwen and 109/81 for Llama. The zR accuracy on full endpoints in the gold-graph primary diagnostic is 81.4/74.2%. A separate trajectory-exchange experiment uses 45 training-disjoint sources in original-pair contexts.

To examine a different source of error, the generated-label pathway, a Qwen intervention fixes evidence and residual. Preceding-label K/V replacement recovers correct conditional second states in 28/39 replay-consistent natural failures through blocks 1–32, 3/39 through 33–36 and 22/39 through all blocks, versus 0/39 for self-replacement. These fixed-prefix results identify a label-context pathway.

Q3 — The same failure without graph injection. Residual removal above tests an already trained graph model. To ask whether the architecture is necessary for the failure, we instead train NoGraph LoRA models without any graph branch. On the primary 133-claim/97-source diagnostic:

ModelOriginal-pair partial accuracyPartial-diagonal JointCrossed token effect
Qwen NoGraph89.18%0.00%+90.21 pp
Llama NoGraph82.47%0.00%+83.51 pp

The token effect is forced-full minus forced-partial conditional second-state accuracy with full companion/history. Extension B gives +73.94 pp for Qwen (241 paired-defined claims/188 sources) and +74.87 for Llama (242/189). One Qwen target is undefined and not zero-imputed. Primary contrasts are complete.

Head-local graph injection is therefore unnecessary for both the natural failure and causal label effect. The architecture enables controlled interventions. The graph-free failure, the direction and magnitude of token effects, and the matched visibility gains are empirical findings. Together with the coverage controls, they establish a controllable failure of evidence-grounded joint judgment rather than a conclusion obtained merely by defining two architectural branches.

The revised manuscript reports these diagnostics in Sections 6.2–6.5 and Appendices C–E.

We hope these controls and the clarified contribution support reconsideration of your overall assessment.

Dear Reviewer Uw6M,

Thank you for your time and effort in reviewing our paper and for your thoughtful comments and questions.

With the discussion period nearing its end, we would like to briefly follow up on our responses and revisions. We would be happy to clarify any points or address any remaining concerns you may have.

Best regards,

The Authors

Reviewer_yYqiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces EIC to evaluate evidence-conditioned claim revision. A hypergraph branch supplies fixed, view-local residuals while autoregressive context varies. Experiments with two 8B models show source-update benefits and partial-preservation failures under endpoint recombination. Crossed interventions identify contextual effects; augmentation and endpoint-factorized generation improve preservation with different tradeoffs.

Critical strengths
  1. Fixed-prefix interventions establish a causal effect of the preceding state token on second-state predictions. Crossed controls distinguish this effect from companion evidence and earlier-history effects.
  2. Matched StaticSource training tests source-update utility, while multiple Qwen seeds and prospective source-disjoint replication strengthen the main observations. Reporting unfavorable configurations and repair tradeoffs makes their scope assessable.
Critical weaknesses
  1. At fixed δ\delta, Δℓ=WLMΔhmain.\Delta\ell=W_{\mathrm{LM}}\Delta h_{\mathrm{main}}. Context-induced logit changes therefore pass through the main decoder by construction. Since hmainh_{\mathrm{main}} processes both evidence views and is jointly trained with the graph branch, this does not distinguish successful inference followed by readout failure from context-dependent inference. Source-update gains establish utility; symbolic recoverability establishes information availability; correct first-position, single-view, and original-pair predictions establish capability in those contexts. None establishes successful learned inference before the failed second-state prediction, as the interpretation in §7 requires.

  2. Prior work establishes prefix-dominated answer selection [1] and teacher-forcing-induced prefix shortcuts [2]. EIC’s incorrect-full-prefix rescue is not simple copying, and its fixed-residual and crossed controls add methodological value. These controls identify architectural dependence, but do not resolve whether context changes inference or only answer selection. Restricting “dissociation” to architectural branches makes the claim defensible, but leaves a narrower diagnostic contribution whose added explanatory value over prior prefix-driven failure studies I find insufficiently established.

  3. As Sec 6.7 and Appendix F acknowledge, factorization jointly changes histories, grammar, objective, and computational budgets. Its gains cannot independently validate the proposed inference–readout separation. On identical all-partial inputs, its advantage over Exact-View Copy is 0.64 percentage points, with a source-bootstrap 95% interval of [−2.14,3.54][-2.14,3.54]. The non-identical test only duplicates facts while preserving proof groups and coverage. It establishes robustness to redundant evidence; an advantage over consistency enforcement after deduplication remains untested. Thus, the repair demonstrates improved preservation under the evaluated transformations, but supplies neither the missing mechanistic identification nor an established capability beyond simpler consistency rules.

[1] Liu. The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models. arXiv.

[2] Bachmann and Nagarajan. The Pitfalls of Next-Token Prediction. ICML, 2024.

Questions for authors
  1. What does “successful evidence propagation” mean operationally? If it means correct inference before the failed prediction, which result distinguishes this from context-dependent inference within hmainh_{\mathrm{main}}? If it means beneficial source updates, what further conclusion does the claimed dissociation establish?
  2. Which conclusion about history isolation itself follows from the bundled repair? What capability does the non-identical evaluation establish beyond consistent judgments under duplicate-fact transformations?
5 comments
AuthorsOpenReview ↗

Response to Reviewer yYqi (1/2)

Thank you for the careful comments. We have added tests of state information on failed endpoints, native causal influence and a matched label-visibility intervention to clarify what propagation establishes and what EIC adds beyond prior prefix-failure studies.

Q1 / W1 — What does successful propagation mean? We assess the graph pathway through source-update utility, decodable correct-state information and native causal influence. These are complementary experiments, not successive stages of verified correct inference. Here z0 and zR are claim representations before and after propagation.

PropertyMeasurement and result
Source-update utilityMatched Dynamic/StaticSource training: three-seed Qwen multihop Joint gains of +34.34 pp on EntailmentBank and +9.01 pp on FLD.
Decodable correct-state informationFrozen probes on each model's failed, graph-recoverable partial endpoints: zR 78.5%/69.8% versus z0 60.1%/40.1% for Qwen/Llama.
Native causal influenceQwen source-trajectory exchange with weights and receiving decoder state fixed: native counterfactual logit margins shift +7.94 [6.41, 9.52] toward full and +8.28 [6.63, 9.88] toward partial, relative to same-state donors.

Bracketed intervals are 95% source-bootstrap intervals. Probes are fitted only on training endpoints, with source-grouped cross-validation. Qwen/Llama failed subsets contain 102/109 claims from 79/81 sources. On full endpoints in the primary diagnostic, zR accuracy is 81.4%/74.2%, ruling out an always-partial probe. They read zR, not the injected residual δ=z(R)−z(0)\delta = z^{(R)} - z^{(0)}.

Trajectory exchange uses 45 training-disjoint sources in original-pair contexts, separately from the failed-endpoint probe. Effects remain +3.46/+3.85 logits in the 24 cases whose source changes state.

A complementary Qwen intervention tests the generated-label pathway. With target evidence and residual fixed, preceding-label K/V replacement recovers correct conditional second states in 28/39 replay-consistent failures through blocks 1–32, 3/39 through blocks 33–36, and 22/39 through all blocks. Self-replacement gives 0/39. These fixed-prefix results identify a label-context pathway. Part 2 tests whether controlling it also improves freely generated pair judgments.

W2 — What EIC explains beyond prefix-driven failure. The cited studies establish prefix-dominated answer selection and teacher-forcing shortcuts. Our additional question is when label dependence violates an evidence-defined judgment, and whether it can be reduced while retaining necessary updates.

EIC defines each target from its endpoint evidence and registered justification, distinct from the filtered visible graph. Changing another generated label leaves that target unchanged. Our 2×2×2 crossed design fixes the target evidence, visible graph, residual, weights and output position. It varies the current claim's preceding partial/full code, the companion evidence, and natural histories generated under full/partial companions. The preceding-code effect is +67.01 to +82.99 pp across the four companion/history contexts for the reference Qwen checkpoint on primary FLD (133 claims/97 sources). These are conditional second-state accuracy effects, not free-running Joint gains. They identify specific contextual influences on evidence-defined judgments. The incorrect-full-prefix rescue is not copying: forcing full can make the next output partial.

Shared/Masked tests the intervention suggested by this diagnosis. With training coverage, paired grammar, targets, loss weights and main-decoder forward-token budget matched, non-identical preservation rises from 8.37% to 59.67% across three seeds. N1 views lack different necessary premises but warrant the same partial state, and remain different after deduplication. This isolates label visibility beyond the bundled factorization and duplicate-fact comparisons.

Preservation alone could reward copying, so Part 2 also measures required updates. Partial-conditioned copying nearly matches Masked on N1 but retains far less affected-claim and multihop-update accuracy. Q1 establishes the value of evidence propagation. Q2 tests control over generated labels as a complementary way to improve correct preservation across different evidence, with high but reduced update accuracy. The contribution connects evidence-defined correctness, causal diagnosis of label dependence, and a measured preservation–update trade-off. We have clarified this distinction in Section 2 and expanded the diagnostics in Sections 6.2, 6.4–6.5 and Appendices C–E.

AuthorsOpenReview ↗

Response to Reviewer yYqi (2/2)

Q2 / W3 — Preserving judgments across different evidence while retaining the ability to update. The added capability is selective preservation, not agreement enforced by copying. N1 pairs views missing different necessary premises while both warranted states remain partial: if a complete justification requires a and b, one view lacks a and the other b. Deduplication cannot make these inputs identical.

Matched label visibility. To separate the proposed label pathway from the other changes in factorization, Shared and Masked use identical training pairs, paired grammar, targets, loss weights, updates and main-decoder forward-token budgets, with prefix recomputation in both. When predicting a state for one temporal endpoint (before or after), Masked replaces the other endpoint's already generated state codes with a fixed non-state token. Both evidence views and the current endpoint's own history remain available. Neither arm adds partial-to-partial training. This isolates the visibility change from the grammar/objective changes bundled with factorization.

N1 contains 740 pairs/174 sources. N2:partial contains 330 pairs/103 sources where deletion of an irrelevant fact preserves partial coverage. Both are development edits of original FLD evaluation sources held out from training. Joint requires both labels correct. A/P/M measure affected, preserved and multihop-affected claims on the unchanged original FLD task. Reported intervention differences use unrounded scores.

Three-seed Qwen3-8B means (%)N1N2:partialAPM
Shared8.376.2095.7698.4692.35
Masked59.6727.7193.7398.3688.44

The N1 gain is +51.30 pp, positive in every seed (+60.70/+49.53/+43.67), at A/P/M costs of 2.03/0.10/3.91 pp. Its paired-source 95% interval is [46.18, 56.26] pp, conditional on these checkpoints rather than training randomness. Factorized reaches 59.79% N1. Matched masking attains a similar mean while retaining the paired format and isolating label visibility.

The trained comparison measures the benefit of learning with this visibility rule. To test visibility without changing weights, inference-only replacement at fixed Shared weights raises N1 from 4.92% to 54.64% (+49.71 pp from unrounded scores). It also yields 36.13% invalid development outputs, counted as errors, and a 33.35 pp original-task P loss. This is a causal diagnostic. Trained masking is the remedy evaluated above.

Preservation together with required updates. A gain on preservation alone cannot distinguish evidence-based judgment from copying. We therefore score both preservation and necessary updates. All rules below operate on the same Shared outputs without gold labels or preservation flags. Masked uses no copying rule. These are reference-checkpoint source-macro Joint percentages, distinct from the three-seed table.

PolicyN1N2:partialAM
Shared / exact or canonicalized copy4.923.2094.7891.33
Always copy65.2334.560.000.00
Copy if first prediction is partial65.2334.5666.2047.45
Copy if ancestor structure is equal35.2034.5694.7891.33
Masked65.6321.8792.3785.20

Exact/canonicalized copying leaves N1 at 4.92%. Partial-conditioned copying nearly matches Masked on N1 (65.23% versus 65.63%), but Masked retains 26.17 pp more affected-claim accuracy and 37.75 pp more multihop-update accuracy (differences from the displayed scores).

The ancestor-structure rule is stronger on N2 (34.56% versus 21.87%) and retains Shared's A/M in this evaluation.

Masked improves differing-premise preservation while retaining substantially more update accuracy than aggressive copying. This is a trade-off among the tested methods, not uniform superiority over consistency rules. On identical inputs, exact copying on Original Joint scores 46.07% versus Masked's 40.23%. This uses different trained checkpoints, not the matched Shared/Masked comparison. Our capability claim concerns differing-premise preservation together with required updates.

Shared and Masked both recompute prefixes. Masked replaces the preceding labels described above, whereas Shared leaves them visible. Their mean per-checkpoint median latencies are 3.994/3.985 seconds per complete pair, with no observed increase from label replacement in this benchmark. Both take about 1.6 times cached Joint latency (2.491 s).

We have aligned the Abstract, Section 1 and Section 7 with these contributions, and added the matched visibility and rule comparisons to Section 6.7 and Appendix F.

We hope these new controls address your concerns about soundness and added explanatory value. If so, we would appreciate your reconsideration of the overall rating.

Dear Reviewer yYqi,

Thank you for your time and effort in reviewing our paper and for your thoughtful comments and questions.

With the discussion period nearing its end, we wanted to follow up on our responses and revisions. We would be happy to clarify any points or address any remaining concerns you may have.

Best regards,

The Authors

Reviewer_KU7xOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Evidence–Intervention–Claim, a task that labels supporting and opposing evidence coverage before and after an intervention. A hypergraph branch supplies view-local residuals to a language model's state-token readout. Experiments with two 8B base models distinguish the benefits of updating intermediate graph sources from failures caused by shared autoregressive context. On the graph-recoverable FLD cohort, duplicating a partial endpoint sharply reduces joint accuracy despite an unchanged graph residual. Local token substitutions and crossed interventions identify strong effects of the preceding state token, companion evidence, and generated history. Further studies include prospective source-disjoint replication, predicted graphs, training-data augmentation, and endpoint-factorized generation. The latter substantially improves partial preservation, with measured losses on the original task.

Critical strengths
  1. Convincing interventions on the readout mechanism. The paper holds target evidence and graph residuals fixed while manipulating the preceding token, companion view, and earlier generated history. The main token effect is large in both models, and the prospective extension reports 15 of 16 positive simultaneous lower bounds on additional sources. The analysis correctly distinguishes conditional next-token accuracy from free-running joint correctness: forcing an incorrect first token can improve the second prediction while leaving the pair incorrect.
  2. Useful controls and transparent negative results. Matched StaticSource training isolates intermediate-source updates and yields mean multihop gains of 34.34 points on EntailmentBank and 9.01 on FLD across three Qwen seeds. The paper also provides exact copying, code-permutation, output-order, redundant-fact, and predicted-graph controls. It reports the shallow-propagation deficit, severe state redistribution under heavy augmentation, and the original-task costs of factorization. These results support a diagnostic contribution beyond a single favorable benchmark comparison.
Critical weaknesses
  1. The mechanism is established in a deliberately structured setting with a specific missing composition. The models are trained on registered coverage labels, partial-preservation pairs are absent from the original training set, and only five of the nine representable states occur. Multi-seed evidence is concentrated on Qwen, with small multihop cohorts outside FLD. The predicted-graph control is valuable, but substitutes graphs at inference for one gold-trained Qwen consumer while retaining supplied views, claims, and graph-supervised predictor training. It therefore does not establish how broadly the same failure occurs in ordinary evidence revision with unstructured inputs or naturally conflicting evidence.
  2. The remedy does not isolate history separation and remains incomplete. Endpoint-factorized training changes generation histories, grammar, and objective, with differing token and compute budgets; the authors explicitly acknowledge this. Table 22 reports 90.64% partial-diagonal Joint on the graph-recoverable subset, but 45.27% across all partial endpoints. On identical inputs the latter is close to Exact-View Copy's 44.64%. The improvement on non-identical inputs, from 11.45% to 45.23%, is more informative, but those inputs are redundant-fact perturbations. Broader preservation edits and a matched history-isolation control would clarify the practical value and mechanism of the remedy.
  3. The evidence supports a readout contribution without establishing an internally correct propagated state. Residual invariance follows from the view-local architecture, and the residual's partial-minus-full margins are negative in the principal diagnostic. Thus graph recoverability and a fixed residual do not by themselves show that the neural propagation branch has encoded the correct partial judgment. The intervention results convincingly locate a causal readout pathway; stronger language about successful propagation should remain tied to the matched propagation experiments rather than implying a verified correct intermediate representation for each failed case.
Questions for authors
  1. Can history isolation be varied while holding the output grammar, loss weighting, training tokens, and inference budget as closely matched as possible?
  2. How do factorization and low-dose Partial1 augmentation compare across seeds, runtime, and original-task versus preservation accuracy, including their combination?
  3. Does the effect persist on non-identical preservation interventions beyond duplicating a fact, and on examples with simultaneous support and opposition?
  4. Can you evaluate a consumer trained with predicted graphs and extend the graph-robustness result to additional checkpoints or Llama?
  5. What diagnostics distinguish useful propagation from correct state representation on the failed partial endpoints, given that the fixed graph residual itself favors full coverage?
3 comments
AuthorsOpenReview ↗

Response to Reviewer KU7x (1/3)

Thank you for the constructive review. We added the matched visibility control and the remedy comparison you requested. A/P/M denote source-macro Joint accuracy for affected, preserved and multihop-affected claims. Joint requires both endpoint states to be correct.

Q1 / W2 — Label visibility at matched grammar and budget. Shared and Masked use identical training pairs, paired output grammar, targets, loss weights, updates and main-decoder forward-token budgets. Masked replaces the other endpoint's preceding state codes with a fixed non-state token in the recomputed prefix. Both evidence views and the current endpoint's own history remain visible. Neither arm adds partial-to-partial training.

N1 pairs views missing different necessary premises while both targets remain partial (740 pairs/174 sources). N2 deletes an irrelevant fact. N2:partial contains partial-preserving cases (330 pairs/103 sources). These development tests edit original FLD evaluation cases whose sources were held out from training. N1 remains non-identical after deduplication.

Training / inference conditionN1 Joint (%)
Shared / labels visible4.92
Same Shared checkpoint / labels replaced at inference54.64
Masked / labels replaced during training and inference65.63

Relative to visible-label Shared, inference-only replacement at fixed weights improves N1 by 49.71 pp. The separately trained Masked arm improves it by 60.70 pp. Differences use unrounded scores. The inference-only intervention also yields 36.13% invalid development outputs, counted as errors, and a 33.35 pp original-task P loss. It diagnoses the visibility effect. The trained Masked comparison evaluates the remedy.

Across three paired seeds, Shared/Masked N1 is 8.37/59.67%, with gains of +60.70/+49.53/+43.67 pp. Thus the gain repeats without changing transition coverage or the joint output format.

Q2 — Partial1, factorization, their combination and Masked. All entries below are three-seed Qwen3-8B means (%). Diag. covers all partial endpoints (276 endpoint observations/174 sources), rather than only the graph-recoverable subset. A/P/M use the original FLD task.

MethodN1N2:partialDiag.APM
Original Joint15.098.0911.9795.7498.8192.18
Partial140.5733.0846.5895.6598.6592.01
Factorized59.7925.6845.2793.8298.1389.46
Partial1 + Factorized62.4337.4549.5894.0098.3391.33
Shared8.376.206.7795.7698.4692.35
Masked59.6727.7141.1693.7398.3688.44

Partial1 largely retains original-task accuracy. Factorization improves N1 more. Their combination increases N2 over Factorized by 11.77 pp on average, with a gain in every seed.

Masked reaches a similar N1 mean to Factorized (59.67/59.79), with larger M costs. Its contribution is isolating label visibility within the joint format. Relative to Shared, its A/P/M costs are 2.03/0.10/3.91 pp.

Shared/Masked both use state-by-state prefix recomputation during training and inference. Original Joint uses full-sequence training and cached decoding. Shared is the matched comparator for Masked.

Runtime is measured per complete endpoint pair, including both calls for factorized methods: three exclusively held A800s, BF16, batch size 1, 100 fixed pairs and three checkpoints per method. Loading/warm-up are excluded. Latencies average the per-checkpoint median/p95.

MethodCallsMedian / p95 (s)Peak reserved GiB
Joint12.491 / 4.88217.33
Partial112.496 / 4.85517.33
Factorized24.132 / 8.06617.33
Partial1 + Factorized24.130 / 8.07917.33
Shared13.994 / 10.84017.46
Masked13.985 / 10.75117.46

Masked replaces cross-endpoint state labels, while Shared leaves them visible. With the same prefix-recomputation implementation, their mean per-checkpoint median latencies are 3.985 and 3.994 s per complete pair. Label replacement therefore adds no observed latency over matched Shared in this benchmark. Both take about 1.6 times cached Joint latency (2.491 s). Factorization uses two calls per pair and takes about 1.66 times cached Joint latency.

The revised manuscript reports these controls in Section 6.7 and Appendix F (Tables 4 and 29–32), with implementation details in Appendix B.

AuthorsOpenReview ↗

Response to Reviewer KU7x (2/3)

Q3 / W1 — Preservation under different premises and conflicting evidence. N1 removes different necessary premises rather than duplicating a fact. The three-seed Shared/Masked increase from 8.37% to 59.67% therefore demonstrates preservation on genuinely different evidence, where exact and canonicalized copying do not trigger. N1 is constructed from the original training-held-out FLD evaluation sources.

For simultaneous support and opposition, we trained a synthetic nine-state model with all 81 transitions represented. The conflict-state tests below use synthetic examples, rather than the FLD N1/N2 edits. Each intervention changes one preceding code while fixing target evidence, graph residual and the remaining prefix. Codes 4/5/7/8 represent (partial,partial)/(partial,full)/(full,partial)/(full,full) support/opposition coverage. The table reports conditional second-state accuracy (%) after a correct versus an incorrect preceding code.

Preceding codeDiagonal: correct → incorrectNon-identical: correct → incorrect
4→7, support changes78.65 → 30.0587.72 → 22.81
4→5, opposition changes78.65 → 30.2087.72 → 23.68
5→8, support changes92.72 → 20.27100.00 → 28.82
7→8, opposition changes89.31 → 11.7381.25 → 5.21

The conflict-state intervention cohorts contain 576 diagonal targets and 96 non-identical pairs. All four contrasts per cohort exclude zero under adjusted source-bootstrap intervals. The replay consistency check reports 671/672 target predictions matching generation. This is execution agreement, not prediction accuracy. Incorrect prefixes harm these judgments, whereas an incorrect full prefix rescues partial predictions in the original diagnostic.

The label effect extends to simultaneous support and opposition, including non-identical evidence views. The four preceding-state-code interventions reduce conditional second-state accuracy on non-identical views by 64.04–76.04 percentage points, with adjusted source-bootstrap intervals excluding zero.

A separate free-generation analysis evaluates 432 conflict-partial endpoints from 111 sources. Accuracy at the original, diagonal-first and diagonal-second positions is 42.90%, 44.64% and 44.51%, respectively. These results show that cross-endpoint labels can causally affect predictions without a pronounced free-generation positional collapse.

Q4 / W1 — Predicted-graph training in Qwen and Llama. We trained consumers on predicted graphs in both families, using 3,209 training pairs and 1,206 updates. A fixed two-fold routing policy supplies out-of-fold graphs during training and is retained at evaluation. The predictor remains graph-supervised. The held-out primary evaluation for these predicted-graph-trained consumers contains 471 pairs/300 sources. Its partial/full diagnostic contains 208 claims/130 sources.

ModelOriginal-pair Joint (%)Partial-diagonal Joint (%)Label effect [source-bootstrap interval], pp
Qwen80.000.38+18.85 [13.59, 24.62]
Llama73.720.00+17.31 [11.67, 23.21]

The label effect is forced-full minus forced-partial conditional second-state accuracy, with target evidence, graph residual and the remaining prefix fixed. Joint columns instead evaluate both freely generated states. Replay agreement is 99.62%/99.23% for Qwen/Llama, and all sham checks pass.

At residual scale 1 on the primary evaluation, zeroing the graph residual lowers Qwen A/P/M by 43.75/44.44/30.00 pp and Llama A/P by 34.33/37.59 pp. The corresponding 97.5% intervals exclude zero. Together, residual removal and label intervention show that a graph pathway used in native prediction can coexist with label-dependent preservation failures when consumers are trained and evaluated on learned graphs.

Under predicted-graph training, Qwen/Llama partial-diagonal first-position accuracy is 30.00%/21.92%, dropping to 0.38%/2.44% at the second position. Fixed-prefix interventions still yield label effects of +18.85/+17.31 percentage points, showing that this causal pathway persists in both model families when trained on predicted graphs.

The gold-graph diagnostic additionally covers three seeds each for Qwen-8B, Llama-8B and Qwen-14B, plus one Qwen-30B-A3B checkpoint. All ten show low partial-diagonal Joint and positive token effects on the same 133-claim/97-source cohort. The 30B model is an NF4 MoE with about 3B active parameters.

The model, transition and predicted-graph tests extend the diagnostic in different directions. Matched masking is evaluated on Qwen, and conflicting evidence is synthetic. All tests use supplied claims and evidence views.

These results appear in revised Sections 6.3, 6.5 and 6.7–6.8 and Appendices D.1, E and G.2 (Tables 14, 23 and 37). Table 37 uses the 208-claim/130-source diagnostic.

AuthorsOpenReview ↗

Response to Reviewer KU7x (3/3)

Q5 / W3 — Propagation utility, correct-state information and native causal influence. These three measurements establish what the graph pathway contributes. Residual invariance supplies the control for changing generated context. The probe and trajectory experiments assess information and its use.

Propagation utility. Matched Dynamic/StaticSource training gives three-seed Qwen multihop Joint of 94.95/60.61% on EntailmentBank and 91.84/82.82% on FLD: paired gains of +34.34/+9.01 pp (revised paper Table 6).

Correct-state information on failed endpoints. Frozen linear probes are fitted only on training endpoints with source-grouped cross-validation. We evaluate recoverable partial endpoints where the native model fails at the diagonal second position:

ModelFailed claims / sourcesBefore propagation z0 (%)After propagation zR (%)zR on full endpoints (%)
Qwen102 / 7960.178.581.4
Llama109 / 8140.169.874.2

Propagation increases access to correct-state information on these failures. The full-endpoint column uses the primary diagnostic, not the failed-partial subset, and rules out an always-partial probe. The probe reads the propagated representation zR, distinct from the additive residual δ=z(R)−z(0)\delta = z^{(R)} - z^{(0)}.

Native causal influence. In Qwen, exchanging intermediate-source trajectories while fixing model weights and the receiving decoder state shifts native counterfactual logit margins by +7.94 toward full and +8.28 toward partial relative to same-state donors. This separate experiment uses 45 training-disjoint sources in original-pair contexts, not the failed-diagonal probe cohort. Effects remain +3.46/+3.85 on the 24 cases whose source changes partial/full. Self-swaps reproduce the baseline.

Why a fixed residual can accompany a wrong judgment. On 121 graph-recoverable partial claims/89 sources, Qwen/Llama residual partial-minus-full margins stay at -5.90/-9.52. Main-decoder margins fall from 13.78/16.73 in original pairs to 0.73/1.61 at the diagonal second position, reversing the combined preference. The residual is an additive contribution learned jointly with the decoder, not an independent verdict.

A complementary Qwen intervention fixes target evidence and residual and replaces only preceding-label K/V:

Patched blocksCorrect conditional second states among 39 replay-consistent failures
Baseline / self replacement0/39
1–3228/39
33–363/39
All blocks22/39

The final four blocks alone recover fewer failures than blocks 1–32. Scores are fixed-prefix predictions rather than free-running pair recovery.

Together, the controls distinguish an evidence contribution that improves revision, contains decodable state information and influences native predictions from a label-context pathway that destabilizes judgments. The matched visibility experiment in Part 1 reduces that instability without adding the missing training composition. We have adopted this operational claim in the Abstract, Section 1 and Section 7, and incorporated the representation, trajectory and label-pathway controls in Sections 6.2, 6.4–6.5 and Appendices C–E.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their constructive feedback. EIC identifies a gap between effective evidence propagation and reliable joint judgments.

Matched source-update controls, probes on failed endpoints and native-path interventions establish propagation utility, decodable correct-state information and causal influence, respectively. In fixed-prefix interventions on the same claim's before/after judgments, replacing only the first generated state label can change the second prediction. The target endpoint's evidence, graph residual and correct state are held fixed.

The key new control varies whether previously generated state labels for the other temporal endpoint (before or after) remain visible when predicting the current endpoint, while matching training data, paired output format, objective and main-decoder forward-token budget. On N1, where the views lack different necessary premises but both warranted states remain partial, Joint accuracy rises from 4.92% to 65.63% at the reference checkpoints. Across three Qwen3-8B seeds, the corresponding means are 8.37% and 59.67%, with improvement in every seed and no added partial-to-partial training. Original-task affected/preserved/multihop Joint changes by -2.03/-0.10/-3.91 percentage points across seeds.

The replies test this distinction against graph-free models, complete transition coverage, predicted-graph training and additional model scales. Comparisons with explicit consistency rules evaluate preservation together with required updates, distinguishing selective judgment from enforced agreement. The revised manuscript clarifies this operational claim and task motivation, strengthens the related-work discussion, and incorporates the reported experiments and their scope.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.