acceptodds
Under review as a conference paper at ICLR 2027

The Answer Is Not the State: Trajectory-Level Corrigibility in Diffusion Language Models

Abstract

Diffusion language models commit tokens across a shared response canvas, making unfinished outputs appear globally revisable. We show that the answer is not the full decision state. Remasking it can leave visible rationale that preserves trajectory history and reconstructs a prior decision despite current evidence. Matched answer and path resets isolate the retained rationale state. Transplanting a coherent rationale between paired trajectories changes the rebuilt decision, whereas shuffling the same tokens weakens the effect. On source-disjoint multi-hop questions, path reset makes decisions in Dream, LLaDA, and iLLaDA responsive to evidence identity, including controlled corrupt evidence. Answer reset retains path dependence, and the absent condition rules out a generic regeneration benefit. In the two-model natural development analysis, direct leave-one-out attribution improves repair prioritization, but even the best partial reset remains far below path reset. Final-answer regeneration alone does not reveal whether a masked-dLLM trajectory remains revisable.

Primary area:
generative models
Keywords:
diffusion language models, trajectory-level corrigibility, token commitment, visible rationale state, evidence control, rationale transplantation, AI alignment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4jB7OpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper distinguishes between decision tokens and rationales in diffusion language models, it finds that if previously generated rationales are retained, the model reconstructs the prior decision, whereas otherwise, control shifts to the current evidence. The authors validated this by comparing three types of interventions (no reset, answer reset, and path reset) across three dLLMs (Dream, LLaDA, and iLLaDA).

Critical strengths
  1. The paper systematically distinguishes, for the first time, between "open-ended answers" and "revisable trajectories."
  2. Replication across three different dLLM checkpoints enhances generalizability.
Critical weaknesses
  1. The primary experiments rely on Deep Arithmetic Chains and HotpotQA multi-hop question answering, that are tasks characterized by short answers, multiple-choice formats, or single-token decisions. What about open-ended generation tasks (e.g., long-form reasoning, code generation, multi-turn dialogue)?
  2. The paper repeatedly emphasizes that "path reset" represents a causal endpoint rather than a deployment strategy. However, the best partial reset during testing achieved only 70%, far below the 99% seen with "path reset", indicating a significant discrepancy.
  3. The three models (Dream, LLaDA, iLLaDA) all belong to the class of masked diffusion language models, what about other types of diffusion language models?
  4. While the "top-50" and "mass-80" direct attribution methods outperformed the random control, the "top-25" and "decision-mask" methods did not. The rank correlation between the two attribution scores is merely 0.115, suggesting they select different tokens—a point that lacks comprehensive analysis or explanation.
Questions for authors

see weakness

3 comments
AuthorsOpenReview ↗

Response to concerns on scope and attribution

Thank you for the concrete questions. We respond to each weakness below.

W1: The evaluation uses short or one-token decisions; what about open-ended generation?

We froze a new 100-question source-disjoint HotpotQA evaluation before inference. It provides no answer choices or proposed answer; gold and controlled false targets contain 2–8 tokens. All 3,600 expected outputs are retained, and the minimum per-cell format-pass rate is 98%.

ModelCorrect EM: answer → pathFalse-target EM: answer → pathCorrect-specific effect [95% CI]
Dream65% → 66%44% → 46%+1 [−3, 5]
LLaDA8% → 54%2% → 27%+46 [36, 56]
iLLaDA4% → 67%2% → 46%+63 [53, 73]
Pooled——+36.7 [31.3, 42.3]

This extends the result to reliably parsed free-form short answers and reveals checkpoint dependence: the additional path-reset effect is large in LLaDA/iLLaDA and negligible in Dream. It does not establish long-form reasoning, code generation, or multi-turn dialogue, and we revised the scope accordingly.

W2: The best partial reset reaches 70%, far below the 99% path-reset result.

We agree that the gap is large; it is the mechanism result we call the attribution–repair gap. Direct top-50 and mass-80 exceed matched random selection by +16.8 [9.0, 25.2] and +18.2 [11.0, 26.0] points, so the score improves token prioritization. Yet the best partial reset reaches 70% versus 99% after path reset. We now present this as a negative result on compact repair sufficiency, not as a near-deployable method.

W3: What about other diffusion families?

The evidence covers Dream, LLaDA, and iLLaDA, all fully bidirectional masked dLLM checkpoints. The 64-step and reset-time robustness studies vary decoding within this family; they do not replace cross-family validation. We removed broader architectural implications and do not claim coverage of continuous diffusion or all diffusion language models.

W4: Why do top-50/mass-80 help while top-25/decision-masking do not, and why is rank correlation only 0.115?

The scores measure opposite directions. Decision masking asks how the answer supports an observed rationale token; direct leave-one-out asks how removing a rationale token changes the answer distribution. Their mean Spearman correlation is therefore low (0.115; top-quarter overlap 30%). Direct top-50 and mass-80 establish a matched-random advantage, whereas top-25 and the decision-masking ranking do not. Figure 4 now presents the repair frontier and matched-random effects together; Appendix D.6 retains per-model results and all policies. Redundancy and nonlinear interactions remain possible explanations for the gap, not established conclusions.

The free-form protocol and full cells are in Section 3.5, Table 3, Appendix C.5, and Appendix D.4; attribution revisions are in Section 3.7, Figure 4, and Appendix D.6.

Thank you for the detailed response. The new free-form evaluation partly addresses my W1, and I agree that the attribution–repair gap is better framed as a negative result. However, after reading the full discussion, including Reviewer UzAb’s review and your reply to it, I have come to share the concern about what is specific to diffusion. Now you say that AR models may show the same anchoring, and that path reset is operationally close to regenerating on the same canvas. Without a matched AR comparison or a restart-with-evidence baseline, it is unclear how much of the finding is specific to diffusion models rather than a general property of retained rationale. My own concerns are also only partly resolved. The free-form gain depends strongly on the checkpoint (Dream: +1 [−3, 5]) and is still extractive short-answer HotpotQA (W1), and there is no evidence beyond masked dLLMs (W3).

Taken together, I am lowering my score from 3 to 2. I mention them as what would most change my assessment, not as a condition for this round.

AuthorsOpenReview ↗

Prompt-restart results and relation to AR research

We have completed the prompt-restart comparison you highlighted.

1. Why study dLLMs when AR research reports related phenomena?

AR models generate tokens sequentially; masked dLLMs predict multiple positions on a shared canvas and allow generated positions to be remasked. Whether that editing operation enables revision with new evidence requires direct testing.

Related AR studies examine CoT dependence [1] and self-correction without external feedback [2]. Our answer/path interventions, rationale transplantation and selective repair test which retained state obstructs revision, how its content affects decisions, and whether influential tokens suffice for repair. Related AR findings do not replace these tests. This is the contribution of our work.

2. Prompt-restart comparison

On the same 100 questions and three checkpoints, we placed evidence in the user prompt and generated from scratch, retaining the 64/16-token rationale/answer fields, 32-step greedy decoding and the same parser. All 900 outputs across correct, corrupt and absent evidence are included; every cell has 100% first-turn format pass.

Correct-evidence answer EM:

ModelAnswer resetPath resetPrompt restart
Dream65%66%64%
LLaDA8%54%57%
iLLaDA4%67%70%

Restart-minus-path differences are −2 [−8, 3], +3 [−3, 10] and +3 [−3, 10] percentage points (paired 95% intervals).

3. What does the comparison show?

LLaDA and iLLaDA reach 57% and 70% with prompt restart, versus 8% and 4% with answer reset. Reopening the rationale restores 54% and 67%, close to the restart estimates. Together with the matched answer/path intervention, this shows that reopening the answer can leave the decision tied to earlier rationale, even when the model can answer correctly from the current evidence. The contribution is identifying this revision barrier and testing its transfer and repair properties.

4. Checkpoint and task coverage

Dream shows little separation in this interface. The barrier therefore occurs in some tested checkpoints, rather than universally. The experiments cover extractive short-answer QA and three masked dLLM checkpoints. Long-form generation and other diffusion families remain untested; these boundaries limit generalization while leaving the observed revision barrier in LLaDA and iLLaDA intact.

With this additional comparison and clarification of the contribution, we hope you will reconsider the decision to lower your rating.

[1] Lanham et al. (2023), Measuring Faithfulness in Chain-of-Thought Reasoning. https://arxiv.org/abs/2307.13702

[2] Huang et al. (ICLR 2024), Large Language Models Cannot Self-Correct Reasoning Yet. https://arxiv.org/abs/2310.01798

Reviewer_usz7OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies whether diffusion language models can revise their decisions after receiving new evidence during generation. The authors argue that reopening the answer tokens alone is insufficient because previously committed rationale tokens can preserve the original decision trajectory. They investigate this through answer reset, full-path reset, rationale transplantation, and selective token-level repair. Experiments on Dream, LLaDA, and iLLaDA show that removing committed rationale makes decisions substantially more responsive to current evidence. The paper further demonstrates that coherent transplanted rationale can steer reconstructed decisions, while attribution-guided partial resets remain considerably less effective than resetting the full rationale.

Critical strengths
  1. The paper provides a convincing intervention-based analysis of how committed rationale affects subsequent decisions. Rather than merely examining attention patterns or final prediction confidence, the authors directly intervene on the generated token state. The comparison between answer reset and path reset is particularly useful because the two interventions retain the same prompt, evidence, and reconstruction policy while changing whether the committed rationale remains visible. On the three-model HotpotQA holdout, path reset achieves 96.7% accuracy with correct evidence, compared with 55.6% under answer reset. The controlled corrupt-evidence experiments further show that this improvement reflects responsiveness to evidence identity rather than a generic preference for the correct answer. This makes the main mechanism substantially more convincing.

  2. The rationale transplantation and selective-repair experiments provide complementary evidence rather than just repeating the main result. The transplantation experiments keep recipient evidence fixed and change only the donor rationale. The coherent-versus-shuffled comparison produces a pooled 28.3-point directional accuracy effect, supporting the claim that the retained rationale influences reconstruction. I also appreciate the negative result from selective repair: direct leave-one-out attribution improves token prioritization, but the best tested partial reset achieves only 70% accuracy compared with 99% for full-path reset. This is an informative finding because it shows that identifying influential tokens is not the same as finding a sufficient repair set.

Critical weaknesses
  1. The primary evaluation is based on a relatively constrained decision interface, which limits generalization to realistic language generation. Most of the main experiments use a fixed response canvas with an explicitly separated rationale region and a constrained one-token A/B/C/D decision. This provides good experimental control, but real diffusion language models may generate longer answers without such a clean separation. The free-form multi-token stress test is therefore important. Unfortunately, the reported parsing integrity ranges from 1.1% to 88.6% across experimental cells, which prevents a reliable replication of the main exact-decision results. As a result, the main claim is well supported for constrained decisions, but its extension to unrestricted generation remains uncertain.

  2. The experimental interventions identify persistent decision state more convincingly than they establish a practical correction mechanism. Full-path reset essentially discards the entire previously generated rationale and reconstructs the answer under the current evidence. The identical early- and late-arrival results after full reset also follow directly from the identical post-reset states under deterministic reconstruction, as formalized in Equation 8. This is a useful implementation and state-equivalence check, but not independent evidence that the model can efficiently revise an existing reasoning trajectory. Moreover, the coherent-versus-shuffled transplantation experiment mixes semantic organization with text naturalness: shuffling tokens may create severely unnatural rationale sequences. A more controlled comparison with fluent but semantically different rationale would strengthen the interpretation of this effect.

Questions for authors
  1. How robust is the answer-reset versus path-reset effect when the decision is generated as unconstrained multi-token text? Given the parsing issues in the current free-form stress test, could the authors evaluate a more reliable multi-token output format or a small set of natural question-answering cases with exact answer extraction?

  2. In the rationale transplantation experiment, could the authors add a fluent, length-matched control, such as paraphrased or semantically neutral rationale, rather than relying only on shuffled tokens? This would help separate the influence of coherent decision-related content from the general degradation caused by unnatural token sequences.

3 comments
AuthorsOpenReview ↗

Response to free-form and transplant concerns

Thank you for the accurate summary of the intervention logic. We address each concern directly.

Q1: Does the answer-reset/path-reset difference hold for reliably parsed free-form multi-token answers?

We froze a new 100-question source-disjoint HotpotQA evaluation before inference. It provides no answer choices or proposed answer; gold and controlled false targets contain 2–8 tokens. All 3,600 expected outputs are retained, and the minimum per-cell format-pass rate is 98%, above the requested 95% threshold.

ModelCorrect EM: answer → pathFalse-target EM: answer → pathCorrect-specific effect [95% CI]
Dream65% → 66%44% → 46%+1 [−3, 5]
LLaDA8% → 54%2% → 27%+46 [36, 56]
iLLaDA4% → 67%2% → 46%+63 [53, 73]
Pooled——+36.7 [31.3, 42.3]

The result confirms path-level evidence control in LLaDA and iLLaDA and shows checkpoint dependence: the additional path-reset effect is negligible in Dream under this interface. Controlled corrupt evidence also increases false-target generation, so the intervention transfers control to current evidence rather than generically improving accuracy. We restrict the claim to free-form short answers.

W2: Is deterministic state equality only a sanity check, and is full path reset a practical correction mechanism?

Yes. In the new experiment, path reset and an independent same-canvas fresh reconstruction match in all 900/900 model–item–evidence pairs, for both answer token IDs and rationale text. We now describe this as the expected state-equivalence/implementation check, not independent evidence for efficient revision. Full path reset is a causal endpoint and incurs additional reconstruction; it is not presented as a deployment policy.

Q2: Does the shuffled-token control confound semantics with naturalness?

Yes. The primary donor-identity contrast already compares coherent rationales associated with opposite evidence while holding recipient evidence fixed; it produces a +17.8-point donor-consistent accuracy contrast and a +18.9-point wrong-target-following contrast. The structured-vs-shuffled comparison preserves token multiset and occupied positions and yields +28.3 points in donor-consistent accuracy. However, shuffling also changes fluency, syntax, and local composition. We therefore revised the claim to structured donor arrangement/content, not semantics independently of naturalness. We have not added a strictly length-matched fluent-neutral or paraphrase control and state this limitation explicitly.

The new free-form results appear in Section 3.5, Table 3, Appendix C.5, and Appendix D.4. The state-equivalence audit and revised transplant interpretation appear in Appendix D.4–D.5 and Limitations.

Reviewer_usz7OpenReview ↗

Follow-up

Section 3.5 and Appendix D.4 resolve my main parsing concern: the new short-answer evaluation retains all outputs, including failures, with at least 98% format pass per cell. The answer-to-path EM gains in LLaDA (8 to 54) and iLLaDA (4 to 67) support the retained-state interpretation; Dream (65 to 66) limits its generality. The revised interpretation of Eq. 8 as state equivalence is appropriate. A fluent, length-matched transplant control is still missing, so the shuffled comparison cannot isolate semantics from naturalness. This remains a limitation rather than a reason to change my overall assessment. I maintain 4.

AuthorsOpenReview ↗

Thank You for the Follow-up

Thank you for carefully reviewing the additional experiments! The new evaluation resolves your main parsing concern and further supports the finding that retained reasoning affects answer revision. Reopening the reasoning raises accuracy from 8% to 54% in LLaDA and from 4% to 67% in iLLaDA, while Dream’s results help clarify the scope of this finding.

Your feedback helped us strengthen the experiments and clarify the conclusions. Thank you for your continued support for acceptance after reviewing these changes.

Reviewer_UzAbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

On masked diffusion LMs (Dream-7B, LLaDA-8B, iLLaDA-8B) the question is: if evidence arrives after part of the canvas is committed, can the decision still change? Canvas has an evidence slot E, a free-rationale region R (64 tokens) and a decision D (one A/B/C/D token). Interventions at step 24: answer reset (re-mask D only), path reset (re-mask R and D), and rationale transplant between paired trajectories with a shuffled-token control. On a 150-question source-disjoint HotpotQA holdout converted to 4-way multiple choice, path reset gets 96.7% with correct evidence vs 55.6% (answer reset) and 41.3% (standard decoder); with matched absent evidence path reset is 33.6%; with corrupt evidence wrong-proposal following is 90%. Path reset also makes early and late arrival give identical outputs (300/300). A coherent opposite-donor rationale moves decisions 28.3 points more than a shuffle of its tokens. Leave-one-out attribution beats count-matched random resets (+16.8) on a 50-item set, but the best partial reset gets 70% vs 99%.

Critical strengths
  • Clean methodology: matched pairs, three evidence identities, an estimand (∆corr) that subtracts the generic effect of regenerating more tokens, frozen protocols, failed gates kept in a ledger. Code, trajectories and a checksum-verified reproduction are released.
  • Transplant with token-shuffled control (+28.3 [25.0, 31.8]) and the attribution-repair gap (top-75% reset gets 70%; mass-80 reopens ~31 tokens for 65%) are concrete numbers.
Critical weaknesses
  • Little conceptual payoff and the diffusion-specific part is not shown. Visible earlier tokens conditioning later ones is anchoring, known from autoregressive CoT, and the appendix itself says AR models show it too. No AR comparison, and no comparison with "restart generation with the evidence in the prompt", which path reset more or less is. Path reset is not a usable policy either, as the authors say. So the practical message is "if you want the model to use new evidence, regenerate".
  • Narrow setting, big claims. One-token multiple choice on extractive HotpotQA, author-built corruption (swapping the answer entity), a prompt that tells the model the proposed option is an untrusted belief, 32 steps, three similar models. The conclusion about "alignment evaluations for masked dLLMs" is far beyond what was tested.
Questions for authors
  1. What is specific to diffusion? An AR model with the same rationale in context and evidence appended later should show this. Did you try?
  2. How does path reset differ from re-running generation with the evidence in the prompt?
  3. Why is LLaDA below chance on the arithmetic task and why does it follow the proposal ~70% for any evidence? Does the arrival-sweep conclusion survive a task LLaDA can do?
  4. Do the effects hold for free-form answers with a parser that passes 95%?
2 comments
AuthorsOpenReview ↗

Response to reviewer questions and new free-form results

Thank you for asking us to sharpen the distinction between general rationale anchoring and the causal object studied here.

Q1: What is specific to diffusion? Would an AR model also be anchored by retained rationale?

Yes, an AR model may also be anchored by rationale retained in its context. We did not run a matched AR experiment and no longer frame anchoring or revision failure as diffusion-exclusive. What is specific to our masked-dLLM study is not anchoring itself, but the intervention enabled by a shared canvas: after the same partial trajectory, we can reopen the answer alone or reopen both rationale and answer without changing the prompt or evidence. This separates answer openness from trajectory revisability.

Q2: How does path reset differ from restarting generation with evidence in the prompt?

Path reset is a current-canvas intervention: it remasks the rationale and answer without restarting the prompt, evidence slot, or intervention time. Operationally, it is close to discarding the old rationale and regenerating. In the new experiment, path reset exactly matches an independent same-canvas fresh reconstruction in all 900/900 model–item–evidence pairs, for both answer token IDs and rationale text. We now describe this as the expected state-equivalence/implementation check, not evidence for an efficient correction algorithm. Its role is to show why regenerating the answer alone is insufficient: retained rationale remains decision state. A standard restart with evidence moved into the user prompt changes both layout and initialization; we have not run that distinct baseline and state this explicitly.

Q3: Why is LLaDA below chance on arithmetic, and does the arrival-sweep conclusion survive on a task it can perform?

The arithmetic sweep does not identify why LLaDA is below chance; early commitment, proposal following, and task competence are entangled, so we do not claim a mechanistic cause. Its preregistered cross-model deadline gate failed: LLaDA's early-minus-late gain is −1 point [−4, 2]. We removed the shared-deadline interpretation and retain the negative result. Natural-language QA supports the narrower path-state claim: LLaDA's correct-specific path effect is +51.3 [42.0, 60.7] on the original holdout and +46 [36, 56] in the new free-form study. The latter is not a six-time natural-task arrival sweep, and we do not present it as one.

Q4: Do free-form answers pass the requested 95% parsing threshold?

Yes. We froze a new 100-question source-disjoint HotpotQA evaluation before inference. It provides no answer choices or proposed answer; gold and controlled false targets contain 2–8 tokens. All 3,600 expected outputs are retained, and the minimum per-cell format-pass rate is 98%.

ModelCorrect EM: answer → pathFalse-target EM: answer → pathCorrect-specific effect [95% CI]
Dream65% → 66%44% → 46%+1 [−3, 5]
LLaDA8% → 54%2% → 27%+46 [36, 56]
iLLaDA4% → 67%2% → 46%+63 [53, 73]
Pooled——+36.7 [31.3, 42.3]

The pooled false-target interaction is +23.7 [18.3, 29.7]. Thus path reset transfers control to the evidence currently present in LLaDA and iLLaDA, including controlled corrupt evidence; Dream is already responsive after answer reset in this interface.

Scope. The new interface removes multiple-choice answers and the untrusted-proposal prompt, but it remains extractive short-answer QA. Corrupt evidence is a controlled counterfactual entity/value substitution, not natural misinformation. Existing 64-step and reset-time 22/24/26/28 studies establish schedule robustness within the tested masked-dLLM family. We now restrict claims accordingly and do not extend them to long-form generation, code, dialogue, continuous diffusion, or all diffusion architectures. The new protocol and complete results appear in Section 3.5, Table 3, Appendix C.5, and Appendix D.4.

AuthorsOpenReview ↗

Follow-up on Q2: evidence-in-prompt restart results

In Q2, you asked how path reset compares with restarting with evidence in the prompt. We have now completed that comparison.

1. Prompt-restart results

On the same 100 questions and three checkpoints, we placed evidence in the user prompt and generated from scratch. We retained the 64/16-token rationale/answer fields, 32-step greedy decoding and the same parser. All 900 outputs across correct, corrupt and absent evidence are included; every cell has 100% first-turn format pass.

Correct-evidence answer EM:

ModelAnswer resetPath resetPrompt restart
Dream65%66%64%
LLaDA8%54%57%
iLLaDA4%67%70%

Restart-minus-path differences are −2 [−8, 3], +3 [−3, 10] and +3 [−3, 10] percentage points (paired 95% intervals). The point estimates are close; these intervals do not establish equivalence.

2. Why study dLLM revision separately?

AR models generate subsequent tokens sequentially; masked dLLMs predict multiple positions on a shared canvas and can remask generated positions. These different editing operations make it necessary to test whether reopening a dLLM's answer enables revision with new evidence.

Related AR work, such as Lanham et al. [1], intervenes on CoT to test answers' dependence on reasoning text. Our question is whether the editing capability offered by masked dLLMs is sufficient to update a decision.

LLaDA and iLLaDA can use the evidence when starting over, yet often fail when only the answer is reopened and the old rationale remains. Reopening the rationale brings accuracy close to the restart estimates. Dream shows little separation in this interface. Transplantation further shows that changing retained rationale changes the rebuilt decision; selective repair shows that locating influential tokens does not necessarily identify a sufficient repair set.

The contribution is to distinguish an editable answer from a revisable decision, and to test which retained state accounts for the difference. Related AR phenomena do not substitute for these tests in masked dLLMs.

Does this comparison address your Q2? We would also appreciate knowing which remaining concern weighs most in your assessment, so we can respond before discussion closes.

[1] Lanham et al. (2023), Measuring Faithfulness in Chain-of-Thought Reasoning. https://arxiv.org/abs/2307.13702

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.