- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PRICE, a framework for learning what to recompute in diffusion language models. It formulates repair-scope selection as a state-conditioned action selection problem, using a predicted dependency graph to construct a bounded pool of candidate repair scopes. The method learns reward–cost values of complete repair actions from matched offline continuations and selects a single action during inference without online branching. A selective retention executor further combines quarantine and verification-gated reuse to avoid unnecessary recomputation. The paper also provides theoretical analyses of selective-repair advantages and regret bounds related to candidate coverage and value estimation errors. Experiments across three diffusion language models and four reasoning benchmarks demonstrate improvements in reasoning performance over full-closure repair, while reducing model evaluations by approximately 12%.
Critical strengths
-
Novel Graph-Based Perspective on Recomputation. The paper provides a novel perspective on recomputation in diffusion language models by modeling dependencies among reasoning units through a graph. It formulates repair-scope selection as a state-conditioned action selection problem and learns to balance expected reward and computational cost, enabling more selective recomputation and reuse of intermediate results.
-
Relatively Comprehensive Experimental Evaluation. The experiments cover multiple diffusion language models and reasoning benchmarks. The paper also includes budget-matched comparisons and detailed ablation studies on major components, providing useful evidence for the effectiveness of the proposed framework.
Critical weaknesses
-
Unclear mechanism behind recomputation gains. Although recomputation improves performance, it remains unclear why remasking previously generated tokens leads to more accurate reasoning, especially when the upstream context remains unchanged. The paper lacks controlled ablations distinguishing the effects of changed masking contexts, stochastic regeneration, and corrected dependencies.
-
Presentation and reproducibility issues. Some parts of the paper are difficult to follow due to dense notation and explanations. Figure 1 is overly complicated, with small text and crowded arrows, while several experimental tables are too compact. Furthermore, the provided code appears to be a simplified implementation, lacking key components such as automatic DAG construction and complete benchmark evaluation scripts, making the reported results difficult to reproduce.
Questions for authors
-
What is the main source of improvement from recomputation? It remains unclear why remasking previously generated reasoning units improves accuracy, particularly when the upstream context remains unchanged. Could the authors clarify or experimentally distinguish the effects of changed masking contexts, stochastic regeneration, and corrected upstream dependencies?
-
How effective is PRICE at correcting errors in early reasoning steps? The paper seems to lack an analysis of how repair effectiveness varies with the position or dependency depth of erroneous reasoning units. Could the authors report correction success rates and downstream regeneration/reuse statistics for early-stage errors, particularly in longer reasoning sequences?
3 comments
Thank you for the additional experiments and clarifications, including the controlled analysis of recomputation, the error-position experiments, and the revised method figure and explanations, which help address my earlier concerns. I appreciate the novel perspective presented in this work. However, I still have concerns about the overall clarity of the presentation and the effectiveness of the method on more complex reasoning tasks. I therefore maintain my original rating.
Response to Reviewer NzxW
Thank you for giving the revised figure and error analyses another careful read.
Section 3 and the revised Figure 1 separate the learned choice of what to recompute from the fixed check for reusing a stored result.
Consider an intermediate number that changes from 4 to 5. Its double changes and its sign stays positive. If the predicted graph contains both dependencies, full-closure repair discards both stored results. PRICE can retain a value outside the model’s input and check it against the updated context before restoring it. In Figure 1, the scoring panel shows the scope choice and the verification panel shows the check before reuse.
PRICE used fewer model evaluations on the longest MATH traces in both comparisons. The small accuracy differences favored different methods.
We examined trace length as one aspect of reasoning difficulty. Before comparing the methods, we divided all 5,000 MATH problems into four groups of 1,250 by the effective length of independently generated native traces. Both methods used LLaDA’s native stopping rule and the same three evaluation seeds.
| Native trace length | Full closure accuracy | PRICE accuracy | Full closure mean NFE | PRICE mean NFE |
|---|---|---|---|---|
| Shortest 25% | 47.2% | 49.5% | 40.1 | 35.9 |
| Second quartile | 45.8% | 48.6% | 41.1 | 36.8 |
| Third quartile | 44.0% | 47.5% | 42.6 | 37.7 |
| Longest 25% | 41.0% | 41.6% | 44.6 | 38.8 |
| All 5,000 | 44.5% | 46.8% | 42.1 | 37.3 |
With native stopping, PRICE’s accuracy advantage was 2.3–3.5 percentage points in the three shorter groups and 0.6 points in the longest, where it used 13.0% fewer model evaluations. In the separate 48-NFE MATH subset, PRICE also saved evaluations in the longest group, where full closure was slightly more accurate.
Response to Reviewer NzxW
We thank the reviewer for recognizing graph-based recomputation and our evaluation.
W1 / Q1. “Unclear mechanism behind recomputation gains.”
Stale downstream text can reinforce an earlier error in bidirectional denoising; hiding it enables revision even when upstream values stay unchanged.
We compare four branches from 300 unrepaired states (150 GSM8K/150 MATH; one/question), selected before outcomes with auditable target/ancestor errors. Branches share snapshot, target slot, prior costs and local budget: one replacement call under a 48-NFE total cap, identical KV rules and separately rebuilt visible inputs.
A→C isolates visibility (+21 net corrections; 7.0 points), A→B sampling (+7; 2.3), and C→D ancestor correction (+23; 7.7). D uses a reference intermediate value only offline. These conditional effects do not add up to the total system gain.
The separate native-termination control (Table 3b; Appendix B.15) matches states/scopes/continuations/budgets: accuracy/NFE is 79.8%/34.6 with stale text visible, 80.4%/40.9 with full regeneration and 82.6%/36.0 with selective reuse.
W2 (code). “lacking key components such as automatic DAG construction and complete benchmark evaluation scripts”
We are submitting updated code with this response, including automatic DAG construction, matched interventions, scorer training, repair execution and a runner for selected backbones/tasks comparing PRICE, native decoding, Confidence and Full closure.
A frozen question-only planner produces a validated DAG with fixed slots. The lowest-confidence eligible stored unit seeds six scopes: no repair, target, depth-1/2, closure and global. Duplicate/infeasible actions are removed; the eligible frontier stays fixed within each control interval. From the current trace and budget, the scorer predicts each action's terminal-reward gain and normalized full remaining-cost change versus no repair, including later checks/regeneration. One action executes without online branching.
Selected values are discarded; other affected values stay externally cached and hidden. Inputs are rebuilt with KV reuse disabled. Once current parents resolve, a greedy forward probes the hidden slot. External token/termination matching gates reuse with current parent-version records; rejection discards the probe and generates afresh. All forwards count, and fallback retains prior costs.
Problems are split before collection. Candidates share state, remaining budget and continuation. Slot draws pair problem/slot/position/local update/repeat seed; native fallback shares the repeat seed. Reference answers only score terminal rewards. Accuracy/pass@1 denominators include failures and exhausted budgets (Appendix C).
W2 (presentation). “dense notation and explanations”
We redrew Figure 1 and explain operations before formal notation.
The figure separates four online stages from offline learning. The text first explains discard scope R, hidden affected region Ds(R) and eligible updates U. Core equations/key comparisons stay in the main text; detailed tables/state fields move to appendices.
Q2. “How effective is PRICE at correcting errors in early reasoning steps?”
PRICE locally corrects 36/100 early errors and 7/26 early errors in the longest-trace quartile.
This separate audit uses 300 baseline-error events (150/task; one/question), 100 per early/middle/late third of original unit-completion order. PRICE resumes at the next legal checkpoint under a 48-NFE total cap, retaining prior costs and its ordinary selector without target forcing. Targets are independently assessed after their first completed regeneration.
Rows regroup the same events; the last is an intersection. Original trace length excludes padding/post-termination tokens. Downstream means count distinct units per category, conditional on target regeneration. Of 26 long/early errors, 12 have no completed regeneration and 7 remain wrong.
Shortest root distance is the shortest directed path's edge count from any root to the audited unit (root=0):
Observed correction declines across these distance groups.