- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the reveal policy in masked diffusion language models: after each model evaluation, the decoder must decide which masked position to reveal next. Because later predictions are conditioned on previously revealed tokens, different causal reveal policies can induce different distributions over completed answers even when the underlying model is fixed. The paper defines the set of answer distributions reachable from a partial state under admissible causal reveal policies. It proves that satisfies a recursive characterization and is a convex polytope. It then connects linear optimization over this set to Bellman recursions and occupancy-measure programs, covering both hard per-generation budgets and expected evaluation budgets. A two-position construction shows that current per-position marginals can be insufficient for selecting the optimal first reveal. Empirically, the paper reports four main findings. First, confidence-based reveal ordering outperforms random ordering on GSM8K across four diffusion language models at matched sequence-evaluation budgets. Second, a four-path portfolio using confidence, margin, entropy, and left-to-right policies outperforms four confidence paths on Dream-7B and LLaDA-1.5. Third, exact enumeration on constrained three-slot graphs reveals large policy-dependent answer-law diameters. Finally, MMLU states selected by an option-confidence threshold exhibit concentrated outcomes under randomized future reveal orders, enabling an early-stopping rule that reduces model evaluations while approximately preserving accuracy.
Critical strengths
- The reachable answer-law formulation cleanly separates reveal-policy effects from sampling noise and answer correctness.
- The recursion for covers randomized, history-dependent causal policies rather than only fixed reveal orders.
- The paper clearly distinguishes exact reachable-set characterization from lower bounds obtained using a finite tested policy family.
- The matched-budget confidence-versus-random results are consistent across four trained models and include appropriate controls at block size one.
- The three-slot experiments exhaustively enumerate all deterministic history-dependent policies and independently verify the resulting values.
- The portfolio gain is replicated on a second model and remains positive under parser sensitivity analyses.
- The prospective stopping experiment uses disjoint prompts and reports both accuracy and hardware timing.
Critical weaknesses
- The main polytope, Bellman, and occupancy-measure results largely follow from standard finite-horizon MDP/CMDP machinery after projecting terminal distributions through an answer map. The paper should distinguish the conceptual contribution from the technical novelty more explicitly.
- The portfolio comparison omits four-path margin-only, entropy-only, and left-to-right-only ensembles. Consequently, the evidence shows that the fixed portfolio outperforms four confidence paths, but not that diversity or complementarity is responsible.
- The exact optimization assumes a fully known finite DAG of transition kernels and terminal utilities. Its state space grows exponentially with output length, and no estimation-error or scalable-planning guarantee is given for full-length models.
- Corollary 1 is a useful insufficiency construction, but its interpretation is somewhat broad. The two-position example establishes the value of knowing the response structure beyond current marginals; it does not directly demonstrate an online controller exploiting a nontrivial revealed history over several decisions.
- “Matched compute” should be qualified as matched sequence-evaluation budget. The compared arms can have different physical invocation counts, batch shapes, memory use, and wall-clock behavior.
- The MMLU concentration result concerns one randomized continuation procedure on threshold-reaching prompts, not concentration over all admissible reveal policies. It is also conditional on a model-level inclusion criterion.
- Comparisons with stronger learned, reward-guided, or adaptive reveal-order baselines are limited.
Questions for authors
- Could you report four-path homogeneous ensembles for margin, entropy, and left-to-right, as well as their paired differences from the portfolio? Without these controls, how can the portfolio gain be attributed to policy complementarity rather than one stronger constituent policy?
- Do leave-one-policy-out portfolios retain the gain? Which policy contributes most, and how sensitive is the result to the allocation of paths among policies?
- The appendix refers to a “subset ablation below.” Which experiment does this refer to, and are any intended ablation results missing from the current manuscript?
- Can you report wall-clock time, FLOPs, peak memory, and physical model invocations for the principal portfolio comparison? Please also clarify which claims use “compute” to mean sequence-model evaluations.
- How were the block size, portfolio composition, and stopping threshold selected? Were any GSM8K or MMLU test results inspected during these choices?
- Can the portfolio uncertainty analysis include random seed groups, for example through a crossed or hierarchical bootstrap over prompts and seeds?
- What part of Theorems 1 and 2 goes beyond applying standard finite-horizon MDP/CMDP reachable-distribution and occupancy-measure results followed by projection through the answer map?
- Can the response-information result be extended to at least three positions, where an observed token genuinely changes a later reveal decision? How should the cost of obtaining or estimating the response structure be accounted for?
- What guarantees remain when the transition kernels are estimated from finitely many model evaluations rather than known exactly?
2 comments
Thank you for the detailed rebuttal.
Thank you for the detailed rebuttal. The additional homogeneous-ensemble comparisons, allocation ablations, hardware measurements, and revised uncertainty analysis substantially address my main empirical concerns. I also appreciate the more qualified claims, particularly regarding when policy mixing helps and the scope of the concentration results. The theoretical extensions help clarify the contribution, although I retain some reservations about its technical novelty and the scalability of exact planning. Overall, the response strengthens the paper and provides sufficient clarification of my concerns. I have no further questions and maintain my weak-accept recommendation.
Response to reviewer xUzx
Thank you for your helpful questions and for noting that the reachable-set formulation separates policy effects from sampling noise and correctness. Numbers refer to the revised paper, where changes are in red.
Q1 and W2 (single-policy ensembles). General response G2 gives every single-policy ensemble (Table 13). On Dream-7B the portfolio beats all of them. On LLaDA-1.5 and JustGRPO (Ni et al., 2026), LLaDA-8B-Instruct trained with GRPO, four left-to-right (LTR) paths are best, so §3.2 now chooses the composition on 500 train prompts. It beats the best single-policy ensemble on test by +.0447 and +.0191 (Table 15).
Q2 and Q3 (leave-one-out, allocation, the "subset ablation below"). It is the drop-one ablation on a fixed 400-prompt LLaDA-1.5 subset. Table 11 restores it, with all eight contrasts. Every drop-one variant falls below the portfolio, and four LTR paths beat it, as in G2. On Dream-7B the 70 allocations span .4393–.6680. With blocks of 32, the four best mixes contain no LTR path.
Q4 and W5 (cost). On one A100 40GB (Table 17), four-path variants make 64 calls × 4 sequences per prompt and take 9.3 s and 9.7 s, at 17.1 and 16.8 GiB. One deep confidence path makes 256 × 1 calls in 11.7 s and 11.5 s. All reveal rules read their scores from the same model pass, so FLOPs are about equal. The paper now says "matched sequence-evaluation budget".
Q5 (choices). Block size 32 is common for LLaDA on GSM8K, as in JustGRPO. The portfolio was fixed in the Dream-7B plan. Before the choice we had seen the LLaDA-1.5 subset and 252 Dream-7B test prompts. On all other prompts the gains hold. The 0.65 threshold was frozen before the test on 4,250 new MMLU prompts (Table 30).
Q6 (seed runs). Intervals now resample prompts and seed runs. Chosen mix minus portfolio is +.0235 [+.0116, +.0357] and +.0611 [+.0455, +.0766]. An earlier resampling error made them too narrow. After the fix, Dream-7B portfolio minus four LTR includes zero.
W6 (MMLU scope). The concentration result covers random-order continuations, not every policy, as the paper now says. The inclusion rule (App. C.9) excludes SDAR-8B, now named.
W7 (baselines). Margin, from Kim et al. (2025), is one of our five policies. Against tuned self-consistency the mix wins on LLaDA-1.5 (+.0198) and ties on Dream-7B. It beats greedy one-step lookahead by +.0339 and +.0432 (G4). The learned policy of Hong et al. (2026) has no public checkpoints, so we could not run it.
Q7 and W1 (beyond MDP machinery). Theorem 3 makes the earlier sufficient condition exact. For completed outputs, the policy does not matter exactly when the predictions agree with one joint distribution, which holds exactly when swapping two reveals never changes it. Theorem 4 shows a mix can strictly beat every single-policy ensemble. Jury theorems (List and Goodin, 2001; Chen et al., 2024) take voters as given. Here each voter's answer distribution is chosen through its policy.
Q8 and W4 (three positions). Corollary 1(ii) extends the example to three positions. The first token decides which copy rule the next two follow. Any rule whose second reveal ignores it scores 1/2, and the rule that uses it scores 0.9. The model calls this needs enter Theorem 2 as action costs.
Q9 and W3 (estimated predictions). By Proposition 2, an answer distribution moves by at most the prediction error times the number of reveals, and a policy planned on estimates loses at most twice that (Table 5).
References