- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper addresses a failure mode in parallel decoding of diffusion LLMs (dLLMs): a bundle of tokens committed together can be jointly unsafe even when every pairwise dependency test judges it safe (motivated by a parity construction where variables are pairwise-independent but jointly constrained). The authors define Higher-Order Residual Risk (HOR) as the residual of an offline joint-risk proxy R_J(S) after regressing out pairwise risk, confidence, bundle size, and diffusion stage (Eq. 1, 7). R_J(S) itself is a tractable "sequential-reveal vs. parallel" log-score discrepancy averaged over up to 8 permutations (Eq. 5). Experiments are done on Dream-7B and LLaDA across GSM8K/MATH500/HumanEval+/MBPP.
Critical strengths
The paper is unusually explicit that HOR is an operational residual, that the diagnostic prevalence is sampling-conditional, and that latency is hardware-specific.
Critical weaknesses
-
The empirical evidence is not credible as presented. The combination of (a) highly regular, round-number tables across many independent cells, (b) an AI-use statement describing reconstruction of the manuscript from a PDF with figures "prepared from tabulated results," and (c) the absence of variance/CIs, raw logs, or released code for the main decoding experiments makes it impossible to verify that the reported gains (4.1 points, 8.0% NFE, 10.3% latency) come from executed experiments.
-
The core construct (HOR) is not shown to measure what its name claims, and its marginal value is small. By the authors' own framing HOR is a catch-all residual; the hidden-state predictor could be exploiting generic error correlates rather than higher-order dependence, and the residual formulation beats direct R_J prediction by only ~0.03 AUROC. Thus even taking the numbers at face value, the paper does not establish that modeling higher-order structure is responsible for the gains.
Questions for authors
- Will you release the code, raw per-run logs, and per-seed numbers (with variance/CIs) for Tables 1, 3, 7, and 9? Given the AI-reconstruction note, can you confirm every reported number is the output of an executed run, and provide the run artifacts?
- Several tables show strikingly regular values (e.g., Table 9 scaling, Table 6/10 HOR means, Table 7 NFE). Can you provide the underlying distributions / histograms and seed-level dispersion that produced these aggregates?
- Can you show that HOR captures higher-order dependence specifically, e.g., on a semi-synthetic benchmark with controllable k-wise-but-not-pairwise structure, rather than generic bundle-error signal? What does the hidden-state predictor's advantage reduce to when the true joint structure is purely pairwise?
- Table 5 shows residual vs. direct R_J prediction differs by only ~0.03 AUROC. Why is the residual formulation worth the added two-stage complexity?