- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Diffusion language models typically default to 64 denoising steps during inference, resulting in significant wasted computation; however, no single short schedule is optimal for all tasks. The paper introduces TCDA, which utilizes approximately 128 calibration samples and a lower confidence bound test with family-wise error rate control to select the lowest-cost, quality-certified schedule from a candidate pool, falling back to the full schedule if standards are not met. Experiments demonstrate that summarization allows for early truncation (3× speedup), translation requires state reuse for continuation (2×), and unconditional generation is limited to coarser grids (1.3×), confirming that the optimal inference schedule is task-dependent.
Critical strengths
- By modeling scheduling compression as a non-inferiority test with family-wise error rate control—complemented by a safe fallback mechanism—the violation rate of the retained set is kept below 0.4% (compared to up to 32.6% for naive selection), a capability that is key to the method's practical deployment.
- Three distinct optimal scheduling topologies were identified across three tasks and prospectively validated on independently trained ELF-M models using a pre-registration protocol (achieving a 3.26× measured speedup), thereby ruling out inflation due to multiple testing.
Critical weaknesses
Weakness 1: All conclusions are based on a single model family (ELF-B/ELF-M) and four manually designed candidate topologies; there is a lack of evidence regarding their applicability to other diffusion-based language models or a broader scheduling space, limiting the external validity of the claim regarding "task dependency."
Weakness 2: The candidate pool is entirely pre-specified by humans and tied to specific task structures (e.g., the specific parameters of prefix19 and H32); TCDA merely selects from existing candidates rather than discovering new schedules, making the method's contribution appear limited when compared to existing calibration or multiple-testing tools (such as Westfall–Young or Holm).
Weakness 3: The writing, especially the plot is hard to read. also why put the SHA here (Appendix K)?
Questions for authors
-
How are the specific parameters in the candidate pool (e.g., prefix19, H32, resamp_K24) determined? If the candidate pool were larger or more granular, would TCDA’s family-wise error rate control become overly conservative, thereby negating the speedup benefits?
-
The method was validated solely on the ELF model family; does it hold equally true for other diffusion/flow-matching language models (e.g., those with different noise schedules or decoding heads)? Is there any preliminary evidence?
-
The calibration set (n=128) is fixed and randomly sampled; if the input distribution at deployment differs from the calibration distribution, can the certified schedule maintain its non-inferiority? Has a recalibration trigger mechanism for distribution shift been considered?
-
Guardrail thresholds (such as n-gram repetition and entropy for OpenWebText) were added after degradation was observed; does this constitute a form of post-hoc adjustment? How can it be ensured that the guardrails themselves do not introduce new multiple-testing issues?
2 comments
Dear Reviewer RWoq,
Thank you again for your review. The discussion period closes soon, so we wanted to check whether our response answered your questions and whether anything remains unclear.
In brief, the revised paper addresses each point:
- Other model families (W1, Q2): on LLaDA-8B (masked discrete diffusion, with a different noise process and decoding head), TCDA certifies 8.0× fewer forward passes on WMT14, 4.05× on GSM8K and 1.7× on XSum, with 0/500 false certifications.
- Hand-picked candidates (W2, Q1): with a mechanically generated 127-schedule library, error control does not become overly conservative: the certified speedup rises (XSum 3.37× → 6.75×, WMT 2.0× → 3.8×) with at most 3/500 false certifications.
- Distribution shift (Q3): certified schedules stay within tolerance on CNN/DailyMail, newstest2016 and IWSLT17; we also evaluate recalibration and a shadow monitor as a trigger.
- Post-hoc guardrails (Q4): guardrails are now certified jointly with utility (Proposition 1), which reduces OWT false certifications from 36/500 to 0/500; a prospective test froze every threshold before generation and confirmed every selection on fresh seeds.
- Readability (W3): all figures regenerated with text of at least 9 pt; hashes moved to the supplement.
If any concern remains, we would be glad to address it before the discussion closes. If our response has resolved your concerns, we would appreciate it if you would consider updating your assessment.
Thank you for your practical, deployment-focused review. Your questions directly inspired our new LLaDA experiments.. All updates are in the revised PDF (Appendix A lists all changes).
W1 & Q2 (Validation Beyond ELF on Other Diffusion Families): We validated TCDA across 2 model families, 3 model scales, and 7 datasets:
(On LLaDA XSum, a static back-loaded 16-step schedule at 4× is acceptable on the full pool, so at n = 128 TCDA conservatively certifies Fast-dLLM instead.)
prefix48(Δ = −0.002 nats) and ELF-L certifiesH48(Δ = −0.027), both achieving 1.33× wall-clock speedup even at a strict 5% PPL tolerance. (Comparisons against SharpEuler, Heun, and adaptive early exit are also added in §5.)W2 & Q1 (Hand-Picked Candidates vs. Automated 127-Schedule Pool): Originally,
prefix19andH32came from small ELF-B pilot runs. To test whether a large, fine-grained pool makes error control overly conservative, we generated 127 schedules automatically (prefix at every step, grids every 4 steps, and 33 continuations):prefix8already matches the full 64-step schedule!) and 2.0× → 3.8× on WMT, with ≤ 3/500 false certifications across all settings.Q3 (Distribution Shift & Online Recalibration Trigger):
prefix19: Δ = −0.17 [−0.36, +0.01] ROUGE-L); WMT14 → newstest2016 (H32: +0.09 [−0.03, +0.21] chrF) and → IWSLT17 (+0.04 [−0.12, +0.22] chrF). Recalibrating directly on shifted data (n = 128) certifiesprefix19on CNN/DM (379/500 splits; 491/500 at n = 256; shifting safely toprefix26at ε = 0.5) andH32on newstest2016 (491/500) and IWSLT17 (401/500), with ≤ 1/500 false certs. (We now state the exchangeability assumption explicitly.)Q4 (Post-Hoc Guardrails & Multiple Testing): OWT guardrails were originally set after observing
resamp_K16degenerate. we have tested the following:resamp_K24; XSum 500/500, WMT 486/500 matches, 0/500 false certs across one unified 500-split cohort).W3 (Readability, Plots & SHA Hashes): We regenerated all figures with ≥ 9 pt fonts to make it easier to view.