acceptodds
Under review as a conference paper at ICLR 2027

TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

Abstract

Diffusion language models predict many masked positions at each denoising step, but their efficiency and output quality depend on which predictions enter the evolving context. Confidence is informative but incomplete for this decision: at matched confidence, proposals with different identity and support histories exhibit different one-step revision rates. We propose Trajectory-Aware Commit Gating (TACG), a training-free decoder that separates the current token proposal from the decision to commit it. TACG scores temporal support for the current proposal by contrasting its logits with an exponential moving average of earlier logits, requires the proposal identity to persist through a History Gate, and promotes a capped number of additional positions per step. The gate decides which positions to commit and when; every written token remains the contemporaneous base proposal. On complete four-task LLaDA evaluations under the stated single-seed protocol, measured task-specific operating points reach 51.02% mean accuracy with 67.82 mean model evaluations, compared with 48.00% and 76.59 for Native Confidence. Under a matched per-event commit-count replay, history-guided selection exceeds confidence ranking by 8.3 accuracy points at identical per-sample NFE. History-reference interventions alter commitment timing while preserving the base-proposal write rule. Experiments on Dream and LLaDA2-Mini characterize quality–cost behavior across backbones and generation lengths.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Masked Diffusion Language Models, Efficient Decoding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mzX1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper points out that using only confidence to do decoding for masked diffusion models is not enough. It is helpful to incorporate historical information along the decoding trajectory to improve decoding quality and speed. They present TACG, a decoding method based on this insight. TACG surpasses several baselines on benchmarks for various commonly-used masked diffusion models.

Critical strengths
  1. The experiments are comprehensive. Empirical results show improvements over selected baselines.
  2. The method combines signed temporal logit contrast and explicit token-identity persistence. This is well-motivated.
Critical weaknesses
  1. Similar methods have been proposed last year and the paper does not seem to compare with them. See question 1.
Questions for authors
  1. There might be missing citations. For example, the icml 2026 paper https://arxiv.org/abs/2512.02044 also proposes a trajectory-aware decoding method. It may be helpful to include a comparison table to compare TACG with relevant methods.
  2. I am interested to know the authors' opinion on using TACG in RL rollouts. And while being a training-free method, I wonder if specialized training can further improve TACG's performance.
3 comments

[W1 & Q1] Missing citation and comparison table with CCD (ICML 2026, arXiv:2512.02044) and related methods.

Thank you for bringing this relevant ICML 2026 work (Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models, proposing Coherent Contextual Decoding, CCD / CCD-DS, arXiv:2512.02044) to our attention. In the revised manuscript, we have cited and discussed CCD / CCD-DS in Section 3 (Related Work) and added the structured comparison table below in Appendix B (Table 10; we will also move the table into Section 3 in the final version as space permits), situating it alongside concurrent commitment methods (VSB, TraceLock) and the trajectory-based baselines evaluated under our unified protocol in Table 1 (where TACG achieves 49.62%–49.90% / 67.87–67.96 NFE shared and 51.02% / 67.82 NFE default on LLaDA-8B, compared with DSB at 49.35% / 75.15 NFE, KLASS at 49.03% / 128.45 NFE, AHD at 47.81% / 76.65 NFE, and Native Confidence at 48.00% / 76.59 NFE).

MethodTraining-FreeGranularitySignal Beyond Current ConfidenceCommitment ControlWrite Rule
Native ConfidenceYesTokenNone (instantaneous confidence ctc_t)Static threshold τ\tauNative argmax
KLASSYesTokenConsecutive-step posterior KL divergenceDual confidence + KL thresholdsNative argmax
CCD / CCD-DS (ICML 2026)YesTokenHistorical context-marginalized distribution + step-level stability/entropyCoherent score + dynamic step budget (CCD-DS)Trajectory-rectified distribution
VSBYesBlockNF vs. FA context-conditioning divergence (block self-containedness)Dynamic block boundary sizingNative argmax
TraceLockNoTokenMulti-step trace states scored by a learned controllerParameterized commitment policyNative argmax (learned commit gate)
TACG (Ours)YesTokenDecoupled: per-position top-1 streak (HG) + positive EMA support gain (TILG)Per-position persistence gate + TILG-ranked capped promotion (kmax⁡k_{\max})Strict native argmax

Mechanistically, TACG differs from CCD / CCD-DS along two key axes. First, CCD aggregates historical contexts into a trajectory-rectified marginal distribution—coupling historical evidence with token selection—whereas TACG strictly decouples when to commit from what token to write, writing only the backbone's unmodified current-step argmax token (as Section 5.3, Table 2 shows, altering written tokens via historical logits hurts accuracy). Second, while CCD-DS leverages cross-step top-1 consistency and marginal entropy to adapt the step-level unmasking budget, TACG enforces a per-position discrete persistence hard gate (streak≥m\text{streak} \ge m, HG) before ranking eligible candidates by candidate-conditioned positive EMA support growth (TILG) under a capped promotion budget (kmax⁡k_{\max}). Within TACG's own component ablation on LLaDA-8B, removing this per-position HG gate while keeping continuous TILG promotions lowers 4-task accuracy from 49.62% to 48.70% (-0.92 pp), confirming that continuous temporal scores alone cannot replace position-wise discrete persistence filtering.

[Q2] Using TACG in RL rollouts and potential gains from specialized training.

Thank you for raising both questions—we evaluated TACG under the online group rollout sampling protocol of d1 (diffu-GRPO, NeurIPS 2025) on an uncontended NVIDIA H200 GPU (25 GSM8K prompts ×\times group size G=4G=4 = 100 stochastic trajectories at T=0.5,L=256,B=64T=0.5, L=256, B=64, ABBA-interleaved timing):

Backbone & Online Rollout Sampler (G=4,T=0.5G=4, T=0.5)Trajectory NFELatency / GroupRollout Speedup (vs. d1 / Native)Rollout Pass@1
LLaDA-8B — Original d1 Fixed (256 steps)256.0026.87 s1.00× / —63.0%
LLaDA-8B — Native Confidence (τ=0.90\tau=0.90)82.6910.39 s2.59× / 1.00×63.0%
LLaDA-8B — TACG Rollout (Zero-retuning)72.04 (-12.9%)9.50 s2.83× / 1.09×69.0% (+6.0 pp)
LLaDA-1.5 (VRPO) — Original d1 Fixed (256 steps)256.0032.72 s1.00× / —67.0%
LLaDA-1.5 (VRPO) — Native Confidence (τ=0.90\tau=0.90)80.1510.57 s3.10× / 1.00×69.0%
LLaDA-1.5 (VRPO) — TACG Rollout (Fast / Conservative)68.44 / 74.829.47 / 10.14 s3.46× / 1.12× (fast)70.0% / 71.0% (+2.0 pp)

Because online rollouts dominate GRPO wall-clock cost and synchronize on the slowest trajectory in each group (G=4G=4), TACG accelerates group rollout generation by 2.83×–3.46× over fixed-step d1 and 1.09×–1.12× over Native Confidence without modifying d1's loss formulation (which scores completed responses yy via resampled masked-ELBO likelihoods; as in confidence-based rollouts, altering commit order shifts the behavior rollout distribution while leaving the masked-ELBO update interface unchanged, and pairs even more cleanly with ratio-free objectives such as wd1; we leave closed-loop RL training dynamics to future work). At the same time, filtering unstable commits during stochastic exploration (T=0.5T=0.5) raises LLaDA-8B rollout Pass@1 by +6.0 pp (63.0% →\rightarrow 69.0%) and group Pass@4 by +12.0 pp (72.0% →\rightarrow 84.0%). Moreover, across all 2,240 items of our 4-task benchmark suite on the RL post-trained LLaDA-1.5 checkpoint, conservative TACG reaches 50.41% / 72.09 NFE, improving both accuracy (+0.37 pp) and NFE (-5.7%) over the LLaDA-1.5 default (50.04% / 76.43 NFE).

Regarding specialized training, our results point to clear headroom: comparing our single shared vector (49.62%–49.90%) against task-specific operating points (51.02%) on LLaDA-8B—and zero-retuning (49.62%) against conservative gating (50.41%) on LLaDA-1.5—shows roughly 0.8–1.4 pp of domain- and calibration-dependent headroom. Learning a lightweight context-aware calibration head over TACG's white-box statistics (ct(i),bt(i),streakt(i))(c_t(i), b_t(i), \text{streak}_t(i)) to adapt (τfloor,λ,kmax⁡)(\tau_{\text{floor}}, \lambda, k_{\max}) per block could capture this gain automatically while retaining full interpretability; likewise, adding cross-step consistency regularization during post-training to penalize high-confidence token flips would align the backbone's confidence calibration directly with trajectory-aware commitment at inference time. In the revised manuscript, we have added the online RL rollout and LLaDA-1.5 experiments to Section 5.8 and Appendix E.3 (Table 21) and noted lightweight calibration and RL training extensions in Section 6 (which we will expand further in the final version).

Thank you for the detailed rebuttal and presenting the new experiment results. It is inspiring to know that a good inference-time method like TACG could be beneficial in RL rollout. My main concern is still partially unresolved though - like the other two reviewers mentioned, several previous works also studied using history information to guide dllm inference. TACG's main contribution is a unifying commitment framework. But it seems like there is currently insufficient theoretical or empirical evidence to support this framework, to show that it is superior to previous works. I would lean toward keeping my score unchanged at this stage.

Reviewer_RDjTOpenReview ↗
Rating
3(+1)
Confidence
4(+1)
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

Masked diffusion LMs propose tokens at many positions per step; in non-revising decoders the commit schedule decides which proposals enter context. The paper argues that confidence alone is incomplete and proposes TACG, a training-free commit gate that combines temporal logit support (TILG), a history/persistence gate, and capped extra promotions while writing the contemporaneous base argmax.

Reported results are mainly on LLaDA-8B (with Dream / LLaDA2-Mini adapters). Under task-specific operating points (Table 1), mean ACC rises from 48.00% (Native Confidence) to 51.02% while mean NFE falls from 76.59 to 67.82. A matched per-event commit-count replay attributes an ACC gap to history-guided vs confidence-ranked position choice at matched commit counts (Tables 3–4).

Critical strengths

The matched per-event commit-count replay (Tables 3–4) is a useful attempt to isolate which positions commit from how many commits. The propose-vs-commit framing is conceptually clear even if the delivered method is a large composite of knobs.

Critical weaknesses
  1. Presentation and Readability: The method's presentation is needlessly opaque. Components like TILG, History Gate, escape floors, and KmaxK_{\text{max}} are introduced via dense equations (Eqs. 4–7) before providing an algorithmic overview or pseudo-code, making the mechanics hard to trace. Additionally, the HOLD-probe schematic in Figure 1 lacks sufficient self-contained context. The text also tends to over-claim: the abstract and introduction emphasize broad "trajectory-aware" improvements, yet Table 1 shows marginal or negligible per-task gains (e.g., GSM8K accuracy remains virtually unchanged at 77.10 vs. 77.03).

  2. Empirical Rigor and Hyperparameter Tuning: The primary results appear fragile. The headline LLaDA figures rely on per-benchmark tuning selected from a candidate pool rather than a single, fixed hyperparameter configuration across tasks. Results are reported over a single run (seed 516), omitting standard deviations or multi-seed validation. Furthermore, trade-offs are inconsistent—Table 5 shows Dream NFE actually increasing from 75.72 to 77.43 for a modest +0.77 accuracy bump. Without demonstrating robustness under a uniform parameter setting, the claimed efficiency advantages seem overstated.

Questions for authors
  1. Can you report Table 1 (and the Dream panel of Table 5) under one shared (w, λ, HG, floor) vector across all four tasks, with mean±std over ≥3 seeds? How much of the +3.02 mean ACC remains?

  2. For the LLaDA adapter, what fraction of the gain survives if KLASS eligibility (kl_ok) is removed so the comparison is purely confidence + TACG vs confidence?

  3. Please clarify the HOLD-probe protocol in Fig. 1 in plain language (what is held, for how many steps, and how bins are matched)—the current write-up is difficult to reproduce from text alone.

  4. How does TACG interact with revision-capable / remasking decoders where committed tokens can be revisited?

6 comments

[W1 & Q3] Method readability, plain-language HOLD-probe protocol (Fig. 1), and claim calibration.

Thank you for this candid and constructive feedback on readability. In plain terms, the HOLD probe in Figure 1 measures how often a high-confidence masked position revises its top-1 token if commitment is deferred by a single step. Specifically, at denoising step tt, we take a currently [MASK] candidate position ii with top-1 prediction x^t(i)=arg⁡max⁡vzt(i,v)\hat{x} _ t(i) = \arg\max _ v z _ t(i, v) and instantaneous confidence ct(i)c _ t(i). While all other selected positions commit normally at step tt, position ii is kept as [MASK] (held) for exactly 1 step (t→t+1t \to t+1); we then run step t+1t+1 on the updated context and record a flip whenever x^t+1(i)≠x^t(i)\hat{x} _ {t+1}(i) \neq \hat{x} _ t(i). To rule out instantaneous confidence as a confounder, all probed events are grouped into equal-width bins of ct(i)c _ t(i) and compared strictly within the same confidence bin: Figure 1 (left) contrasts persistent (streak≥3\text{streak} \ge 3) against non-persistent (streak<3\text{streak} < 3) proposals, and Figure 1 (right) contrasts positive two-step support growth (Δp>0\Delta p > 0) against non-positive change (Δp≤0\Delta p \le 0) at matched confidence and streak.

To make the method immediately clear, in the revised manuscript we have added a concise four-step workflow overview at the beginning of Section 4 prior to Equations (4)–(7)—summarizing the loop of extracting the base proposal and streak, computing TILG support against the EMA reference, filtering via HG with capped promotion (kmax⁡k _ {\max}), and committing native argmax writes—and we have expanded Figure 1's caption and Section 2.2 so the HOLD-probe protocol is fully self-contained. In the final version (using the extra camera-ready page), we will also move a compact Algorithm 1 box from Appendix A directly into Section 4.

Finally, we agree that per-task trade-offs should be stated without over-generalization. In the revised manuscript, we have calibrated the wording in the Abstract, Introduction, and Section 5.2 to distinguish benchmarks where TACG simultaneously improves accuracy and reduces NFE (HumanEval, MBPP, and MATH500 on LLaDA-8B) from GSM8K on LLaDA-8B, where the gain is primarily a 9.6%–14.0% NFE reduction at comparable accuracy (while on Dream-7B, TACG improves GSM8K accuracy by +2.20 to +3.87 pp).

[W2 & Q1] Shared (w, λ, HG, floor) results across tasks, seed variance, and Dream NFE trade-off.

Below we report the complete four-benchmark tables for LLaDA-8B (Table 1) and Dream-7B (Table 5, retaining each backbone's audited native base threshold) under one fixed (w, λ, HG, floor, k_max) vector shared across all four benchmarks—distinguishing pre-declared paper rows (Tables 13–14 & 18, originally Tables 12–13 & 15) from rebuttal sensitivity points—with 10,000-resample stratified paired bootstrap 95% CIs across all 2,240 test items:

Model & Configuration (ACC % / NFE)HumanEvalMBPPGSM8KMATH5004-Task MeanΔACC\Delta\text{ACC} & 10k Paired Bootstrap 95% CI
LLaDA-8B Native Confidence37.20/52.2847.08/83.3477.10/74.2630.60/96.4848.00 ±\pm 0.00 / 76.59Baseline (temp = 0.0, seed-invariant)
LLaDA-8B TACG Shared (Table 18 [orig. 15] pre-declared: w=0.35,λ=0.10,HG=3,floor=0.74w=0.35, \lambda=0.10, \text{HG}=3, \text{floor}=0.74)39.63/48.0250.19/74.5276.04/63.8332.60/85.0949.62 ±\pm 0.00 / 67.87 (-11.4%)+1.62 pp (53.6% retained); ΔACC:[+0.36,+3.00]\Delta\text{ACC}: [+0.36, +3.00] (p=0.0072p=0.0072), ΔNFE:[−9.42,−8.03]\Delta\text{NFE}: [-9.42, -8.03]
LLaDA-8B TACG Shared (Rebuttal: w=0.35,λ=0.15,HG=3,floor=0.75w=0.35, \lambda=0.15, \text{HG}=3, \text{floor}=0.75)39.63/47.9950.97/74.7976.19/64.0932.80/84.9649.90 ±\pm 0.00 / 67.96 (-11.3%)+1.90 pp (62.9% retained); preserves 98.4% of NFE reduction
LLaDA-8B TACG Task-Specific (Default, Table 1)42.07/45.7150.97/74.4277.03/67.1234.00/84.0451.02 ±\pm 0.00 / 67.82 (-11.4%)+3.02 pp; ΔNFE:[−9.18,−7.83]\Delta\text{NFE}: [-9.18, -7.83] steps
Dream-7B Native Confidence (seed = 516)47.56/51.1963.42/82.4673.16/74.6339.00/94.6055.79 / 75.72Baseline (fallback temp = 0.2, top-p = 0.95)
Dream-7B TACG Shared — Balanced (Rebuttal: w=0.35,λ=0.10,HG=2,floor=0.885,kmax⁡=2w=0.35, \lambda=0.10, \text{HG}=2, \text{floor}=0.885, k_{\max}=2)47.56/50.0862.26/83.8075.36/77.8939.80/94.6756.24 / 76.61 (NFE-neutral)+0.46 pp (59.7% retained); ΔNFE:[−0.47,+2.19]\Delta\text{NFE}: [-0.47, +2.19]; GSM8K +2.20+2.20 pp (p=0.0199p=0.0199)
Dream-7B TACG Shared — Zero-Drop (Table 14 [orig. 13] pre-declared: w=0.30,λ=0.035,HG=5/3,floor=0.88w=0.30, \lambda=0.035, \text{HG}=5/3, \text{floor}=0.88)47.56/55.1063.42/110.2375.82/91.7939.60/108.1556.60 / 91.32+0.81 pp (exceeds Default); zero drop on all 4/4 benchmarks
Dream-7B TACG Task-Specific (Default, Table 5)50.00/50.8063.04/93.5373.62/74.2639.60/91.1456.56 / 77.43+0.77 pp; improves both ACC & NFE on 3/4 benchmarks

Fixing one shared vector across all four tasks retains 53.6%–62.9% of the +3.02 pp task-specific gain on LLaDA-8B (49.62%–49.90%) while preserving ~100% of the NFE savings (67.87–67.96 vs. 67.82). Even without target-task peeking, all 10/10 pre-declared vectors in Table 13 (orig. Table 12) beat Native Confidence on both macro ACC and NFE (worst vector: 48.43% / 66.54 NFE; strict leave-one-benchmark-out blind selection: 49.12% / 66.96 NFE). On Dream-7B, a balanced shared vector (HG=2,τesc=0.98,kmax⁡=2\text{HG}=2, \tau_{\text{esc}}=0.98, k_{\max}=2) retains 59.7% of Default's +0.77 pp gain (56.24%, with +2.20 pp on GSM8K at p=0.0199p=0.0199 and +0.80 pp on MATH500) at 76.61 NFE—statistically indistinguishable from Native Confidence's 75.72 NFE (95% CI [-0.47, +2.19] steps). On multi-seed variance: LLaDA-8B uses deterministic greedy argmax (temp = 0.0) over full ordered splits (std = 0.00), whereas Dream-7B uses deterministic argmax on thresholded commits and official stochastic fallback (temp = 0.2, top-p = 0.95, seed = 516). Beyond the 10k paired bootstrap CIs above, under fully stochastic rollouts (T=0.5T = 0.5, 100 trajectories), TACG lifts LLaDA-8B accuracy by +6.0 pp (63.0% →\rightarrow 69.0%) with -12.9% NFE.

In Table 5, Dream-7B Default actually improves both accuracy and NFE on 3/4 benchmarks—HumanEval (47.56/51.19 →\rightarrow 50.00/50.80), GSM8K (73.16/74.63 →\rightarrow 73.62/74.26), and MATH500 (39.00/94.60 →\rightarrow 39.60/91.14). Higher NFE occurs only on MBPP and under strict 5-step persistence (Zero-Drop at 56.60% / 91.32 NFE, or Table 18 [orig. 15] Across-task at 56.75% / 90.82 NFE), where stricter HG defers Dream-7B's sharper early probabilities to prevent premature errors. In the revised manuscript, we have added the shared cross-task rows to Tables 1 and 5, reported the paired bootstrap CIs and leave-one-benchmark-out check in Sections 5.1–5.2, 5.7, and Appendix E.1 (Table 18), and clarified Dream-7B's NFE trade-off in Section 5.7.

[Q2] Gain surviving on LLaDA when KLASS eligibility (kl_ok) is removed.

100% of the reported gain survives—both in the shared setting (48.00% →\rightarrow 49.62%–49.90%, -11.4% NFE) and in the task-specific setting (48.00% →\rightarrow 51.02%, -11.4% NFE)—because KLASS eligibility is already completely disabled in all Confidence + TACG experiments. In our codebase, Confidence + TACG sets the KL threshold to an inactive constant (τKL=106\tau_{\text{KL}} = 10^6, history window 1), which short-circuits kl_ok to True for all masked positions without computing step-wise KL divergence (kl_ok is active only when evaluating the KLASS baseline itself). We apologize that retaining the unified adapter argument in Appendix A caused this confusion; in the revised manuscript, we have updated both Section 4.3 and the pseudocode in Appendix A to state explicitly that Confidence + TACG uses pure confidence + TACG (kl_ok ≡\equiv True).


[Q4] Interaction with revision-capable / remasking decoders.

Although our experiments focus on standard absorbing-state masked diffusion decoders, TACG's two trajectory signals pair naturally with revision-capable / remasking decoders (such as ReMDM or predictor-corrector dLLMs) both before and after commitment. During forward unmasking, gating candidates with HG and TILG prevents oscillating tokens from being written prematurely, which reduces the corrective forward passes a remasking step must spend undoing early errors. Conversely, when a remasking controller selects previously committed tokens to revert back to [MASK], a sharp negative shift in temporal support (qt(i,xi)−ptref(i,xi)<0q_t(i, x_i) - p_t^{\text{ref}}(i, x_i) < 0) as surrounding context fills in offers a natural trajectory-aware remasking cue alongside low instantaneous confidence; once a position is remasked, resetting its counter (streak = 0) re-enters it directly into TACG gating. We have added a brief note on this extension to Section 6 of the revised manuscript (and will expand the discussion further in the final version).

Reviewer_ZwUmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TACG, a training-free decoding method for dLLMs. The authors argue that current confidence alone is not enough to decide when a token should be committed because tokens with similar confidence can have different prediction histories. Therefore, TACG basically compare current logits with historical EMA logits, and use a History Gate to check whether the predicted token has remained stable across diffusion steps. The method also allows a limited number of extra token commitments, while keeping the current base token proposal unchanged.

Critical strengths
  1. I think the paper raises a good question about a well known problem of token commitment in dLLMs. To me, the motivation is reasonable, since two tokens with similar confidence may have different prediction histories. I also like the HOLD analysis in Figure 1, which shows that proposal stability and support changes provide information beyond current confidence.
  2. The method is simple and training-free. TILG uses logits that are already available during decoding, while HG checks whether the current proposal has remained stable.
Critical weaknesses
  1. Missing closely related work and baseline comparisons. I think the paper is missing some important related work on commitment decisions in dLLMs as this problem is well studied. For example, VSB [1] also studies when to commit, although it focuses on block-level decisions. Also, TraceLock [2] directly studies token commitment using decoding history, which is quite close to the motivation of TACG in this paper.
  • Although TACG is training-free while TraceLock learns a commitment policy, both methods use information beyond current confidence to decide when a token should be committed. I think the authors should discuss these works and better explain what is new in TACG. It would also be useful to compare with them where the decoding settings are compatible.
  1. The contribution of each component is not very clear. TACG combines TILG, HG, extra commitments and fallback strategies. Yet, the selection experiment in Table 2 changes several of these components together. I am not sure how much of the improvement actually comes from the proposed TILG rather than the persistence gate or the extra commit budget.
  • I can see the authors do provide logit reference interventions in Sec 5.5, which is useful, but these mainly focus on changes in decoding trajectories and final texts. I think a more direct ablation on actual accuracy and NFE would be helpful. For example, I would like to see confidence + HG, confidence + extra commitments, and full TACG compared under the same decoding budget.

  • I also noticed that Figure 1 shows decreasing historical support is associated with less stable predictions, but Eq 6 only uses a positive support bonus. In other words, TILG does not and can not penalize tokens whose historical support is decreasing. I am not sure why the authors chose this design.

  1. The actual inference speedup is not fully clear. Since TACG is a training-free decoding method intended to improve inference efficiency, I think end-to-end inference time is more important than just NFE. The method introduces additional operations such as maintaining historical logits, computing EMA and temporal support, applying HG, and ranking extra commit candidates. Although these operations do not require extra model forward passes, I think they still introduce computational and memory overhead. Also, App E.2 uses block length 32 for the original decoder and 64 for TACG, which makes it difficult to attribute the speed difference entirely to TACG.
  • I think the authors should provide matched end-to-end latency, throughput, and memory comparisons using the same block size, batch size, and decoding settings. This would make the actual efficiency benefit much clearer.
  1. The connection between token-level commitment and block-based decoding is not very clear. TACG makes commitment decisions at the token level, but the tested dLLMs such as LLaDA and LLaDA 2 still follow their original block-based generation schedules with block size 32. The method does not directly change block boundaries or block sizes.
  • Also, I am not sure how TACG interacts with different block sizes and dependencies between blocks as the method does not seem to focus on block factors. I can see App A mentions that LLaDA keeps proposal histories across blocks and tracks proposal streaks over the full sequence. Yet, a token that appears stable before earlier blocks are completed may not remain stable after more context is generated. Also, changing the block size or the number of denoising steps may change how much history is available for TILG and HG.

  • I think the authors should clearly explain how token-level commitment interacts with block-level generation and provide experiments with different block sizes as you are studying on block diffusion. It would also be very useful to compare keeping historical information across blocks with resetting it at block boundaries.

  1. The plug-and-play ability of TACG is not sufficiently tested. Since TACG is a training-free module, I would expect it to be relatively easy to apply to existing dLLM training methods such as d1 (NIPS2025) [3], wd1(ICLR2026) [4], StableDRL(ICML2026) [5], which are RL post-training methods for dLLM that can improve dLLM reasoning. Yet, the paper only evaluates the selected pretrained backbones and does not study whether TACG works with dLLM training method or works on RL-trained models.
  • Therefore, I think this is an interesting missing experiment because RL post-training can change token confidence, denoising trajectories, and the stability of intermediate predictions. It is unclear whether the same TILG and HG settings would still work well on these existing dLLM training methods.

[1] Wang, D., Qiu, R., & Huang, Z. (2026). When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models. arXiv:2604.23994.

[2] Sun, B., Zhu, M., Caso, F., Gu, J., Yu, J., Torr, P., Liò, P., & Yu, J. (2026). The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models. arXiv:2605.24697.

[3] Zhao, S., Gupta, D., Zheng, Q., & Grover, A. (2025). d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning. NeurIPS 2025.

[4] Tang, X., Dolga, R., Yoon, S., & Bogunovic, I. (2026). wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models. ICLR 2026.

[5] Zhong, J., Wang, K., Ding, D., Feng, Z., Bai, H., Xiang, Y., Sun, J., & Xu, Q. (2026). Stabilizing Reinforcement Learning for Diffusion Language Models. ICML 2026.

Questions for authors

Q1 Could the authors discuss the closely related commitment-based dLLM work VSB [1] and TraceLock [2] and clarify how TACG differs from these methods? In particular, TraceLock also uses decoding history to guide token commitment. What is the main advantage of TACG beyond being training-free?

Q2 How were the task-specific parameters in Table 10 and 11 selected? Were they tuned on separate validation data? I didn’t find any details in validation set split in the paper which is confusing for how you select the hyperparameters.

Q3 Could the authors provide a more direct ablation of TILG and HG? How much improvement does TILG provide over confidence + HG with the same extra commit budget? Also, why does Equation (6) only use positive historical support rather than directly penalizing decreasing support?

Q4 Could the authors explain why the confidence-ranked baseline in Tables 3 and 4 achieves only 41.16 accuracy compared with 48.00 for Native Confidence in Table 1? Since the replay follows TACG's commit-count schedule, could both methods also be compared under independently chosen decoding schedules at similar NFE?

Q5 Could the authors report matched end-to-end inference latency, throughput, and memory usage? Since TACG introduces additional computations beyond the model forward pass, I think these measurements are important. Could the authors also use the same block size for the baseline and TACG in App E.2?

Q6. How does TACG interact with block-based decoding? In particular, how sensitive are TILG and HG to block size and the number of denoising steps? Why is proposal history retained across blocks in LLaDA, and what happens if this history is reset at each block boundary?

Q7. Because TACG is tuning free, can it work with dLLM training methods? I would like to see whether it can improve these RL trained checkpoints without additional tuning. Could it also be used during RL rollout generation and would this require any changes to the training objective?

9 comments

[W1 & Q1] Comparison with VSB [1] and TraceLock [2]; main advantages beyond being training-free.

Thank you for pointing out these closely related concurrent studies. Because VSB [1] restructures variable-size block boundaries and TraceLock [2] trains a backbone-specific policy head (with neither providing plug-in weights compatible with our fixed-block training-free protocol), we have added a dedicated discussion to Section 3 (Related Work) and the comparison table below in Appendix B (Table 10) of the revised manuscript (and will move the table into Section 3 in the final version), situating them alongside all compatible training-free trajectory baselines empirically evaluated under our unified protocol in Table 1 (where TACG reaches 49.62%–49.90% / 67.87–67.96 NFE under a single shared setting and 51.02% / 67.82 NFE default, compared with DSB at 49.35% / 75.15 NFE, KLASS at 49.03% / 128.45 NFE, and AHD at 47.81% / 76.65 NFE).

MethodDecision GranularityTraining-FreeSignal Beyond Current ConfidenceToken Write Rule
VSB [1]Block-level boundaryYesNF vs. FA context-conditioning divergence (block self-containedness)Native argmax
TraceLock [2]Token-levelNo (learned controller)Multi-step trace states scored by a self-supervised controllerNative argmax (learned commit gate)
TACG (Ours)Token-level (any block schedule)YesExplicit decoupling: discrete persistence (HG) + positive support growth (TILG)Strict native argmax

Beyond requiring no policy training, TACG differs fundamentally from TraceLock [2] in both mechanistic transparency and cross-checkpoint generalization. TraceLock trains a parameterized controller on self-supervised trace states from a specific frozen backbone; although reusable across sequence lengths or step budgets on that same checkpoint, moving to a new architecture or applying RL post-training—which reshapes confidence sharpness and intermediate trajectory dynamics—requires recollecting traces and retraining the controller. By contrast, TACG explicitly separates discrete proposal persistence (HG, which filters transient top-1 oscillations) from continuous temporal support growth (TILG, which ranks stable candidates for promotion), while strictly writing the backbone's current-step argmax token. This white-box separation enables clean counterfactual verification (Tables 3–4 and Figure 3) and transfers directly across dense (LLaDA-8B, Dream-7B) and MoE (LLaDA2-Mini) models as well as RL post-trained checkpoints (LLaDA-1.5) and online d1 rollouts (see [W5 & Q7]).

From a runtime and scheduling perspective, TACG requires no auxiliary network forward pass and adds less than 1 ms/step of active-block gating compute, delivering 1.15×–1.57× end-to-end wall-clock speedups over Native Confidence at matched block sizes (see [W3 & Q5]). Moreover, whereas VSB [1] adapts macro-level block boundaries, TACG governs micro-level token commitment within each active block, making the two mechanisms orthogonal and directly composable.

[W2 & Q3] Direct ablations of TILG, HG, and extra commits; why Eq. (6) uses positive clipping.

To isolate the exact contribution of each component under a fixed budget, we ran a controlled orthogonal ablation across all four benchmarks (2,240 items) on LLaDA-8B using the single shared configuration (w=0.35,λ=0.10,β=0.95,τfloor=0.74,HG=3,kmax⁡=2w=0.35, \lambda=0.10, \beta=0.95, \tau_{\text{floor}}=0.74, \text{HG}=3, k_{\max}=2):

Configuration (LLaDA-8B, Single Shared Setting)4-Task Mean ACC (%)Mean NFE
Native Confidence (τ=0.90\tau=0.90)48.0076.59
Confidence + HG (kmax⁡=0k_{\max}=0, no TILG)48.1979.99
Confidence + Extra (kmax⁡=2k_{\max}=2, no HG, no TILG)47.8859.79
Confidence + HG + Extra (kmax⁡=2k_{\max}=2, w/o TILG)48.6068.15
Full TACG (HG + TILG + Extra, kmax⁡=2k_{\max}=2)49.6267.87

Comparing Confidence + HG + Extra (w/o TILG) directly against Full TACG under the exact same extra commit budget (kmax⁡=2k_{\max}=2) and virtually identical NFE (68.15 vs. 67.87), adding TILG improves 4-task mean accuracy from 48.60% to 49.62% (+1.02 pp), driven by gains of +1.83 pp on HumanEval, +1.55 pp on MBPP, and +1.00 pp on MATH500. Meanwhile, enabling extra commitments without HG or TILG (Confidence + Extra, 47.88%) degrades accuracy by prematurely locking in unstable tokens, whereas Confidence + HG alone improves accuracy (48.19% on LLaDA-8B, and 56.98% [+1.19 pp] across all four tasks on Dream-7B) at the cost of higher NFE. Combining HG and TILG is what achieves higher accuracy and lower NFE simultaneously.

Regarding Equation (6), bt(i)=max⁡(qt(i)−ptref(i),0)b_t(i) = \max(q_t(i) - p_t^{\text{ref}}(i), 0) uses positive clipping because TACG decouples eligibility filtering from promotion ranking. Unstable tokens whose top-1 identity is still oscillating are already excluded from eligibility by HG (streak≥3\text{streak} \ge 3) and τfloor\tau_{\text{floor}}. Among stable candidates that have already passed HG, local context updates as neighboring tokens unmask can cause minor step-to-step probability dips; positive clipping ensures TILG acts purely as a promotion bonus for candidates with strengthening support, leaving candidates with qt≤ptrefq_t \le p_t^{\text{ref}} at bt(i)=0b_t(i)=0 so they naturally yield limited kmax⁡k_{\max} promotion slots without perturbing the base admission threshold. When we replace positive clipping with unclipped signed differences in gating and ranking, 4-task accuracy drops from 49.62% to 48.83% (-0.79 pp). In the revised manuscript, we have clarified positive clipping in Section 4.2 and added this orthogonal ablation to Section 5.3 and Appendix D.1 (Table 15; to be moved into Section 5.3 in the final version).

[W3 & Q5] Matched-block latency, throughput, and memory profiling (including Appendix E.2).

We completely agree that block sizes must be matched to isolate TACG's true system efficiency. On an uncontended NVIDIA H200 GPU (matching Section 5.8, Tables 8–9), we re-profiled Original (fixed steps), Native Confidence, and TACG (shared setting) on LLaDA-8B under strictly identical block sizes (BL = 32 across all methods, and BL = 64 across all methods) for both GSM8K requests and sequence-length scaling:

Workload & Matched Block (Batch = 1)Original Latency / Tok/sNative Conf Latency / Tok/s (NFE)TACG Latency / Tok/s (NFE)TACG Speedup (vs. Orig / Native)Peak VRAM (Native / TACG)
GSM8K (BL = 32, L = 256)5,818.0 ms / 44.01,937.4 ms / 132.1 (84.2)1,688.7 ms / 151.6 (70.7)3.45× / 1.15×15.13 / 15.44 GiB (+0.31)
GSM8K (BL = 64, L = 256)5,777.3 ms / 44.31,832.6 ms / 139.7 (81.6)1,497.8 ms / 170.9 (64.9)3.86× / 1.22×15.19 / 15.57 GiB (+0.38)
Scaling (BL = 32, L = 256)6,266.5 ms / 40.92,746.9 ms / 93.2 (112)1,901.5 ms / 134.6 (79)3.30× / 1.44×15.11 / 15.43 GiB (+0.32)
Scaling (BL = 32, L = 512)18,999.9 ms / 27.05,002.9 ms / 102.3 (138)3,932.9 ms / 130.2 (102)4.83× / 1.27×15.17 / 15.85 GiB (+0.68)
Scaling (BL = 64, L = 512)29,160.4 ms / 17.68,228.6 ms / 62.2 (141)5,243.5 ms / 97.6 (92)5.56× / 1.57×15.23 / 15.90 GiB (+0.67)
Scaling (BL = 64, L = 1024)108,260.6 ms / 9.515,331.4 ms / 66.8 (140)10,692.2 ms / 95.8 (90)10.13× / 1.43×15.36 / 16.74 GiB (+1.38)

Because candidate gating and ranking operate without extra forward passes, at standard decoding length (L = 256) TACG adds only 0.62–0.87 ms/step of compute (< 4% of a single Transformer step, e.g., 23.08 ms vs. 22.46 ms/step on GSM8K at BL = 64) and +0.31–0.38 GiB of peak allocated VRAM (~2% of model memory). As a result, NFE savings translate directly into 1.15×–1.57× end-to-end speedups over Native Confidence (3.30×–10.13× over Original), and batched decoding (Batch = 4, L = 512) similarly reaches 200.9 tok/s (1.44× faster than Native Confidence at 139.3 tok/s). In the revised manuscript, we have updated Section 5.8 and added these matched-block profiles to Appendix E.2 (Table 20), and will regenerate the Appendix E.2 scaling figures under matched blocks in the final version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.