- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Codpula, which combines a probabilistic inference layer with diffusion large language models. Codpula includes context-dependent token potentials and coupling gates, and uses a dual-flow update for training. Token dependencies are also used for guiding the decoding. Experiments show that Codpula achieves better efficiency-quality trade-offs.
Critical strengths
- The proposed method targets a critical problem for diffusion large language models. The authors also provide a theoretical analysis to show that the proposed update preserves valid probability parameters. In some settings, the proposed method achieves good performance and better accuracy-latency trade-offs.
- The proposed method addresses two limitations of CoDD. CoDD uses a frozen backbone and fixed coupling parameters. Instead, Codpula introduces backbone adaptation and context-dependent coupling gates.
Critical weaknesses
- The experiments are not comprehensive. The benefits of context-dependent coupling are unclear. The experiments on LLaDA only use Input Modulation without Sum Modulation. In Table 2, adding Sum Modulation sometimes makes performance worse. The experiments are only on math and coding tasks. There are no evaluations on general tasks like QA. And CoDD is absent in Figure 3 and Figure 4.
- The proposed method is limited. The authors contend that they jointly optimize both components, but for LLaDA 1.0, they train the PC and adapter in two stages. And Codpula uses separate models for math and code tasks. So the generalizability of Codpula may be limited.
Questions for authors
- Please see the weaknesses.
- Is the proposed Codpula compatible with KV cache?
- How are the checkpoints in Table 5 selected? Are they selected based on the test sets?
5 comments
(Continued W1, Q1) Codpula's has lower per-step overhead than the CoDD across all four backbones. We compare both methods at batch sizes 1 and 64 and include CoDD in revised Figures 3-4. At batch size 1, Codpula (Input) adds 3.4-15.4% overhead, compared with 62.8-287.8% for CoDD. On SDAR, Codpula (Input + Sum) adds 14.8% versus CoDD's 187.9% at batch size 1 and 22.1% versus 53.3% at batch size 64. Adding Sum modulation increases overhead by at most 1.9 percentage points across the measured configurations.
Table 4. Per-step overhead over the base model (%); B denotes batch size.
| Backbone | Task | CoDD, B=1 | Codpula Input, B=1 | Codpula Input + Sum, B=1 | CoDD, B=64 | Codpula Input, B=64 | Codpula Input + Sum, B=64 |
|---|---|---|---|---|---|---|---|
| LLaDA-1.0 | GSM8K | 266.3 | 10.4 | 12.0 | 5.0 | 2.5 | 2.6 |
| LLaDA-1.0 | MATH500 | 184.6 | 7.2 | 8.3 | 2.8 | 1.4 | 1.4 |
| LLaDA-1.0 | HumanEval | 184.8 | 7.1 | 8.2 | 2.4 | 1.2 | 1.2 |
| LLaDA-1.0 | MBPP | 185.0 | 7.1 | 8.2 | 2.5 | 1.2 | 1.3 |
| Dream | GSM8K | 287.8 | 15.4 | 17.3 | 6.1 | 3.3 | 3.4 |
| Dream | MATH500 | 212.0 | 11.3 | 12.8 | 3.5 | 1.9 | 1.9 |
| Dream | HumanEval | 184.7 | 9.9 | 11.1 | 3.1 | 1.6 | 1.7 |
| Dream | MBPP | 209.3 | 11.2 | 12.6 | 3.2 | 1.7 | 1.8 |
| LLaDA-2.0 | GSM8K | 65.2 | 3.5 | — | 10.8 | 6.1 | — |
| LLaDA-2.0 | MATH500 | 62.8 | 3.4 | — | 7.0 | 4.0 | — |
| SDAR | Code | 187.9 | 13.4 | 14.8 | 53.3 | 21.0 | 22.1 |
Overhead is per decoding step (one NFE: one backbone call and one circuit call). The circuit costs about the same on every backbone (2.6-3.8 ms at B=1, 14-35 ms at B=64); the backbone step does not. LLaDA-1.0 and Dream re-encode the full sequence at every step, so their step cost grows with sequence length (GSM8K generates 256 tokens, the other tasks 512, hence the higher GSM8K overhead) and with batch size (24-38 ms at B=1, 1.0-2.3 s at B=64). SDAR decodes one 16-token block against a KV cache, so its step is short (19.5 ms at B=1, 66 ms at B=64) and the same for every task; the same circuit cost is therefore a larger fraction, most of all at B=64. The LLaDA-2.0 forward grows only 5-8$\times$ from B=1 to B=64, so its overhead also rises with batch size.
(W2, Q1). Joint optimization is not a requirement for Codpula, and generalization is supported by more evidence.
Joint optimization. Our new LLaDA-1.0 experiments jointly optimize the backbone adapter, PC, and router, matching or exceeding the paper's staged Input-only model in 9 of 12 settings. Thus, two-stage training is not a requirement of Codpula. The original schedule was a backbone-specific optimization choice: Appendix A.1 discusses inconsistent or negative effects of standard SFT on LLaDA reported by d1 [1], DSFT [2], and Blockwise SFT [3], alongside related fine-tuning sensitivity reported by CDLM [4].
Generalization. Our protocol follows established practice in diffusion LM post-training: d1 (Section 4.1) [1] trains separate models per task, while JustGRPO (Section 5) [5] uses task-specific mathematical training splits and a coding dataset. We also directly evaluate generalization beyond the training domains: we reuse the paper's checkpoints on general QA benchmarks (MMLU and GPQA) without additional training, obtaining improvements across three backbones. For example, Dream improves from 18.2% to 45.9% on MMLU at 8 tokens/step, and LLaDA-2.0 improves from 18.3% to 31.5% on GPQA at 4 tokens/step.
Q2. KV caching is compatible with Codpula and orthogonal to our contribution. Circuit refinement operates on the backbone's outputs without changing its attention structure. Existing PC algorithms [6] also provide a basis for caching circuit computations. Fully exploiting these caching opportunities requires additional engineering, which we leave to future work.
Q3. We select a single checkpoint for each backbone, method variant, and task family, rather than independently selecting checkpoints for each evaluation benchmark or decoding budget. Specifically, we use MATH500 for math and MBPP for code, screening intermediate checkpoints on subsets before evaluating the candidates on the full selection benchmark. The selected checkpoints are then reused across the remaining benchmarks in each task family. We also evaluate these existing checkpoints on MMLU and GPQA without additional training or checkpoint selection.
We apply the same accuracy-based checkpoint-selection protocol to the independently tuned CE-SFT and CoDD baselines. This is consistent with best-checkpoint reporting in prior diffusion LM post-training work, including d1 [1], wd1 [7], and SPG [8]. We acknowledge that MATH500 and MBPP serve as model-selection benchmarks rather than strictly held-out evaluations. Importantly, improvements on the other benchmarks provide independent evidence beyond checkpoint selection.
We hope these clarifications and additional results address your concerns. Please let us know if any points would benefit from further clarification. We would appreciate it if you could consider raising your score.
[1] d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning, Zhao et al.
[2] DSFT: Inspiring Diffusion Large Language Models to Comprehend Mathematical and Logical Patterns, Chen et al.
[3] Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding, Sun et al.
[4] CDLM: Consistency Diffusion Language Models for Faster Sampling, Kim et al.
[5] The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models, Ni et al.
[6] Lossless Compression with Probabilistic Circuits, Liu et al.
[7] wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models, Tang et al.
[8] SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models, Wang et al.
We thank the reviewer for recognizing the significance of the problem, our theoretical analysis of the proposed learning algorithm, the improved performance and accuracy-latency trade-offs, and Codpula's contributions in addressing limitations other work did not. We next address the reviewer's concerns through additional training ablations, evaluations on general QA tasks, and direct comparisons of inference overhead.
(W1, Q1). Broader evaluation and clearer evidence for context-dependent coupling.
We provide additional LLaDA-1.0 results comparing Input-only and Input + Sum modulation under both staged and joint training, along with evaluations on the general QA benchmarks MMLU and GPQA. We add CoDD to Figures 3-4 and show that Codpula's implementation incurs roughly half CoDD's per-step overhead (43-61% lower at batch size 64) across all four backbones.
Sum modulation improves LLaDA-1.0 in 11 of 12 settings under joint training. We now evaluate both modulation variants under both staged and joint training. Compared with jointly trained Input-only Codpula, Input + Sum improves all four benchmarks at 4 and 8 tokens per step, including 13.4% → 22.4% on MATH500 at 8 tokens/step and 32.3% → 41.6% on MBPP at 4 tokens/step. The jointly trained Input + Sum model also matches or exceeds the paper's staged Input-only model in 9 of 12 settings. These results demonstrate the benefit of context-dependent coupling, particularly at higher parallelism, while the mixed staged-training results show that the training schedule matters.
Table 1. LLaDA-1.0 accuracy (%). Column numbers indicate tokens per step. CE-SFT results are from Table 3 of the paper. Bold indicates the best result in each column.
The paper's checkpoints improve QA accuracy without additional training. We provide additional QA results on MMLU and GPQA by reusing the checkpoints from the paper, without additional training. On GPQA, Codpula improves LLaDA-2.0 from 18.3% to 31.5% at 4 tokens/step and SDAR from 29.7% to 34.2% at 2 tokens/step over their respective base models.
Table 2. GPQA accuracy (%). Column numbers indicate tokens per step. Bold indicates the best result in each column.
On MMLU, we reuse the Dream and LLaDA-2.0 checkpoints from the paper without additional training. Codpula achieves the highest accuracy in all seven settings. With entropy-based decoding at 8 tokens/step, Input + Sum modulation improves Dream's accuracy from 18.2% to 45.9% over the base model and exceeds CE-SFT by 5.9 percentage points.
Table 3. MMLU accuracy (%). Column numbers indicate tokens per step. Dream uses entropy-based decoding. Bold indicates the best result in each column.