- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes content-before-position decoding: selecting a token using evidence across positions before deciding where to place it. A one-step Bayesian analysis motivates this approach. The resulting COLA decoder combines token-first proposals with local acceptance and is evaluated on DiffusionGemma and Nemotron-Labs-Diffusion.
Critical strengths
-
COLA improves a competitive model using measured runtime comparisons. On DiffusionGemma, ParallelBench quality increases from 93.12% to 93.47%, while mean latency falls from 3.105 to 2.074 seconds. This is direct evidence of a useful improvement over the model’s EB decoder.
-
Table 3 compares token-first and entropy position-first selection within the same controller. Token-first reduces model evaluations from 90.80 to 41.96 per 100 generated tokens without reducing the overall score. This supports the contribution of the proposed selection rule.
Critical weaknesses
-
The speed improvement is not consistent across tasks or models. On DiffusionGemma’s text-generation tasks, latency increases from 1.881 to 3.968 seconds. On Nemotron, COLA requires more model evaluations and is slower in the small GSM8K and MBPP timing samples. The evidence supports a narrower benefit than general decoding acceleration.
-
The controlled comparison uses entropy position-first selection but does not include maximum-confidence selection within the same controller. Since maximum confidence is an endpoint of the paper’s Bayesian formulation, this comparison is important for establishing the benefit of aggregating evidence across positions.
Questions for authors
-
Does token-first selection outperform maximum-confidence selection when the rest of the controller is unchanged?
-
What exact EB version, temperature, stopping rules, and forward limit were used? How were the EB and COLA settings selected?
-
On a code-generation benchmark, does COLA improve quality at equal runtime over EB, with output lengths and truncation rates also reported?
2 comments
3. Code-generation quality under a fixed time budget
On DiffusionGemma MBPP, COLA correctly completes more problems at every reported per-example time budget.
Comparison under the same time budget. We keep the decoding configurations from Table 2 unchanged and associate each example’s recorded generation time with its test outcome. For a given time budget, an example counts as successful when generation finishes within that budget and the generated program passes all tests. Examples that finish later or fail the tests count as unsuccessful. The denominator is always all 450 problems.
Timings use batch size 1, with one BF16 model instance sharded across three NVIDIA RTX 3090 GPUs, each with 24 GB of memory, without CPU or disk offloading. The timed interval includes prompt prefill and the complete generation process, including decoder control. Model loading, input preparation and GPU transfers, output text processing, test execution, and result writing are outside the timed interval.
| Per-example time budget | EB completed | EB correct within budget | COLA completed | COLA correct within budget |
|---|---|---|---|---|
| 2 s | 11 | 10/450 (2.22%) | 77 | 69/450 (15.33%) |
| 5 s | 68 | 62/450 (13.78%) | 298 | 262/450 (58.22%) |
| 10 s | 158 | 143/450 (31.78%) | 404 | 338/450 (75.11%) |
| 20 s | 262 | 216/450 (48.00%) | 442 | 349/450 (77.56%) |
| 30 s | 325 | 242/450 (53.78%) | 450 | 350/450 (77.78%) |
| 60 s | 434 | 257/450 (57.11%) | 450 | 350/450 (77.78%) |
| All runs finished | 450 | 257/450 (57.11%) | 450 | 350/450 (77.78%) |
At a shared 10-second budget, COLA correctly completes 195 more problems, improving the correct-completion rate by 43.33 percentage points. At 5 seconds, COLA already correctly completes 262 problems, exceeding EB’s 257 correct solutions after all 450 runs finish.
Output lengths and truncation. Both methods use a 512-token output budget. The following statistics describe the complete runs over all 450 examples. Output truncation means exhausting the output-token budget without normal termination.
| Method | Mean output length (tokens/example) | Output truncation (%) |
|---|---|---|
| EB | 210.74 | 3.33 |
| COLA | 158.42 | 5.11 |
These results show that, under the fixed Table 2 configurations, COLA delivers more correct programs within the same per-example generation-time budgets.
Thank you for your thoughtful review and constructive questions. We appreciate your recognition of the measured runtime improvements over DiffusionGemma’s EB decoder and the controlled evidence supporting token-first selection. Your comments help us clarify the comparison with maximum confidence, document the decoding setup, and examine code-generation quality under a shared time budget. We address these points below with controlled comparisons, experimental settings, and a fixed-budget analysis on MBPP, followed by clarification of the two weaknesses.
1. Maximum-confidence versus token-first selection under the same controller
We compare maximum-confidence and token-first proposals while keeping the remaining COLA components and settings fixed, including residual support, KL suppression, local acceptance, risk budgeting, and revision. All results below therefore use the full COLA controller, with only the proposal rule changed.
On ParallelBench, evaluated over 1,700 examples:
We also evaluate the same controlled comparison on downstream mathematics and code tasks. Each entry reports Score (%) / NFE/100.
Token-first (aggregate evidence) yields higher Score and lower NFE on ParallelBench and all four downstream tasks, with particularly large quality gains on code generation. These comparisons directly support the contribution of aggregate-evidence ordering within the complete COLA controller.
2. EB implementation, decoding settings, and operating-point selection
We use the DiffusionGemma EB implementation bundled with Hugging Face Transformers 5.14.1 (
huggingface/transformers, release tagv5.14.1). The sampler isEntropyBoundSamplerinsrc/transformers/models/diffusion_gemma/generation_diffusion_gemma.py, configured throughEntropyBoundSamplerConfigand invoked viaDiffusionGemmaForBlockDiffusion.generate().Decoding settings. EB uses an entropy bound of 0.1 and retains the official stability and low-entropy early-stopping mechanisms, with
stability_threshold=1andconfidence_threshold=0.005. Its maximum number of model forwards per canvas is the official default of 48.For COLA, we deliberately set a loose upper bound of 512 forwards per canvas; actual denoising finishes well before this limit. Reported NFE counts the model forwards actually executed. Both methods use linear temperature scaling from 0.8 to 0.4 over their respective per-canvas step budgets, restarting the schedule at each new canvas.
Both methods use the official processor and apply temperature scaling before computing probabilities. We apply no additional top-k or top-p truncation, repetition penalty, or special-token or EOS logit masking. EOS is used for normal generation termination and output truncation.
Operating-point selection. For each downstream task, COLA and EB use the same 50-example development set, disjoint from the final test set. Development sampling uses seed
20260921, and both methods’ configurations were fixed before test evaluation.For COLA, we started from the ParallelBench settings and evaluated approximately three to five configurations per task, manually adjusting canvas length and controller parameters using the development set. For EB, we initially tested canvas lengths of 32, 64, 128, and 256. Later trials focused on 32 and 64. The other EB controller parameters retained the official defaults recommended for generation quality and latency.
The final task-specific settings are listed below. Output caps apply to both methods and are measured in generated tokens. COLA margins are listed in commitment / rollback order.
COLA fixes the residual-coverage weight
beta=1, the KL parametergamma=1, and the numerical floor at1e-6. Local acceptance and the risk budget determine the number of commitments per forward, without a fixed count cap.