acceptodds
Under review as a conference paper at ICLR 2027

Content Before Position: A Bayesian Decision Framework for Diffusion Language Model Decoding

Abstract

Diffusion language models predict tokens across a canvas in parallel, yet decoding typically prioritizes positions by local confidence. This can overlook collective evidence for a token distributed across several candidate positions. We introduce content-before-position decoding, a principle that separates what to generate from where to commit it. A one-step Bayesian formulation connects maximum confidence position-first decisions to an aggregate-evidence, token-first endpoint. Building on this framework, we propose COLA (Collective-evidence Ordering and Local Acceptance), a training-free decoder that uses token-first proposals and validates placement locally. It coordinates parallel allocations from each model prediction and regulates commitment and revision across denoising steps. On DiffusionGemma, COLA modestly improves on the already high PARALLELBENCH quality of the model's default Entropy Bound (EB) decoder, while reducing model evaluations per generated token by 39.4% and latency by 33.2%. Across six downstream tasks, aggregate model evaluations per generated token fall by 73.7% relative to EB, with higher accuracy on both mathematics benchmarks and pass@1 gains of 20.7 and 10.5 percentage points on MBPP and HumanEval, respectively. These results establish content selection as a practical design axis for efficient parallel diffusion decoding.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Diffusion Language Models, Content-First Decoding, Bayesian Decision Theory, Parallel Decoding, Inference Efficiency

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pxmtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes content-before-position decoding: selecting a token using evidence across positions before deciding where to place it. A one-step Bayesian analysis motivates this approach. The resulting COLA decoder combines token-first proposals with local acceptance and is evaluated on DiffusionGemma and Nemotron-Labs-Diffusion.

Critical strengths
  1. COLA improves a competitive model using measured runtime comparisons. On DiffusionGemma, ParallelBench quality increases from 93.12% to 93.47%, while mean latency falls from 3.105 to 2.074 seconds. This is direct evidence of a useful improvement over the model’s EB decoder.

  2. Table 3 compares token-first and entropy position-first selection within the same controller. Token-first reduces model evaluations from 90.80 to 41.96 per 100 generated tokens without reducing the overall score. This supports the contribution of the proposed selection rule.

Critical weaknesses
  1. The speed improvement is not consistent across tasks or models. On DiffusionGemma’s text-generation tasks, latency increases from 1.881 to 3.968 seconds. On Nemotron, COLA requires more model evaluations and is slower in the small GSM8K and MBPP timing samples. The evidence supports a narrower benefit than general decoding acceleration.

  2. The controlled comparison uses entropy position-first selection but does not include maximum-confidence selection within the same controller. Since maximum confidence is an endpoint of the paper’s Bayesian formulation, this comparison is important for establishing the benefit of aggregating evidence across positions.

Questions for authors
  1. Does token-first selection outperform maximum-confidence selection when the rest of the controller is unchanged?

  2. What exact EB version, temperature, stopping rules, and forward limit were used? How were the EB and COLA settings selected?

  3. On a code-generation benchmark, does COLA improve quality at equal runtime over EB, with output lengths and truncation rates also reported?

2 comments

Thank you for your thoughtful review and constructive questions. We appreciate your recognition of the measured runtime improvements over DiffusionGemma’s EB decoder and the controlled evidence supporting token-first selection. Your comments help us clarify the comparison with maximum confidence, document the decoding setup, and examine code-generation quality under a shared time budget. We address these points below with controlled comparisons, experimental settings, and a fixed-budget analysis on MBPP, followed by clarification of the two weaknesses.

1. Maximum-confidence versus token-first selection under the same controller

We compare maximum-confidence and token-first proposals while keeping the remaining COLA components and settings fixed, including residual support, KL suppression, local acceptance, risk budgeting, and revision. All results below therefore use the full COLA controller, with only the proposal rule changed.

On ParallelBench, evaluated over 1,700 examples:

Proposal rule within COLAScore (%)NFE/100
Maximum-confidence92.4748.80
Token-first (aggregate evidence)93.5342.01

We also evaluate the same controlled comparison on downstream mathematics and code tasks. Each entry reports Score (%) / NFE/100.

TaskMaximum-confidenceToken-first (aggregate evidence)
GSM8K95.10 / 19.7996.50 / 8.17
MATH-50042.80 / 17.7145.20 / 6.24
HumanEval68.42 / 12.1081.58 / 6.72
MBPP58.89 / 24.7377.78 / 12.08

Token-first (aggregate evidence) yields higher Score and lower NFE on ParallelBench and all four downstream tasks, with particularly large quality gains on code generation. These comparisons directly support the contribution of aggregate-evidence ordering within the complete COLA controller.

2. EB implementation, decoding settings, and operating-point selection

We use the DiffusionGemma EB implementation bundled with Hugging Face Transformers 5.14.1 (huggingface/transformers, release tag v5.14.1). The sampler is EntropyBoundSampler in src/transformers/models/diffusion_gemma/generation_diffusion_gemma.py, configured through EntropyBoundSamplerConfig and invoked via DiffusionGemmaForBlockDiffusion.generate().

Decoding settings. EB uses an entropy bound of 0.1 and retains the official stability and low-entropy early-stopping mechanisms, with stability_threshold=1 and confidence_threshold=0.005. Its maximum number of model forwards per canvas is the official default of 48.

For COLA, we deliberately set a loose upper bound of 512 forwards per canvas; actual denoising finishes well before this limit. Reported NFE counts the model forwards actually executed. Both methods use linear temperature scaling from 0.8 to 0.4 over their respective per-canvas step budgets, restarting the schedule at each new canvas.

Both methods use the official processor and apply temperature scaling before computing probabilities. We apply no additional top-k or top-p truncation, repetition penalty, or special-token or EOS logit masking. EOS is used for normal generation termination and output truncation.

Operating-point selection. For each downstream task, COLA and EB use the same 50-example development set, disjoint from the final test set. Development sampling uses seed 20260921, and both methods’ configurations were fixed before test evaluation.

For COLA, we started from the ParallelBench settings and evaluated approximately three to five configurations per task, manually adjusting canvas length and controller parameters using the development set. For EB, we initially tested canvas lengths of 32, 64, 128, and 256. Later trials focused on 32 and 64. The other EB controller parameters retained the official defaults recommended for generation quality and latency.

The final task-specific settings are listed below. Output caps apply to both methods and are measured in generated tokens. COLA margins are listed in commitment / rollback order.

TaskOutput capCOLA canvasCOLA risk budgetCOLA KL strengthCOLA marginsEB canvas
GSM8K512640.250.501 / 132
MATH-5001,0241280.100.251 / 132
HumanEval512640.250.501 / 832
MBPP512640.100.251 / 832
IFEval1,0241280.250.501 / 864
XSum128320.100.501 / 864

COLA fixes the residual-coverage weight beta=1, the KL parameter gamma=1, and the numerical floor at 1e-6. Local acceptance and the risk budget determine the number of commitments per forward, without a fixed count cap.

3. Code-generation quality under a fixed time budget

On DiffusionGemma MBPP, COLA correctly completes more problems at every reported per-example time budget.

Comparison under the same time budget. We keep the decoding configurations from Table 2 unchanged and associate each example’s recorded generation time with its test outcome. For a given time budget, an example counts as successful when generation finishes within that budget and the generated program passes all tests. Examples that finish later or fail the tests count as unsuccessful. The denominator is always all 450 problems.

Timings use batch size 1, with one BF16 model instance sharded across three NVIDIA RTX 3090 GPUs, each with 24 GB of memory, without CPU or disk offloading. The timed interval includes prompt prefill and the complete generation process, including decoder control. Model loading, input preparation and GPU transfers, output text processing, test execution, and result writing are outside the timed interval.

Per-example time budgetEB completedEB correct within budgetCOLA completedCOLA correct within budget
2 s1110/450 (2.22%)7769/450 (15.33%)
5 s6862/450 (13.78%)298262/450 (58.22%)
10 s158143/450 (31.78%)404338/450 (75.11%)
20 s262216/450 (48.00%)442349/450 (77.56%)
30 s325242/450 (53.78%)450350/450 (77.78%)
60 s434257/450 (57.11%)450350/450 (77.78%)
All runs finished450257/450 (57.11%)450350/450 (77.78%)

At a shared 10-second budget, COLA correctly completes 195 more problems, improving the correct-completion rate by 43.33 percentage points. At 5 seconds, COLA already correctly completes 262 problems, exceeding EB’s 257 correct solutions after all 450 runs finish.

Output lengths and truncation. Both methods use a 512-token output budget. The following statistics describe the complete runs over all 450 examples. Output truncation means exhausting the output-token budget without normal termination.

MethodMean output length (tokens/example)Output truncation (%)
EB210.743.33
COLA158.425.11

These results show that, under the fixed Table 2 configurations, COLA delivers more correct programs within the same per-example generation-time budgets.

Reviewer_M1LbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes content-before-position decoding: aggregate evidence across positions to select a token, then determine and validate its placement. A one-step Bayesian formulation motivates this ordering. The resulting training-free decoder, COLA, combines token-first proposals with residual support, token-specific discounting, uncertainty budgeting, and revision. Evaluation covers 17 ParallelBench tasks and six downstream benchmarks, primarily on DiffusionGemma, with supplementary Nemotron experiments. Reported improvements include 33.2% lower ParallelBench latency and substantial code-generation gains; benefits vary across tasks and backbones.

Critical strengths

A useful and distinct decoding principle. Selecting vocabulary identities through aggregate support differs meaningfully from refining position-level commitment scores, as in Mean-Field Parallel Decoding. Table 3 provides encouraging evidence: within the COLA controller, token-first proposals reduce NFE/100 from 90.80 to 41.96 relative to the tested entropy-based alternative, while slightly improving quality.

Consequential practical gains on DiffusionGemma. COLA improves the quality–cost trade-off against the model’s official EB decoder. Particularly convincing are the measured ParallelBench latency reduction and the 20.7-point MBPP improvement, whose paired confidence interval excludes zero. The code diagnostics clarify that much of the benefit comes from avoiding malformed programs—a practically valuable improvement.

Critical weaknesses
  • The central contribution is only partially isolated. The theory’s position-first endpoint is maximum-confidence selection, but Table 3 compares against entropy computed from normalized, discounted scores. This leaves uncertainty about whether the advantage persists against a simpler maximum-confidence rule under the same controller. Moreover, every downstream ablation retains token-first proposals, and removals are cumulative. Consequently, the strongest downstream gains cannot be specifically attributed to content-first ordering, and individual component effects remain entangled.

  • The efficiency advantage has a restricted demonstrated scope. ParallelBench text-generation latency increases from 1.881 to 3.968 seconds; overall savings are driven mainly by waiting-line tasks (§7.3). On Nemotron, downstream NFE/100 increases from 56.1 to 60.4 with mostly small quality changes (Table 13). The main downstream comparison reports aggregate token-normalized NFE without corresponding per-task wall-clock measurements. These findings limit confidence in a broadly transferable efficiency improvement.

Questions for authors
  • With the remaining controller fixed, how do maximum-confidence and raw-entropy proposals compare with token-first proposals on the downstream mathematics and code tasks? This would directly clarify the contribution responsible for the headline gains.

  • How were COLA and baseline operating points selected, and were tuning budgets comparable? Please specify the downstream canvas lengths, forward caps, logit processing, and development/test partition procedure; these are not fully recoverable from Appendix I.

  • Can you provide per-task downstream latency, forwards per example, output lengths, and truncation rates for EB and COLA? These would clarify whether the aggregate NFE reduction translates into practical speedups and whether truncation contributes to the code-error differences.

4 comments

Thank you for your thoughtful review and constructive suggestions. We appreciate your recognition of the content-before-position principle, COLA's practical gains on DiffusionGemma, and the value of the code-generation diagnostics. Your questions help us clarify the contribution of proposal ordering, the selection of operating points, and the relationship between model evaluations and generation latency. We address these points with controlled comparisons, detailed experimental settings, and per-task efficiency and truncation analyses, followed by clarification of the two weaknesses.

1. Proposal ordering on downstream tasks

We vary only proposal ordering within COLA, keeping residual support, KL suppression, local acceptance, risk budgeting, revision, and all other settings fixed. Raw entropy is computed from the original model probabilities. Results are Score (%) / NFE/100:

TaskToken-firstRaw entropy (Position-first)Maximum-confidence (Position-first)
GSM8K96.50 / 8.1794.40 / 28.2295.10 / 19.79
MATH-50045.20 / 6.2443.60 / 36.1142.80 / 17.71
HumanEval81.58 / 6.7271.93 / 27.5368.42 / 12.10
MBPP77.78 / 12.0864.00 / 42.4358.89 / 24.73

Token-first yields higher Score and lower NFE on all four tasks, with the largest quality gains on code generation. These comparisons directly test both position-first alternatives under the same controller and show that the advantage persists against maximum-confidence and raw entropy. They support proposal ordering as a key contributor to the downstream quality-cost improvements within COLA, particularly on code tasks.

2. Operating-point selection and evaluation protocol

Both methods use the same 50-example development set per task, disjoint from the test set, with configurations fixed before test evaluation. Starting from the ParallelBench settings, we evaluated approximately three to five COLA configurations per task, manually adjusting canvas length and controller parameters using the development set. For EB, we initially tested canvas lengths of 32, 64, 128, and 256. Later trials focused on 32 and 64. EB retained the official DiffusionGemma controller defaults recommended for generation quality and latency.

Output caps are shared and measured in tokens. COLA margins are commitment / rollback thresholds.

TaskOutput capCOLA canvasCOLA risk budgetCOLA KL strengthCOLA marginsEB canvas
GSM8K512640.250.501 / 132
MATH-5001,0241280.100.251 / 132
HumanEval512640.250.501 / 832
MBPP512640.100.251 / 832
IFEval1,0241280.250.501 / 864
XSum128320.100.501 / 864

Forward caps and logit processing. EB uses the official maximum of 48 model forwards per canvas. COLA uses a loose upper bound of 512 forwards per canvas; actual denoising finishes well before this limit. NFE counts the model forwards actually executed. Both methods use linear temperature scaling from 0.8 to 0.4 over their respective per-canvas step budgets, restarting the schedule at each new canvas.

EB uses an entropy bound of 0.1 and retains the official stability and low-entropy early-stopping mechanisms, with stability_threshold=1 and confidence_threshold=0.005. COLA fixes residual-coverage weight beta=1, KL parameter gamma=1, and floor 1e-6. Local acceptance and the risk budget determine commitments per forward, without a fixed count cap.

Both use the official processor and apply temperature scaling before computing probabilities, with no additional top-k or top-p truncation, repetition penalty, or special-token or EOS logit masking. EOS is used for normal termination and output truncation.

Development/test partition. All development-set sampling used seed 20260921. The partition procedures are listed below.

TaskDevelopment-set selectionFinal test size
GSM8KShuffle the train split of openai/gsm8k, exclude prompts duplicated in the test set, and select 50 examples.1,000
MATH-500Shuffle the train split of DigitalLearningGmbH/MATH-lighteval, exclude prompts duplicated in the test set, and select 50 examples.500
HumanEvalRandomly reserve 50 examples from the original HumanEval test set for development; evaluate on the remaining examples.114
MBPPRandomly reserve 50 examples from MBPP full/test for development; evaluate on the remaining examples.450
IFEvalRandomly reserve 50 examples from the original input_data for development; evaluate on the remaining examples.491
XSumFrom the XSum test split, exclude the frozen 1,000 test examples, filter out empty summaries and articles over 6,000 characters, shuffle, and select 50 examples.1,000

3. Per-task latency, generation statistics, and truncation analysis

We measure generation latency at batch size 1, processing one example at a time. DiffusionGemma runs in BF16 as a single model instance sharded across three NVIDIA RTX 3090 GPUs, each with 24 GB of memory, without CPU or disk offloading.

The timed interval includes model-side prompt encoding/prefill and the complete subsequent generation process, including canvas generation, denoising, and decoder control. Model loading and input preparation, including prompt formatting, tokenization, and transfers to the GPUs, are outside the timed interval. Output text processing, task evaluation, and result writing are also excluded.

Per-task efficiency and output statistics. Each pair is reported in EB / COLA order. Generation latency, forwards, and output length are means per example. The truncation rate is the percentage of examples that exhaust the output-token budget without normal termination.

TaskExamplesGeneration latency (s/example)Forwards/exampleOutput tokens/exampleTruncation (%)
GSM8K1,00023.73 / 6.61106.65 / 22.50277.44 / 275.341.20 / 0.60
MATH-50050046.36 / 8.35225.84 / 27.83443.91 / 445.654.80 / 4.60
HumanEval11420.14 / 4.7489.34 / 15.75260.22 / 234.392.63 / 2.63
MBPP45021.68 / 5.12104.07 / 19.14210.74 / 158.423.33 / 5.11
IFEval49113.47 / 11.5456.55 / 43.46255.87 / 261.481.63 / 1.43
XSum1,0002.25 / 2.778.79 / 11.4932.11 / 31.320.00 / 0.00

COLA achieves lower measured generation latency on five of the six tasks, with approximately 3.6–5.6× speedups on mathematics and code generation. IFEval shows a smaller improvement, while XSum generation latency increases from 2.25 to 2.77 seconds per example.

On GSM8K and MATH-500, the two methods generate outputs of very similar average length, while COLA substantially reduces both model evaluations and generation latency. These results demonstrate improved decoding efficiency at comparable output lengths. On MBPP, COLA produces shorter outputs, and its length-normalized NFE/100 also decreases from 49.38 to 12.08.

Code quality and truncation. HumanEval has the same truncation rate of 2.63% for both methods. On MBPP, the rates are 3.33% for EB and 5.11% for COLA. To examine code quality among completed outputs, we compare the full MBPP test set with the paired subset where neither method’s output is truncated.

Evaluation subsetExamplesEB pass@1 (%)COLA pass@1 (%)Gain (percentage points)Syntax/indentation errors (EB / COLA)
Full test set45057.1177.78+20.67139 / 22
Neither output truncated42160.8181.47+20.67116 / 14

Gains are calculated from unrounded scores. On the 421 paired examples where neither output is truncated, EB passes 256 examples and COLA passes 343, yielding a 20.67-percentage-point advantage. Syntax or indentation errors decrease from 116 to 14. Thus, COLA’s advantage in functional correctness and syntactic validity persists among outputs that terminate without truncation.

4. Clarifications on contribution isolation and efficiency scope

Contribution isolation. The fixed-controller comparisons assess proposal ordering while holding the other COLA components constant. The cumulative ablations evaluate progressively simplified decoder configurations, with each removal assessed in the context of the components retained at that stage.

Efficiency scope. COLA’s efficiency varies across tasks and decoding interfaces. The new measurements establish practical acceleration on DiffusionGemma’s mathematics and code tasks. ParallelBench text generation and XSum incur higher latency, while the Nemotron configuration provides modest quality gains with more model evaluations.

Thank you for the additional experiments. The controlled proposal-order comparisons substantially address my attribution concern. The latency measurements and truncation analysis also strengthen the evidence for practical gains on DiffusionGemma. Two clarifications remain:

  1. Temperature annealing spans 48 forwards for EB versus 512 for COLA, so the cap affects predictions even without exhaustion. Do the gains persist with the same temperature schedule?
  2. Please reconcile the MATH-500 output limit (512 tokens in the paper versus 1,024 in the response) and XSum development-set size (10 versus 50), confirming which settings produced the original results.

I maintain 3 — Weak acceptance, with increased confidence. Please incorporate the new evidence and corrected settings while retaining the qualifications regarding XSum and Nemotron.

Thank you for the careful follow-up. We appreciate your comments on the proposal-order comparisons, latency measurements, and truncation analysis. Your observation that the annealing horizon affects predictions even when a canvas finishes early prompted us to test COLA with EB’s schedule. This comparison helped us identify a better default setting for COLA on DiffusionGemma.

1. Matching the temperature schedule

We set both COLA’s per-canvas forward cap and its annealing horizon to 48, matching EB. Both decoders use the same temperature at each denoising step. The temperature decreases linearly from 0.8 to 0.4 over 48 forwards and resets to 0.8 at each new canvas.

Each cell reports Score ↑ / NFE/100 ↓, using the same task metrics as Table 2.

TaskEB 48COLA 48Original COLA 512
GSM8K94.90 / 38.4496.50 / 8.0796.50 / 8.17
MATH-50042.60 / 50.8845.00 / 6.0345.20 / 6.24
HumanEval71.05 / 34.3384.21 / 6.5381.58 / 6.72
MBPP57.11 / 49.3878.67 / 11.5677.78 / 12.08
IFEval89.71 / 22.1089.19 / 14.9987.76 / 16.62
XSum88.44 / 27.3788.48 / 34.5788.49 / 36.70
Macro average73.97 / 37.0880.34 / 13.6379.55 / 14.42

Moving from COLA 512 to COLA 48 raises the macro-average score from 79.55 to 80.34 and lowers NFE/100 on every task. Macro-average NFE/100 falls by 5.5%, from 14.42 to 13.63. Using Table 2’s pooled aggregation, NFE/100 falls by 5.0%, from 10.54 to 10.01. All 3,555 examples converge at 48 steps. With the schedule matched, COLA also achieves higher scores and lower NFE/100 than EB on all four mathematics and code tasks.

We also varied COLA’s per-canvas forward cap and annealing horizon together to examine how this setting affects quality, cost, and convergence.

COLA cap and annealing horizon (steps)Macro-average score ↑Macro-average NFE/100 ↓Nonconverged examples (of 3,555)
1274.6511.86437
2479.5513.0937
4880.3413.630
9679.6913.970
19279.7014.170
51279.5514.420

We count an example as nonconverged if any of its canvases reaches the forward cap before completion. These examples remain in the scoring denominator. Output-length truncation refers to exhausting the separate output-token budget before normal termination.

Across these six DiffusionGemma tasks, 48 steps gives the highest macro-average score among all tested settings and the lowest macro-average NFE/100 among those that converge on every example. We are especially grateful for your suggestion, which led us to a setting that improves both quality and computational efficiency over our original configuration. We will use 48 steps as the default per-canvas forward cap and annealing horizon for COLA’s DiffusionGemma adapter in the open-source implementation.

On Nemotron, both COLA and the native Threshold baseline use a temperature that is constant across denoising steps. The cap determines when iteration stops. Across all 3,555 examples in the original COLA evaluation, the maximum observed count was 31 denoising iterations per canvas. Caps of 48, 96, or 192 would therefore yield the same outputs and forward counts as the original cap of 512 on these examples. On this backbone, COLA’s modest quality gains over the native Threshold baseline come with higher pooled NFE/100 (60.4 vs. 56.1).

2. Clarifying the settings used for the original results

We used the 512-token limit and the 10-example development set in earlier experiments and inadvertently retained those values in the manuscript. We apologize for the confusion. The results reported in Table 2 used a 1,024-token output limit for MATH-500 and 50 development examples for XSum. The XSum development examples were disjoint from the frozen 1,000-example test set. We will correct both descriptions in the revised paper.

Reviewer_LFuROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes content-before-position decoding for diffusion language models, motivated by the observation that evidence for a token may be distributed across multiple candidate positions even when no individual position has sufficiently high confidence. The authors introduce a one-step Bayesian decision framework that separates token identity from placement, connecting conventional position-first decoding to a token-first strategy based on aggregate evidence. Building on this formulation, the paper develops COLA, a training-free decoder that combines token-first proposals, local acceptance, KL-based suppression, residual support, uncertainty budgeting, and commitment revision. Experiments primarily on DiffusionGemma demonstrate substantial reductions in model evaluations and improved performance on mathematical reasoning and code generation. Additional analyses examine decision ordering, decoding efficiency, code-generation failures, and the contributions of different algorithmic components.

Critical strengths
  1. The paper identifies a potentially useful limitation of conventional position-first decoding: local confidence does not necessarily capture evidence for a token distributed across multiple candidate positions. The proposed separation between content selection and placement provides an interesting perspective on diffusion language model decoding.

  2. The approach does not require retraining the underlying model and can be integrated with existing diffusion decoding infrastructure.

  3. The experiments on DiffusionGemma demonstrate meaningful improvements in inference efficiency while largely preserving or improving generation quality.

  4. The paper includes multiple decoding baselines, controlled decision-order comparisons, component ablations, paired evaluations, and detailed error analyses.

Critical weaknesses
  1. The principal conceptual contribution is the content-before-position strategy. However, COLA combines this strategy with several additional mechanisms, including residual coverage, KL-based suppression, uncertainty budgeting, and commitment revision.

  2. The proposed Bayesian analysis is based on an auxiliary one-step loss with separately specified token-identity and placement costs. However, the loss and its cost ratio are modeling assumptions, and the resulting Bayes action does not directly characterize the complete multi-step decoding process.

  3. COLA requires a proposed token to be the raw local top-one prediction at its selected position before commitment. Consequently, the proposed method primarily changes the ordering of locally supported commitments rather than performing non-greedy token assignment.

  4. The primary experiments show strong improvements on DiffusionGemma. However, the supplementary Nemotron evaluation requires a substantially modified decoding procedure and does not reproduce the efficiency gains.

Questions for authors
  1. Can the authors provide a more controlled comparison that isolates aggregate token evidence from residual coverage, KL-based suppression, and commitment control? In particular, how much improvement remains when only the initial proposal ordering is changed?

  2. How should the auxiliary token-identity and placement costs be interpreted in relation to actual generation quality?

  3. How frequently does the highest-ranked token-first proposal fail local acceptance? When this occurs, why does the decoder terminate packing instead of considering the next candidate?

  4. Why does COLA achieve substantial efficiency improvements on DiffusionGemma but require more model evaluations and higher latency on Nemotron?

3 comments

Thank you for the thoughtful review and detailed questions. We appreciate your recognition of the content-before-position perspective and the practical efficiency gains on DiffusionGemma. Your comments help us clarify the contribution of proposal ordering, the role of the Bayesian formulation, and the decoder's behavior across settings. We provide the controlled comparisons and execution diagnostics below, followed by further clarification of the theoretical analysis and local acceptance.

1. Isolating the contribution of aggregate token evidence

We compare maximum-confidence, raw-entropy, and aggregate-evidence proposal rules on all 1,700 ParallelBench examples using DiffusionGemma under two settings.

First, we compare the three ordering rules in a simplified serial decoding setting. All other COLA components are removed. Each model forward commits exactly one token, and predictions are refreshed before the next commitment. Only the ordering rule differs across the three variants.

Proposal ruleCorrectScore (%) ↑NFE/100 ↓
Position-first (maximum confidence)1,57992.88228.80
Position-first (raw entropy)1,58193.00218.95
Token-first (aggregate evidence)1,58393.12221.07

Token-first (aggregate evidence) yields four and two additional correct examples relative to maximum confidence and raw entropy, respectively, corresponding to gains of 0.24 and 0.12 percentage points. The serial controls show small score differences and comparable NFE/100.

Second, we replace only the proposal rule within the full COLA decoder, keeping the remaining components and their parameters fixed, including residual coverage, KL-based suppression, local acceptance, uncertainty budgeting, and commitment revision. The raw-entropy alternative computes entropy directly from the model probabilities.

Proposal rule within COLACorrectScore (%) ↑NFE/100 ↓
Position-first (maximum confidence)1,57292.4748.80
Position-first (raw entropy)1,58293.0667.95
Token-first (aggregate evidence)1,59093.5342.01

With the surrounding controller fixed, Token-first (aggregate evidence) improves score by 1.06 percentage points over maximum confidence and 0.47 points over raw entropy, while reducing NFE/100 by 13.9% and 38.2%, respectively. The larger quality–cost benefit appears when aggregate evidence guides parallel commitment within COLA.

Token-first provides an alternative way to organize parallel decoding by selecting content through aggregate evidence before determining its placement. These results support content-first proposal selection as a useful design choice whose benefits emerge through its integration with allocation coordination and commitment control.

2. Interpreting the auxiliary costs in relation to generation quality

The token-identity and placement costs encode relative priorities in an auxiliary one-step decision problem. The auxiliary target is sampled from a distribution constructed from the current model predictions. The token-identity cost penalizes a mismatch with the auxiliary target token; the placement cost penalizes a position mismatch conditional on the token matching.

This preference has a precise effect on proposal ordering. Under a uniform location prior, the cost ratio ρ=ctok/cpos\rho=c_{\mathrm{tok}}/c_{\mathrm{pos}} induces the score

Gρ(v)=(ρ−1)A(v)+P(v),G_{\rho}(v)=(\rho-1)A(v)+P(v),

where A(v)A(v) is the aggregate probability across eligible positions and P(v)P(v) is the highest probability at any eligible position. At ρ=1\rho=1, this recovers maximum-confidence selection. Increasing ρ\rho places greater weight on aggregate evidence, and COLA adopts the identity-priority endpoint as ρ→∞\rho\to\infty. The Bayes action optimizes the specified auxiliary one-step decision.

These costs influence generation through the proposal policy. Aggregate evidence prioritizes candidate tokens, raw local probabilities and margin checks assess placements, uncertainty budgeting regulates additional commitments, and revision reconsiders earlier commitments as context changes. Task evaluations and controlled comparisons assess the quality of the resulting multi-step generation.

3. Frequency of local-acceptance failures and the stopping rule

We measured the frequency of local-acceptance failures and evaluated a bounded-search variant that considers additional candidates.

At each packing iteration, local acceptance requires the highest-ranked proposal to satisfy both the raw local top-one condition and the ratio-margin threshold. Across the six downstream tasks, these checks terminate packing 20.41–33.68 times per 100 model forwards. The breakdown below reports termination events per 100 forwards, with risk-budget stops listed separately.

TaskLocal top-one conflictMargin failureRisk-budget stop
GSM8K8.3018.1150.81
MATH3.8416.5765.71
HumanEval11.5317.0440.53
MBPP5.4618.4159.06
IFEval13.4520.2359.39
XSum7.3914.1764.88

A termination event can occur after several commitments have already been accepted within the same forward. Across tasks, the mean number of accepted token commitments per forward ranges from 4.17 to 17.20.

COLA treats the first local-acceptance failure as a signal to refresh predictions. It retains the accepted commitments, performs the next model evaluation on the updated canvas, and recomputes proposal priorities. This greedy policy gives priority to obtaining predictions conditioned on the updated context before continuing allocation.

Following your suggestion to consider the next candidate before terminating packing, we tested whether continuing candidate search could improve the quality–computation trade-off. We implemented a bounded-search variant, termed Next4: when the highest-ranked proposal fails the raw local top-one check, the decoder examines up to four subsequent candidates with distinct token identities using the same model predictions. The original margin and risk-budget checks are retained. We compared Next4 with full COLA across all six downstream tasks to measure the effect of continuing the search at this stopping point.

TaskFull COLA Score ↑Next4 Score ↑Full COLA NFE/100 ↓Next4 NFE/100 ↓
GSM8K96.5096.408.177.68
MATH45.2045.006.246.18
HumanEval81.5879.826.726.08
MBPP77.7877.7812.0811.52
IFEval87.7687.2416.6214.99
XSum88.4988.4936.7034.04
Unweighted task mean79.5579.1214.4213.42

Next4 reduces NFE/100 on all six tasks, with an approximately 7.0% reduction in the unweighted task mean. Scores remain unchanged at the reported precision on MBPP and XSum and decrease on the other four tasks. The descriptive mean score decreases by 0.43 points. HumanEval shows the largest decrease, from 81.58% to 79.82%, a difference of 1.76 percentage points.

These results provide two task-dependent operating points. Immediate stopping preserves higher observed quality on four tasks, while Next4 offers fewer model evaluations across all six tasks, including matched reported quality on MBPP and XSum. A configurable candidate-search budget could expose this quality–computation trade-off to users.

4. Different efficiency outcomes on DiffusionGemma and Nemotron

COLA applies content-first proposals to both backbones through their respective decoding interfaces. DiffusionGemma uses uniform-corruption diffusion with prediction refresh and commitment revision. Nemotron uses masked-block decoding with permanent commitments, conservative local and placement checks, high-confidence companions, and bounded fallback. The reference decoders are Entropy Bound for DiffusionGemma and confidence-threshold decoding for Nemotron.

On DiffusionGemma, all new canvas commitments pass through the token-first path. Aggregate evidence organizes multiple allocations from a shared model prediction, and local acceptance and revision regulate their reliability. On ParallelBench, COLA reduces NFE/100 by 39.4% and mean latency by 33.2% relative to EB.

On Nemotron, token-first packing initiates each step and accounts for 10.9%–17.1% of commitments across the six downstream tasks (Table 14, excluding autoregressive boundary seeds). Companions and fallback supply the remaining commitments. These execution paths show that the two implementations use aggregate proposals to different extents. In the Nemotron adaptation, conservative local and placement checks together with bounded fallback favor high-confidence commitments, deferring some allocations to later model evaluations. The increased forward counts and profiling results are consistent with this quality–computation trade-off.

The additional forward computation is visible in the profiling study of ten GSM8K and ten MBPP examples. Mean forwards increase from 101.5 to 117.0 and from 200.1 to 222.3, respectively, with latency increasing from 3.77 to 4.40 seconds and from 7.40 to 8.35 seconds. Model forwards account for approximately 94% of COLA's runtime, while KL updates account for 0.28%–0.33% (Table 15). These measurements identify model forward computation as the dominant cost.

This adaptation achieves a different quality–computation trade-off. On Nemotron ParallelBench, score increases from 85.59% to 87.35%, with NFE/100 increasing from 79.3 to 88.1. Across the six downstream tasks, quality is maintained or slightly improved and aggregate NFE/100 increases from 56.1 to 60.4. The two implementations demonstrate how the content-first principle can be used under different diffusion interfaces, with the measured efficiency depending on the backbone and commitment configuration.

5. Scope of the Bayesian analysis and the role of local acceptance

The Bayesian analysis formalizes proposal selection at a fixed denoising state. The auxiliary loss and cost ratio make the relative priority of token identity and placement explicit, yielding the score family discussed in Response 2. Its mathematical contribution is a unified characterization of proposal ordering. COLA implements the identity-priority endpoint through allocation coordination and commitment control across decoding steps.

The raw local top-one condition validates a proposed placement. Aggregate evidence first prioritizes a token identity, after which the decoder selects and validates its candidate position. A token with support distributed across positions can take priority over a token with a larger individual probability peak. Each accepted allocation updates residual support and selected-token discounts, changing subsequent priorities and the composition of the parallel pack. These commitments then shape the context used by the next model evaluation. The algorithm therefore uses global content evidence to order and coordinate locally supported commitments.

In the full DiffusionGemma ParallelBench traces, we compare each accepted token with the token of the globally highest-probability eligible position–token pair. The disagreement rate averages 50.69% across examples (Section 7.1 and Appendix D). These traces show that aggregate ordering frequently changes actual content selection. The controlled comparisons in Response 1 assess the resulting quality and computational cost under the same controller.

Comments on the paper

1 comment

Dear Area Chairs, Senior Area Chairs, and Program Chairs,

We sincerely thank all three reviewers for their careful reading and constructive suggestions. We especially appreciate the positive feedback on COLA’s content-before-position approach and its measured efficiency gains on DiffusionGemma.

COLA is a training-free decoder that pools evidence across positions to propose a token, then validates its placement locally. This allows several commitments from a single model prediction. Compared with DiffusionGemma’s default decoder, COLA reduces mean ParallelBench latency by 33.2% with a modest quality gain and improves MBPP pass@1 by 20.7 percentage points.

During the rebuttal, we compared token-first, maximum-confidence, and raw-entropy proposals with the rest of COLA fixed. Token-first achieved the highest score and used the fewest model evaluations per 100 generated tokens on each of the four mathematics and code benchmarks. We also compared COLA and the default decoder under a common 48-step temperature schedule, reported per-task latency, and evaluated how many MBPP problems each decoder solved within the same time budget.

We have uploaded the revised manuscript, with the new material marked in blue. We hope the additional evidence is helpful in your assessment of the paper.

Best regards,
The COLA Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.