acceptodds
Under review as a conference paper at ICLR 2027

CT-OPD: Counterfactual Trace On-Policy Distillation for Diffusion Vision-Language Models

Abstract

Diffusion vision-language models generate answers by gradually resolving masked tokens, making accurate conditional prediction in partially resolved states central to post-training. Masking completed answers yields coherent contexts and targets, but prescribed masks do not reflect the model's reveal decisions. Its trajectories capture these decisions, yet their provisional visible tokens can conflict with the target response. Outcome-based reinforcement learning follows these trajectories but provides only response-level feedback, which loses contrast when sampled rewards tie. To align coherent token-level supervision with the model's reveal decisions, we introduce Counterfactual Trace On-Policy Distillation (CT-OPD), which combines completed teacher responses with trajectory masks from the current student. CT-OPD retokenizes each teacher response in the student's vocabulary and extracts unresolved-position masks at successive stages of the student's reverse process. For each mask, it discards provisional rollout values and reconstructs the partial state from the teacher endpoint, so the supervised positions follow the current trajectory while the visible context and targets remain consistent with the same response. The student is trained on these reconstructed states with its native categorical loss, and trajectories are refreshed as the model evolves. Across dense and sparse diffusion architectures, CT-OPD consistently enhances multimodal understanding and reasoning capabilities, with gains of up to 9.80 points on the nine-benchmark average. On the unified understanding-and-generation architecture, it also improves both visual understanding and image generation, showing that the same principle transfers across architectures and modalities. Ablations further attribute these gains to coherent reconstruction and current-model trajectory masks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On-Policy Distillation, Diffusion Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LFGTOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes CT-OPD, a distillation method for diffusion vision-language models that applies masks from the current student’s generation trajectories to teacher responses, using teacher tokens to provide consistent visible context and prediction targets at masked positions.

Critical strengths
  1. Broad empirical validation. The evaluation covers dense, sparse, and unified diffusion VLMs across multimodal understanding, reasoning, and image generation. CT-OPD improves average understanding performance over the matched random-mask baseline on all evaluated backbones and improves all three image-generation metrics on MMaDA, supporting its applicability across architectures and output modalities.
  2. Carefully controlled and informative ablations. The random-mask baseline shares teacher responses and matches stage-wise mask counts and update budgets, helping distinguish the method’s contribution from stronger teacher supervision alone. Additional controls examine teacher-consistent reconstruction, online trajectory refresh, and prompt–mask matching, providing targeted empirical support for the individual design choices rather than relying solely on aggregate benchmark improvements.
Critical weaknesses

Weaknesses

  1. Unclear novelty over adaptive masking. PUMA [1] and LIFT [2] already combine model-dependent position selection with reference-token supervision. CT-OPD transfers masks from student trajectories to teacher answers, but it is unclear why this is better than selecting masks directly on teacher answers. Comparisons with these methods under the same supervision and compute budget are needed.

  2. Limited evidence for the proposed mechanism. The diagnostic uses training prompts and teacher-reconstructed contexts. Although the benchmark results show improvements, this diagnostic does not directly test whether the method improves continuation from the student's own potentially incorrect states. Evaluating this on held-out prompts would provide stronger evidence.

  3. Unclear value of the extra computation. The Random baseline also runs student rollouts to match mask counts, although random masking can be implemented without full rollouts. Therefore, this comparison does not show whether online trajectory collection is worth its cost. Please compare performance under the same total compute budget, including rollout costs, against rollout-free masking and the frozen-trace variant.

  4. The counterfactual framing needs clarification. The method replaces student-generated visible tokens with teacher tokens while keeping the original masks. However, changing these tokens may also change which positions the student would reveal next. Please distinguish this fixed-mask reconstruction from a trajectory the student would generate under the modified context, and explain what the counterfactual interpretation adds beyond the reconstruction procedure.

References

[1] Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training. arXiv:2602.10314, 2026.

[2] Learnability-Informed Fine-Tuning of Diffusion Language Models. arXiv:2605.22939, 2026.

Questions for authors

Please address the questions and concerns raised in the Weaknesses section.

6 comments
AuthorsOpenReview ↗

Response to LFGT (1/2): adaptive masking and complete method results

Thank you for highlighting PUMA and LIFT as closely related works. We have cited and discussed both methods in the revised paper.

CT transfers free-generation reveal decisions into coherent supervision.(Weakness 1) PUMA progressively reveals reference tokens using a model-dependent policy. LIFT selects supervision using reference-token confidence and diffusion time. CT first collects masks from the student's free trajectory, then reconstructs their visible context and targets from a teacher endpoint. Retokenized teacher responses can provide reference data to all three methods, CT separates free-running position selection from endpoint-based state construction. Provisional tokens influence the history encoded in the collected masks.

Complete implementations and training runs.(Weakness 1) We implemented PUMA's progressive transitions and schedule and LIFT's two-pass selection and loss on LLaDA-V. The runs share teacher endpoints, initialization, global batch 16, and 11,140 updates. Checks against released implementations cover 360 PUMA transition cases and 90 LIFT mask, loss, and gradient cases.

MethodMMMUMMMU-ProMMStarMMBenchSeedBenchMathVerseMathVistaGPQAGPQA-DAverage
Native SFT39.1131.2757.4785.3674.6720.6949.2034.1534.3447.36
PUMA38.4430.5857.4084.5174.8521.1948.2032.3733.8446.82
LIFT40.0030.8157.2784.7274.6221.0748.2031.9230.3046.54
CT-OPD46.2233.1255.8783.1674.2627.7961.3032.5933.8449.79

Direct selection under teacher tokens.(Weakness 1) The LaViDa teacher-conditioned control selects positions under visible reference tokens while matching the current free trajectory's active counts at every reverse transition. A fixed-position control reveals a left-to-right reference prefix.

MethodMMMUMMMU-ProMMStarMMBenchSeedBenchMathVerseMathVistaGPQAGPQA-DAverage
Teacher-context42.0029.7745.9375.5366.4618.2729.0029.9127.7840.52
Fixed-position41.6729.3145.5375.8366.4917.5130.7030.5826.7740.49
CT-OPD43.6729.8347.0075.3766.5318.1547.6030.1327.7842.90

These studies retain each method's masking rule and objective under the shared update budget. They specify the practical alternatives to the free-trajectory collection and subsequent reconstruction used by CT.

AuthorsOpenReview ↗

Response to LFGT (2/2): held-out states, cost, and counterfactual meaning

Held-out prediction.(Weakness 2) We evaluated 512 prompts excluded from post-training, checking overlap by image and question identities. A frozen Base supplies common states. Reconstructed-state NLL is 3.2781 for CT, 3.4146 for Random, 3.3327 for Endpoint-only, and 4.4733 for Direct Trace-Target. CT−Random is −0.1366 [−0.1699, −0.1024], establishing that the reconstructed-state effect extends beyond training prompts.

We measured full-run training cost, including online trajectory collection.(Weakness 3) The table combines four-GPU update times with full-run GPU-hours per 1,000 scheduled training prompts:

Methodseconds/updateFull-run GPU-h/1k prompts
Native SFT16.323.56
CT-OPD30.188.54
MDPO88.9823.67
MaskGRPO58.8817.54

Four-GPU timings use A100 PCIe 80GB GPUs, BF16, and ZeRO-2, including trajectory collection and optimization. SFT/CT updates contain 16 supervised states, RL updates contain four prompts with four sampled responses each. CT timing averages complete four-update cycles. Means cover 36 SFT updates, 13 CT cycles, and 20 updates per RL method after warm-up. These short-run timings and the full-run allocation costs measure different scopes.

Costs include the complete training allocation, model initialization, checkpointing, and final export. CT reuses each trajectory for four updates, while the RL recipes sample four completions per prompt.

CT-OPD requires 8.54 GPU-hours per 1,000 prompts in the completed run, compared with 23.67 for MDPO and 17.54 for MaskGRPO. Its additional cost over SFT purchases current-model trajectories whose masks support four supervised updates. This reuse amortizes trajectory collection across the four training stages. Appendix E.6 includes the same cost table and measurement boundaries.

The counterfactual intervention fixes positions and changes visible content.(Weakness 4) Let Z be the student's free trajectory, M_t its unresolved mask, and YTY^T the teacher endpoint:

Mt=Gt(Z),Xt=C(YT,Mt).M_t=G_t(Z),\qquad X_t=C(Y^T,M_t).

The question is what the student predicts at these same unresolved positions when visible tokens come from one coherent teacher response. The intervention happens after mask collection. Re-executing the reveal policy under replaced tokens would select another mask and answer a different question; the teacher-conditioned experiment tests that alternative selection context. On-policy collection supplies the masks, and reconstruction supplies the learning inputs. Section 3.2 makes this sequence explicit.

Thank you for the additional experiments and clarification. I understand that the masks reflect the student's generation history. Once the student-generated visible tokens are replaced with teacher tokens, why are these masks still more useful for training than masks selected directly under teacher-provided context?

Reviewer_fX3QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CT-OPD, a post-training method for diffusion vision-language models. The main idea is to combine student-generated diffusion trajectories with teacher-generated responses: the student trajectory determines which positions remain unresolved and should be supervised, while the teacher endpoint provides coherent visible context and prediction targets. The method aims to address the mismatch between provisional rollout tokens and teacher targets, and is evaluated on several diffusion VLM architectures with improvements across understanding and generation benchmarks.

Critical strengths
  1. The paper identifies an important challenge in diffusion VLM post-training, that is, constructing training states that match the model’s actual reverse diffusion process. It clearly highlights the trade-off between existing approaches, where SFT provides consistent context-target pairs but relies on predefined masks, while trajectory-based methods better capture model-specific decoding behaviors but may introduce inconsistent visible tokens.

  2. CT-OPD separates the unresolved-position selection from response reconstruction. The student trajectory provides the mask pattern, while the teacher endpoint supplies the visible context and prediction targets.

  3. The method is evaluated on diverse diffusion VLM architectures, including dense, sparse, and unified models, with ablations covering different training strategies. The consistent improvements across settings demonstrate the general applicability of the method.

Critical weaknesses
  1. CT-OPD relies heavily on teacher endpoints, but its robustness to different teacher qualities is unclear. Experiments with weaker teachers or noisy supervision would strengthen the analysis.

  2. The actual computational overhead of CT-OPD compared with standard SFT remains unclear despite trace reuse and matched budgets. Reporting training efficiency would strengthen the analysis.

3, It remains unclear how much of the improvement comes from trajectory-aware supervision versus adaptive mask selection, since visible rollout tokens are discarded and only the unresolved-position masks are retained.

Questions for authors

Please see Weakness.

3 comments
AuthorsOpenReview ↗

Response to fX3Q (1/2): smaller teacher and adaptive selection

Thank you for highlighting teacher dependence and the role of trajectory information.

Smaller teacher.(Weakness 1) We replaced Qwen3-VL-32B-Instruct with Qwen3-VL-8B-Instruct and regenerated teacher endpoints for the same 44,547 prompts, keeping the LaViDa-LLaDA student, answer-anchored endpoint construction, and training budget unchanged.

MethodMMMUMMMU-ProMMStarMMBenchSeedBenchMathVerseMathVistaGPQAGPQA-DAverage
CT / 32B43.6729.8347.0075.3766.5318.1547.6030.1327.7842.90
CT / 8B43.2229.9445.4775.6066.6519.4245.5030.1328.7942.75

The 8B teacher achieves a nine-task average of 42.75, only 0.15 points below the 32B teacher (42.90). This indicates that CT-OPD remains effective with a substantially smaller teacher, rather than relying exclusively on the 32B model.

Trajectory information is preserved through student-selected masks.(Weakness 3) Although CT-OPD discards provisional token values during reconstruction, these tokens influence subsequent predictions, confidence estimates, and reveal decisions during the student's free-running trajectory. Their effects are therefore reflected in the selected unresolved positions. In contrast, teacher-conditioned selection determines reveal positions under visible teacher tokens, rather than the student's freely generated context.

We evaluated this distinction using teacher-conditioned and fixed-position controls, both matching the relevant active mask counts:

MethodMMMUMMMU-ProMMStarMMBenchSeedBenchMathVerseMathVistaGPQAGPQA-DAverage
Teacher-context42.0029.7745.9375.5366.4618.2729.0029.9127.7840.52
Fixed-position41.6729.3145.5375.8366.4917.5130.7030.5826.7740.49
CT-OPD43.6729.8347.0075.3766.5318.1547.6030.1327.7842.90

CT-OPD achieves the highest average accuracy (42.90 vs. 40.52/40.49), supporting the value of trajectory-derived mask selection beyond teacher-conditioned or fixed-position alternatives.

Held-out prediction diagnostics further support this finding. On 512 prompts excluded from post-training, CT-OPD achieves a reconstructed-state NLL of 3.2781, compared with 3.4146 for Random, with a paired difference of −0.1366 (95% CI [−0.1699, −0.1024]).

AuthorsOpenReview ↗

Response to fX3Q (2/2): measured training overhead

We measured full-run training cost, including online trajectory collection.(Weakness 2) The table combines four-GPU update times with full-run GPU-hours per 1,000 scheduled training prompts:

Methodseconds/updateFull-run GPU-h/1k prompts
Native SFT16.323.56
CT-OPD30.188.54
MDPO88.9823.67
MaskGRPO58.8817.54

Four-GPU timings use A100 PCIe 80GB GPUs, BF16, and ZeRO-2, including trajectory collection and optimization. SFT/CT updates contain 16 supervised states, RL updates contain four prompts with four sampled responses each. CT timing averages complete four-update cycles. Means cover 36 SFT updates, 13 CT cycles, and 20 updates per RL method after warm-up. These short-run timings and the full-run allocation costs measure different scopes.

Costs include the complete training allocation, model initialization, checkpointing, and final export. CT reuses each trajectory for four updates, while the RL recipes sample four completions per prompt.

CT-OPD requires 8.54 GPU-hours per 1,000 prompts in the completed run, compared with 23.67 for MDPO and 17.54 for MaskGRPO. Its additional cost over SFT purchases current-model trajectories whose masks support four supervised updates. This reuse amortizes trajectory collection across the four training stages. Appendix E.6 includes the same cost table and measurement boundaries.

Thank you very much for your time and effort in reviewing our submission.

We would like to kindly follow up on our rebuttal, as the rebuttal period is approaching its deadline. We have carefully addressed each of your concerns point by point in our response, and would greatly appreciate it if you could take a moment to review our rebuttal and share any further thoughts or clarifications you may have.

We would be happy to address any remaining concerns or questions.

Thank you again for your time and consideration!

Reviewer_YKRbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CT-OPD, a post-training recipe for masked-diffusion VLMs. A frozen AR teacher (Qwen3-VL-32B-Instruct) answers Vision-SR1-47K prompts once. Answer-verified responses are detokenized, retokenized into the student vocabulary, and fitted to the student's canvas (Sec. 3.1, App. A.1).

In each training cycle, the current student decodes the prompt, and its unresolved-position sets at the 100/75/50/25% stages are overlaid on the teacher response. Visible positions take teacher tokens, so the student's provisional tokens are discarded. Masked teacher tokens are supervised with the student's native, per-example-normalized cross-entropy, with four sequential updates per rollout (Eqs. 3, 5, 7). Appendix D formalizes the reconstruction and objective (Props. 1–5).

Experiments cover LLaDA-V, LaViDa-LLaDA, an in-house MoE model (DiMoE-VL-Base), and MMaDA (understanding and image generation). Comparisons include MDPO/MaskGRPO/UniGRPO, Endpoint-only (fully masked), count-matched Random masks, OPDLM+BPM, and Direct Trace-Target. CT-OPD has the best average on every backbone (e.g., 40.00→49.79 on LLaDA-V, Table 1). Controlled variants and probes (Tables 2, 3, 11, 14) are used to attribute the gains to partial-state training, coherent reconstruction, and online, prompt-matched masks.

Critical strengths
  1. The controlled comparisons test the method's ingredients under matched teacher data and update budgets, with a consistent ordering across all four backbones. Endpoint-only, Random, and CT-OPD share endpoints, initialization, optimizer, and state exposure. Random also matches CT-OPD's per-stage mask counts and nested schedule (Sec. 4.1, App. A.2.1), providing a controlled comparison of random and trajectory-derived mask selection. The averages order the same way on every model (Table 1):

    • LLaDA-V: 43.12 < 45.85 < 49.79
    • LaViDa-LLaDA: 40.30 < 41.84 < 42.90
    • DiMoE-VL: 35.65 < 37.78 < 41.48
    • MMaDA understanding: 56.38 < 58.33 < 59.51

    Direct Trace-Target (45.99 vs. 49.79, Sec. 4.4) also yields a lower average than coherent reconstruction in this experiment, a useful comparison that helps explain the role of reconstruction. The consistent ordering across architectures supports the contribution of trajectory-derived masks, while leaving some uncertainty that the clarifications below could help address.

  2. The recipe is straightforward to implement and applies across architectures and output modalities using the native loss. It uses teacher outputs re-encoded in the student's own token space, one student rollout per four updates, and the existing masked cross-entropy (Sec. 3.4). The same construction trains an MoE decoder, including its routers and experts (Sec. 3.4, A.2.2), and MMaDA's image-token branch. There, GenEval/CLIP/ImageReward rise from 55.45/28.43/0.585 to 61.69/29.16/0.774, above every compared arm on all three metrics (Table 8). This breadth makes the method potentially useful for diffusion-VLM post-training.

Critical weaknesses
  1. Clarifying the differences between the reported Base scores and published backbone results would help establish how to interpret the absolute gains. I appreciate the consistent gains under the paper's evaluation setup. My main question is how that setup relates to the published backbone evaluations:

    • LLaDA-V. The original paper reports 48.6 MMMU, 60.1 MMStar, and 59.7 MathVista (lmms-eval). Table 1 lists 41.78/50.07/53.60 for Base. After CT-OPD the scores are 46.22/55.87/61.30, still below those published Base results on MMMU and MMStar. It would be helpful to confirm whether the checkpoint versions, benchmark splits, and evaluation settings agree.
    • LaViDa. The original paper reports 44.8 on MathVista versus 33.00 here, so the highlighted +14.60 gain (p. 7) ends at 47.60. Clarifying the evaluation differences would help interpret this gain relative to the published result.
    • GPQA. LLaDA-V Base scores 16.96 on GPQA and 16.67 on GPQA-D, below the 25% random-choice reference for four-option questions. Could the authors help explain these scores, including whether prompting, decoding, or answer extraction contributes? The scores alone do not identify the cause.
    • Protocol. Appendix A provides useful implementation details, but I would appreciate a more explicit description of the understanding-benchmark evaluation prompts, generation length, decoding steps, remasking, and answer parsing. The endpoint-trained arms all learn the "Reason: … Final answer:" format (A.1.2).

    The within-protocol improvements are encouraging, but differences in prompts, decoding, or answer extraction could contribute to the discrepancy with published Base scores. Clarifying these differences would help assess how far the abstract's "up to 9.80 points" can be interpreted as a capability gain over the published checkpoints. Relative comparisons among the endpoint-trained arms are less affected. Helpful clarifications: the protocol and checkpoint/split correspondence, parse-failure rates, and an explanation of the Base-score differences. If re-evaluation is needed, a representative comparison of Base, Random, and CT-OPD under the official settings would be particularly informative.

  2. The attribution of gains to student-selected positions would benefit from uncertainty estimates and probes beyond the training-prompt distribution.

    • Training-run uncertainty. Benchmark results are reported for one seed per configuration, without variability across training runs (Tables 4, 7; A.2.4). The reported margins over count-matched Random are 3.95/1.05/3.70 (p. 7) and 1.17 on MMaDA; Table 3 factor effects are 0.88–2.37. The prompt-paired bootstrap intervals for the probes are useful, but address a different source of uncertainty.
    • Benchmark concentration. The average gaps are uneven across tasks. On DiMoE-VL, Random and Endpoint-only fall 12.18 and 12.86 points below Base on MMBench, and MMBench alone supplies 16.72 of the 33.31 summed per-benchmark points separating CT-OPD from Random. On LaViDa, Endpoint-only and Random differ by 17.40 on MathVista. These are differences between methods, rather than estimates of seed variability, but they make per-task consistency and uncertainty important to the interpretation.
    • One backbone. The reconstruction and mask-source ablations provide useful evidence on LLaDA-V; their applicability to the other backbones is less directly established.
    • Training-prompt probes. Tables 2 and 11 use 512 training prompts with trajectory-derived masks (B.2.3). The common probe states make the comparisons useful, but evaluation on held-out prompts would strengthen the evidence that the mechanism generalizes.
    • Table 14. The margin T=1.1123\mathcal T=1.1123 is almost entirely the reconstruction term E=1.0976\mathcal E=1.0976, while the raw-state terms nearly cancel (H=0.2975\mathcal H=0.2975, A=−0.2828\mathcal A=-0.2828). The pooled point estimates therefore imply little excess teacher-target NLL at retained positions in the raw student context (H+A≈0.015\mathcal H+\mathcal A\approx0.015). An uncertainty estimate for this sum or a positional control would help clarify how much of the effect depends on student-specific selection.

    Helpful additional evidence, if feasible: repeated Random vs. CT-OPD runs on a representative backbone and the existing probes on held-out prompts would be the most informative checks. Positional controls and replication on a second backbone would provide useful additional evidence if feasible.

Questions for authors
  1. [Primary clarification] Could you provide a fuller description of the evaluation protocol for the understanding benchmarks: prompt template, canvas/generation length, number of reverse steps, remasking rule, and answer extraction. Was it identical for Base and all trained arms? It would also help to understand how the checkpoints, splits, and settings compare with the original backbone evaluations, and to see answer-parse failure rates, especially for GPQA/GPQA-D. If re-evaluation is needed to explain the Base discrepancy, a representative Base/Random/CT-OPD comparison under the official settings would be valuable. These clarifications would help me assess the scope of the reported gains.
  2. [Primary clarification] If additional runs are feasible, could you report the mean and variability of the Random vs. CT-OPD comparison across a few training seeds, preferably three, on a representative backbone such as LLaDA-V? This would help assess the robustness of the mask-selection contribution. Replication on a second backbone or for the Table 3 variants could be considered as additional follow-up evidence if resources permit.
  3. [Clarification] Table 14 reports only an identity residual for the 100%→75% transition. There, raw and reconstructed parents coincide, so T=H+A\mathcal T=\mathcal H+\mathcal A measures the allocation margin without context replacement. Could you share T\mathcal T, H\mathcal H, and A\mathcal A at that transition? Could you also report an uncertainty estimate for the pooled H+A\mathcal H+\mathcal A and describe how retained versus just-revealed positions differ in relative position or distance to visible tokens? This would help distinguish the effects of selection, reconstruction, and positional structure.
  4. [Optional suggestion] A control using count-matched masks from a fixed positional prior, such as left-to-right selection or distance to visible tokens, would help assess whether the gain specifically depends on student-selected positions. Another useful comparison would be the backbone's native diffusion-SFT objective, with its standard mask-ratio sampling and weighting, on the same endpoints and token budget. Random currently uses the four nested ratios matched to the student's counts. Either control would be informative if feasible.
  5. [Clarification] Could the Table 2/11 probes be repeated on held-out prompts? As run on training prompts with trajectory-derived masks (B.2.3), they leave some uncertainty about generalization beyond that probe distribution. Evaluation on uniformly masked reconstructed states would be a useful secondary analysis, with the held-out-prompt check being the more informative priority.
7 comments
AuthorsOpenReview ↗

Response to YKRb (1/4): evaluation protocol

Thank you for identifying the details needed to interpret the absolute gains.

Base, Random, and CT use the same task prompts and generation settings (Weakness 1.4; Question 1) Their saved LLaDA-V configurations agree:

BenchmarksGeneration lengthReverse steps
MMMU, MathVerse, MathVista4848
MMMU-Pro, MMStar, MMBench, SeedBench, GPQA, GPQA-Diamond88

All three arms use temperature zero and native confidence-based decoding. Training uses a separate 128-token canvas and 32 reverse steps. Multiple-choice inputs contain the question and options followed by “Answer with the option letter only,” unless the benchmark supplies its own prompt. Question-answering tasks retain their provided prompts. Each backbone uses its native conversation wrapper, LaViDa uses its prefix-LM interface. “Reason: … Final answer:” is the training-endpoint format, not an extra CT-specific evaluation instruction.

LLaDA-V versus published Base. (Weakness 1.1; Question 1) The released evaluation script uses two tokens/two steps for MMMU and MMStar, versus our 48/48 and 8/8. During case studies, we observed that LLaDA-V and LaViDa-LLaDA often produced outputs appearing as "A" under short generation budgets. With longer budgets, some responses instead formed natural-language phrases such as "A picture..." or "A person...", rather than explicit selections of option A. Such truncation can create ambiguity in answer extraction. Moreover, some benchmarks includes free-form questions for which a two-token limit may be insufficient. These observations motivated our use of longer generation budgets. For MathVista, the released configuration uses 96 tokens/48 steps, versus our 48/48. Thus, the published and submitted Base numbers reflect different decoding and scoring configurations.

We reran the same Base, Random, and CT-OPD checkpoints using the released MMMU/MMStar/MathVista configurations:

MethodMMMUMMStarMathVista
Published Base48.660.159.70
Our Base rerun47.8960.1459.80
Random rerun49.2263.9362.40
CT-OPD rerun53.2264.7366.10

Our Base rerun closely reproduces the published MMMU and MMStar scores. Importantly, CT-OPD consistently outperforms both Base and Random under the released settings, improving over Base by 5.33/4.59/6.30 points on MMMU/MMStar/MathVista. Thus, Table 1 reports gains under our common evaluation configuration, while these reruns confirm that CT-OPD's advantages persist under the officially released settings.

LaViDa MathVista. (Weakness 1.2; Question 1) We use the released lavida-llada-v1.0-instruct checkpoint. The LaViDa paper specifies LMMS-Eval's mathvista_testmini_format task with a 100-token generation length, whereas our evaluations use a fixed task prompt, a 48-token generation canvas, and 48 reverse steps with prefix-LM decoding. Additionally, we replace the legacy GPT-3.5 answer extractor with Qwen3-8B, consistently applying it to all evaluated checkpoints. To verify this discrepancy and assess the robustness of our improvements, we reran both Base and CT-OPD using the released LaViDa evaluation protocol:

MethodOur protocolLaViDa protocol
Published Base44.80
Our Base rerun33.0044.80
CT-OPD rerun47.6057.80

Our Base rerun reproduces the published 44.80 under the released protocol. Importantly, CT-OPD consistently outperforms Base under both protocols, achieving gains of +14.60 and +13.00 points, respectively. These results demonstrate that CT-OPD's improvements persist under the officially released evaluation settings.

GPQA: parsing failures and below-chance Base accuracy. (Weakness 1.3; Question 1) We inspected the model responses and extended answer extraction to recognize more fields before applying the original option matcher. For responses without such fields, the original parsing behavior is preserved.

BenchmarkModelOriginal parsing failuresAfter extended extraction
GPQABase1515
CT-OPD1712
GPQA-DBase1212
CT-OPD85

The extended extraction resolves 5 and 3 previously unparsed CT-OPD responses on GPQA and GPQA-D, respectively, while leaving Base's parsing failures unchanged. Importantly, these parsing failures alone cannot explain the below-chance Base scores, even if all unparsed Base responses were counted as correct, its accuracy would reach only 20.31% on GPQA and 22.73% on GPQA-D, both below the 25% random-choice reference.

ModelGPQA Acc.GPQA-D Acc.
Base16.9616.67
CT-OPD33.7135.35

The extended extraction rule resolves only 5/448 and 3/198 previously unparsed CT-OPD responses, corresponding to 1.12 and 1.52 percentage points at most. These changes are far smaller than the overall gains of 16.75 and 18.68 points over Base. Since both checkpoints are evaluated under identical settings, the substantial improvements primarily reflect the effectiveness of CT-OPD rather than artifacts of answer extraction.

AuthorsOpenReview ↗

Response to YKRb (2/4): task concentration and second-backbone components

Training-seed robustness. (Weakness 2.1; Question 2) Following the reviewer's suggestion, we conducted two additional independent training runs for both Random and CT-OPD on LLaDA-V, yielding three training seeds per method under identical training and evaluation configurations.

Training SeedRandomCT-OPDImprovement
Seed 1 (original)45.8549.79+3.94
Seed 244.9348.81+3.88
Seed 347.6749.34+1.67
Mean ± Std46.15 ± 1.3949.31 ± 0.49+3.16 ± 1.29

CT-OPD consistently outperforms the count-matched Random baseline across all three independent training seeds, with an average improvement of 3.16 percentage points. Moreover, CT-OPD exhibits lower across-seed variability (0.49 vs. 1.39), providing additional evidence of training stability. Together with the consistent improvements across four backbones, these results strengthen the empirical support for the effectiveness and robustness of CT-OPD.

The Table 1 gains extend beyond MMBench. (Weakness 2.2) We computed win counts and leave-one-benchmark-out averages from the results:

BackboneRandomCT-OPDCT wins/ties/lossesGap excluding MMBench
LLaDA-V45.8549.797/1/1+3.41
LaViDa41.8442.909/0/0+0.94
DiMoE37.7841.488/1/0+2.07
MMaDA understanding58.3359.516/0/0+0.84

The derived statistics use task scores. DiMoE retains +1.35 points after excluding both MMBench and SeedBench. Across the four backbones, CT wins 30 of 33 comparisons and ties on two. Thus MMBench contributes substantially to DiMoE's gain, while the advantage extends to other tasks and architectures.

Second-backbone component experiments are complete. (Weakness 2.3; Question 2) We already compares CT/Random on four backbones. The additional LaViDa studies examine reconstruction, online refresh, and prompt matching. DTT denotes Direct Trace-Target, FM/FS denote frozen matched/shuffled masks, OS denotes online shuffled masks.

MethodMMMUMMMU-ProMMStarMMBenchSeedBenchMathVerseMathVistaGPQAGPQA-DAverage
DTT41.6729.3146.0075.6566.5218.1530.2030.5827.7840.65
FM41.5629.6046.5375.6966.3718.2730.1029.2427.7840.57
FS42.2229.4245.5375.5166.4618.6529.4028.5726.7740.28
OS42.8929.4846.0775.0566.3917.8929.3029.0226.7740.32
CT-OPD43.6729.8347.0075.3766.5318.1547.6030.1327.7842.90

Among the new variants, coherent-context averages range from 40.28 to 40.57, versus 40.65 for DTT. Frozen prompt matching improves the average by 0.29 points over frozen shuffled masks.

AuthorsOpenReview ↗

Response to YKRb (3/4): Allocation and held-out prediction

Initial allocation before context replacement. (Weakness 2.5; Question 3) At the 100%→75% transition, the raw and reconstructed parents coincide. Across 504 valid prompts, we obtain H=0.2071, A=−0.1127, E=0, and T=0.0945, with a paired 95% bootstrap CI of [0.0387, 0.1544] for T. This indicates a positive initial allocation margin even without context replacement.

For the original pooled visible-parent analysis, directly bootstrapping the per-prompt sum yields H+A=0.0147 [−0.1316, 0.1639], while E=1.0976 and T=1.1123. Thus, the pooled decomposition shows a substantial reconstruction-associated contribution, whereas the initial transition isolates the allocation margin before context replacement.

Positional structure. (Weakness 2.5; Question 3) At the initial transition, retained and just-revealed tokens have mean relative positions of 0.6311 and 0.2387, respectively. At later transitions, these means are 0.7076 and 0.5642, with normalized distances to visible response tokens of 0.3291 and 0.1559. The later statistics cover 437 prompts with recoverable active coordinates. These descriptive results reveal positional differences between the selected token sets, although they do not independently isolate positional effects from trajectory-based selection.

Held-out prediction diagnostics. (Weakness 2.4; Question 5) To test whether the observed prediction benefits extend beyond training prompts, we evaluated 256 ScienceQA and 256 M3CoT prompts excluded from post-training, checking overlap using question and image identities. All models were scored on identical trajectory states generated by a single frozen Base checkpoint.

ModelHeld-out reconstructed-state NLL
Base5.3776
Endpoint-only3.3327
Random3.4146
Direct Trace-Target4.4733
CT-OPD3.2781

Using 10,000 paired prompt-bootstrap draws, the CT-OPD NLL differences (95% CIs) are:

  • vs. Random: −0.1366 [−0.1699, −0.1024]
  • vs. Endpoint-only: −0.0547 [−0.1044, −0.0037]
  • vs. Direct Trace-Target: −1.1953 [−1.2826, −1.1040]

CT-OPD achieves the lowest NLL on held-out trajectory-reconstructed states, with paired confidence intervals excluding zero for all three comparisons. In contrast, on uniformly masked reconstructed states, CT-OPD yields a slightly higher NLL than Random (+0.0302 [0.0203, 0.0406]). This contrast suggests that CT-OPD's prediction advantage is specific to trajectory-shaped states rather than uniform across masking distributions.

Section 4.6 and Appendix E provide the held-out experiments and additional allocation statistics.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We sincerely thank all Program Chairs, Senior Area Chairs, Area Chairs and three reviewers for the time and care they devoted to our paper and for their thoughtful, constructive feedback. We are particularly encouraged by their recognition of the context–target construction, the controlled ablations, and the empirical validation across dense, sparse, and unified diffusion VLMs. We also appreciate their assessment that CT-OPD is straightforward to implement and applicable to both multimodal understanding and image generation. Their questions have helped us sharpen the explanation of the method and expand the evidence for its design.

Our central contribution is to align coherent token-level supervision with the student's own reveal decisions. CT-OPD separates the two sources of information, a free student trajectory determines the unresolved positions, while a completed teacher response supplies consistent visible content and prediction targets. This construction supports different teacher and student tokenizers and uses the student's native training objective. The benchmark results establish its effectiveness across architectures and modalities, while the component ablations and new held-out diagnostics explain the roles of reconstruction, online refresh, and prompt-specific position selection.

In response to the reviews, we have added complete-method comparisons with native diffusion SFT, PUMA, and LIFT on LLaDA-V, replicated the reconstruction and mask-source ablations on LaViDa-LLaDA, and evaluated teacher-conditioned selection, fixed-position selection, and an 8B endpoint teacher. We have also extended the prediction diagnostics to 512 held-out prompts, supplemented the allocation analysis with the initial-stage decomposition and positional statistics, and analyzed benchmark concentration. To clarify practical cost and evaluation, we now report full-run training costs for SFT, CT-OPD, MDPO, and MaskGRPO, document the generation and answer-extraction settings, and make the fixed-mask counterfactual construction explicit. Each specific response presents the relevant results and explanations directly, with the corresponding details incorporated into the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.