acceptodds
Under review as a conference paper at ICLR 2027

CODA: Contrastive Decision Advantages for Unified Text-to-Image Generation

Abstract

Unified models integrate language understanding and image synthesis within a single model, and reinforcement learning further improves their generation capability through outcome-level rewards on completed images. However, relying on a single final-image reward to jointly optimize semantic reasoning and visual generation leaves two difficulties: ambiguous semantic attribution and sparse visual supervision. The former mixes high-level semantic reasoning with its low-level visual realization, while the latter leaves a long chain of visual decisions without feedback before the image is completed. To address them, we propose **COntrastive Decision Advantag (CODA)**, a framework that derives contrastive decision advantages by comparing each decision with a matched alternative using shared randomness, which contains **Semantic Reasoning Guidance** and **Masked Trajectory Guidance**. Specifically, Semantic Reasoning Guidance isolates the effect of prompt optimization by comparing images generated from the optimized and original prompts. Masked Trajectory Guidance contrasts each probed commitment with equally sized alternatives and weights its outcome gain by the induced distribution shift, promoting earlier commitments that are both correct and structurally influential. A trajectory-level outcome advantage is further incorporated to provide a global signal. Grounded in the same constraint-anchored evaluation from the original instruction, the semantic advantage supervises the text tokens forming the optimized prompt, while the process advantage supervises each probed commitment. Extensive experiments on GenEval, T2I-CompBench, and WISE demonstrate that CODA achieves outstanding performance across tasks involving complex scene composition and knowledge-intensive image generation.

Primary area:
generative models
Keywords:
Unified Multimodal Models, Text-to-Image Generation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_funWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CODA, a reinforcement learning framework for unified text-to-image generation that addresses ambiguous semantic credit assignment and sparse visual supervision. CODA introduces two components: Semantic Reasoning Guidance, which estimates the benefit of prompt optimization through paired generations with shared randomness, and Masked Trajectory Guidance, which evaluates visual token commitments against alternative commitments and weights their outcome differences using KL-based distributional influence. Experiments on GenEval, T2I-CompBench, and WISE demonstrate substantial improvements over the Show-o baseline.

Critical strengths

The paper addresses an important limitation of outcome-based RL for unified image generation, where semantic reasoning and visual generation decisions receive the same trajectory-level reward despite their different contributions.

Critical weaknesses

CODA incurs substantial computational overhead from additional counterfactual rollouts. Without training-cost analysis or compute-matched comparisons with standard outcome-based RL, it remains unclear whether the improvements stem from better credit assignment or increased sampling budgets.

Questions for authors

How does CODA compare with Mask-GRPO or outcome-only RL under matched GPU-hours or total sampling budgets?

3 comments
AuthorsOpenReview ↗

Evaluating CODA under matched computational budgets

Q1. Comparison under matched GPU-hours or sampling budgets

We thank the reviewer for raising this important concern. To ensure a fair comparison, we evaluate CODA against GRPO, Mask-GRPO, and outcome-only RL using the same Show-o 1.3B backbone, training data, and evaluation settings, with a matched budget of 80 GPU-hours. GRPO and Mask-GRPO are algorithm baselines. The outcome-only ablation retains CODA's visual-update formulation and uses only LoutL_{\mathrm{out}} supervision on visual actions. Its rewriter is the shared Show-o model, matching Table 4 and our response to Reviewer gKdN.

MethodGPU-hoursGenEval ↑
GRPO8074.61%
Mask-GRPO8078.58%
Show-o self-rewriting + outcome-only RL8072.15%
CODA8084.25%

Notably, GRPO, Mask-GRPO, and outcome-only RL reach their best performance with less training time than CODA. Extending their training to the matched budget yields no substantial further improvements, suggesting that their performance has largely saturated. In contrast, CODA achieves 84.25% under the same 80 GPU-hour budget, outperforming GRPO, Mask-GRPO, and outcome-only RL by 9.64pp, 5.67pp, and 12.10pp, respectively. These results demonstrate that CODA provides consistent performance gains beyond simply increasing training computation, highlighting the effectiveness of its contrastive semantic and visual process supervision.

Thanks for your rebuttal. I maintain my positive score.

We sincerely thank the reviewer for the positive feedback. Your comments are very valuable for improving our work. We will ensure that these revisions are incorporated in the subsequent version. We sincerely appreciate your time and effort in supporting the review of our paper.

Reviewer_gKdNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CODA, providing more fine-grained RL feedback for a unified text understanding and image generation model, to address the issues of ambiguous semantic and sparse visual feedback in existing methods. Specifically, the paper introduces a Semantic Reasoning Guidance branch and a Masked Trajectory Guidance branch. The former measures semantic contribution by comparing the score differences of images generated from the original and improved prompts. Meanwhile, the latter refines the contribution of a specific step in the trajectory by calculating the score difference between the actual commitment and the same number of alternative commitments, weighted by the KL divergence of the predicted distributions before and after the commitment. The authors conduct experiments on Show-o 1.3B, showing improvements on GenEval, T2I-CompBench, and WISE, and also perform ablation studies on each module.

Critical strengths

The overall motivation and main module logic are clear and complete, easy to understand. The experimental results are competitive and include ablation experiments. Overall performance on GenEval increased from 68.2 to 84.25, and on WISE from 35 to 50, with a significant performance boost and better results compared to other methods; ablation experiments show that both main modules have positive effects and the complete method performs the best.

Critical weaknesses

The contrastive process advantage is over-claimed and has technical gaps. Changing a decision also alters subsequent generation states and trajectories, meaning that the resulting reward difference may still reflect complex downstream interactions rather than a precisely isolated causal effect.

Questions for authors
  1. The setup of the experiment training seems inconsistent, which affects reproducibility. In the manuscript, the probed step set consists of two steps uniformly sampled from the full T = 50 decoding steps. However, it is different in Appendix (32).
  2. CODA requires additional matched semantic generations and multiple counterfactual visual rollouts, but the paper does not systematically report training time, GPU cost, or performance improvements under matched computational budgets against standard GRPO or Mask-GRPO baselines.
  3. The experimental validation is limited to the 1.3B Show-o backbone, leaving its generalizability to other unified multimodal architectures and visual decoding mechanisms unverified.
  4. Is the rewriting stage active at inference in every row of Table 4? Please add: (a) base Show-o with self-rewrite and with a Qwen3.5 rewrite, no RL; (b) the same rewrites plus L_{out} training.
  5. As no knowledge-related rewards are included, could the authors explain in detail the improvements on WISE?
  6. In the main experimental setup, the authors probe only two steps from a 50-step decoding trajectory. It looks the additional feedback remains sparse relative to the complete sequence of visual decisions.
3 comments
AuthorsOpenReview ↗

Addressing reproducibility, computational cost, and generalizability

Q1. We evaluated both settings and found that 50-step decoding performs better. All experiments in the main text use 50 steps. The appendix retained an earlier, more efficient configuration and will be corrected.

StepsGenEval ↑
3282.93%
5084.25%

Q2. CODA uses 8 NVIDIA H20 GPUs for 80 GPU-hours. Under the same GPU-hour budget, Show-o 1.3B backbone, training data, and evaluation settings, we compare:

MethodGPU-hoursGenEval ↑
GRPO8074.61%
Mask-GRPO8078.58%
CODA8084.25%
CODA outperforms GRPO and Mask-GRPO by 9.64pp and 5.67pp, respectively, achieving the best performance among the compared methods under the same computational budget.

Q3. CODA contrasts actual decisions with matched alternatives, using token commitments for masked decoding and image-token choices with matched continuations for autoregressive decoding.

BackboneSettingGenEval ↑
Janus-Pro-1BOriginal paper73%
Janus-Pro-1B+ CODA84%
Janus-Pro-7BOriginal paper80%
Janus-Pro-7B+ CODA89%
Emu3-8BOriginal paper54%
Emu3-8B+ CODA64%

CODA improves over the original scores by 11pp, 9pp, and 10pp, respectively. This supports transferability across model scales and autoregressive architectures.

Q4. Table 4's first row uses the original prompt without RL (68.29%). All other rows use rewriting at inference. Removing LsemL_{\mathrm{sem}} removes semantic supervision, not rewriting. The Show-o self-rewriting + LoutL_{\mathrm{out}} row below is Table 4's outcome-only ablation, with shared model parameters. Qwen3.5-2B remains frozen during training and evaluation. All four settings start from the same Show-o generator; RL rows use only LoutL_{\mathrm{out}} supervision.

RewriterVisual supervisionGenEval ↑
Show-o self-rewritingNo RL69.37%
Qwen3.5-2B (frozen)No RL74.18%
Show-o self-rewritingLoutL_{\mathrm{out}}72.15%
Qwen3.5-2B (frozen)LoutL_{\mathrm{out}}79.13%

Qwen rewriting improves GenEval by 4.81pp without RL and 6.98pp with LoutL_{\mathrm{out}}. Adding LoutL_{\mathrm{out}} gives 2.78pp and 4.95pp for Show-o and Qwen rewriting. Full CODA achieves 84.25%, exceeding the best outcome-only setting in this table by 5.12pp. The 3.86pp difference between 68.29% and 72.15% includes enabling rewriting and outcome-only training; RL's gain with Show-o rewriting already enabled is 2.78pp.

Q5. A plausible explanation for the WISE gains without a dedicated factual-knowledge reward is better use and visual realization of pretrained knowledge. Rewriting can make implicit knowledge explicit as visual conditions, while process guidance helps render the requested objects, attributes, and relations.

Table 4 is consistent with this explanation: WISE increases from 38 with outcome-only training to 43 with LsemL_{\mathrm{sem}}, 41 with LprocL_{\mathrm{proc}}, and 50 with full CODA. Process supervision adds 7 points when semantic supervision is retained; semantic supervision adds 9 points when process supervision is retained. Both modules contribute to WISE performance.

Q6. Thanks for your review. We agree that two probe steps provide limited temporal coverage within an individual 50-step trajectory. CODA’s focus, however, is to obtain training signals attributable to specific visual decisions through random sampling, rather than evaluating every step in every rollout. We substantiate this design through four aspects: stochastic estimation of the process objective, attribution of decision gains, step coverage across rollouts, and the existing ablation results. (1) Stochastic estimation. Uniformly sampling two eligible steps provides an unbiased estimate of the mean process loss over eligible steps:

ES[1∣S∣∑t∈Sℓt(θ)]=1∣T∣∑t∈Tℓt(θ).\mathbb E_{\mathcal S}\left[\frac{1}{\lvert\mathcal S\rvert}\sum_{t\in\mathcal S}\ell_t(\theta)\right]=\frac{1}{\lvert\mathcal T\rvert}\sum_{t\in\mathcal T}\ell_t(\theta).

Here, T\mathcal T denotes eligible steps, S\mathcal S the sampled probes, and ℓt\ell_t the per-step process loss. Thus, probe count mainly affects estimation variance. (2) Decision attribution. Each probe compares actual and equal-sized alternative commitments from the same predecessor state under matched continuations, with outcome differences measuring the benefit of the actual commitment. (3) Coverage across rollouts. Probe locations are resampled across rollouts to cover different eligible steps, while KL influence weighting captures their impact on subsequent predictions. (4) Empirical support. Adding process supervision improves GenEval from 76.88% to 84.25% (+7.37pp), supporting sparse process supervision. CODA measures total decision effects under matched continuations, including downstream interactions, rather than strict causal effects.

Thank you for the rebuttal. The submitted results especially experimental analysis on generalizability have addressed most of my concerns. I choose to raise my score.

We sincerely thank the reviewer for the positive feedback. Your comments are very valuable for improving our work. We will ensure that these revisions are incorporated in the subsequent version. We sincerely appreciate your time and effort in supporting the review of our paper.

Reviewer_7CiDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CODA, a reinforcement learning framework for unified text-to-image generation. Using Show-o as the backbone, the method first rewrites the input prompt and then performs masked image generation. It assigns credit to prompt rewriting through the reward difference between images generated from the original and rewritten prompts under shared random seeds. For visual generation, it evaluates alternative token commitments through matched continuations and weights the resulting process advantages using changes in predictive distributions measured by KL divergence. These signals are combined with an outcome objective to update the shared model. Experiments on GenEval, T2I-CompBench, and WISE report improvements over the baseline and component ablations.

Critical strengths
  1. The paper addresses a meaningful credit-assignment problem. Separating the contribution of prompt rewriting from decisions made during image generation is a useful direction. Comparing matched continuations provides an interpretable way to evaluate the consequences of intermediate visual decisions, rather than relying exclusively on a single terminal score.

  2. The reported improvements and component ablations are encouraging. The results across compositional alignment and knowledge-dependent generation benchmarks suggest that the proposed training signals are useful in aggregate. Table 4 also provides evidence that both semantic and visual process supervision contribute to the final performance.

Critical weaknesses
  1. The benefit of jointly learning the rewriter is not fully isolated. Table 4 supports the effectiveness of the semantic and process objectives. However, it does not compare joint training with a fixed, independent rewriter paired with the same Show-o visual training. Removing the semantic loss does not itself freeze the rewriting behavior when parameters remain shared. This leaves the benefit of coupled optimization, beyond improved prompt conditioning, unclear.

  2. The incremental contribution over Visual-CoG requires clearer differentiation. Visual-CoG [1] already uses Show-o and evaluates semantic rewriting through the reward difference between original-prompt and rewritten-prompt images generated under the same seed. CODA’s visual commitment comparisons and KL influence weighting are potential advances, but their distinction from—and benefit over—the closely related prior framework should be established more directly.

Questions for authors
  1. In Table 4, does the model still generate rewritten prompts when the semantic loss is removed? How does CODA compare with a frozen external LLM rewriter used consistently during both training and evaluation, with comparable visual training budgets and reported rewriting costs?

  2. How does the semantic advantage differ from Visual-CoG’s Equation (4)? Can a controlled comparison isolate the incremental benefit of CODA’s visual process supervision over that prior framework?

  3. As a supplementary analysis, how sensitive are the results to the relative loss weights? Gradient alignment or relative gradient norms could help explain interactions between the objectives.

3 comments
AuthorsOpenReview ↗

Clarifying semantic supervision and validating contrastive decision advantages

Q1. Rewriting behavior and frozen rewriters

Yes. Without LsemL_{\mathrm{sem}}, the model still generates P′P' and uses it for image generation. Table 4 removes semantic supervision, not rewriting. Visual optimization may still change rewriting behavior through shared parameters.

We compare CODA with independently frozen Show-o 1.3B and Qwen3.5-2B rewriters, used during both training and evaluation. The downstream Show-o generator receives Lout+LprocL_{\mathrm{out}}+L_{\mathrm{proc}} supervision.

RewriterSupervisionGenEval ↑Rewrite latencyWall time
Show-o (joint CODA)Lsem+Lout+LprocL_{\mathrm{sem}}+L_{\mathrm{out}}+L_{\mathrm{proc}}84.25%1.5 s/prompt10 h
Show-o 1.3B (frozen)Lout+LprocL_{\mathrm{out}}+L_{\mathrm{proc}}78.13%1.5 s/prompt8 h
Qwen3.5-2B (frozen)Lout+LprocL_{\mathrm{out}}+L_{\mathrm{proc}}80.99%1.6 s/prompt8.8 h

All runs use 8 H20 GPUs and the same number of visual updates. Wall time includes rewriting, rollouts, reward computation, and optimization, corresponding to 80, 64, and 70.4 GPU-hours. This comparison matches visual updates.

CODA improves over the frozen rewriters by 6.12pp and 3.26pp, respectively. Its semantic advantage compares outcomes under P′P' and PP to supervise rewriting jointly with visual generation. Inference consists of one rewrite followed by image generation; matched semantic pairs and counterfactual branches are training only.

Q2. Relation to Visual-CoG and process supervision

  1. Despite shared-seed outcome contrasts, attribution differs. Visual-CoG sums rewards (Visual-CoG Eq. 10), which may mask weak semantic gains; CODA instead uses separate clipped objectives for rewritten text tokens and visual commitments (Eq. 15).

  2. Visual-CoG requires an external teacher, yet reconstruction agreement does not prove decision gains. CODA contrasts actual and equal-sized alternative commitments from the same predecessor state under matched continuations, using original-prompt outcomes and KL influence weighting for task-aligned supervision.

  3. CODA averages semantic reward differences over K=2K=2 matched image pairs for less noisy gain estimates.

We compare process modules with the semantic and outcome objectives retained:

Process supervisionGenEval ↑
None76.88%
Visual-CoG-style teacher reconstruction82.55%
CODA process guidance84.25%

Teacher reconstruction improves GenEval by 5.67pp over no process supervision. CODA improves it by 7.37pp, exceeding the teacher-reconstruction setting by 1.70pp. The backbone, data, evaluator, and semantic/outcome objectives are the same; only process supervision differs. The 82.55% result is this controlled module comparison, not the complete published Visual-CoG system.

Q3. Loss-weight sensitivity and gradient interactions

We fix the semantic weight at 1 and vary the process weight:

Semantic weightProcess weightGenEval ↑
1.00.2583.27%
1.00.584.25%
1.01.083.59%

Across a fourfold change in process weight, GenEval remains within 83.27%–84.25%, with a maximum difference of 0.98pp. The default weight of 0.5 performs best.

Following [1], we sample 200 minibatches and compute semantic and process gradients on shared Transformer parameters. Let gsem,bg_{\mathrm{sem},b} and gproc,bg_{\mathrm{proc},b} denote the gradients for minibatch bb:

Cb=gsem,b⊤gproc,b∥gsem,b∥2∥gproc,b∥2.C_b=\frac{g_{\mathrm{sem},b}^{\top}g_{\mathrm{proc},b}}{\lVert g_{\mathrm{sem},b}\rVert_2\lVert g_{\mathrm{proc},b}\rVert_2}.

ρb=∥gsem,b∥2∥gproc,b∥2.\rho_b=\frac{\lVert g_{\mathrm{sem},b}\rVert_2}{\lVert g_{\mathrm{proc},b}\rVert_2}.

P_{-}=\frac{100\\%}{200}\sum_{b=1}^{200}\mathbf{1}(C_b < 0).

ModelCosine similarityNorm ratioNegative rate
Janus-Pro-7B [1]−0.0001 ± 0.00290.07347.5%
Janus-Pro-1B [1]−0.0030.09379.8%
CODA0.0014 ± 0.00310.10345.5%

Janus-Pro values are references from [1], while CODA is measured under our default setting. CODA's near-zero positive mean cosine and 45.5% negative rate suggest that the two objectives are nearly orthogonal overall, without strong conflict. With process weight 0.5, the semantic-to-weighted-process norm ratio increases from 0.103 to 0.206, reducing the gradient magnitude gap. These results support the compatibility of the objectives under the default setting.

[1] Abinav Rao and Sujan Rachuri. Do Understanding and Generation Fight? A Diagnostic Study of DPO for Unified Multimodal Models. arXiv:2603.17044, 2026.

Thank you for the rebuttal. The frozen-rewriter comparison with matched visual updates addresses my concern about the benefit of joint training, and the controlled comparison with a Visual-CoG-style teacher-reconstruction module helps isolate the contribution of CODA’s visual process supervision. The loss-weight and gradient analyses also address my concern about interactions between the objectives. I am raising my rating from 2 to 3 (weak acceptance). In the revision, please include these comparisons and their training costs, clarify that the same-seed semantic reward contrast is shared with Visual-CoG, and explain that the measured effects of visual decisions include downstream interactions under matched continuations.

We sincerely thank the reviewer for the positive feedback. Your comments are very valuable for improving our work. We will ensure that these revisions are incorporated in the subsequent version. We sincerely appreciate your time and effort in supporting the review of our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.