- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes DUETGRPO, which supplies direct rewards for reasoning correctness and generation-prompt fidelity. This textual supervision is combined with shared image–text alignment feedback to enable online co-adaptation of the language and visual policies. Relative gains from the GRPO stage are reported on WISE (+0.05) and T2I-CoReBench (+1.6).
Critical strengths
- The core motivation of directly checking both the reasoning trace and the generation prompt against the original user instruction is reasonable and addresses a gap in existing think-then-generate pipelines.
- The joint RL recipe that integrates composite textual supervision with shared image–text alignment feedback is supported by carefully designed ablation studies, providing empirical verification of the proposed combination.
Critical weaknesses
- The reasoning reward, which constitutes the primary claimed motivation, can be viewed as a straightforward instantiation of multi-reward or process-reward ideas. This raises concerns about the degree of novelty and the extent of new inspiration offered to the community.
- The additional data construction and training cost do not yield state-of-the-art results, nor do they produce substantially larger relative (SFT -> RL) improvements than those observed for other baseline methods. This raises questions about the efficiency and necessity of the reasoning-reward component.
- Several introduced modules (e.g., checklist construction for the text-only evaluator and the selection of coefficients for combining textual rewards) rely on manually designed components. Such design choices may introduce instability when assessing the effectiveness and robustness of the overall approach.
Questions for authors
- Could the authors provide deeper discussion of the experimental results beyond numerical values? For example, Table 2 shows that DUETGRPO behaves quite differently across categories. Why does this variation occur, and what guidance does it offer regarding the conditions under which reasoning rewards are most effective?
- Please clarify the key differences between DUETGRPO and Dual-GRPO. Both methods follow the think-then-generate paradigm. From the presented material, the reward for reasoning correctness appears to be the principal addition; clarification on whether this constitutes a more substantial technical distinction would be helpful.
7 comments
Weakness 2: The additional data construction and training ...
We thank the reviewer for raising the question of whether the additional supervision justifies its cost. All experiments in Table 4 of the manuscript use the same training data and SFT initialization, so the data-construction cost is shared across these comparisons. Regarding final performance, DuetGRPO achieves the highest Overall scores among the methods compared on both WISE (0.77) and T2I-CoReBench (60.7). While the starting backbone leaves room for improvement in some subcategories, these overall results support the effectiveness of our training recipe.
To further examine the contribution of the text reward and its combination with joint optimization, we replace text reward with VLM-based language reward . All scores below are WISE Overall:
VLM-based reward with a frozen DiT: We train the autoregressive (AR) policy using UnifiedReward as the sole reward while keep clip score training DiT. This configuration yields 0.72 → 0.74, whereas text reward+clip score yields 0.72 → 0.77 (Table R3-1). This comparison supports the benefit of directly evaluating reasoning correctness and generation-prompt fidelity using text alone, independently of rendered-image quality, over the tested VLM-based feedback.
Table R3-1. Comparison of UnifiedReward as a language reward.
| Training variant | AR training reward | Cultural | Time | Space | Biology | Physics | Chemistry | Overall |
|---|---|---|---|---|---|---|---|---|
| DuetGRPO (ours) | Text reward + CLIPScore | 0.81 | 0.82 | 0.80 | 0.62 | 0.72 | 0.74 | 0.77 |
| UnifiedReward baseline | UnifiedReward | 0.80 | 0.76 | 0.71 | 0.65 | 0.74 | 0.71 | 0.74 |
The matched ablation in Table 4 of the manuscript isolates the text reward's contribution during joint optimization: removing it reduces Overall from 0.77 to 0.75, with the remaining configuration unchanged. Together, these results show a measurable benefit from direct text supervision and further gains from combining it with online joint optimization. Our text reward specifically supervises reasoning and prompt correctness, while a separate image–text alignment reward supports coordination with the renderer. The shared training data and these controlled comparisons provide evidence for the reward's empirical value.
Weakness 3: Several introduced modules ...
We thank the reviewer for highlighting the need to validate the scoring rules and reward coefficients. Each checklist is generated by a fixed evaluator from the instruction and its reference explanation, cached, and shared across candidates, allowing semantically equivalent realizations. Manual review of 6,000 CoT–PE pairs on held-out WISE instructions shows agreement with human judgments: checklist coverage achieves Pearson correlations of 0.82/0.77 and ICCs of 0.81/0.79 for DuetGRPO/SFT, respectively, while prompt and reasoning correctness achieve κ = 0.80–0.86 (Table R2-4 in our response to Reviewer 2).
The coefficient sweep in Table R3-2 further shows that WISE Overall remains within 0.73–0.74 across the tested nonzero coefficients, with multiple settings reaching 0.74. Setting the coefficient of reasoning correctness (), prompt coverage (), or prompt correctness () to zero yields 0.73, 0.72, and 0.72, respectively. These results support retaining all three components and indicate limited sensitivity of aggregate performance to the tested coefficient values.
Table R3-2. Ablation of text-reward coefficients in DuetGRPO (ours). Bold values indicate the best result within each coefficient sweep, including ties.
| Coefficient | Value | Cultural | Time | Space | Biology | Physics | Chemistry | Overall |
|---|---|---|---|---|---|---|---|---|
| 0 | 0.79 | 0.79 | 0.73 | 0.63 | 0.60 | 0.66 | 0.73 | |
| 1/5 | 0.80 | 0.79 | 0.73 | 0.64 | 0.70 | 0.67 | 0.74 | |
| 2/5 | 0.80 | 0.80 | 0.74 | 0.64 | 0.69 | 0.67 | 0.74 | |
| 3/5 | 0.79 | 0.78 | 0.72 | 0.63 | 0.71 | 0.73 | 0.74 | |
| 0 | 0.79 | 0.77 | 0.76 | 0.68 | 0.63 | 0.60 | 0.72 | |
| 1/5 | 0.79 | 0.79 | 0.72 | 0.63 | 0.67 | 0.65 | 0.73 | |
| 2/5 | 0.80 | 0.80 | 0.74 | 0.64 | 0.69 | 0.67 | 0.74 | |
| 3/5 | 0.80 | 0.79 | 0.72 | 0.64 | 0.70 | 0.67 | 0.74 | |
| 0 | 0.78 | 0.79 | 0.73 | 0.63 | 0.57 | 0.64 | 0.72 | |
| 1/5 | 0.79 | 0.81 | 0.72 | 0.66 | 0.65 | 0.65 | 0.73 | |
| 2/5 | 0.79 | 0.79 | 0.73 | 0.63 | 0.67 | 0.66 | 0.73 | |
| 1/2 | 0.80 | 0.80 | 0.74 | 0.64 | 0.70 | 0.67 | 0.74 | |
| 3/5 | 0.79 | 0.81 | 0.77 | 0.62 | 0.68 | 0.70 | 0.74 |
We thank the reviewer for the opportunity to clarify the distinction. DuetGRPO differs from the original Dual-GRPO in the supervision target and the definition and assignment of rewards.
1. Direct supervision of reasoning and prompt correctness. In the original Dual-GRPO, the language policy is rewarded by averaging the semantic consistency scores between generated images and the original user instruction; the reasoning and refined prompt are not directly checked for factual correctness. (Dual-GRPO, Sec. 3.3) DuetGRPO instead directly evaluates reasoning correctness, generation-prompt coverage, and prompt correctness with a text-only evaluator, using a candidate-independent checklist for prompt evaluation. This supplies correctness feedback on language outputs independently of rendering quality.
2. Separate checks of instruction satisfaction and rendering fidelity. Dual-GRPO uses image-based semantic feedback for the language policy and a combination of aesthetic, physical-consistency, and semantic scores for the DiT. DuetGRPO checks language outputs against the user instruction and measures image alignment against the generated prompt. The language policy receives Rtext+Ralign, while the visual policy receives only Ralign. This explicitly rewards both a correct description of what should be generated and faithful rendering of that description.
The matched ablations support the value of these complementary signals: removing either the text reward or the language policy's alignment feedback reduces WISE Overall from 0.77 to 0.75 (Table 4 of the manuscript). For a stricter baseline comparison, the revised Dual-GRPO baseline in Table R2-2 of our response to Reviewer 2 additionally receives our text reward and uses generated prompts for CLIPScore; the distinctions above concern the published Dual-GRPO objective.
Regarding the gradient-routing question, with the full rewards retained, detached conditioning achieves 0.77, compared with 0.75 for coupled gradients. Appendix F analyzes early, archived runs in which visual-loss gradients introduced additional language updates; these unrestricted extra updates were constrained before the experiments in Table 4 of the manuscript. The diagnostics identify a potential source of optimization imbalance, while the ablation in Table 4 of the manuscript evaluates the effect of conditioning-gradient routing under the revised schedule.