- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
UMM-Reflection trains a unified BAGEL model to iteratively inspect and revise its generated images. After SFT on externally constructed reflection trajectories, the method samples complete trajectories from a shared initial image and uses a trajectory-level advantage to update both reflection tokens and flow-based revisions. Experiments on GenEval and three benchmarks not used for RL training evaluate generation quality and the contributions of joint training and iterative refinement.
Critical strengths
-
The improvements over reflection SFT are substantial, including 12.05 points on GenEval and 10.97 points on WISE, with additional gains on OneIG-Bench and T2I-CompBench++. Improvements on three benchmarks not used for RL training support the generalization of the learned refinement policy beyond the training setting.
-
The experiments provide useful evidence for jointly optimizing reflection and image revision. Table 3 reports GenEval scores of 84 for joint RL, compared with 78 for text-only RL and 73 for flow-only RL. The instruction-swapping experiment further supports the benefit of improved textual guidance. Comparisons with sample selection and repeated SFT editing under a four-image budget also strengthen the evidence that the gains are not explained simply by generating more images.
Critical weaknesses
- I find the methodological advance over prior unified-model RL refinement limited. AlphaGRPO [1] already fixes a common initial image for refinement and uses a shared advantage to jointly optimize reflection text and flow-based image revision, while Vinci [2] already studies multi-round reflection with RL in a unified model. UMM-Reflection extends joint text–flow optimization to complete multi-round repair trajectories with a shaped trajectory reward. The extensive experiments demonstrate the effectiveness of this extension. However, the two highlighted design choices—sharing the initial image and jointly updating text and image policies with one advantage—already appear in AlphaGRPO, and their extension across repair rounds is not positioned against Vinci. This overlap narrows the methodological novelty relative to the paper’s framing.
References:
[1] Runhui Huang, Jie Wu, Rui Yang, Zhe Liu, and Hengshuang Zhao. AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in Unified Multimodal Models via Decompositional Verifiable Reward. ICML, 2026.
[2] Wang Lin, Wentao Hu, Liyu Jia, Kaihang Pan, Majun Zhang, Zhou Zhao, Fei Wu, Jingyuan Chen, and Hanwang Zhang. Vinci: Deep Thinking in Text-to-Image Generation using Unified Model with Reinforcement Learning. NeurIPS, 2025.
Questions for authors
-
Could you clarify the technical differences from AlphaGRPO’s joint text–flow optimization and Vinci’s multi-round RL, and explain how these differences address challenges specific to multi-round text–flow refinement? A controlled comparison with a single-round joint RL policy, using the same SFT initialization, prompt pool, verifier, and a comparable training budget, would help establish the value of this extension. Both policies should be evaluated under the same multi-round inference protocol.
-
Under Equations 1–2 and the stated reward coefficients, the verifier-score sequences (0, 1, 1, 1) and (0, 1, 0, 1), each with three edits and no premature DONE, receive rewards of 1.30 and 1.40, respectively. Thus, losing and recovering previously achieved progress can receive a higher reward than preserving it. How often do sampled trajectories become verifier-correct, regress, and then recover? Do the gains persist when the progress terms are computed only from improvements beyond the best previous verifier score?
For future work, evaluation on another unified backbone would help establish how broadly the training approach applies beyond BAGEL.
Additional comment on related work:
I also suggest discussing several related approaches to learning reflection and refinement. GenAgent [3] trains multi-turn interactions with image-generation tools using both final-image rewards and pairwise improvement rewards. RL-RIG [4] uses Reflection-GRPO to train an editing-instruction actor and an image editor in separate stages. R³-Bench [5] evaluates visual diagnosis and rectification separately and trains a refiner using feedback from executed image edits. These works provide useful context for clarifying what UMM-Reflection gains from jointly optimizing reflection and rendering within one model, and how the quality of the learned reflection is assessed.
Additional references:
[3] Kaixun Jiang et al. GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning. ECCV, 2026.
[4] Tianyu Wang et al. RL-RIG: A Generative Spatial Reasoner via Intrinsic Reflection. arXiv:2602.19974, 2026.
[5] Junjie Wang et al. Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation. ICML, 2026.
8 comments
Reviewer LQFT — Part 2/2
Related work: GenAgent, RL-RIG, R³-Bench
We have made this distinction explicit and cite all three works in the revised Related Work. Each splits reflection and rendering across separate models: GenAgent's MLLM agent calls FLUX.1-dev as a fixed tool; RL-RIG trains its VLM Actor and image editor separately with Reflection-GRPO; and R³-Bench's R³-Refiner is an MLLM refiner attached to external T2I models. UMM-Reflection trains reflection and rendering in one unified model under a single advantage.
Direct evaluation isolates the learned reflection: with the image, renderer, and noise fixed, the RL policy's instruction repairs 48.4% of failing states vs. 21.4% for SFT (App. J), and masking the image given to reflection while still passing it to the renderer drops repair from 64.9% to 21.4%.
Other backbones
UMM-Reflection requires only a single model that both reads and renders images with an evaluable rendering policy likelihood (here, Flow-GRPO's SDE sampler); nothing in the reward, shared-root sampling, or shared advantage is specific to BAGEL. BAGEL is the most widely used open unified model that both understands and generates images (Sec. 4.1), and the method applies directly to other unified models of this kind. Extending UMM-Reflection to more recent unified models such as SenseNova-U1 is our future work.
Thank you again for your thoughtful feedback! We hope our responses have addressed your concerns. If any concerns remain, we'd be happy to discuss them further. If you feel your concerns have been resolved, we’d appreciate it if you would reconsider your overall score in light of our responses.
Reviewer LQFT — Part 1/2
W1 + Q1. Difference from AlphaGRPO and Vinci; single-round control
Our task is interleaved generation itself: in a unified model, rendering an image, diagnosing it, and re-rendering with the full text–image history in context forms a natural reflection chain, and UMM-Reflection trains this entire multi-round chain with one trajectory-level RL reward. AlphaGRPO trains only a single reflection step without image history ("we utilize single-turn conversations rather than multi-turn conversations (containing the history of the initially generated images)", AlphaGRPO, App.), so it cannot learn to act on its own earlier revisions. Vinci regenerates a new image each round on Emu3, a discrete-token model where standard GRPO applies unchanged; its cold-start chains are built from independently sampled candidates ordered by score (Vinci, Sec. 3.2), sibling groups start from different first images, and the text is rewarded by a separate MLLM judge. UMM-Reflection instead repairs the current image, preserving correct regions while fixing errors; all siblings share one initial image, so the advantage compares repair strategies rather than first draws; and reflection text is credited only through the image it produces, eliminating the need for a text judge. This positioning is now explicit in the revised paper, with AlphaGRPO and Vinci cited in Related Work.
Multi-round text–flow refinement differs because a revision is evaluated only after later rounds render it. Credit must therefore propagate to every reflection and every flow transition in the trajectory: shared-root sampling isolates the repair strategy from initial-draw luck, and a single trajectory-level advantage updates both heads without the KN branching of per-round credit (Sec. 3.4).
The controlled single-round comparison uses the same SFT parent, prompt pool, verifier, and K, with 500 updates each and the same 3-round inference at evaluation.
3-round training outperforms single-round training on all four benchmarks (GenEval +5.4, WISE +3.9, OneIG +1.7, T2I-CompBench++ +2.2) and raises the repair rate from 35.1% to 61.1%. We have added this experiment to the revised paper.
Q2. Path dependence of the reward
β credits improvement accumulated across multiple edits, e.g., (0.1, 0.3, 0.45, 1), rather than a single lucky fix (Sec. 3.4); with β=0, GenEval drops from 84 to 79 (Table 3). The (0, 1, 0, 1) ordering is not exploitable: a failed repair, (0, 1, 0, 0), scores −0.20, so deliberately breaking a correct image yields an expected 1.6r−0.2 versus 1.30 for keeping it, which is lower for every per-edit repair probability r<15/16, while the policy's per-edit repair rate is at most 38% (Sec. 6.1). Trajectories that rise, drop, and recover make up 2.4% of training rollouts and 1.4% on GenEval.
When progress and multi-improvement terms are computed only from improvement beyond the best previous score, as the reviewer suggests ((0, 1, 0, 1) then scores 0.80), the gains persist:
Both rewards match within 0.3 points across all four benchmarks. We have added this ablation to the revised paper.