acceptodds
Under review as a conference paper at ICLR 2027

ReasonUni: Reasoning-Augmented Reconstruction Advances Unified Multimodal Models

Abstract

Image-to-text (I2T) understanding and text-to-image (T2I) generation are dual tasks in unified multimodal models (UMMs). Reconstructive post-training couples these tasks, but existing reconstruction-based methods remain perception-centric and provide limited supervision for reasoning about structure, intent, and transformation. We propose ReasonUni, a reasoning-augmented reconstructive post-training framework that extends this coupling beyond perception. Given an original image and an editing instruction, the UMM first reasons about what and how to edit and describes the target state, then generates the corresponding target image conditioned on this understanding output. To jointly optimize both modules, we introduce TreeGRPO, a tree-structured reinforcement learning method that samples multiple reasoning paths and candidate images per path, then aggregates visual rewards to update both modules under a unified objective. Experiments across multiple benchmarks demonstrate consistent gains in both understanding and generation, showing that reasoning-augmented reconstruction yields genuine mutual gains for UMMs.

Primary area:
generative models
Keywords:
Unified Multimodal Models, Reinforcement Learning, Post-Training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QoakOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the joint enhancement of understanding and generation capabilities in Unified Multimodal Models (UMMs). It proposes ReasonUni, which connects visual understanding and image generation through an inference-driven image editing and reconstruction task, and designs TreeGRPO, which decomposes the reward for generated images into an understanding branch reward and a generation reward to achieve joint reinforcement learning.

Critical strengths
  1. The paper introduces a novel approach to supervising visual reasoning through reasoning-augmented reconstruction. The model infers the target state from an editing instruction and uses reconstruction feedback to improve this inference, offering a promising way to strengthen visual reasoning through image generation.
  2. TreeGRPO is intuitive and well motivated, allowing relatively separate evaluation of the understanding and generation policies for cleaner credit assignment. The latent reward complements this design by avoiding image decoding and external encoding, improving training efficiency while maintaining strong performance.
Critical weaknesses
  1. The reward ablation settings need more detail. For example, how is GPT-4.1 used for scoring, and how are external encoder features converted into rewards? Please clarify these implementations and whether all variants use the same training settings.
  2. The motivation for branch-level averaging could be supported more directly. An analysis of within-branch and between-branch reward variance using existing rollouts could help quantify the uncertainty that TreeGRPO aims to mitigate.
Questions for authors
  1. Could the author provide more details of the reward ablation settings?
  2. Could the authors provide an analysis of within-branch and between-branch reward variance using existing rollouts to better quantify the uncertainty that TreeGRPO?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Qoak

We thank the reviewer for the concrete suggestions.

W1. Reward ablation settings

All reward variants share the same data, training steps, initialization, and optimization settings (K=M=4); only the reward differs:

  • Latent (ours): cosine similarity between mean-pooled SigLIP-VQ codebook embeddings of the generated and target tokens (Eq. 2). No decoding is needed.
  • SigLIP2 / DINOv3: generated tokens are decoded to pixels and re-encoded; the reward is the cosine similarity of global features (pooled embedding for SigLIP2, CLS token for DINOv3). Decoding and re-encoding explain the longer training time (~60h vs. 25h).
  • GPT-4.1: given the source, target, and generated images, the editing instruction, and the reference effect description, it rates instruction adherence and content preservation on a 1–5 scale (temperature 0), mapped to [0, 1] by (s−1)/4. With only five score levels, many candidates in a group tie and get zero advantage, which explains its weaker results despite the highest cost (68h).

We will add these details to the supp.

W2. Motivation for averaging rewards within branches

Branch averaging nearly halves ranking errors among descriptions. From existing rollouts, the reward variance is 0.001172 within branches and 0.006012 across branch means. The und. policy, however, learns from the ranking of the K descriptions of an input, which single-image rewards flip often.

We fixed four target descriptions and scored them twice with disjoint sets of generated images. The two evaluations pick different top descriptions 39.90% of the time with one image per description, vs. 22.51% with the average of four (random: 75%). Averaging thus gives the und. policy a markedly more stable signal.

Thanks for the reply. The rebuttal addresses most of my concerns. I think the comparison with dedicated understanding RL and the multi-seed results are reasonable. I maintain my initial rating and recommend acceptance.

We appreciate your positive assessment of the two experiments we conducted to address the other reviewers’ concerns about ReasonUni’s understanding gains. Thank you again for your thoughtful review and continued support!

Reviewer_uaYPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ReasonUni post-trains a unified autoregressive model (X-Omni) on reasoning-based image editing. Given a source image and an edit instruction, the model first writes a text description of the edited scene. It then generates the target image from that description alone. The reward is the cosine similarity between mean-pooled SigLIP-VQ codebook embeddings of the generated image and the ground-truth edited image. TreeGRPO samples K descriptions with M images each. Descriptions get advantages from their branch-mean rewards, and images get advantages relative to their siblings. On X-Omni SFT and RL checkpoints, the method improves 12 understanding benchmarks (mostly by 0.1 to 2.7 points) and several generation benchmarks (e.g., GenEval++ 40.00 to 46.79, WISE 45 to 48 in the RL setting). It also beats a reimplemented UAE.

Critical strengths

The idea is natural and clearly motivated. Turning reconstruction into “describe the edited target, then render it” pushes the understanding side beyond captioning what is already there. Training both directions in one AR model with one objective is cleaner than the split design in UAE.

TreeGRPO is a nice design in a two-stage pipeline. The fixed-budget comparison with flat GRPO (Table 5b) and the latent reward’s lower cost (25h vs about 60h) are useful practical results.

Critical weaknesses

The understanding gains are small, and the “reasoning” claim rests on very few questions. Most gains are 0.1 to 2 points, from one training run with no variance. The headline MMBench logical-reasoning gain (48.31 to 55.08) is about 8 extra correct answers out of roughly 118. MMVP +2.00 is 3 pairs. GenEval++ is scored in 2.5-point steps, so Pos/Size +25 is about 10 prompts i guess? Without seeds or confidence intervals, I can’t tell which gains are real.

It’s unclear that the gains come from reasoning-augmented reconstruction rather than the data. The training data has synthetic edits made with Gemini-3-Pro and Nano Banana, so some gains may be distillation from stronger models. A matched-data control exists only for UAE in the SFT setting. The most important ablation is missing from the main paper: same data and TreeGRPO, but the model describes the source image instead of the target. The paper says this is in the supplement, but it should be in the main text.

The reward may not measure the edit. Generation sees only the description, so most of the reward comes from reproducing the unchanged scene, which is perception again. The reward also mean-pools token embeddings, which discards layout. The claim that it “retains implicit spatial cues” is supported only by two examples in Figure 4.

Questions for authors

Can you report multiple seeds or confidence intervals, at least for MMBench reasoning, MMVP and GenEval++?

I am leaning towards a borderline accept though... happy to revise my scores based on the rebuttal

1 comment
AuthorsOpenReview ↗

Response to Reviewer uaYP

We thank the reviewer for the careful reading and helpful comments. In summary:

  1. Robust across seeds. Over 5 seeds, ReasonUni beats the base on every reported metric by 1.33-5.00 points, with std ≤ 1.05.
  2. Reasoning augmentation, not data, drives the gains. With the same data and TreeGRPO, it beats reconstruction-only on all metrics; UAE† trained on the same data gains far less.
  3. Reward measures the edit. Shared unchanged content cancels out in group-relative advantages; what remains requires reasoning about what to change and what to keep. Under the same reward, replacing reconstruction with editing improves reasoning-heavy metrics (W2).

W1. Statistical reliability

The gains hold across seeds. We re-ran the main RL setting with 4 extra seeds (2026–2029); mean ± std over all 5 runs:

ModelMMBenchMMB. ReasoningMMVPCV-BenchGenEval++
Base74.3166.3750.6768.2640.00
ReasonUni (5 runs)75.81 ± 0.1868.86 ± 0.6752.00 ± 1.0569.62 ± 0.6145.00 ± 1.04

Fine-grained subsets (e.g., MMBench LR, GenEval++ categories) are small and inherently noisy, so we report seed variance on the aggregate scores that contain them; the revision will report 5-run mean ± std.

W2. Contribution of reasoning augmentation vs. data

Reasoning augmentation, not data, drives the gains. Reconstruction-only shares ReasonUni's source images, TreeGRPO config, training steps, and seed, but describes and reconstructs the source image instead of the edited target; the only difference is whether the model must reason about the edit.

MethodMMBenchMMB. ReasoningMMVPCV-BenchGenEval++
Base74.3166.3750.6768.2640.00
+ Reconstruction-only + TreeGRPO75.8668.0150.6767.9143.21
+ Reasoning + TreeGRPO (ReasonUni)75.9669.4252.6770.0146.79

As expected, reconstruction-only already helps perception-heavy MMBench, while reasoning adds the most on reasoning-heavy metrics (MMB Reasoning +1.41, MMVP +2.00, CV-Bench +2.10) and on generation (GenEval++ +3.58).

Not distillation. (1) UAE† trained on exactly the same data gains far less (Tab. 1, SFT): GenEval++ 6.43 vs. 23.21, WISE 26 vs. 37. (2) ReasonUni never imitates teacher outputs: Gemini-written descriptions are never training targets, and edited images enter only via a scalar reward. We will move this comparison into the main text.

W3. Does the reward measure the edit?

Yes. The reward covers both perception and the edit, and the edit is what drives learning beyond reconstruction:

  1. Shared content cancels out. GRPO advantages are group-relative (Eq. 3): all candidates for one input share the same source scene, so its reward largely cancels out. What remains is whether each candidate changes the right thing and keeps the rest, which requires reasoning about the edit's scope, not just perception (e.g., in Fig. 5 the baseline keeps the target text but removes the license plate, failing both at once).
  2. The edit signal is measurable. Reconstruction-only TreeGRPO uses the same reward, hence the same perception signal; replacing reconstruction with editing is the only change, yet it adds +1.41 MMB Reasoning, +2.00 MMVP, +2.10 CV-Bench, and +3.58 GenEval++ (W2).
  3. The model learns the edited state. On 120 held-out questions requiring reasoning about the edits, target-state description accuracy rises from 48.3% to 63.3% (multiple-choice: from 71.2% to 92.9%).
  4. Every training pair has a visible edit. We drop pairs whose source–target latent similarity exceeds 0.95 (to be added to the supp.).

Mean pooling. SigLIP-VQ tokens are quantized from SigLIP features with positional embeddings and self-attention, so each token already carries spatial context. Notably, external-encoder rewards (pooled SigLIP2, DINOv3 CLS) are equally global, yet our latent reward outperforms both (Tab. 5a). We will clarify this in the revision.

Reviewer_dXu6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a post-training framework for unified autoregressive multimodal models that replaces perception-only image reconstruction with reasoning-driven image editing. Given a source image and an editing instruction, the model predicts a textual target-state description and then synthesizes the target image conditioned solely on this description. Both capabilities are jointly optimized via TreeGRPO using cosine similarity of mean-pooled codebook embeddings as the reward, yielding simultaneous gains across 12 understanding benchmarks and 3 text-to-image generation benchmarks on X-Omni.

Critical strengths
  1. The motivation is natural and well-founded: extending self-reconstruction from passive image captioning to instruction-based editing creates an information bottleneck that requires the model to infer scene transformations before generating the target image.

  2. TreeGRPO outperforms flat GRPO under a matched sampling budget, and computing rewards directly in the native SigLIP-VQ codebook space cuts training time by more than half while outperforming external reward encoders.

Critical weaknesses
  1. All experiments are conducted on a single backbone, X-Omni, leaving it unverified whether the method generalizes to other unified multimodal models.

  2. Although the paper emphasizes advancing visual understanding and reasoning beyond perception-centric methods like UAE, the empirical gains are heavily concentrated on generation while understanding improvements are marginal. Over half of the 12 understanding benchmarks improve by less than 1 point, and on MMBench overall and MathVerse, ReasonUni outperforms UAE and the matched-data UAE baseline by only 0.13 to 0.89 points; moreover, scaling the tree from 4x4 to 8x8 causes understanding and MMBench reasoning to drop below flat GRPO.

Questions for authors
  1. During evaluation on pure text-to-image benchmarks where no source image is provided, does the model generate visual tokens directly from the benchmark prompt, or does it first run the understanding policy to expand the prompt into a target-state description?

  2. Despite training entirely on image-editing triplets, the model is never evaluated on image editing benchmarks because generating solely from text alters source geometry. Did the authors evaluate the model on image editing benchmarks?

I am willing to raise my score if the authors can convincingly address the concerns.

2 comments
AuthorsOpenReview ↗

Response to Reviewer dXu6

We thank the reviewer for the constructive comments. In summary:

  1. Backbone-agnostic. The gains transfer to a new backbone, Tar (NeurIPS'25), improving both und. and gen. benchmarks.
  2. Understanding gains. We argue that the observation "und. gains are smaller than gen. gains" is because the base models are already near saturation on und. benchmarks (e.g., MMBench), where even dedicated und. RL improves little. Notably, our joint training matches dedicated und. RL.
  3. 8×8 improves both und. and gen. over 4×4. The reported step-900 checkpoint is gen.-favoring (Gen. Avg 49.02, the highest of all settings); every checkpoint from step 1000 on beats 4×4 on both und. and gen. (see W3).

W1. Backbone-agnostic

ReasonUni transfers to Tar, a fully autoregressive UMM with a different visual tokenizer (TA-Tok) and training recipe, improving both und. and gen. under the same experimental setup. Und. Avg. averages the five und. benchmarks below; Gen. Avg. averages WISE (+1%) and GenEval++ (+4.28%). We will include these results in the revision.

ModelUnd. Avg.Gen. Avg.MMBenchMMVPCVBenchVStarOCRBench
Tar50.2338.1869.6746.0061.9339.2734.30
w/ ReasonUni52.4340.8271.1348.0063.7042.9336.40

W2. Understanding gains

The smaller und. gains reflect limited headroom: base models are already extensively trained on und. benchmarks and near saturation on them. Within this headroom, ReasonUni's und. gains are competitive, stable, and supervision-free:

  1. On par with dedicated und. RL. An und.-only GRPO baseline [1] from the same initialization reaches similar und. scores but drops GenEval++ to 36.43, while ReasonUni raises it to 46.79.
  2. Stable across seeds. Over 5 runs (see R2-W1), MMBench reaches 75.81 ± 0.18 and MMBench Reasoning 68.86 ± 0.67, vs. 74.31 and 66.37 for the base.
  3. No und. supervision. ReasonUni is trained only on editing triplets (no QA data), with LoRA (r=8) for one epoch.
ModelMMBenchMMBench ReasoningMMVPCV-BenchGenEval++
Base74.3166.3750.6768.2640.00‾\underline{40.00}
UAE (CVPR'26)75.4367.6650.0068.0538.57
Understanding-only GRPO [1]75.52‾\underline{75.52}68.40‾\underline{68.40}53.3370.3836.43
ReasonUni75.9669.4252.67‾\underline{52.67}70.01‾\underline{70.01}46.79

[1] Li et al. To Think or Not To Think: A Study of Thinking in Rule-Based Visual Reinforcement Fine-Tuning. NeurIPS 2025, Spotlight.

W3. Scaling from 4×4 to 8×8

8×8 improves both und. and gen. over 4×4; the reported step-900 checkpoint is simply gen.-favoring. The paper selects checkpoints by the sum of und. and gen. gains (ΔU + ΔG), which picks step 900 for its Gen. Avg of 49.02, the highest of all settings. Every 8×8 checkpoint from step 1000 on beats both 4×4 and flat GRPO on und. (54.28–54.60 vs. 54.26 and 53.90) and gen. (47.51–47.78 vs. 47.40 and 45.40). A criterion rewarding balanced gains, the product ΔU × ΔG, selects step 1100; the revision will report it together with the full training curve.

SamplingMM-PMM-RUnd. AvgGen. AvgΔU + ΔGΔU × ΔG
Base75.8966.3752.6642.50——
Flat 1677.0367.8353.9045.40——
K=4, M=476.9369.4254.2647.40——
K=8, M=8 (900)76.5768.3653.2549.027.11003.8468
K=8, M=8 (950)77.1670.1054.5847.536.94609.6499
K=8, M=8 (1000)76.8269.7654.4847.787.09799.5986
K=8, M=8 (1050)77.0569.9354.2847.386.50407.9121
K=8, M=8 (1100)77.1669.5954.6047.516.95519.7430

ΔU and ΔG denote improvements over the base model in Und. Avg and Gen. Avg, respectively.

AuthorsOpenReview ↗

Response to Reviewer dXu6

Q1. Text-to-image evaluation protocol

Images are generated directly from the original benchmark prompts, without the und. policy or any prompt expansion. The gen. gains thus come from the improved generation policy itself, not prompt rewriting.

Q2. Image editing evaluation

Standard editing benchmarks require reference-image-conditioned generation, which X-Omni does not support; ReasonUni keeps this design, as editing triplets serve as a training signal rather than to build an editor. We therefore directly evaluate whether the model infers the edited state: on 120 held-out post-edit inference questions, target-state description accuracy rises from 48.3% to 63.3% (multiple-choice: from 71.2% to 92.9%).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Reviewer Feedback and Our Response

Dear Area Chair and Reviewers,

Thank you for the careful review. We summarize below the contributions highlighted in the reviews and our response to the reviewers’ main concern.

  1. Reviewers recognized reasoning-augmented reconstruction as novel (Qoak), well-founded (dXu6), and clearly motivated (uaYP). By requiring the model to infer the edited scene, it extends reconstruction-based supervision from perception to reasoning about how a scene should change.
  2. TreeGRPO is a “nice design” (uaYP) that enables clearer credit assignment (Qoak) and outperforms flat GRPO at the same sampling budget (dXu6, uaYP), while the native reward outperforms external reward encoders (dXu6) and cuts training time from about 60 to 25 hours (uaYP).
  3. Training understanding and generation in one autoregressive model with a single objective is cleaner than the split design used in prior work (uaYP).

Reviewers dXu6 and uaYP questioned the magnitude of ReasonUni’s understanding gains, and uaYP also asked whether the gains hold across seeds. We addressed these concerns with two experiments that Reviewer Qoak has already called “reasonable.”

  1. Comparing with understanding-only RL. From the same initialization, understanding-only GRPO (NeurIPS 2025 Spotlight) achieves similar understanding scores but lowers GenEval++ to 36.43, while ReasonUni raises it to 46.79.
  2. Stable across seeds. Across five runs, ReasonUni improved on all three benchmarks uaYP asked about (base score to five-run mean ± SD): MMBench Reasoning (from 66.37 to 68.86 ± 0.67), MMVP (from 50.67 to 52.00 ± 1.05), and GenEval++ (from 40.00 to 45.00 ± 1.04).

These results support the effectiveness and stability of ReasonUni’s understanding gains. We appreciate your time and consideration. We hope you will consider our paper for acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.