- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CoUPLe provides a unified framework for adapting pretrained text-to-image flow models to visual understanding, image generation, and image editing. The framework integrates decodable continuous text representations, distinct interfaces for text conditioning and text generation, and causal blockwise flow matching. Evaluations on Flux2-9B and Z-Image, encompassing extensive benchmark comparisons and ablation studies, substantiate the effectiveness of latent adaptation and blockwise generation, while demonstrating positive transfer from image generation to visual understanding.
Critical strengths
- The separation of text conditioning from text generation addresses a concrete compatibility problem when adapting pretrained image generators, making the approach technically well motivated.
- Validation across two backbone architectures and multiple understanding, generation, and editing tasks demonstrates broad applicability, with ablations supporting key design choices.
Critical weaknesses
- Direct comparisons with the closest continuous multimodal methods, particularly GenFirst and OmniFlow, are missing. This limits assessment of the incremental contribution beyond existing continuous and blockwise formulations.
- Training-exposure accounting and matched-budget controls are insufficiently specified. Consequently, performance gains and generation-to-understanding transfer cannot be cleanly attributed to the proposed modeling choices.
Questions for authors
- How do the reported dataset sizes reconcile with the sample exposure implied by update counts and batch sizes? Are the joint and text-only controls matched in data exposure and compute?
- What attention structure does the Text VAE encoder use, and can preceding clean latent blocks contain information from future response text?
- What evidence establishes the incremental benefit over the closest continuous multimodal methods under comparable training conditions?
6 comments
Response to Reviewer 76Vt (2/3)
W2/Q2: Training exposure.
Both full CoUPLe and the understanding-only control reported in the paper use the hardware configuration and training duration. We report the total non-padding text tokens processed during each training stage. These counts include repeated presentations of training examples and therefore measure cumulative training exposure rather than unique corpus tokens.
| Training stage | Non-padding text tokens, including repetitions |
|---|---|
| Stage I | 34B |
| Stage II | 95B |
| Stage III | 3B |
Controlled evaluation. We separately disable the Stage-I image-generation loss, the later image-generation losses, and image-generation losses throughout training. These comparisons isolate the contributions of generation supervision at different stages. For the understanding-only control, we disable image-generation losses throughout Stages I–III and allocate the saved budget to additional understanding-training updates. This tests whether longer understanding-only training can recover the benefit of joint image-and-text training.
| Configuration | Stage-I image loss | Later image losses | MMMU ↑ | MathVista ↑ | MME ↑ |
|---|---|---|---|---|---|
| Full CoUPLe | On | On | 68.19 | 80.12 | 1712 |
| No Stage-I image loss | Off | On | 66.01 | 76.32 | 1677 |
| No later image losses | On | Off | 65.22 | 74.89 | 1647 |
| No image loss in any stage | Off | Off | 63.42 | 72.00 | 1609 |
| Understanding-only, hardware-and-time matched | Off | Off | 63.98 | 74.02 | 1643 |
| Full CoUPLe minus matched control | — | — | +4.21 | +6.10 | +69 |
Removing generation supervision in either Stage I or the later stages lowers performance on all three understanding benchmarks. Additional understanding updates recover part of this loss, but full CoUPLe still exceeds the hardware-and-time-matched control by 4.21 percentage points on MMMU, 6.10 percentage points on MathVista, and 69 points on MME-P. These results support a benefit from generation supervision that additional understanding-only training does not recover within the same resource budget.
We have added these results in the revised manuscript in sec 4.2
Q2: Text VAE.
The Text VAE encoder uses causal self-attention, so each text representation depends only on the current and preceding tokens. Consequently, preceding clean latent blocks encode only the corresponding response prefix and cannot contain information from subsequent response blocks. The backbone additionally enforces block-causal attention: each noisy response block can attend to the conditioning context, preceding clean response blocks, and its own noisy positions. Its clean counterpart and future response blocks are inaccessible. Together, these restrictions prevent future-response information from leaking through the preceding clean latent blocks.
Response to Reviewer 76Vt (3/3)
Q3: Evidence beyond the closest continuous multimodal methods under comparable training conditions. We thank the reviewer for requesting comparisons under comparable training conditions. We fine-tuned GenFirst and OmniFlow on the same training dataset used for CoUPLe, with the same hardware configuration and training duration. This matches the additional adaptation budget across methods. All models were evaluated under our common benchmark protocol.
The table reports baseline performance before and after adaptation, alongside CoUPLe.
| Model | Training setting | MMMU ↑ | MathVista ↑ | MME ↑ | GenEval ↑ | DPG-Bench ↑ |
|---|---|---|---|---|---|---|
| OmniFlow | Base checkpoint | 39.34 | 22.21 | 1111 | 0.6700 | 66.93 |
| GenFirst | Base checkpoint | 42.29 | 61.29 | 1312 | 0.8957 | 84.68 |
| OmniFlow | Matched adaptation | 44.05 | 30.11 | 1192 | 0.7211 | 72.04 |
| GenFirst | Matched adaptation | 49.17 | 68.22 | 1412 | 0.9003 | 87.31 |
| CoUPLe (Flux2-9B) | Matched adaptation | 67.59 | 78.45 | 1488 | 0.8800 | 89.88 |
| CoUPLe (Z-Image) | Matched adaptation | 68.19 | 80.12 | 1512 | 0.8900 | 90.29 |
Both baselines benefit from adaptation to our dataset. Under the matched adaptation budget, CoUPLe achieves stronger understanding performance across all three benchmarks and the highest DPG-Bench score among the compared models. Compared with adapted GenFirst, CoUPLe (Z-Image) improves MMMU by 19.02 percentage points, MathVista by 11.90 percentage points, and DPG-Bench by 2.98 points. GenFirst achieves the highest GenEval score. These comparisons support the empirical benefit of CoUPLe’s adaptation framework and complement the encoder-separation and alignment ablations that isolate its specific design choices.
We have added this results in the Appendix H.8 of the revised manuscript
We appreciate the reviewer’s constructive feedback, which has helped us substantially strengthen the paper. The new comparisons demonstrate substantial understanding gains over the closest continuous multimodal methods, while the controlled ablations establish the benefits of generation supervision, encoder separation, and conditioning alignment. We respectfully ask the reviewer to consider these new results and the stronger evidence supporting CoUPLe’s contributions.
Response to Reviewer 76Vt (1/3)
We thank the reviewer for recognizing the motivation behind separating text conditioning from text generation, and for appreciating the validation across architectures, tasks and design ablations.
W1 Comparisons with the closest continuous multimodal methods.
We thank the reviewer for requesting these comparisons. We have expanded Tables 1 and 3 with GenFirst, LLaDA2.0-Uni, and additional OmniFlow evaluations, using the same benchmark protocol as CoUPLe. GenFirst is concurrent work whose paper was not publicly available at the time of our original submission.
Methodological advance and importance. CoUPLe addresses a central challenge in extending pretrained image generators to multimodal understanding: the representations used to condition image synthesis must remain compatible with the pretrained backbone, while text-generation targets must support continuous modeling and reliable decoding. CoUPLe separates these roles through a fixed conditioning encoder and an adaptive text-target encoder, connects the conditioning interface to the pretrained backbone through alignment, and jointly adapts the text-generation latents with reconstruction and reference-posterior regularization. This enables pretrained visual flow backbones to acquire causal blockwise language generation and multimodal reasoning while retaining image-generation capabilities.
This contribution differs from the objectives and designs of the closest methods. GenFirst develops stable end-to-end latent learning and trains its unified block-causal generative prior from scratch. OmniFlow extends SD3 to any-to-any generation using a text VAE. CoUPLe establishes how causal blockwise text generation and adaptive text representations can be integrated into pretrained visual flow backbones, with the conditioning and decoding requirements explicitly separated and validated. We demonstrate this approach across both MM-DiT and single-stream DiT architectures through Flux2-9B and Z-Image.
Please also see our response to Reviewer
JTnhW1/Q1Methodological contribution for additional discussion on the technical contributions of CoUPLe and controlled validation of CoUPLe’s encoder separation and alignment components.Direct empirical comparison.
Both CoUPLe variants outperform the compared baselines on MMMU, MathVista, MME-P, and DPG-Bench. Relative to GenFirst, CoUPLe (Z-Image) improves MMMU by 25.90 points, MathVista by 18.83 points, and DPG-Bench by 5.61 points, while matching its GenEval score. These results demonstrate substantial improvements in understanding alongside strong image generation.
The expanded comparisons appear in Tables 1 and 3, and the controlled encoder and alignment ablations are provided in Appendix H.6 of the revised manuscript.