acceptodds
Under review as a conference paper at ICLR 2027

CoUPLe: Continuous Flow Unifying Pixels and Language

18 views

Abstract

We present CoUPLe, a framework that turns pretrained text-to-image flow models into unified multimodal generators by modeling text responses and images with a shared continuous velocity field. Our key design decouples text conditioning from text generation: a frozen reference encoder and learned connector align conditioning features with the pretrained backbone, while a Text VAE is jointly adapted with the flow model under reconstruction and reference-posterior KL regularization to maintain response decodability. Causal blockwise flow matching enables variable-length responses through autoregressive generation across latent blocks and parallel refinement within each block, with image synthesis treated as a single image-latent block. This formulation supports visual understanding, multimodal reasoning, and instruction-guided editing through interleaved reasoning–image generation. Instantiated on Flux2-9B and Z-Image, CoUPLe supports both MM-DiT and single-stream DiT architectures. Moreover, we find that training CoUPLe with both generation and understanding losses achieves better understanding performance than training with understanding loss alone, demonstrating that generation can help understanding in our settings.

Primary area:
generative models
Keywords:
Unified MultiModel, Image and text generation, Diffusion and FlowMatching, Discrete Diffusion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_76VtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CoUPLe provides a unified framework for adapting pretrained text-to-image flow models to visual understanding, image generation, and image editing. The framework integrates decodable continuous text representations, distinct interfaces for text conditioning and text generation, and causal blockwise flow matching. Evaluations on Flux2-9B and Z-Image, encompassing extensive benchmark comparisons and ablation studies, substantiate the effectiveness of latent adaptation and blockwise generation, while demonstrating positive transfer from image generation to visual understanding.

Critical strengths
  • The separation of text conditioning from text generation addresses a concrete compatibility problem when adapting pretrained image generators, making the approach technically well motivated.
  • Validation across two backbone architectures and multiple understanding, generation, and editing tasks demonstrates broad applicability, with ablations supporting key design choices.
Critical weaknesses
  • Direct comparisons with the closest continuous multimodal methods, particularly GenFirst and OmniFlow, are missing. This limits assessment of the incremental contribution beyond existing continuous and blockwise formulations.
  • Training-exposure accounting and matched-budget controls are insufficiently specified. Consequently, performance gains and generation-to-understanding transfer cannot be cleanly attributed to the proposed modeling choices.
Questions for authors
  • How do the reported dataset sizes reconcile with the sample exposure implied by update counts and batch sizes? Are the joint and text-only controls matched in data exposure and compute?
  • What attention structure does the Text VAE encoder use, and can preceding clean latent blocks contain information from future response text?
  • What evidence establishes the incremental benefit over the closest continuous multimodal methods under comparable training conditions?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 76Vt (1/3)

We thank the reviewer for recognizing the motivation behind separating text conditioning from text generation, and for appreciating the validation across architectures, tasks and design ablations.

W1 Comparisons with the closest continuous multimodal methods.

We thank the reviewer for requesting these comparisons. We have expanded Tables 1 and 3 with GenFirst, LLaDA2.0-Uni, and additional OmniFlow evaluations, using the same benchmark protocol as CoUPLe. GenFirst is concurrent work whose paper was not publicly available at the time of our original submission.

Methodological advance and importance. CoUPLe addresses a central challenge in extending pretrained image generators to multimodal understanding: the representations used to condition image synthesis must remain compatible with the pretrained backbone, while text-generation targets must support continuous modeling and reliable decoding. CoUPLe separates these roles through a fixed conditioning encoder and an adaptive text-target encoder, connects the conditioning interface to the pretrained backbone through alignment, and jointly adapts the text-generation latents with reconstruction and reference-posterior regularization. This enables pretrained visual flow backbones to acquire causal blockwise language generation and multimodal reasoning while retaining image-generation capabilities.

This contribution differs from the objectives and designs of the closest methods. GenFirst develops stable end-to-end latent learning and trains its unified block-causal generative prior from scratch. OmniFlow extends SD3 to any-to-any generation using a text VAE. CoUPLe establishes how causal blockwise text generation and adaptive text representations can be integrated into pretrained visual flow backbones, with the conditioning and decoding requirements explicitly separated and validated. We demonstrate this approach across both MM-DiT and single-stream DiT architectures through Flux2-9B and Z-Image.

Please also see our response to Reviewer JTnh W1/Q1Methodological contribution for additional discussion on the technical contributions of CoUPLe and controlled validation of CoUPLe’s encoder separation and alignment components.

Direct empirical comparison.

ModelMMMU ↑MathVista ↑MME ↑GenEval ↑DPG-Bench ↑
OmniFlow39.3422.211111.00.6766.93
Show-o2 (7B)48.90--1620.50.7686.14
LLaDA2.0-Uni50.1068.101422.00.8987.76
GenFirst42.2961.291312.00.890084.68
CoUPLe (Flux2-9B)67.5978.451688.00.8889.88
CoUPLe (Z-Image)68.1980.121712.00.8990.29

Both CoUPLe variants outperform the compared baselines on MMMU, MathVista, MME-P, and DPG-Bench. Relative to GenFirst, CoUPLe (Z-Image) improves MMMU by 25.90 points, MathVista by 18.83 points, and DPG-Bench by 5.61 points, while matching its GenEval score. These results demonstrate substantial improvements in understanding alongside strong image generation.

The expanded comparisons appear in Tables 1 and 3, and the controlled encoder and alignment ablations are provided in Appendix H.6 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 76Vt (2/3)

W2/Q2: Training exposure.

Both full CoUPLe and the understanding-only control reported in the paper use the hardware configuration and training duration. We report the total non-padding text tokens processed during each training stage. These counts include repeated presentations of training examples and therefore measure cumulative training exposure rather than unique corpus tokens.

Training stageNon-padding text tokens, including repetitions
Stage I34B
Stage II95B
Stage III3B

Controlled evaluation. We separately disable the Stage-I image-generation loss, the later image-generation losses, and image-generation losses throughout training. These comparisons isolate the contributions of generation supervision at different stages. For the understanding-only control, we disable image-generation losses throughout Stages I–III and allocate the saved budget to additional understanding-training updates. This tests whether longer understanding-only training can recover the benefit of joint image-and-text training.

ConfigurationStage-I image lossLater image lossesMMMU ↑MathVista ↑MME ↑
Full CoUPLeOnOn68.1980.121712
No Stage-I image lossOffOn66.0176.321677
No later image lossesOnOff65.2274.891647
No image loss in any stageOffOff63.4272.001609
Understanding-only, hardware-and-time matchedOffOff63.9874.021643
Full CoUPLe minus matched control——+4.21+6.10+69

Removing generation supervision in either Stage I or the later stages lowers performance on all three understanding benchmarks. Additional understanding updates recover part of this loss, but full CoUPLe still exceeds the hardware-and-time-matched control by 4.21 percentage points on MMMU, 6.10 percentage points on MathVista, and 69 points on MME-P. These results support a benefit from generation supervision that additional understanding-only training does not recover within the same resource budget.

We have added these results in the revised manuscript in sec 4.2

Q2: Text VAE.

The Text VAE encoder uses causal self-attention, so each text representation depends only on the current and preceding tokens. Consequently, preceding clean latent blocks encode only the corresponding response prefix and cannot contain information from subsequent response blocks. The backbone additionally enforces block-causal attention: each noisy response block can attend to the conditioning context, preceding clean response blocks, and its own noisy positions. Its clean counterpart and future response blocks are inaccessible. Together, these restrictions prevent future-response information from leaking through the preceding clean latent blocks.

AuthorsOpenReview ↗

Response to Reviewer 76Vt (3/3)

Q3: Evidence beyond the closest continuous multimodal methods under comparable training conditions. We thank the reviewer for requesting comparisons under comparable training conditions. We fine-tuned GenFirst and OmniFlow on the same training dataset used for CoUPLe, with the same hardware configuration and training duration. This matches the additional adaptation budget across methods. All models were evaluated under our common benchmark protocol.

The table reports baseline performance before and after adaptation, alongside CoUPLe.

ModelTraining settingMMMU ↑MathVista ↑MME ↑GenEval ↑DPG-Bench ↑
OmniFlowBase checkpoint39.3422.2111110.670066.93
GenFirstBase checkpoint42.2961.2913120.895784.68
OmniFlowMatched adaptation44.0530.1111920.721172.04
GenFirstMatched adaptation49.1768.2214120.900387.31
CoUPLe (Flux2-9B)Matched adaptation67.5978.4514880.880089.88
CoUPLe (Z-Image)Matched adaptation68.1980.1215120.890090.29

Both baselines benefit from adaptation to our dataset. Under the matched adaptation budget, CoUPLe achieves stronger understanding performance across all three benchmarks and the highest DPG-Bench score among the compared models. Compared with adapted GenFirst, CoUPLe (Z-Image) improves MMMU by 19.02 percentage points, MathVista by 11.90 percentage points, and DPG-Bench by 2.98 points. GenFirst achieves the highest GenEval score. These comparisons support the empirical benefit of CoUPLe’s adaptation framework and complement the encoder-separation and alignment ablations that isolate its specific design choices.

We have added this results in the Appendix H.8 of the revised manuscript

We appreciate the reviewer’s constructive feedback, which has helped us substantially strengthen the paper. The new comparisons demonstrate substantial understanding gains over the closest continuous multimodal methods, while the controlled ablations establish the benefits of generation supervision, encoder separation, and conditioning alignment. We respectfully ask the reviewer to consider these new results and the stronger evidence supporting CoUPLe’s contributions.

Reviewer_JTnhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CoUPLe, a unified multimodal framework based on continuous Flow Matching that extends pretrained text-to-image (T2I) models to visual understanding, image generation, and editing. The authors identify two main limitations of existing approaches. First, unified multimodal models typically use different generative processes for text and images. Second, existing fully continuous approaches face challenges in decodable text representations, causal generation of long sequences, and support for general multimodal tasks.

To address these challenges, the authors introduce several designs. First, they reuse a pretrained CoLA Text VAE to represent text in a decodable continuous latent space. A frozen reference text encoder and a trainable Connector align text-conditioning features with the pretrained T2I backbone, while the target Text VAE and Flow Backbone are jointly adapted. Second, the authors adopt Causal Blockwise Flow Matching, which generates text autoregressively across blocks and performs parallel denoising within each block. Images are generated as single latent blocks, allowing both modalities to share the same Flow Backbone.

The authors further develop a three-stage training strategy consisting of bidirectional image-text pretraining, visual instruction tuning, and Interleaved Training that generates a text Rationale before using it to condition edited-image generation. Experiments are conducted using two pretrained backbones, Flux2-9B and Z-Image. The model is evaluated across multimodal understanding, image generation, editing, and multi-image reasoning benchmarks, achieving competitive results.

Critical strengths

1. A valuable exploration of unified continuous multimodal generation that effectively leverages pretrained visual generation models.

Existing unified multimodal models commonly combine autoregressive text generation with image Diffusion or Flow Matching, using different generative objectives for the two modalities. CoUPLe explores a unified Continuous Flow Matching framework for both text and image generation, allowing the two modalities to share a common generative process. This is a valuable research direction and provides an alternative to mainstream hybrid multimodal architectures.

The authors also build upon strong pretrained T2I backbones, making effective use of existing visual generation priors and reducing the need to train a unified multimodal model from scratch. The experiments demonstrate that this approach can extend pretrained image generators with visual understanding, reasoning, and editing capabilities while maintaining strong image generation performance. These results highlight the practical potential of the proposed framework.

2. Extensive experimental evaluation, competitive results, and substantial ablation studies.

The authors evaluate CoUPLe on two pretrained backbones with different architectures, Flux2-9B and Z-Image, across a wide range of tasks, including visual question answering, mathematical reasoning, image generation, image editing, and multi-image understanding. CoUPLe achieves strong results among the selected baselines, including 80.12 on MathVista, 90.29 on DPG-Bench, and 4.64 on ImgEdit Overall.

The paper also provides extensive ablations covering training strategies, loss functions, Blockwise Generation, and different visual representations. These experiments support the effectiveness of the overall framework and several important components. In addition, the authors investigate whether generative training can improve visual understanding, providing useful experimental evidence and insights into unified multimodal learning.

Critical weaknesses

1. Limited methodological novelty and insufficient validation of the independent contributions of the proposed designs.

Several core components of CoUPLe, including the Text VAE, Causal Blockwise Flow Matching, and joint latent-prior training, have already been explored in prior work such as CoLA-DLM [1]. Unified continuous generation frameworks have also been studied in earlier work. The main methodological contributions of CoUPLe therefore lie in adapting and integrating these existing techniques into pretrained T2I backbones through text-conditioning alignment and multistage training.

While this integration has practical value, the paper introduces relatively few new core mechanisms. Moreover, although the experiments provide strong evidence for the effectiveness of the overall framework, some distinctive design choices lack direct comparisons that isolate their contributions. For example, the alignment-loss ablation demonstrates the benefit of text-conditioning alignment, while the necessity of using separate text encoders for conditioning and generation remains unclear without a direct comparison against a shared-encoder design.

As a result, the extent of the methodological advances beyond the integration and adaptation of existing methods remains somewhat unclear.

Questions for authors

Q1. What are the main methodological contributions beyond existing approaches?

CoLA-DLM [1] has already introduced a decodable continuous Text VAE and Causal Blockwise Flow Matching. GenFirst has also explored bidirectional text-image generation under a shared continuous generative framework, while FullFlow has studied extending pretrained T2I Flow Backbones to bidirectional vision-language generation. The main contribution of CoUPLe appears to lie in adapting and integrating these techniques into pretrained T2I backbones. Could the authors clarify which core technical mechanisms are newly introduced in this work and what specific advantages they provide over existing methods and their direct combinations? In particular, could the authors provide a comparison between a single shared Text Encoder and the proposed separate encoders for conditioning and generation? This would help establish the specific benefit of the proposed representation separation.

Q2. Baseline selection, evaluation comparability, and SOTA claims

The paper reports competitive results across multiple benchmarks. However, some existing unified multimodal models have reported higher scores on several of these benchmarks. For example, UniDFlow [2], published at ICML 2026, reports 74.3 on MMMU, 85.9 on MathVista, 0.95 on GenEval, and 91.19 on DPG-Bench, compared with 68.19, 80.12, 0.89, and 90.29 for CoUPLe, respectively. In addition, Emu3.5 reports an Overall Score of 7.59 on the GEdit-Bench-EN Full Set, compared with CoUPLe's best score of 6.87. These works may use different training settings and evaluation protocols. Could the authors explain why these relevant baselines were omitted and clarify the scope of the paper's SOTA claims? Furthermore, some reported baseline scores differ from those published by the original models. For example, the official BAGEL [3] repository reports an MME score of 2388, while Table 1 reports 1312. Similarly, MammothModa-2 [4] officially reports a GenEval score of 0.87, while Table 2 reports 0.81. Could the authors clarify whether these differences result from different model versions, evaluation subsets, scoring definitions, or inference settings? Were the baseline results reproduced under a unified evaluation protocol, or taken from existing publications?

References [1] Continuous Latent Diffusion Language Model (CoLA-DLM) [2] Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matching (UniDFlow) [3] Emerging Properties in Unified Multimodal Pretraining (BAGEL) [4] MammothModa2: A Unified AR-Diffusion Framework for Multimodal Understanding and Generation

4 comments
AuthorsOpenReview ↗

Response to Reviewer JTnh (1/2)

W1/Q1: Methodological contribution.

We thank the reviewer for highlighting the importance of clarifying our contributions and isolating the proposed design choices. CoUPLe contributes an adaptation framework that extends pretrained T2I flow backbones to unified multimodal understanding and generation. Building on the Text VAE, causal blockwise flow matching, and joint latent-prior learning explored in CoLA-DLM, our design addresses the compatibility between pretrained image-generation conditioning and newly learned text-generation targets. Specifically, CoUPLe retains a frozen text-conditioning encoder while adapting a separate encoder for decodable text-generation targets during Stage I. Conditioning alignment connects this interface to the pretrained visual backbone, and multistage training extends its capabilities to instruction following and interleaved rationale–image generation. We demonstrate this adaptation across the Flux2-9B and Z-Image backbone architectures.

Contributions relative to GenFirst and FullFlow. GenFirst is concurrent work that jointly learns continuous text–image latent representations and subsequently trains a fresh multimodal prior from scratch. CoUPLe instead adapts an existing pretrained T2I flow backbone, addressing how to introduce decodable continuous text generation while maintaining compatible conditioning for image synthesis. FullFlow also adapts pretrained T2I backbones, but generates text through discrete insertion; CoUPLe models both text and image targets through continuous velocity prediction.

Our methodological contribution addresses the different requirements of text conditioning and text generation: a frozen conditioning encoder and aligned connector anchor the conditioning pathway to the pretrained backbone, while a separate target encoder and decoder adapt jointly with the flow model under reconstruction and reference-posterior regularization. This separation allows text-generation representations to adapt without simultaneously changing the conditioning encoder’s representations. Its importance is supported by the shared-versus-separate encoder comparison below: even with alignment enabled, separation improves both understanding and image generation. These results identify a concrete design principle for extending pretrained visual generators: conditioning and generation targets benefit from different adaptation constraints and establish the benefit of our design beyond alignment alone.

We have expanded our comparison in Tables 1 and 3 to include OmniFlow, LLaDA2.0-Uni, and GenFirst, alongside both CoUPLe variants. GenFirst is concurrent work whose paper appeared after our initial submission. FullFlow’s published results overlap with our evaluation only on MS-COCO captioning and VQAv2, and its implementation has not yet been released.

Independent validation of encoder separation and alignment. To directly address the requested comparison, we evaluate shared and separate text encoders, each with and without conditioning alignment, using Z-Image. In the shared configuration, one encoder supplies both image-generation conditioning and text-generation representations.

SettingAlignmentMMMUMathVistaGenEvalCOCO-FID
Full CoUPLe — separate encodersOn68.1980.120.897.75
Separate encodersOff67.1278.890.8310.68
Shared encoderOn65.2173.980.7812.21
Shared encoderOff64.3172.230.7514.44

Encoder separation improves understanding and image generation under both alignment settings. Alignment also improves both encoder configurations. In particular, the comparison with alignment enabled directly establishes the benefit of separate encoders beyond alignment supervision alone. These results provide direct evidence for the distinct contributions of encoder separation and conditioning alignment within CoUPLe’s adaptation framework.

We have added this results in the Appendix H.6 of the revised manuscript

AuthorsOpenReview ↗

Response to Reviewer JTnh (2/2)

Q2: Clarify omitted baselines, the scope of SOTA claims, and whether baseline score discrepancies reflect model versions, evaluation protocols, or reproduced versus published results.

We thank the reviewer for identifying these issues. We have clarified the scope of our performance claims, expanded the baseline comparison, and corrected the MammothModa-2 GenEval entry. We prioritized baselines with comparable reported training-token budgets. UniDFlow reports approximately 3.6T training tokens, which differs substantially from our multimodal training budget, where as Emu3.5 is a substantially larger 34.1B-parameter model than our backbones and compared baselines.

These differences motivated the original comparison scope. Our comparative claims are now scoped to the evaluated models and the stated training and evaluation conditions. Accordingly, we have replaced the broad performance claim in the Introduction with qualified statements about the highest scores as comparisons among the evaluated models. There was a minor typographical error in the MME scores for both CoUPLe (Z-Image) and CoUPLe (Flux), as well as an entry for BAGEL, in Table 1. We have corrected these values in the revised manuscript.

To broaden the comparison, we have included Show-O2, OmniFlow, LLaDA2.0-Uni and GenFirst alongside both CoUPLe variants. GenFirst’s paper was not publicly available at the time of our original ICLR submission, which explains its omission from the initial comparison.

ModelMMMU ↑MathVista ↑MME ↑GenEval ↑DPG-Bench ↑
OmniFlow39.3422.211111.00.6766.93
Show-o2 (7B)†48.90--1620.50.7686.14
LLaDA2.0-Uni50.1068.101422.00.8987.76
GenFirst42.2961.291312.00.890084.68
CoUPLe (Flux2-9B)67.5978.451688.00.8889.88
CoUPLe (Z-Image)68.1980.121712.00.8990.29

MammothModa-2 correction. We thank the reviewer for identifying this discrepancy. The correct GenEval score for MammothModa-2 is 0.87, as reported in Appendix Table 13. The 0.81 entry in Table 2 was a typographical error, which we have corrected in the revised manuscript. The baseline results in Tables 1 and 2 were reproduced under our unified evaluation protocol rather than copied directly from the original publications.

Regarding MME, Table 1 reports our reproduced MME-P results, which measure perception. MME-S combines perception and cognition. We additionally evaluated CoUPLe on MME-S, with the reported results summarized below.

MethodMME-S ↑
BAGEL2388
MammothModa-22000
FUDOKI1711
CoUPLe2409

We will explicitly distinguish MME-P and MME-S in the revised Table 1 and identify the evaluated checkpoints and inference settings. Our comparative claims are scoped to the evaluated checkpoints under the reported protocol.

We have updated Table 1 from the revised manuscript and added the MME-S scores.

We appreciate the reviewer’s constructive feedback, which has helped us strengthen the paper. The additional controlled ablations isolate the benefits of encoder separation and conditioning alignment, while the expanded comparisons provide stronger evidence for CoUPLe’s effectiveness across two pretrained T2I backbones. We respectfully ask the reviewer to consider these new results and clarifications.

Dear Reviewer JTnh,

Thank you again for your thoughtful review. We would appreciate your feedback on our rebuttal and revised manuscript. Our main responses include:

  1. Methodological contribution: Clarified CoUPLe’s adaptation design and its distinctions from CoLA-DLM, GenFirst and FullFlow.
  2. Direct validation: Added shared-versus-separate text encoder comparisons with alignment enabled and disabled, supporting the contributions of both design choices.
  3. Comparisons and reporting: Expanded baseline coverage, qualified our performance claims, corrected the MammothModa-2 GenEval entry, and distinguished MME-P from MME-S.

We would be grateful for your updated assessment in light of these additions. Please let us know if any concerns remain or further clarification would be helpful.

Dear Reviewer JTnh,

If you have any further questions, concerns, or suggestions, please do not hesitate to reach out. We would be delighted to engage in a constructive discussion to enhance the quality and impact of our work.

Thank you once again for your time and thoughtful review.

Reviewer_DCQvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper turns pretrained text-to-image flow models into unified models in which one continuous velocity field generates both text and images. Text is encoded with the CoLA Text VAE and generated block by block with causal flow matching; images are generated as a single block. After three training stages, it reports the best unified-model results on understanding, generation and editing , and claims that generation training helps understanding.

Critical strengths

An interesting question with a sensible design. The paper tests whether a pretrained T2I backbone can become a unified model under one continuous objective, without an LLM and without quantizing images. Thorough method details and broad architectural coverage, with the experiments covering both major diffusion backbone designs, MMDiT (FLUX.2-9B) and single-stream DiT (Z-Image).

Critical weaknesses

The paper's headline claims are relative gains over "the strongest unified baseline", so the baseline results decide whether those claims hold. Several of them do not match published results. For example, The "Show-o2" row (MME 1097, GQA 58.0) actually contains the results of Show-o v1; while Show-o2-7B reports 1620.5 / 63.1 in its released paper. The baseline set is somewhat outdated. Only two of the compared models (OmniGen2 and LaViDa-O) are 2026 work, and several recent unified models are missing, such as LLaDA2.0-Uni. The understanding results are implausible. CoUPLe has no pretrained LLM in its understanding path; the original T2I text encoder only serves as a frozen alignment target during warmup. Yet it reports MMMU 68.2, about 10 points above Qwen2.5-VL-7B. Even a caption-only Stage-I model reaches 59.2. The authors should explain where this knowledge comes from. The "generation helps understanding" claim is the paper's main scientific finding, but the evidence is weak. It needs a stricter control. In the current "text-only" setting, image generation is removed only in Stage I and is still trained in Stages II–III, so the comparison does not isolate the effect of generation.

Questions for authors

First, I hope the authors can address the issues raised in Critical Weaknesses. Second, Appendix D (Table 11) lists 4.5M AnyEdit pairs and 5M examples "selected from the Emu Edit training corpus". However, AnyEdit officially released only 2.5M pairs. Emu Edit only release its test set; its training set was never made public. How were the AnyEdit 4.5M and Emu Edit training data obtained?

9 comments
AuthorsOpenReview ↗

Response to Reviewer DCQv (1/2)

We thank the reviewer for recognizing the value of our research question, the sensible adaptation design, and the thorough method details and evaluation across both backbone architectures.

W1: Baseline accuracy and coverage, the source of CoUPLe’s understanding gains, and stricter controls isolating generation-to-understanding transfer.

We thank the reviewer for highlighting the Show-o naming typo. The corresponding row refers to Show-o. Our expanded comparison includes Show-O2, Omniflow, LLaDA2.0-Uni, and GenFirst, alongside both CoUPLe variants. We note that during ICLR submission, GenFirst had not released their paper.

ModelMMMU ↑MathVista ↑MME ↑GenEval ↑DPG-Bench ↑
OmniFlow39.3422.211111.00.6766.93
Show-o2 (7B)48.90--1620.50.7686.14
LLaDA2.0-Uni50.1068.101422.00.8987.76
GenFirst42.2961.291312.00.8984.68
CoUPLe (Flux2-9B)67.5978.451688.00.8889.88
CoUPLe (Z-Image)68.1980.121712.00.8990.29

We have updated the Table 1 and 3 from the main paper.

Sources of understanding capability. CoUPLe combines a pretrained visual DiT, semantic image encoder, and CoLA text autoencoder with joint image–text training and subsequent instruction tuning. The instruction mixture in Table 10 covers VQA, OCR, science, mathematics, and rationale-enriched responses. To examine the contribution of DiT pretraining, we compare pretrained and randomly initialized DiT weights while retaining the other pretrained interfaces and matching the training protocol. The comparison supports the contribution of pretrained DiT initialization at both checkpoints. Figure 12 provides complementary evidence of stronger semantic representations after joint training.

Sources of understanding capability. CoUPLe inherits pretrained visual and language representations through its pretrained visual DiT, semantic image encoder, and CoLA text autoencoder with joint image–text training and subsequent instruction tuning. The instruction mixture in Table 10 covers VQA, OCR, science, mathematics, and rationale-enriched responses. To examine the contribution of DiT pretraining, we compare pretrained and randomly initialized DiT weights while retaining the other pretrained interfaces and matching the training protocol. The comparison supports the contribution of pretrained DiT initialization at both checkpoints.

DiT initializationCheckpointMMMU ↑MathVista ↑
PretrainedStage I, joint training59.2349.13
Randomly initializedStage I, joint training45.4236.06
PretrainedFinal68.1980.12
Randomly initializedFinal55.0964.02

Our initialization ablation directly tests whether DiT pretraining contributes to the reported understanding results. With the other pretrained interfaces and training protocol held fixed, pretrained DiT initialization improves Stage-I MMMU from 45.42 to 59.23 and MathVista from 36.06 to 49.13. These gains occur before instruction tuning, directly addressing the reviewer’s concern about the caption-trained Stage-I model. Subsequent instruction tuning adds explicit supervision for VQA, OCR, science, mathematics, and reasoning.

Moreover, our additional experiments presented in Figure 12 provide complementary evidence of stronger semantic representations after joint training: with the backbone frozen, peak probing accuracy reaches approximately 69% for Flux2-9B and 61.23% for Z-Image. Joint multimodal training increases these peaks to approximately 74% and 75.28%, respectively.

Generation helps understanding. To address the requested stricter control, we disable image-generation losses throughout all three stages while retaining text-response and rationale training. We also evaluate an understanding-only model that uses the same training-compute budget through additional understanding-training updates. These comparisons support the benefit of generation supervision beyond additional compute devoted to understanding.

ConfigurationStage-I image lossLater image lossesMMMU ↑MathVista ↑MME ↑
Full CoUPLeOnOn68.1980.121712
No Stage-I image lossOffOn66.0176.321677
No later image lossesOnOff65.2274.891647
No image loss in any stageOffOff63.4272.001609
Understanding-only, hardware-and-time matchedOffOff63.9874.021643
Full CoUPLe minus matched control——+4.21+6.10+69

Removing image-generation losses from all three stages reduces MMMU from 68.19 to 63.42 and MathVista from 80.12 to 72.00. Allocating the same hardware and training duration entirely to understanding improves the control to 63.98 and 74.02, but full CoUPLe retains advantages of 4.21 and 6.10 points, respectively.

AuthorsOpenReview ↗

Response to Reviewer DCQv (2/2)

Dependence on visual input. We evaluate the same questions using correctly paired, blank, and shuffled images while retaining the question text and answer options. The performance differences shows strong sensitivity to the supplied visual input.

Image conditionMMMU ↑MathVista ↑
Correct image68.1980.12
Blank image2.223.33
Shuffled image2.925.21

We have added these results in the revised manuscript in sec 4.2 in the main paper and Fig. 12 in the Appendix.

Q1: Provenance and reported counts of the AnyEdit and Emu Edit training data.

We thank the reviewer for noticing the dataset provenance and counts. Emu Edit. We obtained a training corpus containing approximately 10M examples directly from the Emu Edit team. We curated approximately 5 M examples from this corpus for our training. This access was provided directly by the team, separately from the publicly released test set.

AnyEdit and UniCEdit. We used approximately 2.5M pairs from AnyEdit and 2.5M pairs from UniCEdit. These are separate datasets and should be individually attributed. The corrected breakdown is shown below.

DatasetTraining examples used
PicoBanana-4000.5M
OpenGPT-4o-Image0.08M
AnyEdit2.5M
UniCEdit2.5M
Emu Edit, obtained directly from its team4.5M
UltraEdit2M
Total Dataset12.48M

We have revised Appendix D and Table 13 to reflect these separate sources, their corresponding counts, and the provenance of the Emu Edit training corpus.

We appreciate your constructive feedback and believe the new experiments and clarifications materially strengthen the paper. CoUPLe offers a unified continuous formulation with a technically motivated separation of text conditioning and generation, validated across two backbone architectures and diverse understanding, generation, and editing tasks. Our new controlled experiments further substantiate the benefit of generation supervision for understanding. We hope these additions address your concerns and respectfully ask you to reconsider your score in light of this additional evidence.

Dear Reviewer DCQv,

Thank you again for your careful review. We would appreciate your feedback on our rebuttal, which includes:

  1. Baseline comparisons: Corrected the Show-o identification and added Show-o2, OmniFlow, LLaDA2.0-Uni and GenFirst.
  2. Understanding capabilities: Added pretrained-versus-random DiT initialization controls and clarified the pretrained interfaces and instruction supervision.
  3. Generation-to-understanding transfer: Disabled image-generation losses across all stages and added an understanding-only control with matched hardware and training duration. CoUPLe retains gains of 4.21 MMMU and 6.10 MathVista points.
  4. Data provenance: Corrected the AnyEdit/UniCEdit attribution and documented access to Emu Edit training data directly from its team.

We would be grateful for your feedback and reconsideration of your assessment in light of these additions. We are happy to answer further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.