- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Multimodal Flow, a unified framework that models language and vision using Flow Matching in continuous embedding spaces. It introduces ordered hyperchunks and a shared chunk-causal Transformer to support both text and image generation under a single training objective. The authors train MF-1 up to 1.6B parameters and demonstrate competitive performance on image generation and visual understanding benchmarks. Scaling experiments, controlled comparisons with discrete and hybrid models, and ablation studies further support the proposed approach.
Critical strengths
-
A promising direction for unified multimodal modeling. The paper explores a promising alternative to existing discrete and hybrid approaches by modeling both language and vision through a shared continuous generative process. Beyond the specific architecture, this work demonstrates the potential of fully continuous modeling as a new direction for unified vision-language models.
-
Strong experimental validation. The authors train models up to 1.6B parameters and evaluate them across image generation and visual understanding tasks. The controlled comparisons with discrete and hybrid architectures under matched training budgets provide useful evidence for the effectiveness of the proposed framework.
Critical weaknesses
Limited insights into scaling and data efficiency. While Figure 4 studies the scaling behavior of MF-1, the comparison with discrete and hybrid modeling paradigms in Table 4 is limited to a single model scale and pretraining budget. It remains unclear whether the advantages of fully continuous modeling persist or grow with increasing model size and training data, and whether this approach offers better data efficiency. Given that the paper proposes a new modeling paradigm, a deeper understanding of these properties would be valuable.
Questions for authors
-
Data efficiency. Table 4 compares different modeling paradigms at a fixed pretraining budget of 50B tokens. Could the authors provide performance comparisons at different training stages? This would help clarify whether Multimodal Flow is more data-efficient than the discrete and hybrid baselines.
-
Scaling behavior. Figure 4 demonstrates scaling trends within MF-1, but it remains unclear whether its advantages over other modeling paradigms persist as model size increases. Do the authors have any evidence or preliminary observations about how these relative advantages change with model scale?
5 comments
Response to Question 2 (and Weakness)
Question 2: Preliminary Observations at Larger Model Scale (3B)
Do the authors have any evidence or preliminary observations about how these relative advantages change with model scale?
We appreciate your interest in how paradigm differences behave at larger model scales. Exploring this question provides valuable guidance for scaling unified continuous models.
Due to compute limits during the rebuttal period, pretraining multiple 3B models from scratch for hundreds of billions of tokens was not feasible. To obtain preliminary evidence at a larger scale, we trained matched 3B versions of Multimodal Flow and Transfusion-style on the same SFT data under identical optimization settings:
| Model (Matched 3B) | POPE ↑ | MMBench ↑ | SEEDB ↑ | GQA ↑ | Avg ↑ |
|---|---|---|---|---|---|
| Multimodal Flow (3B) | 87.27 | 47.59 | 48.84 | 56.68 | 60.10 |
| Transfusion-style (3B) | 80.18 | 32.99 | 24.58 | 56.26 | 48.50 |
At the 3B scale, Multimodal Flow continues to show clear gains across all four VQA benchmarks (+11.60% higher average score). We hypothesize that optimizing a single continuous Flow Matching objective allows added backbone capacity to be shared more coherently across vision and language than jointly optimizing two heterogeneous objectives. While full pretraining at larger scales is still needed for definitive conclusions, both the 150B trajectories and this 3B comparison are encouraging, and we plan to further scale Multimodal Flow in both data and model size.
Thank you again for your insightful comments and constructive engagement with our work. We hope these observations address your questions, and we welcome any further discussion.
Dear Reviewer,
Thank you again for recognizing the promise of fully continuous multimodal modeling and for your thoughtful questions about data efficiency and scaling!
With less than two days remaining in the discussion period, we wanted to briefly follow up on our responses. Your questions motivated us to extend the matched learning curves to 150B pretraining tokens and to conduct a preliminary comparison of matched 3B models trained on the same SFT data.
We would welcome your thoughts on these additional results and would be happy to discuss any further questions or observations.
Thank you for your time and consideration.
Best regards,
The Authors
Response to Question 1 (and Weakness)
We sincerely thank you for the encouraging review and thoughtful questions. We truly appreciate your recognition of fully continuous modeling as a promising new direction for unified multimodal models. We completely agree that understanding data efficiency and model scaling across paradigms is a central question for this line of research, and we are glad to share new experimental observations and thoughts on both aspects below.
Question 1 (and Weakness): Data Efficiency Across Pretraining Stages
Tracking intermediate pretraining checkpoints is very helpful for understanding how data efficiency evolves across modeling paradigms.
Following your suggestion, we extended the controlled comparison of the three matched 1.6B architectures (Multimodal Flow, Transfusion-style, and Chameleon-style) from 50B up to 150B pretraining tokens (40K to 240K steps) under identical data mixtures and optimization schedules:
These trajectories highlight two interesting patterns: