- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper presents Prism, a post-training framework for unified multimodal models that utilizes self-generated, interleaved text-image trajectories. The core mechanism relies on a frozen copy of the base model, acting as an internal judge to evaluate generated text and image chains independently. Based on their identified reliability, Prism dynamically routes target construction: either reconstructs text from accepted images, rewrites the failing modality using the complementary chain, or applies a text-unlikelihood loss when both modalities are deemed unreliable. All rewritten targets are verified prior to fine-tuning. While external assistance is used to construct ~2.4K initial task-prompt seeds, the core judging and rewriting loop is entirely self-contained within the base model. The authors evaluate Prism on SenseNova-U1 and BAGEL under a brief post-training regime across three primary capabilities: image generation, visual understanding, and interleaved synthesis. The empirical study is supported by controlled ablations (e.g., standard self-training, removal of rewriting, randomized routing, and modified loss components) alongside diagnostic analyses on judge reliability, rewrite acceptance rates, and broader capability trade-offs.
Critical strengths
1.The experimental controls are well-designed and convincingly isolate the core contribution. Rather than just showing baseline gains, the comparisons against random routing and standard self-training prove that improvements stem directly from the target-construction strategy rather than compute or action distribution shifts.
2.The empirical evaluation is solid and backed by unusually insightful diagnostic analyses. Testing across backbones like SenseNova-U1 and BAGEL prevents single-architecture bias, while human audits and detailed rejection rates confirm that verification acts as a genuine quality filter.
Critical weaknesses
1.The verification mechanism evaluates modalities independently, which fails to guarantee joint cross-modal consistency. Because text and image checks do not cross-reference the anchor modality, individually accepted chains could still diverge in intermediate details, leaving a core premise of bidirectional teaching unverified.
2.The performance gains appear selective, and small improvements lack statistical uncertainty estimates. Several benchmarks show noticeable regressions (e.g., TIIF and GEdit), while gains on smaller datasets like MMVP lack paired confidence intervals or repeat runs to rule out random variance.
Questions for authors
1.On a sample of accepted rewrites, what fraction is jointly consistent between modalities, and do actual verification prompts access the anchor modality during implementation?
2.Can you provide paired uncertainty estimates or multi-run results to confirm the statistical significance of Prism over random routing?
3.How do you bound your claims regarding general capabilities given the regressions in Appendix B, and do loss ablations shed light on these trade-offs?
5 comments
W2/Q2: Selective gains and paired uncertainty estimates
A. Thank you for the suggestion. Given the limited time and computational resources available during rebuttal, we are unable to conduct repeated training runs. Instead, we provide paired bootstrap 95% confidence intervals for Prism relative to Random Routing on seven benchmarks to address your concern about the stability of smaller improvements.
We perform 20,000 paired, grouped bootstrap resamples with replacement. Each resample uses the same test units for both methods and recomputes the score difference under the original evaluation protocol. For GenEval, we stratify by its six categories and resample prompts, retaining the four images for each prompt. For DPG, we resample prompts and retain their four-image scores. For MME, we stratify by subtask and resample images, keeping the two questions for each image together. For MMBench, we resample original questions, and report row accuracy. The resampling units are multiple-choice questions for MMMU, individual questions for MMStar, and complete question pairs for MMVP.
| Benchmark | Prism | Random Routing | Difference | Paired bootstrap 95% CI |
|---|---|---|---|---|
| GenEval | 89.3 | 87.6 | 1.7 | [0.44, 2.96] |
| DPG | 87.2 | 86.4 | 0.8 | [0.60, 1.00] |
| MME | 2307.5 | 2296.3 | 11.2 | [1.18, 22.22] |
| MMBench | 87.2 | 86.5 | 0.7 | [0.18, 1.20] |
| MMMU | 60.7 | 59.4 | 1.3 | [0.24, 2.36] |
| MMStar | 66.5 | 65.7 | 0.8 | [0.07, 1.53] |
| MMVP | 62.0 | 59.3 | 2.7 | [0.67, 5.33] |
Prism outperforms Random Routing with statistical significance on each of the seven benchmarks under the current evaluation setting, as all individual paired 95% confidence intervals for the score differences exclude zero.
Regarding the regressions on TIIF and GEdit, we acknowledge that Prism involves capability trade-offs, although the absolute decreases are relatively small: 0.011–0.035 on TIIF on a 0–1 scale and 0.06–0.27 on GEdit on a 0–10 scale. As discussed in Appendix B, Prism emphasizes planning and visual state progression conditioned on preceding text and images. In contrast, long prompts and style control in TIIF emphasize satisfying all constraints in a single generation, while GEdit requires preserving input-image pixel details and making local changes. Text rendering also requires character-level accuracy and often benefits from specialized post-training, such as reinforcement learning, to improve stability and correctness. These requirements are not directly covered by the current training objectives, which may limit transfer to these capabilities.
Thanks for the authors’ detailed responses. I have increased the soundness score from 2 to 3 and maintain my initial rating.
W1/Q1: Independent verification and joint cross-modal consistency
A. Thank you for raising this issue. In our implementation, the verification prompts do not access the anchor modality. We adopted this design because, during development, the base model's joint judgments tended to overreport inconsistencies and reject otherwise usable targets. We acknowledge that this choice does not guarantee joint cross-modal consistency.
To address your concern, we conduct a joint audit of 100 t2i and 100 i2t targets that were accepted for training. Three trained reviewers manually examine all 200 sampled trajectories for detailed text-image consistency. In the table below, Base denotes the joint judgment of the frozen base model.
The base model reports substantially more inconsistencies than the human reviewers, suggesting that directly using its joint judgments as a filter may reject some usable targets. Nevertheless, the limitation you identified remains: the human reviewers confirm detailed inconsistencies in 33% of t2i and 15% of i2t samples. These may be related to the base model's limited ability. Despite these residual discrepancies, the downstream experiments and ablations in the paper indicate that the constructed targets can provide effective supervision overall.
Q3: Bounding general capability claims and interpreting loss trade-offs
A. Thank you for encouraging us to clarify the scope of our claims. Our results support joint improvements in understanding, generation, and interleaved generation, while the regressions in Appendix B highlight the limits of these gains. Table 5 further shows that different objectives make different contributions to understanding and generation.Replacing the image-to-text rewriting objective reduces MMVP from 62.0 to 56.7. Replacing the text-to-image rewriting objective reduces GenEval from 89.2 to 88.0. Replacing the both-pass reconstruction objective slightly raises GenEval to 89.4 but lowers four of the five understanding metrics. This supports the different capability emphases of the objectives, rather than suggesting that each objective improves every metric. These results highlight the complementary roles and trade-offs of the objectives. We will clarify these findings and the scope of our capability claims more explicitly in the main text.