acceptodds
Under review as a conference paper at ICLR 2027

Understand, Imagine, then Reason: Thinking with Foresight in Unified Latent Space

Abstract

Multimodal Large Language Models (MLLMs) have made substantial progress in visual understanding, yet reasoning about spatial relationships, object interactions, and state changes remains challenging. Thinking about images primarily operates in text, thinking with images often relies on external tools for visual evidence, and existing latent visual reasoning methods largely focus on static representations. Unified multimodal models offer an opportunity to simulate scene dynamics, yet how to leverage these dynamics for visual reasoning remains underexplored. We propose **Understand-Imagine-Reason (UnI-R)**, a framework that feeds internally simulated future dynamics back into reasoning, forming an understanding, imagination, and reasoning loop within a unified latent space. We first introduce Anchor-Conditioned Dynamic Coverage to select complementary future dynamics beyond the observed static scene, while a Foresight-to-Reasoning Bridge aggregates and integrates them into the reasoning stream. During training, Consistent and Discriminative Foresight Learning promotes consistency across rollouts and discrimination across inputs. Contrastive Future-Grounded Self-Distillation further aligns predictions based on imagined futures with teacher predictions grounded in actual future observations while contrasting them with those induced by counterfactual futures. UnI-R empirically improves visual reasoning across both dynamic-centric and general scenarios, achieving an average gain of 8.76% on BabyVision and an 8.34% improvement on MME-RealWorld, demonstrating the promise of leveraging dynamics for reasoning in unified latent space.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Reasoning, Multimodal Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XNjdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

UnI-R feeds imagined future states into visual reasoning through token selection and a query-based bridge. Two-stage training encourages consistent representations and factual-future grounding. Experiments show gains across dynamic and general benchmarks and multiple backbones.

Critical strengths

The connection between future generation and reasoning is interesting, with substantial gains across tasks and backbones. Component ablations provide useful supporting evidence.

Critical weaknesses

The paper does not clearly establish whether the trained model uses future content at inference. Without controlled removal or replacement tests, the gains cannot be cleanly attributed to the imagined futures.

Questions for authors
  1. What is the end-to-end inference latency and memory overhead relative to the backbone and competing methods, including future generation, ACDC, and F2R? How does accuracy trade off against this cost?
  2. How does ACDC compare with random selection at the same budget, and how does the same bridge perform when reading current-image features instead of future states?
5 comments
AuthorsOpenReview ↗

Response to Reviewer XNjd (Part 1)

We greatly appreciate Reviewer’s insightful comments. We address the concerns in detail below. If you have any further questions, please feel free to raise them, and we will be glad to address them.

W1.

The paper does not clearly establish whether the trained model uses future content at inference. Without controlled removal or replacement tests, the gains cannot be cleanly attributed to the imagined futures.

We thank the reviewer for raising this question. We would like to clarify that Figure 4 and Table 10 of the original paper already examine the contribution of future content by varying the number of future frames at inference using the same fully trained UnI-R model. Disabling future generation (T=0) leaves only limited gains over the backbone. Figure 5 and Appendix Figure 7 additionally provide 12 qualitative examples illustrating how imagined futures inform the model’s answers.

To further address this concern, we have added controlled removal and replacement experiments. For removal, we first generate a fixed T=32 rollout and progressively remove its future representations. As shown in Table below, removing all future content brings performance close to the backbone across all five benchmarks, while retaining imagined futures restores the gains.

Rebuttal Table 1. Impact of future-content removal.

Retained imagined futuresBabyVisionVTBenchV*MME-RealWorldSpatialEval
Cosmos38.5123.8775.9244.4064.69
0 (Remove all)8.7624.5275.9244.6665.02
8 (Remove 24)13.4029.6879.5846.9367.03
16 (Remove 16)16.4930.3280.6351.4067.79
24 (Remove 8)17.0130.3281.6751.5667.57
32 (No removal)17.2731.2981.1652.7468.54

The replacement tests further examine whether the content of these representations matters. With the same UnI-R model and T=32 setting, we apply Gaussian perturbations, disrupt the ordering of future states, or replace them with futures from other samples, random vectors, or current-image features. Every intervention reduces performance on all five benchmarks relative to intact futures. The results support the importance of imagined futures specific to the input.

Rebuttal Table 2. Impact of future-content replacement.

Future representationBabyVisionVTBenchV*MME-RealWorldSpatialEval
Cosmos38.5123.8775.9244.4064.69
UnI-R17.2731.2981.1652.7468.54
+Gaussian perturbation13.4026.4579.0649.6867.40
+Shuffled states7.9924.1976.4446.8064.25
+Sample mismatch7.2224.5274.3544.5264.37
+Random vectors6.1919.0372.7743.5162.11
+Current-image features9.0221.9476.4445.0364.90

The removal tests establish the contribution of future representations, while the replacement tests show that the gains depend on their content. The quantitative controls and qualitative examples demonstrate that UnI-R successfully feeds imagined future information back into the reasoning process at inference.

AuthorsOpenReview ↗

Response to Reviewer XNjd (Part 2)

Q1.

What is the end-to-end inference latency and memory overhead relative to the backbone and competing methods, including future generation, ACDC, and F2R? How does accuracy trade off against this cost?

We thank the reviewer for this question. We add an efficiency evaluation on V*, reporting end-to-end latency, component timings, peak GPU memory, and accuracy for the backbone, competing methods, and UnI-R under different future generation budgets.

Rebuttal Table 3. Inference efficiency and accuracy trade-offs with backbone and competing Methods.

MethodFuture generations (s)ACDC (s)F2R (s)Latency (s)Memory (GB)V*
Cosmos3---0.85981 (1.00x)30.7175.92
+VISER---0.86820 (1.01x)30.9575.92
+ICoT---2.00165 (2.33x)31.1276.44
+Monet---2.03167 (2.36x)31.0978.53
+UnI-R (T=32)0.407490.017000.001871.28503 (1.49x)36.1481.16
+UnI-R (T=16)0.248860.013660.001851.12728 (1.31x)34.8082.20
+UnI-R (T=8)0.180290.010030.001891.04232 (1.21x)33.6280.63

Future frame tokens are generated in parallel, and UnI-R directly uses their hidden states without decoding them into images. The additional operations are lightweight, with ACDC and F2R together taking only 18.87 ms at T=32. Compared with competing methods, it achieves higher accuracy and lower latency.

UnI-R enables flexible trade-offs between accuracy and cost, as shown in Figure 4 and Table 10 of orginal paper. With generation budget T=16, UnI-R achieves a V* score of 82.20 (+6.28) at1.31x latency, while T=8 retains a 4.71 improvement at 1.21x latency. These results demonstrate UnI-R's practical efficiency and flexibility and can be selected according to accuracy, latency, and memory requirements. We anticipate that continued advances in generation acceleration will further improve UnI-R's efficiency and scalability within unified multimodal models.

Q2.

How does ACDC compare with random selection at the same budget, and how does the same bridge perform when reading current-image features instead of future states?

We thank the reviewer for the comment. To evaluate the contribution of ACDC, we compare it against random selection under the same token budget. ACDC is designed to identify future states that provide complementary information beyond the observed scene. As shown below, replacing ACDC with random selection consistently degrades performance across all five benchmarks and still outperforms the baseline, validating the effectiveness of its coverage-driven selection strategy and UnI-R framework.

Rebuttal Table 4. Comparison of ACDC with random selection and future-state features with current-image features.

MethodBabyVisionVTBenchV*MME-RealWorldSpatialEval
Cosmos38.5123.8775.9244.4064.69
Full UnI-R17.2731.2981.1652.7468.54
ACDC → Random Selection14.6927.4278.0149.6867.14
Reading Current-Image Features9.0221.9476.4445.0364.90

We also evaluate the same F2R bridge using current-image features instead of imagined future states. The results show that simply reading the current visual features repeatedly provides little benefit over the backbone and performs substantially worse than using imagined futures. This suggests that repeatedly accessing existing visual information does not provide the complementary evidence needed for reasoning and may introduce redundant or distracting representations. In contrast, imagined future states provide additional information about potential scene evolution, which can be effectively integrated through F2R.

These experiments validate both the motivation behind ACDC and the importance of providing F2R with imagined future states instead of repeatedly reading static visual features.

AuthorsOpenReview ↗

A Gentle Follow-up on Our Responses

Dear Reviewer XNjd,

We hope this message finds you well. We sincerely appreciate your time and the thoughtful feedback you have provided. As the discussion period is nearing its end, with less than two days remaining, we would like to kindly ask whether our responses have adequately addressed your concerns. If any points remain unclear or warrant further discussion, we would be grateful for the opportunity to clarify them.

Thank you again for your time, consideration, and valuable feedback.

Best regards,

The Authors of Paper ''Understand, Imagine, then Reason: Thinking with Foresight in Unified Latent Space''

Reviewer_mG6TOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes UnI-R, a framework that closes the loop between generation and reasoning in unified multimodal models. A diffusion-based generation branch simulates future scene dynamics; Anchor-Conditioned Dynamic Coverage (ACDC) greedily selects complementary latent states from T=32 imagined future frames relative to anchor coverage of the observed image; a Foresight-to-Reasoning Bridge (F2R) aggregates the selected states into summary tokens injected into the final transformer block via gated cross-attention. Training proceeds in two stages: CDFL (response CE + FutureNCE + rollout Self-Align on a 148K SFT mixture) and CFGSD (on-policy self-distillation from an EMA teacher conditioned on factual vs. counterfactual futures from 4K DualityVidQA pairs, using JS-divergence grounding and a contrastive margin).

Critical strengths
  1. Well-motivated and genuinely novel framing. The observation that unified understanding+generation architectures (BAGEL, Cosmos3, Lance) do not automatically let imagined dynamics inform reasoning is astute, and the asymmetric-attention analysis (Eq. 1) crisply identifies why. Closing this loop in latent space—rather than via explicit generated pixels or external tools—is a clean conceptual contribution that distinguishes the work from both "thinking with images" and prior latent reasoning methods (CoVT, LVR, Monet).
  2. Evidence that naive access is insufficient. Table 6 is the most informative experiment in the paper: full attention to generated states catastrophically degrades performance (BabyVision 8.51→3.09), and SFT only partially recovers it. This strongly supports the claim that selection (ACDC) and learned integration (F2R) are doing real work, not merely adding capacity.
  3. Comprehensive empirical evaluation. Gains are consistent across 14 benchmarks, three backbones (Cosmos3-Nano-8B, Cosmos3-Edge-2B, Lance-3B), and survive component ablations (Table 3), loss ablations (Table 4), and hyperparameter sweeps (Fig. 4) showing a broad plateau rather than a narrow tuning optimum.
Critical weaknesses
  1. No inference-cost analysis, and the likely cost is severe. At inference the model must generate T=32 diffusion future frames for every question, then run ACDC over all candidate tokens and the F2R readout. The paper never reports latency, throughput, or FLOPs; Fig. 4 only shows accuracy vs. T and K. For static single-image questions (MMBench, V*, etc.) paying 32 diffusion rollouts per query is a dramatic overhead relative to the backbone, and without wall-clock numbers it is impossible to judge practicality. This is the paper's largest gap.
  2. The mechanism behind gains on static general benchmarks is under-evidenced. Improvements on MME-RealWorld (+8.34) and V* (+5.24) are attributed to "imagined changes in viewing scale" enabling closer inspection, but the only support is a single qualitative case (Fig. 5b, mouse brand). It is unclear what the diffusion branch actually generates for a static question, whether these gains could instead come from the extra training data (148K SFT mixture) or auxiliary losses, and Table 3's "standard SFT" control only partially addresses this since UnI-R's losses act on future representations even for static inputs.
  3. Several inference-time details are unspecified: single vs. multiple rollouts at inference (training uses two), how ACDC behaves when the imagined rollout is degenerate, and the text contains small typos ("dyanmics", "fine-turning").
Questions for authors

See weakness above

5 comments
AuthorsOpenReview ↗

Response to Reviewer mG6T (Part 1)

We sincerely thank the reviewer for thoughtful evaluation and valuable feedback, which we have carefully considered and addressed in the responses below. And if you have any further questions, please feel free to post them, and we would be glad to address them.

W1.

No inference-cost analysis, and the likely cost is severe. At inference the model must generate T=32 diffusion future frames for every question, then run ACDC over all candidate tokens and the F2R readout. The paper never reports latency, throughput, or FLOPs; Fig. 4 only shows accuracy vs. T and K. For static single-image questions (MMBench, V*, etc.) paying 32 diffusion rollouts per query is a dramatic overhead relative to the backbone, and without wall-clock numbers it is impossible to judge practicality. This is the paper's largest gap.

We appreciate the reviewer’s suggestion. We have conducted additional experiments on V*, reporting component-wise latency, overall latency, throughput, and TFLOPs. Future frame tokens are generated in parallel, and UnI-R directly utilizes their latent hidden states without decoding them into images. The additional ACDC and F2R modules introduce minimal computational overhead, requiring only 0.01700s and 0.00187s at T=32. The results are summarized below.

Rebuttal Table 1. Inference cost and accuracy trade-offs with backbone and competing Methods.

MethodFuture Generation (s)ACDC (s)F2R (s)Overall Latency (s)Throughput (sample/s)TFLOPsV*
Cosmos3---0.85981 (1.00x)1.10951.73275.92
+VISER---0.86820 (1.01x)1.06752.78475.92
+ICoT---2.00165 (2.33x)0.47064.35576.44
+Monet---2.03167 (2.36x)0.44760.38778.53
+UnI-R (T=32)0.407490.017000.001871.28503 (1.49x)0.73980.85081.16
+UnI-R (T=16)0.248860.013660.001851.12728 (1.31x)0.85571.80682.20
+UnI-R (T=8)0.180290.010030.001891.04232 (1.21x)0.90864.65380.63

We also appreciate the reviewer's recognition that UnI-R exhibits a broad performance plateau rather than a narrow tuning optimum. Across 14 benchmarks, UnI-R achieves consistent gains with an average improvement of 18.93% over the backbone. Meanwhile, it offers flexible accuracy-costtrade-offs, as shown in Fig. 4, Tab. 10 of original paper, and Rebuttal Table 1 above. With T=16, it achieves 82.20 on V* at only 1.31x latency, while T=8 achieves 80.63 at 1.21x latency. These results demonstrate a favorable balance between broad reasoning improvements and inference cost. We anticipate that advances in generation acceleration will further improve its efficiency and scalability.

AuthorsOpenReview ↗

Response to Reviewer mG6T (Part 2)

W2.

The mechanism behind gains on static general benchmarks is under-evidenced. Improvements on MME-RealWorld (+8.34) and V* (+5.24) are attributed to "imagined changes in viewing scale" enabling closer inspection, but the only support is a single qualitative case (Fig. 5b, mouse brand). It is unclear what the diffusion branch actually generates for a static question, whether these gains could instead come from the extra training data (148K SFT mixture) or auxiliary losses, and Table 3's "standard SFT" control only partially addresses this since UnI-R's losses act on future representations even for static inputs.

We thank the reviewer for raising this question. We would like to clarify that Appendix Fig. 7 of original paper also provides five additional qualitative case studies specifically on static image understanding tasks, beyond the example in Fig. 5(b). In Cosmos3, future generation is conditioned on the input query, allowing imagined representations to emphasize task-relevant objects and visual details even for static questions.

Fig. 4 and Tab. 10 of the original paper provide quantitative evidence that the improvements cannot be attributed solely to additional training data or auxiliary losses. In these ablations, we use the same fully trained UnI-R model, varying only the number of future frames generated at inference. When T=0, the gains on all benchmarks are negligible, whereas generating future representations yields substantial improvements. Since the training data and loss objectives remain unchanged, these results directly demonstrate the contribution of imagined future representations at inference, beyond the benefits of training alone. We put the results of original paper below for clarity.

Rebuttal Table 2. Effect of future representations on static and dynamic benchmarks.

VTBenchV*MME-RealWorldSpatialEval
Cosmos322.5075.9244.4064.69
UnI-R (T=0)23.2275.9244.6665.02
UnI-R (T=8)26.4580.6347.1667.35
UnI-R (T=16)28.8182.2051.2868.08
UnI-R (T=24)29.6882.7251.7567.90
UnI-R (T=32)31.2981.1652.7468.54

The qualitative and quantitative results demonstrate that UnI-R's improvements on static general benchmarks stem from imagined changes in viewing scale, which UnI-R successfully integrates back into the reasoning process to enhance visual understanding.

W3.

Several inference-time details are unspecified: single vs. multiple rollouts at inference (training uses two), how ACDC behaves when the imagined rollout is degenerate, and the text contains small typos ("dyanmics", "fine-turning").

We thank the reviewer for highlighting these implementation details. During training, two imagined rollouts are used to compute the Self Alignment loss, which encourages consistent utilization of imagined future dynamics. This consistency is learned during training, so only a single rollout is required at inference, without additional generation or consistency computation.

Regarding degenerate rollouts, ACDC follows the same greedy coverage selection procedure, prioritizing future states that provide the largest marginal information gain beyond the observed scene and previously selected states. When imagined futures largely repeat the observation or contain incorrect dynamics, their contribution to reasoning can be limited. CDFL and CFGSD are specifically designed to mitigate such issues by encouraging consistent future representations and grounding reasoning in actual future observations. We have added failure cases and corresponding discussions in the revision to clarify this behavior.

We also thank the reviewer for identifying the typo errors, which have been corrected in revision.

AuthorsOpenReview ↗

A Gentle Follow-up on Our Responses

Dear Reviewer mG6T,

We hope this message finds you well. We would first like to sincerely thank you for your detailed, constructive review and positive assessment of our work. As the discussion period is nearing its end with less than two days remaining, we would like to kindly check whether our responses have satisfactorily addressed all of your concerns. If there are any points you would like to discuss, we would be truly grateful for your guidance. Your insights are invaluable to us.

We deeply appreciate the time, care, and thoughtful effort you have devoted to reviewing our manuscript. Thank you once again for your invaluable contribution.

Best regards,

The Authors of Paper ''Understand, Imagine, then Reason: Thinking with Foresight in Unified Latent Space''

Reviewer_wZMxOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary
  • This work presents a training framework that allows multlimodal LLMs to use a shared latent reasoning process for understanding, imagination of future dynamics, and reasoning.
Critical strengths
  1. The idea of integrating vision, language, and imagination of future dynamics into a shared reasoning process is simple, elegant, and useful. In my view, this is well-motivated: existing multimodal models have quite impoverished interfaces and lack flexibility in the manners in which they reason. The domainant paradigms of LLMs and visual world models today have very inflexible reasoning interfaces relative to humans, and this is a topic well worth addressing.
Critical weaknesses
  1. THe paper introduces an architecture and method that seems to involve lots of moving components and complexity, and is generally written in a manner that is difficult to understand. A consequence of this is that the paper introduced several new terms, e.g., ACDC, F2R, CDFL, CFGSD -- that corresponded to new methodology, which made it challenging to understand as well. Each method introduced lots of new math. For these reasons, it took me a long time to understand the new methodology. This is a shame, because the high-level idea of the method is simple, but the implementation does not seem so. For example, they discuss how removing any of the three auxiliary losses reduces performance across all five benchmarks -- so it seems like they had to use lots of different components to get this system to work. This is worrying as the method seems quite hand-crafted, which suggests it is not clear if the method will scale to larger models or different tasks.
  2. The results are not particularly impressive. The paper shows modest gains over existing baselines. At the same time, they make no comparisons to frontier models. This might be important because readers may wish to know if these tasks are still worth working on, or if the tasks have already been solved by existing systems. It was also difficult to understand if their proposed latent reasoning even was useful for tackling their proposed goal: imagination of future dynamics. In their figures (e.g., Fig 5), the examples did not involve this capability. So it's not clear if the method is helping solve the key goal.
Questions for authors

None.

5 comments
AuthorsOpenReview ↗

Response to Reviewer wZMx (Part 1)

We thank the reviewer for the comments. We address the concerns as follows.

W1-1.

THe paper introduces an architecture and method that seems to involve lots of moving components and complexity, and is generally written in a manner that is difficult to understand.

A consequence of this is that the paper introduced several new terms, e.g., ACDC, F2R, CDFL, CFGSD -- that corresponded to new methodology, which made it challenging to understand as well.

Each method introduced lots of new math.

For these reasons, it took me a long time to understand the new methodology.

The paper follows a standard progression from motivation to method formulation and experimental results. We have introduced each method component in its relevant context with an explicit description. The review’s broad characterization of the presentation as difficult to understand does not specify where the exposition or logical progression becomes unclear. Listing component names and noting the amount of mathematics does not identify a concrete ambiguity in these explanations. The assessment leaves unspecified which definitions, equations, or algorithmic steps are difficult to follow. We invite the reviewer to identify these points so that we can address each one directly.

Each component has a distinct role and an explicit description. ACDC uses the current observation as a coverage baseline and selects future states with the largest marginal coverage gain, updating coverage after each selection, as detailed in Algorithm 1. F2R aggregates and integrates future information through Transformer readout blocks, cross attention, and gated residual connections, with the workflow illustrated in Fig. 2 and formalized in Eqs. 4–6. CDFL combines CE supervision with NCE contrastive and L2 alignment losses to make internally generated dynamics useful for visual reasoning (Eqs. 7–11). CFGSD builds on OPSD with contrastive supervision from teacher predictions conditioned on real and counterfactual futures (Eqs. 12–13). These equations provide precise definitions of the operations and training objectives.

W1-2.

This is a shame, because the high-level idea of the method is simple, but the implementation does not seem so. For example, they discuss how removing any of the three auxiliary losses reduces performance across all five benchmarks -- so it seems like they had to use lots of different components to get this system to work.

We find this comment puzzling. Each design choice has an explicitly stated motivation and intuition tied to the high-level idea. The ablations systematically examine whether these choices deliver their intended benefits. A decrease relative to the complete method supports the contribution of each specifical design. Providing comprehensive ablations should not count against a method merely because those experiments demonstrate the benefits of its individual components.

The Reviewer's claim that the "loss components are required to get this system to work” is unsupported by the reported results. No matter which component is removed, the performance is still higher than the baseline (Cosmos3), as explicitly shown in Tables 3 and 4 in the original paper. We welcome further discussion of any specific design choice whose motivation or empirical contribution remains unclear.

W1-3.

This is worrying as the method seems quite hand-crafted, which suggests it is not clear if the method will scale to larger models or different tasks.

UnI-R is designed around a clear objective of incorporating imagined future dynamics into reasoning within unified multimodal models, with ablations validating the contribution of each component.

Original paper's Tab. 5 have already demonstrated the effectiveness of UnI-R across 2B, 3B, and 8B backbones from different model families. We further report gains on Cosmos3-Super-32B in the table below, extending this evidence to a substantially larger model.

Rebuttal Table 1. Performance comparison across model scales and backbone families.

ModelBabyVisionVTBenchV*MME-RWSpatialEval
Cosmos3-Super-32B16.4928.3980.6349.7969.80
+UnI-R29.3838.0685.3456.3573.03
Cosmos3-Nano-8B8.5123.8775.9244.4064.69
+UnI-R17.2731.2981.1652.7468.54
Cosmos3-Edge-2B5.4119.0370.6839.6650.31
+UnI-R10.8226.7778.0147.1154.84
Lance-3B5.6714.8460.7335.2842.78
+UnI-R10.0520.3267.5442.2645.90

The concern about effectiveness across different tasks does not make sense with the reported evaluations. Tab. 1 of the original paper covers five groups of benchmarks centered on dynamics understanding, Tab. 2 includes nine general image understanding benchmarks with different sub-task categories, and Appendix Tab. 7 provides a finer breakdown by sub-tasks. These evaluations span physical interactions, spatial changes, visual tracking, recognition of fine visual details, and understanding of real scenes.

AuthorsOpenReview ↗

Response to Reviewer wZMx (Part 2)

W2-1.

The results are not particularly impressive. The paper shows modest gains over existing baselines.

We do not agree with the characterization of the gains as “modest” considering with the magnitude and consistency of the improvements (as explicitly reported in Tables 1 and 2 of the original paper). Our evaluation covers 12 reasoning methods spanning three paradigms and 14 benchmarks, achieving consistent gains with average relative improvements of 18.93% over the backbone baseline and 10.61% over best competing method. These significant improvements are achieved with limited additional training, using only 152K samples to optimize 2 lightweight modules while keeping the backbone frozen. We present a performance comparison of UnI-R with the backbone and the best competing methods in the table below.

Rebuttal Table 2. Significant performance gains over backbone and best competing methods.

ModelBabyVisionBlock towersMichotteVTBenchOpenWorldQACausalSpatialMME-RW
Backbone baseline8.5153.0619.0223.8735.8038.4444.40
Best competing result10.82 (+2.31)56.19 (+3.13)22.18 (+3.16)27.10 (+3.23)38.20 (+2.40)41.11 (+2.67)49.04 (+4.64)
UnI-R17.27 (+8.76)62.35 (+9.29)26.53 (+7.51)31.29 (+7.42)41.40 (+5.60)43.28 (+4.84)52.74 (+8.34)

W2-2.

At the same time, they make no comparisons to frontier models. This might be important because readers may wish to know if these tasks are still worth working on, or if the tasks have already been solved by existing systems.

We clarify that our objective is to improve the visual reasoning for existing Unified Multimodal Models instead of publishing a new foundation model. The Unified Multimodal Models lack an effective mechanism for feeding imagined future dynamics back into reasoning, making the development of this capability a valuable research direction for improving their understanding of how scenes may evolve. We then compare UnI-R with alternative reasoning methods on shared open backbones across different model families and sizes, enabling a fair assessment of the methods under comparable conditions.

To address whether existing frontier systems have already solved these tasks, we have additionally evaluated models from the GPT, Kimi, and DeepSeek families through their APIs. The results show that these systems still face substantial challenges across the evaluated reasoning tasks, particularly those involving dynamics. These findings demonstrate that the tasks remain challenging for the evaluated systems and reinforce the relevance of improving these reasoning capabilities even in leading models.

Rebuttal Table 3. Comparison with frontier API models.

ModelBabyVisionVTBenchOpenWorldQACausalSpatialMME-RealWorld
Cosmos38.5123.8735.8038.4444.40
+UnI-R17.2731.2941.4043.2852.74
API models
GPT-5.6 Sol52.8447.1057.4061.0670.93
Kimi K341.7543.2452.2059.3172.04
DeepSeek V4.1 Flash39.6936.4553.0056.9865.37

W2-3.

It was also difficult to understand if their proposed latent reasoning even was useful for tackling their proposed goal: imagination of future dynamics. In their figures (e.g., Fig 5), the examples did not involve this capability. So it's not clear if the method is helping solve the key goal.

We are particularly puzzled by this comment. The Reviewer's statement directly contradicts the content of the cited Fig. 5, which presents the input images, imagined continuations, and the corresponding answer corrections achieved by UnI-R. The car example explicitly visualizes imagined motion trajectories. We have also provided additional examples in Fig. 6 (Appendix E) of the original paper . It is therefore difficult to understand the criticism.

Figure 4 of original paper further provides quantitative evidence through the ablation on the number of future frames, showing limited gains when no future dynamics are generated. Together with the benchmark results, this supports UnI-R’s central goal of using internally generated future information to improve visual reasoning.

AuthorsOpenReview ↗

A Gentle Follow-up on Our Responses

Dear Reviewer wZMx,

We hope this message finds you well. We appreciate your time and feedback. As the discussion period is nearing its end, with less than two days remaining, we would like to kindly ask whether our responses have adequately addressed your concerns. Please let us know if you have any further comments or points that require clarification.

Best regards,

The Authors of Paper ''Understand, Imagine, then Reason: Thinking with Foresight in Unified Latent Space''

Comments on the paper

2 comments
AuthorsOpenReview ↗

Rebuttal Summary

We sincerely thank the reviewers and the AC for their thoughtful feedback and careful evaluation. Many of the insights greatly helped us strengthen the clarity, completeness, and empirical depth of the paper. For ease of comparison with the original paper, we have highlighted all newly added content in the Revision in red.

We summarize the responses and the corresponding improvements made in the revised manuscript as follows.

  1. Clarification of the UnI-R framework. We clarified the logical progression of the paper and emphasized that the mathematical formulations and definitions are provided to better explain the workflow and methodological operations of UnI-R (wZMx).

  2. Clarification of experimental results. We clarified the implications of the ablation results and highlighted the broad range of task categories covered by Tables 1 and 2, together with the substantial performance improvements achieved by UnI-R. We further added comparisons with frontier API models, including GPT, Kimi, and DeepSeek. Building upon the existing experiments across 2B, 3B, and 8B models from different model families, we additionally evaluated Cosmos3-Super-32B, with the results included in Revision Table 6 (wZMx).

  3. Clarification of visualization experiments. We clarified that the visualization experiments already demonstrate precisely the imagination of future dynamics that the reviewer considered missing. The original submission Figure 4 and Appendix Table 10 also provide supporting evidence for this capability (wZMx).

  4. Clarification of inference-time details and typos. We clarified that inference requires only a single rollout, while two rollouts are used during training. We also corrected the identified typos and updated the manuscript accordingly (Section 3.3, line 256) (mG6T).

  5. Clarification of ACDC behavior and additional failure cases. We further explained the behavior of ACDC and added failure cases to illustrate situations where the imagined rollout becomes degenerate (Appendix E, Revision Figure 6) (mG6T).

  6. Analysis of computational cost and performance trade-offs. We added an end-to-end efficiency table comparing UnI-R with the backbone and competing methods, providing a direct assessment of its computational cost and performance trade-offs (Revision Table 5) (mG6T, XNjd).

  7. Further validation of the contribution of future dynamics. We explained how the original Figure 4, Appendix Table 10, and visualization results in Figures 5–7 demonstrate that imagined future dynamics are the source of UnI-R's performance gains. We further introduced controlled removal and replacement experiments to provide additional evidence for this contribution (Revision Tables 8 and 9) (mG6T, XNjd).

  8. Additional ablation of ACDC against random selection. We conducted additional experiments replacing ACDC with random selection to further validate the effectiveness of ACDC (Revision Table 3) (XNjd).

We sincerely appreciate the reviewers’ insightful suggestions and the AC’s time and effort throughout the review process. We would be grateful for the opportunity to engage in any further discussion.

AuthorsOpenReview ↗

A Gentle Follow-Up on Our Responses

Dear Reviewers,

We hope this message finds you well. As the discussion period is nearing its end with three days remaining, we would like to kindly check whether our responses have satisfactorily addressed all of your concerns. If there are any points you would like to discuss, we would be truly grateful for your guidance. Your insights are invaluable to us.

We deeply appreciate the time, care, and thoughtful effort you have devoted to reviewing our manuscript. Thank you once again for your invaluable contribution.

Best regards,

The Authors of Paper ''Understand, Imagine, then Reason: Thinking with Foresight in Unified Latent Space''

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.