acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Multimodal Reasoning with Visual Imagination

Abstract

Vision-Language Models increasingly rely on textual reasoning to solve advanced multimodal tasks. Despite its effectiveness, this paradigm faces two fundamental limitations. First, relying on textual reasoning alone can limit perceptual ability in vision-centric questions, where accurate answers depend on interpreting task-relevant visual evidence. Second, applying textual reasoning uniformly across inputs overlooks question-dependent variation in both reasoning modality and depth, often producing verbose and unnecessary traces when little logical inference is required. To this end, we propose **A**daptive **V**isual-**T**extual **R**easoning (AVTR), which introduces visual imagination as a complementary reasoning modality to textual thinking, enabling the model to revisit its visual observations and distill the salient evidence through generative visual reflection. Furthermore, we formulate reasoning-modality selection as a learned routing problem, allowing a single policy to adaptively harness the complementary strengths of visual reflection and textual reasoning in response to the perceptual and inferential demands of the input. Across 16 benchmarks, AVTR improves the average score by 4.5 points over Qwen3.5 baseline, while reducing inference latency by 33%. It also scales effectively into a smaller size backbone, outperforming the evaluated 2B-scale baselines. Our code will be made publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Vision Language Models

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_zPgDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Visual-Textual Reasoning (AVTR), which uses generative supervision to learn question-conditioned continuous visual states and adaptively selects among direct answering, visual imagination, textual reasoning, and their composition. Training comprises visual replay, question-conditioned visual reflection, imagination-guided supervised finetuning, and outcome-guided routing optimization. The visual generator is used only during training; inference directly uses the continuous visual states without generating pixels.

Experiments cover 16 benchmarks spanning image understanding, spatial reasoning, video understanding, and temporal grounding, with evaluations on both 9B and 2B backbones. The paper reports a 4.5-point improvement in average score and approximately 33% lower inference latency relative to the backbone’s textual-reasoning mode.

Critical strengths

1. The method is well motivated and supported by comparisons within the same model. AVTR uses generative supervision to learn question-conditioned visual states while avoiding image-generation overhead at inference. Table 4 compares reasoning routes using the same checkpoint, showing complementary strengths of visual and textual reasoning and that no fixed route is consistently optimal across tasks. This provides direct empirical support for adaptive selection of reasoning modalities.

2. The evaluation is broad and includes informative ablations. Results across diverse image and video tasks, two model scales, and ablations of training stages and reasoning routes collectively support the framework’s effectiveness. Beyond aggregate improvements, the paper examines which tasks benefit from each reasoning modality. These findings support the method’s practical value and are an important basis for my positive recommendation.

Critical weaknesses

1. The contribution of generative supervision is not fully isolated. Table 5 includes pure-SFT and feature-alignment controls, but it is unclear whether these variants use the same relevant training data, Stage-3 optimization, and inference routes as full AVTR. If these conditions differ, the performance gaps may reflect differences in supervision, training, and routing. The results therefore support the complete framework, while the specific advantage of generative supervision over alternative objectives requires a clearer explanation.

2. The efficiency measurement protocol needs further clarification. Figure 5 reports an appealing improvement in both performance and latency, but the hardware, batch size, timing boundaries, and averaging procedure are not sufficiently specified. Appendix B also introduces a fallback to non-thinking predictions when generation reaches the length limit, without clarifying whether the preceding reasoning and additional fallback computation are included in latency. Since efficiency is a central contribution, these details are important for interpreting the conditions under which the reported 33% latency reduction holds.

Questions for authors

1. Contribution of generative supervision: Do the pure-SFT, feature-aligned, and full AVTR settings in Table 5 use the same training data, Stage-3 optimization, and inference routes? If not, please identify the main differences. If results controlling for these factors already exist, please provide them or indicate their location in the paper. Otherwise, please explain the limitations of the current comparison in separating the contribution of generative supervision from other training factors; retraining models is not requested.

2. Efficiency measurements: Please specify the hardware, batch size, timing boundaries, and averaging procedure for the AVTR and backbone textual-reasoning results in Figure 5, and clarify whether both were measured under the same conditions. When the fallback described in Appendix B is triggered, does the reported latency include both the preceding reasoning and the additional fallback computation? Please provide fallback frequencies if they are already recorded; otherwise, clarification of the accounting procedure and its limitations is sufficient, without new evaluations.

3 comments
AuthorsOpenReview ↗

Review Appears to Refer to a Different Submission

Thank you for your review. However, the comments above appear to refer to a different paper and do not match the content or method of our submission. Could you please update the review with comments specific to our paper and revise the score accordingly? We would be happy to respond to any concerns about our work once the correct comments are provided.

Reviewer_zPgDOpenReview ↗

Apology and Correction of the Review

Thank you for pointing this out. You are correct—the review was intended for another submission. I sincerely apologize for the confusion and the rebuttal time this has cost you. I have corrected the review and rating, with clarification requests focused on existing experimental details and no new experiments requested.

AuthorsOpenReview ↗

Response to Reviewer zPgD

Thank you for timely updating your review and for your positive assessment of our work. We appreciate your constructive feedback and address the two concerns below.

W1 / Q1. Contribution of generative supervision

We would like to clarify that the Pure SFT and Feature Alignment variants are compared against our model before Stage-3 RL instead of the final AVTR with adaptive routing. This reference corresponds to the "AVTR w/o adaptive RL (S3)" row in the lower half of Table 5. We report the results side by side here for clarity.

MethodV*HR-4KHR-8KBLINKERQAMMSIVSIRWQAMVBenchVideo-MMELVB
No visual supervision86.982.380.672.147.737.565.375.069.572.168.7
Feature supervision87.482.379.172.548.037.265.175.669.872.969.4
Generative supervision88.486.783.875.950.738.467.976.371.574.469.4

Training data and supervision. All three variants use the same answer-SFT dataset, with the following differences:

  • Pure SFT uses no visual targets, as it does not involve visual supervision.
  • Feature alignment uses the same question-conditioned visual targets as generative supervision and follows the same Stage-1 and Stage-2 training, replacing the generative objective with feature-space cosine alignment.
  • Generative supervision uses the same visual targets and Stage-1/2 training, with an aligned generator providing visual supervision. Stage-0 is a necessary VLM–generator alignment stage for our generative supervision, during which the VLM remains frozen and its reasoning ability is untouched.

Evaluation protocol. All three variants are evaluated before Stage-3 RL, without adaptive routing. Pure SFT directly generates short answers, while feature alignment and generative supervision use a forced Visual route followed by short-answer generation. Therefore, this comparison isolates the effects of different supervision methods without involving Stage-3 optimization or adaptive routing.

Thank you for raising this concern. We will clarify these training and evaluation details in the revision.

W2 / Q2. Efficiency measurement protocol

Thank you for pointing this out. We clarify the efficiency measurement protocol below and will add these details to Appendix B.

Hardware and batching. All methods are evaluated under the same settings, using NVIDIA H100 80GB GPUs, one vLLM engine per GPU, greedy decoding, 8192 new generation token limit, and same visual input budgets. We use continuous batching with up to 32 concurrent sequences per engine and data parallelism across GPUs.

Timing and averaging. The measurement includes multimodal input processing, vision encoding, prefill, decoding, and routing decisions. We calculate the average per-query time cost.

Length limit and fallback. When a thinking trace reaches the 8192 token limit, generation is truncated before producing a final answer. To avoid penalizing accuracy solely due to truncation, we use the corresponding short-answer prediction for reliable evaluation. In these cases, the reported latency includes the full reasoning generation but excludes the additional cost of the short-answer replacement. On average, 7.2% of Qwen3.5 thinking traces and 5.4% of AVTR thinking traces reach this limit.

Reviewer_SYzZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AVTR, which uses learnable visual queries to form question-conditioned continuous visual states trained with generative visual supervision. The method comprises visual replay, visual reflection, imagination-guided SFT, and adaptive routing. At inference, the generator and connector are removed, and a policy selects direct answering, visual reasoning, textual reasoning, or their combination. Experiments cover 16 image and video benchmarks and extend to a 2B backbone. Relative to the textual-reasoning mode of Qwen3.5-9B, the authors report an average improvement of approximately 4.5 points and a latency reduction of approximately 33%.

Critical strengths
  1. The motivation is clear and addresses a practical problem. The paper distinguishes limited visual perception from the need for textual inference and explains why uniformly generating long textual traces may be unhelpful. Selecting the reasoning mode according to the input is a valuable research direction.

  2. Separating generative training supervision from inference computation is appealing. Visual queries are processed in parallel, while the generator and connector are used only during training. Inference directly uses continuous visual states, avoiding pixel generation and sequential generation of latent tokens.

  3. The evaluation is broad and includes smaller models. The 16 benchmarks cover high-resolution images, spatial reasoning, video understanding, and temporal grounding. Results on both 9B and 2B backbones support the practical potential of the framework.

  4. The route and training-stage ablations provide useful diagnostic evidence. Table 4 compares reasoning modes within the same checkpoint, while Table 5 includes Pure SFT, feature alignment, and stage ablations. These experiments help explain the roles of visual supervision and adaptive selection.

Critical weaknesses
  1. More direct evidence is needed that visual imagination preserves the details required to answer. Reconstruction similarity (SSIM) and generated examples do not establish whether the model's internal states retain important details or whether its answers actually use them. Checking these details and how answers change when the relevant information is altered would help address this concern.

  2. The reasoning-route selection policy needs more targeted comparisons. Fixed-route experiments show that different questions benefit from different routes, but it remains unclear how much learned selection improves over simple rules at similar inference latency. The effects of the chosen route costs and of excluding training questions on which all four routes succeed or fail also need clarification.

  3. Whether the original textual reasoning ability is preserved needs clarification. On some tasks in Table 4, AVTR's textual route scores below the original backbone, and the combined route scores below the textual-only route. The paper should more clearly separate the effects of training changes from those of adding visual information at inference, and clarify whether performance on text-only tasks is retained.

Questions for authors

Overall, I am very interested in the significance and motivation of the paper. My primary questions stem from the section on critical weaknesses.

  1. Regarding weakness 1: Beyond reconstruction similarity and generated examples, is there evidence that the model retains the important details needed to answer and actually uses them? How do its answers change when this information is altered or removed?

  2. Regarding weakness 2: At similar inference latency, how much does learned route selection improve over simple rules? How are the results affected by different route costs and by retaining or excluding training questions on which all four routes succeed or fail?

  3. Regarding weakness 3: How do the authors explain the task-specific declines of the textual and combined routes in Table 4? Is there evidence separating the effects of training changes from those of additional visual information, and showing whether text-only task performance is retained?

5 comments
AuthorsOpenReview ↗

Response to Reviewer SYzZ Part [1/3]

Thank you for the positive assessment of the motivation, methodology, evaluation breadth, and existing diagnostic ablations. We address the three weakness/question pairs below.

W1 / Q1. Preservation and use of answer-relevant visual details

Thanks for this valuable suggestion of checking the use of visual states.

In addition to reconstruction similarity and generated examples, we also visualize several examples of the visual imagination in Figure 4,5,9,10 for your reference.

To validate whether the answer actually uses the visual states and whether altering visual states will change the answer, we therefore directly modify the visual state at inference and check how the answers change.

MethodV*HR-4KHR-8KBLINKERQAMMSIVSIRWQAMVBenchVideo-MMELVB
AVTR Direct87.984.782.873.347.037.165.875.870.074.368.5
AVTR Visual88.486.783.875.948.737.469.976.370.374.469.4
Random queries84.880.977.573.346.236.465.373.369.072.967.8
Random sampled image85.983.180.172.547.235.367.475.369.473.366.7

We discuss those three variants below.

  1. Compared with the removing visual states (Direct), the Visual route improves consistently across benchmarks.
  2. When we replace the learned visual queries with random queries, performance drops substantially, in many cases even below Direct.
  3. We also replace the insert visual queries with a randomly sampled image from the input, which leads to a similar drop. This suggests that the improvement comes from the task-relevant information carried by the learned visual state.
AuthorsOpenReview ↗

Response to Reviewer SYzZ Part [2/3]

W2 / Q2. Selection rules and route costs

(1) At similar inference latency, how much does learned route selection improve over simple rules?

The gain from adaptive routing is not simply due to using more expensive routes. At closely matched inference cost, Adaptive achieves 73.23 average score at 8.9s, compared with 70.67 at 9.3s for always-Text and 70.66 at 9.4s for always-Visual+Text. Thus, learned per-question selection provides about +2.6 points over the closest non-adaptive policies at similar latency. It also outperforms Qwen3.5 Thinking by +2.91 points while being substantially faster (8.9s vs. 13.2s).

Model / routeV*HR-4KHR-8KBLINKERQAMMSIVSIRWQAMVBenchVideo-MMELVBAvg.Time(s)
Qwen3.5 Direct86.384.782.570.046.034.757.275.469.172.467.84.5
Qwen3.5 Think88.488.785.172.452.335.054.880.571.875.469.113.2
AVTR Direct87.984.782.873.347.037.165.875.870.074.368.54.3
AVTR Visual88.486.783.875.948.737.469.976.370.374.469.45.1
AVTR Text89.588.384.573.148.236.463.277.172.174.970.19.3
AVTR Visual+Text88.587.883.574.250.737.763.577.470.474.469.29.4
AVTR Adaptive89.589.485.175.254.239.871.081.572.476.970.58.9

(2) How are the results affected by different route costs

For the route costs, we find that performance is stable when the penalties remain small relative to the correctness reward. Varying (C(\textsc{Textual})) from 0.01 to 0.05 gives similar results, while a larger value of 0.10 begins to reduce performance. This is consistent with the cost acting only as a secondary efficiency preference, with correctness remaining the main routing signal.

C(Textual)0.010.020.040.050.1
V*89.090.189.589.089.0
HR-Bench-4K87.489.489.488.887.1
HR-Bench-8K84.084.985.184.983.5
BLINK74.874.075.274.971.4
ERQA53.253.654.253.654.7
MMSI-Bench38.837.539.839.738.9
VSI-Bench70.770.371.070.570.3
RealWorldQA81.481.581.579.978.3
MVBench71.971.472.470.870.6
Video-MME75.875.076.976.275.8
LongVideoBench70.770.170.570.869.8
MLVU80.581.081.780.980.7

(3) How are the results affected by excluding training questions on which all four routes succeed or fail?

For questions where all four routes are correct or all four are wrong, there is no correctness signal for choosing one route over another. They also do not help the GRPO update: if all rollouts within a route receive the same reward, the advantage is zero and there is no content-learning signal. Our implementation already excludes (A_i=0) rollouts for this reason. Filtering out all correct/wrong questions is generally used in GRPO training (such as DAPO) because they provide no effective learning signal.

AuthorsOpenReview ↗

Response to Reviewer SYzZ Part [3/3]

Question 3: Regarding weakness 3: How do the authors explain the task-specific declines of the textual and combined routes in Table 4? Is there evidence separating the effects of training changes from those of additional visual information, and showing whether text-only task performance is retained?

The textual route after AVTR post-training is not expected to reproduce the original Qwen3.5-Think distribution exactly. Training introduces a second reasoning modality through visual imagination, so the model learns to allocate reasoning differently across visual and textual routes. One clear change is that the forced AVTR Text route produces much shorter reasoning traces: averaged over all questions of the 11 benchmarks, generated tokens decrease from 3050 for Qwen3.5 Think to 1191 for AVTR Text route

Model / routeV*HR-4KHR-8KBLINKERQAMMSIVSIRWQAMVBenchVideo-MMELVBAvg (question-weighted)
Qwen3.5 Think (Acc)88.488.785.172.452.335.054.880.571.875.469.1
Qwen3.5 Think (#Tokens)112314741583295538925709472720341960260531323050
AVTR Text (Acc)89.588.384.573.148.236.463.277.172.174.970.1
AVTR Text (#Tokens)3757307251446175023641370473831137213151191
AVTR Visual+Text (Acc)88.587.883.574.250.737.763.577.470.474.469.2
AVTR Visual+Text (#Tokens)4157977491527171422721204512857148213561218
AVTR Adaptive (Acc)89.589.485.175.254.239.871.081.572.476.970.5
AVTR Adaptive (#Tokens)33076674065384876374295421235613516

Similarly, forcing Visual+Text is not expected to dominate Text on every task. The two modalities have different strengths, and adding visual imagination can be unnecessary or even interfere with tasks that favor discrete textual reasoning. This is why our method does not force either Text or Visual+Text universally, but learns to select the appropriate route per question.

The relevant comparison is therefore the learned Adaptive policy against the original Qwen3.5 Think. Adaptive reaches 73.23 vs. 70.32 average accuracy (+2.91 points) across these 11 benchmarks, while generating only 516 vs. 3,050 tokens on average. Thus, post-training redistributes reasoning across visual and textual modalities, and adaptive routing is important for realizing the benefit of this new reasoning space.

Reviewer_zSkqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an adaptive vision-text reasoning framework to address the limitations of visual perception in multimodal large language models and the computational redundancy introduced by fixed reasoning strategies. Specifically, the proposed approach learns task-relevant visual representations through generative supervision and dynamically selects reasoning pathways via an Adaptive Routing mechanism. Experimental results demonstrate that the proposed method improves model performance across multiple multimodal benchmarks while reducing inference latency compared with fixed text-based reasoning.

Critical strengths
  1. The proposed framework integrates visual representation learning with adaptive reasoning strategies, enabling the model to dynamically select different reasoning modes based on task-specific requirements.
  2. The Visual Imagination module does not require explicit image generation during inference, thereby avoiding additional computational overhead. Moreover, the adaptive routing mechanism reduces unnecessary text-based reasoning, further improving inference efficiency.
  3. The authors evaluate the proposed method across multiple multimodal benchmarks and further validate its effectiveness through ablation studies and experiments with different model scales.
Critical weaknesses
  1. The limitations of visual perception in multimodal large language models, as well as the need for task-dependent reasoning strategies, have been extensively discussed in prior studies. The paper does not sufficiently establish the novelty or distinctiveness of the research problems it aims to address.
  2. Compared with using Visual Imagination alone, the Adaptive Routing mechanism provides only marginal performance improvements while introducing substantial additional inference latency. The authors should further justify whether these limited performance gains warrant the increased computational overhead.
  3. The proposed framework primarily integrates existing techniques, including Visual Imagination, Adaptive Routing, and GRPO. However, the paper does not sufficiently demonstrate substantive methodological innovations beyond the combination of established approaches.
Questions for authors

Please refer to the Weaknesses section for my detailed concerns and questions regarding the research motivation, performance–efficiency trade-off, and methodological novelty.

3 comments
AuthorsOpenReview ↗

Response to Reviewer zSkq

Weakness 1: The limitations of visual perception in multimodal large language models, as well as the need for task-dependent reasoning strategies, have been extensively discussed in prior studies.

Existing multimodal LLMs took multimodal inputs but reasoning only in text. Our has significant changes: a multimodal LLM generates different reasoning modalities in the output space, and adaptively chooses the appropriate modality for each question. Latent visual reasoning methods introduce visual states in latent feature space, while our generative reflection learns a question-conditioned visual state in the image-generator space by replaying task-relevant visual evidence. Adaptive methods such as AdaptThink and VideoAuto-R1 only adjust the amount of textual reasoning. Our method instead makes reasoning modality itself adaptive, choosing among Direct, Visual, Textual, and Visual+Textual reasoning.

Our analysis further shows why this distinction matters: visual reasoning is stronger on metric, geometric, and viewpoint-sensitive tasks, while textual reasoning is stronger on temporal ordering, counting, and discrete relational reasoning. This task-dependent complementarity motivates adapting both reasoning modality and reasoning depth, rather than reasoning depth alone.

Weakness 2: Compared with using Visual Imagination alone, the Adaptive Routing mechanism provides only marginal performance improvements while introducing substantial additional inference latency.

  1. The extra latency comes from textual reasoning, not from the router itself. Some questions are simply hard to solve with the Visual route alone. In these cases, the model needs a longer autoregressive reasoning chain to reach the correct answer, so some extra computation is unavoidable.

  2. The performance is not marginal. The gain is larger on the tasks that actually need textual reasoning. Adaptive improves over Visual by 1.7 points on average, but the gains are +5.5 on ERQA, +5.2 on RealWorldQA, +2.5 on Video-MME, and +2.1 on MVBench. The point of routing is to use the more expensive textual route only for these harder cases, instead of paying that cost for every input.

  3. Adaptive is also clearly better than always using Qwen3.5 thinking in both efficiency and accuracy. It reaches 70.08% at 8.9s, while Qwen3.5 with textual reasoning gets 65.54% at 13.2s. In other words, Adaptive is 4.5 points better while being 33% faster. So the extra cost over the Visual-only route is used to recover cases that need textual reasoning, without paying the full cost of always-on thinking.

Weakness 3: The proposed framework primarily integrates existing techniques, including Visual Imagination, Adaptive Routing, and GRPO. However, the paper does not sufficiently demonstrate substantive methodological innovations beyond the combination of established approaches.

We disagree with the characterization of AVTR as an integration of existing techniques. “Visual imagination,” “adaptive routing,” and “GRPO” are broad research directions, while the concrete mechanisms in AVTR are methodologically different from prior work in each case. AVTR introduces multi-modality reasoning in VLM's output space, together with new formulations for generative visual reflection, and modality-level routing. We detail below how each contribution differs from existing work.

  • Visual imagination. Prior latent visual reasoning methods such as LVR, Monet, and VisReflect mainly construct visual states through latent-feature alignment. We instead introduce generative visual reflection: question-conditioned visual queries are supervised in the image-generator space to replay task-relevant evidence from the previously observed visual input. The queries are produced in parallel, directly condition the answer, and require neither the generator nor pixel decoding at inference. To the best of our knowledge, there is no existing literatures that use visual imagination as guidance for VLM to visually reflect salient evidence.

  • Adaptive routing. Existing adaptive reasoning methods mainly adapt textual reasoning depth: AdaptThink selects Thinking vs. NoThinking, while VideoAuto-R1 decides whether to enter textual reasoning based on an initial answer. AVTR instead makes reasoning modality itself adaptive. For every question, we rollout Direct, Visual, Textual, and Visual+Textual routes and learn a correctness and cost aware policy to select or compose different output-side reasoning modalities.

  • GRPO. We do not directly apply standard GRPO. Standard GRPO compares rewards across a sampled group. In AVTR, rollouts come from four explicitly controlled reasoning routes, and we center the reward within each route, rather than across routes, so differences in route quality do not enter the content update. Route selection is optimized separately through our soft routing objective, while GRPO only learns how to reason within the selected route.

Thank you for your response. Your clarifications have addressed my concerns. Therefore, I am willing to increase my score.

Dear Reviewer zSkq,

Thank you for your prompt response and positive assessment of our work. We are glad that our clarifications have addressed your concerns. We sincerely appreciate your valuable feedback and the time and effort you devoted to reviewing our paper.

Reviewer_QWoFOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Adaptive Visual-Textual Reasoning (AVTR), which learns question-conditioned visual states through generative supervision and adaptively selects among direct answering, visual imagination, textual reasoning, and their composition. A visual generator supervises the latent states during training but is removed at inference. Experiments report improvements in both accuracy and inference latency across image and video benchmarks.

Critical strengths
  1. The paper provides a meaningful extension from adaptive reasoning depth to adaptive reasoning modality. The framework allows the model to select visual processing, textual reasoning, or their composition according to the question. This provides a more expressive alternative to uniformly enabling long textual reasoning.

  2. A coherent separation between visual-state learning and routing optimization. Generative supervision gives the continuous queries a concrete visual target, while route-complete evaluation supervises gate selection separately from within-route content optimization. Removing the generator at inference makes the learned representation practically useful without requiring image generation.

Critical weaknesses
  1. Presentation quality. The figures and writing are somewhat rough. Some figures are difficult to interpret without reading the text closely, and parts of the exposition are unclear or inconsistent in terminology and notation. A more careful pass on figure design and writing would make the paper considerably easier to follow.

  2. Limited insight. The proposed method is largely heuristic. The design choices are not well motivated, and the paper offers little principled analysis of why the approach works or when it would fail. As a result, the contribution feels incremental and lacks deeper insight. A clearer hypothesis, a theoretical or empirical analysis of the underlying mechanism, and ablations that isolate the contribution of each component would strengthen the paper.

  3. The specific necessity of generative supervision is not fully isolated. Table 5 includes useful pure-SFT and feature-alignment controls, but the full method also benefits from large-scale reconstruction training and curated visual targets. The evidence does not completely distinguish the proposed generative mechanism from the benefits of additional visual alignment and evidence-focused training.

  4. The routing objective uses a simplified representation of computational cost. Each route receives a fixed cost despite question-dependent variation in textual reasoning length and runtime. Consequently, the learned policy optimizes an approximate route preference rather than an explicit per-example accuracy–latency trade-off. This limits how broadly the efficiency mechanism can be interpreted.

Questions for authors
  1. Under matched training data and computational budgets, does generative supervision outperform reconstruction-free alternatives that learn question-conditioned visual queries from the same evidence targets?

  2. How sensitive is routing to the cost coefficients and the sharpness of the soft routing targets? Would measured or predicted per-example latency improve the trade-off?

  3. The visual targets are primarily salient crops or evidence frames. Which capabilities require “visual imagination” beyond evidence selection and compression, and can the authors clarify this distinction?

4 comments
AuthorsOpenReview ↗

Response to Reviewer QWoF Part [1/3]

Thank you for recognizing our method from adaptive reasoning depth to adaptive reasoning modality, and the separation of visual-state learning from routing optimization. We address the concerns about presentation, mechanism, supervision controls, and computational cost below.

Weakness 1: Presentation quality.

Thank you for the feedback. We are happy to further refine the figures, terminology, notation, and overall exposition in the revision. The comment does not identify which specific figures are difficult to interpret, or which terms or notation appear unclear or inconsistent. If the reviewer could point us to the particular figures, notation, or terminology that caused confusion, we would be glad to address those points directly and carefully. We will in any case perform another full pass over the paper to improve figure readability, terminology consistency, and presentation wherever possible.

Weakness 2: Limited insight.

Our main insight is that visual imagination and textual reasoning have complementary strengths across different question types, rather than one reasoning mode being uniformly better. Figure 3 and Appendix E.2 provide a task-level analysis supporting this hypothesis: visual imagination is stronger on metric, geometric, and viewpoint-sensitive tasks, while textual reasoning is stronger on counting, temporal ordering, and discrete relational reasoning. Figures 9-11 further show corresponding success and failure cases; for example, Figure 11 shows that visual imagination can alter count-relevant details, whereas textual reasoning preserves the correct count. We also provide controlled ablations for both the training design and the routing mechanism. Table 5 isolates each training stage and compares generative supervision against Pure SFT and feature alignment:

Supervision ablations

MethodsV*HR-4KHR-8KBLINKERQAMMSIVSIRWQAMVBenchVideo-MMELVB
no visual supervision86.982.380.672.147.737.565.375.069.572.168.7
feature supervision87.482.379.172.548.037.265.175.669.872.969.4
generative supervision88.486.783.875.950.738.467.976.371.574.469.4

The result shows generative supervision outperforms the feature alignment. Visual imagination guides the model to visually reflect the salience evidence in the given vision context.

Component ablations

MethodsV*HR-4KHR-8KBLINKERQAMMSIVSIRWQAMVBenchVideo-MMELVB
w/o recon-align (S0)84.880.877.170.447.538.261.675.167.170.867.0
w/o target-align (S1)87.486.382.574.353.638.469.580.371.875.368.7
w/o adaptive RL (S3)88.486.783.875.950.738.467.976.371.574.469.4
AVTR89.589.485.175.254.239.871.081.572.476.970.5

Reasoning routes

MethodV*HR-4KHR-8KBLINKERQAMMSIVSIRWQAMVBenchVideo-MMELVB
Direct87.984.782.873.347.037.165.875.870.074.368.5
Visual88.486.783.875.948.737.469.976.370.374.469.4
Textual89.588.384.573.148.236.463.277.172.174.970.1
Both88.587.883.574.250.737.763.577.470.474.469.2
Adaptive89.589.485.175.254.239.871.081.572.476.970.5

These results show that no fixed reasoning route is uniformly optimal and that simply using more reasoning is not necessarily better. This observation directly motivates adaptive routing. We will make this hypothesis and the connection between these analyses more explicit in the revision.

AuthorsOpenReview ↗

Response to Reviewer QWoF Part [2/3]

Weakness 3: The specific necessity of generative supervision is not fully isolated. Table 5 includes useful pure-SFT and feature-alignment controls, but the full method also benefits from large-scale reconstruction training and curated visual targets.

Stage-0 is an alignment stage rather than additional VLM finetuning. The VLM is frozen throughout this stage; only the visual queries, connector, and SANA generator are trained for reconstruction. Its purpose is to align the VLM states with the generator so that the generator can provide meaningful visual supervision in the later stages, rather than to improve the VLM's answer-generation ability. Therefore, feature-aligned control and our generative reflection starts from the same point since stage-1.

The feature-aligned control already matches the two factors raised by the reviewer.

  • The feature-aligned uses the same curated visual targets for evidence-focused training.
  • It goes through the same Stage-1 and Stage-2 training as our generative supervision.

The difference is the supervision objective: feature-space cosine alignment replaces the generative objective.

Question 1: Under matched training data and computational budgets, does generative supervision outperform reconstruction-free alternatives that learn question-conditioned visual queries from the same evidence targets?

The feature-aligned variant is our reconstruction-free control for this comparison. It goes through the same Stage-1 and Stage-2 training with the same question-conditioned evidence targets as our generative variant; the difference is the supervision objective. Feature alignment applies cosine alignment to the visual target, whereas our method uses the aligned generator as generative supervision.

Importantly, Stage-0 should not be viewed as additional VLM pretraining. The VLM is completely frozen during this stage; only the visual queries, connector, and SANA-1.6B generator are trained. Its purpose is to establish the VLM--generator alignment required before the generator can provide meaningful supervision. Our Stage-0 ablation further shows that this alignment is necessary for generative reflection.

Under the matched Stage-1/2 evidence targets and VLM training, generative supervision outperforms feature alignment on 10 of 11 benchmarks and ties on the remaining one:

SupervisionV*HR-4KHR-8KBLINKERQAMMSIVSIRWQAMVBenchVideo-MMELVB
No visual supervision86.982.380.672.147.737.565.375.069.572.168.7
Feature supervision87.482.379.172.548.037.265.175.669.872.969.4
Generative supervision88.486.783.875.950.738.467.976.371.574.469.4

Thus, the gain cannot be explained by the curated evidence targets alone: the feature-aligned baseline sees the same targets but yields only limited improvement over response-only SFT, while generative supervision provides substantially stronger gains.

AuthorsOpenReview ↗

Response to Reviewer QWoF Part [3/3]

Weakness 4: The routing objective uses a simplified representation of computational cost. Each route receives a fixed cost despite question-dependent variation in textual reasoning length and runtime.

We would like to clarify two points about the routing cost.

  • First, the relative cost ordering is example-independent and follows naturally from the computation required by each route: C(Direct) < C(Visual) < C(Textual) < C(Visual+Textual). Direct introduces no additional reasoning, Visual processes a small set of visual queries in parallel, Textual requires sequential autoregressive decoding, and Visual+Textual performs both. Therefore, although the exact length of textual reasoning may vary across questions, the relative computational ordering of the four routes remains unchanged.

  • Second, correctness is prioritized over efficiency. We use small cost values (0, 0.01, 0.04, 0.05) only to distinguish the relative cost of the four routes when their correctness is comparable. With g=4 rollouts per route, the smallest non-zero difference in empirical route reward is 0.25, larger than the maximum cost difference of 0.05. Thus, the cost cannot override an observed correctness advantage and only favors a more efficient route when correctness is similar. Importantly, the cost term is used only for gate learning; route-conditioned GRPO uses answer reward only, without any cost or response-length penalty. Therefore, response length does not enter the content optimization within each route.

The routing objective therefore uses question-dependent correctness as the primary signal, with route cost serving only as a secondary efficiency preference. Figure 5 shows that this design improves the average score by 4.5 points while reducing measured latency from 13.2s to 8.9s.

Question 2: How sensitive is routing to the cost coefficients and the sharpness of the soft routing targets?

The route-level cost is independent of response length and only reflects the inherent computational ordering of the four routes. As discussed above, correctness is the primary routing signal, while the cost only serves as a small secondary preference when routes have comparable correctness. Moreover, the route-conditioned GRPO objective uses only the answer reward and does not include any cost term, so optimization within each route is independent of response length.

C(\textsc{Textual})0.010.020.040.050.1
V*89.090.189.589.089.0
HR-Bench-4K87.489.489.488.887.1
HR-Bench-8K84.084.985.184.983.5
BLINK74.874.075.274.971.4
ERQA53.253.654.253.654.7
MMSI-Bench38.837.539.839.738.9
VSI-Bench70.770.371.070.570.3
RealWorldQA81.481.581.579.978.3
MVBench71.971.472.470.870.6
Video-MME75.875.076.976.275.8
LongVideoBench70.770.170.570.869.8
MLVU80.581.081.780.980.7

The table shows that routing performance is insensitive to small cost penalties relative to the correctness reward. Per-example latency prediction is not required for our criterion: the default assigned cost ordering Direct < Visual < Textual < Both is fixed across questions and independent of response length.

Question 3: The visual targets are primarily salient crops or evidence frames. Which capabilities require “visual imagination” beyond evidence selection and compression, and can the authors clarify this distinction?

We use “visual imagination” to refer to the model internally forming a question-conditioned visual state, rather than generating novel visual content. The supervision is grounded in observed crops or frames because our goal is to replay and retain task-relevant visual evidence for reasoning, not to hallucinate unseen content. Thus, the “imagination” lies in constructing and reasoning through this intermediate visual state in a generative reflection manner. We will clarify this scope in the revision.

Dear Reviewer QWoF,

As the rebuttal period approaches its end, we would like to kindly follow up on our response and check whether our clarifications and additional experiments have addressed your concerns.

We would greatly appreciate any further feedback and would be happy to provide additional clarification if needed.

Best regards,
The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision Summary

Summary of Revisions

We thank the reviewers for their constructive feedback. We have uploaded a revised PDF with all changes highlighted in blue. Below are the main updates and the reviewer comments they address.

Main Paper

  • Sec. 2 (Related Work). Clarified our technical contributions and difference over prior work, particularly generative visual supervision, adaptive multimodal reasoning, and gate learning. [zSkq W1, W3]

  • Sec. 3.1 (Visual Imagination). Clarified that visual imagination revisits task-relevant evidence from the input rather than generating unseen content. [QWoF Q3]

  • Sec. 3.3 (Route Cost). Explained that route costs reflect computational differences independent of input or response length. Small penalties favor cheaper routes without overriding correctness and only affect gate learning. [QWoF W4; SYzZ W2/Q2]

  • Sec. 4.3, Table 4 (Reasoning Routes). Added latency for each route and explained changes in textual reasoning behavior after post-training. [SYzZ W2/Q2, W3/Q3; zSkq W2]

  • Sec. 4.3, Figure 5 (Qualitative Example). Moved a camera-motion example from the appendix to the main paper, showing how visual imagination succeeds where textual reasoning fails. [QWoF W2; SYzZ W1/Q1]

  • Sec. 4.4, Table 5 (Ablations). Clarified the controlled comparison of supervision objectives. All variants use the same instruction data and are evaluated before Stage-3 without adaptive routing. [QWoF W2, W3/Q1; zPgD W1/Q1]

  • Sec. 4.6 (Efficiency). Clarified when adaptive routing benefits from textual reasoning (+5.5 on ERQA, +5.2 on RealWorldQA) and its advantage over fixed routes at similar latency (~2.5 points). Added references to detailed timing and token analyses. [zSkq W2; SYzZ W2/Q2; zPgD W2]

Appendix

  • App. A (Stage-3). Clarified why training excludes questions where all routes succeed or fail, and why route costs cannot override correctness differences. [SYzZ W2/Q2; QWoF W4]

  • App. B (Evaluation). Clarified the supervision-ablation protocol and added latency measurement details, including hardware (H100), batching (vLLM, up to 32 sequences), timing, fallback handling and rates. [zPgD W1/Q1, W2/Q2; QWoF W3]

  • App. D (Ablations). Added route-cost sensitivity (Table 9), showing stable performance under small penalties, and visual-state intervention experiments (Table 10), showing accuracy drops when learned visual states are replaced with random queries or images. Also corrected a typo in Table 7. [QWoF Q2; SYzZ W1/Q1, W2/Q2]

  • App. E (New: Inference Cost and Reasoning Length). Added accuracy, latency, and generated-token comparisons across all reasoning routes (Table 11) to explain changes in reasoning behavior after post-training. [zSkq W2; SYzZ W2/Q2, W3/Q3]

  • App. F–G (Figures). Corrected the caption of Figure 10 and clarified Figure 12 and its latency range. [QWoF W1]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.