- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper focus on the joint reasoning of both visual and textual content for MLLMs, and proposes a multimodal reasoning framework L2T, converting tool-produced visual states into vector-quantized visual tokens via a trained visual codebook, which permits the visual tokens to be processed in the same way as textual tokens. And then it trains a model to select the proper reasoning mode. This framework is claimed to be more efficient than tool-calling methods. Experiments cover eight visual-centric and reasoning-centric benchmarks, and different model scales, 3B and 7B Qwen2.5-vl models, and additional OOD physics reasoning benchmarks. The 7B model reports the best average scores among the evaluated benchmarks and compared methods, and shorter generated traces.
This paper involves some relatively old but very solid concepts and technecally sound approaches, altogether making the framework reasonable and effective. And the corresponding results was promising. (By "was" I mean, the employed models and compared methods are not quite up-to-date). Some minor experiments (I suppose they can be solved during rebuttal) against more recent models would make this paper better suitable to be publishe.
Critical strengths
-
The proposed framework is a practical.
- It discretizing intermediate tool outputs via VQ, allowing the model to use ordinary next-token training and policy optimization without external tool execution at inference.
- This helps to bridge visual and textual reasoning into the same space.
-
This paper present statistical evidence to support their claim of efficiency.
- Compared to the selected baselines, the proposed method improves the mean score while generating shorter traces.
- As report under "Average Length of CoT" in Table 2, with rout reward (standard) significantly reduces visual-centric/reasoning-centric CoT length relative to its ablation (w/o route reward); with the fast–slow component (standard) also reduces the length by around 10%. These align with the finding that this method "improves reasoning efficiency by reducing token consumption without sacrificing accuracy" (conclusion took from the paper).
Critical weaknesses
-
The comapred baselines and the used models are relatively out-dated.
- This paper used Qwen2.5-vl, which was released more than a year and a half ago. Although the proposed method seems to be useful on Qwen2.5-vl, it is unclear if the method remains effective during a year's development of VL models.
- If this method is still effective for models such as Qwen3, 3.5, 3.6, and 3.8, this could be viewed as a topic worth investigating, otherwise, the motivation of this paper could be considered as somehow solved during the year and a half.
-
The comparison of similar VQ-base methods are missing and the unique contribution of this paper is unclear.
-
Some prior work already reached the contribution mentioned by this paper (contribution #1 at the end of introduction section): [1] already turns intermediate visual representations into VQ-based perception tokens to form a unified space with textual tokens two years ago; [2] and [3] ([3] is already included in Table 1 though) also use discrete visual tokens for intermediate tool outputs (But they are only preprints, so we can exclude them for this discussion). Throughout the paper, it seems that the codebook is fixed after pre-training the VQ-VT. So authors should compare this pre-trained codebook to similar methods' resulting codebook to state the uniqueness of the codebook trained in this paper, and why they did not use pre-trained codebooks (or directly using the training mechanism) in other methods, and what features garantee the following performance.
-
Follow the last point, the codebook is not updated together with the following training stages. This raises an important question, whether the frozen generated tokens remain meaningfully tied to the visual representations learned by VQ-VT when the reasoning mode might be changing during the stage 2 and 3 training.
[1] Bigverdi, Mahtab, et al. "Perception tokens enhance visual reasoning in multimodal language models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
The followings are preprints, they are only here as examples:
[2] Guo, Ziyu, et al. "ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both." arXiv preprint arXiv:2605.15198 (2026).
[3] Chen, Xiuwei, et al. "Latent Visual States for Efficient Multimodal Reasoning." arXiv preprint arXiv:2606.24233 (2026).
Questions for authors
-
Are there any accuracy-cost joint metric to directly demonstrate that the proposed method has successfuly reached a balance?
-
Can you provide the detailed analysis and statistics of constructed training data? e.g., what's the portion of different tasks, and benchmarks. How are they affect the training results? Will the training collapse if changing the data distribution, and how?
-
After RL, do generated visual tokens still represent meaningful intermediate visual states, or has the model repurposed them as internal reasoning symbols? Appendix A.2 shows image sensitivity, but doesn’t establish that stronger correspondence.
-
Not a question, but just to mention there are some errors when compiling the pdf (line 461 "¡unk¿ tokens")
2 comments
Response to Reviewer eciQ (2)
Q1
The training curves and failure analysis support controlled rather than indiscriminate CoT compression.
-
Stage 3 dynamics (appendix A.7.2, Figure 12). Over 30 training steps, the routing score rises rapidly to around 0.9 (Approximate 5 steps); the response length decreases early and then fluctuates around a stable level rather than shrinking indefinitely (Approximate 10 steps). Notably, response length does not decrease indefinitely but instead stabilizes alongside the accuracy reward.
-
The Theoretical Analysis. As theoretically demonstrated in appendix A.3.1, our method guarantees that correct responses are always preferred over incorrect ones, regardless of their lengths. Consequently, when reducing response length does not affect correctness, our method encourages shorter responses. Conversely, when further compression would negatively affect reasoning accuracy, our method prioritizes correctness rather than blindly minimizing response length. This mechanism enables safe and adaptive response compression while preserving reasoning accuracy.
The reported results favor adaptive length control while preserving substantive reasoning on difficult tasks.
Q2
We provide exact dataset composition, benchmark coverage, training settings, and a restricted-data sensitivity experiment.
-
SFT data (appendix A.7.3, Figure 13a). There are 433,256 SFT samples, of which 235,217 (54.29%) receive visual-token supervision and 198,039 (45.71%) do not. The sources include Thyme-SFT, Zebra-CoT from Monet-SFT, and Mulberry from Vision-R1-SFT.
-
RL data (appendix A.7.3, Figure 13b). There are 96,144 RL samples: 40,048 (41.65%) visual-centered, 35,582 (37.01%) mixed, 14,735 (15.33%) simple, and 5,779 (6.01%) text-centered. The RL sources are Thyme-RL, ViRL39K, and Geo3K.
-
Evaluation benchmarks. Our evaluation covers three complementary groups: Visual-centric perception: V* evaluates fine-grained visual search and detail recognition in high-resolution images. HRBench4K and HRBench8K assess high-resolution visual understanding, requiring models to identify small objects, recognize fine details, and understand spatial relationships. MME-RealWorld-Lite evaluates perception in complex real-world images, including object recognition, counting, spatial understanding, and text recognition. Reasoning-centric understanding: MathVista mini evaluates mathematical reasoning grounded in visual inputs, such as diagrams, charts, and geometric figures. MMMU-Pro-10-option evaluates multidisciplinary knowledge and reasoning through challenging multimodal questions with ten answer options. MMMU-Pro-V uses a vision-only question format, requiring models to extract the question and answer choices from images before solving the problem. MMVet evaluates the integration of multiple capabilities, including recognition, OCR, knowledge, spatial understanding, and mathematical reasoning. SeePhys and SeePhysPro SeePhys and SeePhysPro assess physics reasoning from visual evidence. We use them to evaluate generalization beyond the training distribution. Together, these benchmarks cover visually grounded reasoning, integrated multimodal understanding, and out-of-distribution generalization.
| Method | Three-benchmark average accuracy | Three-benchmark average time (s/sample) | HRBench8K accuracy | HRBench8K time (s/sample) |
|---|---|---|---|---|
| Thyme | 58.53% | 12.6 | 72.00% | 9.6 |
| L2T (Thyme-RL only) | 58.00% | 4.1 | 72.13% | 3.6 |
| L2T (full RL data) | 60.90% | 3.0 | 72.70% | 2.6 |
Sensitivity to RL dataset composition: comparison across three benchmarks and on HRBench8K.
- Distribution sensitivity. Restricting GRPO RL training to Thyme-RL gives 58.00% at 4.1 s/sample, versus 60.9% at 3.0 s/sample with full RL data. Compared with Thyme’s 58.53% at 12.6 s/sample, the Thyme-RL only variant is 3.07× faster with an accuracy change of −0.53 . On HRBench8K it reaches 72.13% at 3.6 s/sample, versus 72.00% at 9.6 s/sample for Thyme.
The restricted-data change affects downstream performance, but we did not observe training collapse in this experiment: the resulting model retains competitive accuracy and efficiency.
Q4
The compilation error has been fixed in the revised manuscript.
Response to Reviewer eciQ (1)
W1
L2T’s gains persist on both the newly evaluated 2B, 4B model and the larger 72B model.
Qwen3-VL-4B. On V*Bench, accuracy changes from 77.0% to 86.4% (+9.4); on HRBench8K, from 72.1% to 74.5% (+2.4); and on MME-RealWorld-Lite, from 47.4% to 53.9% (+6.5).
Qwen3.5-2B. V*Bench accuracy improves from 75.9% to 79.1% (+3.2) and MME-RealWorld-Lite from 49.5% to 53.9% (+4.4).
Qwen2.5-VL-72B. The corresponding results are 75.4% to 81.5% (+6.1) on V*Bench, 69.3% to 71.0% (+1.7) on HRBench8K, and 48.0% to 52.2% (+4.2) on MME-RealWorld-Lite.
Additional model-scale and architecture results.
All evaluated models achieve average accuracy improvements over their corresponding base models.
W2 W3
Our contribution is an RL-optimized discrete visual-reasoning framework, not a claim that the codebook is a universally transferable plug-in.
Comparison with ATLAS variants on V*Bench.
W4 Q3
The generated visual-token positions remain attentive to task-relevant spatial regions after RL training.