- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper aims to solve the limitations of sequential visual latent tokens, especially the weak sensitivity to decisive evidence and limited support for subsequent reasoning. This paper constructs a dataset AVR-112K and proposes a framework called ELVIS that constructs evidence grounded latent visual states in parallel and invokes new states as reasoning progresses. Diverse benchmarks, ablations, and analyses show the effectiveness of the proposed method.
Critical strengths
-
The paper addresses important limitations of existing latent visual reasoning methods. The proposed architecture is well motivated.
-
The experimental evaluation is comprehensive and the component ablations and latent transplantation experiments provide useful evidence for the effectiveness of the method.
Critical weaknesses
The comparison with vanilla SFT is not compute-matched, as ELVIS needs an additional computation budget, such as extra SFT stages. Notably, on Zebra Maze, vanilla SFT already improves accuracy from 15.25% to 35.59%, while the full ELVIS only further improves it to 37.29%. The paper does not provide budget-controlled comparisons with vanilla SFT. Consequently, it remains unclear whether the proposed latent mechanism provides sufficient benefits to justify its additional computational and training complexity.
Questions for authors
-
Could the authors compare against a non-latent VLM trained with the same total computational budget, for example through additional SFT or SFT followed by GRPO? How much of the improvement can be attributed specifically to latent reasoning rather than additional training?
-
Could the authors report end-to-end inference latency and computational cost relative to a standard VLM without latent states? Does ELVIS still provide a good accuracy–efficiency trade-off under matched inference budgets?
Minor: The AI-use statement is currently placed in Appendix A.5 rather than before the references as recommended by the template. Additionally, it would be better if you stated the data processing via Qwen2-VL (Sec. 3.3) in the AI use statement section.
2 comments
Part 2/2: Response to Reviewer Lqtb
Q2: End-to-End Inference Efficiency
We thank the reviewer for highlighting this distinction. We have added end-to-end inference latency and FLOPs comparisons with both tool-based and recurrent latent reasoning methods, and vanilla SFT in Appendix A.6 (Tabs. 12–13). All experiments are conducted on a single NVIDIA A800 GPU with BF16 precision, batch size 1, and a maximum generation length of 1,024 tokens.
Table 12. End-to-end inference latency and FLOPs compared with tool-based visual reasoning methods. Parentheses indicate speedups over the fastest baseline and FLOPs changes relative to the lowest-FLOP baseline.
\hline \text{Dataset} & \text{Model} & \text{Time (s)}\downarrow & \text{FLOPs (T)}\downarrow \\\\ \hline \text{V*} & \text{Pixel-Reasoner} & 6.72 & 194.88 \\\\ & \text{DeepEyes} & 7.09 & 226.22 \\\\ & \text{Thyme} & 10.59 & 155.64 \\\\ & \mathbf{ELVIS} & \mathbf{2.64}\\;(2.54\times) & \mathbf{99.19}\\;(-36.27\\%) \\\\ \hline \text{HR4K} & \text{Pixel-Reasoner} & 12.23 & 432.25 \\\\ & \text{DeepEyes} & 12.39 & 434.67 \\\\ & \text{Thyme} & 13.54 & 246.64 \\\\ & \mathbf{ELVIS} & \mathbf{4.26}\\;(2.87\times) & \mathbf{193.55}\\;(-21.53\\%) \\\\ \hline \text{Maze} & \text{Pixel-Reasoner} & 19.73 & 22.94 \\\\ & \text{DeepEyes} & 24.49 & 59.37 \\\\ & \text{Thyme} & 23.32 & 24.19 \\\\ & \mathbf{ELVIS} & \mathbf{6.34}\\;(3.11\times) & \mathbf{12.23}\\;(-46.69\\%) \\\\ \hline \end{array}$$ As shown in **Tab. 12**, ELVIS achieves **2.54×, 2.87×, and 3.11× speedups** over the fastest tool-based baselines on V*, HR4K, and Maze, respectively, with FLOPs reductions of **36.27%, 21.53%, and 46.69%**. These substantial latency gains reflect the additional overhead of tool execution in tool-based methods. **Table 13. End-to-end inference latency and FLOPs compared with latent methods across latent-state widths K.** Parentheses indicate speedups over the fastest baseline and FLOPs changes relative to the lowest-FLOP baseline. $$\begin{array}{llcccccc} \hline \text{Dataset} & \text{Model} & \text{Time} & \text{Time} & \text{Time} & \text{FLOPs} & \text{FLOPs} & \text{FLOPs} \\\\ & & K=8 & K=16 & K=32 & K=8 & K=16 & K=32 \\\\ \hline \text{V*} & \text{LVR} & \mathbf{2.50} & 2.89 & 3.05 & \mathbf{99.14} & 100.97 & 101.22 \\\\ & \text{Monet} & 2.86 & 2.95 & 3.08 & 100.89 & 100.95 & 101.05 \\\\ & \text{ILVR} & 3.76 & 4.21 & 5.43 & 101.61 & 101.95 & 102.86 \\\\ & \mathbf{ELVIS} & 2.68\\;(0.94\times) & \mathbf{2.64}\\;(1.09\times) & \mathbf{2.65}\\;(1.15\times) & 99.20\\;(+0.06\\%) & \mathbf{99.19}\\;(-1.75\\%) & \mathbf{99.22}\\;(-1.81\\%) \\\\ \hline \text{HR4K} & \text{LVR} & 4.34 & 4.74 & 5.02 & 193.66 & 193.90 & 194.77 \\\\ & \text{Monet} & 4.45 & 4.95 & 5.49 & 194.75 & 194.89 & 195.13 \\\\ & \text{ILVR} & 4.85 & 5.69 & 6.54 & 195.13 & 195.73 & 196.34 \\\\ & \mathbf{ELVIS} & \mathbf{4.29}\\;(1.01\times) & \mathbf{4.26}\\;(1.11\times) & \mathbf{4.21}\\;(1.19\times) & \mathbf{193.45}\\;(-0.11\\%) & \mathbf{193.55}\\;(-0.18\\%) & \mathbf{193.67}\\;(-0.57\\%) \\\\ \hline \text{Maze} & \text{LVR} & 7.54 & 8.71 & 9.07 & 13.75 & 14.86 & 15.09 \\\\ & \text{Monet} & 11.40 & 11.50 & 13.73 & 16.95 & 17.03 & 18.50 \\\\ & \text{ILVR} & 6.94 & 8.90 & 11.43 & 14.12 & 17.09 & 20.46 \\\\ & \mathbf{ELVIS} & \mathbf{6.31}\\;(1.10\times) & \mathbf{6.34}\\;(1.37\times) & \mathbf{6.41}\\;(1.41\times) & \mathbf{12.21}\\;(-11.19\\%) & \mathbf{12.23}\\;(-17.68\\%) & \mathbf{12.26}\\;(-18.78\\%) \\\\ \hline \end{array}$$ Compared with recurrent latent reasoning methods (**Tab.13**), at the default $K=16$, ELVIS achieves **1.09×, 1.11×, and 1.37× speedups** on the three benchmarks, respectively. Furthermore, as $K$ increases from 8 to 32, the inference time and FLOPs of recurrent methods progressively increase, whereas **ELVIS remains nearly constant in both, yielding greater relative speedups at larger $K$**. These results confirm that the efficiency gains of ELVIS's parallel latent-state construction extend to the complete inference process. **Table 14. End-to-end inference latency and FLOPs of ELVIS and Vanilla SFT.** | Dataset | Model | Time (s) ↓ | FLOPs (T) ↓ | | ------- | ----------- | ---------: | ----------: | | V* | Vanilla SFT | 2.57 | 99.03 | | | **ELVIS** | 2.64 | 99.19 | | HR4K | Vanilla SFT | 4.26 | 193.47 | | | **ELVIS** | 4.26 | 193.55 | | Maze | Vanilla SFT | 6.41 | 12.41 | | | **ELVIS** | 6.34 | 12.23 | Compared with vanilla SFT, ELVIS achieves **nearly identical end-to-end latency and FLOPs**, while improving accuracy by **+6.28 pp on V\* and +9.33 pp on Jigsaw** (Tab.1). Latent-state computation accounts for only **~1% of total inference time**, further demonstrating the negligible overhead of parallel latent construction.
Part 1/2: Response to Reviewer Lqtb
W1 & Q1: Training-Budget-Matched Comparisons with Non-Latent Baselines
We thank the reviewer for raising this important concern. We first clarify that the original vanilla SFT baseline corresponds to the first-stage SFT (SFT1) of ELVIS. To examine whether ELVIS's improvements arise from additional training, we have added comparisons with non-latent baselines under matched training steps in Appendix A.5 (Tab. 11). Specifically, starting from the shared SFT1 checkpoint (3,500 steps), we continue training the vanilla model for another 1,766 steps (matching ELVIS's SFT2 stage) on AVR-112K using the standard CE objective. We further apply vanilla GRPO to both models under the same RL setup, enabling controlled comparisons at both the SFT and RL stages.
Table 11. Training-budget-matched comparisons between ELVIS and non-latent baselines.
Under matched SFT training steps, ELVIS outperforms the non-latent baseline by +5.24, +1.76, +7.33, and +11.86 pp on V*, HR4K, Jigsaw, and Maze, respectively. After vanilla GRPO, ELVIS maintains improvements of +1.05, +1.75, +8.67, and +6.78 pp on the same benchmarks. These results indicate that additional training alone cannot reproduce the benefits of ELVIS's latent-state learning.
Regarding the reviewer's observation on Zebra Maze, we acknowledge that vanilla SFT1 already achieves 35.59%. Interestingly, continuing vanilla SFT for the same additional 1,766 steps reduces Maze accuracy from 35.59% to 24.58%, while Jigsaw also decreases from 48.67% to 46.67%. This may reflect overfitting to the training trajectories and the difficulty of transferring knowledge learned from intermediate images to inference without such observations. In contrast, ELVIS learns evidence-grounded latent states to bridge this gap.
Furthermore, the controlled ablations in Sec. 5.4 (Tab. 4 and Fig. 7) separately examine the contributions of latent-state construction and evidence supervision. Together with the matched-step comparisons, these results support that ELVIS's improvements arise from its latent-state learning design.
Q3(Minor): Relocation of the AI-Use Statement
We thank the reviewer for the reminder. We have moved the existing AI-use statement from the appendix to immediately before the references and updated it to add the use of LLM and VLM for dataset processing.