acceptodds
Under review as a conference paper at ICLR 2027

Looping Visual Tokens for Scaling Multimodal Intelligence

Abstract

We introduce Looping Visual Tokens (LVT), a mechanism for learning visual and spatial understanding through recurrent computation over visual tokens. LVT uses a shared transformer block to iteratively update a persistent visual state at the native image-token positions of a vision-language model (VLM). By reusing the same parameters and token positions across iterations, the block deepens visual computation without expanding the token sequence. We train this recurrent computation by aligning successive states with the backbone's perceptual representations of intermediate observations, and pass the final state to the model's task readout. Human videos and continuous environment-state sequences from simulation can provide supervision at scale, connecting LVT to learning from visual experience. We evaluate LVT on visual and spatial reasoning tasks. On fresh VSP maps beyond training path lengths, ordered supervision achieves 97.0% closed-loop success, compared with 93.2% for final-state supervision and 64.3% for answer-only recurrence. For unseen rotations at prescribed horizons, ordered supervision improves accuracy from 79.1% to 92.1%, while visual supervision alone reaches 87.2%. Textual process supervision remains stronger on the training grid size, while LVT generalizes better to larger grids. Decoding and interventions examine visual structure and state use in prediction. These results motivate LVT as a scalable mechanism for general-purpose visual learning. Anonymous repository: https://anonymous.4open.science/r/LoopVLM-6F55/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Looped Transformers, Vision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3NeeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work presents Looping Visual Tokens (LVT), a recurrent mechanism for vision‑language models that iteratively updates native image tokens via a shared transformer block, without expanding token sequence length. Trained by aligning evolving visual states to perceptual targets derived from successive intermediate observations, LVT supports multi‑step spatial reasoning. Evaluated on visual spatial planning (VSP) and mental‑rotation tasks, ordered sequence supervision substantially improves generalization to longer transformations and larger unseen grids compared with answer‑only or final‑state supervision; LVT outperforms textual chain‑of‑thought on size extrapolation, where text suffers transcription failures. Diagnostic decoding and state interventions further distinguish perceptually recoverable visual content from the actual usage of recurrent states for downstream prediction.

Critical strengths
  1. The method proposed in the paper doesn’t add extra latent tokens but directly reuses the model’s original image tokens; the backbone model is frozen, and only LoRA is fine-tuned. The recursive output calculation is directly connected to the existing pre-trained task readout head, so there’s no need to train a multi-modal model from scratch, making it easy to plug into existing VLMs.

  2. The paper compares training with just the answer, the final state, ordered sequences, and shuffled supervision. Experiments show that supervising with ordered intermediate observations greatly boosts long-sequence and out-of-distribution generalization, performing significantly better than just using the final labels on longer paths and larger grid tasks.

  3. The paper is written smoothly, and the main ideas are clear.

Critical weaknesses
  1. During the reasoning phase, we need to run the shared Transformer blocks multiple times, and the more iterations there are, the higher the computation cost. Even though the number of tokens stays the same, the number of times the decoder layers are called increases linearly with each loop.

  2. Most experiments in the paper use a preset number of loop iterations; although they tested convergence stopping, automatically deciding when to terminate the inner loop is still an unsolved problem.

  3. Training requires a complete, aligned sequence of intermediate frame observations as teacher signals. In real-world scenarios, it's really hard to get this kind of precise step-by-step supervision, which limits practical application.

Questions for authors

This issue mainly affected my impression of this piece of work: While the paper includes internal ablations and one re‑implemented latent‑token baseline (Mirage), its external baseline suite remains incomplete. It misses key competitors including LoopViT, LVR, CoVT and Monet from latent visual reasoning literature. Critically, it lacks a compute‑matched baseline that simply increases static transformer depth at inference, making it hard to disentangle gains from iterative recurrent state updates versus extra test‑time compute. All main experiments rely solely on Qwen3‑VL‑4B‑Instruct without cross‑backbone validation. Besides, textual‑CoT baseline uses mismatched training sampling and output budget, introducing confounding variables for comparison.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 3Nee (1/4)

Thank you for your careful and constructive review. We appreciate your recognition that LVT reuses the model's original image tokens and plugs into existing VLMs, and that ordered intermediate supervision improves long-sequence and out-of-distribution generalization. We respond to the three weaknesses first and then to your questions, starting with the baseline suite, which you noted mainly affected your impression (Q1).

W1: Inference cost of recurrence

Measured per test item, LVT takes as long as direct fine-tuning on VR-Bench (1.17 vs. 1.18 s) and 2–24% longer on VSP, rotation and MindCube. We now report FLOPs and measured latency for every task. Relative to direct fine-tuning, on one RTX 5090:

Task (rounds)Training FLOPs per updateInference FLOPs per callLatency per test item (LVT vs. direct FT)
VR-Bench [1] (8)2.0×1.65×1.17 vs. 1.18 s
VSP [2] (20, closed loop)1.5×2.9×2.95 vs. 2.88 s (5.1 map re-renders per item, the fewest of all models)
Rotation (angle/30°)1.3×1.2×0.26 vs. 0.21 s
MindCube [3] (4)1.7×1.25×0.32 vs. 0.28 s

Training FLOPs include the frozen teacher, counted once per training example because its targets are fixed. The number of rounds can also be reduced: on VSP, propagating two rings per round halves the rounds and still reaches 85.3%, against 59.2% for direct fine-tuning (Appendices M and U).

W2: Automatic stopping

LVT does not need to decide when to stop: it runs a fixed number of rounds, and the rounds after the last frame are trained to keep the final frame, so extra rounds leave the state unchanged and the budget only needs to be large enough. This needs no stop label and no extra head, and it already runs on three tasks: every VSP call runs 20 rounds, trained with 0–4 such holding rounds, every VR-Bench item runs 8 rounds and every MindCube question 4, and all their gains are obtained with this fixed budget. On rotation, the one task whose main result sets the rounds from the angle stated in the question, both a learned stop (a halting head on the mean probe-image state) and a fixed 7-round budget with padding stay above direct fine-tuning at the unseen 150° and 180° (60.0% and 57.4%, 61.0% with write-back, vs. 49.1%, first fresh set), and the angle-derived rounds give 84.2% (Sections 3.3 and 4.1, Appendix P).

W3: Availability of frame supervision

LVT needs frames only during training; at test time it sees the same input as direct fine-tuning. The frames can also come from data that already exists: on VR-Bench they are rendered once offline, and on MindCube they are built from the real views supplied with each question plus a drawn top-down map. Frames can also be sampled from video. On MindCube, we generated a short clip between pairs of real views of each training scene with a video model conditioned on the real first and last frames (Wan 3.0 [4]), and supervised the rounds with frames sampled from these clips (8,225 of 10,000 training questions). This variant reaches 79.7% on the official test, against 59.9% for direct fine-tuning and 77.2% for the answer-only loop, and is above both in every seed. The clips are generated rather than recorded, but they show that frames sampled from video can supervise LVT, suggesting that recorded videos could also serve as a source of frames.

[1] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

[2] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

[3] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

[4] Team Wan, A. Wang, B. Ai, et al. Wan: Open and Advanced Large-Scale Video Generative Models. arXiv:2503.20314, 2025.

AuthorsOpenReview ↗

Response to Reviewer 3Nee (2/4)

Q1: Latent visual reasoning baselines

We ported LoopViT [1], LVR [2], CoVT [3], Monet [4] and Mirage [5] to our backbone, data, update budget and scorer, and evaluated them on VR-Bench [6] (trained on its official training split), VSP [7], mental rotation and MindCube [8], with three seeds each:

MethodVR-Bench (official split)VSP, closed loopRotation 150°+180° (sets 1 / 2)MindCube
LVT40.8 (8 rounds)96.484.2 / 84.086.5
Direct FT33.259.2 (57.9 trained on the question and image only)49.1 / 51.759.9
LoopViT35.1 (6 iterations)55.392.6 / 93.563.4
LVR30.356.052.9 / 54.863.7
CoVT33.062.250.1 / 50.267.7
Monet26.041.245.5 / 47.356.7
Mirage32.655.959.1 / 60.359.3

LVT is above every port on VR-Bench, VSP and MindCube, and above four of the five on rotation. Per port:

  • LoopViT iterates tied blocks 10–17 of the backbone's vision encoder. On VR-Bench (6 iterations, original image only, as in the original) it is 1.9 points above direct fine-tuning (35.1% vs. 33.2%) and 5.8 below LVT, both in every seed. On VSP it is level with direct fine-tuning (55.3% vs. 57.9%) and 41.1 points below LVT, and on MindCube it is 23.1 below LVT. On rotation it is above LVT (92.6% / 93.5% vs. 84.2% / 84.0%).
  • LVR uses the input image as the latent target and, like the original, sees only the original image. It is below direct fine-tuning trained the same way on VR-Bench (30.3% vs. 33.2%) and level with it on the other three tasks (VSP 56.0% vs. 57.9%).
  • CoVT in the table is our surrogate, which uses targets from the frames and the backbone's own visual features, with linear heads, and inserts its 16 latent tokens before every answer. It is above direct fine-tuning on MindCube and level with it on VSP, rotation and VR-Bench (33.0% vs. 33.2%). We also ran the original expert-backed version (frozen SAM, Depth-Anything-V2 and DINOv2 targets, its four-stage curriculum, latent tokens emitted by the model): it is below the surrogate on VR-Bench, VSP and MindCube (26.0%, 51.0% and 53.8%), in every seed on VR-Bench and VSP, and on rotation it is 24.6 / 23.0 points below LVT (59.6% / 61.0%).
  • Monet is trained with its auxiliary images, as in the original. It is below direct fine-tuning on VR-Bench and VSP and level with it on MindCube and at unseen rotation angles.
  • Mirage first aligns four latent positions with the features of a helper image, each item's final frame, under a down-weighted answer loss, and then trains on the answer alone. On VR-Bench it is level with direct fine-tuning (32.6% vs. 33.2%) and 8.3 points below LVT in every seed. On VSP it is level with direct fine-tuning trained the same way (55.9% vs. 57.9%) and 40.6 below LVT. At unseen rotation angles it reaches 59.1% / 60.3%, 25.1 / 23.7 points below LVT. On MindCube it is level with direct fine-tuning (59.3% vs. 59.9%) and 27.2 points below LVT.

Training FLOPs per update relative to direct fine-tuning, frozen targets counted once per example (inference FLOPs per call in parentheses), and latent positions:

MethodLatent positions (train / test)VR-BenchVSPRotationMindCube
LVTnone; 8 / 20 / angle/30° / 4 rounds over the image tokens2.0× (1.65×)1.5× (2.9×)1.3× (1.2×)1.7× (1.25×)
LoopViTnone; iterated layers (6 iterations on VR-Bench)1.2× (1.2×)1.2× (1.7×)1.2× (1.2×)1.5× (1.3×)
LVR100–369 / 161.4× (1.04×)1.4× (1.04×)1.4× (1.06×)1.3× (1.01×)
CoVT16 / 161.1× (1.04×)1.1× (1.04×)1.1× (1.06×)1.1× (1.01×)
Monet8 / 82.4× (1.02×)1.9× (1.02×)1.8× (1.03×)1.7× (1.01×)
Mirage4 / 41.03× (1.01×)1.03× (1.01×)1.05× (1.02×)1.02× (1.00×)

We have included these results in Section 4.2 and Appendices H, I and U of the revision.

[1] W.-J. Shu, X. Qiu, R.-J. Zhu, et al. LoopViT: Scaling Visual ARC with Looped Transformers. arXiv:2602.02156, 2026.

[2] B. Li, X. Sun, J. Liu, et al. Latent Visual Reasoning. ICLR 2026.

[3] Y. Qin, B. Wei, J. Ge, et al. Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens. ECCV 2026.

[4] Q. Wang, Y. Shi, Y. Wang, et al. Monet: Reasoning in Latent Visual Space Beyond Image and Language. CVPR 2026.

[5] Z. Yang, X. Yu, D. Chen, et al. Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens. CVPR 2026.

[6] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

[7] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

[8] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

AuthorsOpenReview ↗

Response to Reviewer 3Nee (3/4)

Q2: Compute-matched baseline

Looping the direct fine-tuned checkpoint itself at test time, without training the loop, gives LVT's inference compute and does not help: it collapses on VSP and VR-Bench and changes nothing on MindCube and rotation (Q2 continued). We therefore also trained a compute-matched baseline: an answer-only loop that runs the same shared block for the same number of rounds with exactly LVT's parameters, so it adds the same depth at inference (the four shared layers per round) and has the same test-time compute, and is trained on the same data for the same number of updates, only without frames: no frame loss and, on VSP, no starts from intermediate frames (LVT without these starts still reaches 94.1%; three seeds). It separates the effect of the extra rounds from that of supervising the recurrent states with frames. On VR-Bench it runs without write-back, as LVT does there, and like every VR-Bench model here it sees only the question and the original image as input, never an intermediate frame:

TaskLVTAnswer-only loop
VSP [1], closed loop, 20 rounds96.466.1
MindCube [2], 4 rounds86.577.2
Mental rotation, 150°+180°, fresh set 184.281.5
VR-Bench [3], official split only, 8 rounds40.839.4
VR-Bench, official + generated training levels, 8 rounds57.156.0

LVT is above this answer-only loop on all four tasks: by 30.3 points on VSP, 9.3 on MindCube and 2.7 on rotation (84.2% vs. 81.5%), and on VR-Bench in every seed, by 1.4 points on the official split (40.8% vs. 39.4%, p = 0.009), 1.1 with generated levels (57.1% vs. 56.0%) and 6.1 on the 258 official test mazes whose layouts do not occur in training (51.2% vs. 45.1%, with generated levels). We have included these results in Section 4.2 and Appendices J and U of the revision.

[1] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

[2] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

[3] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

Reviewer_vxpbOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose Looping Visual Tokens, a learning framework that aims to enhance the spatial understanding of VLMs in maze solving. The method partitions the model into a prelude, a recurrent block, and a coda. The recurrent block is adapted to recursively iterate on the visual state by aligning successive states with encoded representations of intermediate observations, such as sequential rotations or successive planning states. The authors validate their method using a small (4B) model, comparing it to two baselines on a grid planning task and a mental rotation task.

Critical strengths

None

Critical weaknesses
  • The paper, and especially the methodology, is difficult to follow. Notions are discussed before being defined, and many symbols are left undefined. To the best of my knowledge, these notions are not standard nomenclature: ordered supervision, answer-only recurrence, prescribed horizons, textual process supervision, observation sequences, layer boundary, pretrained readout, VSP maps, map-image/probe-image positions, inter alia. Some of these are defined later in the paper, long after being introduced. In the methodology, uku_k is defined twice with competing definitions, Cˉ\bar{C}, Pˉ\bar{P}, sk(i)s_k^{(i)}, Laux\mathcal{L}_{\text{aux}} are undefined.
  • Many modelling choices are presented but not discussed at all (see questions)
  • The experimental section is severely lacking. The evaluated tasks are very narrow, the method is too specific to them, and the evaluation spans a single, small model (4B) on two datasets. Despite this, a naive chain-of-thought baseline seems to outperform the proposed method. Finally, much of the accompanying text in the results simply restates the numerical findings in the tables with no analysis. The authors simply re-encode the information in the tables into text format.
Questions for authors
  • Could you define all nonstandard jargon and simplify the exposition in the methodology? (see weaknesses)
  • Why does the Text CoT baseline perform so well on the task in Table 1? Why are the results of the proposed method highlighted, given that it underperforms the baseline row?
  • Were the following modelling choices ablated: λ\lambda in the loss, the number of terminal holding steps, the sampling probability in L187, the penalty in L189, and which specific layers form the prelude and the coda?
  • How did you do model selection? Was there a validation split? How did you choose the hyperparameters mentioned in Sec. 4.1?
  • How is the Shepard & Metzler study on humans relevant? Why is it cited at L323?
  • Does the paper’s title faithfully represent the paper’s contents?
  • What happens to the pre-existing capabilities of the VLM? Can you report the performance before and after your training on external benchmarks that Qwen reports?
8 comments
AuthorsOpenReview ↗

Response to Reviewer vxpb (1/5)

Thank you for your detailed and constructive review, which helps a lot in improving the overall clarity of our paper. We take your concerns seriously, especially about our AI usage statement, the clarity of the method, the modelling choices and the scope of the evaluation. We first address the ethics concern and then respond to the other issues point by point. We hope these clarifications and the new results alleviate the concerns underlying your current assessment.

Ethics: AI usage statement

We hold research integrity and transparency in the highest regard, and we thank you for raising this concern. This is why we released the code of our experiments together with the submission in the anonymous repository (https://anonymous.4open.science/r/LoopVLM-6F55/), including the analysis script paired_ci.py that you mention, so that the implementation can be checked and the experiments rerun, and why we keep it updated: the repository now also contains the code for the experiments added in the revision.

Regarding the AI usage statement, our original wording was not specific enough: it mentioned only manuscript polishing and did not describe how we used AI coding assistants. We therefore clarify it. Following the ICLR 2027 AI policy for authors, we have updated the AI use statement in the revised version (marked in blue): “The research questions, the method and the experimental protocols were decided by the authors. AI coding assistants helped implement the data-generation, training, evaluation and analysis code, launch and monitor experiment runs under author-specified settings, give feedback on experiment design, and summarise intermediate results; generative AI tools were also used to edit the manuscript. The authors reviewed all AI-assisted work, every reported number is computed by scripts from the saved per-item evaluation outputs, and the authors take full responsibility for all content of the paper.”

W1: Clarity of the method and nonstandard terms

We rewrote Section 3 in full (marked in blue). It first states in plain words what LVT does and then introduces each part in the order it is used: the three parts of the decoder, the visual state, one round of the update, the frame targets and their loss, and the fixed number of rounds and the readout. Every symbol is defined at its first use, u_k is defined once, and the section comes with a notation table (Appendix A) and a figure of one unrolled forward pass (Fig. 2). The terms you list now read:

  • Prelude, shared block, coda: layers 0–11, 12–15 and 16–35 of the 36-layer decoder. The shared block is applied repeatedly; one application is a round.
  • Map-image / probe-image positions: the image tokens of the rendered VSP [1] map (planning) or of the rotated probe object (rotation). Their hidden states form the looped visual state v_k.
  • VSP maps: the grid images of the VSP benchmark [1], each with a start, a goal and holes to avoid.
  • State update: x_k = W_in[h_{k−1}; e] at those positions, u_k = C_θ(x_k) is the shared block's output at round k (now defined once), and v_k = v_{k−1} + g_k ⊙ ([u_k]_V − v_{k−1}), where g_k is the write-back gate, learned on VSP and fixed to 1 (replace the state) on the other tasks.
  • Observation sequence (now called frames): the frames I_1, …, I_m of the procedure (rotated views, solver expansions). At test time the model sees only the question and the original image; frames are training targets, and on VSP some training rollouts also start from an intermediate frame (Section 3.3).
  • Ordered supervision: the state after round k is trained to match the frozen model's encoding of frame k. Final-state supervision uses only the last frame; answer-only recurrence (λ = 0) trains the loop with the answer loss alone.
  • P̄, C̄ and the layer boundary: layers 0–11 and 12–15 of the pretrained model, i.e., LVT with its LoRA adapters switched off. The target for frame j is [C̄(P̄(E(c, I_j)))]_V, how the pretrained model represents frame j at the output of layer 15, where the student state is read.
  • s_k^(i) and L_aux: s_k^(i), now written v̂_k^(i) (Eq. 4), is the RMS-normalised, centred student feature at position i after round k, compared with the teacher feature by a weighted cosine loss; L_aux is the gate-opening penalty (planning) or the halting loss (rotation), in L = L_ans + λ·L_feat + L_aux.
  • Prescribed horizon: the number of rounds is set by the protocol rather than chosen by the model: a fixed 20 rounds per call on VSP, 8 on VR-Bench and 4 on MindCube, and angle/30° on rotation.
  • Pretrained readout: the coda with the pretrained output head, which produces the answer from the updated states.
  • Textual process supervision: the Text CoT control, which writes the map and the search out as text.

[1] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

AuthorsOpenReview ↗

Response to Reviewer vxpb (2/5)

W2: Modelling choices that were not discussed

We ran one-factor ablations, changing one choice at a time from the default configuration (three seeds each). Each cell is the change in accuracy from the default, in points; “default (x)” marks the default setting with its own accuracy x; bold: the 95% interval excludes zero.

Choice (default: VSP / rotation)VariantVSP [1], closed loopRotation, 150°+180°, fresh set 1
Frame-loss weight λ (1 / 1)0.3−25.8−1.2
3+3.0−18.3
Holding rounds (up to 4 / none)none−95.7default (84.2)
up to 2−0.6−23.3
up to 4default (96.4)−24.1
up to 8+0.7−23.9
Intermediate-start probability, submission L187 (0.5 / 0)0−2.3default (84.2)
0.250.0+8.1
0.5default (96.4)+8.6
0.75+0.7+8.1
Auxiliary weight, submission L189 (gate penalty 0.01 / halting loss 0.1)0+1.1−0.7
gate 0.1 / halting 0.3+0.6−0.5
Shared-block layers (12–15 / 12–15)8–11+0.9+8.8
16–19−5.1−16.8

All reported results keep the default configuration, fixed before these ablations and not tuned per task; on the test sets, λ = 3 on VSP (+3.0) and intermediate starts or layers 8–11 on rotation (+8.1 to +8.8) score higher. We also ran write-back on and off on every task (three seeds each). It helps on VSP, where the number of rounds is large (96.4% with vs. 83.7% without at 20 rounds), and on the other three tasks replacing the state each round, the setting we use there, is as good or better: on VR-Bench [2] 40.8% vs. 39.9% with write-back on the official split (p = 0.09) and 57.1% vs. 55.5% with generated levels (p = 0.005), on rotation 84.2% vs. 80.2% at unseen 150°+180°, and on MindCube [3] 86.5% vs. 83.4%. We have included these results in Section 4.4 (Table 5) and Appendices M and N of the revision.

[1] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

[2] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

[3] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

AuthorsOpenReview ↗

Response to Reviewer vxpb (3/5)

W3: Scope of the evaluation

The revision substantially broadens the evaluation. It adds the full official test sets of VR-Bench [1] (all 1,512 items the official scorer can score) and MindCube [2] (11,154 items) to VSP [3] and mental rotation, compares LVT with eight baselines on all four tasks, and repeats the main comparisons on Qwen2.5-VL-3B, Qwen3-VL-8B and InternVL3.5-4B, the last from a different model family (InternViT vision encoder, MLP projector). Main results (%, three seeds each, final checkpoints; Table 1 of the revision; bold: best per column on Qwen3-VL-4B):

Method (%)VR-Bench [1], official splitVR-Bench, + generated levelsMindCube [2]VSP [3], closed loopRotation, 150°+180°
Qwen3-VL-4B: direct FT33.247.659.959.249.1
Direct FT, looped at inference3.85.660.10.649.3
LVR [4]30.340.863.756.052.9
CoVT [5] (surrogate)33.044.867.762.250.1
Monet [6]26.035.856.741.245.5
Mirage [7]32.644.559.355.959.1
LoopViT [8]35.156.863.455.392.6
Answer-only loop39.456.077.266.181.5
LVT40.857.186.596.484.2
Qwen2.5-VL-3B: LVT − direct FT+4.8+3.2+8.1+1.8+25.9
Qwen3-VL-8B: LVT − direct FT+5.9+9.9+12.2+35.2+14.7
InternVL3.5-4B: LVT − direct FT+6.2+9.7+3.5+31.9+10.8

On Qwen3-VL-4B, LVT has the highest accuracy on VR-Bench, with and without generated levels, on MindCube and on VSP; on the three other backbones it is above direct fine-tuning in all 15 cells (+1.8 to +35.2 points). We answer the point on the Text CoT baseline under Q2 below.

W3 (continued): Analysis in the results

We restructured Section 4 so that the results are analysed rather than restated. The main table above (Table 1 of the revision) is followed by analyses in Sections 4.3 and 5 and an ablation study in Section 4.4, which explain each result with evidence from decoding and interventions. On VSP [3], success follows how much of the search the looped states contain: a decoder trained only on frozen features recovers 1.5%, 42.9%, 77.6% and 89.6% of the correct arrows after 20 rounds for the answer-only, shuffled, final-frame and ordered models, which reach 64.3%, 69.7%, 93.2% and 97.0% (the submission's runs, Table 2). Ordered frames matter most where each frame replaces the previous one: on rotation, supervising only the final view gives 68.8% against 92.1% with ordered frames (the submission's test set; Sec. 4.3 and Appendix O), whereas on VSP each frame adds a ring of arrows to the previous one, so the final field already contains the whole search. Appended latent tokens can be bypassed: replacing the latent tokens of Mirage [7] with those computed for a moved goal leaves 93–100% of its plans unchanged, whereas the same replacement in LVT's state redirects the first action on 98–100% of eligible maps.

[1] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

[2] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

[3] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

[4] B. Li, X. Sun, J. Liu, et al. Latent Visual Reasoning. ICLR 2026.

[5] Y. Qin, B. Wei, J. Ge, et al. Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens. ECCV 2026.

[6] Q. Wang, Y. Shi, Y. Wang, et al. Monet: Reasoning in Latent Visual Space Beyond Image and Language. CVPR 2026.

[7] Z. Yang, X. Yu, D. Chen, et al. Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens. CVPR 2026.

[8] W.-J. Shu, X. Qiu, R.-J. Zhu, et al. LoopViT: Scaling Visual ARC with Looped Transformers. arXiv:2602.02156, 2026.

Reviewer_BihwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Looping Visual Tokens (LVT), which repeatedly updates a VLM’s existing image-token states using a shared transformer block. Intermediate observations provide perceptual supervision through a frozen backbone, and the final state feeds into the model’s task readout. Experiments on visual spatial planning and mental rotation show benefits from visual-state supervision, with additional gains from ordered intermediate targets. A Maze-Hard extension studies recurrence with a different supervision and readout interface.

Critical strengths
  1. The method connects recurrent visual computation to a pretrained readout without expanding the token sequence.

  2. Answer-only, final-state, and ordered-target controls help identify where the gains come from. Rotation training without answer labels is particularly interesting.

  3. Decoding and intervention studies distinguish recoverable visual content from state use, and the paper openly reports cases where these diverge.

Critical weaknesses
  1. Generalization remains constrained. Strong planning results rely on closed-loop feedback, while custom-split open-loop success is only 18.1%. Rotation uses horizons prescribed from the true angle. The synthetic, task-specific experiments do not yet establish broader learning from visual experience.

  2. Cross-method comparisons remain confounded. Matching training steps does not match computation, and baselines differ in latent capacity or data sampling. The results establish the value of supervision within LVT more clearly than its advantage over alternative architectures.

Questions for authors
  1. How well does rotation work with a fixed inference budget or learned stopping, without using the true angle to set the horizon?

  2. Does LVT retain its advantage when baselines receive comparable total computation and latent capacity, accounting for teacher encoding and repeated environment calls?

6 comments
AuthorsOpenReview ↗

Response to Reviewer Bihw (1/4)

Thank you for your thoughtful and constructive review. We appreciate your recognition that LVT connects recurrent visual computation to a pretrained readout without expanding the token sequence, that our answer-only, final-state and ordered-target controls help identify where the gains come from, and in particular your interest in rotation training without answer labels. Below, we address your concerns point by point.

W1: Generalization beyond synthetic tasks, closed-loop feedback and prescribed horizons

To test whether the gains extend beyond synthetic tasks, we added two established visual reasoning benchmarks and evaluated on their full official test sets: VR-Bench [1] for 2D reasoning, where the model plans a whole path in one pass without any environment feedback, and MindCube [2] for 3D reasoning, which asks about spatial relations across multiple views of a scene. Every comparison uses the same training data and number of updates for both models, reports the final checkpoint, and uses three seeds.

TaskTest setDirect FTLVT
VSP, closed loop300 fresh maps, 7–12 steps59.296.4
Mental rotation, unseen 150°+180°fresh test sets 1 / 2 (300 new objects each)49.1 / 51.784.2 / 84.0
VR-Bench (2D), official training split only, 8 roundsofficial, 1,512 items33.240.8
VR-Bench (2D), official + generated training levels, 8 roundsofficial, 1,512 items47.657.1
MindCube (3D), 1,250 updatesofficial, 11,154 items59.986.5

LVT improves on both benchmarks. On VR-Bench, which is open-loop by construction, we train on its official training split: LVT raises success from 33.2% to 40.8% at 8 rounds in every seed. Adding generated levels (17,583 rows over all five games, about 22% official, deduplicated against the test set) raises both models: LVT reaches 57.1% against 47.6% in every seed. In the official test set, 102 of the 360 mazes, all easy, share their layout, up to rotation and reflection, with a maze in the official training split. We removed those training mazes, and any generated maze with a test layout, so no model trains on a test layout. On the other 258 mazes, mostly medium and hard, LVT trained with generated levels solves 51.2% against 31.0% for direct fine-tuning, and the answer-only loop of W2 solves 45.1% (LVT higher in every seed, p < 0.001). On MindCube, accuracy rises from 59.9% to 86.5%.

The gains also generalize across backbones. In addition to Qwen3-VL-4B, we repeated the main comparisons on Qwen2.5-VL-3B, Qwen3-VL-8B and InternVL3.5-4B, the last from a different model family (InternViT vision encoder, MLP projector). LVT minus direct fine-tuning, three seeds each:

TaskQwen3-VL-4BQwen2.5-VL-3BQwen3-VL-8BInternVL3.5-4B
VSP, closed loop+37.2+1.8+35.2+31.9
Rotation, 150°+180° (fresh sets 1 / 2)+35.1 / +32.3+25.9 / +26.7+14.7 / +18.4+10.8 / +10.7
VR-Bench (official split)+7.7+4.8+5.9+6.2
MindCube+26.6+8.1+12.2+3.5

For rotation, the horizon is one round per 30° of the angle stated in the question, so it uses only input information; we discuss fixed budgets and learned stopping under Q1. Unlike closed-loop VSP [3], where the map is re-rendered after every two executed actions, VR-Bench is open-loop by construction, and LVT improves there in every seed (33.2% to 40.8% on the official split, 47.6% to 57.1% with generated levels). We have included these results in Section 4.2 and Appendices E, G and K of the revision.

[1] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

[2] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

[3] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

AuthorsOpenReview ↗

Response to Reviewer Bihw (2/4)

W2: Confounds in cross-method comparisons

At equal parameters and inference FLOPs, LVT is above an answer-only loop on all four tasks (below), and we report the training FLOPs of every method. We address the comparison from three perspectives.

Equalizing training compute would confound compute with data exposure. On a fixed training set, the only way to give direct fine-tuning LVT's training FLOPs is to train it for 1.3–2.0× as many passes over the same examples. The baseline would then differ from LVT both in compute and in how often it sees each example. We therefore keep the data and the number of updates fixed and report the per-update cost of every method (Q2).

Our protocol matches data and updates and reports the training FLOPs of every method. Mirage [1], LVR [2], CoVT [3] and Monet [4] each compare against fine-tuning on the same data; to our reading, none states that its baseline used the same number of updates, and none reports training FLOPs.

Inference compute can be matched directly. We compare LVT with an answer-only loop that has exactly LVT's shared parameters and runs the same number of rounds, so it has the same inference FLOPs; it is trained without frames: no frame loss and, on VSP, no starts from intermediate frames (LVT without these starts still reaches 94.1%; three seeds). On VR-Bench it runs without write-back, as LVT does there, and like every VR-Bench model here it sees only the question and the original image as input, never an intermediate frame:

TaskLVTAnswer-only loop
VSP [5], closed loop, 20 rounds96.466.1
MindCube [6], 4 rounds86.577.2
Mental rotation, 150°+180°, fresh set 184.281.5
VR-Bench [7], official split only, 8 rounds40.839.4
VR-Bench, official + generated training levels, 8 rounds57.156.0

At equal parameters and inference FLOPs, LVT is above the answer-only loop on all four tasks: by 30.3 points on VSP, 9.3 on MindCube and 2.7 on rotation (84.2% vs. 81.5%), and on VR-Bench in every seed, by 1.4 points on the official split (40.8% vs. 39.4%, p = 0.009) and 1.1 with generated levels (57.1% vs. 56.0%). The ported baselines cost 1.02–2.4× the training FLOPs of direct fine-tuning per update, with the latent positions listed under Q2 below. We have included these results in Section 4.2 and Appendices J and U of the revision.

[1] Z. Yang, X. Yu, D. Chen, et al. Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens. CVPR 2026.

[2] B. Li, X. Sun, J. Liu, et al. Latent Visual Reasoning. ICLR 2026.

[3] Y. Qin, B. Wei, J. Ge, et al. Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens. ECCV 2026.

[4] Q. Wang, Y. Shi, Y. Wang, et al. Monet: Reasoning in Latent Visual Space Beyond Image and Language. CVPR 2026.

[5] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

[6] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

[7] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

AuthorsOpenReview ↗

Response to Reviewer Bihw (3/4)

W2 (continued): Frames on VR-Bench, and looping the direct fine-tuned model

On VR-Bench [1], against the answer-only loop, both without write-back, LVT is higher on the 258 official test mazes whose layouts do not occur in training (51.2% vs. 45.1% with generated levels, every seed, p < 0.001); on the official training split alone it is 1.4 points higher overall (40.8% vs. 39.4%, every seed, p = 0.009).

We also looped the direct fine-tuned model itself at inference for the same number of rounds, without training the loop, so that its inference FLOPs equal LVT's. This does not help. It collapses where the loop is long (VSP [2] 0.6% vs. 57.9% for direct fine-tuning trained on the question and the original image only; VR-Bench, official split, 3.8% vs. 33.2%) and changes nothing where it is short (MindCube [3] 60.1% vs. 59.9%; rotation 150°+180° 49.3% vs. 49.1%). Extra inference compute alone therefore does not explain LVT's gain (Appendix J).

Q1: Rotation with a fixed budget or learned stopping

Without the angle-derived horizon, LVT stays above direct fine-tuning at the unseen angles on both fresh sets, with a learned stop and with a fixed budget of 7 rounds (57.4–61.8% vs. 49.1–51.7%), and keeps 99.6% at the trained angles. The main rotation result sets the rounds from the angle stated in the question (angle / 30°); VSP, VR-Bench and MindCube already run one fixed number of rounds for every item. Replacing this horizon, with training on angles up to 120° and testing at the unseen 150° and 180° (three seeds; two fresh test sets of 300 new objects each, generated after the configuration was fixed and reported as 1 / 2), gives:

Unseen 150°+180°Accuracy (%), fresh sets 1 / 2
Direct fine-tuning49.1 / 51.7
LVT, rounds = angle / 30°84.2 / 84.0
LVT, learned stop60.0 / 61.8
LVT, fixed 7 rounds with fixed-point padding57.4 / 58.6
LVT, fixed 7 rounds with fixed-point padding and write-back61.0 / 61.6

Learned stop. We attach a halting head to the loop. After round k it averages the RMS-normalised probe-image states, s̄_k = mean_{i∈V} RMSNorm(v_k^(i)), and outputs a stop logit ℓ_k = wᵀ s̄_k + b. It is trained with L_halt = (1/R) Σ_{k=1}^{R} BCE(σ(ℓ_k), 1[k = R]), weight 0.1, where R is the number of frames of the item; at test, the loop stops at the first round with ℓ_k > 0, at most 8 rounds. It is as accurate as the angle-derived horizon at the trained angles (99.6% at 0–120°) and reaches 60.0% at 150° and 180°.

Fixed budget with fixed-point padding. This approach needs no stop label and no extra head: we run a fixed number of rounds and train the rounds after the last frame to keep the final frame, so that extra rounds leave the state unchanged and the budget only needs to be large enough. It already runs on our other three tasks: every VSP call runs 20 rounds, trained with 0–4 holding rounds after the plan is complete, and every VR-Bench item runs 8 rounds and every MindCube question 4, with shorter frame sequences holding their final frame. Their gains over direct fine-tuning (VSP 59.2% to 96.4%, VR-Bench 33.2% to 40.8%, MindCube 59.9% to 86.5%) are all obtained with this fixed budget, without a per-item horizon. On rotation, training every item for 7 rounds with this padding makes the answer independent of the budget (running 11 or 15 rounds changes at most 1.2% of answers), keeps 99.6% at the trained angles and reaches 57.4% at 150° and 180° (61.0% with write-back), on par with the learned stop. The angle-derived horizon, which uses only the angle stated in the question, gives 84.2% (Sections 3.3 and 4.1, Appendix P).

[1] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

[2] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

[3] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Review and Rebuttal Summary

Dear Area Chairs and Reviewers,

We thank all reviewers for their time and constructive feedback. To assist the final evaluation, we summarize the strengths noted in the reviews, the main concerns raised, and how the rebuttal and the revised paper (changes marked in blue) address them.

Strengths noted in the reviews

  • LVT "connects recurrent visual computation to a pretrained readout without expanding the token sequence" (Bihw), "directly reuses the model’s original image tokens" and is "easy to plug into existing VLMs" (3Nee).
  • The supervision controls "help identify where the gains come from" (Bihw), and "supervising with ordered intermediate observations greatly boosts long-sequence and out-of-distribution generalization" (3Nee); "rotation training without answer labels is particularly interesting" (Bihw).

Main concerns and how we addressed them

  1. Generalization beyond synthetic tasks and one backbone (Bihw, vxpb, 3Nee). On the full official test sets of VR-Bench [1] and MindCube [2], LVT raises success from 33.2% to 40.8% (47.6% to 57.1% with generated training levels) and accuracy from 59.9% to 86.5% over direct fine-tuning. On Qwen2.5-VL-3B, Qwen3-VL-8B and InternVL3.5-4B, LVT is ahead of direct fine-tuning in all 20 backbone–task cells.
  2. Baselines and compute (Bihw, 3Nee). We ported five latent visual reasoning baselines, LoopViT [3], LVR [4], CoVT [5], Monet [6] and Mirage [7], to the same backbone, data and updates: LVT is above every port on VR-Bench, VSP [8] and MindCube, and above four of the five on rotation. A compute-matched answer-only loop with exactly LVT's parameters and inference FLOPs stays below LVT on all four tasks, and looping the direct fine-tuned model at inference does not help. We report training and inference FLOPs for every method and measured latency (1.17 vs. 1.18 s per item on VR-Bench).
  3. Stopping and the number of rounds (Bihw, 3Nee). VSP, VR-Bench and MindCube run a fixed number of rounds, so all their gains need no stopping decision. On rotation, a learned stop and one fixed budget with fixed-point padding both stay above direct fine-tuning at the unseen angles (60.0% and 57.4% vs. 49.1%).
  4. Availability of frames (3Nee). Frames are needed only for training and come from simulators, offline rendering or the views supplied with each question; supervising MindCube with frames sampled from generated video clips reaches 79.7%, against 59.9% for direct fine-tuning.
  5. Text CoT baseline (vxpb, 3Nee). Text CoT uses a larger training and decoding budget; on unseen 7×7 maps it reaches 60.9% / 39.3% on short / long paths, against 100.0% / 98.4% for LVT.
  6. Clarity and modelling choices (vxpb). We rewrote Section 3 so that every term is defined at its first use, with one state-update equation, a notation table and a figure of one unrolled forward pass, and ablated each modelling choice the reviewer listed with three seeds.
  7. AI use statement (vxpb). We clarified the statement in the revised paper to describe how AI coding assistants were used. Our code has been public in the anonymous repository since submission and now also covers every experiment added in the revision.

After the discussion, Reviewer Bihw wrote: "After reading your response, my concerns are addressed", and that the new results "show that LVT improves over baseline methods with tolerable costs."

Thank you very much for your time and effort.

Sincerely,

The Authors of Submission 32466

[1] C. Yang, H. Wan, Y. Peng, et al. Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks. arXiv:2511.15065, 2025.

[2] Q. Wang, B. Yin, P. Zhang, et al. MindCube: Spatial Mental Modeling from Limited Views. arXiv:2506.21458, 2025.

[3] W.-J. Shu, X. Qiu, R.-J. Zhu, et al. LoopViT: Scaling Visual ARC with Looped Transformers. arXiv:2602.02156, 2026.

[4] B. Li, X. Sun, J. Liu, et al. Latent Visual Reasoning. ICLR 2026.

[5] Y. Qin, B. Wei, J. Ge, et al. Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens. ECCV 2026.

[6] Q. Wang, Y. Shi, Y. Wang, et al. Monet: Reasoning in Latent Visual Space Beyond Image and Language. CVPR 2026.

[7] Z. Yang, X. Yu, D. Chen, et al. Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens. CVPR 2026.

[8] Q. Wu, H. Zhao, M. Saxon, et al. VSP: Assessing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for VLMs. arXiv:2407.01863, 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.