acceptodds
Withdrawn from ICLR 2027

VERSE: Verifiable Spatial Self-Evolution with Geometric Process Rewards

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spatial reasoning remains a fundamental bottleneck for vision–language models (VLMs), largely because models reason purely in 1D text tokens without external geometric anchors, whereas humans routinely sketch and mark up scenes. Equipping VLMs with agentic visual tools narrows this gap, yet training such tools traditionally demands prohibitively expensive human-annotated reasoning trajectories. While the self-evolution paradigm eliminates human annotation in code and mathematics, directly porting it to tool-assisted spatial reasoning encounters a critical failure mode: outcome-only rewards are completely blind to intermediate tool fidelity. Consequently, a policy can draw arbitrary annotations or cross physical obstacles, yet still be falsely reinforced via lucky textual guesses. To address this, we propose VERSE, an annotation-free framework that trains a VLM to actively propose, draw, and self-evolve within 3D environments. At the core of VERSE is the Agentic Verifier: by exploiting underlying 3D scene metadata and procedural topologies, it evaluates intermediate visual tool calls in closed form via computing deterministic grounding IoUs for bounding boxes and sequence-alignment fidelities for traced routes. This yields dense, engine-verified process rewards without relying on learned reward models or LLM judges. Extensive experiments across video, multi-view, and navigation domains demonstrate substantial improvements across scales: on Qwen2.5-VL-7B, VERSE elevates VSI-Bench accuracy from 33.68% to 40.52% and boosts MAZE performance from 33.50% to 90.80%, confirming that engine-verified process rewards provide the critical supervisory signal for multimodal self-improvement.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM, VLM, Spatial reasoning, Self-evolve, Reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VHMUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VERSE, a new framework for training VLMs’ geometric tool use. It focuses on the process reward during vision tool calls, on top of the conventional outcome reward. The process reward is verified by object grounding and geometric path fidelity. On VSI-Bench and MAZE, VERSE improves upon the base Qwen2.5-VL.

Critical strengths

(1) The motivation of adding a process reward to supervise tool-call soundness, on top of just the outcome, is insightful, and the method is designed accordingly to exactly oversee visual tool-use trajectories.

(2) The rich analyses in the paper also support this intuition of process supervision, as the process fidelity correlates with the final outcome, and the grounding accuracy evidently improves throughout training. These are interesting findings.

Critical weaknesses

(1) A fairer baseline in support of the idea of "process reward" would be pure outcome-driven RL, rather than the base VLMs. The authors ablated this in Tab. 2, but it’s limited to VSI-Bench, and the comparison should be made on all benchmarks where this paper claims improvement in Tab. 1. Furthermore, although the intuition behind process supervision for visual tool use is clear, the advantage compared to the conventional outcome-only reward seems marginal in Tab. 2.

(2) Judging from Tab. 1, it seems VERSE works much better on the smaller VLM (3B) than on the 7B one, and its effects may diminish with more capable base models.

Questions for authors

(1) Considering that the path fidelity correlates strongly with the outcome (Fig. 4), would outcome-only RL also induce better path fidelity and object grounding (Tab. 3), even without explicit supervision over them?

(2) Related to weakness (2), how does the method scale with larger VLM variants?

Reviewer_iQ9kOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VERSE, a framework that improves spatial reasoning in vision-language models through self-evolution with geometric process rewards. It introduces an Agentic Verifier that evaluates intermediate visual tool actions using bounding-box IoU and path overlap, rather than relying only on final answer correctness. The framework combines supervised initialization with a proposer-solver RL pipeline. Experiments on VSI-Bench, SPAR-Bench, and MAZE show improvements for both 3B and 7B models.

Critical strengths
  • The paper addresses a meaningful limitation of outcome-only rewards in tool-assisted spatial reasoning. The motivation is clear and well explained.
  • The proposed geometric rewards are simple, interpretable, and can be computed directly from ground-truth geometry without learned reward models.
  • The experiments cover different spatial reasoning tasks and model sizes, with consistent improvements across three benchmarks.
  • The ablation studies and fidelity analyses provide useful evidence for the effectiveness of intermediate tool supervision.
Critical weaknesses
  • The improvement from process rewards alone is relatively small on VSI-Bench (40.52% vs. 38.89%). Additional ablations on MAZE and SPAR-Bench would help clarify their contribution to the overall gains.
  • The LCS-based path reward does not explicitly penalize unnecessary detours or invalid transitions. A path may receive a high score even if it is not physically valid.
  • The grounding reward uses the maximum IoU across predicted boxes, which may allow the model to receive a high reward despite producing multiple incorrect boxes.
  • The method still relies on supervised tool-use initialization and detailed geometric metadata. Its applicability to less structured environments remains unclear.
  • The paper does not report variance across training runs, making it difficult to assess the stability of the reported improvements.
  • The figures are not clear enough and can be further improved.
Questions for authors

Please see the weaknesses part.

Reviewer_WS9KOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VERSE to train VLMs for spatial reasoning with visual drawing tools. After supervised initialization using ViLaSR’s interface and trajectories, a proposer–solver reinforcement loop generates scene and maze tasks, with a SpatialEvo-derived geometric engine verifying validity and answers. Its Agentic Verifier rewards intermediate drawings through ground-truth box overlap and maze-path alignment, alongside answer correctness. Experiments with two Qwen2.5-VL sizes report gains on three benchmarks, supported by ablations, fidelity analyses, and a matched ViLaSR comparison.

Critical strengths
  • Geometric supervision with informative diagnostics. Engine-derived rewards target faithful tool use beyond answer correctness. Within-question and endpoint-controlled analyses support path fidelity as a useful process signal beyond destination matching.
  • Careful controls and transparent evaluation. The matched ViLaSR comparison and Stage-3 split audit strengthen the evaluation. The authors acknowledge comparable performance and show that tool access alone substantially degrades the untuned model’s accuracy.
Critical weaknesses
  • Limited evidence for the process reward’s contribution. The verifier adds 1.63 points over outcome-only RL on VSI-Bench, without reported training-seed uncertainty. Missing cross-benchmark ablations and fidelity comparisons leave its role in preventing reward hacking unclear. Performance comparable to matched ViLaSR further limits evidence of a distinctive benefit.
  • Incomplete enforcement of geometric fidelity. As written, the box reward ignores extra incorrect boxes, while the path reward can fully credit routes that include the reference path plus invalid excursions. These potential loopholes weaken the claim of ensuring faithful tool use, although actual exploitation is not demonstrated.
Questions for authors
  • How do reflective SFT, outcome-only RL, and full VERSE compare on MAZE and SPAR-Bench? Can you provide training-seed uncertainty for the process reward’s gains?
  • Does adding process rewards improve held-out box and path fidelity over outcome-only RL? Is there direct evidence that process rewards reduce reward hacking or training collapse?
  • Does the implementation check for extra incorrect boxes, wrong-frame matches, invalid path transitions, or wall crossings beyond the displayed equations? What rewards do deliberately constructed invalid examples receive?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.