acceptodds
Withdrawn from ICLR 2027

VeriACT: Action-Grounded Verification for Efficient Interactive Visual Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language models can reason effectively about static visual inputs, yet interactive visual reasoning remains challenging because errors in perception or action can propagate across subsequent decisions. An agent may fail to notice the effect of an action, repeat ineffective actions, or continue planning from stale beliefs, wasting costly environment interactions. We introduce VeriACT, a task-agnostic interaction framework that makes verification an explicit and consequential part of the agent loop. VeriACT requires each action to state a falsifiable expectation and, before the next action, verifies the resulting environment transition against that expectation. It combines model-based verification with controller-grounded visual signals, including frame-change and repetition detection, multi-frame transition analysis, click-target confirmation, and terminal-state verification. Verification outcomes can trigger replanning and conditional memory updates, allowing new evidence to directly modify the agent's subsequent behavior. We evaluate VeriACT on ARC-AGI-3 against a free-form control while holding the vision-language model, observation interface, memory, and evidence tools fixed. In a 13-game evaluation, VeriACT improves mean RHAE from to and reduces environment actions by % on levels successfully completed by both conditions, at the cost of increased inference-time computation. These results suggest that explicit, action-grounded verification can trade additional inference-time computation for greater task progress and fewer environment interactions, even when the underlying reasoning model remains unchanged.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Visual Abstract Reasoning, Interactive Reasoning, Verification, Agentic Reasoning, Vision-Language Models, ARC-AGI-3

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9e1wOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces VeriAct, a framework within an agent loop that explicitly and consequentially integrates the verification of actions and updates its working memory when needed. The motivation centers on the capability of VLMs to handle interactive visual reasoning cases, specifically in terms of performance and efficiency. They claim that agents are not verifying their observations against their expectations, and raise the question of whether this is the root cause for missing changes, repeating actions, or continuing planning from stale thoughts. They evaluate it on a subset of the ARC-AGI-3 benchmark against a baseline without verification.

Critical strengths

The idea of making the model think about its expectations before execution is a compelling tier of the verification modeling, alongside the macro-verification of objective preservation. Additionally, making the verification operational achieves a notable improvement in action efficiency.

Critical weaknesses
  • Lack of explanation: The paper assumes preliminary familiarity with ARC's tasks, game mechanics, no proper introduction of the task setup and evaluation metrics.
  • Limited baselines: The framework is only evaluated against a "vista-shaped" agent, which lacks a verification phase. What about comparing it against other baselines that include a verification phase (e.g., reflection, retry, self-critique) or non-vision-based agents (text-only)?
  • Limited experimental setup (Is the framework robust?):
    • Model variety: Why were no other VLMs evaluated besides Qwen3.8? Models differ significantly in scale and in their text/vision encoders.
    • Task selection: The evaluation uses only 13 games. Why wasn't the framework evaluated on the full public game set? How do these selected games differ from one another, and what different challenges do they represent?
    • Generalizability: Are there other types of games or interactive visual reasoning setups where this framework could be tested?
Questions for authors
  • Level progression: How does verification within the framework change (e.g., does it become more challenging) as the levels progress?
  • Uncertainty computation: Can the uncertainty somehow be computed internally from the VLM agent (for example, by using its token probabilities)? Furthermore, is the uncertainty report calibrated to this?
Reviewer_AS2VOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a task-agnostic interaction framework, VeriACT, that requires VLM to perform an explicit verification inside agent loop. Before action execution, the VLM would be required to state an execution result expectation, which is compared with the real execution result. The framework combines this model-based verification with controller-grounded visual signals. The framework is evaluated on 13 games from the ARC-AGI-3 benchmark using a fixed Qwen3.8-27B model. The VeriACT framework improves the mean RHAR from 1.72 to 6.36 and shows environment actions reductions.

Critical strengths
  • The paper provides this explicit verification step, which is an intuitive way of approaching interactive visual reasoning with unknown rules. This mechanism enables the framework to incorporate action feedback and updates its own understanding and memory, which is practical.
  • The experiments show that the framework could effectively improve the RHAR score, and also show environment action reductions.
Critical weaknesses
  • The evaluations are not extensive enough to prove the effectiveness of the method. The method is only evaluated on 13 games from a single ARC-AGI-3 benchmark. Especially, table 1 shows that a large portion of the gain comes from a single game ft09. And out of the 13 games, 4 games’ RHAE decreases. Without testing on more games on diverse benchmarks, it is hard to prove that the method is a task-agnostic framework as it is claimed, especially when the gains from other games are much smaller than from a single game.
  • Although the experiments show that the framework can reduce environment actions on the shared solved levels, it uses 2.29× as many tokens since the framework adds separate reasoning stages. The paper also does not provide targeted ablations to separate the gains from this introduced verification step or simply additional checking. This explicit verification is the central contribution claimed in the paper but is not sufficiently supported.
Questions for authors
  • The paper mentioned VISTA is already predicting action outcomes and describing subsequent changes, which makes the central difference to be “controller-enforced, action-grounded verification”. Could the authors provide ablation studies to separate the effects of different parts of the proposed methods and isolate the benefit of controller enforcement?
  • Could the authors provide more evaluations on different games on diverse benchmarks?
  • Since the number of games and trials is small, could the authors report how were the 13 games selected, and how many independent runs were performed for each game and each method? Since in the paper you mentioned using temperature 1.0, could the authors repeat the experiments for multiple times to assess whether and show if the large gain on ft09 and the decrease on the four other games are reproducible? Also, more analysis of why ft09 has such a large gain is needed, and more analysis of failure cases is also needed to better interpret the results.
Reviewer_3Z4NOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes an agentic framework with verification for interactive visual reasoning. The experiment is conducted on ARC-AGI-3, and VeriACT outperforms the control group in terms of RHAE and Actions/Level in some game splits. However, the idea is not novel and the paper seems to be incomplete.

Critical strengths

The verifier-enabled framework makes the agent outperform the control group in terms of RHAE and Actions/Level in some game splits.

Critical weaknesses
  1. The method is not novel. There are already a lot of works using verifier to enhance agents, like [1]. The paper doesn't state the uniqueness of VeriACT clearly. The verifier logic seems to be simple. Therefore, the novelty is incremental.
  2. The setting is not stated clearly. For example, what kind of task is this paper focusing on?
  3. The method section lacks technical details, and the architecture figure is merely flow chart. The detail of each component should be specified.
  4. The benchmark is insufficient, and the paper doesn't compare with recent agents.
  5. There is no ablation study in the paper.
  6. The paper seems to be incomplete, as the appendix is empty.

[1] Madaan, Aman, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon et al. "Self-refine: Iterative refinement with self-feedback." Advances in neural information processing systems 36 (2023): 46534-46594.

Questions for authors

I believe substantial improvement is required before this paper is ready for acceptance. The authors should focus on following questions:

  1. State the novelty of this paper, compared against other verifier-enabled agentic workflows.
  2. State clearly what kind of setting this method is focused on.
  3. Rewrite method part with technical details, and revise the architecture figure to include more information rather than information flow.
  4. Add more benchmarks, and compare with representative agent workflows.
  5. Add ablation to each component.
  6. Show more detailed examples with prompts and responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.