acceptodds
Withdrawn from ICLR 2027

ASK THE WORLD: Correction-guided active perception for VLAs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action (VLA) policies can fail when the spatial relation needed for an action decision is not visible from the current observations. Inference- time correction offers a way to improve frozen policies, but a proposed correc- tion can itself be uncertain: the available evidence may be insufficient to deter- mine whether applying it would actually help. We introduce Ask the World, a correction-guided active perception framework that uses a tentative correction to determine what the robot should observe before intervening. Our key idea is to treat the correction as a decision-directed visual query: it identifies the spatial re- lation that must be resolved to accept or reject the intervention. When the current evidence is insufficient, Ask the World selects a viewpoint that exposes this rela- tion, acquires a new observation, and reassesses the correction before execution. We instantiate the framework with LingBot as the frozen base VLA and Astra as an inference-time visual reasoning assistant. LingBot provides the nominal action chunk, while Astra proposes bounded local corrections, assesses whether the native observations provide sufficient evidence, and acquires additional views only when needed. The newly acquired evidence is used to reassess the correction against the same nominal action before execution.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Active Perception, Bimanual Manipulation, Vision-Language-Action Models, Vision-Language Models, Task-Guided Viewpoint Selection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JycmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work addresses a core problem, the vulnerability of VLA policies when crucial spatial relations are occluded or missing in current views. This can cause both nominal actions and tentative inference-time corrections to carry high uncertainty. To solve this problem, the authors introduce "Ask the World", a correction-guided active perception framework that treats a tentative correction as a decision-directed visual query to identify key spatial relations that must be resolved. The framework uses LingBot as the frozen base VLA policy to gather nominal action chunks, paired with Astra as an inference-time visual reasoning assistant to process local corrections and manages active view acquisition.

Critical strengths

(a) Novel formulation of action corrections as visual queries. In this work, instead of treating inference-time corrections isolated as direct trajectory modifications or relying on generic visual exploration, authors innovatively treat a tentative correction as a decision-directed visual query. This novel concept explicitly links active perception to the spatial relation needed to accept or reject an intervention.

(b) Inference-time assistance without retraining base models. The proposed framework operates around a frozen VLA policy and uses a general-purpose visual reasoning assistant without updating the parameters of model and no need retraining the base policy. This provides a flexible, modular pathway to enhance policy robustness at inference stage.

Critical weaknesses

(1) My first concern is that: the evaluation relies only on a virtual camera in the RoboTwin simulator to acquire new viewpoints. The authors also acknowledge that this bypasses the real-world physical constraints, collision risks, trajectory execution time, and hardware costs associated with physically moving an embodied camera.

(2) Despite performance gains are significant on spatially occluded tasks, the macro-average improvement across all 16 tasks over the base frozen LingBot policy is relatively margin (93.13% vs. 91.38%). On several tasks (e.g., Grab Roller, Scan Object), the system actually performs even slightly worse than the baseline policy.

Questions for authors

Q1: How does the framework balance the physical time, energy, and collision risks of moving a physical camera when transitioning from simulator virtual views to real world scenarios?

Q2: Why authors designed Astra to introduce unnecessary corrections that drop performance on tasks where the base policy already perform well (e.g., Grab Roller falling from 100% to 95%)?

Reviewer_mwXXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Ask the World, an inference-time framework built on a frozen VLA (LingBot). A VLM (Astra) proposes a local correction to LingBot’s action chunk. When the native views cannot judge that correction, the VLM renders an additional virtual-camera view chosen to resolve the relevant spatial relation, then decides whether to apply the correction. Experiments on RoboTwin 2.0 manipulation tasks report a modest improvement in average success rate over LingBot.

Critical strengths
  1. Distinguishing object visibility from decision sufficiency is a reasonable motivation, and using a pending correction to decide what to observe is intuitive.

  2. The method is training-free and keeps the base policy frozen.

Critical weaknesses
  1. The central claim is untested. No ablation isolates correction-guided view selection, such as correction without extra views, random extra views, or the fixed-residual baseline the authors themselves call strong. The comparison to RoboDawn and RoboProbe, which use VLMs as policies, does not address this claim.

  2. The significance of the gain is unclear. The improvement over LingBot is small. The paper does not report the number of evaluation episodes, seeds, variance, or confidence intervals. The average gain comes mainly from a single task (Hanging Mug), and the method underperforms LingBot on several other tasks.

  3. The manuscript appears unfinished. Template text remains in the title (“Conference Submissions”), and the appendix contains only placeholder text.

Questions for authors
  1. How many episodes and seeds were used per method and task? Were all methods evaluated on the same episodes and seeds?

  2. Could the authors report ablations on the main benchmark that isolate query-guided view selection?

Reviewer_fBqeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed a correction-guided active perception process for bimanual robot manipulation in simulation, where a frozen LingBot model is used as the base VLA to derive the action chunks, and then Astra is employed to answer visual query questions: whether a correction action is needed and whether new visual evidences from a virtual camera is needed to evaluate the correction action. The process is referred as Ask the World, which was implemented and evaluated on 16 bimanual manipulation tasks in the RoboTwin simulator.

Critical strengths

The set up of Ask the World, i.e., using a frozen LingBot VLA model to generate the actions and using Astra to guide the active perception for the plausible correction actions, is novel and very well motivated.

The implementation and evaluation on 16 bimanual manipulation tasks in RoboTwin achieve somewhat reasonable improvement to an average success rate of 93%.

Critical weaknesses

The novelty and contributions of the paper lie on the set up of combining LingBot and Astra in Ask the World. In some sense, this is a work to evaluate and verify the capability of Astra in active perception for VLAs.

The major empirical results, i.e., improving the average success rate from 91.38% of baseline LingBot to 93.13% in Ask the World, are not that significant and convincing. Actually, if excluding the improvement from 49% to 70% for \emph{Hanging Mug}, the average success rate would drop for Ask the World. No improvement on 7 out 16 tasks. Thus, this about 2% gain on average success rate may be biased. Therefore, the generalization capability of Ask the World is in question.

Overall, the set up of Ask the World is novel and quite cute. Yet the empirical results are not comprehensive and convincing to support the claimed advantages. I am on the fence of weak rejection and weak acceptance.

Questions for authors

Any way to verify this Ask the World process work for sim-to-real on real robot manipulation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.