- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies rapid deployment-time adaptation of robots to new environments. The authors argue that repeatedly refining a provided reference task may concentrate experience around a single solution and fail to acquire knowledge useful under changed conditions. To address this, the paper introduces ProactiveVLA, which allocates part of the limited interaction budget to proactive environment exploration and acquires reusable knowledge about object affordances, state-changing interactions, and interaction compositions. The acquired experience is then consolidated into memory to guide subsequent task execution.
Critical strengths
-
Compared with Harness VLA, ProactiveVLA extends deployment-time adaptation from pure task refinement to proactive environment exploration, and combines visual verification with memory consolidation to reuse the acquired experience in subsequent tasks. The overall motivation is intuitive and well aligned with the goal of rapid adaptation to new environments.
-
The method is evaluated on LIBERO-Pro and RoboCasa365 Composite-Seen. On LIBERO-Pro, ProactiveVLA improves over Harness VLA in seven of the eight settings, increasing the overall success rate from 72.1% to 76.4%. In addition, on LIBERO-Pro Goal-T, when at most one VLA primitive invocation is allowed during evaluation, ProactiveVLA achieves 48% success compared with 19% for Harness VLA, an improvement of 29 percentage points.
Critical weaknesses
- Presentation and clarity
1.1. The adaptation-budget protocol appears internally inconsistent. Sec. 3.1 suggests that a fixed b_init turns are allocated to the provided task, after which the remaining (B - b_init) turns are used for exploration. However, Algorithm 1 does not explicitly contain b_init, while the implementation section states that exploration uses the remaining budget “after task success.” It should be clarified whether exploration begins after a fixed number of initial turns or only after successful task completion, and what happens if the reference task cannot be completed within the budget.
1.2. The variable e_i in Algorithm 1 is not clearly defined. The paper should explicitly state whether it represents a scene, an environment instance, or another entity.
1.3. Figure 3 is difficult to interpret. The task instruction and target object are not clearly shown, and the localization outputs are not visualized. The key distinction between “RE-LOCALIZE,” “NO RE-LOCALIZE,” and “WRONG POSITION” is mainly conveyed through textual annotations rather than observable evidence. Therefore, the figure does not convincingly demonstrate that an exploration-derived recovery rule caused the successful behavior. Showing the retrieved memory rule, localization/tool traces, and the supporting adaptation evidence would make the analysis more convincing.
1.4. The auxiliary model configuration is insufficiently specified. Codex is identified as the high-level agent, but it is unclear whether the language-model proposer uses exactly the same model/configuration. More importantly, the specific model used as the independent visual checker is not identified.
- Experimental validation
2.1. The sets of comparison methods differ substantially across the two benchmarks. This is not necessarily inappropriate, but the paper should better explain which baselines are available or applicable to each benchmark and why some related agentic methods are absent from the RoboCasa365 comparison.
2.2. The statistical support on RoboCasa365 is limited. GR00T N1.5 and the StarVLA variants are evaluated with 30 or 50 rollouts per task, whereas Harness VLA and ProactiveVLA use only five. Although the main Harness VLA vs. ProactiveVLA comparison is matched, the resulting estimates have substantially different statistical precision across methods.
2.3. Five rollouts per task make the per-task success rates very coarse. A single additional success changes the reported rate by 20 percentage points. Therefore, improvements such as 20%→40% correspond to only one additional successful rollout. More rollouts, confidence intervals, or statistical tests would be needed to support fine-grained per-task conclusions.
2.4. The current experiments do not fully establish that the improvement specifically comes from proactively acquired environment knowledge. ProactiveVLA simultaneously changes the exploration prompt, memory organization, high-level task decomposition, and tool-use behavior. The paper itself shows that ProactiveVLA produces substantially more fine-grained action-level VLA instructions and uses analytic control more frequently. A matched baseline such as Harness VLA with explicitly enforced fine-grained action-level decomposition/prompting would help disentangle the effect of proactive environment knowledge from improved prompting and tool coordination.
2.5. It would also be valuable to study whether the advantage of proactive exploration persists as ordinary task-execution experience accumulates over time, for example by comparing task-only continual memory with task memory + proactive exploration. This would clarify whether the method primarily benefits cold-start rapid adaptation or remains useful during longer-term deployment.
- Novelty
3.1. ProactiveVLA is largely instantiated on top of the Harness VLA agentic execution stack, with the main additions being proactive exploration, three-level exploration objectives, visual verification, and global memory consolidation. The system is reasonably structured, but the technical novelty appears relatively incremental. In particular, the ablation results show only modest drops when removing initial-task guidance or memory management, from 71% to 68% and 67%, respectively, whereas removing the three-level exploration structure reduces performance to 42%. This suggests that a substantial portion of the gain comes from the manually specified three-level exploration scaffold/prompt. Since the exploration categories, ordering, and minimum exploration requirements are explicitly encoded in the prompt, it remains unclear how much of the contribution should be regarded as a new algorithmic mechanism rather than carefully engineered prompting and memory organization.
Questions for authors
For the submission, the following should be addressed:
-
For presentation and clarity, the issues summarized in the Weaknesses section should be clarified, particularly the adaptation-budget protocol, notation in Algorithm 1, interpretation of Figure 3, and the auxiliary model configurations.
-
The evaluation protocol on RoboCasa365 would benefit from further clarification and stronger statistical support. The compared methods use substantially different numbers of rollouts, while Harness VLA and ProactiveVLA are evaluated with only five rollouts per task, meaning that a single additional success changes the reported success rate by 20 percentage points. Could the authors explain the choice of evaluation protocol and provide more rollouts, confidence intervals, or statistical tests to support the per-task conclusions?
-
The current results do not fully disentangle whether the improvements come from proactively acquired environment knowledge or from changes in prompting, task decomposition, and tool usage. Since ProactiveVLA produces more fine-grained action-level VLA instructions and uses analytic control more frequently, it would be helpful to include a matched Harness VLA baseline with similarly fine-grained action-level decomposition or prompting.
-
It would also be valuable to study whether the advantage of proactive exploration persists as ordinary task-execution experience accumulates over time. For example, how does task-only continual memory compare with task memory + proactive exploration? Such an experiment could clarify whether the proposed method mainly benefits cold-start rapid adaptation or remains useful during longer-term deployment.
24 comments
2. What happens if the reference task cannot be completed within the budget? (Weakness 1.1, Question 1)
Thank you for raising this question. In practice, the initial task is successfully completed in the vast majority of adaptation instances, as the allocated interaction budget allows multiple attempts and recovery from execution failures. Therefore, ProactiveVLA can typically proceed to proactive exploration as intended.
For the small fraction (about 2.5%) of cases where the initial task cannot be completed within the budget, the run does not enter autonomous exploration, but the agent still preserves the failed attempts and their execution feedback as negative experience for subsequent memory construction. These records provide useful information about unsuccessful interactions and physical constraints, even without a completed reference task. Importantly, all attempts share the same interaction budget, ensuring a consistent evaluation setting.
We will clarify this behavior and the transition to proactive exploration in the revised paper. As future work, we plan to investigate more adaptive strategies for cases where the initial task cannot be completed, such as targeted failure diagnosis and exploration aimed at identifying the underlying physical constraints and possible recovery strategies.
3. Presentation and clarity (Weakness 1.2, Weakness 1.3, Weakness 1.4, Question 1)
Notation of the variable e_i in Algorithm. We will define e_i as the reference environment instance, including its scene and initialization, paired with the provided task instruction g_i.
Presentation of Figure 3. We sincerely thank you for the specific suggestions and have revised the paper (now it's Figure 4). The key decision is what happens after the manipulator is moved away: ProactiveVLA localizes the target again before continuing, whereas HarnessVLA proceeds without that update in the illustrated execution. We have revised our paper to make this comparison directly inspectable by showing the task instruction and target, the localization outputs at that decision point, the retrieved memory rule, and its supporting adaptation evidence.
Report of the auxiliary model configuration. Thanks for your feedback, we have carefully revised our paper to include them. The task proposer, visual checker, and memory-processing modules use GPT-5.6 Sol with high reasoning effort, with verification and evidence review performed in separate contexts. For VLA control, we use the RLinf-released π0.5-SFT checkpoint on LIBERO-Pro and RLDX-1 on RoboCasa365.
1. The adaptation-budget protocol and when exploration begins (Weakness 1.1)
We sincerely thank you for your effort in reviewing our paper. We sincerely thank you for the detailed review and for recognizing the motivation of proactive exploration and the gains under limited VLA calls. During the discussion period, we revised the method description and recovery figure to improve clarity, and conducted additional experiments, including a real-robot comparison, evaluations with an alternative planner and VLA controller, and an expanded 800-rollout RoboCasa365 evaluation. We further introduced a matched memory-processing baseline, repeated memory acquisition, and pooled-memory evaluation to better characterize the contribution of proactive exploration in the revised paper. We address each point below.
Total budget for adaptation. The adaptation budgets are:
Each attempt in adaptation uses min(24, remaining_turns) and min(12, remaining_actions). Actual consumption is deducted from the shared totals, so the procedure is not equivalent to exactly four attempts. Following an unsuccessful attempt, the agent resets and retries the provided task while both total allowances remain positive.
When exploration begins. After the first success, ProactiveVLA resets once to the seed-0 initial state and enters autonomous exploration if at least 4 requests and 1 action call remain. Exploration receives only the unused allowance. HarnessVLA continues refining the provided objective under the same total limits.