acceptodds
Withdrawn from ICLR 2027

VISTA: Active Exploration for Self-Improvement in Embodied Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

17 views

Abstract

Agentic embodied intelligence plays a foundational role in open-ended, dynamic physical environments. Rather than treating policy learning as an end-to-end mapping from instructions to actions, recent studies reveal that embodied systems depend not only on pre-trained foundation models, but also on harness. Prior approaches largely treat rollouts as static logs for post-hoc reflection, producing harness updates without autonomously testing and refining them through subsequent embodied interaction. To address this challenge, we propose VISTA, a meta-agent framework that transforms post-hoc rollout reflection into active self-exploration for embodied harness improvement. Specifically, VISTA maintains a harness DAG that records the lineage of proposed harnesses, which enables the meta-agent to retrieve prior hypotheses, build on effective mechanisms, and avoid repeatedly exploring failed directions. At each iteration, VISTA first evaluates the current harness to obtain rollout evidence and initiaze a candidate harness. Rather than directly committing an update from these historical trajectories, the meta-agent actively explores in the environment. It autonomously generates new rollouts, probes execution failures and recovery behaviors, and iteratively modifies the harness based on the resulting evidence. The refined harness is then evaluated afresh and added to the harness DAG. We evaluate VISTA on the LIBERO-Pro and RoboCasa benchmarks. Experimental results demonstrate that VISTA progressively improves task success and is more rollout-efficient than static reflection, achieving higher success rates with fewer simulator rollouts.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Harness, Recursive self improvement, Embodied agent

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_57sxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VISTA uses a coding meta-agent to improve an embodied agent’s harness while keeping the underlying models fixed. It combines active simulator diagnosis, code revision, and a DAG of previous harnesses. Experiments show large improvements on LIBERO-Pro development instances and smaller gains on RoboCasa365.

Critical strengths
  1. Active diagnosis is a useful idea for failures that recorded rollouts cannot explain well. Figure 6 gives a concrete example: correcting the lift reference and release behavior improves actual execution.

  2. The gains cover all 40 LIBERO-Pro tasks in Table A2. This supports the usefulness of harness adaptation beyond a few selected examples, although these are development results.

Critical weaknesses
  1. Appendix C.1 reports results on the same 16-cell development matrices used for harness selection. Fresh episodes on these instances do not establish generalization to held-out seeds. This is my main concern about the near-perfect LIBERO-Pro results.

  2. The RoboCasa improvement is unclear. Table 2 gives an initial Seen score of 18.4%, while Table A3 reports 59/256, or 23.0%. The latter implies only about a 2.7-point gain to 25.8%, rather than the claimed 7.8 points. This materially changes the evidence from the second benchmark.

Questions for authors
  1. Evaluation clarification: Do the final frozen harnesses retain their gains on seeds excluded from all development and selection?

  2. Results clarification: Which RoboCasa scores are correct, and how should Tables 2/A3 and Section 4.2 be reconciled?

  3. Budget clarification: How does the 110% token cap in Section 4.3 allow 94.2M passive versus 42.2M active tokens in round 1? What are the total simulator rollout counts, including diagnostic interactions, for both methods?

Reviewer_FQf9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VISTA, a self-improvement framework for embodied-agent harnesses. VISTA maintains a directed acyclic graph (DAG) of previously developed harness versions and introduces active diagnostic exploration, allowing a meta-agent to interact with a private simulator, investigate execution failures, and iteratively refine the harness. The framework achieves strong performance on LIBERO-Pro, with a mean success rate of approximately 97% on the reported development instances, and demonstrates improvements on RoboCasa365.

Critical strengths
  1. Strong performance on LIBERO-Pro. The proposed framework achieves impressive success rates on LIBERO-Pro, demonstrating its effectiveness in optimizing task-specific harnesses.

  2. Effectiveness of active diagnostic exploration. The active-versus-passive ablation study shows that active exploration achieves better performance while consuming fewer meta-agent tokens in the evaluated tasks. This provides useful evidence supporting the effectiveness of active diagnostic exploration.

  3. Insightful qualitative analysis. The qualitative results clearly illustrate how the harness improves through successive revisions. In particular, the examples of failure diagnosis, branching exploration, and harness refinement make the improvement process easy to understand.

Critical weaknesses
  1. Lack of an ablation study for the DAG-based history representation. Although the paper introduces a DAG to maintain the history of harness revisions, it does not quantitatively demonstrate the contribution of this representation. For example, limiting access to only the previous one or two harness versions could help evaluate the importance of historical information. A comparison against a simpler linear history representation would further clarify the benefits of the DAG structure.

  2. Insufficient clarification of the Harness VLA comparison on RoboCasa365. The original Harness VLA paper reports higher success rates on the RoboCasa365 composite task categories than those reported for VISTA. Although VISTA uses a reimplementation of Harness VLA as its initial harness, the differences between the evaluation protocols and the substantial gap in reported performance are not sufficiently explained. This makes it difficult to assess the improvement relative to prior work.

  3. Concerns about the reliability and completeness of the reported results. The manuscript states that the work "is still in progress and reflects the current snapshot," in Table A3 raising concerns about whether the reported results are sufficiently finalized and validated. Furthermore, there are inconsistencies in the numerical scores reported in the main text, Table 2, and Table A3. These discrepancies raise questions about the reliability of the experimental results and make it difficult to assess the validity of the conclusions. The authors should clarify the status of the experiments and carefully verify the consistency of the reported results throughout the manuscript.

Questions for authors
  1. Evaluation protocol for active diagnostic exploration. Section 3.3 and Appendix B.1 explain that active diagnostic exploration uses private simulator sessions that are separate from formal autonomous evaluation. However, it is unclear whether the task instances and seeds used during diagnostic exploration and iterative harness selection are also included in the final reported evaluation. Could the authors clarify how the development and evaluation instances are separated? In particular, are the final harnesses evaluated on held-out seeds that were not used during diagnostic exploration or harness selection?

  2. Comparison with Harness VLA on RoboCasa365. The paper states that the initial harness for RoboCasa365 is a reimplementation of Harness VLA. However, the success rates reported for the initial harness are substantially lower than those reported in the original Harness VLA paper. Could the authors clarify the differences in experimental settings, including the task configurations, and underlying models, that might explain this discrepancy? A more directly comparable evaluation would help establish the effectiveness of the proposed framework relative to Harness VLA.

  3. Differences in improvement across task categories and refinement rounds. In Figure 3, the Spatial and Goal categories appear to have similar initial success rates at R0, but their success rates differ substantially after the first refinement round (R1). What factors might explain this difference? Additionally, the improvements across successive refinement rounds appear relatively modest for some categories. Could the authors provide further analysis of the factors limiting improvement in these cases?

Reviewer_2bGqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

VISTA proposes a meta self-improvement approach to embodied reasoning, where developing the harness itself is a self-improvement problem. VISTA allows the embodied system to reflect on the outcomes of rollouts (e.g. self-reflect), then utilize the self reflection to augment the harness and build a DAG of harnesses. The authors report performance gains across simulator tasks over foundation model VLAs and HarnessVLA.

Critical strengths
  • Overall, the idea of making the harness itself subject to exploration is interesting. The related work section appears to be thorough, and it seems like a reasonable next step given the importance of harnesses to autoresearch frameworks and their current heuristic nature.

  • The empirical results appear quite impressive, reporting big performance gains in a diversity of tasks over SOTA VLA models in simulated tasks.

Critical weaknesses
  • The paper does not include a Generative AI use disclosure statement, as is required in ICLR submissions.

  • The text was very difficult for me to parse as someone familiar with VLAs, but not completely up-to-date on current literature using autoresearch and VLAs. This is particularly a problem in the methods section. For example, I do not know what ``public capability interface'' means. Grounding Section 3 in more specifics -- e.g. clearer demonstration of Fig 5 (for example, videos that indicate each step of development in an anon github repo), including examples of the evolving harness for inspection, etc. -- would be useful.

  • Similarly, Section 4 describes benchmarks as "For LIBERO-Pro, we evaluate on across Spatial, Object, Goal, and Libero-10 under instruction redirection." I do not know what instruction redirection means. Also, examples of some of these tasks would greatly improve reader understanding, so it is clear what is and is not a part of the Harness, and what an autoresearch pipeline without harness meta self-improvement would look like as a comparison.

Questions for authors
  • How does VISTA compare (conceptually, I am not asking for new baselines) with a VLA autoresearch pipleline that does not adjust a harness? The definition of harness seems very expansive, here, and I struggle to understand what exact degrees of freedom can be subject to active exploration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.