acceptodds
Under review as a conference paper at ICLR 2027

RoboHarn-Evo: Evolving Hierarchical Physical Knowledge for Self-Improving Robotic Manipulation

Abstract

Vision-language models can coordinate long-horizon robot manipulation, yet successful task reasoning still depends on whether local physical interactions produce the intended effects. We study how repeated interaction can improve this capability without updating the base model. We introduce RoboHarn-Evo, a dual-loop harness that evolves Hierarchical Physical Knowledge (HPK) from physical experience. HPK couples two levels of reusable knowledge: Task Knowledge captures which subtask should be executed and when it is complete, while Action Knowledge captures object-relative geometric strategies and their physical effects. During execution, the agent retrieves knowledge at the corresponding decision level and grounds it in the current scene under the task goal. Across episodes, physical feedback is used to revise historical knowledge, update its applicability, and organize reusable entries for subsequent retrieval. Experiments on RMBench show that HPK improves average success by up to 24.2 percentage points across different agent models. With 80 interaction rollouts, held-out success rises from 48.3% to 75.0% for GPT-5.5 and from 70.0% to 88.3% for GPT-6. RoboHarn-Evo also resolves over 83% of historical knowledge errors while retaining 95.8% of valid knowledge, and transfers zero-shot from RMBench to RoboDojo with gains of 35.0 and 25.0 percentage points. These results demonstrate that physical interaction can be accumulated into reusable knowledge for improving subsequent manipulation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Agentic Manipulation, Self-Evolving Robot Manipulation, Robot Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rtGYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RoboHarn-Evo, a dual-loop intelligent agent framework that enables the improvement of a robot’s operational capabilities through physical interaction, without the need to update the underlying VLM or lower-level actuators. The hierarchical physical knowledge structure employed by this framework includes task knowledge used to guide the selection of subtasks and to evaluate their completion level, as well as action knowledge used to describe the relative geometric relationships between objects and local physical effects. The internal execution loop is responsible for retrieving and applying this knowledge, while the external evolution loop utilizes physical feedback to verify, refine, and maintain this knowledge.

On the RMBench platform, the authors tested RoboHarn-Evo using various VLM models, and they conducted analyses of the effectiveness of knowledge maintenance, experiments on continuous interaction, comparisons across different benchmarks, as well as tests in real-world scenarios. The results show that, with unchanged model parameters, the robot’s task completion rate and performance can be improved through the accumulation of interaction experience over time.

Critical strengths
  1. A sound design for the maintenance of hierarchical knowledge. This paper separates the completion of sub-tasks from the actual physical actions performed, and utilizes this distinction to maintain knowledge related to tasks and actions. This design addresses a major issue in long-duration robotic operations: faults and the evidence related to them can occur at different decision levels.

  2. A relatively comprehensive empirical evaluation. The authors report that this method results in significant improvements in performance across various visual language model architectures and operational tasks. They also investigated the correction of historical knowledge, the performance gains resulting from continuous interaction, the ability to transfer knowledge across different environments, and the performance in real-world applications. In particular, comparisons using data from the same source provide strong evidence for the effectiveness of the entire knowledge maintenance framework.

Critical weaknesses

This paper addresses a meaningful issue and proposes a framework with a solid theoretical foundation, while also showing improved empirical results. However, the differences between its core methodological contributions and those of existing research are not significant; furthermore, the hierarchical evidence attribution mechanism proposed lacks direct and controllable comparative analysis. As a result, the current evidence is not sufficient to fully support the innovative aspects it presents.

  1. The concept of hierarchical evolution of physical knowledge does not differ markedly from that of existing research.

Previous studies have explored topics such as hierarchical representation of experience, knowledge updating based on feedback, as well as the verification and reuse of physical knowledge. For example, XSkill separates skills at the task level from experience at the action level; PhysMem investigates how knowledge that has been physically verified can be accumulated; EmbodiSkill distinguishes between skill deficiencies and execution errors.

In particular, the paper does not clarify in what physical interaction scenarios existing methods may incorrectly extract, attribute, or update knowledge at different decision levels. Although Figure 1 illustrates fault conditions at various levels, these examples are not directly linked to the specific limitations of the existing methods.

Therefore, it remains unclear which specific limitations of existing knowledge evolution methods the proposed hierarchical evidence attribution mechanism addresses, and what fundamental technical contributions it offers compared to these existing approaches.

  1. The hierarchical evidence attribution mechanism proposed has not yet been directly tested, and comparisons with other self-evolving methods are also limited.

The three experimental scenarios listed in Table 2—“only task level,” “only action level,” and “planar reflection”—demonstrate the usefulness of different knowledge levels and their combinations; the knowledge maintenance experiments shown in Figure 4 also highlight the advantages of evidence-based verification methods. Nevertheless, these experiments do not directly prove whether the hierarchical evidence attribution and maintenance mechanism is superior to those knowledge evolution mechanisms based on uniform verification standards.

A more precise experimental design should maintain the same “task+behavior” knowledge representation, information source trajectories, retrieval mechanisms, underlying models, and executors, and then compare the proposed hierarchical verification mechanism with a unified knowledge verifier that can access complete trajectory evidence. This would allow for a more accurate assessment of the actual contributions of each verification mechanism at different levels.

Additionally, the existing external benchmark methods are mainly strategy-based approaches and Harness VLA. Comparisons with methods related to self-evolution of knowledge or skills are still insufficient. The authors should consider incorporating mechanisms such as PhysMem, EmbodiSkill, or others similar to them, or explain why these methods are difficult to apply effectively in the current evaluation environment.

There are also some issues related to cross-references and inconsistent terminology:

  1. When describing the retrieval and maintenance processes, Section 4.3 refers to Figure 8, whereas the relevant overview should be Figure 3; moreover, Figure 3 is not mentioned anywhere in the main text.
  2. Appendix E.7 references Section 3.4, which does not exist.
  3. Appendices E.1 and E.7 use the term “Action-Feedback Knowledge” (AFK), while the main text uses “Hierarchical Physical Knowledge” (HPK). These terms should be consistent, or it is necessary to clarify the relationship between these two concepts.

These issues relate mainly to the way things are presented and to consistency, rather than the two core technical problems mentioned earlier.

Questions for authors
  1. Compared to XSkill, PhysMem, and EmbodySkill, what specific unsolved problems does the proposed hierarchical physical evidence attribution mechanism address? Can the authors provide a concrete example of a failure case, illustrating how existing methods would incorrectly update knowledge, and how RoboHarn-Evo avoids such errors?
  2. Can the authors design a controlled experiment in which, while keeping the same task and representation of action-related knowledge, execution trajectories, retrieval mechanisms, and computational resource consumption, the hierarchical evidence verification mechanism is replaced by a unified knowledge verification mechanism? This would help quantify the independent contribution of the hierarchical evidence attribution mechanism.
  3. Can the authors further explain the rationale behind selecting these external benchmark methods? Additionally, it would be useful to compare them with PhysMem, EmbodySkill, or other methods capable of knowledge evolution or feedback-based skill refinement. If direct comparison is not feasible, please describe the specific differences in their implementation or experimental setup.
  4. In Appendix A.1, it is mentioned that Appendix D contains detailed information regarding the resources required for knowledge construction, model configurations, and repeated experiments. However, in reality, Appendix D mainly describes the definitions of metrics used for knowledge correction and transfer evaluation. Can the authors provide the relevant experimental configurations and details of the repeated experiments, or at least indicate where such information can be found?
Reviewer_NtFrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RoboHarn-Evo, a framework that improves robot manipulation by updating an external knowledge store while keeping other main components fixed. There are two knowledge types: Task Knowledge describes what subtask to perform and when it is complete, while Action Knowledge describes how to carry out an operation and what physical effect to expect.

Critical strengths
  1. The key finding is very interesting: The knowledge maintenance procedure should be more carefully designed than simply storing past experience. The evaluation also distinguishes repairing incorrect knowledge from deactivating it, and measures whether correct knowledge is preserved.
Critical weaknesses
  1. Important experimental details are unclear. I could not find a complete account of the trajectories used to build the knowledge stores, including their number, source, and allocation across tasks. The exact model configurations, action budgets, and retry limits are also unclear. Furthermore, the specific benefit of the hierarchical design is not fully ablated. The ablations show that the complete system works better than its simpler variants. However, the variants may also differ in the amount of information, retrieval quality, prompt length, and model calls.

  2. The real-world evidence is still limited. The task difficulty or complexity, the trials and the comparison settings with the baselines could be further carefully designed.

Questions for authors
  1. Could you provide the knowledge construction budget, trajectory sources, model configurations, and learning/evaluation splits for each experiment, especially the real-world study?

  2. How was the initial incorrect knowledge pool constructed?

Reviewer_QMy7OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes RoboHarn-Evo, a framework for improving long-horizon robotic manipulation through persistent knowledge evolution without updating the underlying vision-language model (VLM) or robot executor. The central contribution is Hierarchical Physical Knowledge (HPK), which contains Task Knowledge (which subtask to execute, the relevant target and goal, and the conditions for verifying completion) and Action Knowledge (object-relative manipulation strategies, including grasp geometry, contact conditions, and expected physical effects).

Critical strengths
  1. The paper's problem is timely about self improvement of agentic robotics.
  2. The paper's central claim is to provide the clear separation between task-level and action-level feedback, which is reasonable and sound.
  3. The paper provides comprehensive evaluation on a lot benchmarks, including new ones such as RMBench or RoboDojo from this year.
Critical weaknesses
  1. The paper overall writing is hollow, and non-scientific. For example, the usage of advertisement-alike words such as "RSI" without definition, component definition such as "action grounding" "primitive execution" "Relative geometry and effect" ... in every major figures. These are very loose terms that lack any referral from main text and caption.
  2. Separating the Task Knowledge and Action Knowledge can date back to decades ago to Task and Motion Planning (TAMP) to later this year of TipTop by Shen et al. Also, I don't think the paper fully distinguish itself from its related work such as PhysMem and XSkill. It's not clear that how much the proposed mechanisms advance beyond combining established hierarchical memory and physical verification methods.
  3. The overall method and its evaluation is also very loose - for example, relying on GPT 6-as-judge, task generalization, comparison to only VLA policies which have less data.
Questions for authors
  1. What is the fundamental contribution of the paper that would distinguish from prior work?
  2. What's the major ingredients that support the authors' claims and how would you curate the evaluation data to support those claims?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.