- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the limited efficiency with which existing embodied agentic harnesses utilize costly physical robot interactions. The authors introduce EmbodiedEvo, a self-evolving agentic harness built around a frozen robot foundation model. The goal is to let the harness actively decide what physical experiment to conduct next and transform the resulting evidence into reusable improvements in executable code, skills, and memory.
The framework contains three main components. Value-of-Information Experiment Selection maintains code and skill hypotheses in a Hypothesis Graph and selects physical experiments according to their expected uncertainty reduction relative to their cost. Code-Skill Co-Evolution explicitly evaluates code, skill, and joint effects through matched trials. Reward-Grounded Memory Learning learns memory-management decisions according to their downstream impact on task performance, exploration uncertainty, and physical-experiment cost.
Overall, the work reframes the relatively heuristic execution-feedback-update loop of prior agentic robot systems as an interaction-efficient continual self-evolution problem.
Critical strengths
- The overall system is well integrated, and the Hypothesis Graph provides structured tracking of experimental evidence and code-skill relations.
- Reward-Grounded Memory Learning is interesting, as memory operations are learned according to downstream utility rather than fixed heuristics.
- The method achieves strong results on RoboCasa365 and LIBERO-Pro and is further evaluated on real robots.
Critical weaknesses
- The generation of hypotheses and candidate experiments is under-specified and may depend heavily on LLM proposal quality.
- The Value-of-Information Experiment Selection module is mainly compared with Random Exploration; stronger baselines such as uncertainty-only, information-gain-only, greedy success gain, or LLM-guided selection would be more convincing.
- The cold-start stability of the PPO-based memory manager and correction of early erroneous memories are not sufficiently analyzed.
- In the real-robot experiments, SmolVLA is first fine-tuned with 50 teleoperated demonstrations per task before being frozen for evaluation. This setting demonstrates that EmbodiedEvo can further improve a task-adapted VLA, but it does not fully test one of the key motivations of an agentic harness: improving generalization without updating the underlying VLA. An additional comparison using the pretrained SmolVLA without task-specific fine-tuning would better show whether EmbodiedEvo can provide gains purely through the harness.
Questions for authors
The following points should be addressed.
-
The generation of hypotheses and candidate experiments is not sufficiently clear. It would be helpful to provide more details on how these hypotheses and experiments are proposed, and to what extent the process depends on the quality of the underlying LLM.
-
The Value-of-Information Experiment Selection module is mainly compared with Random Exploration. It would be beneficial to include stronger experiment-selection baselines, such as uncertainty-only selection, information-gain-only selection, greedy success-gain selection, or LLM-guided selection, to better isolate the contribution of the proposed VOI formulation.
-
The Reward-Grounded Memory Learning module uses a PPO-based memory manager, but its behavior during the cold-start stage is not sufficiently discussed. How reliable are the memory decisions before enough downstream rewards have been collected? In addition, if incorrect memories are retained or abstracted at an early stage, how effectively can the system identify and correct them later? Further analysis of the stability and error-correction capability of the memory manager would be helpful.
-
In the real-robot experiments, SmolVLA is first fine-tuned and then frozen for evaluation. While this setting demonstrates that EmbodiedEvo can further improve a task-adapted VLA, one of the main motivations of an agentic harness is to improve generalization without updating the underlying VLA. It would therefore be valuable to additionally evaluate EmbodiedEvo with the pretrained SmolVLA without task-specific fine-tuning, in order to better demonstrate the generalization gains provided purely by the harness.
5 comments
Response to reviewer eHSg (2/3)
W3. Cold-Start Stability and Memory Error Correction
Thank you for your thoughtful observation regarding memory reliability, particularly the importance of cold-start stability and error correction for continually evolving robotic agents.
First, the memory manager does not require a fully trained PPO policy to maintain reliable physical evidence during cold start. As described in Section 3.3 and Appendix C, raw execution records (M1) and evidence-based hypothesis updates (M2) are maintained independently of the memory manager's decisions. The manager operates on memory retrieval and abstraction, with Skip always available when no update is selected. Moreover, PPO training occurs during evolution, and the trained manager is frozen during evaluation.
Second, incorrect memories are not permanently committed. Every M3 abstraction preserves links to its underlying evidence. New physical outcomes continuously update hypothesis confidence, while Merge incorporates new evidence and Forget removes outdated or contradicted memories from active retrieval without deleting their original records. The PPO reward further evaluates memory decisions through subsequent task success, uncertainty reduction, and experiment cost, providing feedback for correcting ineffective memory-management strategies.
To quantitatively evaluate cold-start reliability and error-correction capability, we further analyze memory-decision trajectories recorded during evolution, measuring early-stage contradictions, subsequent memory corrections, and correction latency.
Table R2. Cold-Start Memory Stability and Error Correction
| Metric | Result |
|---|---|
| Cold-Start Contradiction Rate ↓ | 15.28% |
| Memory Correction Rate ↑ | 84.90% |
| Mean Correction Latency ↓ | 3.71 memory decisions |
These results indicate that 84.90% of identified contradictory memories are successfully corrected, requiring an average of only 3.71 memory decisions after contradiction detection. This provides additional evidence that early memory errors can be identified and corrected through subsequent physical feedback rather than permanently propagating through the memory hierarchy.
Finally, Table 5 provides empirical support for reward-grounded memory management. Our full method achieves 71.3% Composite-Unseen success and 0.635 Learning AUC, outperforming PPO with task reward only (65.2%, 0.589) and retrieval-based selection (61.7%, 0.556). These results demonstrate the benefit of learning memory decisions from downstream outcomes rather than relying solely on retrieval similarity or immediate task rewards.
W4. Real-World Generalization with a Frozen VLA
Thank you for highlighting the importance of distinguishing backbone adaptation from harness-level generalization. Your suggestion addresses an important question for agentic robotics: how to improve the capabilities of robot foundation models without requiring repeated parameter updates.
First, we clarify that "frozen VLA" refers to the harness adaptation and inference stages, rather than the absence of any prior fine-tuning. In our real-world experiments, SmolVLA is first adapted to the physical SO-101 setup and target manipulation tasks using 50 demonstrations per task. This establishes the low-level manipulation capabilities required for deployment.
Second, the VLA weights remain completely frozen throughout harness evolution and real-world evaluation. Both SmolVLA and EmbodiedEvo use exactly the same fine-tuned backbone. The only difference is the agentic harness, whose code and skills are evolved entirely in simulation and transferred directly to the physical robot.
Importantly, EmbodiedEvo improves real-world success from 46.0% to 71.3% (+25.3 percentage points) without any additional VLA parameter updates. This controlled comparison directly isolates the generalization gains contributed by the harness rather than additional backbone training.
Thank you again for your valuable insights and constructive suggestions. Your feedback has helped us strengthen our experimental evidence, clarify our underlying mechanisms, and better articulate the broader significance of our work. We believe these additions further demonstrate the potential of evidence-driven, self-evolving agentic harnesses for advancing autonomous robotic learning.
Response to reviewer eHSg (3/3)
Dear reviewer eHSg,
We are very grateful for your valuable comments, and we spare no effort to address your concerns. Please let us know if there are any additional concerns. We also hope our responses above address your concerns and merit a better confidence score.
Best, Authors
Response to reviewer eHSg (1/3)
Thank you for your exceptionally insightful review! We particularly appreciate your deep understanding of the fundamental challenges facing agentic robotics. Your suggestions highlight four important research directions: evidence-grounded hypothesis generation, principled experiment selection, reliable long-term memory, and generalization beyond the environments where agents learn.
We believe these questions are essential for advancing agentic robotics toward more autonomous systems. Your insights not only strengthen our work but also offer valuable directions for future research in this field. Inspired by your suggestions, we have conducted two additional experiments to further validate our central contributions.
We sincerely appreciate the opportunity to address your questions and present additional evidence. Our detailed responses follow.
W1. Hypothesis and Experiment Generation
Thank you for this insightful question regarding hypothesis generation and LLM dependency.
First, hypothesis generation is grounded in physical execution feedback rather than unconstrained LLM generation. As described in Section 3.2 and Algorithm 1, the Fast System uses execution trajectories, task outcomes, and the existing Hypothesis Graph to propose new code-skill hypotheses or refine existing ones. Candidate experiments are then constructed from these hypotheses under interaction-budget and evaluation constraints. Each experiment explicitly identifies the hypotheses it tests.
Second, our core mechanism does not rely on the LLM to judge hypothesis correctness. Hypotheses are evaluated through matched physical trials. Their confidence is updated using Bayesian posterior estimates, and subsequent experiments are selected through cost-normalized Value of Information. The LLM influences which hypotheses are proposed, while physical evidence determines which hypotheses are supported and which experiments are worth pursuing.
We will expand the implementation details of hypothesis generation, candidate construction, and the associated prompts in the revised manuscript.
W2. Stronger Experiment-Selection Baselines
Thank you for your insightful suggestion to evaluate stronger adaptive exploration strategies. As you suggested, we conducted two additional experiments: Information Gain Only, which maximizes expected information gain without cost normalization, and Thompson Sampling, a standard Bayesian adaptive exploration strategy. All methods use the same interaction budget.
Table R1. Experiment Selection Ablation
Full VOI outperforms Information Gain Only by 11.1 percentage points in Success@50, while improving Learning AUC from 0.556 to 0.635. It also outperforms Thompson Sampling by 47.2 percentage points, achieving 1.96× higher Learning AUC.
These results directly isolate the benefit of cost-normalized information gain and demonstrate that our VOI formulation substantially outperforms both uncertainty-reduction-based selection and Bayesian posterior sampling under the same interaction budget.