acceptodds
Under review as a conference paper at ICLR 2027

EmbodiSkill: Skill-Aware Reflection for Self-Evolving Embodied Agents

Abstract

Reusable skills are essential for embodied agents because they provide procedural guidance for acting under changing environments. However, embodied skills often express high-level guidance that must be translated into context-dependent actions, creating a gap between skill guidance and action realization. A failed trajectory may therefore arise from either a skill defect or an execution lapse in which valid guidance is present but overlooked. Existing skill self-evolution methods often do not distinguish these causes and may unnecessarily revise guidance that is already correct. We introduce EmbodiSkill, a training-free framework that uses skill-aware reflection to separate skill-changing evidence from execution lapses. It revises the skill body when guidance needs to be added, refined, or corrected, while updating an appendix to emphasize valid but overlooked guidance. Experiments on ALFWorld and EmbodiedBench show consistent improvements over direct-execution, memory-based, and skill-based baselines. On ALFWorld, under a fully local and model-matched setting, EmbodiSkill improves a frozen Qwen3.5-27B executor from 61.19% to 86.57% task success. Using GPT-5.2 only for offline skill evolution further increases task success to 93.28%.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied AI, LLM Agents, Agent Skills, Self-Evolving Agents, Reflection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_H65FOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the phenomenon that failure in embodied tasks may stem from inherent skill deficiencies or the execution lapse to follow existing effective guidance. It proposes EmbodiSkill, a method that requires no training and no updating of model parameters. Its skill-aware reflection categorizes trajectory evidence into Discovery, Optimization, Skill Defect, and Execution Lapse. The first three categories are used to update the skill text; the Execution Lapse is used to update the independent appendix, emphasizing rules already present but ignored in the text.

Experimental results on ALFWorld, EB-Habitat, and EB-Navigation demonstrate that EmbodiSkill effectively enhances the embodiment capabilities of general multimodal models. The authors also provide analysis of reflection and appendix ablation, evolutionary processes, hyperparameter sensitivity analysis, and cue templates.

Critical strengths
  1. The problem is practically relevant, and the method design corresponds to the problem. It distinguishes between knowledge that needs modification and re-emphasis, making it suitable for long-term skill library maintenance.
  2. Fully local configuration also offers significant gains. Multiple models on ALFWorld and EB-Habitat show substantial improvements, indicating that the method does not entirely rely on closed-source evolvers.
Critical weaknesses
  1. The core novelty has not yet been fully established. While the AutoManual already addresses failure attribution and rule emphasis[1], this paper needs to more directly demonstrate the unique advantages of independently maintaining the appendix.
  2. Ablation and validation remain insufficient. Existing experiments cannot independently demonstrate the role of failure attribution and the independent appendix, and validation of attribution accuracy, repeatability, and statistical stability is lacking.

References:

[1]Automanual: Constructing instruction manuals by llm agents via interactive environmental learning. In NeurIPS 2024.

Questions for authors
  1. AutoManual already includes failure attribution and rule emphasis. What has been added in this article? Can we compare in-rule emphasis and independent appendixes under the same executor and budget?
  2. "w/o Skill-Aware Reflection" removes both reflection and failure attribution. Can we retain reflection and only remove failure categories to verify the effectiveness of attribution itself?
2 comments

We sincerely thank the reviewer for their constructive feedback. We appreciate the reviewer's positive assessment of the practical relevance of our problem, the suitability of our approach for long-term skill maintenance, and the substantial performance improvements demonstrated in our experiments. The reviewer's important concerns regarding the independent Appendix and Failure Attribution have helped us strengthen the paper. We have incorporated additional experiments and analyses into the revised manuscript, with all changes highlighted in blue. We address these concerns in detail below.

Q1. Compared to AutoManual, what are the additional contributions of EmbodiSkill? (part 1)

Highlight. Compared with AutoManual, EmbodiSkill offers two key advantages: (1) AutoManual's per-trajectory updates risk overfitting to environment-specific experiences, whereas EmbodiSkill consolidates multiple trajectories to derive more generalizable skill guidance, improving success from 61.94% to 79.85%. (2) AutoManual's inline bolding or annotations may not sufficiently reinforce critical guidance due to LLMs' positional biases and the potential benefits of prompt repetition. EmbodiSkill instead selectively repeats overlooked guidance in an independent Appendix at the end of the prompt, outperforming inline bolding and inline reminders by 6.71 and 8.95 percentage points, respectively. Overall, EmbodiSkill consistently outperforms AutoManual across three embodied benchmarks.

EmbodiSkill enables continuous self-evolution through multi-trajectory experience. In embodied tasks, the same goal may require different action sequences across environments, making updates based on individual trajectories prone to overfitting to the specific environment in which they were collected. Unlike AutoManual[1], which updates rules after each individual trajectory, EmbodiSkill employs Skill-Aware Reflection and consolidates experience across multiple trajectories before updating skills. The updated skills are then reused in subsequent interactions, forming a continuous cycle of reflection, consolidation, and skill evolution.

Our newly added Table 7 provides supporting evidence. Under the same 500-trajectory evolution budget (K=1), batch updates with B=50 and B=100 achieve 79.85% and 73.88% success, respectively, outperforming per-trajectory updates (B=1, 61.94%) by 17.91 and 11.94 percentage points.

Table 7 in the paper. Sensitivity to the number of trajectories per update B and reflection limit K. The total evolution budget is fixed at 500 trajectories, with N = 500/B evolution stages.

KB=1 (N=500)B=50 (N=10)B=100 (N=5)
161.9479.8573.88
253.7376.1276.87
467.1678.3671.64

The independent Appendix is a targeted reinforcement mechanism designed for local embodied executors. Given the latency requirements of embodied interaction, frozen local models are commonly employed as executors. Prior studies have shown that LLMs exhibit positional biases, often underutilizing information in the middle of a prompt [2], while prompt repetition can improve model performance in certain settings [3]. Consequently, AutoManual's in-place bolding or annotations may still leave critical guidance vulnerable to positional biases or insufficient repetition. Instead, EmbodiSkill introduces an independent Appendix following the skill body, where overlooked guidance identified by Skill-Aware Reflection is selectively reiterated. The Appendix therefore enhances its salience and reinforces its influence during execution, mitigating the mentioned limitation.

Our newly added Table 4 directly validates this design. With the same Skill, emphasized content, and executor, our Appendix achieves 93.28% success, outperforming Inline Bolding and Inline Reminder by 6.71 and 8.95 percentage points, respectively. It also surpasses Random Reminders (79.85%), Full-Body Repetition (82.84%), Top-k Body Retrieval (85.07%), and Top-k Appendix Retrieval (83.58%), demonstrating the effectiveness of our proposed design.

Table 4 in the paper. Comparison of skill reinforcement strategies on ALFWorld.

Reinforcement StrategySuccess Rate (%)
Without Appendix75.37
Inline Bolding86.57
Inline Reminder84.33
Random Reminder79.85
Full-Body Repetition82.84
Top-k Body Retrieval85.07
Top-k Appendix Retrieval83.58
Skill-Aware Appendix (Ours)93.28

Q1. Compared to AutoManual, what are the additional contributions of EmbodiSkill? (part 2)

New direct comparison experiments further demonstrate the overall advantages of EmbodiSkill over AutoManual. As shown in Tables 1 and 2, under the same executors and evolution budgets, EmbodiSkill consistently outperforms AutoManual across ALFWorld, EB-Habitat, and EB-Navigation, with improvements of up to 16.42, 11.00, and 10.00 percentage points, respectively.

Table 1&2 in the paper. Direct comparison between AutoManual and EmbodiSkill across embodied benchmarks. All values are task success rates (%).

BenchmarkExecutorAutoManualEmbodiSkill (Ours)
ALFWorldQwen2.5-14B63.4379.85
ALFWorldQwen3.5-27B73.1386.57
EB-HabitatQwen3-VL-8B31.6742.67
EB-HabitatQwen3-VL-32B48.0051.33
EB-NavigationQwen3-VL-8B39.0049.00
EB-NavigationQwen3-VL-32B49.6756.00

Q2. How Can We Independently Validate the Effectiveness of Failure Attribution?

Highlight. We provide two key findings regarding Failure Attribution: (1) Effectiveness: Removing Failure Attribution while retaining Reflection reduces success from 85.82% to 54.48%, even below removing Reflection entirely (67.91%), demonstrating that incorrect attribution can be more harmful than omitting Reflection. (2) Reliability: Human evaluation achieves 96% agreement, supporting the reliability of distinguishing Skill Defects from Execution Lapses.

We introduce a new w/o Failure Attribution ablation to independently evaluate the contribution of Failure Attribution while retaining Reflection. This variant preserves Discovery and Optimization but treats all failed-trajectory reflections as Skill Defects, using them to revise the Skill Body without updating the Appendix. As shown in Table 3, with the same Gemini-3-flash evolution model and Qwen2.5-14B executor, removing Failure Attribution reduces success from 85.82% to 54.48% (−31.34 percentage points), even below w/o Skill-Aware Reflection (67.91%). This suggests that incorrect attribution can be more harmful than omitting Reflection, as execution lapses may trigger unnecessary revisions to correct guidance. The GPT results show a similar numerical trend (76.12% vs. 86.57%), although the evolution models differ.

Table 3 in the paper. Ablation study of Failure Attribution on ALFWorld.

ConfigurationGPTGemini
EmbodiSkill86.5785.82
-w/o Skill-Aware Reflection70.9067.91
-w/o Failure Attribution76.1254.48
-w/o Appendix76.8778.36

Note: As GPT-5.2 is no longer accessible, the new w/o Failure Attribution experiment uses GPT-5.6-sol, while the other experiments in the GPT column use GPT-5.2.

We further conduct an independent human evaluation to assess the reliability of Failure Attribution. We randomly sampled 50 failed-trajectory reflections and independently annotated them using complete trajectories and existing Skills, assessing whether following the original guidance could have prevented failure. As shown in Table 5, the model agrees with human annotations on 48/50 cases (96%). These results demonstrate the reliability of the model's failure attribution.

Table 5 in the paper. Comparison between human annotations and model predictions for Failure Attribution.

Human LabelPred. DefectPred. LapseTotal
Skill Defect14014
Execution Lapse23436
Total163450

We sincerely thank the reviewer again for their valuable feedback. We hope that the additional experiments and clarifications adequately address the reviewer's concerns and further strengthen the paper.

References

[1] Chen et al. (2024). AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning. NeurIPS 2024. https://arxiv.org/abs/2405.16247

[2] Liu et al. (2024). Lost in the Middle: How Language Models Use Long Contexts. TACL, 12:157–173. https://aclanthology.org/2024.tacl-1.9/

[3] Leviathan et al. (2025). Prompt Repetition Improves Non-Reasoning LLMs. arXiv:2512.14982. https://arxiv.org/abs/2512.14982

Reviewer_YXHEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EmbodiSkill, a training-free framework for evolving procedural skills in embodied agents. The paper distinguishes failures caused by incorrect or insufficient skill guidance from execution lapses, where the agent fails to follow already-valid guidance. To this end, the method uses skill-aware reflection to analyze execution trajectories and separates skills into a body and an appendix. The body is revised when guidance needs improvement, while the appendix emphasizes existing guidance that was overlooked during execution. Experiments on ALFWorld and EmbodiedBench show improvements over memory-based and skill-based baselines using frozen LLM executors.

Critical strengths

EmbodiSkill improves task success over model-matched memory-based and skill-based baselines on diverse benchmarks including ALFWorld, EB-Habitat, and EB-Navigation, implying that evolving external procedural guidance can improve frozen executors without additional model training.

Critical weaknesses

The two main contributions are distinguishing skill revision from execution lapses and separating skills into a body and an appendix (Sections 3.2–3.3). However, both components are implemented primarily through LLM prompt designs (Appendix G), without introducing a substantially new technical mechanism. While these design choices may be useful in practice, it is unclear what methodological novelty they offer beyond prompt-based failure analysis and skill refinement.

Questions for authors
  • Both skill-aware reflection and the body–appendix separation appear to be implemented through LLM prompting. What is the main technical novelty beyond prompt-based failure analysis and skill refinement?
  • How reliably can the reflection model distinguish Skill Defect from Execution Lapse? For example, when an instruction is technically correct but too vague for the executor to follow, how does the method determine whether the failure should trigger skill revision or execution emphasis?
  • Regarding the concern above how does the reflection model determine exactly which existing skill instruction was overlooked during execution? Since the agent's internal reasoning is not directly observable, how can the method verify that a particular instruction was overlooked rather than incorrectly understood or applied?
  • Could the authors compare the proposed appendix against potential baselines such as randomly selected reminders, simply repeating existing skill instructions, retrieving top-k relevant appendices, etc.?
  • Comparisons with baselines in Tables 1, 2, and 4 show noticeable performance differences when Qwen, GPT, or Gemini is used for skill evolution, and no single model consistently performs best across settings. How sensitive is EmbodiSkill to the choice of evolution model, and to what extent do the improvements depend on the model's reflection capabilities rather than the proposed framework itself?
7 comments

We sincerely thank the reviewer for their constructive feedback. We appreciate the reviewer's positive assessment of EmbodiSkill's improvements over model-matched baselines across diverse embodied benchmarks without additional training. The reviewer's important questions regarding technical novelty and reliability have helped us strengthen the paper. We have incorporated additional experiments and analyses into the revised manuscript, with all changes highlighted in blue. We address these questions in detail below.

Q1. What Is the Main Technical Novelty Beyond Prompt-Based Failure Analysis and Skill Refinement? (part 1)

Highlight. Building on the widely adopted, training-free Agent Skills paradigm, EmbodiSkill goes beyond prompt engineering by introducing three key technical innovations for reliable embodied skill self-evolution: (1) Skill-Aware Reflection distinguishes skill defects from execution lapses to prevent harmful revisions of correct skills, improving success from 54.48% to 85.82%; (2) Multi-trajectory Consolidation aggregates experience across trajectories to mitigate overfitting to individual environments, improving success from 61.94% to 79.85%; and (3) Skill-Aware Appendix mitigates LLMs' positional biases and leverages the benefits of prompt repetition by selectively reiterating overlooked guidance in an independent Appendix following the Skill Body, improving success from 86.57% to 93.28%. Overall, EmbodiSkill consistently achieves the highest success rates across all evaluated embodied benchmarks, outperforming all compared baselines.

Agent Skills have emerged as a widely adopted, training-free approach to enhancing agent capabilities[3]. Although expressed through prompts, skill evolution goes beyond prompt engineering by accumulating and refining reusable knowledge from interaction experience. Unlike digital agents, embodied agents must ground skills in dynamic physical environments, where failures may stem from defective skills or overlooked guidance, making blind skill revisions potentially harmful. EmbodiSkill addresses these challenges by distinguishing when to revise guidance, when to reinforce it, and how to consolidate experience across interactions. Its main contributions lie in three complementary mechanisms:

(1) Skill-Aware Reflection prevents unnecessary skill revisions caused by execution lapses. In embodied tasks, executors may overlook correct guidance when acting on changing environmental observations. Treating such failures as skill defects can lead to unnecessary revisions. Table 3 validates the importance of this distinction. With Gemini-3-flash, w/o Failure Attribution retains Reflection but removes failure categorization, reducing success from 85.82% to 54.48%. In contrast, w/o Skill-Aware Reflection removes the entire reflection mechanism and achieves 67.91%. Notably, the former performs even worse than the latter, suggesting that incorrect failure attribution can lead to unnecessary skill revisions and hinder skill evolution.

Table 3 in the paper. Ablation study of Failure Attribution on ALFWorld.

ConfigurationGPTGemini
EmbodiSkill86.5785.82
-w/o Skill-Aware Reflection70.9067.91
-w/o Failure Attribution76.1254.48
-w/o Appendix76.8778.36

Note: As GPT-5.2 is no longer accessible, the new w/o Failure Attribution experiment uses GPT-5.6-sol, while the other experiments in the GPT column use GPT-5.2.

(2) Multi-trajectory consolidation enables continuous skill self-evolution. The same embodied task may require different action sequences across environments, making updates based on individual trajectories prone to overfitting to the specific environments in which they were collected. EmbodiSkill therefore consolidates reflections across multiple trajectories before updating skills, reducing redundancy and resolving conflicts. The updated skills are reused in subsequent interactions, forming a continuous evolution cycle. As shown in Table 7, under the same 500-trajectory budget, batch updates (B=50, K=1) achieve 79.85% success, outperforming per-trajectory updates (B=1, K=1) at 61.94%.

Table 7 in the paper. Sensitivity to the number of trajectories per update B and reflection limit K. The total evolution budget is fixed at 500 trajectories, with N = 500/B evolution stages.

KB=1 (N=500)B=50 (N=10)B=100 (N=5)
161.9479.8573.88
253.7376.1276.87
467.1678.3671.64

Q1. What Is the Main Technical Novelty Beyond Prompt-Based Failure Analysis and Skill Refinement? (part 2)

(3) The independent Appendix is a targeted reinforcement mechanism designed for local embodied executors. In embodied tasks, frozen local executors may fail to apply correct guidance at critical decision points. Prior studies have shown that LLMs exhibit positional biases, often overlooking information in the middle of a prompt[1], while prompt repetition can improve performance in certain settings[2]. Motivated by these findings, EmbodiSkill selectively reiterates overlooked guidance identified by Skill-Aware Reflection in an independent Appendix at the end of the prompt, combining favorable prompt positioning with targeted repetition to reinforce critical guidance during execution. As shown in Table 4, with the same Skill, emphasized content, and executor, our Appendix achieves 93.28% success, outperforming Inline Bolding (86.57%), Inline Reminder (84.33%), Random Reminder (79.85%), Full-Body Repetition (82.84%), Top-k Body Retrieval (85.07%), and Top-k Appendix Retrieval (83.58%). These results demonstrate the effectiveness of our targeted reinforcement mechanism.

Table 4 in the paper. Comparison of skill reinforcement strategies on ALFWorld.

Reinforcement StrategySuccess Rate (%)
Without Appendix75.37
Inline Bolding86.57
Inline Reminder84.33
Random Reminder79.85
Full-Body Repetition82.84
Top-k Body Retrieval85.07
Top-k Appendix Retrieval83.58
Skill-Aware Appendix (Ours)93.28

Q2. How Reliably Can the Model Distinguish Skill Defect from Execution Lapse, Especially When Existing Guidance Is Correct but Too Vague to Follow?

Highlight. Human evaluation confirms the reliability of Failure Attribution, achieving 96% agreement. Importantly, even when existing guidance is technically correct but too vague to execute reliably, Skill-Aware Reflection can refine it into more explicit and actionable instructions, enabling successful execution.

Human evaluation supports the reliability of Failure Attribution. We randomly sampled 50 failed-trajectory reflections and independently annotated them using complete trajectories and existing Skills, assessing whether following the original guidance could have prevented failure. As shown in Table 5, the model agrees with human annotations on 48/50 cases (96%). These results demonstrate the reliability of the model's failure attribution.

Table 5 in the paper. Comparison between human annotations and model predictions for Failure Attribution.

Human LabelPred. DefectPred. LapseTotal
Skill Defect14014
Execution Lapse23436
Total163450

Further analysis of the misclassified cases reveals an illustrative example. In the task "Heat an apple and place it on the dining table," the original Skill states:

For tasks involving object transformation, complete the necessary processing steps before delivery. Ensure the required conditions are met, address any issues that arise, and proceed to the destination once the object is ready.

The executor failed by approaching the microwave without first picking up the apple. While the human annotator considered this an Execution Lapse, the model classified it as a Skill Defect and revised the Skill as follows:

For tasks requiring cleaning, heating, or cooling, locate and pick up the object, process it at the appropriate fixture, confirm success, and deliver it to the destination. If processing fails, check the prerequisites and retry. Once successful, proceed directly to delivery without further exploration.

The revised Skill enabled successful execution. This case shows that when existing guidance is technically correct but too vague or difficult for the executor to follow, Skill-Aware Reflection may classify the failure as a Skill Defect and revise the Skill to make it more explicit and actionable.

Q3. How Does the Method Identify Which Specific Skill Instruction Was Overlooked Rather Than Misunderstood or Misapplied?

We use Skill-Aware Reflection to attribute failures by analyzing the consistency between execution behavior and existing skill guidance. Specifically, Skill-Aware Reflection examines the complete interaction trajectory and current Skill, generates up to K reflection records identifying relevant skill guidance and corresponding update suggestions, and determines whether the failure stems from incorrect or insufficient instructions, or from the executor overlooking guidance that is already correct and sufficiently clear to follow. Although the executor's internal reasoning is not directly observable, our attribution is grounded in observable execution behavior and its consistency with existing skill guidance.

Q4. How Does the Appendix Compare with Random Reminders, Simple Repetition, and Top-k Retrieval?

We introduce controlled comparisons of different reinforcement strategies (Table 4) to directly validate the effectiveness of Skill-Aware Appendix. All strategies use the same set of Skills and the same frozen executor. In particular, Inline Bolding, Inline Reminder, and Skill-Aware Appendix reinforce the same guidance, differing only in how and where it is emphasized. As shown in Table 4, our Appendix consistently outperforms all other reinforcement strategies, demonstrating the effectiveness of our proposed design.

Q5. How Sensitive Is EmbodiSkill to the Choice of Evolution Model, and How Much Do the Improvements Depend on the Model's Reflection Capabilities?

The choice of evolution model affects performance but does not determine the effectiveness of EmbodiSkill. With Qwen3.5-27B as the executor on ALFWorld, evolution using Qwen, GPT-5.2, and Gemini-3-flash achieves 86.57%, 93.28%, and 87.31% success, respectively. While stronger Reflection capabilities may improve failure attribution and skill updates, the substantial gains achieved with local models demonstrate that EmbodiSkill's effectiveness is not dependent on a particular closed-source evolution model.

More importantly, the evolution strategy appears to have a greater impact on performance than the choice of evolution model. As shown in Table 3, under the same Gemini-3-flash evolution model, removing Failure Attribution reduces success from 85.82% to 54.48%. Table 7 further demonstrates the impact of reflection limits (K) and update batch sizes (B): under the same 500-trajectory budget, increasing B from 1 to 50 with K=1 improves success from 61.94% to 79.85%. Moreover, Table 4 shows that our Skill-Aware Appendix achieves 93.28%, outperforming other reinforcement strategies. These results suggest that EmbodiSkill's effectiveness depends not only on the evolution model's capabilities but also on how reflections are generated, consolidated, and translated into targeted skill updates.

We sincerely thank the reviewer again for their valuable feedback. We hope that the additional experiments and clarifications adequately address the reviewer's concerns and further strengthen the paper.

References

[1] Liu et al. (2024). Lost in the Middle: How Language Models Use Long Contexts. TACL, 12:157–173. https://aclanthology.org/2024.tacl-1.9/

[2] Leviathan et al. (2025). Prompt Repetition Improves Non-Reasoning LLMs. arXiv:2512.14982. https://arxiv.org/abs/2512.14982

[3] Zhang, B., Lazuka, K., and Murag, M. (2025). Equipping Agents for the Real World with Agent Skills. Anthropic Engineering Blog, October 16, 2025. https://www.anthropic.com/research/equipping-agents-for-the-real-world-with-agent-skills

Reviewer_uWMaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

this paper shows a training-free framework for improving a frozen embodied agent through an evolving textual skill manual. It distinguishes trajectories that warrant adding or revising procedural knowledge from execution lapses in which existing valid guidance was not followed. The former update the skill body; the latter update a separate appendix intended to emphasize existing guidance without introducing new procedures. Reflections are consolidated over batches of trajectories before the manual is revised. Evaluation covers ALFWorld, EB-Habitat, and EB-Navigation, including matched local-model settings and stronger offline evolution models. On ALFWorld, the local Qwen3.5-27B setting reaches 86.57% success versus 81.34% for the strongest reported matched baseline, while GPT-5.2-based evolution reaches 93.28%. The proposed contribution is an attribution-guided memory-update policy that separates knowledge revision from execution emphasis, rather than parameter learning or acquisition of new low-level control policies.

Critical strengths
  1. An unsuccessful execution does not necessarily imply that the underlying guidance is wrong. Explicitly separating knowledge revision from reminders addresses a real failure mode of self-improving agents: incidental execution errors can otherwise become redundant or conflicting procedural rules. The resulting update records are also interpretable.

  2. the paper includes local, model-matched evolution and evaluation across two executor scales and multiple embodied benchmarks. The ALFWorld improvement in the fully local 27B setting indicates that the reported benefit is not solely attributable to using a more capable closed-source evolution model. This supports the practical usefulness of the system, subject to the validation concerns below.

Critical weaknesses

the paper cited AutoManual but not directly compared, AutoManual already distinguishes imperfect rules from an imperfect agent. Its Appendix H.3, Listings 16 and 18, also directs the system to emphasize overlooked rules or add reminders while retaining existing details. Thus, the key incremental question is whether a separately maintained appendix improves on rule-local emphasis. The current ablations do not answer this: removing skill-aware reflection also removes a reasoning stage, while removing the appendix at test time removes useful text.

Questions for authors
  1. Beyond AutoManual’s imperfect-rules/imperfect-agent distinction and rule emphasis, what does the separate appendix contribute?

  2. Can you independently assess a sample of defect/lapse decisions, including whether following the original guidance would actually avoid the failure? Please report uncertainty or disagreement, not only successful examples. How do decisions change when reflection sees the full trace rather than the version with invalid actions and numeric identifiers removed? How often does an unmatched lapse target fall back to a defect, and why is this treated as a semantic label rather than an unresolved case?

5 comments

We sincerely thank the reviewer for their constructive feedback. We appreciate the reviewer's positive assessment of the practical value and interpretability of our approach, as well as the effectiveness of our experiments. The reviewer's important concerns regarding AutoManual and Failure Attribution have helped us strengthen the paper. We have incorporated additional experiments and analyses into the revised manuscript, with all changes highlighted in blue. We address these concerns in detail below.

Q1. What Does the Independent Appendix Contribute Compared with AutoManual? (part 1)

Highlight. Compared with AutoManual, EmbodiSkill offers two key advantages: (1) AutoManual's per-trajectory updates risk overfitting to environment-specific experiences, whereas EmbodiSkill consolidates multiple trajectories to derive more generalizable skill guidance, improving success from 61.94% to 79.85%. (2) AutoManual's inline bolding or annotations may not sufficiently reinforce critical guidance due to LLMs' positional biases and the potential benefits of prompt repetition. EmbodiSkill instead selectively repeats overlooked guidance in an independent Appendix at the end of the prompt, outperforming inline bolding and inline reminders by 6.71 and 8.95 percentage points, respectively. Overall, EmbodiSkill consistently outperforms AutoManual across three embodied benchmarks.

EmbodiSkill enables continuous self-evolution through multi-trajectory experience. In embodied tasks, the same goal may require different action sequences across environments, making updates based on individual trajectories prone to overfitting to the specific environment in which they were collected. Unlike AutoManual[1], which updates rules after each individual trajectory, EmbodiSkill employs Skill-Aware Reflection and consolidates experience across multiple trajectories before updating skills. The updated skills are then reused in subsequent interactions, forming a continuous cycle of reflection, consolidation, and skill evolution.

Our newly added Table 7 provides supporting evidence. Under the same 500-trajectory evolution budget (K=1), batch updates with B=50 and B=100 achieve 79.85% and 73.88% success, respectively, outperforming per-trajectory updates (B=1, 61.94%) by 17.91 and 11.94 percentage points.

Table 7 in the paper. Sensitivity to the number of trajectories per update B and reflection limit K. The total evolution budget is fixed at 500 trajectories, with N = 500/B evolution stages.

KB=1 (N=500)B=50 (N=10)B=100 (N=5)
161.9479.8573.88
253.7376.1276.87
467.1678.3671.64

The independent Appendix is a targeted reinforcement mechanism designed for local embodied executors. Given the latency requirements of embodied interaction, frozen local models are commonly employed as executors. Prior studies have shown that LLMs exhibit positional biases, often underutilizing information in the middle of a prompt [2], while prompt repetition can improve model performance in certain settings [3]. Consequently, AutoManual's in-place bolding or annotations may still leave critical guidance vulnerable to positional biases or insufficient repetition. Instead, EmbodiSkill introduces an independent Appendix following the skill body, where overlooked guidance identified by Skill-Aware Reflection is selectively reiterated. The Appendix therefore enhances its salience and reinforces its influence during execution, mitigating the mentioned limitation.

Our newly added Table 4 directly validates this design. With the same Skill, emphasized content, and executor, our Appendix achieves 93.28% success, outperforming Inline Bolding and Inline Reminder by 6.71 and 8.95 percentage points, respectively. It also surpasses Random Reminders (79.85%), Full-Body Repetition (82.84%), Top-k Body Retrieval (85.07%), and Top-k Appendix Retrieval (83.58%), demonstrating the effectiveness of our proposed design.

Table 4 in the paper. Comparison of skill reinforcement strategies on ALFWorld.

Reinforcement StrategySuccess Rate (%)
Without Appendix75.37
Inline Bolding86.57
Inline Reminder84.33
Random Reminder79.85
Full-Body Repetition82.84
Top-k Body Retrieval85.07
Top-k Appendix Retrieval83.58
Skill-Aware Appendix (Ours)93.28

Q1. What Does the Independent Appendix Contribute Compared with AutoManual? (part 2)

New direct comparison experiments further demonstrate the overall advantages of EmbodiSkill over AutoManual. As shown in Tables 1 and 2, under the same executors and evolution budgets, EmbodiSkill consistently outperforms AutoManual across ALFWorld, EB-Habitat, and EB-Navigation, with improvements of up to 16.42, 11.00, and 10.00 percentage points, respectively.

Table 1&2 in the paper. Direct comparison between AutoManual and EmbodiSkill across embodied benchmarks. All values are task success rates (%).

BenchmarkExecutorAutoManualEmbodiSkill (Ours)
ALFWorldQwen2.5-14B63.4379.85
ALFWorldQwen3.5-27B73.1386.57
EB-HabitatQwen3-VL-8B31.6742.67
EB-HabitatQwen3-VL-32B48.0051.33
EB-NavigationQwen3-VL-8B39.0049.00
EB-NavigationQwen3-VL-32B49.6756.00

Q2. Reliability and Implementation of Failure Attribution (part 1)

Highlight. We provide four key findings regarding Failure Attribution: (1) Reliability: Human evaluation achieves 96% agreement, supporting attribution reliability. (2) Effectiveness: Removing Failure Attribution reduces success from 85.82% to 54.48%, even below removing Reflection entirely (67.91%), revealing the harm of incorrect attribution. (3) Existing Skill Validity: Reinforcing existing guidance through the Appendix improves success from 76.87% to 86.57% without modifying the Skill Body, demonstrating that existing Skills can provide correct guidance when properly followed. (4) Implementation: Reflection uses complete trajectories, and unmatched Execution Lapses are discarded to avoid unjustified skill revisions.

Human evaluation supports the reliability of Failure Attribution. We randomly sampled 50 failed-trajectory reflections and independently annotated them using complete trajectories and existing Skills, assessing whether following the original guidance could have prevented failure. As shown in Table 5, the model agrees with human annotations on 48/50 cases (96%). These results demonstrate the reliability of the model's failure attribution.

Table 5 in the paper. Comparison between human annotations and model predictions for Failure Attribution.

Human LabelPred. DefectPred. LapseTotal
Skill Defect14014
Execution Lapse23436
Total163450

Further analysis of the misclassified cases reveals an illustrative example. In the task "Heat an apple and place it on the dining table," the original Skill states:

For tasks involving object transformation, complete the necessary processing steps before delivery. Ensure the required conditions are met, address any issues that arise, and proceed to the destination once the object is ready.

The executor failed by approaching the microwave without first picking up the apple. While the human annotator considered this an Execution Lapse, the model classified it as a Skill Defect and revised the Skill as follows:

For tasks requiring cleaning, heating, or cooling, locate and pick up the object, process it at the appropriate fixture, confirm success, and deliver it to the destination. If processing fails, check the prerequisites and retry. Once successful, proceed directly to delivery without further exploration.

The revised Skill enabled successful execution. This case shows that, despite a minor attribution disagreement, the model refines the skill by adding explicit operational details, making the guidance easier for the executor to understand and follow.

Q2. Reliability and Implementation of Failure Attribution (part 2)

We further provide execution evidence that reinforcing existing guidance can prevent failures. In Table 3, adding the Appendix without changing the Skill Body improves success rates from 76.87% to 86.57% and 78.36% to 85.82%. Figure 8 further shows that, with the same Skill Body, the executor repeatedly searches visited locations and fails without the Appendix, but succeeds with a targeted reminder. These results support that some failures arise from overlooked guidance rather than missing skill knowledge.

Table 3 in the paper. Ablation study of Failure Attribution on ALFWorld.

ConfigurationGPTGemini
EmbodiSkill86.5785.82
-w/o Skill-Aware Reflection70.9067.91
-w/o Failure Attribution76.1254.48
-w/o Appendix76.8778.36

Note: As GPT-5.2 is no longer accessible, the new w/o Failure Attribution experiment uses GPT-5.6-sol, while the other experiments in the GPT column use GPT-5.2.

We introduce a new w/o Failure Attribution ablation to independently evaluate the contribution of Failure Attribution while retaining Reflection. This variant preserves Discovery and Optimization but treats all failed-trajectory reflections as Skill Defects, using them to revise the Skill Body without updating the Appendix. As shown in Table 3, with the same Gemini-3-flash evolution model and Qwen2.5-14B executor, removing Failure Attribution reduces success from 85.82% to 54.48% (−31.34 percentage points), even below w/o Skill-Aware Reflection (67.91%). This demonstrates that the benefit stems not merely from an additional reasoning stage, but from correctly attributing failures to avoid unnecessary revisions of valid skill guidance. Notably, incorrect attribution can be more harmful than omitting Reflection altogether. The GPT results show a similar numerical trend (76.12% vs. 86.57%), although the evolution models differ.

Reflection always uses the complete interaction trajectory for failure attribution. During the Reflection stage, we retain all actions, environmental feedback, and original numeric object identifiers, including invalid actions. We do not remove any of the information mentioned by the reviewer. Therefore, the reported attribution results are already obtained using complete trajectories. The term Invalid Reflection in our paper refers to empty, malformed, or otherwise invalid Reflection outputs that fail to satisfy the required format, rather than filtered execution trajectories.

Unmatched Execution Lapses are not automatically reclassified as Skill Defects. Each Execution Lapse must match existing guidance in the Skill Body. Otherwise, the reminder is discarded. Since a matching failure does not necessarily indicate a skill defect, discarding unmatched reminders prevents unnecessary skill revisions based on insufficient evidence.

We sincerely thank the reviewer again for their valuable feedback. We hope that the additional experiments and clarifications adequately address the reviewer's concerns and further strengthen the paper.

References

[1] Chen et al. (2024). AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning. NeurIPS 2024. https://arxiv.org/abs/2405.16247

[2] Liu et al. (2024). Lost in the Middle: How Language Models Use Long Contexts. TACL, 12:157–173. https://aclanthology.org/2024.tacl-1.9/

[3] Leviathan et al. (2025). Prompt Repetition Improves Non-Reasoning LLMs. arXiv:2512.14982. https://arxiv.org/abs/2512.14982

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.