acceptodds
Under review as a conference paper at ICLR 2027

HumanoidHarness: Self-Improving Humanoid Task Execution through Experience-Guided Agentic Policy Revision

Abstract

Pretrained robot policies provide reusable action capabilities, but completing humanoid tasks requires maintaining situational awareness, adjusting body–target alignment, and adapting execution to the environment. Whole-body motion changes these conditions throughout an interaction, making feedback and recovery essential to coordinating learned capabilities. We introduce HumanoidHarness, a self-improving agentic framework that organizes pretrained controllers through an editable execution policy. The policy specifies controller stages, observation checks, and recovery transitions, and is validated and compiled into a state machine. During execution, the harness bounds each action segment and uses new observations to assess progress and select the next state. Across attempts, a reflection journal and task memory retain execution evidence to guide policy revision and reuse. This design separates immediate execution feedback from experience-guided changes to the policy, while keeping pretrained controller parameters fixed. The same agent logic supports simulation and hardware using egocentric vision, proprioception, and task instructions. Results on HumanoidArena show improved task completion and a generally steady increase in success rate across policy revisions with episode memory, while hardware examples illustrate how coordinated actions can adapt to environmental constraints.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Humanoid, Harness, agent

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_coRtOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper presents HumanoidHarness, an agentic framework for coordinating pretrained policies for humanoid whole-body tasks. Its main components are an editable execution-policy graph, which specifies controller stages, observation checks, and recovery transitions, and an experience-guided revision process that uses execution history and memory to update this graph while keeping the underlying controllers fixed. The framework is evaluated on HumanoidArena and a physical Unitree G1, with experiments examining task completion, execution strategies, and the effect of episode memory.

Critical strengths

Complete system and substantial engineering effort. The paper integrates policy authoring, graph validation and compilation, supervised execution, and experience memory into a working system supporting both simulation and physical deployment.

Technically reasonable approach with experimental support. The overall framework appears technically sound. The experiments across multiple simulated tasks and the hardware demonstrations provide evidence that the system can improve the execution of pretrained controllers.

Critical weaknesses

1. Insufficiently polished writing and unclear presentation

The manuscript gives me the impression of insufficiently revised AI-assisted writing. I do not object to using AI for writing assistance. However, the authors are responsible for ensuring that the paper is logically coherent, precise, and readable. The manuscript frequently introduces notation and terminology without sufficient explanation at the point of use. Examples include:

  • Πaux\Pi_{\mathrm{aux}} (line 168): The notation is introduced without definition.
  • Terms such as “editable object” (line 169), “compiler” (line 173), and “bounded segment” (line 174) are introduced without sufficient explanation at first use. The paper should clearly define what the agent can edit, what compilation entails, and what limits govern each execution segment.
  • The fourth paragraph of the introduction: The connection between execution recovery, experience-guided policy revision, and the specific research contribution needs a clearer logical progression.

There are many other similar issues. Although subsequent sections explain some aspects of these concerns, their abrupt introduction makes the method unnecessarily difficult to follow. The paper needs substantial revision to introduce concepts in a logical order, define terminology consistently, and explain the actual procedure through concrete examples.

2. Limited methodological novelty and insufficient insight into design choices.

The contribution appears primarily to be an engineering integration of agentic components. The paper does not sufficiently establish what generalizable insight distinguishes this particular combination from existing approaches to agentic robot control. The rationale for the individual design choices is also underdeveloped. It remains unclear why the proposed graph representation, supervision mechanism, and reflection process are necessary, or under what conditions each provides a meaningful advantage over simpler alternatives.

Section 4.4 includes an episode-memory ablation, and Table 2 compares several execution strategies. However, these experiments provide limited isolation of the contributions of individual components and their interactions. More targeted ablations and controlled alternatives are needed to explain which design choices account for the improvements. As presented, the paper demonstrates a functioning implementation more clearly than it establishes a substantive research contribution appropriate for a top-tier AI conference.

Questions for authors

See weakness

Reviewer_NvrYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HumanoidHarness, an agentic framework for improving humanoid task execution through experience-guided policy revision. The framework integrates pretrained VLA/WAM controllers, an editable execution policy, state-machine-based supervision, observation checks, and persistent experience memory. Experiments on HumanoidArena report improved task success rates, while real-world Unitree G1 demonstrations show obstacle avoidance and long-horizon task execution. The paper addresses a relevant practical problem and presents a functional system. However, I have concerns regarding the methodological novelty and whether the experiments sufficiently support the central claim of self-improvement.

Critical strengths
  1. Meaningful practical problem. The paper addresses an important challenge in humanoid robotics: coordinating pretrained controllers, monitoring execution, and recovering from failures without modifying low-level policy parameters.

  2. Reasonably complete system design. The framework integrates policy compilation, supervised execution, recovery transitions, and experience memory. The separation between high-level agent decisions and bounded low-level execution is well motivated.

  3. Promising experimental results. The reported improvements across multiple HumanoidArena tasks and the real-world demonstrations provide evidence of practical effectiveness. The real-robot videos demonstrate successful obstacle avoidance and controller switching.

Critical weaknesses
  1. Limited methodological novelty. Most components, including LLM-based task planning, executable state machines, feedback-driven recovery, reflection, and persistent memory, have been explored in previous work. Although the proposed framework extends these ideas to humanoid whole-body tasks, the paper does not sufficiently establish its technical differences from related approaches such as Harness VLA, ASPIRE, and existing behavior-tree-based robotic systems. The specific novelty of the proposed policy revision mechanism remains unclear.

  2. Insufficient evidence for generalizable self-improvement. The paper reports increasing success rates over 200 policy revisions, but the evaluation appears to reuse the same set of task configurations. Consequently, the observed improvements may reflect adaptation to previously encountered settings rather than generalizable learning. A held-out evaluation of revised policies on unseen environments or task instances would be necessary to substantiate the self-improvement claim. Moreover, the ablation studies do not fully isolate the contributions of policy revision, online feedback, and memory. The attempt budgets and success-rate accounting across different experimental tables also require clarification.

  3. Limited real-world evaluation of efficiency and policy improvement. Although the real-world demonstrations show successful obstacle avoidance and task completion, the robot frequently pauses between execution segments, resulting in noticeable discontinuities and slow task execution. The paper reports visual-assessment latency but does not provide a comprehensive breakdown of end-to-end execution time or controller-switching overhead. Furthermore, the reported 10/10 success rates are measured after policy development, without matched quantitative comparisons before and after policy revision. These experiments demonstrate system feasibility but provide limited evidence for the effectiveness of the self-improvement mechanism on physical hardware.

Questions for authors
  1. How does the proposed experience-guided policy revision fundamentally differ from existing LLM-based program repair, behavior-tree editing, and memory-guided robot execution frameworks? Which specific mechanism constitutes the main methodological contribution?

  2. Are the policy-revision experiments evaluated on the same task configurations used to accumulate experience? Can the authors demonstrate improvements on held-out environments or task instances, and clarify whether the direct-execution and harness baselines use matched attempt budgets?

  3. Can the authors provide a breakdown of end-to-end real-world execution latency, including visual assessment, waiting time, and controller switching? Additionally, can the contribution of policy revision be evaluated quantitatively on real hardware using matched experimental conditions?

Reviewer_zzz5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HumanoidHarness uses an LLM to revise a restricted execution program over frozen humanoid controllers and support primitives. The program specifies controller stages, checks, and recovery transitions, and is validated and compiled into a state machine. Execution records guide later revisions. Evaluation includes seven HumanoidArena tasks, comparisons under interaction budgets, memory ablations, and two physical-robot tasks.

Critical strengths

The experiments address whether improvement comes simply from extra retries. With at most four attempts per episode, the system reaches 81.9% mean success, compared with 76.1% for a primitive-call agent and 79.3% for agent-only checks. The Boxing study also compares tool-call budgets. These controls support a practical benefit from the execution framework.

Critical weaknesses

Major

  1. The methodological advance is limited. Program-based control, execution checks, reflection, and reusable experience are established ideas, including in Code as Policies and ASPIRE [1,2]. The restricted template, compiler, and supervisor form a useful implementation, but the paper does not clearly identify a new revision or memory mechanism. The experiments compare execution strategies without establishing what is gained over a closely matched prior program-repair system.

  2. The strongest self-improvement evidence does not establish generalization after revision. Figure 4 repeatedly revises and evaluates on the same 60 settings per task. This demonstrates improvement on those settings, but needs evaluation with the final program and memory fixed on independent starting states and layouts. The two physical tasks achieve 10/10 each after development, without accounting for development attempts or providing a matched comparison of the improvement process.

Minor

Please specify the memory configuration used in each experiment and report the reliability and latency of visual checks. The additional tool-call budget analysis and memory ablation cover only one and two tasks, respectively.

[1] Liang et al. Code as Policies: Language Model Programs for Embodied Control. https://arxiv.org/abs/2209.07753 [2] Lu et al. ASPIRE: Agentic /Skills Discovery for Robotics. https://arxiv.org/abs/2607.00272

Questions for authors
  1. Evidence affecting the assessment: After freezing the final program and memory, what success rate is obtained on independent starting states and object layouts? Please distinguish this test from the Figure 4 revision settings.
  2. Clarification: What specific revision or memory mechanism differs from the cited program-repair systems, and which matched comparison isolates its effect?
  3. Evidence affecting the assessment: How many physical trials, agent calls, and manual changes were needed before the final 10/10 evaluations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.