- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies test-time evolution for new humanoid loco-manipulation tasks by repurposing existing skills without retraining the underlying controller. The core system, InterEvolve, represents a task as an editable, staged reward program: an LLM modifies the stage structure and reward terms based on the task, scene, and execution feedback, while CMA-ES tunes numerical constants. The resulting program is executed by a forward-backward behavioral foundation model augmented with an object-aware residual. Verified programs are stored in a skill library to support reuse and composition on subsequent tasks. Object-aware residual branches are trained for the actor and the forward and backward maps while the pretrained body branches remain frozen. Stage rewards weight the cached backward embeddings of a fixed state bank, and their projected sum supplies the latent prompt without per-task RL training. The experiments cover reference motion tracking, eight families of goal-conditioned tasks, composite long-horizon tasks, and deployment on a Unitree G1. The reported reward evolution increases the macro-average success rate from 34.6% for the fixed program to 86.5%, and raises reference-tracking success from 60% to 72%; the skill library achieves 8/10, 4/10, and 5/10 on three composition tasks. Overall, the paper presents a clear and appealing paradigm of “searching over task specifications while keeping the controller fixed.” However, the current evidence comes primarily from privileged-state simulation search, the real-robot experiments are small, and the treatment of search stochasticity and comparisons with general test-time planning methods remain insufficient.
Critical strengths
-
The paper decomposes multi-stage contact tasks into rewards, completion conditions, and tunable constants, allowing LLMs and CMA-ES to handle structural editing and numerical calibration respectively. The program semantics, feature namespace, and code constraints enforced by the validator and documented in the appendix make the method relatively easy to inspect, and make it easier to express stages such as contact, transport, and release than directly generating joint trajectories.
-
Table 1 shows that the body-only BFM-Zero performs substantially worse in terms of object error and success rate, whereas adding object states and link geometry gives the controller reusable object-interaction capabilities. Tables 2, 3, and 13 further show that the multi-stage structure and constant calibration jointly provide significant gains. The drift-correction, interaction-input, and residual-capacity ablations also indicate that the gains do not come solely from changing the network size.
-
Beyond reference tracking, the paper evaluates carrying, tipping, kicking, and obstacle-navigation tasks, and reports a task-level verifier, earned tiers, search cost, and the reported search/evaluation scene splits. The program and lessons ablations for the skill library further suggest that the reuse gains come from adapting existing programs rather than simply copying them. The appendix provides search budgets, scene splits, feature interfaces, and the real-robot perception pipeline, making the overall reproducibility information fairly complete.
Critical weaknesses
-
The staged design of the reward program, the object-aware residual, LLM-based structural editing, CMA-ES constant tuning, and skill-library reuse each have clear similarities to hierarchical reward shaping, residual control, program search, and skill-library methods. Section 2 relates the design to Language to Rewards and FB reward inference. The object-aware controller and staged program interface are meaningful differences, but stronger evidence against alternative reward-search procedures would clarify how much additional value their joint test-time evolution provides.
-
The reward-inference bank is built from body-object states and surface geometry in pretraining replay, and the verifier directly reads simulator states. Within the reported search and confirmation grids, the object bearing varies by ±12° and the distance by ±0.15 m around nominal initial states. The real G1 experiments show only a few qualitative kicking and pushing sequences, without success rates across trials, perception dropouts or errors, different objects, or substantial scene changes. It is therefore still unclear whether program evolution in simulation is robust to real-world perception noise.
-
Reference tracking is compared with ULTRA and BFM-Zero, but the goal-conditioned tasks are mainly compared with human/initial fixed programs using the same controller; there is no compute-matched comparison with non-LLM test-time planners, continuous reward search, or other task-specification optimization methods. A full search uses 6.7–14.2M environment steps per task family, averaging 9.5M, plus LLM calls. The budget-matched component ablations provide useful evidence for the design choices, but do not establish an advantage over alternative search procedures under matched total simulation and compute budgets.
Questions for authors
-
Within the reported three-stage limit and feature vocabulary, how can failures due to limited expressiveness be distinguished from those caused by insufficient search?
-
How sensitive are bank-based reward-to-prompt projection and the object-aware residuals to unseen geometries, pose errors, and poorly represented contact configurations?
-
Do final test scenarios influence program selection? Please reconcile Section C.3's separate evaluation set with Section D.1's use of test scenarios for confirmation.
-
How are incompatible contact assumptions resolved during skill-library composition, and what timing or trajectory deviations can the frozen controller tolerate during physical latent-prompt replay?
2 comments
Response to Reviewer Mgfc (2/2)
Q1: Do the remaining failures come from limited expressiveness or from insufficient evolution? Success improves during the initial evolution rounds and remains unchanged over three additional rounds (Table R3), supporting convergence within the original budget on these composite tasks.
Table R3. Success rate (%) on the composite tasks across evolution rounds (revised manuscript: Table 21, p. 33, L1763–1768)
| Composite task | Round 1 | Round 2 | Round 3 | Round 4 (new) | Round 5 (new) | Round 6 (new) |
|---|---|---|---|---|---|---|
| Relocate between supports | 31.3 | 43.8 | 71.9 | 71.9 | 71.9 | 71.9 |
| Stack two boxes | 4.7 | 28.1 | 45.3 | 45.3 | 45.3 | 45.3 |
| Carry, place, and kick | 50.0 | 84.4 | 90.6 | 90.6 | 90.6 | 90.6 |
Q2: How sensitive are projection and the object-aware residuals to unseen geometry, pose errors, and contact coverage? Pose errors are covered by training-time randomization (Table G2 in the general response), a controller trained on one box achieves higher tracking success than ULTRA on unseen suitcase and plastic-box geometries (Table R4), and projection works when the bank holds states relevant to the task.
Q2.1: Unseen geometry? We use suitcase and plastic-box interaction sequences from OMOMO, retargeted to the Unitree G1 using the pipeline described in Sec 4.1 (p. 6, L309–319), together with their corresponding object meshes. These object categories are unseen during training of the large-box-only controller. Without retraining, it achieves 32% and 30% tracking success on 50 reference motions per category, respectively, compared with 30% and 22% for tracking baselines (ULTRA).
Table R4. Reference tracking on OMOMO suitcase and plastic-box objects, unseen by the large-box-only controller and ULTRA baseline controllers (revised manuscript: Table 22, p. 34, L1783–1790)
| Object | Controller | SR (%) | E_h (cm) | E_o (cm) |
|---|---|---|---|---|
| Suitcase | Ours (Large-box only) | 32.0 | 27.26 | 42.51 |
| Suitcase | ULTRA | 30.0 | 27.86 | 39.74 |
| Plastic box | Ours (Large-box only) | 30.0 | 32.75 | 53.21 |
| Plastic box | ULTRA | 22.0 | 32.45 | 49.09 |
Q2.2: Poorly represented contacts? Projection handles them when the reward focuses on the few relevant bank states. On the feet-only task, whose foot contacts are rare in the replay data, uniform weighting gives 0% success, a moderate focus gives 59.4%, and banks of 12.5k to 100k states give 53.1–65.6%, with drops only at the extremes of bank size or focus (Table 9 in Sec. B.3, pp. 19–20, L1010–1042).
Q3: Do the final test scenarios influence program selection? No. Selection uses only the search and confirmation scenarios, and the final test set is used once, after the program is fixed (Sec. C.3, p. 23, L1199–1206).
Q3.1: Which scenario sets are used? Three disjoint sets:
- Search grid, 16 scenarios: the agent proposes candidates, and CMA-ES tunes their constants.
- Confirmation grid, 64 scenarios with 3 repeats: a candidate replaces the current best only if it wins by more than the evaluation noise, so lucky edits are rejected.
- Final test set, 64 scenarios: evaluates the fixed program; evolution never sees these scenarios.
Q3.2: Sec. D.1 (p. 26, L1382–1383) has a typo. Thanks for catching this. Its phrase "confirmation re-evaluates candidates on the test scenarios" is a typo for "on the confirmation scenarios", which the revision corrects. Scenario seeds in our logs confirm that the confirmation and final test sets share no scenario.
Q4.1: How are incompatible contact assumptions resolved during composition? A reward only encourages a behavior within its own stage; it is not a hard constraint and not applied in all stages. When carry, place, and kick switches from the carrying reward, which favors hand contact, to a kicking reward such as no_hand · (push + θ0 · strike) from Fig. 2(a) (p. 4, L178–183), the kicking reward is zero at the switching point. The controller is then prompted toward the states this reward favors, so the robot releases the box and kicks it. Execution feedback tells the agent when a contact term needs revision.
Q4.2: What timing and trajectory deviations can execution tolerate? We now run both latent-prompt replay and closed-loop stage transitions on the real G1 (Table G1 in the general response). Replay succeeds in 11 of 15 trials with start shifts of up to 0.5 m and in 4 of 5 with a disturbance during carrying, as the controller is trained with deviations. Closed-loop stage transitions also re-time each stage switch to the observed state and recover from failures by returning to an earlier stage. Please see “10 / Deploy" and "Real G1 / Recovery" on the supplementary webpage.
We hope the physical trials, the budget-matched comparisons, and the protocol clarification address the reviewer's main concerns and we would be glad to answer further questions.
Response to Reviewer Mgfc (1/2)
We thank the reviewer for the detailed review, and for recognizing the inspectable program interface, the gains from object-aware control and staged optimization, and the breadth of the evaluation and reproducibility details. In short, evolution beats matched-budget search baselines (W3), programs transfer to the real G1 without tuning (W2), and test scenarios never influence selection (Q3).
W1: Why do the object-aware controller and the staged program interface matter beyond similar methods? They are what make test-time evolution for loco-manipulation possible. The comparison with alternative searches, requested here and in W3, is answered in W3.
W1.1: Our contribution. To our knowledge, InterEvolve is the first framework for test-time evolution of humanoid loco-manipulation. Some of its components resemble established methods, but their interface is new: staged reward programs run at test time rather than through per-candidate policy training as in Eureka, on a frozen object-aware controller rather than a body-only one such as BFM-Zero. Test-time execution also makes CMA-ES calibration affordable, since each constant setting costs only rollouts, whereas Eureka relies on the LLM to set the constants.
W1.2: Object-aware controller supplies the interaction competence that programs repurpose: the body-only BFM-Zero succeeds on 8% of tracking clips against 72% for ours (Table 1, p. 7, L325–337), and without object inputs the controller follows the body but rarely completes the task (Table 6, p. 17, L865–871).
W1.3: Staged program interface versus hierarchical reward shaping. Hierarchical reward shaping usually fixes phase rewards before training, whereas our stages, rewards, and completion conditions stay editable at test time. Staging is the largest single contributor to evolution: allowing only one stage drops success from 86.5% to 44.7% (Table 3, p. 8, L393–403).
W2: Is program evolution in simulation robust to real-world perception noise? We clarify that perception noise does not change what a reward program specifies: the program defines the task, not how it is sensed. For example, a carrying program rewards reducing the distance from the box to the marker. It describes what the box and the body should do, whether the box pose is observed exactly or with noise, so programs are developed and verified in simulation by design. The sim-to-real gap lies in execution: whether the frozen controller, given the program, tolerates real observation noise.
W2.1: Training for observation noise. The controller is trained with domain randomization over object properties, robot dynamics, actuation, control delay, and observation noise (Table G2 in the general response).
W2.2: New videos. The updated supplementary webpage shows real-world executions from shifted starts ("10 / Deploy") and recovery from disturbances ("Real G1 / Recovery").
W2.3: New quantitative evaluation. Our controller succeeded in 27 of 35 real-world trials across pushing, kicking, and carrying, including shifted starts and a disturbance during carrying (Table G1 in the general response).
W2.4: Do the ±12° and ±0.15 m grids limit the tested variation? They bound only the scenarios used during evolution. Execution is tested under larger variances, e.g., real-world trials add shifted starts of up to 0.5 m ("10 / Deploy").
W3: How does evolution compare with search baselines under matched budgets? Evolution with LLM feedback is best or tied on all three task families tested (Table R2).
W3.1: Which baselines, and under what controls? All four methods share the frozen controller, task definitions, verifier, and scenarios, and each task gives every method the same simulation budget: 9.7 M steps for pushing, 7.6 M for kicking, and 9.6 M for lifting. The alternatives cover the reviewer's request:
W3.2: What does the comparison show? Execution feedback matters most: Search reward without LLM feedback never solves kicking, which requires dedicated body use, although it matches evolution on pushing and comes close on lifting.
Table R2. Success rate (%) under matched simulation budgets (revised manuscript: Table 19, p. 32, L1721–1727)