- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ACSE generates task variants from solver failures, validates physical feasibility, filters by completion probability, and weights training through a population meta-game. Its AWAM implementation updates experience memory, action-head LoRA, and a supervised physical head while freezing Qwen3.5 and the future predictor. Evaluation covers RoboDojo, BiCoord, and two physical manipulation tasks.
Critical strengths
-
Practical task construction. Asset, constraint, and reference-execution checks distinguish physical feasibility from difficulty for the current solver. This is a useful design for generating robotic training tasks.
-
Evidence of adaptation gains. BiCoord success improves from 38.2% to 46.0%, exceeding random sampling (40.1%) and failure-weighted sampling (41.6%). Table 5 also supports combining memory and parameter updates. The physical evaluation shares randomized layouts and fixes model parameters during testing.
Critical weaknesses
-
The population-game formulation is not connected to the generator algorithm. Eq. 6 defines a generator response against a mixture of solvers. Algorithm 1 instead generates and filters tasks using the incumbent solver before computing that mixture; it does not show how the solver mixture affects subsequent generation or selection.
The concrete meaning of a generator strategy and its population update is also insufficiently specified. Calling generation a heuristic response does not establish its relationship to Eq. 6. Consequently, the distinctive algorithmic contribution beyond failure-guided generation and population-based task weighting remains unclear.
-
The capability-boundary and generalization claims are not isolated experimentally. Table 4 compares sampling weights, and Table 5 tests solver updates. Neither separates failure-conditioned proposals or capability filtering from data expansion and reweighting under matched interaction and RL budgets. Nor is the simulation task/scene separation between generation, adaptation, and final evaluation specified. The results support adaptation, but do not establish that boundary-targeted generation drives improvement on genuinely unseen conditions.
Questions for authors
- What concretely defines a generator strategy, and where does the solver mixture enter its proposal or selection procedure?
- Which existing controls isolate capability filtering and targeted generation? Please clarify matched budgets and the held-out simulation tasks/scenes.
6 comments
Response to Reviewer 93gq
Q1: The population-game formulation is not connected to the generator algorithm. Eq. 6 defines a generator response against a mixture of solvers. Algorithm 1 instead generates and filters tasks using the incumbent solver before computing that mixture; it does not show how the solver mixture affects subsequent generation or selection.
The concrete meaning of a generator strategy and its population update is also insufficiently specified. Calling generation a heuristic response does not establish its relationship to Eq. 6. Consequently, the distinctive algorithmic contribution beyond failure-guided generation and population-based task weighting remains unclear.
R1: Thank you for raising this question. The solver mixture first enters the generator response in line 8 of Algorithm 1. In lines 3–5, the generator uses failure evidence from the incumbent solver to propose candidate tasks and adds physically feasible candidates to the training pool. Lines 6–7 estimate the payoff matrix and compute the solver and generator mixtures. In lines 8–9, the generator samples solver checkpoints according to the solver mixture and uses unmet subgoals in their execution traces to propose a new generation rule.
Generator strategy. A generator strategy is a reusable rule that defines a distribution over task instances. For example, a bimanual handover rule can preserve the handover goal while specifying how the handover position and the relative timing of the two arms may vary. Scenes generated by this rule are instances of the same strategy. Each rule corresponds to one column of the payoff matrix. Each entry in that column estimates one solver checkpoint’s mean return on tasks sampled from the rule.
Connection to Eq. (6). Equation (6) seeks an admissible rule that minimizes the expected return of the solver mixture. In practice, the generator samples solver checkpoints according to the mixture weights in Eq. (6), identifies unmet subgoals in their execution traces, and constructs a new rule targeting those subgoals. Checkpoints with higher weights are sampled more often in expectation, so the difficulties they encounter are more likely to guide the proposal. This procedure provides a heuristic response to Eq. (6).
Population update. In lines 10–11, the solver samples training tasks according to the generator mixture and updates its policy. Lines 12–13 consolidate the Harness memory and form the new solver checkpoint. Line 14 adds the checkpoint and the new generation rule to their respective populations. In the next round, the new rule corresponds to a new column in the payoff matrix. We have revised Algorithm 1 and the main text to make both population updates explicit.
Response to Reviewer 93gq
Q2: The capability-boundary and generalization claims are not isolated experimentally. Table 4 compares sampling weights, and Table 5 tests solver updates. Neither separates failure-conditioned proposals or capability filtering from data expansion and reweighting under matched interaction and RL budgets. Nor is the simulation task/scene separation between generation, adaptation, and final evaluation specified. The results support adaptation, but do not establish that boundary-targeted generation drives improvement on genuinely unseen conditions.
R2: Thank you for raising this point. We added a uniformly sampled, failure-guided baseline to Table 4. Comparing it with Random tests whether failure diagnoses improve scene generation when the number of new scenes and the training budget are held fixed. In our formulation, the capability interval describes the target difficulty range. Algorithm 1 admits physically feasible candidates to the training pool without applying a fixed completion-rate threshold, so there is no separate filtering step to ablate.
Extended Table 4. Scene generation and task sampling on BiCoord. SR denotes full-task success rate, and SSR denotes stage-wise success rate. We have updated Table 4 of the manuscript accordingly.
Table 1. Scene generation and task sampling on BiCoord
| Method | Failure-guided generation | Sampling | SR (%) ↑ | SSR (%) ↑ | TL ↓ |
|---|---|---|---|---|---|
| Base | — | — | 38.2 | 53.8 | 491 |
| Random | No | Uniform | 40.1 | 55.4 | 470 |
| Random + failure guidance | Yes | Uniform | 40.9 | 55.0 | 452 |
| Failure-Weighted | Yes | Failure-weighted | 41.6 | 56.3 | 438 |
| ACSE | Yes | Payoff-weighted | 46.0 | 61.1 | 410 |
1. Random scene expansion and additional training. Continued training on randomly generated scenes increased SR from 38.2% to 40.1% and SSR from 53.8% to 55.4%, gains of 1.9 and 1.6 percentage points, respectively.
2. Failure-guided generation (Random vs. Random + failure guidance). Both methods generate new scenes and sample uniformly from their respective scene pools. The added control uses unmet subgoals identified in execution traces to guide scene modifications. It outperforms Random on the same held-out instances, indicating that failure-guided generation improves performance on unseen instances when the number of new scenes and the training budget are matched.
3. Adaptive sampling and co-evolution (Failure-Weighted vs. ACSE). Failure-Weighted allocates training rollouts according to the current solver’s empirical failure rates. ACSE uses returns across solver checkpoints to weight generator strategies and samples checkpoints from the solver mixture to guide subsequent scene generation. With the same number of solver-training rollouts, ACSE outperforms Failure-Weighted by 4.4 percentage points in SR (41.6% to 46.0%) and 4.8 percentage points in SSR (56.3% to 61.1%). These gains quantify the benefit of the full ACSE configuration over the failure-weighted baseline.
Budgets and evaluation split. All adaptation methods used the same initial solver, number of solver-training rollouts, and solver-update rules. Random and Failure-Random also used the same mutation operators, number of new scenes, total simulation rollouts, and RL update steps. We counted each rollout once, including rollouts reused for payoff estimation. Scene generation and solver adaptation used training data only. We selected checkpoints on validation data and evaluated all methods on the same held-out BiCoord instances. These results therefore measure generalization to new instances within the 18 known task categories.
Response to Reviewer 93gq
Thank you for carefully reviewing our manuscript. We appreciate your feedback and respond to each comment below.
Clarification of our core contribution. We thank the reviewer for the thoughtful feedback. Our core contribution is ACSE, an agentic framework that enables the training task distribution and the solver to co-evolve in long-horizon embodied manipulation. ACSE addresses three questions:
What should the solver practice after a failure? The Capability Analyzer uses execution traces and observed object states to identify the first unmet subgoal. The generator then creates physically feasible task variants that allow the solver to practice the operation that failed.
How should a limited training budget be allocated? For each generation rule, ACSE estimates each solver checkpoint’s mean return on tasks sampled from that rule. It uses the resulting payoff matrix to compute sampling weights. Two rules with the same failure rate for the current solver can therefore receive different weights if earlier checkpoints achieved different returns on their tasks.
How can the solver revisit earlier skills while acquiring new ones? ACSE retains historical generation rules so the solver can continue practicing earlier tasks. Failures observed after each solver update guide the construction of new task variants that target difficulties the solver still encounters.
This feedback loop extends the agent’s autonomy from executing assigned tasks to shaping its own training curriculum through experience. It represents an initial exploration of recursive self-improvement in embodied systems.