acceptodds
Withdrawn from ICLR 2027

STEPS: Scene-Grounded Task Escalation through Progressive Skill Learning toward Open-Ended Goals

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Code-as-Policy robot agents increasingly improve through practice, yet evalua- tion on predefined tasks makes it difficult to distinguish genuine capability acqui- sition from adaptation to practiced instances. We present STEPS, a framework for evaluating self-improvement on human-specified goals such as “tidy the office desk”. STEPS generates physically valid, oracle-solvable scenes; defines the full goal as an L5 task and derives L1-L4 diagnostic tasks probing subtasks, compo- nent capabilities, and constraints; and evaluates a frozen agent on the practiced L5 task, pose-perturbed L5 instances, and held-out lower-level tasks. Across five goals and 105 tasks, a skill library developed with human assistance achieves over 90% success on practiced L5 tasks but performs substantially worse on regener- ated instances and held-out L3-L4 tasks. On one evolution route, skills distilled from L1-L4 practice usually pass isolated validation but are rarely invoked when solving the full goal. Despite this fitting, some learned capabilities transfer be- yond the development scenes to external manipulation tasks, and weaknesses ex- posed by STEPS provide actionable targets for capability updates that further im- prove cross-domain performance. These results show that practiced-task success alone can overstate capability acquisition and should be complemented with goal- derived diagnostics that reveal what transfers, where improvement breaks down, and what capability to target next.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Agent, Simulation, 3d generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_xv9ZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

STEPS is a framework for testing self-improving Code-as-Policy robot agents on goals like "set up the desk for the day". It generates a scene from the goal, keeps it only if a privileged oracle solves the full goal (L5), and derives L1-L4 tasks that reuse the goal's relations and the L5 starting poses. The agent builds a skill library on L5 only. The library is then frozen and tested on the same L5, on L5 with objects moved, and on the L1-L4 tasks it never practiced. A human-assisted library solves 24/25 L5 trials but only 61.6% of L3 and 40.8% of L4 trials. When objects in L5 are moved slightly or moderately, the library succeeds only 58% and 31% of the time. The authors argue that high success on the practiced goal does not show that the agent has learned what the goal requires.

Critical strengths
  1. The task design separates real capability from (over)-fitting to one scene. The core design is good. The L1-L4 tasks reuse the goal relations and starting poses of L5, so a failure cannot be blamed on a harder layout, and each task is labeled with what its failure means. Other self-improving agents could be evaluated the same way.

  2. The stage records in Fig. 5b are useful. They show that learned skills are almost always offered on L5 but rarely executed unmodified. Other skill-library papers could use this check without the rest of the pipeline. This is good.

Critical weaknesses
  1. The evidence does not yet show that STEPS works as an evaluation tool. Only one library is tested, and code inspection already shows that it is tied to the objects of its development scene (Section 4.3), so its drop on scenes with moved objects is expected. Also, the L3 and L4 results in the abstract come from a first run with runtime faults. After the fix, the gap between L3 goal subtasks and L5 shrinks from about 32 to about 7 points, and the intervals overlap (Table 5).

  2. Limited novelty, and hard to reuse. Table 1 already credits GenSim, LIBERO-PRO and ASPIRE with the main components, so the new part is mostly the combination. No prompts are given, and the policy LLM and the harness agent are not named, which makes it hard for others to run STEPS on their own agents. I hope some experiments can be conducted to test whether STEPS can work reasonably well under different models.

Questions for authors
  1. Which LLM writes the policy programs in Tables 3 and 4, and what is the "harness agent"? How much human help went into the library, and could you add all prompts to the appendix?
  2. How were ASPIRE and CaP-Agent0 run, and did ASPIRE get an L5 development phase with a comparable model and budget? Were the Table 4 RATs numbers rerun or taken from their paper?
  3. Could you rerun L1, L2, L4 and the probes under the fixed runtime? And how much do scenes, tasks and per-level success change if STEPS is rerun on the same goal?

Minor:

  • Running two or more self-improving agents under the same develop-then-freeze protocol, with matched models and budgets, would show whether STEPS can rank agents. This could be interesting.
  • Minor: the framework and the evaluated agent are both called STEPS (Tables 3 and 4). Different names would make the paper easier to follow.
Reviewer_d8eDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes STEPS, a framework for evaluating whether self-improving agents actually acquire manipulation capabilities or just get better at the specific tasks they have practised.

Critical strengths
  • The paper covers the related literature well.
  • The hierarchical task generation aspect is interesting.
Critical weaknesses
  • The paper is very simplified in terms of tasks and environments.
Questions for authors
  • Please rewrite the abstract. It is very verbose.
  • Given the framework's largely automated nature, the evaluation seems surprisingly limited in goal diversity. Although the paper evaluates 105 tasks, they derive from only five underlying goals. How diverse are the resulting task hierarchies in terms of manipulation skills, geometric complexity, and planning requirements? More importantly, how well would the generation and validation pipeline scale to a substantially larger number of goals (e.g., 100)?
  • Oracle-based solvability is presented as a central contribution, but Appendix C.2 reports that only 87 of the 105 historical task specifications passed the validation criteria. Furthermore, the authors acknowledge that they did not verify complete oracle validation of the integrated evaluation suite. How can the benchmark's validity be established without verifying that its tasks satisfy the proposed solvability and nontriviality criteria? Could the authors report results restricted to validated tasks?
  • Since the scenes and task decompositions are largely generated using foundation models, how do the authors ensure the resulting evaluations faithfully represent the original human-specified goals rather than the generating model's interpretation? Although predefined predicates and oracle validation ensure some consistency and feasibility, they do not necessarily guarantee semantic correctness. Has any independent human evaluation of the generated task specifications been conducted?
  • The baselines used for scene generation (Holodeck, SceneWeaver and SAGE) all use foundation models (LLMs) in their pipelines. Did you change the ones they use (Holodeck and SceneWeaver use GPT-4; SAGE uses open-source ones) to match the one you use (GPT-5.6)? Are the improvements from your pipeline or from a better LLM?
  • Scene evaluation uses an LLM for most of Table 2. I urge the authors to find another way to evaluate their method.
  • The evaluation focuses on Code-as-Policy agents, and the authors acknowledge that extension to learned visuomotor policies remains future work. However, it is unclear whether the proposed diagnostics would remain meaningful for such policies, whose capabilities cannot necessarily be decomposed into explicit programmatic skills. Could the authors discuss how STEPS would evaluate learned behaviours and what changes would be required?
  • The evaluated goals predominantly involve object rearrangement and relatively simple spatial relationships. Appendix C.1 also describes oracle programs using prescribed manipulation sequences. How would the framework handle more challenging manipulation tasks requiring complex contact interactions, nontrivial motion planning, or multiple alternative action sequences? To what extent does the oracle-based validation restrict the generated tasks to those solvable by predefined manipulation routines?
  • The framework uses predefined spatial-relation predicates for goals. Its expressiveness is extremely limited. How do you establish that these predicates are enough to capture the human goal intent?
Reviewer_4ycbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents STEPS, an evaluation framework for self-improving Code-as-Policy agents. STEPS includes a task-scene generation pipeline, design for 5 different level tasks and evaluation on the self-improving code-as-policy agent. The authors evaluate the scene-generation pipeline against existing methods, use the generated tasks to diagnose the capabilities of a developed code-as-policy agent, and skill transferability on RoboSuite and a smaller LIBERO evaluation.

Critical strengths
  • Meaningful research question: The paper raises an important question about whether improvements on practiced tasks reflect genuine capability acquisition and whether learned skills contribute to downstream task performance.

  • Comprehensive scene-generation evaluation: Scene generation is evaluated on both visual quality and physical plausibility.

  • Revealing limitations of learned capabilities: The authors demonstrate that high success rates on practiced L5 tasks do not necessarily translate to reliable performance on held-out lower-level tasks or regenerated instances, highlighting the limitations of evaluating self-improvement through task success alone.

  • Transferable learned capabilities: The authors show that some capabilities acquired during L5 development can transfer to external tasks, despite fitting to the practiced scenes.

Critical weaknesses
  • Unclear development procedure and baseline fairness: The authors do not specify the number of improvement iterations, development trials, or human interventions involved in constructing the skill libraries, nor whether the baselines received equivalent assistance and development budgets. This makes it difficult to determine whether the performance differences arise from the learned capabilities or unequal development conditions. Reporting these details and providing comparisons under matched budgets would strengthen the experimental fairness and reproducibility.

  • Insufficient analysis of hierarchical task performance: Table 3 shows that ASPIRE outperforms STEPS on L4 tasks in the office and living-room scenes, despite performing substantially worse on L5 tasks. The authors should further analyze this discrepancy and clarify how performance on lower-level tasks relates to the capabilities required for L5 success.

Overall, the paper raises an interesting and important question about whether success on practiced tasks reflects genuine capability acquisition. Although the experiments demonstrate the diagnostic value of the proposed framework, the lack of matched development budgets and unclear human assistance across baselines weaken the broader conclusions about skill improvement and generalizability.

Questions for authors
  1. For results in Table 3 and Table 4, how many improvements/iterations were given to each method before evaluation on downstream tasks? Did the baselines also receive the same human-specified L5 goals (e.g., "Set up the desk") and comparable human assistance?
  2. Could the authors conduct additional baseline experiments in which ASPIRE and RATs receive equivalent human assistance?
  3. Could the authors provide a more detailed comparison between the learned skill libraries evaluated with STEPS and those of the baselines, particularly RATs? Which specific skills or capabilities account for the differences in downstream performance?
  4. Most qualitative examples in Appendix A.2 involve lifting, placing, or pushing objects. Could the authors discuss whether the learned skill library generalizes to more diverse manipulation tasks, particularly articulated-object interactions such as opening drawers?
Reviewer_cPvgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

STEPS generates scenes and related tasks from human goals to test whether a robot agent learns reusable skills. The paper shows that high success on practiced goals can hide failures on subtasks and new layouts. The evaluation idea is interesting and intuitively worthy explore, but I have concerns about how the experiments were validated and compared.

Critical strengths
  • The distinction between goal subtasks, component skills, and extra constraints is useful. These tests reveal different weaknesses that a full-task score would miss.

  • The large performance drop under pose perturbations is an interesting result and makes a good case for testing beyond the practiced layouts.

Critical weaknesses
  • The subtask tests change both the instructions and the scene content. Failures could come from grounding, skill selection, or execution, so it is difficult to tell which capability needs improvement.

  • Human assistance and target-domain guidance make the transfer results hard to interpret. Their roles need to be clearer before the gains can be attributed to skills learned through STEPS.

  • The benefit of the hierarchy over simpler held-out tasks and pose perturbations is not fully demonstrated.

Questions for authors
  • What do the results look like when restricted to tasks validated under the final runtime?

  • Were all baselines rerun after the runtime fixes?

  • What RoboSuite-specific guidance was provided, and how does performance change without it?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.