- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
STEPS is a framework for testing self-improving Code-as-Policy robot agents on goals like "set up the desk for the day". It generates a scene from the goal, keeps it only if a privileged oracle solves the full goal (L5), and derives L1-L4 tasks that reuse the goal's relations and the L5 starting poses. The agent builds a skill library on L5 only. The library is then frozen and tested on the same L5, on L5 with objects moved, and on the L1-L4 tasks it never practiced. A human-assisted library solves 24/25 L5 trials but only 61.6% of L3 and 40.8% of L4 trials. When objects in L5 are moved slightly or moderately, the library succeeds only 58% and 31% of the time. The authors argue that high success on the practiced goal does not show that the agent has learned what the goal requires.
Critical strengths
-
The task design separates real capability from (over)-fitting to one scene. The core design is good. The L1-L4 tasks reuse the goal relations and starting poses of L5, so a failure cannot be blamed on a harder layout, and each task is labeled with what its failure means. Other self-improving agents could be evaluated the same way.
-
The stage records in Fig. 5b are useful. They show that learned skills are almost always offered on L5 but rarely executed unmodified. Other skill-library papers could use this check without the rest of the pipeline. This is good.
Critical weaknesses
-
The evidence does not yet show that STEPS works as an evaluation tool. Only one library is tested, and code inspection already shows that it is tied to the objects of its development scene (Section 4.3), so its drop on scenes with moved objects is expected. Also, the L3 and L4 results in the abstract come from a first run with runtime faults. After the fix, the gap between L3 goal subtasks and L5 shrinks from about 32 to about 7 points, and the intervals overlap (Table 5).
-
Limited novelty, and hard to reuse. Table 1 already credits GenSim, LIBERO-PRO and ASPIRE with the main components, so the new part is mostly the combination. No prompts are given, and the policy LLM and the harness agent are not named, which makes it hard for others to run STEPS on their own agents. I hope some experiments can be conducted to test whether STEPS can work reasonably well under different models.
Questions for authors
- Which LLM writes the policy programs in Tables 3 and 4, and what is the "harness agent"? How much human help went into the library, and could you add all prompts to the appendix?
- How were ASPIRE and CaP-Agent0 run, and did ASPIRE get an L5 development phase with a comparable model and budget? Were the Table 4 RATs numbers rerun or taken from their paper?
- Could you rerun L1, L2, L4 and the probes under the fixed runtime? And how much do scenes, tasks and per-level success change if STEPS is rerun on the same goal?
Minor:
- Running two or more self-improving agents under the same develop-then-freeze protocol, with matched models and budgets, would show whether STEPS can rank agents. This could be interesting.
- Minor: the framework and the evaluated agent are both called STEPS (Tables 3 and 4). Different names would make the paper easier to follow.