acceptodds
Under review as a conference paper at ICLR 2027

Comparing Models, Skills, Feedback and Curriculum Design for Agentic Robotics

Abstract

Agentic robotics asks coding agents to implement and iteratively improve robot policies from simulated and real-world physical interactions. We study the self-learning loop of agentic robotics through a controlled simulation study spanning nine manipulation tasks, 631 different experiment configurations, and 28K coding agent rollouts, systematically varying model capability, skill priors, feedback structure, and curriculum. The experiment data suggests that the bottleneck is often not the access to sufficient information, but interpreting it into the right policy update. Across these factors, a successful self-learning loop depends on how effectively agents attribute failures, reason task-relevant spatial and geometric structure, and selectively restructure code to improve policies.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Agentic Robotics, LLM for Robotics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DiB4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a simulation study of feedback-driven code revision for robotic manipulation. Using a shared execution framework and skill library, it examines model choice, skill priors, temporal and spatial feedback, and a privileged-to-perception curriculum across five VABAR tasks and four MimicGen tasks. The model comparison evaluates initial policies and two feedback rounds; the curriculum experiments provide longer trajectories with more iterations.

Critical strengths
  1. The study evaluates policy improvement beyond initial generation and reveals meaningful differences in learning dynamics. In Tab.1, Opus 5 xhigh, Fable 5.1, GPT-6 Astra, and Sonnet 5 begin at 43–45% overall success but reach 74%, 64%, 48%, and 46% after two feedback rounds. This shows why initial success alone is an incomplete evaluation of an agent that can revise its policies. Fig.10 adds useful distinctions between partial progress, delayed improvement, and stagnation. For example, perception-only Threading improves its stage reward while task success remains near zero. These observations provide a valuable basis for evaluating iterative robot programming.

  2. The feedback and curriculum comparisons offer concrete design insights, including important negative results. Tab.2 and 3 report overall success of 25/36/52% for episode/motion/skill feedback and 30/40/52% for world-frame/object-frame/part-annotated feedback. The earlier perception error behind Tool Hanging's final reaching failure makes the value of intermediate feedback tangible. The curriculum also has sharply different effects across tasks: Tab.4 reports 86% versus 2% on Threading, but 16% versus 63% on Tool Hanging, for curriculum versus perception-only learning.

Critical weaknesses
  1. The source and robustness of the iterative gains remain insufficiently characterized. Moving from @0 to @1 and @2 adds model inference, execution, candidate policies, and potentially validation-based selection. The main paper does not report a comparison with independent restarts or refinement without execution feedback under matched inference and execution budgets. The feedback ablations compare report formats but do not establish how much of the overall gain comes from using interaction evidence rather than additional attempts. The trajectory evidence also warrants a narrower interpretation: nine of ten model settings have smaller second-round gains, yet Opus 5 high still improves by 19 points and GPT-6 Sol improves more in the second round than the first. Sol's entire overall gain comes from USB-C, with the other four tasks unchanged. The longer curriculum curves include plateaus, delayed gains, and reversals under changing observation conditions. These results establish useful differences within the tested budgets, but leave unclear how sensitive the model comparisons and bottleneck interpretations are to the selected refinement budget.

  2. The central interpretation about information use is stronger than the feedback controls establish. In Tab.2, skill-level feedback includes non-motion operations such as perception that motion-level feedback omits, so information coverage and semantic organization change together. In Tab.3, part annotations add functional geometry; their benefit cannot be attributed solely to reorganizing information already available. The world-frame/object-frame comparison is more directly relevant, but the complete information equivalence needs to be established. These results support useful feedback interfaces more directly than the central claim that interpreting information is often more limiting than obtaining sufficient information.

Questions for authors
  1. Does Table 1's @k evaluate the policy produced at iteration k, or the policy selected by validation from iterations 0 through k? How does this relate to Fig.10's curves and Tab.4's last-three-iteration averages? Are training layouts reused across rounds, resampled, or accumulated, and which execution outcomes are visible to the agent?

  2. Under the same total inference and execution budgets and validation rule, how does the feedback loop compare with independent policy generation/restarts or refinement without execution outcomes? A comparison on representative tasks would help establish the contribution of feedback use.

  3. For representative settings that improve rapidly, stall, or improve late, what happens when refinement continues under unchanged observations and feedback? Please distinguish current-policy performance from the performance of the policy selected by validation, and report variability across runs for these curves and Tables 1–3. This would help assess whether the reported differences are robust to the stopping budget.

  4. Does the world-frame condition contain the full poses and object geometry needed to derive all quantities in the object-frame report? Can the temporal comparison preserve coverage of perception and other non-motion steps while changing only report organization? This would separate representation effects from added information.

  5. Is there a within-model comparison of skill availability that supports the boundary/scaffold interpretation? The initial unmatched-line fraction alone does not directly measure useful adaptation during later rounds. Do code changes across feedback rounds, or changes in policy graphs across curriculum transfer, support the proposed mechanisms beyond initial line matching and total skill counts?

Reviewer_jD6YOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies self-learning in agentic robotics: a coding agent writes a robot policy as code, runs it in simulation, reads a feedback report, and edits the code. One harness is held fixed and four factors vary. Evaluation covers five VABAR contact-rich tasks and four MimicGen tasks. The paper reports six findings: model capability matters more for repair than for initial generation; stronger models depart from the expert skills while weaker ones reuse them; skill-level feedback beats per-motion feedback; explicit object-relative and part geometry helps; collision-aware motion planning remains a bottleneck; and a privileged-to-perception curriculum can either help or constrain.

Critical strengths
  1. The design is the right way to ask the question. Holding the harness fixed and varying one component at a time is what the literature needs, and the scale is unusual. Very few papers in this area describe their protocol this openly.

  2. Finding 3 has a mechanism and a diagnostic. On Tool Hanging, per-episode feedback reaches 7% held-out success while skill-level feedback reaches 90%, because the observed miss originates in an earlier ring estimate. That is a concrete claim with a checkable cause, and it is the paper's contribution.

Critical weaknesses
  1. There is no statistical inference anywhere. Tables 1–4 report point estimates with no interval, and no test. With five runs over ten held-out instances, the standard error near 50% success is about seven points. The paper uses the same comparative language for both cases. Selecting the best iterate on five validation instances adds selection noise that is never quantified.

  2. The four axes are not varied with equal rigour. The skill axis is observational: Section 5.2 attributes lines by verbatim matching against three sources, and the "new code" share may track verbosity or instruction-following style as much as skill use. I could not find a Tool-Only or no-expert-skill condition, although Figure 2 lists those modes.

Questions for authors

Q1. What happens with K fresh zero-shot samples, best-of-K, and no feedback?

Q2. What are the per-cell n and confidence intervals behind Table 1

Q3. Was a Tool-Only or no-expert-skill condition run? If so, please report it.

Reviewer_qDedOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies design decisions in agentic code-as-policy over a series of isolated ablation-style experiments and aims to draw general conclusions from these experiments.

Critical strengths
  • The paper aims to understand the current systems rather than trying to claim some improvement. That makes it more unusual in structure but leads to extremely useful insights.
  • The experiments span two different benchmarks, and some experiments include an open-weights model.
Critical weaknesses
  • The different experiments significantly differ in scale of the gathered data and conclusiveness of the results. The findings that the authors conclude are in most cases too bold and not fully supported by the presented evidence. This hurts the aim for an in-depth, principled study. In detail:
    • Finding 1: The first sentence "Differences in model capability are less apparent at policy initialization but become pronounced during self-learning." is not correct. THere are also significant differences in zero shot. The conclusion is rather that a competitive zero-shot performance does not necessarily lead to a competitive @2 performance.
    • Finding 2: The first sentence "Expert skills are a boundary for weaker models but a scaffold for stronger ones." is not supported by the data. To proof such a boundary, it would be necesary to test different sizes of initial skill sets and check for a correlation with agent performance.
    • Finding 3 is not supported at all by the data. The level of feedback that is most useful is almost randomly distributed over the tasks. I see also a critical risk that there may be a simple bias that with more feedback, agents always attribute failure to a bit earlier tasks, and then dependent on the task that helps or not.
    • Finding 5: It is not wel defined what would satisfy the requirement of a coherent collision plan. Given that success rates between human and agent policies are about the same, I do not see the claim for a key bottleneck supported.
  • Some of the experiments lack critical details. For Findings 2, 3, 4, 6, it is not specified which agentic system resp. coding agent model is used. To draw general conclusions, it would be important to study these experiments over multiple systems. To aide reproducibility, it would be important to report results with an open-weights model. For the comparison to human-written policies it would be important to specify the prior knowledge of these humans. E.g. for finding 5, it may be that the humans that were recruited to write policies are all robotic engineers who work a lot with collision models and are from their background biased to use them as a tool, and it may not reflect how a web developer would write a robot policy.
Questions for authors
  • Why only two rounds of feedback? For agentic systems, that seems a rather low iteration count.
  • Are "round" and "self-learning iteration" the same? do they refer to different things?
  • How is the compute and inference cost derived? Are these actual costs or are these theoretical numbers if one were to make bad economic choices and pay API price instead of a subscription model, resp. rent instead of buy a GPU?
  • line 263: As I understand it, the figure lists @0 performance, so there is no relation to performance after feedback
  • line 266: The discussion mostly looks at relative share of code lines, while the figure shows absolute. That makes it hard to follow.
  • line 266: is "new task specific code" fromt he text the same as "new skill" in the figure?
  • Where do the human policies come from?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.