acceptodds
Under review as a conference paper at ICLR 2027

BABYSTEPS: Learning Task Meaning Along The Way

Abstract

A robot can change its behavior after failure while retaining a mistaken inter- pretation of the demonstrated task. We study this initial-belief anchoring and introduce BABYSTEPS, which trains a vision-language supervisor to revise task interpretations from demonstrations and execution history while guiding a frozen vision-language-action executor. During offline training, a teacher uses the canon- ical task instruction to write a reflection on the demonstration, initial instruc- tion, and failed rollout. We pair this reflection with the canonical instruction for supervised fine-tuning on student-visible histories, without requiring successful correction rollouts. At deployment, the supervisor generates reflections and re- vised instructions without teacher access. Our Qwen3.5-9B supervisor achieves 77.5% Success@5 on LIBERO tasks held out from supervisor training, exceeding Zero-shot and Instruction-only SFT by 30 and 10 percentage points, respectively. Under controlled incorrect-instruction interventions, both BABYSTEPS and a neutral-reflection control eliminate repairs that express only the implanted task, but BABYSTEPS more often recovers the true task without implanted compo- nents: 20/60 repairs versus 8/60. These results distinguish abandoning an incor- rect interpretation from revising toward the demonstrated task

Primary area:
applications to robotics, autonomy, planning
Keywords:
Imitation Learning, VLM, VLA

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sgyBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies "initial-belief anchoring," in which a VLM supervisor infers a task instruction from a demonstration and retains an incorrect interpretation across retries after execution failure. BABYSTEPS fine-tunes a Qwen3.5-9B supervisor with LoRA on 67 failure histories. Each target pairs a reflection written by a privileged teacher (gpt-5.6-sol, given the canonical instruction) with the canonical instruction itself. A frozen π0-LIBERO policy executes the supervisor's instructions. On 80 held-out LIBERO instances, BABYSTEPS reaches 77.5% Success@5, compared with 47.5% for Zero-shot and 67.5% for Instruction-only SFT. An implanted-instruction probe on LIBERO-Goal reports that BABYSTEPS recovers the true task more often than a neutral-reflection control (20/60 vs. 8/60).

Critical strengths
  • The implanted-history probe (Tables 4 and 9) separates abandoning an incorrect instruction from recovering the demonstrated task, two outcomes that are usually conflated. The finding that the neutral control drops planted components more often (5/60 vs. 12/60 retained) but substitutes a different incorrect goal is informative.
  • The shared-failure-history comparison (Table 2) holds repair inputs fixed across methods, which isolates repair behavior from differences in first-attempt histories.
  • The neutral-reflection control is a sensible ablation of reflection content, and its token-weighting confound is disclosed (Appendix A.5).
  • The 4B supervisor matches the 9B result when trained on 9B-generated histories (Table 1), suggesting the supervision does not depend on the student's own failure distribution.
Critical weaknesses
  1. The premise does not transfer to realistic robotics settings, and the robotics component is shallow. The contribution is primarily about VLM instruction revision. The executor is frozen, was trained on the same LIBERO suites used for evaluation, and accepts only LIBERO's canonical instruction wording, so the task reduces to recovering the correct benchmark instruction string within five deterministic retries. The premise that an execution failure signals an incorrect task interpretation does not hold up in practice. Current language-conditioned policies are far from reliable enough that a failure can be attributed to the instruction rather than to execution. In practice, executor failure dominates, and a supervisor that revises its task interpretation after every failure will abandon correct interpretations as often as incorrect ones. The deterministic reset protocol hides this issue: repeating an instruction cannot succeed, so revision is always rewarded regardless of the cause of failure. The paper does not evaluate cases where the original interpretation was correct and the executor failed, which is the dominant failure mode in deployment. The authors acknowledge (Section 2) that retention of correct interpretations is a teacher-prompt requirement that is not independently evaluated. Training reinforces this bias: every target changes the instruction text, including the 31 "wording" cases where the original interpretation was already correct.

  2. The privileged supervision is understated. The paper repeatedly states that it does not require successful correction rollouts. However, every target is the exact canonical instruction, which the executor was trained on, and canonical-instruction rollouts succeed in 37 of the 38 cases where one is available (Appendix A.2). This is goal-label supervision, a stronger assumption than outcome supervision. The motivation cites unlabeled, in-the-wild demonstrations, where no canonical instruction exists. This should be stated in the abstract and introduction, not only as future work in the conclusion.

  3. The gains may come from matching the executor's instruction wording rather than from revising task meaning. Checkpoints are selected by exact match to the canonical instruction, and a supervisor that learns LIBERO phrasing would raise executor success without any change in task interpretation. Two results support this concern. First, BABYSTEPS improves first-attempt success (55.0 vs. 37.5) despite training only on repair turns. Second, on LIBERO-Goal, the suite where task ambiguity is most relevant, BABYSTEPS does not outperform Instruction-only SFT (9 vs. 10, Table 8), and natural retries yield no recovery (Success@1 to Success@5 is 9 to 9, Appendix C.3). Most of the end-to-end gain comes from LIBERO-Object (17 vs. 9). This null result in the setting most aligned with the paper's motivation is not discussed in the main text. A per-suite breakdown of paraphrase versus semantic revision in natural runs is needed.

  4. The prompt may induce some of the anchoring. Every first-attempt prompt includes the example "put the bowl on the plate," which is the official instruction for libero_goal_008 and the exact incorrect instruction shown in Figure 1. An ablation that removes or varies this example is needed before attributing Zero-shot anchoring to the base model.

  5. The method's novelty is limited, and the baselines do not isolate it. The method combines established components: teacher rationale distillation (Hsieh et al., 2023), verbal reflection across retries (Shinn et al., 2023; Liu et al., 2023b), and a VLM guiding a frozen VLA (Shodiev et al., 2026; Shi et al., 2025). The training procedure is standard LoRA SFT. No comparison isolates what this combination adds. Missing baselines include:

    • the teacher model, or another strong VLM, prompted zero-shot as the supervisor with the same repair prompt;
    • a Reflexion-style prompted baseline on the base Qwen model;
    • VLA Grounder, identified in the related work as the closest architecture;
    • a retrieval or enumeration baseline over the canonical instructions available in each scene, which with K=5 attempts and roughly 10 tasks per LIBERO-Goal scene is a cheap reference point.
  6. The statistical support is thin. The paper uses 67 training examples, 80 evaluation instances from 16 tasks, and a single seed. The 10-point gain over Instruction-only SFT corresponds to 8 episodes, and the task-clustered confidence intervals overlap substantially (Table 7: [62.5, 88.75] vs. [51.25, 82.5]). On TEST alone the gap is 28 vs. 20 episodes. Checkpoint selection relies on 12 validation examples with exact-match counts of (0,0,0,0,3,3,8,4) across epochs. There is no sensitivity analysis over LoRA rank, training set size, or reflection length.

  7. The neutral control is only partly evaluated. It appears only in the diagnostic probe and the shared-history comparison, with no end-to-end Success@k result. The main results therefore cannot attribute the improvement to reflection content rather than to repair-turn training in general. The neutral control is also selected at a different epoch (8 vs. 7).

  8. The semantic annotation is non-blind. Probe labels come from keyword rules followed by Claude review with access to model identity and execution outcomes, plus partial human spot-checks. Since the central claim concerns the direction of revision (20/60 vs. 8/60), blinded human annotation of at least the overridden and disputed categories is needed. The human preference study has near-chance agreement (Cohen's κ = 0.09) and cannot substitute for it, as the authors acknowledge.

Minor Issues

  • Eq. 2 defines A_k, but it is never reported under that name. Table 4's "Planted task only" row appears to correspond to it; label it explicitly.
  • Figure 2 is dense, and the "Learn / Try / Fail / Adapt" illustration adds no information.
  • Recent visual history was chosen for primary reporting after both protocols had been evaluated. This is disclosed, but the drop under cumulative history (77.5 to 73.8) warrants explanation.
Questions for authors

Questions

  1. How does the supervisor behave when the original interpretation is correct and the executor fails? What fraction of correct interpretations are retained?
  2. On natural retries, what fraction of BABYSTEPS revisions change task meaning, and what fraction only rephrase toward canonical LIBERO wording?
  3. How does Zero-shot perform without the "put the bowl on the plate" example in the prompt?
  4. Do the gains persist under stochastic execution between attempts?
  5. How does the teacher model perform when used directly as the supervisor at test time?
Reviewer_VJSTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BABYSTEPS, a method to train a VLM to revise its understanding of a task demonstrated after a failed execution. The paper identifies a problem called initial-belief anchoring, where a model continues to follow an incorrect task interpretation even after observing failed attempts.

VLM is used to generate task instructions for VLA executors. During training an offline teacher generates reflections explaining what should be retained or revised in the initial task interpretation. The teacher is given the correct task instruction along with the demonstration, initial instruction, and failed execution history for generating these. These generated reflections are paired with correct task instruction to finetune the VLM using SFT loss. During inference the VLM generates reflections and revised instructions based on the demonstration and execution history without access to the teacher or correct task instruction.

The method is evaluated on held-out Libero tasks using Qwen3.5-9B and Qwen3.5-4B with a frozen pi0 executor. BABYSTEPS outperforms Zero-shot by 30 percentage points and Instruction-only SFT by 10 percentage points with the 9B model. The paper also shows that BABYSTEPS recovers the intended task more often than a neutral-reflection baseline when given incorrect initial instructions.

Critical strengths
  1. The paper identifies an interesting problem where the model is anchored to a wrong task interpretation after failure.
  2. Only the VLM is trained to revise the task instructions using the generated reflections from the teacher, without requiring correct rollouts or updating the robot policy.
  3. The shared-failure and controlled incorrect-instruction experiments provide evidence that BABYSTEPS can revise its task understanding toward the intended task rather than just changing the instruction
  4. Experiments show improvements in Success@5 over Zero-shot and Instruction-only SFT on held-out LIBERO tasks. BABYSTEPS also recovers the intended task more often than the neutral-reflection baseline in the controlled experiments.
Critical weaknesses
  1. The ablation study does not fully isolate the contribution of teacher-generated reflections, as the Neutral-reflection SFT baseline uses shorter reflection targets. It is unclear how much improvement is from the content of the reflections rather than the neutral-reflection baseline.
  2. The current experiments test recovery from incorrect task interpretations but not failure caused by robot execution.
  3. The method is not tested on other VLM families, robot policies, or benchmarks, which makes it unclear how well it generalizes to different settings.
  4. The improvements are not consistent across LIBERO task suites. BABYSTEPS performs slightly worse than Instruction-only SFT on LIBERO-Goal, achieves the same performance on LIBERO-Spatial, and shows most of its improvement on LIBERO-Object.
Questions for authors
  1. Could you provide a length-matched Neural-reflection SFT baseline to better understand the contribution of teacher generated reflections?
  2. Does the method preserve correct task understanding when the failure is due to execution errors instead of incorrect task instructions?
  3. Could you provide results with different model families or robot policies to show generalization?
  4. Why are the improvements not consistent across LIBERO task suites?
1 comment
Reviewer_VJSTOpenReview ↗

Rebuttal Acknowledgement

I thank the authors for the answers and new experiments. The new results addressed most of my concerns. I will maintain my score and positive assessment.

Reviewer_SwUiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses initial-belief anchoring, where a vision-language supervisor that infers a task from a demonstration keeps its mistaken interpretation even after an execution failure. The paper proposes BABYSTEPS, which fine-tunes the supervisor on teacher-written reflections paired with canonical instructions, so that it revises its instruction for a frozen VLA executor (π0), and evaluates it on LIBERO.

Critical strengths

This paper identifies a practical problem, where a robot keeps its wrong understanding of the task even after failing.

Critical weaknesses

(1) The experiments are not strong enough to support the paper's claim. As discussed in the introduction, the VLM's interpretation can be wrong on unlabeled in-the-wild video data, but there is no experiment on such data; the paper only shows robot experiments in LIBERO simulation, where the demonstrations come from the same environment.

(2) As the reviewer understands, every retry is reset to the same conditions, so repeating the same instruction always fails again. Thus, it is unclear whether the improvement in success rate comes from revising the task interpretation, or from simply changing the wording arbitrarily, especially into expressions familiar to π0.

(3) The paper seems to lack essential comparison baselines. All compared methods are variants built on the same Qwen3.5 backbone, and comparisons with existing failure-reasoning methods (AHA [a], REFLECT [b]), stronger VLMs as the supervisor (e.g., the teacher model used in Appendix A.2 (gpt-5.6-sol)), and other VLA models (π0.5 [c], OpenVLA [d]) are needed.

Questions for authors

Please see the weaknesses section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.