acceptodds
Under review as a conference paper at ICLR 2027

Hierarchical Vision Language Action Model Using Success and Failure Demonstrations

Abstract

Vision–Language–Action (VLA) models are typically post-trained on teleoper- ated successful demonstrations, while the many failed attempts that occur natu- rally during data collection are discarded. Those failures record where and how a policy is fragile, and we show they become a usable learning signal once plan evaluation is posed as a reach–avoid problem. We introduce VINE, a hierarchi- cal VLA that separates high-level reasoning (System 2) from low-level control (System 1) under a semi-Markov option formulation. Its central component is a subgoal-level value function that we prove equals the first-exit probability of reaching the goal set before the failure set. Because that quantity is defined by outcomes rather than by expert actions, it can be regressed directly from mixed- quality offline data, with no reward shaping and no online interaction. System 2 uses this value to run a batched tree search over a text-serialized scene-graph abstraction, proposing subgoal transitions, predicting their successor states, and pruning brittle branches before any motion is executed. System 1 then executes the selected subgoal sequence without altering the agent’s motor skills. Across simulated manipulation, the SIMPLER benchmark, and a real 6-DoF arm, VINE improves success most sharply under distribution shift (1.6→over the ω0 baseline on unseen plug insertion and 5.5→on unseen long-horizon real-world packing), suggesting that failure data discarded during collection can improve execution un- der shift.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Hierarchical RL, Mixed Quality Data

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HpsVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces VINE, a hierarchical learning framework that uses both success and failure demonstrations to increase its robustness. Following standard hierarchical planning setup, this system is composed of a high-level reasoning System 2 followed by a low-level continuous controller System 1. System 2 uses a pretrained visual language model to do a Monte Carlo Tree Search over a 2D scene graph. It optimizes the plan (defined as a series of subgoals and their transitions) using a first-exit value function paired with asymmetric expectile loss to avoid overestimation. System 1 uses flow matching policy that is trained only on successful demonstrations to execute the selected subgoals. The authors evaluated this model on simulations (MuJoCo, SIMPLER) and real manipulators and observed better OOD generalization compared to non-hierarchical baselines. The authors claim that failure demonstration improves test-time evaluation of the subgoal feasibilities, which then leads to this better performance.

Note: the reviewer is familiar with VLAs but not SMDP.

Critical strengths
  • Using failure data to improve first-exit reach-avoid probability and use the value for high-level plan search is innovative.
  • In Section 4.3, the low-level System 1 is trained with perturbed data so that it can tolerate upstream errors. This design follows standard practice.
  • Ablation study validates the effectiveness of using failure data.
Critical weaknesses
  1. In Section 3.2, if I understand correctly, the authors make an assumption that all failures are terminal states. This can make the formulation of success v.s. failure into a simple binary value, but could also lead to the framework being unable to handle error recoveries. It is recommended for the authors to discuss how the model handles recoverable situations.

  2. In Section E, by using 2D bounding boxes to form the scene graph, the model might ignore 3D geometry and 3D dynamics, which are key to manipulators.

  3. The proof in Section 3.2 and the appendix needs clarification.

  4. The text descriptions at Line 196 do not match Fig.1. For example, where are the option generator, the learned world model, and the failure-aware value?

Questions for authors
  1. In hierarchical VLAs, the low-level policy typically uses a different (usually simpler) architecture than the high-level one, so as to ensure higher inference frequency. However, in the proposed system, both are based on the same Vision-language backbone. Does this mean that, at inference time, they take turns executing? From Section 4.1, by saying “latency is a one-time planning cost”, do the authors mean System 2 is only run once?

  2. How is the flow-matching controller aligned with the high-level planner? When is the high-level termination predictor queried? What if termination happens while the controller is executing?

  3. Line 157: if the agent enters an avoid set, but the option does not terminate until later at a safe node, will the return <-> indicator relation still hold? The option does not terminate instantly in this case.

  4. Line 794: if you use termination-on-arrival, at t=0 the equation reduces to βe(s)Pπe(s∗∣s)\beta_e(s)P_{\pi_e}(s*|s), which means it checks the termination at the start state, right? Could the authors clarify?

3 comments
AuthorsOpenReview ↗

Response to Reviewer HpsV

W1. Terminal failures and recoverable situations

Failures are terminal in our formulation, in the sense of a task outcome within the prescribed time window, not physical irreversibility. A transition is labeled a failure when it does not reach its target in time, and the episode ends there; the dataset was not collected to include failure-and-recovery trajectories. A situation that needs a different plan is handled before execution, by ranking the corresponding branch low. Runtime motor recovery is outside the scope of this work. We have added this discussion to Section 3.2 and App. K.

W2. 2D scene graphs omit 3D geometry and dynamics

The scene graph is an abstraction for high-level decisions (object identities and coarse spatial relations), not the representation used for control. System 1 generates actions from the current wrist and third-person images, conditioned on the selected subgoal and target graph, so 3D geometry and contact enter through the images. The value also accounts for physical details that the graph omits when those details produced failures in the training data, as in the clock example of Section 5.4. Details that did not cause failures in the data can still affect plan ranking, which App. K lists as a limitation.

W3 / Q3. Entering the avoid set inside an option and terminating later at a safe node

In our formulation, the failure set is absorbing because a failure label ends the episode. A transition is labeled a failure when it does not reach its target within the prescribed time window, and no trajectory in our data continues after a failure label. A transient execution error inside an option, after which the target is still reached within the time window, is therefore not a failure-set event. Under this definition the scenario described by the reviewer does not arise, and the return–indicator identity holds for the decision-epoch process of App. B.2.

Absorption follows from episode termination, not from physical irreversibility. We have replaced the phrase "irrecoverable manipulation state" in Section 3.2 with this definition and defined both sets at decision epochs, so that Section 3.2 and App. B.2 describe the same process.

W4. Text at Line 196 does not match Fig. 1

We thank the reviewer for pointing this out. We have revised the caption of Fig. 1 and Section 4.1 so that each term is mapped to the component shown in the figure:

Term in Section 4.1Component in Fig. 1
Option generatorLM head, generating the edge string
Learned world modelThe same LM head, generating the successor node string given the edge
Failure-aware valueScore head
Intra-option policyAction expert
Termination ruleDone expert

Q1. Shared backbone and inference scheduling

Yes, System 2 runs only once, and the two systems do not alternate in the default configuration. System 2 builds the tree before the first action (27.4 ± 6.0 s at w = 3, Table 9), and only System 1 runs afterwards, at about 0.14 s per action chunk. Both systems use the same frozen language block with different LoRA adapters and heads. System 1 is fast enough because it generates a chunk of actions per forward pass and does not decode text. The 20 Hz figure is the rate at which actions are executed, not the rate of model inference. We have stated this in Section 4.1.

Q2. Alignment of the controller with the planner, and termination

The planner hands System 1 the selected edge and the predicted target graph of that edge; these condition both the action expert and the done expert. The termination predictor is System 1's done expert, not a part of System 2.

The done probability is predicted in the same forward pass that generates an action chunk and is held while the chunk is executed. When it exceeds the threshold, and a minimum number of steps has elapsed for the subgoal, the controller advances to the next edge, and later chunks are generated under that edge and its target graph. If a subgoal completes while a chunk is being executed, the remaining actions of that chunk are still executed, and the switch takes effect from the next chunk. Task success is checked by the environment before every control action. We have aligned Algorithm 2 with this description.

Q4. Termination-on-arrival at Line 794

We thank the reviewer for checking this expression. With the row-stochastic convention P(s, s′) = P_πe(s′ | s) and B = diag(β_e), the corrected expression is

PoeS=∑t≥0(P(I−B))tPBP_{o_e}^{S} = \sum_{t \ge 0} \bigl(P (I - B)\bigr)^{t} P B

An initiated option first takes a transition and then checks termination at the state it reaches, so the t = 0 term is P·B: one transition followed by a check at the arrival state. This corrects the analytical expression in App. B.1 and changes neither the evaluated controller nor any reported number.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer HpsV,

Thank you again for your careful reading and constructive feedback. We would like to follow up on our responses above, which address the definition of terminal failures and the failure set, the corrected option kernel in App. B.1, the correspondence between Section 4.1 and Fig. 1, and the scheduling and termination of the two systems.

As the discussion period draws to a close, we would be grateful to hear whether these responses have addressed your concerns. If any questions remain, we would be happy to provide further clarification.

Best regards, The Authors

Thanks to the authors for the clarifications. Though I incline towards keeping the current score due to that defining the failure set as terminating timeout label (i.e. does not reach in the time window) in the offline dataset rather than a physical state set reduces the SMDP to simplified binary prediction on truncated trajectories. Secondly, system 2 takes 27.4s (Tab.9), this makes the system difficult to resolve initial errors like those from occlusions (unanswered in Q1).

Reviewer_vitLOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents VINE, a hierarchical vision-language-action framework that learns from both successful and failed demonstrations. A high-level System 2 reasons over scene representations, predicts subgoals and their consequences, and uses a learned value function to guide tree search. A low-level System 1 executes the selected subgoals. The framework is evaluated on custom MuJoCo manipulation tasks, SIMPLER, and real-world manipulation tasks. The paper addresses an important problem: how to exploit unsuccessful demonstrations for more reliable robotic manipulation. The reported results support the effectiveness of the complete system. However, the current experiments do not sufficiently isolate the contribution of learning from failures from those of hierarchical decomposition, additional supervision, and test-time search.

Critical strengths
  1. The paper addresses the important problem of learning from failed demonstrations in robotic manipulation. The hierarchical framework provides a coherent way to incorporate failure information into high-level planning. Real-world experiments, including long-horizon tasks, provide useful evidence beyond simulation.
  2. VINE integrates subgoal generation, successor-state prediction, value estimation, and tree search with a low-level VLA controller. This provides a concrete mechanism for using past failures to evaluate alternative plans before execution, particularly for tasks involving ordering constraints, obstacles, and dependencies between subgoals.
  3. VINE integrates subgoal generation, successor-state prediction, value estimation, and tree search with a low-level VLA controller. This provides a concrete mechanism for using past failures to evaluate alternative plans before execution, particularly for tasks involving ordering constraints, obstacles, and dependencies between subgoals.
Critical weaknesses
  1. Failed demonstrations train the high-level planner, while the low-level controller learns only from successful demonstrations. The benefits may therefore depend on whether failures can be avoided through better subgoal selection or ordering; effectiveness on failures requiring improved low-level control remains less clear. This also complicates the comparison with RECAP, which uses outcome information to improve the action policy. The comparison is useful at the system level, but does not by itself establish a more effective general mechanism for learning from failures. The precise RECAP implementation should also be clarified.
  2. VINE combines several design choices, including structured intermediate supervision, subgoal-conditioned execution, successor-state prediction, and value-guided search. It is therefore difficult to determine how much improvement comes from failure data, the specific System 2 learning objective, or the additional structure and computation.
Questions for authors
  1. Can the authors clarify the separate contributions of failed demonstrations and learned value scoring, given that the Tree-to-Full comparison changes both?
  2. How does VINE handle deviations between predicted and observed states during execution? For the supplementary replanning mechanism, how sensitive is performance to the task-specific threshold, and how much additional latency does replanning introduce?
4 comments
AuthorsOpenReview ↗

Response to Reviewer vitL (1/2)

W1. Failure data trains only the planner

Using failure data at the plan level follows from the problem we target. An instruction generally admits several subgoal sequences, and only some of them are feasible in a given scene; an early placement, for example, can obstruct a later one. Failed demonstrations reveal the conditions under which particular subgoal transitions fail, and this outcome information is useful for plan evaluation even when the recorded motions are unsuitable as imitation targets. Supervising subgoal transitions with outcome labels lets the planner exclude a branch before the robot enters it, whereas action-level methods reweight or re-rank motions. System 2 therefore learns from failed demonstrations, and System 1 is trained on successful demonstrations.

The results are consistent with this choice. The gains are largest under distribution shift, and Table A below shows that they come from training on failed demonstrations together with value-based plan selection. In App. J the selected plan is feasible in 188 of 210 trials. Action-level use of failure data, as in RECAP, is complementary and can be applied to System 1. We have stated this division of roles in Section 4.

W1. Comparison with RECAP and its implementation

In our setting, RECAP uses the same backbone and the same mixed-quality offline demonstrations as VINE, so the two methods differ in the level at which outcome information enters: action generation for RECAP and plan selection for VINE. Under this condition VINE reaches 0.422 unseen success on plug insertion against 0.244 for RECAP, and 0.675 against 0.575 on drawer packing (Table 1).

Our baseline applies RECAP's offline stage (value training followed by advantage-conditioned policy training on demonstration data) and omits the iterations on autonomous rollouts. Its advantage threshold, indicator dropout and guidance scale follow RECAP's reported settings; the full configuration is given in our response to yaJB W2/Q1. RECAP reports its largest gains with on-robot data, so we have labeled the baseline as offline-only and corrected "advantage-weighted learning" in Section 5.1 to "advantage conditioning". The two approaches are complementary: RECAP-style training could be applied to System 1.

W2 / Q1. Separating failed demonstrations from learned value scoring

VINE-Tree → VINE-Full changes both the data and the scorer. To separate them, we completed the 2×2 of training data (success-only or mixed) × scorer (LM confidence or value) on both tasks, with the same search and System 1.

Table A. Training data × scorer. "Mixed + LM" uses the VINE-Full checkpoint unchanged and runs the search from scratch guided by LM confidence, without querying the value head.

TaskTraining dataScorerSeenUnseenAll
DrawerSuccess-onlyLM confidence (VINE-Tree)59/8021/4080/120
DrawerSuccess-onlyValue (new)40/8010/4050/120
DrawerMixedLM confidence (new)51/8021/4072/120
DrawerMixedValue (VINE-Full)64/8027/4091/120
PlugSuccess-onlyLM confidence (VINE-Tree)32/4513/4545/90
PlugSuccess-onlyValue (new)30/4515/4545/90
PlugMixedLM confidence (new)31/4521/4552/90
PlugMixedValue (VINE-Full)36/4519/4555/90
  • VINE-Full has the highest overall success on both tasks (91/120 and 55/90).
  • Failure data. A value trained on successes alone does not help (drawer packing 50/120 against 80/120 for LM confidence; plug insertion 45/90 against 45/90). The value is useful when it is trained on failures.
  • Scorer. With the same mixed-data checkpoint, searching with the value gives higher overall success on both tasks: by a large margin on drawer packing (91/120 against 72/120) and a small one on plug insertion (55/90 against 52/90). On unseen plug-insertion configurations the two scorers are within two trials of each other (19/45 against 21/45).

On the effect of structure and computation: the greedy baseline in App. I uses the same trained System 2 at nearly the same planning time (27.6 s against 28.4 s) and reaches 0.311 unseen success on plug insertion against 0.422 with search, so the gain from search is not explained by extra computation.

AuthorsOpenReview ↗

Response to Reviewer vitL (2/2)

Q2. Deviations between predicted and observed states

By default they are not handled at the plan level. System 2 plans once, before the first action, and the tree is not rebuilt when the observed state departs from the predicted successor. System 1 is closed-loop at the level of action chunks: each chunk is generated from the current images, the planned edge and the predicted target graph, and its training perturbs the target-graph coordinates so that small localization errors are tolerated. Larger deviations surface as execution failures. We have made this explicit in Section 4.1.

Q2. Sensitivity to the replanning threshold and added latency

The replanning module in App. L is not part of the main method, and no result in Tables 1–3 uses it. Its threshold is calibrated manually for each task, and with that calibration it improved unseen drawer-packing success from 67.5% to 75.0% (App. L). To measure sensitivity, we varied the threshold, including one set from the training distribution without reference to evaluation outcomes (q95 of the uncertainty statistic on training episodes).

Table B. Replanning across trigger thresholds on unseen drawer packing (40 trials per condition; run separately from Table 1 and App. L, so rates are comparable only across these rows).

ConditionSuccessTrials triggeredEpisode s / trial
No intervention19/40n/a94.8
q9517/40100%103.5
Fixed 0.00218/4087.5%103.6
Fixed 0.00422/4050%95.9
Fixed 0.00620/4037.5%92.7

The module's effect is governed by how often the trigger fires. With selective thresholds (0.004 and 0.006, triggering in 37.5–50% of trials), success reaches 22/40 and 20/40, against 19/40 without intervention. With thresholds that trigger in most trials (q95 and 0.002), success is 17/40 and 18/40: replanning is then applied to nearly every execution, whether or not it is failing. The trigger therefore has to be selective, which is why it is calibrated per task (App. L). We have revised the sentence in App. K on the open-loop gap accordingly.

Latency. Switching to an alternative branch already stored in the tree takes about 0.1 ms, and the time per System 1 call does not change. The added cost appears as longer episodes when the trigger fires often (about 104 s against 95 s per trial at q95), since the robot executes the newly selected branch and failed trials run until the time limit. With selective thresholds the episode length stays close to the no-intervention level (Table B).

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer vitL,

Thank you again for your constructive feedback. We would like to follow up on our responses above, including the new 2×2 ablation that separates training data from the inference scorer (Table A), the clarification of the RECAP baseline, and the analysis of replanning sensitivity and latency (Table B).

As the discussion period draws to a close, we would be grateful to hear whether these responses have addressed your concerns. If any questions remain, we would be happy to provide further clarification.

Best regards, The Authors

Thank you very much for the detailed response and the additional ablation. The new results help clarify the contributions of the failure data and the value scorer. I still remain concerned about the comparison with RECAP. I will maintain my current rating.

Reviewer_yaJBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents VINE, a hierarchical Vision-Language-Action (VLA) framework de- signed to learn from mixed-quality demonstrations. The core idea is to separate high- level planning (System 2) from low-level control (System 1) while sharing a frozen language backbone with LoRA adapters: System 2: Represents the scene as 2D scene graphs and formulates plan evaluation as a reach-avoid problem. Instead of modifying low-level actions, it trains an offline value function on step-level outcome labels, aiming to estimate the probability of reaching the goal before hitting an irrecoverable failure. At test time, it runs a batched MCTS- style tree search to prune risky branches upfront. This planning is open-loop: the full subgoal sequence is generated once at the initial frame without runtime replanning. System 1: Trained solely on successful data, it executes the selected subgoals using a flow-matching action expert and a learned termination head. Across simulated manipulation, the SIMPLER benchmark, and real world 6-DoF arm tasks, VINE demonstrates notable improvements, especially under out-of-distribution shifts (e.g., improving unseen plug insertion from 0.267 to 0.422, and real-world unseen packing from 6.7% to 36.7% over π0).

Critical strengths

S1. Framing failure at the outcome level makes good sense conceptually: evaluating subgoals by whether they reach the goal avoids penalizing valid low-level motor prim- itives. The ablation sequence is particularly clean. Stepping from success-only chains to LM-confidence search, and then to the failure-trained value, clearly shows that the learned value is doing the heavy lifting (+0.133 on unseen plug insertion, +0.150 on unseen drawer packing). Comparing MCTS to a greedy planner under the exact same System 2 (0.422 vs. 0.311 unseen) further confirms that multi-step lookahead over plans matters, with gains concentrating where one would hope: under distribution shift. S2. The paper offers welcome transparency regarding its empirical scope. Rather than hiding edge cases, the authors are upfront about where the method works and where it struggles: plotting human per-path demonstration stats (Figs. 8–10) clearly illus- trates task difficulty, and the DFS/greedy baselines benchmark the search mechanism effectively. I particularly appreciate the candor in the appendices: Appendix G explic- itly notes what the scene-graph correlation cannot prove, Appendix J documents that 51.6% of failures stem from low-level execution rather than planning, and Appendix K articulates the system’s structural limitations. Being this transparent about failure modes adds genuine credibility to the experimental claims.

Critical weaknesses

W1. Overstating theoretical novelty and a disconnect with the actual training ob- jective. Proposition 1 is essentially the classical reach-avoid / first-exit probability identity. Framing this as a core theoretical contribution overlooks closely related prior works that already formalize learned values as success or safety probabilities from offline outcomes—such as Recovery RL, Reach-avoid Bellman RL and SayCan—none of which are cited or discussed. Furthermore, there is a gap between theory and implementation: the value head is trained with an asymmetric expectile regression (τe = 0.7). Its statistical minimizer is an upper expectile, not the conditional expecta- tion / probability Pr(τg < τf). In fact, the paper’s own qualitative figures explicitly state that node values are raw value-head outputs (unclipped) and can exceed 1.0 (e.g., Fig. 4(a), Fig. 14, and Fig. 17), reporting values such as 1.07, 1.140, and 1.185; the abla- tion text itself contrasts a greedy first step of 0.758 with an MCTS-chosen first step of 0.619 followed by subsequent values of 1.185 and 1.140 (App. I). Claiming that Vθ estimates reach-avoid probability is an intuitive heuristic rather than a mathematically sound property of the learned model. W2. Baseline fidelity and compact empirical diversity in primary environments. The two most competitive baselines come with notable drawbacks: Hume was trained from scratch due to missing weights, and π0(∗) .6-RECAP was adapted to static teleoperated demonstrations without autonomous policy rollouts. Since RECAP’s core strength lies in rollout-driven advantage re-weighting, evaluating it purely on static teleoperation data feels like an apples-to-oranges comparison that risks understating what RECAP can actually do. Additionally, while the SlMPLER evaluation is broad, the primary cus- tom manipulation suite remains somewhat compact: only 3–4 unseen configurations per task, 2–3 unseen configs in the real world, and the diagnostic analysis in Appen- dices G and J is based on a modest pool of n = 210 simulation trials (n = 64 failures). As the authors candidly admit, value calibration degrades beyond ~6 subgoal steps, and planning is strictly open-loop (computed once at t = 0), leaving the system sus- ceptible to contact-rich drift unless manually tuned, task-specific replanning heuristics are introduced.

Questions for authors

Q1. Could the authors provide more details on the tuning protocol for Hume and the adapted π0(∗) .6-RECAP? For RECAP in particular, advantage-weighted regression can be sensitive to advantage temperature scaling and weight clipping on static data. Were hyperparameter sweeps conducted on these baselines, and what reassurances can you share that running RECAP purely on static demonstrations without autonomous rollouts does not limit its performance? Q2. Could the authors better contextualize the reach-avoid formulation against prior works that interpret learned values as transition, safety, or goal-reaching probabilities— such as Recovery RL, SayCan and reach-avoid Bellman RL? Beyond embedding the reach-avoid objective into an SMDP graph abstraction with tree search, what is con- ceptually or theoretically new in Proposition 1 compared to these established formu- lations? Q3. About the failure behavior under open-loop drift and heuristic replanning. The search tree is built strictly once at t = 0, while the uncertainty-triggered replanning in Appendix L (raising unseen success from 67.5% to 75.0%) relies on manually calibrated, task-specific variance thresholds. ls there a more principled way to trigger this without per-task tuning? More fundamentally, when execution drifts away from System 2’s predicted successor scene graphs midway through a plan, how does the default open- loop system actually fail?

3 comments
AuthorsOpenReview ↗

Response to Reviewer yaJB (1/2)

W1 / Q2. Novelty of Proposition 1 and related work

We thank the reviewer for the constructive feedback. Proposition 1 is the classical first-exit identity. Its role is to state which task outcome the planner is trained to predict, and we have revised the text to present it as such. We have added to Section 2 a discussion of Recovery RL (Thananjeyan et al., 2021), SayCan (Ahn et al., 2022) and reach–avoid Bellman RL (Hsu et al., 2021). Unlike these works, VINE uses its value to rank plans rather than to provide safety guarantees.

The idea we contribute is that the outcome of a failed transition is a usable training signal for plan evaluation, even when the failed motion is not a suitable imitation target. VINE connects this signal to subgoal transitions over a generated scene-graph abstraction, to successor prediction and to multi-step search inside a hierarchical VLA. The new ablation (Table A, in our response to vitL W2/Q1) is our evidence that this use of failure data matters.

W1. Expectile objective versus probability

With τe = 0.7 the minimizer is an upper expectile rather than the conditional expectation, and the value head has no output squashing or target clipping, so the score is not bounded to [0, 1]. This explains the values above 1 that the reviewer cites.

The expectile is a deliberate design choice: it weights positive TD residuals more heavily, and the ablation confirms its benefit. On drawer packing, unseen success is 0.575 with the symmetric loss (τe = 0.5) against 0.675 at τe = 0.7, and performance is stable for τe in [0.6, 0.8] (Table 11).

We therefore now describe Vθ as an outcome-supervised planning score rather than as a success probability, and have revised Section 4.2 and the captions of Fig. 4 and Fig. 17 accordingly. The search uses the score only to order candidates: on drawer packing, with the same checkpoint, ranking by this score gives 91/120 against 72/120 for ranking by LM confidence (Table A).

W2 / Q1. Hume implementation and tuning

The public release contains pretrained System 2 weights and LIBERO-specific fine-tuned checkpoints, but no general-purpose pretrained System 1. We initialized System 2 from the released weights and trained it on our task data (185K steps on drawer packing, 130K on plug insertion). It was then frozen, and System 1 (with DINOv2-small) was trained on our data for 145K and 105K steps; these are the checkpoints evaluated in Table 1. We tuned the learning rate over five successive runs (trunk learning rate from 1e−7 to 5e−5) and report the last one: 5e−5 (trunk), 1e−5 (actor), 3e−4 (Q-function), 1K warmup steps, batch 16, discount 0.98, chunk lengths 8/16. We have updated Section 5.1 and noted next to Hume's row in Table 1 that training System 1 without general pretrained weights may understate it.

W2 / Q1. RECAP adaptation and tuning

Implementation. Our baseline applies RECAP's offline stage (value training followed by advantage-conditioned policy training on demonstration data) and omits the subsequent iterations on autonomous rollouts. It is advantage conditioning, not weighted regression, so it has no advantage temperature or weight clipping. The value predicts normalized time-to-success in [−1, 0] with 20 bins (201 in the original). The threshold labels 30% of samples positive (12,638 of 42,126) and the indicator is dropped with probability 0.3, both as in RECAP's pre-training setting; we additionally weight the unconditional loss by 0.5. The policy uses the same optimizer and learning rate as VINE's System 1 (AdamW, 2.5e−5 with 1K warmup and cosine decay) for 50K steps at batch size 4. At inference the indicator is fixed to positive with guidance scale 1.0, RECAP's default. The threshold, indicator dropout and guidance scale follow RECAP's reported settings, without further tuning.

Scope of the comparison. RECAP reports its largest gains with on-robot data. We use its offline stage because VINE is also trained without rollouts, so with the same backbone and the same mixed-quality offline demonstrations the two methods differ in the level at which outcome information enters. RECAP's value conditions action generation at the control rate (0.05 s per step). VINE's value selects among subgoal sequences at subgoal transitions (about 5 s each in Fig. 12), so a shallow tree gives look-ahead over the whole task. The two are complementary and could be combined in one system.

We have corrected "advantage-weighted learning" in Section 5.1 to "advantage conditioning", labeled the baseline as offline-only, and qualified the "1.7× and 1.2×" sentence in Section 5.2 with "under the same offline data".

AuthorsOpenReview ↗

Response to Reviewer yaJB (2/2)

W2. Limited unseen configuration diversity

The custom suite has nine unseen configurations on plug insertion (5 trials each), four on drawer packing (10 trials each) and two to three in the real world. SIMPLER adds breadth across tasks and embodiments. We now state the number of configurations and the number of trials per configuration separately.

W2. Sample size of the diagnostic analyses

App. G and App. J rest on 210 trials and 64 failure cases, and we present them as observational diagnostics. The main claims rest on Tables 1–3 and Table A, not on these analyses.

W2. Value calibration beyond about 6 subgoal steps

We did not measure 6 steps as a general limit. At depth 10 the planner placed an executable ordering with 55% training success above one with 65%, while still ranking the 0% orderings last. Fine-grained ranking errors among executable orderings therefore appear at this depth, whereas infeasible orderings remain separated. App. K now describes this as an observation at depth 10.

W2 / Q3. Open-loop planning: how does the default system fail under drift?

Under drift, failures in our trials take two forms. In the first, the planned subgoal is no longer reachable from the actual state, and System 1 continues to pursue it until the time limit; App. J classifies such trials as action failures. In the second, the done expert fires on a state that differs from the predicted successor, and the next edge starts from that state; App. J classifies such trials as planning–action misalignment.

Q3. Is there a more principled trigger for replanning?

We tested the natural candidate: a threshold set from the training distribution (the 95th percentile of the uncertainty statistic on training episodes), without reference to evaluation outcomes. On unseen drawer packing it triggers in every trial, so replanning is applied whether or not an execution is failing, and success is 17/40 against 19/40 without intervention (Table B, in our response to vitL Q2). Selective thresholds, which trigger in 37.5–50% of trials, reach 22/40 and 20/40. A training-distribution threshold therefore triggers too often, and the module relies on task-specific calibration, as in App. L (67.5% → 75.0%). A dedicated failure detector trained on the failed demonstrations is a natural next step. The module is not part of the main method, and no result in Tables 1–3 uses it.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer yaJB,

Thank you again for your detailed and constructive feedback. We would like to follow up on our responses above, which address the presentation of Proposition 1 and its relation to prior work, the description of the learned value as a planning score, the settings of the RECAP and Hume baselines, and the analysis of the replanning trigger.

As the discussion period draws to a close, we would be grateful to hear whether these responses have addressed your concerns. If any questions remain, we would be happy to provide further clarification.

Best regards, The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

Dear reviewers and area chair,

We sincerely thank the reviewers and the area chair for their time and careful reading. We have uploaded a revised paper in which all changes are marked in blue. In response to the reviews, we completed a 2×2 ablation of training data and inference scorer on both simulation tasks, clarified the relation between the reach–avoid objective and the learned score, and specified the baseline settings and the execution protocol. The central claim is unchanged: outcome supervision from failed demonstrations, used at the level of plans, improves plan selection under shift. The main changes are:

  1. New ablation of training data × scorer (Table A, in the response to vitL W2/Q1). VINE-Full has the highest overall success on both tasks, and a value trained without failures does not help on either. With the same checkpoint, searching with the value gives higher overall success on both tasks, by a large margin on drawer packing and a small one on plug insertion.
  2. Formulation. Proposition 1 is presented as a formalization of the objective, and the learned value is described as a planning score. The option kernel in App. B.1 is corrected, the statement that discounting preserves the ranking is replaced by a bound, and the goal and failure sets are defined at decision epochs.
  3. Baselines. The descriptions of RECAP and Hume in Section 5.1 are clarified, and their full settings are given in the responses below.
  4. Replanning (App. L). The module, which is not used in Tables 1–3, is analyzed further across trigger thresholds, including one set from the training distribution (Table B, in the response to vitL Q2).

We respond to each reviewer below.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewers and Area Chair,

Thank you again for your time and for the constructive reviews. We have posted a response to each reviewer and uploaded a revised paper in which all changes are marked in blue.

As the discussion period draws to a close, we would be grateful if the reviewers could let us know whether our responses and the revision have addressed their concerns, and whether any points remain unresolved. We would be happy to provide further clarification on any of them.

Thank you for your continued engagement.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.