acceptodds
Under review as a conference paper at ICLR 2027

Replan for Error Recovery: Training Embodied VLMs through Verifiable Interaction with Action-Grouped Turn-GSPO

Abstract

Embodied VLMs can execute increasingly complex manipulation tasks, yet execution errors can leave the environment in states that require replanning for recovery. Existing recovery methods either rely on inference-time mechanisms with fixed policies or learn from failure examples and reference corrections rather than verified action outcomes. To enable recovery learning through verifiable interaction, we introduce the Embodied Replanning for Recovery Environment (ERR-Env), which provides feedback on task progress and execution errors. We further construct ERR-Bench, a benchmark with held-out tasks spanning nominal, intermediate, and error states for evaluating recovery capability and execution reliability. We then propose Action-Grouped Turn-GSPO (AGT-GSPO), an online reinforcement learning method combining turn-level rewards for task progress and error states with action-grouped advantage estimation, and use it to train ERR-VLM, a compact 2B embodied VLM. Evaluation on ERR-Bench reveals a mismatch between task progress and execution reliability across diverse VLMs. AGT-GSPO improves ERR-VLM by 10.25 percentage points in Strict Success Rate for error-free completion and 22.95 percentage points in Recovery Rate, while ablations and zero-shot transfer further validate the effectiveness of our method.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied VLM Agent, Error recovery, Reinforcement Learning, Interaction Environment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FoCvOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents an approach to easily recover from states where an error occurred to find a successful trajectory for VLMs. To achieve this, the paper first presents an environment that captures the progress, success, visibility, and penalty of different states within a trajectory. The paper utilizes this environment to collect data to construct the ERR benchmark, which is used to train and evaluate a 2B sized VLM. The experiments show the corresponding trained model has superior recovery rate compared to existing baselines.

Critical strengths
  1. The paper presents a useful dataset focused on recovery from error states within an environment. The environment and the corresponding benchmark could be useful in other applications than the one presented in the paper.
  2. The paper shows that training on the proposed dataset is helpful in improving the recovery rate.
Critical weaknesses
  1. The paper is not well presented in its current format and several parts within it are difficult to follow. The paper also uses several terminology that is not prevalent in the literature and difficult to understand. Please find the details below:

    • Line 162: What is the “pre-defined interface” for task-specific checker.
    • Line 163: What does “task-relevant object relations” mean?
    • Line 176: What does “language-level semantic-action sequence” mean?
    • Eq. 8, what does μinv,μdeg,μimp\mu_{inv}, \mu_{deg}, \mu_{imp} mean? What is the meaning of “small signed thresholds”.
  2. The equations and text around Eq. 7 is very hard to parse. What does rollout group Oi\mathcal O_i mean? Is Gt\mathcal G_t a single set or two? If it’s two sets, then how are the mean and std. defined? If it’s a single set, why are we splitting by the action type?

  3. The overall contribution of the work is ambiguous. Is the primary contribution of the paper the dataset? Apart from that the turn-level rewards have been already explored in the RL literature. Is the reward computation using the task-specific checker supposed to be a key contribution? Details of the task-specific checker are not specified in the main paper. How is it designed? Is it well calibrated with the ground truth rewards? There should be separately analysis for that.

  4. The experiment results are a bit confusing. Many open- and closed-source baselines outperform the proposed method in terms of the success rate metrics. Is the recovery rate improvement the main focus of these results? Could the gains in recovery rate be simply because it was trained on data from the same environment?

  5. Many simple baselines are missing from the experimentation section. What are the results when using other popular online RL methods on the VLM like GRPO, PPO, GSPO, etc. without reward shaping? How does the performance change when we only provide outcome-level rewards?

  6. The task-specific checker provides dense feedback in this environment, which is not easily available on other environments. This could be another reason why the proposed method shows improvement on this dataset. It would be interesting to see results using baselines like PPO or REINFORCE using these dense rewards.

Questions for authors

Please respond to the weaknesses above.

3 comments

We thank the reviewer for recognizing the value of ERR-Env/ERR-Bench and the improvements in recovery performance. We appreciate the detailed feedback and provide the following clarifications.

W1. We appreciate the feedback on presentation and clarify the specific terminology and notation below:

  • Line 162, “pre-defined interface”: This refers to the task-construction/checker API detailed in Appendix A.1.2, which provides operations such as add_actor, pick_and_place, check_grasp, check_on, and count_visible_objects. The checker evaluates simulator states and returns success, progress, penalty, and visibility, which are used for reward computation and evaluation but are never exposed to the policy. We will include a concise API example in the main text.

  • Line 163, “task-relevant object relations”: These are executable predicates describing goal-specific object relationships and manipulation states. For example, the checker in Appendix A.1.2 verifies whether a Rubik's cube is placed in a wooden box and a knife in a steamer. Correct intermediate grasps contribute partial progress, while incorrect placements or irrelevant grasps incur penalties. We will clarify this terminology with concrete examples.

  • Line 176, “language-level semantic-action sequence”: This refers to a symbolic reference plan consisting of named objects and high-level actions, such as pick object_1; place container_1. It is used during task generation to construct and validate executable trajectories and is never provided to the policy during training rollouts or evaluation.

  • Eq. 8, “small signed thresholds”: As specified in Appendix A.2, the thresholds are μinv=−0.02\mu_{\mathrm{inv}}=-0.02, μdeg=−0.01\mu_{\mathrm{deg}}=-0.01, and μimp=0.02\mu_{\mathrm{imp}}=0.02. They respectively constrain the advantages of invalid, non-improving, and improving pick/place actions, suppressing misleading positive advantages for erroneous actions while ensuring positive feedback for verified improvements. We will make these values and their roles explicit alongside Eq. 8.

  • Eq. 7, rollout group and comparison set: For each training prompt or initial snapshot, six sampled interactive rollouts form a rollout group OiO_i, while the update batch BB contains rollout groups from eight prompts. For each turn tt, GtG_t is a single comparison set selected according to the action type: same-type pick/place turns within OiO_i, or same-type home/done turns across BB, since the latter occur less frequently. The mean and standard deviation are computed exclusively over the rewards in the selected GtG_t. We will clarify this piecewise definition and distinguish the rollout group, update batch, and comparison set more explicitly.

W2. The motivation behind Eq. 7 is that different high-level actions have distinct reward semantics and scales. Specifically, pick/place rewards depend on changes in task progress and penalties, home rewards visibility recovery, and done rewards successful termination. Directly comparing these heterogeneous rewards may produce misleading relative advantages. Our method therefore groups turns by action type, using the same initial snapshot for pick/place to preserve task-context comparability and the update batch for less frequent home/done actions to ensure sufficient comparison samples.

We empirically validate this design through the grouping ablations in Table 2. Under the same SFT initialization and RL configuration, our full method achieves 20.89% SSR and 22.16% Recovery SSR, compared with 19.83%/18.16% for Batch (All), 9.25%/9.18% for Sample (All), and 11.11%/8.78% for Batch (Action). These comparisons demonstrate that both action-type grouping and shared initial-task context contribute to reliable recovery. We acknowledge that turns from the same rollout group may still encounter different intermediate states; the grouping is designed to improve contextual comparability rather than guarantee identical states.

W3. Our primary contribution is a verifiable interaction framework for training and evaluating embodied error recovery, consisting of ERR-Env, ERR-Bench, and AGT-GSPO. ERR-Env provides restorable states and executable verification of task progress and errors, while ERR-Bench enables controlled evaluation across nominal, intermediate, and error states. AGT-GSPO builds on these components to optimize recovery decisions through verified interaction outcomes. We do not claim turn-level rewards or group-relative optimization as independently novel; instead, our methodological contribution is their recovery-oriented integration with action-grouped advantage estimation and semantic advantage shaping.

Regarding checker design, Appendix A.1.2 provides a concrete implementation based on executable simulator-state predicates rather than learned reward estimation. The checker evaluates goal completion, partial progress, erroneous manipulations, and object visibility. Generated tasks undergo automatic execution validation, and randomly sampled task programs are additionally inspected to verify consistency between task instructions, action semantics, and checker logic. The checker thus provides programmatically verifiable task-state feedback rather than subjective reward predictions.

The importance of these signals is further supported by the reward ablations in Table 2. Removing action reward reduces SSR from 20.89% to 10.05%, while removing penalty feedback reduces SSR to 2.80%, despite maintaining relatively high PG. This demonstrates that explicit error feedback is essential for distinguishing meaningful task progress from unreliable execution. Although these results establish the utility of checker-based rewards, they do not constitute an independent quantitative calibration study against human-labeled states. We will clarify the checker design and existing validation procedure in the main text.

W4. Our main objective is to improve recovery capability and reliable task completion, rather than outperform larger general-purpose models across every metric. Although some closed-source models achieve higher overall SR, ERR-VLM (Full) achieves the highest aggregate Recovery Rate of 65.47% and outperforms all evaluated open-source and embodied baselines in overall SR and SSR.

Regarding the effect of in-distribution training, SFT and RL data originate from the same ERR-Env task-generation pipeline and follow the same underlying task distribution, covering nominal, intermediate, and recovery states with disjoint training instances. The SFT policy has already learned from reference recovery corrections, while RL further optimizes the policy through verified outcomes of its own actions. Compared with SFT, full RL improves Recovery Rate from 42.52% to 65.47%, Recovery SSR from 8.58% to 22.16%, and SSR from 10.64% to 20.89%.

Moreover, the recovery-data ablation in Table 2 shows that removing recovery snapshots from RL reduces Recovery Rate to 39.12% and Recovery SSR to 11.57%, while PG remains relatively similar (69.14% vs. 71.08%). This demonstrates that nominal interaction alone is insufficient for effective recovery learning. ERR-Bench also uses exclusively held-out tasks unseen during SFT and RL. We further evaluate zero-shot transfer on RMBench, RoboVoLo, and physical robots without additional training, where the full model consistently improves task progress over its SFT initialization. These results support the effectiveness of recovery-oriented learning beyond memorizing training trajectories.

W5. We clarify that several commonly used online RL methods are already evaluated in Table 2, including Turn-GRPO, Trajectory-GSPO, and Trajectory-GRPO. These baselines differ in their optimization granularity and reward assignment. Specifically, Turn-GRPO applies GRPO with turn-level rewards and turn-level policy updates, replacing the sequence-level importance weighting of our Turn-GSPO formulation with token-level GRPO optimization. In contrast, Trajectory-GSPO and Trajectory-GRPO use only outcome-level rewards, assigning a single reward and advantage to the entire trajectory and applying GSPO or GRPO at the trajectory level. All these comparisons use the same SFT initialization and training budget, allowing us to evaluate the effects of optimization strategy and credit-assignment granularity.

The results demonstrate substantial differences in recovery reliability. Turn-GRPO achieves 8.85% SSR and 11.18% Recovery SSR, compared with 20.89% and 22.16% for our full method. Trajectory-GSPO and Trajectory-GRPO achieve Recovery SSR values of only 5.59% and 4.99%, respectively. These results highlight the importance of assigning feedback to individual interaction turns rather than using a single trajectory-level signal, while also demonstrating the benefits of sequence-level GSPO optimization over token-level GRPO in our setting.

We further provide additional algorithm and reward comparisons in Appendix A.5.3 (Table 11). Specifically, Trajectory-GSPO (Outcome) and Trajectory-GRPO (Outcome) correspond to the outcome-only trajectory baselines reported in Table 2. We additionally evaluate Trajectory-GSPO (Sum-of-Turn) and Trajectory-GSPO (Avg-of-Turn), which aggregate dense turn-level rewards into a single trajectory-level reward through summation and averaging, respectively. The four variants achieve Recovery SSR values of 5.59%, 4.99%, 4.99%, and 7.39%, respectively, compared with 22.16% for our full method. Even when dense rewards are available, aggregating them into a single trajectory-level signal removes their temporal specificity and weakens credit assignment for individual corrective actions.

Together, these experiments systematically compare turn-level GRPO, trajectory-level GRPO/GSPO with outcome-only rewards, and trajectory-level GSPO with aggregated dense rewards. They support the effectiveness of preserving fine-grained turn-level feedback and optimizing each reasoning-and-action response as a coherent sequence. We will make these existing comparisons more prominent in the main paper.

W6. We agree that dense checker feedback is an important factor in recovery learning, and providing such verifiable feedback is itself a central motivation for ERR-Env. Unlike environments that provide only final task success, ERR-Env explicitly evaluates intermediate progress, erroneous state changes, execution validity, and visibility recovery, allowing the policy to receive feedback on individual corrective decisions.

We investigate the contribution of these signals through controlled reward ablations in Table 2. Removing grounding, action, penalty, and outcome feedback reduces Recovery SSR from 22.16% to 19.96%, 12.57%, 7.78%, and 14.37%, respectively. Notably, removing penalty feedback causes a substantial decrease in SSR from 20.89% to 2.80%, indicating that progress-based rewards alone can reinforce actions that advance the task while introducing execution errors.

Furthermore, the outcome-only and dense-reward aggregation comparisons in Appendix A.5.3 (Table 11) provide additional evidence concerning the interaction between reward design and credit assignment. These results show that dense feedback alone is insufficient when its temporal information is collapsed into a single trajectory-level reward. The effectiveness of our framework comes from combining explicit error-aware feedback with turn-level optimization, enabling the model to distinguish corrective actions from error-inducing ones. Although PPO and REINFORCE with identical dense rewards are not included, our existing GRPO/GSPO comparisons and reward ablations provide systematic evidence for the contributions of reward design and optimization granularity.

We thank the reviewer again for the constructive feedback and hope these clarifications better communicate our methodology, experimental comparisons, and contributions.

Reviewer_MQL4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets learning recovery from execution errors rather than imitating reference corrections. It contributes (i) ERR-Env, a verifiable environment on RoboTwin 2.0 with task-specific checkers returning success/progress/penalty/visibility; (ii) ERR-Bench, 1503 held-out snapshots spanning nominal, intermediate, and error states across three recovery types (Manipulation/Occlusion/Missing-Target Recovery); and (iii) AGT-GSPO, an online RL method with turn-level rewards (grounding, action, penalty, outcome) and action-grouped advantage estimation, training a 2B ERR-VLM initialized from Qwen3-VL-2B-Thinking. On ERR-Bench, full training raises SSR from 10.64 to 20.89 and Recovery Rate from 42.52 to 65.47 over SFT; zero-shot transfer to RMBench/RoboVoLo and a small Franka/AIRBOT real-world experiment is reported.

Critical strengths
  1. Using executable checkers to supply turn-level verified outcomes (rather than imitating a human correction) is a sensible direction; the recovery-data ablation (w/o recovery data drops Recovery Rate 65.47 to 39.12 while PG stays similar, Table 2) isolates that nominal rollouts alone do not teach recovery.
  2. Trajectory-GSPO/GRPO collapse (Recovery SSR 5.59/4.99 vs. 22.16, Table 2), supporting the turn-level claim; grouping ablations (Batch-All / Sample-All / Batch-Action) show both action-type and sample-context comparability matter.
Critical weaknesses
  1. The action space is a tiny discrete high-level interface (pick / place / home / done with a 2D point), not continuous control.
  2. Training scale and evidence of the RL gain are very limited.
  3. Baseline fairness and reproducibility are unclear.
  4. Real-world transfer is underpowered.
Questions for authors
  1. Please report mean ± std over multiple seeds and statistical tests for Table 1/2, and disclose API versions, temperatures, and number-of-samples for every closed-source model.
  2. Why only 200 RL steps and one 2B size — does the gain persist with more updates and larger models, or plateau?
1 comment

We thank the reviewer for recognizing the value of verifiable turn-level feedback, recovery-specific training data, and the effectiveness of turn-level optimization. We appreciate the constructive feedback and address the concerns below.

W1. Our work focuses on high-level recovery-aware planning rather than continuous low-level control. The compact action space (pick/place/home/done) is deliberately designed to isolate the model's ability to recognize execution errors, select corrective actions, and replan for task completion. For pick/place, the model additionally predicts a 2D image-space point, which is grounded into a 3D target and executed through robot motion primitives. This design enables controlled evaluation of recovery decisions while reducing interference from low-level control errors. We also discuss this scope in Section 5.

W2. Although our experiments focus on a 2B model, the training involves substantially more interaction than 200 individual actions. Each of the 200 RL updates uses 8 prompts with 6 interactive rollouts per prompt, yielding approximately 9,600 multi-turn trajectories. Despite this relatively limited optimization budget, RL improves SSR from 10.64% to 20.89%, Recovery Rate from 42.52% to 65.47%, and Recovery SSR from 8.58% to 22.16%. The 2B setting allows us to systematically investigate recovery learning and its components under controlled computational resources.

W3. To ensure consistent baseline evaluation, all models use the same system prompt, tool definitions, tool-call format, and 500-token thinking budget. This standardized protocol avoids introducing unequal inference-time reasoning budgets across models. Furthermore, Appendix A.3 provides detailed SFT and RL configurations, including optimization hyperparameters, rollout settings, and computational resources, while Appendix A.5.2 analyzes tool-call formatting errors and their execution consequences. We will further clarify the inference configurations of closed-source models to improve reproducibility.

W4. Our real-world experiments evaluate whether recovery behaviors learned in simulation transfer to physical execution without additional training. We test five task settings covering long-horizon manipulation and recovery, with 10 trials per task. Beyond PG and NPR, we introduce Active Re-observation Rate (ARR) to directly evaluate recovery behavior following occlusion. The full ERR-VLM invokes home in 37 of 47 occlusion events (78.7%), compared with 0 of 50 (0.0%) for SFT and 4 of 45 (8.9%) for RoboBrain2.5-8B. Together with consistent PG improvements across all five tasks, these results demonstrate the transfer of learned planning and recovery behaviors to physical execution.

Q1. We acknowledge the importance of multiple training seeds and statistical significance tests. However, the computational cost of online RL with multi-turn environment interactions is substantial, and our available training resources do not permit repeated full training runs across multiple seeds for all methods and ablations. We therefore evaluate all methods on the same 1,503 held-out snapshots to maintain a consistent evaluation protocol. Regarding reproducibility, Appendix A.3 provides detailed training configurations, and all baseline models follow the same inference protocol. We will further specify the exact API versions, decoding temperatures, and sampling configurations of closed-source models.

Q2. We clarify that 200 RL steps correspond to 200 optimization updates rather than individual environment interactions, involving approximately 9,600 multi-turn trajectories. In our training observations, performance had largely stabilized by 200 updates, and evaluating later checkpoints did not yield noticeable further improvements. Given the substantial computational cost of online interaction, we therefore adopt 200 updates consistently across all ablation experiments, ensuring controlled comparisons under the same training budget. Training requires approximately 48 hours using two NVIDIA A800 GPUs for policy optimization and four RTX 3090 GPUs for parallel environment interaction. Due to limited computational resources, we focus on the 2B model to comprehensively evaluate the contributions of recovery data, reward design, action grouping, and optimization strategies.

We thank the reviewer again for the constructive feedback and hope these clarifications better communicate our experimental design, evaluation protocol, and contributions.

Reviewer_zXRfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a framework that trains and evaluates a VLM-based high-level planner to replan and recover from execution errors through verifiable interaction. The planner outputs pick, place, home, and done tool calls together with 2D points. By applying Action-Grouped Turn-GSPO to Qwen3-VL in ERR-Env, the paper achieves large gains in SSR and recovery rate over SFT.

Critical strengths
  • Clear evaluation protocol. By starting episodes from initial, intermediate, and error snapshots, the benchmark separately measures full-task execution, continued execution, and recovery. SSR and Recovery SSR also reveal the mismatch between progress and reliability.
  • Consistent improvement. Applying the proposed method consistently improves SSR, recovery rate, and recovery SSR, and the gains hold for both short- and long-horizon episodes.
  • Reusable infra. The restorable snapshots and checker-based dense feedback in the paper support both online RL and controlled evaluation.
Critical weaknesses
  • Novelty issue. Most components are combinations of existing techniques. Turn-level rewards already appear in VAGEN [1] and ERA [2], and critic-free group-relative optimization is not new either [3, 4]. The new part is the action-grouped advantage estimation, which normalizes rewards within each action type, and it is hard to regard this as a major algorithmic contribution. Specifically, μG\mu_G and σG\sigma_G depend on the action type of the sampled response, and an action-dependent baseline generally biases the policy gradient. As a result, the direct comparison of which action to choose disappears. Since turns from different steps and states are compared even within the same type, the quality of an action also becomes entangled with the value of the state.
  • Lack of a matched control. The motivation of the paper is the gap between imitating reference corrections and optimizing against verified outcomes, yet the only comparison is SFT versus SFT+RL. A control with the same data and compute is therefore needed. For example, one could continue SFT on reference corrections for the RL snapshots, or apply rejection sampling fine-tuning that trains only on successful rollouts filtered by the checker.
  • Fairness of Baseline comparison. ERR-VLM is an in-distribution model, whereas the other models are evaluated zero-shot. The 500-token thinking budget applied to all models may disadvantage large reasoning models. Moreover, the claim that recovery capabilities ''do not naturally emerge at this scale'', which rests on the gap to the base model, is confounded by the effect of tool-call format compliance.
  • Transfer metrics inconsistent with the main claim. The paper presents as one of its key findings that high task progress does not guarantee reliable execution. However, the zero-shot transfer experiments report only Progress in Table 3 and only PG and NPR in Table 4, with no SR or SSR. With 10 trials per task, the sample size is also small. In addition, RoboBrain2.5-8B is the only external baseline.

[1] VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents (Wang et al., NeurIPS 2025)

[2] ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning (Chen et al., ICLR 2026)

[3] Group Sequence Policy Optimization (Zheng et al., arXiv 2025)

[4] Group-in-Group Policy Optimization for LLM Agent Training (Feng et al., NeurIPS 2025)

Questions for authors

See above. As a minor fix, please correct the broken table references in A.5.2.

2 comments

We thank the reviewer for recognizing our evaluation protocol, improvements in recovery and execution reliability, and reusable infrastructure. We address the concerns below.

W1. Novelty issue: We acknowledge that turn-level rewards and group-relative optimization have been explored in prior work. Our contribution integrates verifiable recovery interaction with action-specific feedback and turn-level optimization. In AGT-GSPO, actions have distinct reward semantics: pick/place use task progress and penalties, home uses visibility recovery, and done uses successful termination. To avoid comparing heterogeneous reward scales, we group pick/place turns by action type within the same initial-snapshot rollout group, while less frequent home/done turns are grouped across the batch.

Under identical SFT initialization and RL settings, our method achieves 20.89% SSR and 22.16% Recovery SSR, versus 19.83%/18.16% for Batch (All), 9.25%/9.18% for Sample (All), and 11.11%/8.78% for Batch (Action). These ablations support the benefit of action-type and initial-context comparability. We acknowledge that action-conditioned normalization is not generally an unbiased policy-gradient estimator and that grouped turns may visit different states. Our method instead employs action-conditioned relative optimization with semantic advantage shaping to suppress invalid or non-improving actions and reinforce verified improvements.

W2. Lack of a matched control: SFT and RL data originate from the same ERR-Env task-generation pipeline and underlying distribution, covering nominal, intermediate, and recovery states with disjoint training instances. SFT already learns from reference recovery corrections, while RL further optimizes decisions through verified outcomes of the policy's own interactions rather than introducing a new recovery data source.

Table 2 shows that removing recovery data from RL reduces Recovery Rate from 65.47% to 39.12% and Recovery SSR from 22.16% to 11.57%, while PG remains similar (71.08% vs. 69.14%). This highlights the importance of interacting with error states rather than nominal states alone. Compared with recovery-aware SFT, full RL improves SSR from 10.64% to 20.89% and Recovery Rate from 42.52% to 65.47%, supporting the benefit of verifiable online optimization. A matched continued-SFT baseline would further isolate the RL objective's contribution.

W3. Fairness of Baseline comparison: We use a standardized inference-budget protocol with identical system prompts, tool definitions, tool-call formats, and a 500-token thinking budget. Allowing larger models additional thinking tokens would introduce unequal inference resources, especially given their higher computational cost per token. Our protocol therefore compares models under a consistent reasoning-token budget rather than individually optimized inference settings.

Appendix A.5.2 (Tables 9–10) analyzes tool-call errors and their trajectory-level consequences, showing that formatting errors do not necessarily prevent subsequent valid execution. Moreover, our Base, SFT, and Full models share the same 2B architecture and inference settings, directly isolating the effects of recovery-oriented training. Their improvements support our finding that recovery capabilities do not naturally emerge at this scale without targeted learning.

W4. Transfer metrics inconsistent with the main claim: Our transfer metrics reflect each environment's evaluation capabilities. In Table 3, we use Progress (PG), the native metric of RMBench and RoboVoLo. For multi-stage tasks, PG provides finer-grained discrimination than binary SR by capturing partial completion. These external benchmarks also lack ERR-Env's unified error-specific checkers, preventing directly comparable NPR and SSR measurements.

In Table 4, we combine PG, NPR, and ARR to evaluate task completion, error-free execution, and active re-observation after occlusion, respectively. Thus, real-world evaluation includes both completion and error-related metrics. ERR-VLM improves PG over SFT across all transfer tasks, while ARR increases from 0.0% to 78.7%, demonstrating improved active recovery. These results complement ERR-Bench's comprehensive reliability evaluation. Additional success metrics, trials, and external baselines would further strengthen the evidence.

Q1. Thank you for identifying the broken references in Appendix A.5.2. We will correct them to Tables 9 and 10, respectively.

We thank the reviewer again for the constructive feedback.

Reviewer_zXRfOpenReview ↗

Response to Rebuttal

Thank you for your response. However, the rebuttal restates existing results and adds no new evidence for the concerns I raised.

The central claim of this paper is that optimizing against verified outcomes works better than imitating reference corrections. This still lacks a matched control that separates the RL objective from the additional data and compute. Furthermore, the transfer evaluation still does not measure reliable completion, which is the paper's key message. Success rates appear to be obtainable from the existing evaluations.

Therefore, I will keep my current rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.