- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
CAPSULE partitions generated robot programs into effect-bounded execution groups, preserves their runtime namespace, and associates execution feedback with source regions. An LLM controller patches groups before physical actions occur or appends recovery code afterward. Evaluation across eight main backbones reports success improvements from 36.7% to 49.7% on Robosuite and 8.2% to 16.8% on LIBERO, with component ablations. CC-GRPO additionally uses independently verified repaired programs to improve standalone generation on two tasks.
Critical strengths
The paper provides practical runtime design for recovery. The design enables targeted correction while retaining useful variables and continuing from the current scene. Broad evidence of system-level utility. Full CAPSULE improves over CaP-X-MT for each of the eight main backbones on both benchmarks, with aggregate gains of 13.0 percentage points on Robosuite and 8.6 on LIBERO. Component ablations further support the usefulness of combining local patching and forward recovery.
Critical weaknesses
Limited demonstrated novelty and insufficient isolation of the proposed mechanism. Relative to CaP-X, CAPSULE appears to be an incremental runtime extension providing intermediate LLM intervention, persistent execution state, and code editing.
The evaluation lacks a stepwise code-generation baseline that executes newly generated blocks in a persistent namespace and receives the same feedback at the same execution boundaries. Such a baseline would test whether maintaining and locally editing a full program provides benefits beyond online next-block generation. Consequently, the evaluation does not establish the value of the specific grouping/patching machinery beyond ordinary stepwise replanning, more frequent feedback, and additional inference.
The execution granularity also seems inconsistently specified throughout the manuscript. Section 2.2 places group boundaries after each effectful region, whereas Appendix A groups separate top-level effectful calls together: Group 1 contains both home_pose() and open_gripper(), and Group 2 contains two goto_pose(...) calls followed by close_gripper(). The exception preserving compound statements does not explain these examples. Despite substantial formal notation, the implemented partition remains unclear. This affects intervention frequency and how much physical execution fits within the 30-step controller budget, limiting reproducibility.
Questions for authors
Which grouping rule is implemented in the experiments? Does Appendix A aggregate groups for presentation, and is the LLM controller queried after every executed group?
How does CAPSULE compare with generating only the next execution group from the current state, while matching namespace persistence, feedback, intervention boundaries, and inference budget? This comparison would directly affect my assessment of the methodological contribution.
3 comments
W3: The execution granularity also seems inconsistently specified throughout the manuscript. Section 2.2 places group boundaries after each effectful region, whereas Appendix A groups separate top-level effectful calls together: Group 1 contains both home_pose() and open_gripper(), and Group 2 contains two goto_pose(...) calls followed by close_gripper(). The exception preserving compound statements does not explain these examples. Despite substantial formal notation, the implemented partition remains unclear. This affects intervention frequency and how much physical execution fits within the 30-step controller budget, limiting reproducibility.
Response to W3:
Thank you for carefully checking the consistency between the grouping rules and the case study. You are correct that the original Appendix A did not match Section 2.2. Its group boundaries were retained from an earlier developmental version and were not updated to reflect the final partitioning rules. We apologize for this inconsistency and have corrected the appendix.
The revised case study follows the effect-bounded grouping rules in Section 2.2 for both the initial program and the appended recovery code. Group 1 contains the import and home_pose(), while Group 2 contains only open_gripper(). Each subsequent top-level motion or gripper call ends a separate group, with preceding observation and computation statements included according to the same rules. The initial program comprises Groups 1–9, and the recovery source is partitioned into Groups 10–18. The compound-statement exception is not needed to explain these boundaries.
We have also made the controller-budget accounting explicit. The illustrated sequence through Step 19 comprises 17 run_group decisions, including the failed attempt and subsequent execution of the patched Group 6, one patch_group decision, and one append_recovery decision. It therefore uses 19 of the 30 available controller steps, provided that no additional inspection or repair decisions are inserted. If task completion is reported after Step 19, execution terminates without an additional finish decision, and Groups 17–18 are not executed. This accounting distinguishes controller decisions from robot API invocations and simulator steps, clarifying how execution granularity relates to the controller budget.
Q1: Which grouping rule is implemented in the experiments? Does Appendix A aggregate groups for presentation, and is the LLM controller queried after every executed group?
Response to Q1:
All experiments use the deterministic grouping procedure specified in Section 2.2. This procedure inserts a boundary immediately after each region with , preserves compound statements through control-flow atomicity, and applies dependency-aware closure without crossing effect boundaries.
The original Appendix A example used an outdated developmental fragment that did not match this procedure. We have corrected the case study to follow Section 2.2. The revised Appendix A shows individual execution groups without aggregating them for presentation, including those partitioned from the appended recovery code.
The LLM controller is queried after each executed group unless the runtime has terminated. After each run_group, source-local feedback is returned, and the controller selects the next action if execution continues. When post-action feedback indicates task completion, the runtime exits without another controller query. The illustrated sequence in Appendix A uses 19 of the 30 available controller steps, comprising 17 run_group decisions, one patch_group decision, and one append_recovery decision, provided that no additional inspection or repair decisions are inserted.
Q2: How does CAPSULE compare with generating only the next execution group from the current state, while matching namespace persistence, feedback, intervention boundaries, and inference budget? This comparison would directly affect my assessment of the methodological contribution.
Response to Q2:
We agree that the proposed comparison would help determine whether maintaining and locally editing a full program provides benefits beyond generating only the next execution group under matched conditions.
CAPSULE is deliberately built on the standard Code-as-Policy paradigm used by CaP and CaP-X, where the LLM first synthesizes a complete program. Our contribution is a runtime layer that supports effect-aware local editing and recovery during physical execution. Our comparison with the CaP-X multi-turn baseline evaluates this runtime mechanism against full-program regeneration.
The suggested baseline uses a different generation strategy, but it is a relevant comparison for assessing the value of maintaining and editing a full program. Matching namespace persistence, feedback, intervention boundaries, and inference budget would help distinguish the benefits of CAPSULE’s editing and recovery mechanism from those of stepwise generation with intermediate feedback. Our current evaluation does not establish an advantage over this matched next-group generation baseline. Our claim is therefore limited to the benefits demonstrated relative to the evaluated full-program baselines.
We sincerely appreciate your detailed feedback.
Response to W1:
We acknowledge that CAPSULE builds upon the CaP paradigm. Its contribution, however, is not merely the addition of intermediate LLM intervention, persistent state, or code editing. The stronger frozen controller is used during training to generate repair candidates for both CC-GRPO and GRPO + BC, and is disabled during evaluation. Before physical effects occur, faulty code can be patched in place while preserving the active namespace. Once an effectful API invocation has been recorded, the executed group cannot be patched or replayed, and recovery must proceed through newly appended continuation code. This distinction is absent from the CaP-X multi-turn baseline and is central to our design.
Our existing ablations (Fig. 3) further isolate the two mechanisms by separately evaluating local patching and forward recovery. Full CAPSULE (A4) improves success rates by 16.9 percentage points on Robosuite and 6.5 points on LIBERO relative to the recovery-only control (A2). Enabling forward recovery over the patching-only control (A3) similarly yields gains of 15.4 and 8.8 points, respectively. These results support the complementary contributions of the two recovery mechanisms.
Response to W2:
The suggested stepwise baseline would help determine whether maintaining and locally editing a full program provides benefits over online next-block generation under matched feedback conditions. Our current ablations isolate the roles of patching and recovery within the full-program setting, but do not establish an advantage over this stepwise alternative.
CAPSULE is built on top of the full-program generation paradigm, where the LLM first produces a complete program for the task. Our contribution lies in how to make this full program more robust and editable at runtime. We partition the generated program into groups, maintain persistent execution state, and perform local patching or forward recovery based on intermediate feedback. The suggested baseline changes how the program is generated, so it addresses a broader comparison than an ablation within this setting. Nevertheless, it is relevant to assessing whether the observed benefits require maintaining and editing a full program.
To examine the contributions of patching and recovery within the full-program setting, we provide the ablations in Section 3.2.2 and Figure 3.
append_recoveryenabled andpatch_groupdisabledpatch_groupenabled andappend_recoverydisabledAdding forward recovery to the patching-only configuration (A3 → A4) improves success by 15.4 percentage points on Robosuite and 8.8 percentage points on LIBERO. Adding local patching to the recovery-only configuration (A2 → A4) improves success by 16.9 and 6.5 percentage points, respectively. These results support the contribution of each mechanism when combined with the other within the full-program setting. They do not, however, separate the benefits of maintaining and editing a full program from those of stepwise generation with matched feedback boundaries and inference budgets. Our experimental conclusions are therefore limited to the evaluated full-program configurations.