acceptodds
Under review as a conference paper at ICLR 2027

SCOPE: Selective Certification of Program Edits for Embodied Heuristic Learning

Abstract

Embodied heuristic learning must balance strategic adaptation with the preservation of previously successful behavior. Local repairs may leave structural failures unresolved, whereas broader revisions can introduce regressions that are costly to detect through repeated environment interaction. We introduce SCOPE (Selective Certification Of Program Edits), which uses failed and successful executions as complementary evidence for program revision. Frontier-guided program search annotates a shared diagnostic graph with execution evidence to distinguish supported progress from unresolved checks. Selective behavioral certification uses stateful API replay to restore recorded responses and mutations, checking whether edits preserve selected historical successes. Diagnosis and certification feedback, supplemented by sparse environment diagnostics, guide consecutive repairs across configurations. SCOPE outperforms the strongest baseline in both simulation and real-world tasks. On LIBERO-PRO and Robosuite, it achieves mean success rates of 80.66% and 84.48%, with respective gains of 6.51 and 3.48 percentage points.

Primary area:
applications to robotics, autonomy, planning
Keywords:
embodied heuristic learning, agentic robot

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TnVyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SCOPE lets a fixed language model repair robot control programs from execution feedback. A critic labels each run against a shared diagnostic graph of task checks and prerequisites to find a "failure frontier", and the model picks a local repair, stage redesign, or strategy regeneration. After a repair succeeds and changes the accepted program, up to three stored successful runs are replayed against recorded API requests, responses, and state changes. A full match with normal termination certifies success on that recorded instance, under determinism and API-boundary assumptions. Sparse environments running on uncertified configurations give further repair evidence. Frozen programs reach 80.66% on LIBERO-PRO (+6.51 over ASPIRE on the same GLM-5.3-Flash backbone) and 84.48% on Robosuite (+3.48 over ASPIRE with Opus 4.6). On a physical robot, three tasks give 40/60 successes versus 24/60 for ASPIRE. Ablations on two settings and replay time costs are also reported.

Critical strengths
  1. Same-backbone comparison with three complete learning runs. SCOPE improves all six LIBERO-PRO settings, most on Goal-Task (+13.07) and Spatial-Pos (+10.93), and all seven Robosuite task means.
  2. Removing any of the three components lowers success in both ablated settings.
  3. The theorem's assumptions and scope, and the cost and safety limitations, are stated candidly.
Critical weaknesses
  1. Certification is narrower than its framing. A full match needs every request and pre-call state to equal the tape, so a certified run is one in which the edit altered no API request. Only 38.9-60.6% of checks certify (Table 5); the rest include behavior-changing edits, where regressions arise. Certification also does not gate acceptance (Sec. 3.3, App. H), so "certification of program edits" overstates what the method does. The theorem is a short induction under strong assumptions, and the App. G bound is immediate.
  2. The source of the gain is not isolated. Ablations cover two of six settings, chosen for the largest gains (App. C), with three runs; drops (3.00-6.80) are comparable to SDs (3.72-9.10). No variant removes all three components, and each single removal keeps 6.27-10.07 of the 13.07-point Goal-Task gain and 4.86-7.66 of the 10.93-point Spatial-Pos gain over ASPIRE (my arithmetic from Table 4). I found no model calls, tokens, or environment episodes per method, though SCOPE adds critic, diagnosis and planning calls, and VLA baselines keep their original protocols, while SCOPE uses 15 development seeds with environment feedback (App. B). The replay cost (4.62-6.59% of environment time) has no re-execution baseline. On hardware, one frozen program per method gives 40/60 vs 24/60; by an unpaired Fisher exact test (my calculation) no single task is significant (p about 0.11, 0.33, 0.11).
Questions for authors
  1. Which model is the critic, how accurate are its node labels, and are its calls in the budget?
  2. What are model calls, tokens and environment episodes per method in simulation?
  3. Does the 4.62-6.59% replay ratio include replays that stop at the first divergence?
  4. On hardware, what were the certification and divergence rates, and what does a match mean when execution is not deterministic?
  5. Of the witnesses that diverge, what share fail when diagnosed in the environment? Requests that could change my score (I would raise it if most of the gain remains, and lower it if the gain shrinks under matched calls):
  6. An all-components-removed ablation on all six settings, ASPIRE with matched model calls, replay versus re-running the same witnesses in the environment, and SCOPE with a second backbone. Suggestion, not part of my decision:
  7. State in the abstract that certification is feedback, not an acceptance gate.
3 comments

Thank you for the specific comparisons you suggested. We ran the token-budget comparison, the joint ablation, and the re-execution control with GLM-5.3-Flash, using one learning run per task and the original held-out seeds. We apologize for the time it took to complete these experiments and bring the results together, and hope the delay has not inconvenienced you.

W2 / Q2 / Q6(b). Calls, tokens and environment executions for SCOPE and ASPIRE

We matched each task's input plus output token budget to full SCOPE's first learning run. Model calls are reported as a learning cost. For held-out success, we compared SCOPE with ASPIRE at two endpoints on each task, when ASPIRE first reached or exceeded SCOPE's input plus output token usage, and when ASPIRE completed its full search. The token count included auxiliary requests and retries. At the threshold, ASPIRE could finish validating a candidate it had already generated and freeze its incumbent. If search finished earlier, we used its final program. At both endpoints we evaluated frozen programs on the same 50 seeds per LIBERO-PRO task–setting pair and 100 per Robosuite task.

Frozen programLIBERO-PRO (3,000 trials)Robosuite (700 trials)
SCOPE2,379 (79.30%)596 (85.14%)
ASPIRE at token threshold1,876 (62.53%)519 (74.14%)
ASPIRE allowed to finish, same run2,117 (70.57%)557 (79.57%)

At the token threshold, SCOPE gained 16.77 points on LIBERO-PRO and 11.00 on Robosuite. Allowing ASPIRE to finish reduced those differences to 8.73 and 5.57 points in the same run.

Across the 67 task–setting pairs, full SCOPE made 3,732 calls and used 1,975 environment executions. Full ASPIRE used about 10,400 calls. The full-run token budgets were ≈3.27B input and ≈0.28B output for SCOPE and ≈4.60B input and ≈0.52B output for ASPIRE. The cutoff experiment used token usage returned by each request. Both learners used the same development configurations. The VLA rows report published scores under their original protocols.

W2 / Q6(a). Joint contribution of the three forms of repair guidance on Goal–Task and Spatial–Pos

In Table 4, each single removal lowered mean success across three runs in both settings. Two-sided Welch tests using the reported run means and SDs give unadjusted p-values from .137 to .594 for the six comparisons.

We compared full SCOPE's first run with a new run that removed all three forms of repair guidance together. We hid certification feedback, the first divergence, and graph and frontier evidence from the planner and code generator, and omitted the choice of repair level. Both learners kept the same repair limits, raw observations, API traces, internal replay, diagnostics and follow-ups. Each learner ran to natural termination, and we froze one program per task. Goal–Task and Spatial–Pos each cover 10 tasks. For each task we evaluated both programs on the same 50 held-out seeds, giving 500 trials per setting.

SettingFull SCOPE (%)Joint ablation (%)Full − ablation (pp)
Goal–Task55.8045.60+10.20
Spatial–Pos65.8057.00+8.80

Q6(d). Second backbone. We evaluated SCOPE only with GLM-5.3-Flash. Opus 4.6 in Table 2 is a published ASPIRE result.

W2 / Q1. Which critic is used, how accurate is it, and is it counted?

The critic uses GLM-5.3-Flash. Its 2,235 calls, including retries, count toward SCOPE's 3,732 calls. We measured label consistency against the execution trace and task prerequisites. We did not score labels against independent annotations. The critic produced a usable assessment on 1,904/1,975 executions (96.4%). Checking evidence windows, prerequisite order and invariant coverage changed 5,554/42,667 node labels (13.0%). Masking graph and frontier evidence lowered held-out success by 6.80 points on Goal–Task and 5.47 on Spatial–Pos over three runs.

W2 / Q3 / Q6(c). Replay timing and direct re-execution

Yes, Table 5 includes checks that stop at the first divergence. Its 4.62–6.59% divides replay time by the original environment time for the corresponding witnesses. For a new offline timing test, each pair consisted of an edited program from full SCOPE's first run and a historical success it had checked. We sampled up to 20 pairs per setting and status (certified or divergent) and ran each edited program both against its tape and in the simulator restored to the original witness state and randomness. Across 272 pairs, replay took 1,309 s and live execution took 21,798 s (6.00%, excluding restoration).

To test the effect on learning, we compared full SCOPE with SCOPE-Reexec, which replaces historical replay during search. After each accepted source change, each learner samples up to three successes from its own history. SCOPE checks their tapes and runs sparse diagnostics on eligible unresolved cases. Reexec runs every selected configuration in the environment and uses failures for follow-up repair. Both used GLM-5.3-Flash with at most three main repairs and one follow-up per development configuration. We froze each learner's final program after one run per task and tested both on the same held-out seeds. Full SCOPE and Reexec respectively achieved 79.30% and 78.97% on LIBERO-PRO and 85.14% and 84.86% on Robosuite. Using each task's duration and the same parallel schedule gives elapsed learning times of ≈164.9 hours for SCOPE and ≈182.6 hours for Reexec.

W1 / Q5. What fraction of divergent witnesses actually fail?

For online environment diagnostics, the first run recorded 23 failures among 249 fresh resets of unresolved configurations. The task-level counts bound the failure rate for diagnosed divergences at 6.64–9.54%. Of the eight unavailable checks, one was diagnosed successfully and the remaining seven could account for at most seven failures. Allocating the failures within each task gives a lower bound of 16/241 and an upper bound of 23/241 for the divergent cases.

For the original witness instance, the offline pairs used in the timing test above also give a direct task outcome. On each restored instance, the edited program had to terminate normally and satisfy the benchmark evaluator with live API responses. 14/132 divergent pairs failed (10.61%). All 140 sampled certified pairs succeeded.

W1 / W2 / Q4. What did a hardware match establish?

We counted SCOPE's historical checks during one development run on each of the three physical tasks. Of 28 checks, 13 matched (46.4%), 14 diverged (50.0%), and one was unavailable. A match means the edited program reproduced the complete API interaction with the recorded observations, responses and public-state changes. Sensors, object placement and contact can vary on a new execution. The 13 matches resolved the historical checks locally, without manually rebuilding the scenes. All replay took 126.3 s. Of 13 eligible unresolved configurations tested physically, four failed.

We froze one program per method per task and evaluated each on 20 new placements. SCOPE succeeded on 40/60 placements and ASPIRE on 24/60.

W1 / Q7. Certification in the abstract. We will state the order explicitly. “Once a program succeeds on the current configuration, SCOPE accepts it and replays selected historical successes to guide later repairs.” Appendix H gives the recorded-instance guarantee under faithful capture and deterministic execution. Appendix G explains how often an edit must change behavior to affect aggregate success.

Thank you for the authors' response. I will maintain my score.

Reviewer_WRRhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCOPE, a framework for improving robot control programs through failure-guided repair and selective certification of historical successes. It constructs diagnostic graphs to identify failure frontiers, guides LLM-based program revisions at different levels, and uses stateful API replay to check whether modified programs preserve previously successful behavior. Experiments on LIBERO-PRO, Robosuite, and real-world robots demonstrate improvements over existing approaches, including ASPIRE.

Critical strengths
  1. The paper addresses the important challenge of improving robot control programs while avoiding regressions on previously successful tasks. Combining failure-frontier diagnosis with historical-success feedback provides a systematic approach to iterative program repair.

  2. SCOPE consistently outperforms ASPIRE on LIBERO-PRO and Robosuite, with additional improvements demonstrated on physical robots. Component ablations support the contributions of failure-frontier guidance, certification feedback, and revision-level planning.

Critical weaknesses
  1. The core idea of using recorded successful executions to detect regressions resembles existing record-and-replay and mock-based regression testing approaches, such as RICK [1]. The paper does not sufficiently clarify the technical novelty of its stateful API replay mechanism or compare it against simpler regression-testing alternatives.

  2. The theoretical guarantee requires faithful recording and deterministic execution and only certifies success on the recorded instance, rather than generalizing to future executions. Moreover, replay divergence indicates behavioral differences rather than actual failure, potentially triggering unnecessary diagnostics for alternative but equally successful strategies.

  3. The ablation removing certification feedback still retains replay and environment diagnostics, making it difficult to isolate the benefit of the proposed certification mechanism itself.

[1] Tiwari D, Monperrus M, Baudry B. Mimicking production behavior with generated mocks[J]. IEEE Transactions on Software Engineering, 2024, 50(11): 2921-2946.

Questions for authors
  1. How does SCOPE's stateful full-trace replay extend existing execution-based regression testing approaches, such as RICK (Tiwari et al., 2024)? Could the authors clarify the additional technical contributions and guarantees provided by the proposed certification mechanism?

  2. As noted in Weakness 2, since replay divergence does not necessarily imply task failure, how does SCOPE handle alternative successful trajectories that differ from recorded executions? Could strict trajectory matching unnecessarily reject beneficial program revisions?

5 comments

Thank you for pointing us to RICK and for suggesting a simpler regression check. We will cite RICK in our discussion of replay, and we have now compared replay with direct re-execution of historical successes. We apologize for the time it took to complete that comparison and hope the delay has not inconvenienced you.

W1 / Q1. What does SCOPE add to record-and-replay testing?

RICK (Tiwari, Monperrus and Baudry, IEEE TSE, 2024) records a Java method's interactions with dependencies in production and uses them to generate unit tests with mocks. Its oracles check method output, call arguments, or call order and frequency. SCOPE shares the idea of reusing recorded interactions. The additional question in robot program repair is whether the edited program would still complete a task on a previously successful instance, and which instance to revisit if its behavior changes.

SCOPE records a complete successful episode at the robot API boundary. For each call, it checks the request and the program-visible state before the call, then restores the recorded return or exception and any mutations. Suppose an API updates a pose array in place and the program later reads a slice of that array. The replay must preserve that shared reference for the program to see the correct pose. Certification requires every call and the terminal public state to match, with no extra calls and normal termination. With faithful recording, all external inputs and effects captured, and deterministic execution on the original instance, a full match implies that the edited program succeeds on that instance (Appendix H.1).

After accepting an edit, SCOPE samples up to three historical successes. A match resolves that witness locally. On divergence, the first changed call or state is available to the planner. An eligible unresolved configuration can be tested in the environment, and a failed diagnostic can trigger another repair.

W2 / Q2. Can strict matching reject a beneficial revision?

No. SCOPE accepts a program as soon as it succeeds on the current configuration. If the edit uses a different grasp pose on a historical instance, replay stops at the changed pose. The program stays accepted. The cost of the mismatch is at most a diagnostic on that configuration. If the new grasp succeeds during that diagnostic, no follow-up repair is triggered.

To test replay on the original instance, we paired an edited program from SCOPE's first simulation run with a recorded success that it had checked. We sampled up to 20 pairs without replacement for each setting and replay outcome (certified or divergent) before seeing their task outcomes. For each pair we restored the witness's initial simulator and controller state, program inputs and randomness, then ran the edited program once with live API responses. The benchmark evaluator scored task success.

Replay outcomePassed task evaluationFailed task evaluation
Certified140/1400/140
Divergent118/13214/132

In 118 of the 132 divergent cases, treating the first mismatch as a task failure would have flagged a successful program.

W1 / W3. Direct re-execution as a simpler historical check

The original ablation hides replay feedback from the planner while keeping replay and diagnostics active. For the new experiment we compared full SCOPE with SCOPE-Reexec, a variant that replaces replay during learning. After each accepted source change, each learner samples up to three previously successful configurations from its own history. SCOPE replays their recorded interactions locally and tests eligible unresolved cases in the environment. Reexec runs every sampled configuration in the environment and uses failures for follow-up repair. Both use GLM-5.3-Flash and allow three main repairs and one follow-up repair per development configuration. We ran each learner once per task to its natural endpoint, froze its program and tested both on the same held-out seeds (50 per LIBERO-PRO task–setting pair and 100 per Robosuite task).

Held-out evaluationSCOPESCOPE-Reexec
LIBERO-PRO (3,000 trials)79.30%78.97%
Robosuite (700 trials)85.14%84.86%

To measure retention separately, we saved the initial states of each learner's first three distinct successful development configurations per task and tested its frozen program on those states offline. SCOPE retained 194/201 successes and Reexec retained 193/201. On the 272 program/witness pairs from Q2, replay took 1,309 s and live execution took 21,798 s (6.00%, excluding scene restoration).

Dear Reviewer WRRh,

Thank you again for your thoughtful review. In our response, we describe how SCOPE’s replay differs from RICK, what a full match guarantees on a recorded instance, and how SCOPE handles edits that follow a different successful trajectory. We also report the direct re-execution comparison you suggested and explain what it adds to the original feedback ablation.

If any of your concerns remain open after reading our response, we would be happy to discuss them further.

Thank you for your reply. I believe it has resolved most of my doubts, and I will reconsider my score

Reviewer_VfXVOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

SCOPE repairs robot control programs with a fixed language model, combining prerequisite-aware diagnosis with stateful replay of recorded successful API traces. After accepted program updates, a complete trace match certifies success on the recorded instance under explicit capture and determinism assumptions; selected uncertified configurations receive fresh diagnostics. With the same GLM-5.3-Flash backbone, mean success improves from 74.14% to 80.66% on LIBERO-PRO and from 80.19% to 84.48% on Robosuite. The physical-task comparison improves from 24/60 to 40/60 held-out trials (Tables 1–3).

Critical strengths
  1. Precisely scoped reuse of successful executions. Replay checks API requests and pre-call public states, reproducing recorded returns, exceptions, and mutations. Section 3.3 and Appendix H.1 establish the conditional recorded-instance guarantee and distinguish divergence from task failure. This makes historical successes reusable evidence for program revision while clearly delimiting what is certified.

  2. Consistent gains in the principal comparisons. With the same GLM-5.3-Flash backbone, SCOPE improves both simulation benchmark means (Tables 1–2), with three repetitions of the complete learning and evaluation process. It also improves held-out physical-task successes from 24/60 to 40/60 (Table 3). These results support the overall framework, while the component ablations provide additional evidence for the visible feedback mechanisms.

Critical weaknesses
  1. Replay fidelity needs stronger implementation evidence. The guarantee requires complete capture of behavior-relevant API interactions and faithful restoration of program-visible state. The paper lacks a detailed API-coverage specification and direct empirical validation of certified edited programs; certification rates and costs are reported only for simulation. These gaps limit assessment of how reliably the theorem's conditions hold in practice. Coverage details, replay validation, and physical-task certification statistics would clarify this.

  2. Certification and diagnostic contributions are not fully separated. The paper does not establish that the same-backbone comparison matches actual environment executions, model calls, or total learning costs. Moreover, the certification-feedback ablation hides replay results from planning but retains internal replay, replay-dependent diagnostics, and follow-ups (Appendix C). It therefore isolates visible feedback rather than the entire certification mechanism. Matched-budget comparisons and historical-success retention measurements would clarify the source of the performance and efficiency gains.

Questions for authors
  1. Which interactions and public-state types can the recorder/replayer capture, and how are unsupported cases handled? Please separate supported, unavailable, divergent, and certified checks; provide controlled-simulation validation of certified edited programs and physical-task certification statistics.

  2. Can you provide a matched-budget comparison with same-backbone ASPIRE, reporting environment executions, model calls/tokens, and total learning time, with main search, diagnostics, and follow-ups separated?

  3. How much does replay reduce regressions on historical successes? Please compare against budget-matched random historical diagnostics or historical re-execution, and report retained successes and regressions across accepted updates.

2 comments

Thank you for asking us to test certified edits directly. We apologize for the delay while we restored the original simulator instances and reran the edited programs with live API responses, and we hope it has not inconvenienced you. All 140 sampled certified pairs passed the task evaluator.

W1 / Q1a. Which interactions and state types can replay capture?

The recorder captures calls to get_observation, mask_to_world_points, plan_grasp, solve_ik, goto_pose, and so on. Each event stores the API name, positional and keyword arguments, return value or supported exception, and the public state before and after the call.

Supported values include Python and NumPy scalars, lists, tuples, dictionaries, and numerical or Boolean arrays. These cover images, depth maps, poses, grasp candidates and joint vectors. Replay restores changes to retained arrays and containers and preserves shared references. For array views it records the base, offset, shape, strides and dtype, including transposes and negative strides. Supported built-in exceptions preserve their type, arguments and post-call public state.

Unregistered objects, object or structured arrays, unsupported external buffers, in-place changes to array layout, and unmodeled exception state fall outside this coverage. The guarantee assumes that behavior-relevant I/O passes through the registered boundary, including any time or randomness used by the program. Unsupported state at recording time marks the tape unavailable and lets the live API call proceed. An incompatible runtime fingerprint, corrupt tape, unsupported candidate state or resource limit also yields “unavailable.”

First simulation run, edited programsChecksInterpretation
Certified215All requests and public states matched, the tape was exhausted, and the program terminated normally
Divergent279The executor located a first behavioral difference
Unavailable8The executor could not establish a comparison
Total502All selected historical checks, excluding self-matches

Replay could compare behavior in 494/502 checks (215 certified and 279 divergent). The denominator for certification includes all 502 attempts.

W1 / Q1b. Do edited programs that certify actually succeed in the simulator?

For this offline test, one pair consists of an edited-program snapshot from full SCOPE's first run and a historical success it checked. Across the six LIBERO-PRO settings and Robosuite, we sampled up to 20 pairs without replacement for each setting and replay status (certified or divergent) before seeing the environment outcomes. We restored each witness's simulator, controller, wrapper, random state, program inputs and perception configuration. After checking restoration with an original witness program in each setting, we ran the edited program once with live API responses and scored normal termination and task success with the benchmark evaluator.

Replay statusEnvironment successesEnvironment failures
Certified140/1400/140
Divergent118/13214/132

All 272 restorations yielded valid evaluations. The 118 successful divergences show why a changed interaction requires an environment check before it can be called a task failure.

W1 / Q1c. What happens on the physical robot?

We counted SCOPE's historical checks during one development run on each of the three physical tasks. Overall, 13/28 checks matched (46.4%), 14/28 diverged (50.0%), and one was unavailable. A hardware match means that under the recorded inputs and effects the edited program reproduces the successful interaction through termination. Object placements, sensor observations and contact dynamics can differ on a new physical trial. The 13 matched checks required no physical reconstruction. The 28 checks took 126.3 s of local replay, and four of the 13 physical diagnostics on eligible unresolved configurations failed. We evaluated the frozen programs separately on 20 new placements per task, yielding 40/60 successes for SCOPE and 24/60 for ASPIRE.

W2 / Q2. Token-matched held-out performance and full-run learning costs

We used one learning run per task with GLM-5.3-Flash for both methods. For each task, full SCOPE's first-run input plus output tokens, including critic requests and retries, set ASPIRE's threshold. ASPIRE stopped issuing requests at the first crossing, finished validating any candidate already generated and froze its incumbent. If it finished searching earlier, we froze its final program. We evaluated both methods on the same held-out seeds (50 per LIBERO-PRO task–setting pair and 100 per Robosuite task).

BenchmarkSCOPEToken-budgeted ASPIREDifference
LIBERO-PRO2,379/3,000 (79.30%)1,876/3,000 (62.53%)+16.77 pp
Robosuite596/700 (85.14%)519/700 (74.14%)+11.00 pp

Across the 67 task–setting pairs, full SCOPE used 3,732 calls and full ASPIRE used about 10,400. The full-run token budgets were approximately 3.55B for SCOPE and 5.12B for ASPIRE. The per-task token cutoff used the usage returned by each model request. Full SCOPE used 1,975 environment executions and 699.3 summed task-hours.

SCOPE learning stage, first runModel callsApprox. input + output budget (M tokens)Environment executions
Main search3,2553,2081,703
Historical diagnostics284278249
Follow-up595923
Initialization and other13470

The token budget for each stage is based on the number and average length of requests in each role. Main search and follow-up together accounted for 559.1 hours of model response time and 38.3 hours of environment execution. Historical diagnostics accounted for 58.2 and 17.6 hours, respectively. The 699.3 task-hours also cover initialization and orchestration.

W2 / Q3. Does replay reduce regressions relative to direct historical re-execution?

Table 4 masks the planner's view of replay while keeping replay active. We added SCOPE-Reexec to test replacing replay during learning. After each accepted source change, each learner samples up to three previously successful configurations from its own history. SCOPE checks their tapes locally and tests eligible unresolved cases in the environment. Reexec executes every selected configuration in the environment and uses failures for follow-up repair. Both use GLM-5.3-Flash and allow up to three main repairs and one follow-up per development configuration. Each ran once per task to natural termination, and we evaluated its frozen program on the same held-out seeds.

For the offline retention test, we saved the initial state of each learner's first three distinct successful development configurations per task. We reran its program on the panel instances already collected at each recorded checkpoint and tested the frozen program on the completed panel at termination. Panel outcomes did not feed back into learning. Among instances successful at the preceding checkpoint, 6.39% failed with SCOPE and 5.63% with Reexec. The final programs retained 194/201 and 193/201 successes, respectively.

Across 67 task–setting pairs, one runSCOPESCOPE-Reexec
Retained at recorded checkpoints502/526 (95.44%)512/537 (95.34%)
Previously successful, then failed at next checkpoint20/313 (6.39%)18/320 (5.63%)
Retained by final program194/201 (96.52%)193/201 (96.02%)
Held-out success, LIBERO-PRO / Robosuite79.30% / 85.14%78.97% / 84.86%

The panel analysis covers 208 SCOPE checkpoints and 211 Reexec checkpoints. An instance enters the regression denominator when it succeeds at one recorded checkpoint and is tested again at the next. The rates in the table apply to those observed transitions.

For the direct cost comparison, we timed replay and live execution of the same edited program on each of the 272 restored program/witness pairs described in Q1b. This took 1,309 s for replay and 21,798 s for live execution, excluding scene restoration. Over the full learning run, SCOPE used 1,975 environment executions. The cost model gives ≈2,263 for Reexec. Using each task's duration and the same parallel schedule gives elapsed learning times of ≈164.9 hours for SCOPE and ≈182.6 hours for Reexec.

Comments on the paper

8 comments

Thank you for suggesting that we rerun the historical seeds directly. We compared replay with direct re-execution on LIBERO-PRO and Robosuite using GLM-5.3-Flash. We apologize for the time it took to get these results to you and hope the delay has not complicated your review.

W1 / Q1b. Replay versus re-executing historical seeds

We compared full SCOPE with SCOPE-Reexec, which replaces replay during learning. After each accepted source change, each learner samples up to three previously successful development configurations from its own history. SCOPE replays their recorded interactions locally and tests eligible unresolved cases in the environment. Reexec runs every sampled configuration in the environment and uses failures for follow-up repair. Both allow up to three main repairs and one follow-up repair per configuration. Each learner ran once per task to its natural endpoint. We froze the resulting programs and evaluated them on the same held-out seeds (50 per LIBERO-PRO task–setting pair and 100 per Robosuite task).

Across 67 task–setting pairsSCOPESCOPE-Reexec
LIBERO-PRO held-out success (3,000 trials)79.30%78.97%
Robosuite held-out success (700 trials)85.14%84.86%
Environment executions during learning1,975≈2,263 (from the cost model)
Elapsed learning time from task durations under the same parallel schedule≈164.9 h≈182.6 h

Replacing replay with re-execution changed held-out success by less than 0.34 percentage points on either benchmark, and added approximately 17.6 hours under that schedule. We also timed the two checks on identical instances offline. From full SCOPE's first run, we sampled up to 20 edited-program/witness pairs per setting and replay outcome (certified or divergent). For each pair we ran the same program against its tape and against the live simulator restored to the witness's initial state and randomness. Across 272 pairs, replay took 1,309 s and environment execution took 21,798 s, excluding restoration time (6.00%). Table 5's 0.54–1.36% uses total learning time, including model inference, as its denominator.

W2 / Q1a. Matching ASPIRE's per-task token budget

In a separate one-run control, we used full SCOPE's first-run input plus output token usage on each task as ASPIRE's budget for that task. ASPIRE stopped issuing requests when it first reached or exceeded the threshold, or at natural termination if earlier. It could finish validating a candidate already generated. We included critic calls and retries and evaluated both frozen programs on the same held-out seeds. We report environment executions and time as costs incurred, with no separate stopping rule for either quantity.

BenchmarkSCOPEASPIRE at token thresholdGain
LIBERO-PRO2,379/3,000 (79.30%)1,876/3,000 (62.53%)+16.77 pp
Robosuite596/700 (85.14%)519/700 (74.14%)+11.00 pp

Across 67 task–setting pairs, the first full run used 3,732 model calls for SCOPE and about 10,400 for ASPIRE. ASPIRE generates and evaluates a population of candidates. SCOPE reuses one program across configurations and ends a visit as soon as that program succeeds. Full SCOPE used 1,975 environment executions and 699.3 summed task-hours.

W2. Robosuite baseline in the abstract. Both reported differences follow from SCOPE's 84.48%. The 4.29-point gain in Sec. 4.2 is over same-backbone ASPIRE (80.19%). The 3.48-point gain in the abstract is over ASPIRE–Opus 4.6 (81.00%), the highest published baseline. We will name the comparator in the abstract. The other published methods use their original evaluation protocols.

W2 / Q2. What does replay establish on the physical robot, and how much work does it save?

Camera observations, joint state and contact dynamics can vary when the robot and objects are reset to the same placement. The theorem holds for the recorded instance with fixed inputs and randomness. On hardware, a full match tells us that the edited program, given the recorded observations and API effects, reproduced the successful sequence of calls and terminated normally.

In the original hardware study, each method learned once per task using ten development placements. Of SCOPE's 28 historical checks across three tasks, 13 matched locally, 14 diverged and one was unavailable. Those 13 matches required no manual scene reconstruction. Local replay took 126.3 s in total. We tested 13 eligible unresolved configurations on the robot and observed four failures. Over the full learning process, SCOPE used 122 model calls, 65 physical executions and 10.12 task-hours. ASPIRE used 230 calls, 197 physical executions and 13.65 task-hours. We measured these totals for the two complete methods. The hardware measurement specific to replay is the number of historical checks resolved locally.

W4 / Q3a. Repair-level usage and success

We counted each classified candidate once at its immediately following environment validation, pooling main and follow-up repairs in the first simulation run.

Revision levelShare of 635 repairsSuccessful validations
Local repair333/635 (52.4%)155/333 (46.5%)
Stage redesign197/635 (31.0%)40/197 (20.3%)
Strategy regeneration105/635 (16.5%)13/105 (12.4%)

W4. Replay outcomes and subsequent regressions. In the restored-instance test described in Q1b, all 140 certified program/witness pairs succeeded and 14/132 divergent pairs failed. To track later regressions, we saved the initial states of each learner's first three distinct successful development configurations per task. We tested the programs offline on this panel at successive recorded checkpoints and at termination, withholding the panel results from learning. Among instances successful at one checkpoint and tested again at the next, 20/313 failed with SCOPE (6.39%) and 18/320 with Reexec (5.63%). The final programs retained 194/201 and 193/201 historical successes, respectively. These longitudinal counts pool replay statuses.

W3 / Q3b. Significance of the ablations

We used two-sided Welch tests with the complete learning run as the statistical unit (three per condition). We calculated the tests from the means and sample SDs reported in Table 4. Each cell gives full SCOPE minus the ablation in percentage points and the unadjusted p-value.

Guidance removedGoal–TaskSpatial–Pos
Certification feedback+4.27 (p=.466)+6.07 (p=.137)
Failure frontier+6.80 (p=.341)+5.47 (p=.150)
Explicit revision level+3.00 (p=.594)+3.27 (p=.357)

Full SCOPE scored higher in all six comparisons. The individual differences did not reach statistical significance at the .05 level. For the certification-feedback ablation, the approximate 95% confidence interval is [−10.85, 19.39] points on Goal–Task and [−3.01, 15.15] on Spatial–Pos.

Even on fixed tasks and seeds, GLM-5.3-Flash can generate different programs and repairs. SCOPE's diagnostic graph guides repairs, and stateful replay checks their effects on past successes. Overall LIBERO-PRO success varied less across three runs for SCOPE (80.66 ± 1.46%) than for ASPIRE (74.14 ± 3.11%, mean ± run SD).

W3. Joint ablation on Goal–Task and Spatial–Pos

Given the time available for the rebuttal, we ran the joint ablation with one learning seed per task–setting pair. The original single-component ablations in Table 4 used three independent learning runs per condition. We compared the joint ablation with full SCOPE's first run. We hid certification feedback, graph labels and failure frontiers from the planner and code generator, and removed explicit repair-level selection. Both conditions retained raw traces, internal replay, diagnostics, follow-ups and the same repair limits. Each setting covers 10 tasks. For each task we froze one program per condition and evaluated both on the same 50 held-out seeds, giving 500 trials per setting.

SettingFull SCOPE (%)Joint ablation (%)Full − ablation (pp)
Goal–Task55.8045.60+10.20
Spatial–Pos65.8057.00+8.80

Dear Reviewer qRoT,

Thank you again for your detailed questions. Since our response is split into two parts, we hope this brief guide is helpful. Part 1 covers direct re-execution, the token-budget comparison with ASPIRE, and the Robosuite baselines. Part 2 covers replay on the physical robot, how often each repair level was used and succeeded, regressions on historical successes, and the statistical tests and additional ablation results.

If there is a point in either part that you would like us to clarify or expand on, we would be glad to follow up.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.