acceptodds
Under review as a conference paper at ICLR 2027

Speculative Rollback Correction: Counterfactual Credit Assignment for GUI Agent Imitation

Abstract

Interactive imitation learning supervises agents on learner-induced states, yet conventional methods typically identify errors via disagreement with a reference policy.In multi-solution GUI tasks, this proxy frequently conflates genuinely harmful actions with valid alternative trajectories. To resolve this ambiguity, we define errors through task recoverability: an action is harmful to the extent that it reduces the maximum probability of eventual success achievable by any continuation policy. Because recoverability optimizes over policy space, it reflects an intrinsic property of the task and environment rather than the choices of a specific teacher. We show that the resulting recoverability process admits an exact Doob decomposition, whose predictable compensator yields per-step harm and provides a principled mechanism for localizing feasibility-reducing actions. We instantiate this theoretical framework as Speculative Rollback Correction (SRC). In SRC, the student agent executes a short speculative trajectory segment, after which the teacher localizes the earliest action causing a drop in recoverability. A reset-and-replay mechanism then restores the corresponding state for targeted correction, followed by a hard verifier that filters candidate continuations before updating a standard next-action policy. Across WebArena-Infinity, WebArena-Lite, and OSWorld, augmenting Expert SFT with SRC data improves teacher-free deployment success rates by 9.7, 3.6, and 12.9 percentage points, respectively. Under comparable teacher-query budgets, SRC outperforms OEC-style training across web benchmarks; compared to LEAP-style training, it achieves superior success on WebArena-Infinity and competitive performance on WebArena-Lite (within 2.6 points) while reducing teacher queries by 56–58%. Controlled experiments further confirm the efficacy of corrective labels and teacher-guided localization. At test time, the deployed policy operates autonomously without requiring teacher queries, verifiers, or environment resets, establishing recoverability-guided correction as a practical data-collection principle for resettable GUI environments.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
GUI Agent, Credit assignment, Data aggregation, vision-language agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ST2VOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Speculative Rollback Correction (SRC) for training GUI agents. Rather than treating every deviation from an expert demonstration as an error, it defines harmful actions as those that reduce the task’s recoverability. During data collection, a student executes a short segment, a teacher identifies the earliest harmful step, and the environment is reset and replayed so the teacher can provide a correction. Verified trajectories and correction actions are then used to train a student that operates without the teacher at deployment. The authors report improved success rates on three GUI benchmarks, although the collection process depends on reliable replay and teacher judgments.

Critical strengths
  • Well-motivated problem. Identifying and correcting consequential errors during GUI agent execution is a worthwhile research problem.
  • Strong empirical results. SRC improves teacher-free success rates over Expert SFT on all three evaluated benchmarks, and the paper includes controlled comparisons of corrective labels, rollback localization, and replay reliability.
Critical weaknesses
  • Unclear introduction. The introduction moves into methodological details and formulas too early, making the core problem difficult to grasp.
  • Unclear role of the theory. The extensive formulas and proofs make the method harder to follow, while their necessity for designing or implementing SRC is not clearly established. If the theoretical results are the main contribution, the paper may be better positioned as theory-oriented work.
Questions for authors

Related work and motivation: Could the authors discuss more closely related GUI agent studies, particularly those using reflection or corrective data? Multiple valid paths to a task goal also occur in general agents. Why is this problem framed primarily as GUI-specific?

Replay reliability: Replaying GUI actions may fail to restore the intended state. How are replay mismatches detected and handled? Have the authors considered state snapshots, such as Docker snapshots? How do replay failures affect corrective-data quality and final success rates?

Teacher judgment bias: The teacher reviews short segments of student behavior and decides whether to accept or roll back, rather than performing a full rollout. What biases might this introduce, and how are they controlled?

Localization accuracy: Is there independent annotation or another means of measuring whether the teacher identifies the earliest harmful step correctly? Outperforming random rollback shows that localization is useful, but does not directly establish its accuracy.

Initial data source: Where do the initial Expert SFT trajectories come from?

Teacher-model ablation: How sensitive is SRC to the teacher VLM’s GUI capabilities? How do localization quality and final student performance change when different teacher models are used?

Training tasks and generalization: Where do the tasks used for online collection come from, and how are they separated from evaluation tasks? Does the method generalize to unseen applications or task distributions?

Trajectory length: Could the authors report the average number of steps and the length distribution of SRC-collected trajectories and deployed-student trajectories?

4 comments
AuthorsOpenReview ↗

Response to Reviewer ST2V (1/4): Presentation, Related Work, and Replay Fidelity

Thank you for recognizing the motivation and empirical results, and for identifying how the presentation can better connect the problem, method, and evidence.

W1–W2. The introduction and the role of theory are unclear.

We will lead with the practical problem and position the theory as a design rationale. The introduction will begin with the example in Figure 1: a user request can be completed through either search or menu navigation. Different actions can both be valid, while an apparently reasonable action can create a consequential error that becomes visible only later.

The revised presentation will make the contribution clear in this order:

  1. Problem: identify where a student trajectory needs intervention without requiring it to reproduce one demonstrated route.
  2. Method: inspect a short segment, retain its accepted prefix, restore the selected state, correct locally, and admit verifier-passing trajectories.
  3. Evidence: distinguish autonomous deployment gains, matched supervision ablations, and collector diagnostics.
  4. Formal interpretation: define recoverability and explain the idealized rollback result, with the detailed decomposition and proofs in the appendix.

The implemented procedure can be summarized without mathematical prerequisites:

StageDecision or operationResult
Student executionGenerate a segment of up to three actionsStudent-induced observations and actions
Teacher reviewAccept the segment or choose a rollback positionA proposed intervention boundary
On acceptanceContinue from the current stateRetained student progress
On rejectionReset, replay the retained prefix, inspect the restored state, and correctAn alternative continuation
Trajectory completionApply the task verifier and data filtersAdmitted training examples
Policy trainingCombine admitted SRC data with the expert demonstrations and train from base weightsAn autonomous next-action policy

We will remove wording implying that the theory proves the reliability of the teacher or that the collector explicitly estimates V∗V^*.

Q1. How does SRC relate to reflection and corrective-data methods, and why GUI agents?

We agree that multiple valid solutions are not specific to GUI tasks. The contribution should be situated within interactive imitation and corrective-data collection more broadly.

Related approachConnection to SRCDistinction relevant to this paper
DAgger and interactive imitationSupervision on learner-induced statesSRC adds segment review and explicit restoration of a selected correction state
UI-TARS reflection tuningLearning from errors and corrective tracesUI-TARS describes annotator-labeled correction and post-error reflection pairs; SRC uses teacher-selected rollback and executed alternative continuations
OEC-style takeoverExpert intervention during collectionOur SRC collector chooses an intervention point from a reviewed segment rather than a random switch time
LEAP-style post-hoc correctionFeedback on student-generated trajectoriesSRC interleaves short-segment review, restoration, correction, and resumed execution
Search with state restorationRevisiting earlier environment statesSRC uses restoration to construct training supervision; deployment uses an ordinary policy

The UI-TARS comparison refers specifically to its reflection-tuning procedure in Section 4.5. We will not characterize all prior methods as rejecting every deviation from a single reference action.

GUI environments are a useful testbed because they combine multiple valid interaction routes, stateful consequences, and sandbox-based restoration. Extending the procedure to other resettable tool-use environments is plausible, but has not been evaluated here.

Q2. How are replay mismatches detected and handled?

Replay fidelity is measured by comparing recorded application-state snapshots before rollback with the states regenerated by reset-and-replay. Mismatches are logged, but the current collector does not reject every mismatch online.

Table R7 (overall agreement: Section 5.4; prefix-length breakdown: Table 4 in the revision). Replay agreement reported in Section 5.4 and Appendix C.4.

Environment / criterionSnapshot-verified rollbacksAgreement
WebArena-Infinity: strict match1,23693.6%
WebArena-Infinity: ignoring designated non-functional fields1,23696.3%
OSWorld6,88897.4%
AuthorsOpenReview ↗

Response to Reviewer ST2V (2/4): Replay Handling, Teacher Bias, and Localization Utility

Prefix-length breakdown (= Table 4 in the revision).

Replayed prefix length on WebArena-InfinityAgreement
0–2 actions99.0%
3–7 actions95.5%
8 or more actions92.4%

The WebArena-Infinity agreement denominator excludes 85 rollbacks with missing snapshots. Both missing observations and declining fidelity for longer prefixes limit the interpretation of these rates.

Two safeguards reduce, but do not eliminate, the consequences of replay drift:

  • The corrective action is generated from a fresh screenshot of the actually restored state, rather than solely from the pre-rollback screenshot.
  • A retained completed trajectory must pass the hard task verifier.

These safeguards do not establish that the original state was recovered exactly, that historical observations remain aligned, or that every local label is correct. We do not have an isolated ablation quantifying the effect of replay mismatch on final student SR.

OSWorld restores the initial environment from a VM snapshot; SRC then replays the accepted prefix rather than storing a snapshot after every action. Intermediate VM or container snapshots could reduce replay drift, but their benefit depends on whether they capture all relevant application and external-service state. We will discuss this option without claiming that container snapshots necessarily eliminate all mismatches.

Q3. What biases arise from reviewing only short segments?

The main risks and current mitigations are:

RiskCurrent mitigationRemaining limitation
Delayed consequences are missedInclude preceding context; verify completed trajectoriesA short window cannot reveal every downstream failure
Valid exploration is rejectedPrompt explicitly allows reasonable alternative routes and uncertain casesEfficiency preferences can still remove useful exploration
Teacher misunderstands the GUI stateReview screenshots and use a fresh observation for correctionErrors remain dependent on teacher capability
Successful trajectories are overrepresentedPreserve multiple verifier-passing branchesSuccess filtering does not recover rejected alternatives

The horizon ablation illustrates the trade-off: K=1K=1 yields 45.6% collector SR with 3,324 teacher queries, whereas K=3K=3 yields 51.9% with 2,529 queries (Appendix C.7). This supports the chosen horizon empirically; it does not prove that teacher myopia has been eliminated or that K=3K=3 is universally optimal.

Q4. Is the earliest harmful step independently validated?

Not directly. We agree that better performance than random rollback establishes utility, not localization accuracy.

The current evidence comprises the controlled random-rollback comparison, distributions of teacher-selected positions, and continuation diagnostics. None provides an independently annotated ground-truth harmful position. In particular, the 5.4% success rate when retaining an interior-flagged segment measures success under a particular student continuation, not optimal recoverability.

The closest available diagnostic is the 167-branch replay study in our response to MV6h (W2/Q5–Q6, part b): on branches where the teacher selected an interior position, teacher-selected rollback succeeded on 27/167 branches versus 19/167 for forced segment-start rollback (p=0.0386p=0.0386) and 9/167 for keeping the flagged actions (p=0.0039p=0.0039). This indicates that the selected position is useful, but it is still not ground-truth accuracy: comparing rollback positions is also affected by correction quality and continuation stochasticity. Such a comparison alone would not prove that the selected position is the earliest harmful action. We will describe the result as teacher-guided localization utility, and remove claims of measured earliest-error accuracy.

Q5. Where do the initial Expert SFT trajectories come from?

For WebArena-Infinity, the expert source combines successful Kimi-K2.5 and Qwen3.5 Plus trajectories from the public WebArena-Infinity trajectory dataset. Task coverage is merged using the (environment, task_id) key, rather than adding the two models' trajectory counts. The current public manifest covers 843 distinct task IDs across these two models; after screening for usable action-space outputs, required screenshots, and trajectory completeness, our eligible source coverage is 813 distinct tasks. SRC is collected on the remaining 1,620 − 813 = 807 tasks. This is a valid-source-coverage criterion, not a claim that a teacher failed on every one of those tasks.

AuthorsOpenReview ↗

Response to Reviewer ST2V (3/4): Data Preparation and Teacher Sensitivity

Data preparation also filters unsupported or malformed action outputs, duplicate action emissions, excessive repetitions of the same action, and loop-like trajectories, followed by action-format alignment. The final expert training export contains 8,025 next-action examples from 750 expert trajectory records. The 813 covered source task IDs and the 750 training trajectory records are different reporting units and stages; they must not be substituted for one another. Every augmented policy combines the common expert demonstrations with its own collector's increment and is trained from base weights. The Expert SFT checkpoint serves as the collection student.

For the Kimi-K2.5 SRC collection, the trajectory-length statistics in Table R9 describe 108 newly collected trajectories with 1,382 actions, giving 1,382 / 108 = 12.8 actions per trajectory after rounding. The final SRC training increment contains 1,158 examples, so the combined training set still has 8,025 + 1,158 = 9,183 examples. Collected actions and final training examples are different accounting stages. Action-space validation, screenshot/completeness checks, repetition and loop filtering, and training-format preparation determine which supervision records are retained; the action count of 1,382 must not replace 1,158 in the training-size comparisons. We do not infer the number removed by each individual filter from these aggregate totals. The earlier description of “227 newly collected trajectories” conflated these reporting units and will be corrected.

Within the final 1,158-example SRC increment, 959 examples are accepted student actions and 199 are teacher corrections. The earlier 7,882 / 1,301 figures summed to all 9,183 expert-plus-SRC training examples and were incorrectly presented as SRC's student-action / correction composition. This was a reporting-scope error: that composition should be counted within the SRC increment, while the 8,025 expert examples are reported separately. We withdraw the earlier combined-set attribution and correct Appendix C.1 to distinguish collection trajectories, collected actions, and final training examples. The training mixture and its 9,183 examples are unchanged.

Q6. How sensitive is SRC to the teacher model?

The teacher comparison uses the same 807 WebArena-Infinity tasks without validated teacher-data coverage for all three teachers: 1,620 total tasks minus the 813 distinct task IDs covered by the quality-screened Kimi/Qwen source union. The Kimi-K2.5 run supplies the SRC training increment used by the main Table 1 policy; its query total is 11,475, as reported in Table R10. Its collector SR is conditioned on this selected pool without validated teacher-data coverage and is not directly comparable to the earlier 42.5% full-pool collector diagnostic in Table 5.

Table R8 (= Table 20 in the revision). Teacher-specific collection and downstream deployment results.

TeacherSuccessful collection tasks / 807Collector SRReview callsCorrection callsTraining examples (expert + SRC)Deployed student SR / 1,620
Kimi-K2.513616.97,2504,2258,025 + 1,15835.0
Qwen3.8-max15118.79,6532,3748,025 + 1,663†34.8
GPT-6.1-sol17822.18,0363,7798,025 + 99532.3

These results support two observations:

  1. All three teacher-specific datasets improve the deployed student over the 25.3% Expert SFT baseline.
  2. Higher collection SR does not necessarily yield a better trained student. The teacher with the highest collection SR is not the one with the highest downstream SR.

Each student is trained from base weights on the expert demonstrations plus that teacher's SRC data. †The SRC addition is subsampled to match the student-success addition in Table R3; this is the same Qwen3.8-max model as in Table R3. The comparison does not isolate teacher capability: usable dataset sizes, query totals, and output-format compatibility differ. Consequently, it does not establish equal-budget efficiency, localization accuracy, or teacher insensitivity. The lower GPT-trained student result may reflect these data-processing differences, but no format-normalized ablation currently establishes the cause. We will retain that explanation as a hypothesis rather than a demonstrated finding.

AuthorsOpenReview ↗

Response to Reviewer ST2V (4/4): Generalization and Trajectory Lengths

Q7. Do the results demonstrate generalization to unseen tasks or applications?

The controlled experiments support task/template generalization within the evaluated setting; cross-application generalization has not been demonstrated.

  • WebArena-Infinity: the controlled experiment uses 334 collection tasks and 346 held-out tasks across six applications. Expert demonstrations plus SRC data achieve 13.29%, compared with 8.38% for Expert SFT.
  • OSWorld: the controlled supervision study separates 200 training templates from 125 held-out templates.
  • Main Table 1: collection and evaluation share task pools, so those gains measure adaptation and must not be presented as clean held-out generalization.

The WebArena-Infinity holdout is also harder than its collection set: approximately 83% versus 46% hard tasks. This helps contextualize the low absolute rates but does not justify comparing them directly with Table 1 or claiming unbiased out-of-distribution performance.

Q8. What are the trajectory lengths and their distributions?

Table R9 (= Table 21 in the revision). Actions per trajectory on WebArena-Infinity. Collection uses a 50-action cap; deployed rows use the 30-action evaluation cap (Eval. 1 in Table R6).

PopulationnMeanSDMedianP25P75P90MaxAt cap
Expert demonstrations (Expert SFT data)75010.78.1851321490
SRC-collected, Kimi-K2.5 teacher10812.86.61281621390
SRC-collected, Qwen3.8-max teacher14618.212.11492437503
SRC-collected, GPT-6.1-sol teacher15318.111.415102436507
Deployed Expert SFT, solved41011.26.48513213023
Deployed Expert SFT, failed1,21024.96.83016303030665
Deployed Expert SFT + SRC (Kimi-K2.5), solved56712.06.29614233032
Deployed Expert SFT + SRC (Kimi-K2.5), failed1,05325.36.83017303030683

SRC-collected rows count trajectories admitted without errors.

SRC-collected trajectories are longer than expert demonstrations (median 12–15 versus 8 actions). They contain detours followed by corrections and include harder tasks that the student rarely completes on its own, consistent with the SRC gains in Table R3 being largest on medium tasks. Deployed trajectories are bimodal: solved episodes are short (median 8–9 actions), whereas most failed episodes run until the 30-action cap (665 of 1,210 and 683 of 1,053 failures), so the overall mean mainly reflects the failure rate. Solved episodes of the Expert SFT + SRC policy are slightly longer (mean 12.0 versus 11.2 actions), consistent with its additional successes on medium tasks.


The strongest supported result is that teacher-guided rollback collection produces useful training data for an autonomous GUI policy, with controlled evidence for corrective-label value and within-application held-out performance. We will align the abstract, introduction, experimental claims, and limitations with that evidence, while clearly identifying the controls and measurements that remain incomplete.

We sincerely thank the reviewers again for their thoughtful feedback. We hope the clarifications and additional evidence above address your concerns, and we would appreciate your reconsideration of your assessment in light of these responses. Thank you for your time and consideration.

Reviewer_MV6hOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Speculative Rollback Correction for training GUI agents. A student executes several actions, and a teacher reviews the segment. When the teacher identifies a harmful action, the environment is reset, the accepted prefix is replayed, and the teacher provides a correction. Verified trajectories are then combined with Expert SFT data for training.

The resulting policy runs without a teacher, verifier, or resets at deployment. SRC improves over Expert SFT on WebArena-Infinity, WebArena-Lite, and an OSWorld subset, respectively.

Critical strengths
  1. SRC uses substantially fewer teacher queries than the LEAP-style baseline, while keeping teacher-assisted collection separate from teacher-free evaluation. Replay fidelity is explicitly measured, and the supplied code follows the described collection procedure.
  2. The OSWorld experiments compare corrections against accepted actions at matched data size and template coverage. Teacher-selected rollback also outperforms random rollback. The appendix reports negative and inconclusive results rather than only favorable comparisons.
Critical weaknesses
  1. The formal criterion defines harm as a reduction in the best achievable success probability (Section 3.1, Eqs. 1–3). However, the teacher also rejects reversible mistakes, unnecessary actions, and repeated navigation that need not reduce recoverability (Figure 4). This is broader than the paper's claim that rejection is reserved for actions that reduce task feasibility. Moreover, the method does not estimate the theoretical quantities V∗V^*, Q∗Q^*, or Δ\Delta, and Proposition 1 assumes deterministic transitions and knowledge of the true harmful action. The proofs therefore do not directly justify the implemented rule. The authors should clarify how the theoretical criterion relates to the actual procedure.

  2. The rollback study compares teacher-selected rollback against random rollback (40.3% vs. 25.8% on OSWorld; Appendix C.3), but lacks a control that always returns to the segment start. Therefore, it is unclear whether accurately locating harmful actions is better than simply discarding the entire segment. There is also no data-matched comparison using verifier-filtered student rollouts without review or rollback. In the controlled WebArena-Infinity experiment, adding only corrections to Expert SFT provides no significant improvement (p=0.2478p=0.2478), while adding all SRC data does (p=0.0359p=0.0359; Appendix C.2, Table 3). These missing controls make it difficult to determine whether the gains come from targeted correction or simply from additional successful training trajectories.

  3. Table 1 reports success rates without run counts or confidence intervals, despite the Reproducibility Statement promising repeated evaluations and bootstrap confidence intervals. Important training details for the OEC- and LEAP-style baselines are also missing, including whether they use Expert SFT data, their training data sizes, and update budgets. Table 5 reports teacher queries per collection task but not total collection budgets. Furthermore, the training-task pools used for Table 1 are not specified, making it difficult to verify the claimed separation between training and evaluation tasks. Although data leakage is not established, task-level split records and complete training details are needed to support fair comparisons and the claimed generalization improvements.

Questions for authors
  1. Which tasks produced the Expert SFT and SRC data for each benchmark, and how were evaluation tasks excluded?
  2. For each Table 1 method, please provide initialization, Expert SFT inclusion, verifier filtering, collection budgets, training-example counts, and update steps.
  3. How many runs underlie Table 1, and can the promised paired tests and confidence intervals be reported?
  4. How does the teacher’s reversible-mistake rule implement the recoverability criterion? What proportion of rejected branches still succeeds while retaining the flagged action?
  5. Can an always-rollback-to-segment-start control be added under the same collection and training budgets?
  6. Can Expert SFT plus an equal amount of verifier-filtered student data collected without review or rollback be evaluated?
6 comments
AuthorsOpenReview ↗

Response to Reviewer MV6h (1/6): Theory, Teacher Rule, and Student-Success Control

Thank you for identifying the gap between the formal criterion and the implementation, and for specifying the controls needed to interpret the empirical gains. We agree that these distinctions are essential to the paper's soundness.

W1 / Q4. How does the teacher's rule implement the recoverability criterion?

It is a heuristic proxy with additional efficiency preferences, not an exact implementation of Δ>0\Delta>0. The original wording was too strong.

The formal quantities are

\qquad \Delta(h,a)=V^*(h)-Q^*(h,a).$$ The distinction between the definition and its operational approximation is as follows: | Formal concept | Implemented mechanism | What the mechanism establishes | | :--- | :--- | :--- | | $V^*$ and $Q^*$ | No explicit estimator | No numerical estimate of optimal recoverability | | Positive harm, $\Delta>0$ | Teacher review of screenshots and actions | A fallible judgment about whether to intervene | | Earliest harmful step, $j^*$ | Teacher-selected position, $\hat j$ | A candidate rollback position | | Counterfactual intervention | Reset, replay the retained prefix, and apply a correction | An executed alternative continuation, subject to replay fidelity | | Successful outcome | Hard task verifier | Success of the completed trajectory under that verifier | **Reversible and unnecessary actions.** The formal problem has a finite horizon. An action that leaves the application unchanged can reduce recoverability when it consumes a necessary part of the remaining budget. However, a detour that leaves sufficient budget may have $\Delta=0$. The prompt can still reject such a detour as undesirable training behavior. We will therefore describe its rule as combining **suspected feasibility loss with efficiency-oriented filtering**, and remove the claim that every rejection identifies a strictly harmful action. **Role of the proofs.** The decomposition characterizes the formal harm process. Proposition 1 analyzes rollback under deterministic transitions and knowledge of the true harmful position. Neither result proves that a VLM identifies that position correctly. The uniqueness statement applies to the specified Doob decomposition; it does not establish a unique empirically identifiable causal explanation for each GUI failure. **Rejected branches that still succeed.** In the supplementary diagnostic, 600 rejected WebArena-Infinity branches were sampled from the Qwen3.8-max collection run, and 167 had an interior rollback position. Retaining the entire rejected segment and letting the student continue without correction produced **9/167 successes (5.4%)**. This answers the question for that selected subset, not for all rejected branches. It also does not estimate $V^*$: failure under one student continuation does not establish failure under every possible continuation policy. We will explicitly separate these operational observations from claims about optimal recoverability. ### W2 / Q5–Q6. Do the gains come from targeted correction or simply additional successful trajectories? **SRC outperforms adding more unaided student successes at matched training size and optimizer updates.** Table R3 addresses the proposed successful-trajectory control; the separate supervision ablations compare corrective labels with accepted student-action labels. #### (a) Verifier-filtered student trajectories without review or rollback Following the reviewer's request, both arms add the same number of examples (1,663) to the same expert demonstrations. One adds the Expert SFT student's own verifier-filtered successes, collected without review or rollback; the other adds SRC data. All policies are trained from the Qwen3.5-9B base weights under the Table 1 protocol. **How the additional data are collected.** The student-success collector keeps verifier-passing trajectories generated without intervention. Because unaided student rollouts on the 807 tasks without validated teacher-data coverage provide too few usable examples for this control, it collects on the 750 expert-covered tasks. SRC collects on the 807-task target pool: teacher review identifies where intervention is needed, rollback restores the correction state, and execution resumes after correction. This produces successful training trajectories on tasks the student rarely completes alone. We match the added data at 1,663 examples and train both policies with the same optimizer updates.
AuthorsOpenReview ↗

Response to Reviewer MV6h (2/6): Matched Student Data and Rollback Controls

Table R3 (= Table 14 in the revision). Training-data comparison on 1,620 WebArena-Infinity tasks. The student-success and Qwen-SRC arms have matched training sizes and optimizer updates. Difficulty columns report success counts.

Training data (from base weights)Training examplesSuccesses (SR)EasyMediumHard
Expert demonstrations only (Expert SFT)8,025410 (25.3)125105180
+ verifier-filtered student successes (no review or rollback)8,025 + 1,663449 (27.7)134133182
+ SRC data, Qwen3.8-max teacher8,025 + 1,663†564 (34.8)150220194
+ SRC data, Kimi-K2.5 teacher (Table 1)8,025 + 1,158567 (35.0)151211205

†The SRC addition is randomly subsampled to the size of the student-success addition, so both policies use 9,688 training examples and the same optimizer updates. Student-success examples come from 750 expert-covered tasks; SRC examples come from the 807 tasks without validated teacher-data coverage, as described above.

Adding the student's own verifier-filtered successes improves over Expert SFT by 2.4 pp (53:14 discordant tasks, exact paired p=1.8×10−6p=1.8\times10^{-6}). These 266 student successes average about 6.3 retained action examples per trajectory (1,663 examples), compared with 10.7 for expert demonstrations and 12.8–18.2 actions for SRC-collected trajectories (Table R9). With the added data matched at 1,663 examples and the same optimizer updates, SRC improves by a further 7.1 pp over student-success filtering (139:24, p=6.9×10−21p=6.9\times10^{-21}). The largest difference is on medium tasks (220 versus 133), with gains also on easy (+16) and hard (+12) tasks. Kimi-K2.5 SRC reaches a similar SR with only 1,158 added examples. SRC therefore provides a stronger training signal than collecting more trajectories that the student already completes unaided: it obtains verifier-passing supervision from tasks on which successful student rollouts are sparse.

(b) Teacher-selected versus segment-start rollback

Appendix C.3 provides an end-to-end comparison against random rollback, controlling the teacher, student, collection budget, training schedule, and review horizon:

Rollback ruleDeployed SR on 186 OSWorld tasks
Random position within the segment25.8
Teacher-selected position40.3

This supports the practical value of teacher-guided rollback over random placement, but does not establish superiority over always returning to the segment start.

At K=3K=3, the teacher already chooses the segment start in 82.3% of full-length rejected WebArena-Infinity branches and 71.5% of OSWorld branches. The two rules differ only on the remaining branches. In the supplementary WebArena-Infinity collection run with Qwen3.8-max as teacher (Table R8), we sampled 600 branches uniformly from all rejected branches; 433 (72.2%) rolled back to the segment start and 167 had an interior teacher-selected position. Replaying these 167 branches under three rules, holding the teacher and student fixed, gives: teacher-selected rollback 27/167 (16.2%), forced rollback to the segment start 19/167 (11.4%), and retaining the rejected branch without correction 9/167 (5.4%). Paired exact two-sided McNemar tests give p=0.0386p=0.0386 for teacher-selected versus segment-start rollback (10:2 discordant branches) and p=0.0039p=0.0039 for teacher-selected rollback versus retaining the branch (27:9). This is a collection-stage diagnostic on a selected subset, supporting the value of prefix retention; it is not a substitute for training and evaluating both complete collectors under matched budgets.

AuthorsOpenReview ↗

Response to Reviewer MV6h (3/6): Evidence Scope, Task Overlap, and Generalization

(c) What do the complementary comparisons establish?

QuestionExisting resultInterpretation
Are corrective labels more useful than accepted-action labels under matched data conditions?OSWorld: +6.13 / +8.27 pp; WebArena-Infinity aligned-format test: adjusted p=0.0259p=0.0259Supported in these evaluated settings
Does adding all SRC data to expert demonstrations improve the controlled held-out result?13.29% versus 8.38%; reported paired p=0.036p=0.036Supported
Is correction-only augmentation sufficient?6.36% versus 8.38%; no detected improvementNot supported
Does localization beat random rollback end to end?40.3% versus 25.8% on OSWorldSupported
Does localization beat always-start rollback end to end?No complete matched deployment comparison reportedNot established
Does SRC data beat an equal amount of student successes added to the expert demonstrations?1,663 added examples each: 34.8% vs 27.7%; 139:24, p=6.9×10−21p=6.9\times10^{-21} (Table R3)Supported at matched training size and optimizer updates

Together, these comparisons support collecting useful supervision beyond unaided student successes, training on corrective labels, and selecting effective rollback positions.

W3 / Q1. Were training and evaluation tasks separated?

The separation differs between the main table and the controlled experiments. The blanket statement in Section 5.1 that no policy was trained on its evaluation tasks is incorrect for Table 1 and must be corrected.

Table R4 (= Table 11 in the revision). Evaluation protocols and the conclusions they support.

ExperimentTask relationshipAppropriate interpretation
Table 1: WebArena-Infinity, 1,620 tasksSRC targets 807 tasks without validated teacher-data coverage within the evaluation pool; 816 tasks are represented in the combined Expert/SRC training dataShared-pool adaptation performance
Table 1: WebArena-Lite, 165 tasksPool selected from Expert SFT's Eval-1 failures: 165 − 34 = 131 tasksOutcome-selected adaptation performance
Table 1: OSWorld, 186 tasksPool selected from Expert SFT's Eval-1 failures: 186 − 51 = 135 tasksOutcome-selected adaptation performance
Appendix C.2: WebArena-Infinity334 collection tasks and 346 held-out tasks across six applicationsTask-level generalization within those applications
Controlled OSWorld supervision study200 training templates and 125 distinct held-out templatesTemplate-level generalization in the evaluated setting

For transparency, the supplementary task-coverage breakdown of the main WebArena-Infinity result is:

Table R5 (= Table 12 in the revision). Post-hoc coverage analysis of Table 1.

Evaluation subsetTasksExpert successesSRC successesGainPaired 95% CI for gainExact paired pp
Represented in training data816267 (32.7%)404 (49.5%)+16.79 pp[13.60, 19.98]3.04×10−243.04\times10^{-24}
Not represented in retained training data804143 (17.8%)163 (20.3%)+2.49 pp[0.37, 4.60]0.0308
All tasks1,620410 (25.3%)567 (35.0%)+9.69 pp[7.78, 11.67]3.28×10−223.28\times10^{-22}

Statistical details: These calculations use the supplied paired outcome counts: SRC-only versus Expert-only successes are 167:30, 49:29, and 216:59, respectively. Intervals use 20,000 paired task-bootstrap resamples reconstructed from the four outcome categories, with seed 20261115. The pp-values are two-sided exact McNemar tests. They are nominal, without adjustment across these overlapping subsets, and conditional on the reported outcome counts.

The 804-task subset is not a prospectively excluded holdout: it includes tasks attempted during collection that produced no retained training data. Its difficulty and selection mechanism differ from a clean held-out test. We will not call it an unbiased or conservative estimate of generalization.

The stronger evidence for task generalization is Appendix C.2: 46/346 versus 29/346 successes on tasks excluded from collection, with the original reported paired significance result p=0.036p=0.036. These results come from separately trained models and a different protocol, so their absolute rates should not be compared directly with Table 1. Cross-application generalization remains untested.

Reviewer_8YuyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper considers interactive imitation learning for multi solution GUI agents to focus on task recoverability as opposed to disagreement with expert behavior, pairing student rollouts with environment rollback, expert (teacher) localization, hard verifier filtering to learn an appropriate policy. For this, training requires sandbox environments with resets, a verifier, and a VLM based teacher (Kimi K2.5) with student being a Qwen 3.5-9B model. Through defining notion of an action harm, the student proposes three unmonitored steps, the teacher accepts a block or pinpoints a specific issue with a corrective action. Using these multi-leaf trajectories along with some filters the paper proposes to learn a simple SFT based policy with the resulting samples.

Critical strengths
  1. The method confines all the heavy lifting (resets, verifiers, teacher queries) to training and the inference is just playing the student policy.
  2. Presents interesting and improved practical results while also showing several interesting negative ablations in the appendix.
Critical weaknesses
  1. Somehow, despite introducing the notion of action harm, the paper eventually ends up prompting a teacher policy which might reintroduce all issues with a teacher's potential blindspots.
  2. In general, the notion of reset based access seems to limit the potential of this class of approaches.
  3. It also appears interesting that training just on SRC or expert + correction still underperforms compared to expert sft on controlled web arena split and moreover query counts ignore multi-image VLM tokens and replay wall clock costs.
Questions for authors

None, other than the weaknesses raised.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8Yuy (1/3): Teacher Bias and Reset Access

Thank you for recognizing the practical improvements and the separation between training-time assistance and autonomous deployment. We agree that teacher dependence, reset access, and collection cost require more careful qualification.

W1. Does prompting a teacher reintroduce its blind spots?

Yes, teacher judgments remain fallible. The teacher-independent definition of harm does not make the implemented collector teacher-independent. We will make this distinction explicit.

The formal definition uses the best achievable continuation value, whereas the implementation uses a teacher to approximate the intervention decision:

ComponentRoleLimitation
Optimal recoverability, V∗V^*Defines success potential over all continuation policiesNot estimated by SRC
Teacher reviewAccepts a segment or selects a rollback positionCan miss delayed errors or reject valid alternatives
Teacher correctionSupplies a local action from the restored observationMay itself be suboptimal
Hard verifierFilters completed trajectories according to task successDoes not certify each localization decision

A teacher's actual continuation success probability is bounded above by V∗V^*; its verbal judgment is not automatically a calibrated lower-bound estimate. Likewise, the teacher-suboptimality term in Appendix A is an analytical quantity, not a quantity measured by our collector.

SRC nevertheless differs from training only on teacher demonstrations in three concrete ways:

  1. The student generates most of the data that SRC adds. On WebArena-Infinity, the SRC policy is trained on 8,025 expert next-action examples plus 1,158 SRC-collected training examples. Of these 1,158 examples, 959 (82.8%) are student actions accepted by the teacher and only 199 (17.2%) are teacher corrections. Relative to Expert SFT, teacher judgments therefore enter the added data mainly as accept decisions and a small number of local corrections. (The 7,882 / 1,301 split in Appendix C.1 was computed incorrectly over the combined training set; we will correct it.)
  2. Different valid paths may be retained. The review prompt explicitly discourages rejection solely for different search terms, stylistic choices, or alternative orders of independent subtasks.
  3. Terminal admission depends on the task verifier. Teacher approval alone is insufficient to admit a completed trajectory.

These mechanisms reduce dependence on reproducing a particular teacher trajectory, but they do not eliminate teacher bias. False rejection can reduce trajectory diversity and coverage even when all retained trajectories pass the verifier.

The supplementary teacher comparison also shows that the benefit is not confined to one teacher:

SRC teacherDeployed student SR on 1,620 WebArena-Infinity tasksGain over Expert SFT
No SRC: Expert SFT25.3—
Kimi-K2.535.0+9.7 pp
Qwen3.8-max34.8+9.5 pp
GPT-6.1-sol32.3+7.0 pp

The teacher-specific datasets have different usable sizes, so this is a robustness observation, not an isolated estimate of teacher quality. Our claim is that SRC can produce useful student training data with several teachers, not that it removes their blind spots.

W2. Does reset-based access limit applicability?

Yes. SRC requires a resettable training environment; it does not require resets during deployment.

RequirementData collectionDeployed policy
Teacher review and correctionRequiredNot used
Reset-and-replayRequired for rollbackNot used
Task verifierRequired for trajectory filteringNot used for action selection
Next-action student policyUsedUsed

This makes SRC applicable to sandboxed GUI training environments with recoverable initial states. The measured state agreement is 93.6% on WebArena-Infinity and 97.4% on OSWorld; the remaining mismatches are a real limitation, discussed in our response to Reviewer ST2V.

For environments with irreversible external effects, the intended use is training in a resettable sandbox followed by ordinary policy deployment. Transfer from such a sandbox to an irreversible real-world environment is not established by the present experiments, and we will state this explicitly.

AuthorsOpenReview ↗

Response to Reviewer 8Yuy (2/3): Negative Ablations and Corrective Labels

W3a. Why do SRC-only and Expert + corrections underperform Expert SFT?

Thank you for highlighting these negative results. The controlled experiment is on WebArena-Infinity and uses the same recipe as Table 1: expert demonstrations plus SRC data. It supports this recipe rather than the claim that correction-only training is sufficient.

Table R1 (= Table 3 in the revision; selected rows). Controlled evaluation on 346 held-out WebArena-Infinity tasks, reproduced from Appendix C.2.

Training dataSuccessesSR
Expert SFT298.38
SRC only82.31
Expert + corrections only226.36
Expert + all SRC data4613.29

Three conclusions follow:

  • Adding all SRC data to the expert demonstrations improves over Expert SFT: +4.91 pp, with the original reported paired significance result displayed as p=0.036p=0.036. The 46/346 versus 29/346 success counts and the evaluation protocol are unchanged.
  • Correction-only augmentation does not show an improvement: the exact two-sided McNemar test gives p=0.2477886p=0.2477886 (10 correction-augmented-only versus 17 Expert-only successes), and the point estimate is lower. This tests whether the paired success probabilities differ in either direction; it provides no detected improvement and does not establish equivalence. Different update counts also prevent interpreting this as a clean comparison of data composition alone.
  • SRC-only training does not replace broad expert supervision in this controlled setting. This arm contains only SRC data from tasks that Expert SFT failed and SRC then solved; it is small and concentrated. The main Infinity Table 1 recipe instead selects the 807 tasks not covered by the validated teacher-data pool (1,620 minus 813 covered task IDs after source-quality filtering), and retains the expert demonstrations during training. The controlled experiment uses a separate, prospectively held-out task split, so its absolute rates are not directly comparable with Table 1.

The cleaner evidence for corrective-label value comes from the matched comparisons against accepted student actions: +6.13 pp under sampling and +8.27 pp under greedy decoding on OSWorld, with reported 95% confidence intervals of [1.1, 11.2] and [2.1, 14.7], respectively. On WebArena-Infinity, the aligned-format comparison has multiplicity-adjusted p=0.0259p=0.0259, while the original-format comparison is inconclusive.

We will therefore distinguish the value of corrective labels under matched comparisons from the effectiveness of the full Expert + SRC training recipe. Neither result establishes that correction alone explains the entire deployment gain.

AuthorsOpenReview ↗

Response to Reviewer 8Yuy (3/3): Image-Token and Replay Costs

W3b. Do query counts conceal image-token and replay costs?

Yes. Query count is an incomplete cost measure, and fewer queries do not by themselves establish lower token cost or wall-clock time.

Table R2 (query statistics: Table 5; request structure: Appendix D.6 in the revision). Reported training-time teacher queries per collection task.

Collection methodWebArena-InfinityWebArena-LiteRelevant request structure
OEC-style random switch10.0910.15One screenshot per teacher action request
SRC14.2211.90Up to 9 screenshots per review and 6 per correction
LEAP-style post-hoc correction27.4228.64Full-trajectory context in the post-hoc review request

The SRC increment used by the main WebArena-Infinity policy comes from the 807 tasks without validated teacher-data coverage in Table R8, with 7,250 review + 4,225 correction = 11,475 teacher queries, or 14.22 per collection task. The selection is based on the union of validated Kimi/Qwen teacher-data coverage, not on the deployed Expert SFT checkpoint: 813 covered task IDs are excluded from the 1,620-task pool. Lack of a valid retained teacher trajectory need not mean that no teacher ever solved the task. The earlier 12.01-query average and 42.5% collector SR describe a different diagnostic run and must not be used as the cost or collector SR of this training-data run. OEC and LEAP use the broader collection pools listed in R10, so the query averages above are not matched-pool efficiency comparisons.

For the measured Kimi-K2.5 configuration, one 1920 × 1080 screenshot contributes 2,696 input tokens. Image-token accounting uses the screenshot count for each request:

Timage=2,696∑q=1Qmq,T_{\mathrm{image}}=2{,}696\sum_{q=1}^{Q}m_q,

where mqm_q is the number of screenshots in request qq. Applying the reported caps of nine screenshots per review and six per correction to the actual Infinity run gives an image-only upper estimate of

\approx 0.303\text{ million input tokens per collection task}.$$ For WebArena-Lite, using nine screenshots for every SRC request and the reported 11.90 queries per collection task gives an upper estimate of 0.289 million image tokens/task. These are cap-based estimates, not measured token usage; text inputs and generated outputs must be added separately. For LEAP-style collection, multiplying every teacher query by full trajectory length is justified only if every request actually includes that context. OEC-style requests are smaller and can be cheaper in image tokens. Query counts from different collection pools do not establish matched-pool efficiency or a reduction in total cost. **Replay timing and overhead (= Table 19 in the revision).** Replay adds a separate overhead: | Environment | Mean reset-and-replay time per rollback | Estimated mean replay overhead per collection task | | :--- | ---: | ---: | | WebArena-Infinity | 9.3 s | At most approximately 67 s under the query-based accounting below | | OSWorld | 74 s | At most approximately 3.3 min under the same accounting | The estimates use at most $Q/2$ rollbacks per collection task, because each rollback requires a review query and a correction query; Infinity uses the actual run's $Q=11{,}475/807$, and OSWorld uses the reported $Q=5.29$. They are **aggregate replay-overhead estimates**, not per-task worst-case latency guarantees or end-to-end collection times. They exclude student inference, teacher latency, verification, and training. We will separate these cost categories and avoid describing SRC as uniformly cheaper than all baselines.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response (1/3): Overview and New Evidence

We thank the reviewers for the time and care devoted to evaluating our work and for their constructive feedback. Your comments have helped us identify where the presentation, experimental protocols, and scope of our claims need clarification. Below, we address your concerns point by point and present additional evidence to support our responses.

We also appreciate the reviewers' recognition of the motivation (ST2V), the separation between training-time assistance and autonomous deployment (8Yuy, MV6h), and the controlled comparisons and negative results reported in the appendix (8Yuy, MV6h, ST2V).

What SRC is and what the theory does. SRC is a training-time data-collection procedure. The student executes a short segment; a teacher accepts it or selects a rollback position; the environment is reset, the accepted prefix is replayed, and the teacher supplies one local correction; only verifier-passing trajectories are used for training. Its purpose is to obtain additional successful student-induced trajectories on selected tasks without usable expert supervision, while allowing multiple valid solution routes. The deployed policy is a standard next-action model that uses no teacher assistance, verifier feedback, or resets during execution. Benchmark verifiers score final task success. The recoverability formulation defines what the teacher is asked to judge and motivates two design choices: correcting an action because it reduces recoverability rather than because it disagrees with a reference, and rolling back to the earliest harmful step while keeping the preceding prefix. It is not an estimator of V∗V^* and does not certify the teacher's judgments. We will state this explicitly and map each theoretical quantity to its implementation (response to MV6h, W1).

New evidence in this response.

  • Correction versus additional successful data (MV6h). SRC outperforms adding more unaided student successes when training size and optimizer updates are matched. Adding 1,663 verifier-filtered student-success examples to the expert demonstrations gives 27.7% on WebArena-Infinity; adding 1,663 SRC examples gives 34.8%, a further 7.1 pp (139:24 discordant tasks, p<10−20p<10^{-20}; Table R3). To obtain enough successful examples, the student-success collector uses 750 expert-covered tasks. SRC targets the other 807 tasks without validated teacher-data coverage, where unaided student rollouts yield too little usable supervision. Review, rollback, and correction turn student-induced errors on these tasks into verifier-passing training trajectories. The largest gain is on medium tasks (220 versus 133 successes).
  • Sensitivity to the teacher (8Yuy, ST2V). SRC data collected with each of three teachers improves the deployed student over Expert SFT: 35.0, 34.8, and 32.3 versus 25.3 (Table R8).
  • Task overlap and generalization (MV6h, ST2V). In Table 1, collection and evaluation share the same task pool; Lite and OSWorld collection pools are selected from Expert SFT's Eval-1 failures, giving 165 − 34 = 131 and 186 − 51 = 135 tasks, respectively. These are outcome-selected adaptation results. We will state this explicitly in Section 5.1 and Table 11. On the 804 WebArena-Infinity tasks not represented in the training data, SRC still improves by 2.5 pp (49:29 discordant pairs, exact McNemar p=0.031p=0.031; Table R5). On the controlled split, where held-out tasks were fixed before collection, SRC achieves 13.29% versus 8.38% (reported paired p=0.036p=0.036).
  • Uncertainty, cost, and replay (8Yuy, MV6h, ST2V). We report three repeated evaluations per checkpoint (range ≤ 0.6 pp), task-level confidence intervals and paired tests, screenshot-aware token-cost estimates, and measured reset-and-replay times (9.3 s per rollback on WebArena-Infinity, 74 s on OSWorld).
AuthorsOpenReview ↗

General Response (2/3): Claim Scope and Statistical Corrections

Claims we will narrow.

ConcernClaim we supportClaim we withdraw or qualify
Theory vs. implementationA theoretically motivated procedure; the teacher's review is a heuristic proxy for harmThat the collector estimates V∗V^* or that every rejection is strictly harmful
Correction vs. more dataSRC outperforms extra student successes at matched training size; separate supervision ablations test corrective-label valueDistinguish the collection-strategy comparison from the label-only ablation
GeneralizationHeld-out tasks within evaluated applicationsTable 1 as a held-out result; cross-application transfer
Reliability and costTraining-time benefit under resettable sandboxes, with measured replay fidelity and costUniformly lower cost than all baselines; transfer to irreversible environments

Summary of revisions. Changes are marked in red in the revised manuscript and grouped below by their purpose.

A. Corrections to reported statistics and collection scope

  • Student-action / correction attribution (Appendix C.1, Table 2): 7,882 / 1,301 → 959 / 199. The old numbers summed to the entire 9,183-example expert-plus-SRC export and were incorrectly presented as the composition of SRC-generated supervision. This was a reporting-scope error: the accepted-student-action versus correction split applies to the final 1,158-example SRC increment, comprising 959 accepted student-action labels and 199 corrections (82.8% / 17.2%). The 8,025 expert examples and the combined training size of 9,183 are unchanged; we withdraw the earlier combined-set attribution rather than infer a new attribution for the expert data.
  • Trajectory / action / example units (Appendix C.1, Table 2; Appendix D.8, Table 21): the old 977-trajectory export description → separate collection and training counts. The revised account distinguishes 750 expert training trajectory records, 108 admitted SRC collection trajectories containing 1,382 actions (mean 12.8), and the 1,158 final SRC training examples. The earlier description mixed collection trajectories with processed training records. This is an accounting clarification; the training set remains 8,025 + 1,158 examples.
  • Infinity collector SR (Table 5; Appendix D.7, Table 20): 42.5% → 16.9% for the pool supplying the main SRC increment. The earlier broader-pool diagnostic included expert/SFT-covered tasks. The revised statistic is 136 / 807, conditioned on the selected pool without validated teacher-data coverage; the previous report did not distinguish these collection scopes.
  • Infinity teacher queries per collection task (Table 5; Appendix D.2, Table 13): 12.01 → 14.22. The old average came from the broader diagnostic scope. The main training-data run instead uses 7,250 review + 4,225 correction = 11,475 calls over 807 tasks, giving 14.22 calls/task. Calls count teacher API requests across the complete collection run, including all tasks and branches.
  • Query-reduction claim (Section 5.4; Appendix D.6): “56–58% fewer queries” → removed. That percentage used earlier broader-pool statistics and is not a matched-pool efficiency comparison for the selected 807-task training-data run. The revision reports run-specific query counts and separates image-token and replay costs.
  • Infinity SRC collection scope (Section 5.1; Tables 11, 13, and 20): an ambiguous full-1,620-task description → the actual 807-task collection pool. Quality screening leaves 813 task IDs with validated Kimi/Qwen source coverage; SRC targets 1,620 − 813 = 807 remaining tasks. This corrects the description of the data collection already used for Table 1, rather than introducing a new collection configuration.

Table 1 main results are unchanged.

AuthorsOpenReview ↗

General Response (3/3): Clarified Scope and Revision Locations

B. Narrowed claims and clarified scope

  • Theory and implementation (Sections 3–4; Appendix A): recoverability supplies the formal target and idealized rollback rationale. Teacher review is a fallible proxy; it does not estimate V∗V^* or certify the earliest harmful action. Lemma 1 and its proof are in Appendix A.
  • Evaluation-task selection (Section 5.1; Appendix D.1, Table 11): Lite and OSWorld collection pools are explicitly selected from Expert SFT's Eval-1 failures, giving 165 − 34 = 131 and 186 − 51 = 135 tasks. These are outcome-selected adaptation results; separate controlled held-out protocols support within-application task/template generalization.
  • Student-success control (Section 5.3; Appendix D.3, Table 14): both arms add 1,663 examples and use the same optimizer updates. The student-success collector obtains enough examples from 750 expert-covered tasks; SRC uses review, rollback, and correction on the 807-task target pool, where unaided successes are sparse. The resulting 7.1 pp advantage shows the benefit of obtaining supervision from tasks the student rarely completes on its own.
  • Uncertainty (Section 5.1; Reproducibility Statement; Appendix D.5): three evaluations reuse each fixed checkpoint and do not measure training-seed variability. The Lite gain remains statistically inconclusive.
  • Reliability and cost (Sections 5.4 and 6): replay agreement and teacher-query counts support statements about the measured sandbox setup. They do not establish exact restoration, uniformly lower total cost, or transfer to irreversible environments. Cross-application transfer remains untested.

C. Additional experiments, measurements, and analyses

  • Verifier-filtered student-success control: Section 5.3 and Appendix D.3, Table 14 (Table R3 below). The student-success and Qwen-SRC additions each contain 1,663 examples and use matched training updates.
  • Teacher-specific collection and deployment comparison: Section 5.4 and Appendix D.7, Table 20 (Table R8).
  • Selected-branch continuation diagnostic: Section 5.3 and Appendix D.4, Table 15, comparing teacher-selected rollback, segment-start rollback, and retaining the rejected segment on the same 167 interior-flagged branches.
  • Repeated fixed-checkpoint evaluations: Appendix D.5, Table 16 (Table R6), with Wilson intervals and paired tests in Tables 17–18. These add evaluation and task-level uncertainty reporting, not additional training seeds.
  • Post-hoc training-coverage and trajectory-length analyses: Appendix D.1, Table 12 (Table R5), and Appendix D.8, Table 21 (Table R9). These are analyses of the reported runs, distinct from new controlled experiments.
  • Collection/training ledger and replay-cost measurements: Appendix D.2, Table 13 (Table R10), and Appendix D.6, Table 19. The ledger distinguishes collection tasks, teacher requests, source records, training examples, and optimizer steps.

All success rates are percentages and pp denotes percentage points. Collector SR measures teacher-assisted data collection; deployed SR measures autonomous student execution.

AuthorsOpenReview ↗

Follow-up on Author Responses and Revised Manuscript

Dear Reviewers, thank you again for your valuable comments. With the discussion period nearing its end, we wanted to gently follow up on our point-by-point responses and the revised manuscript (with updates highlighted in red). Could you kindly let us know if our clarifications and additional evidence have addressed your main concerns, or if there are any remaining points we can clarify before the deadline? Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.