- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how an agent can learn from corrections proposed after a failed attempt. A correction may help an externally assisted agent finish a task without teaching the policy to do so independently. RePair tests proposed repairs against baseline continuations from the same state, turns beneficial repairs into action and strategy supervision, and evaluates the trained policy with the assistance removed. In ALFWorld, four successful repair programs provide eight training examples for Qwen2.5-3B. The resulting update leaves selection success unchanged at 3/355 tasks. The paper uses this negative result to examine the gap between verified repairs and learned behavior.
Critical strengths
Useful finding: A verified repair can help an agent finish a task without teaching the policy to reproduce it. The paper documents why this happens: the dual-view update supervises only 4 of the 16 verified action positions, so the later steps remain a generalization demand.
Critical weaknesses
Missing controls: The four repair programs contain 16 action positions, but training covers only their initial actions and strategy descriptions. Without action-only or full-sequence supervision controls, the proposed coverage explanation for failed transfer remains untested.
Unclear local learning: Both policies solve only 3/355 tasks, with most episodes ending after consecutive nonexecuted attempts. Without post-training evaluation from the repaired states, unchanged task-reset success cannot distinguish failure to learn a repair from failure to reach its starting state.
Questions for authors
-
On the same four sources and training budget, how does the dual-view update compare with action-only training and with supervision covering the complete repair sequences?
-
Does the trained policy improve action validity or continuation success when started from the verified source states or their intermediate states?
The stronger-checkpoint experiment gives a positive answer at the repair decision. Starting from a common Qwen2.5-3B checkpoint that solved 81/355 selection tasks, we matched the update budget and exposure to new action targets for RePair and direct teacher-trajectory SFT. After training, we placed each policy at the same ten observed repair states with memory and the repair program disabled. RePair completed 5/10 and 6/10 across the two training seeds. The parent completed 2/10, and direct teacher SFT also completed 2/10 in both fits. The same three tasks that the parent failed were solved independently by both RePair fits (Table 6, p. 9).
The gain was visible in both action choice and task completion. The first executed action matched the verified repair in 7/10 states in each RePair fit, compared with 2/10 for the parent and 1/10 in each teacher fit. In the run that solved six tasks, one successful continuation began with a different action from the demonstration. Any successful continuation counted. Complete teacher trajectories improved the policy’s task-start score in both runs, while the verified repairs produced more recoveries specifically at the decisions where this policy had failed. At the later verified midpoint, both repair fits scored 7/10 against the parent’s 8/10. Evaluating at each starting state shows exactly where the gain occurred (§3.4, pp. 8–9).
For the four sources in the submission, the requested first-action, full-sequence and strategy controls are in Table 5 (p. 8). All four fits completed 0/4 from the repair state and 2/4 when given the first program action. The first attempt executed on 2/4 tasks for both the parent and each fit. All sixteen reset runs reached the original decision, and none of the fifteen runs with matching public input executed the demonstrated first action. This explains why the original task-start score alone could not locate the failure. The stronger-checkpoint result shows that verified repair training can improve autonomous recovery at those kinds of decisions.