- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper defines a Minimal Repair Subtrajectory (MRS): the shortest contiguous span of original steps in a failed agent trace such that replacing it with actions from a fixed candidate library, and then replaying the remaining suffix, makes the task pass an external verifier. It distinguishes a local-recovery certificate from an exact MRS certificate. A training-free doctor–patient–verifier procedure searches for such repairs: the doctor proposes anchors and replacements from the failed trace and the verifier's unmet conditions, and the verifier checks candidates in order of increasing span length. Experiments cover an author-built controlled dependency benchmark, a strict-replay subset of τ², three public environments (ToolSandbox, AppWorld, ALFWorld), a router that chooses between local repair and restart, and the reuse of MRS segments as in-context demonstrations and fine-tuning data.
Critical strengths
- The paper isolates repair extent as its own question and defines it cleanly. Most prior work asks which step caused a failure or simply retries the whole episode; this paper asks how much of a failed trajectory must change for the task to succeed. This angle is clearly distinct from failure attribution and from full retry, and the analogy to delta debugging is apt. The paper is explicit that minimality is relative to a declared candidate library, and it separates "a repair was found" from "the repair is provably minimal", which guards against over-reading its results.
- Recovery is judged by executable verifiers and reported transparently. Every recovery is decided by the environment's own executable verifier rather than by model self-assessment. Sample filtering and denominators for the public environments are disclosed, comparisons use paired tests on the same failures, and negative results are reported: local repair loses to simple rollback or full restart in some environments. Rather than hiding these, the paper turns them into a routing rule based on whether the prefix can be trusted.
Critical weaknesses
- The main controlled findings follow largely from the definition and setup, and the public experiments do not execute MRS. MRS is defined as the shortest passing span and is computed with oracle replacements on failures that are recoverable by construction, so its compactness advantage holds almost by definition. The high share of multi-step repairs also depends on the authors' generator and on replaying the recorded suffix verbatim, which forces downstream actions that consumed corrupted outputs into the span. In the public environments, the suffix is regenerated from a checkpoint, so the method reduces to choosing a rollback point. The authors should report span widths with an adaptive suffix and run bounded-span repair directly in the public environments.
- The method relies on the official verifier at repair time, and its gains over simple baselines are not established under matched conditions. The doctor sees the verifier's unmet conditions and uses its verdict to accept candidates, and the method needs a checkpointable, deterministic environment; both are rarely available in deployment. Even so, a short fixed rollback matches or beats the doctor on AppWorld, full restart wins on ALFWorld, no doctor-versus-retry test is significant, and it is unclear whether the two get the same number of verifier-checked attempts. The authors should add an attempt-matched retry baseline and report the doctor without verifier feedback.
Questions for authors
- In the public environments, how many original positions does the executed repair actually replace (from the checkpoint to the end of the trajectory), and how does that compare with the MRS width found by the post-hoc audit?
- In Table 3, how many verifier-checked attempts do the doctor, fixed rollback, and full retry each receive? Does the doctor use the verifier's pass/fail signal to choose among its candidates?