- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to construct trajectories for supervised fine-tuning of language-model agents. Repair SFT uses a black-box teacher to revise a student's trajectory while retaining valid behavior. When a tool call changes, the authors execute it in the environment and regenerate the dependent continuation so that actions and observations remain consistent. Hybrid SFT chooses between repaired trajectories and direct teacher demonstrations based on training-task outcomes and role-specific rules.
The authors evaluate Qwen2.5-14B and Qwen3.5-9B in a shared memory harness on retrieval-based QA and interactive tasks, with transfer tests on IIRC and BFCL. Repair SFT generally increases recorded completion and reduces budget exhaustion. Hybrid SFT improves QA F1 over Direct SFT for both students. The results also show that completing more trajectories does not consistently improve task performance.
Critical strengths
-
Repair SFT handles a real problem in trajectory editing: changing a tool action can invalidate the observations that follow it. Executing the changed call and regenerating its continuation makes the resulting training example more coherent. It also allows the authors to use black-box teachers without teacher probabilities or aligned tokenizers.
-
I appreciate the paper's separation of execution behavior from task success. It reports results for two students across several task families and breaks QA F1 into answer coverage and conditional answer quality. Qwen2.5 Repair SFT, for example, has nearly complete MuSiQue answer coverage but low F1. The reporting of budget exits, infrastructure failures, and uncertainty makes the completion numbers easier to interpret.
Critical weaknesses
- The experiments leave several parts of the proposed supervision recipe entangled. Hybrid SFT changes outcome-based selection, role-specific source choices, memory-action balance, and target lengths together. Table 11 reports 3.879 million supervised tokens for Qwen3.5 Direct SFT and 2.259 million for Hybrid SFT, despite matched instance counts and optimizer steps. A random Direct/Repair mixture matched for supervised tokens and role composition would help show what the selection rules contribute.
The case for retaining student behavior also needs more direct evidence. Repair SFT is not compared with teacher regeneration under a comparable verification and retry budget, and the paper does not quantify how much of a student trajectory survives repair, how extensive the edits are, or what errors are identified. As presented, the results support the full construction recipes more clearly than the specific value of repairing student errors. I would also like a sharper distinction from AgentRefine (ICLR 2025), which trains on error-and-refinement trajectories but uses synthesized interactions rather than repairs of frozen student trajectories executed in real environments.
- I am less confident in comparisons drawn from the historical evaluation records. Appendix E says that reruns may replace per-item records, attempt counts cannot be checked for every item, and the available records do not establish common output-token limits or timeouts across historical runs. These details matter when answer coverage and budget exhaustion are central outcomes. Appendix G's sensitivity analysis is helpful: Hybrid SFT remains ahead even if failed Direct SFT QA records are assigned perfect F1. That bound does not address differences in retry policies or generation limits, however. The bootstrap intervals also condition on retained outcomes and omit retry and training variability. A controlled comparison with documented generation limits and a uniform retry policy would make the policy differences easier to assess.
Questions for authors
-
Does Hybrid SFT retain its QA advantage over a random Direct/Repair mixture matched for supervised tokens, role composition, and Step Tree memory-action balance? This would help separate the selection rules from the benefit of having access to both sources.
-
Can the historical logs confirm identical output-token limits, timeouts, and retry policies for the main Qwen3.5 comparisons? If first-attempt outcomes cannot be recovered, could you rerun a fixed QA subset under common settings and report F1, answer coverage, and budget exhaustion?
-
How much of each student trajectory remains after repair? How do you identify the earliest causal error, and what verification criteria and retry limits govern acceptance? Repair statistics or a comparison with teacher regeneration under the same verification budget would help substantiate the benefit of retaining student behavior.