- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
HARP trains separate Planner, Executor and Repairer policies, composes them using a phase classifier and a progress verifier, and distils the routed teacher into a standalone Qwen3 model on student-generated trajectories. Executor training includes verifier-based reward, Planner training uses terminal reward, and the Repairer learns from replay-filtered corrections. On AppWorld, the distilled model outperforms the corresponding base model at all three tested sizes and GRPO at 1.7B and 4B, but not 8B. Component results show a substantial 4B Test-N improvement from adding the Repairer to the routed system, while repair gains and their retention after distillation are inconsistent across sizes and splits.
Critical strengths
-
Organising training-time teachers around corrective responsibilities is a meaningful systems contribution. The approach explicitly separates the richer training harness from the single-model deployment interface, and the measured standalone improvements show that some benefit survives that separation. At 1.7B, HARP reaches 13.1%/3.8% TGC on Test-N/Test-C versus 0.6%/0.0% for GRPO.
-
The paper reports limitations unusually concretely: the 8B baseline reversal, the 4B teacher-to-student performance loss, the tiny repair dataset and the verifier's weak regression recall are all visible. This makes the empirical scope assessable and avoids conflating verifier improvement with independently measured task success.
Critical weaknesses
-
The central role-specialisation and repair contribution is not isolated in the final model. HARP differs from GRPO in supervised initialisation, verifier feedback, expert training, repair data and distillation. Table 2 tests routed components, but does not compare final students distilled from matched two-role and three-role teachers or a non-specialised teacher using the same data. Moreover, adding the Repairer also changes the handoff rules from Router2 to Router3. The 4B Test-N teacher gain from 20.2% to 27.4% becomes 19.6% after OPD, and other repair gains are small or negative. Thus the evidence does not yet establish that repair-aware role routing, rather than the other training additions, drives the deployed benefit.
-
The reliability evidence for the corrective signal is narrow. The verifier's labels are model-assisted without independent human adjudication, and only three of nine held-out regression examples are detected; the paper notes that the detected examples come from one task. The strict repair set contains 17 examples from 13 tasks. The lower no-progress rate uses that same verifier on different visited states and an earlier router version (Appendix C.7). These diagnostics do not independently establish improved error diagnosis or repair, which is central to the paper's explanation. Single-run task results further limit interpretation of the smaller differences.
Questions for authors
-
Targeted clarification/validation: is there an existing OPD comparison using the trained Planner–Executor teacher without the Repairer, with the same student budget and handoff rules? This would directly test whether the final model retains a repair-specific advantage. Please also clarify whether the 4B teacher-to-student decline is robust to checkpoint choice.
-
Can you report independently checked outcomes for a small held-out sample of repair activations, distinguishing genuinely corrected tasks, unnecessary interventions and missed regressions? Terminal success or same-state replay would be more informative than another comparison scored solely by V0.
-
Clarification: what are the total training trajectories and teacher/annotation costs for HARP and GRPO, and what data and optimisation settings produce the SFT cold start? Full cost parity is not required, but the comparison needs an explicit resource boundary.
6 comments
[Weakness 2] The reliability evidence for the corrective signal is narrow. The verifier's labels are model-assisted without independent human adjudication, and only three of nine held-out regression examples are detected; the paper notes that the detected examples come from one task. The strict repair set contains 17 examples from 13 tasks. The lower no-progress rate uses that same verifier on different visited states and an earlier router version (Appendix C.7). These diagnostics do not independently establish improved error diagnosis or repair, which is central to the paper's explanation. Single-run task results further limit interpretation of the smaller differences.
[Question 2] Can you report independently checked outcomes for a small held-out sample of repair activations, distinguishing genuinely corrected tasks, unnecessary interventions and missed regressions? Terminal success or same-state replay would be more informative than another comparison scored solely by V0.
Thank you for asking for independent evidence on repair outcomes. We sampled one activation from each of the 1,583 held-out test trajectories in which Router3 invoked the Repairer, yielding 1,572 evaluable pairs. For each pair, we restored the same state immediately before the intervention and let either the Repairer or the regular Planner produce the next plan. Both branches then used the same Planner and Executor, with further repair disabled and the same remaining action budget. We evaluated terminal success with AppWorld’s official state-based tests, independently of .
| Qwen3 size | Evaluable pairs | Rescued | Harmed | Both succeeded | Both failed |
|---|---|---|---|---|---|
| 1.7B | 534 | 2 | 9 | 8 | 515 |
| 4B | 520 | 13 | 3 | 43 | 461 |
| 8B | 518 | 7 | 5 | 56 | 450 |
| Overall | 1,572 | 22 | 17 | 107 | 1,426 |
A rescue means only the Repairer branch succeeded, and a harm means only the Planner branch succeeded. In 107 pairs, both branches succeeded, so the intervention was unnecessary for terminal success in those replays. Overall, the Repairer succeeded in 129/1,572 pairs and the Planner in 124/1,572, a net difference of +0.32 percentage points. The largest observed gain was at 4B, with 13 rescues and 3 harms. This matches the pattern in Table 2, where the configuration with the Repairer also shows the largest improvement at 4B.
In the revised manuscript, Section 5.4 (Routing and Verifier Diagnostics) summarizes the paired replay results. Appendix C.7 (Evaluation Details) presents the replay protocol, independent scoring, and per-model results after the Repairer routing diagnostic. These additions are highlighted in dark blue.
[Question 3] Clarification: what are the total training trajectories and teacher/annotation costs for HARP and GRPO, and what data and optimisation settings produce the SFT cold start? Full cost parity is not required, but the comparison needs an explicit resource boundary.
Thank you for asking us to clarify the training and annotation costs.
- Online training. A complete HARP OPD run uses 120 student-generated trajectories per model size (15 updates × 4 tasks × 2 rollouts), with 24 trajectories used by step 3. The routed experts score tokens on these trajectories. A complete archived GRPO run uses 20,480 trajectories (80 updates × 32 tasks × 8 rollouts), with 15,360 trajectories used by step 60. GRPO trains from the base model using environment rewards, without a separate teacher or annotation pipeline.
- Student SFT cold start. Before Router3 OPD, we collected a shared corpus of 3,000 successful trajectories from the teacher in 8,210 attempts. We used 2,912 trajectories for training and 88 for validation. Each corresponding Qwen3 base model received one epoch of SFT with loss on assistant tokens only, using LoRA (rank 32, alpha 64) and AdamW with a learning rate of .
- Initial SFT data. Qwen3-32B generated 3,600 candidate trajectories, of which 1,215 successful trajectories were retained.
- Phase classifier. 720 historical trajectories yielded 58,659 rule-assisted sentence-level labels, including 46,737 for training.
- Planner and progress verifier. Planner supervision involved model-assisted review of 14,832 assistant turns. The verifier pipeline processed 18,852 candidate records, yielding 9,622 training examples and 1,001 held-out examples. The two pipelines share source trajectories.
- Repairer. We generated 4,500 short branches from 750 intervention points, replayed 485 branches, and used Codex to review 65 flagged states. The final SFT set contained 17 correction examples and 34 ordinary planning examples.
The online rollout counts cover online training only. HARP’s resource requirements also include shared SFT data collection, teacher preparation, and annotation. We report those workloads in their original units because they include overlapping sources, full trajectories, and short continuations.
The revised manuscript now reports the training trajectory counts, student SFT cold start, and teacher preparation and annotation workloads in Appendix C.8 (Training and Annotation Workload). This subsection is highlighted in dark blue.
[Weakness 1] The central role-specialisation and repair contribution is not isolated in the final model. HARP differs from GRPO in supervised initialisation, verifier feedback, expert training, repair data and distillation. Table 2 tests routed components, but does not compare final students distilled from matched two-role and three-role teachers or a non-specialised teacher using the same data. Moreover, adding the Repairer also changes the handoff rules from Router2 to Router3. The 4B Test-N teacher gain from 20.2% to 27.4% becomes 19.6% after OPD, and other repair gains are small or negative. Thus the evidence does not yet establish that repair-aware role routing, rather than the other training additions, drives the deployed benefit.
[Question 1] Targeted clarification/validation: is there an existing OPD comparison using the trained Planner–Executor teacher without the Repairer, with the same student budget and handoff rules? This would directly test whether the final model retains a repair-specific advantage. Please also clarify whether the 4B teacher-to-student decline is robust to checkpoint choice.
Thank you for raising this point. We compared the full HARP student with a student distilled from the trained Planner–Executor teacher without the Repairer. We report OPD step 3 for all three model sizes.
Full HARP scores higher on both splits at 1.7B and 8B, while the Planner–Executor student scores higher on both at 4B. On 4B Test-N, the teacher with the Repairer solves 46/168 tasks versus 34/168 for the Planner–Executor teacher, but their step-3 students solve 33/168 and 36/168, respectively. At 1.7B and 8B, the Repairer-equipped teacher shows no gain on Test-N and a slight decrease on Test-C, while full HARP’s distilled students score higher on both splits. End-to-end teacher performance is therefore an imperfect predictor of student performance after OPD.
One possible explanation is that OPD transfers useful decisions at individual steps along the student’s own trajectories. For example, the Repairer can provide a useful plan revision after a setback even when the routed teacher fails to complete the overall task. This lets the framework bring planning, execution, and recovery supervision into a single deployable policy. The comparison is consistent with this explanation, though it varies the Repairer and the handoff rules together.
In the revised manuscript, we added the step-3 results for the no-Repairer OPD student to Table 2. Section 5.3 (Component Ablations) describes the comparison and discusses the 4B teacher–student ranking reversal. These additions are highlighted in dark blue.