- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies when a teacher should intervene and return control to a student during multi-turn agentic OPD. DASH-OPD standardizes and accumulates directional teacher–student disagreement over action tokens to select the next turn's executor, with a stagnation trigger and a minimum teacher span. Training follows Guided-OPD and covers both reasoning and action tokens; inference uses only the student. The authors report overall performance advantages over five baselines across three environments and two student scales, supported by component ablations and switching-behavior analyses.
Critical strengths
- DASH-OPD uses feedback from the current interaction trajectory to decide when the teacher model should take over and when control should return to the student. This makes rollout scheduling in multi-turn agent distillation more flexible than a predefined curriculum, while retaining student-only inference.
- The paper reports consistent performance gains over the evaluated OPD baselines across three environments and two student model sizes. The ablations also show that removing student resumption or cross-turn accumulation reduces performance, providing experimental support for these design choices.
Critical weaknesses
- The motivating bottleneck is insufficiently established. The introduction motivates teacher guidance through vanilla OPD's limited gains, but does not directly establish that curriculum-based intervention timing is a key limitation. Figure 1 illustrates execution schedules without showing mistimed interventions and their consequences. Given related adaptive-intervention approaches such as ThriftyDAgger and AdaSwitch, I would like clearer evidence of the problem that DASH's specific adaptation resolves.
- The ablations leave the gains from switching decisions insufficiently isolated. Removing teacher execution or student resumption changes both the switching behavior and teacher participation. DASH also permits teacher participation late in training, whereas Guided-OPD has a student-only tail. These comparisons do not sufficiently separate better timing from differences in supervision amount or trajectory composition. Moreover, the recovery statistics partly depend on the controller's own triggers. The action-mask ablation supports the chosen statistic empirically, but does not establish why excluding reasoning disagreement helps; signal mixing or tuning effects remain possible, unverified explanations.
Questions for authors
- Could the authors provide trajectory examples or aggregate evidence of mistimed curriculum interventions and their effects on task success?
- Could the authors clarify whether DASH retains an advantage when teacher turn/token fractions and execution spans are comparable? Is there evidence of recovery independent of the controller's subsequent triggers?
- Could the authors explain how the thresholds were selected, including for the no-action-mask comparison? Beyond Figure 5's local sensitivity analysis, what motivates linear scaling with H_max and the one-third return threshold?
10 comments
1. Limitations of curriculum-based methods (W1, Q1)
We thank you for your insightful questions. Training-progress curricula have a fundamental limitation: they allocate teacher intervention without accounting for whether the current trajectory actually needs it. We have supplemented the revised manuscript with the more detailed analysis below. Our original manuscript already identifies this problem in Section 1, noting that intervention is scheduled "even though its necessity may vary within and across trajectories." Across trajectories, the same training step can contain both productive student rollouts and rollouts trapped in repeated, unproductive actions. Within a trajectory, the need for assistance can rise sharply after an error and diminish after recovery. Neither TCOD's scheduled segment boundary nor Guided-OPD's scheduled teacher probability responds to these changes. Consequently, these schedules can replace useful student execution with unnecessary teacher turns while failing to provide timely assistance when errors begin to compound. This mismatch directly undermines the intended balance between improving trajectory quality through teacher intervention and reducing exposure bias through on-policy student learning. DASH-OPD addresses it by using accumulated feedback from the current trajectory to adaptively determine when the teacher intervenes and when the student resumes control.
Saved training trajectories illustrate how mistimed curriculum interventions can undermine task success and how DASH-OPD's adaptive switching can enable successful completion. On ALFWorld's "put a cool egg in microwave" task, TCOD-B2F hands control to the 1.7B student after a scheduled 14-action expert prefix. The student then repeats look for all 16 remaining turns and fails. On ScienceWorld's "find a plant and move it to the purple box" task, Guided-OPD's 1.7B student repeats look around in the foundry for all 30 turns without any teacher intervention, despite a teacher-selection probability of 15.20%. The task remains incomplete with a score of 8/100. A same-task ALFWorld comparison at 4B further illustrates why sustained assistance matters: on "put two candles in a drawer," both Guided-OPD and DASH-OPD encounter failed attempts to pick up the second candle while carrying the first. Guided-OPD continues interleaving teacher and student execution without resolving this obstacle and fails at turn 30. DASH-OPD maintains teacher control through turn 15: the teacher deposits the first candle at turn 13 and retrieves the second at turn 15, then returns control to the student, which completes the task at turn 17.
The existing experiments in Table 1 already provide aggregate evidence of DASH-OPD's performance advantage over curriculum-based scheduling (TCOD and Guided-OPD). Table R1 extracts the primary performance metrics from Table 1, using mean ± sample SD over seeds 0, 42, and 2026.
Table R1. Main performance of DASH-OPD and curriculum-based baselines (Table 1). Bold marks the best and underlining marks the second-best mean in each row.
| Benchmark / student / metric | DASH-OPD | TCOD | Guided-OPD |
|---|---|---|---|
| WebShop / 1.7B / score | 43.52 ± 1.61 | 32.86 ± 1.68 | |
| WebShop / 4B / score | 43.85 ± 2.34 | 37.68 ± 2.05 | |
| ALFWorld / 1.7B / overall SR (%) | 34.18 ± 2.64 | 26.52 ± 1.72 | |
| ALFWorld / 4B / overall SR (%) | 42.82 ± 2.43 | 30.90 ± 2.32 | |
| ScienceWorld / 1.7B / score | 38.30 ± 1.51 | 30.27 ± 1.77 | |
| ScienceWorld / 4B / score | 49.29 ± 1.98 | 46.11 ± 1.59 |
DASH-OPD outperforms both curriculum-based baselines on every primary metric at both student sizes. Compared with the stronger of TCOD and Guided-OPD in each setting, DASH-OPD improves WebShop score by 5.47 / 5.32 points, ALFWorld overall SR by 4.62 / 7.18 percentage points, and ScienceWorld score by 5.72 / 3.12 points at 1.7B / 4B, respectively. These consistent gains across all six settings support the benefit of adapting teacher intervention to the current trajectory beyond scheduling it solely by training progress.
Changes in the revised manuscript (blue). Section 1 expands the explanation of assistance needs varying within and across trajectories. Figure 1 and its caption ("A research gap in agentic OPD.") are retained from the original submission. Section 5.2 summarizes the comparison with curriculum-based baselines and points to Appendix A.17, which adds the three observed training-trajectory examples and their task outcomes. Appendix A.3 cites ThriftyDAgger when contrasting the use of an auxiliary risk model.
2. Distinctions from prior work (W1)
We thank you for highlighting the connection between prior methods. We have cited ThriftyDAgger in the revised manuscript to acknowledge its contribution to adaptive intervention.
DASH-OPD differs from ThriftyDAgger in switching evidence and application scenarios. ThriftyDAgger (Hoque et al., 2021a) controls robot–human switching in robotic imitation learning: high state novelty or high failure risk estimated by a trained auxiliary risk model triggers takeover, while low risk and low robot–supervisor action discrepancy permit return. In contrast, DASH-OPD controls student–teacher switching during agentic OPD, accumulating standardized action-only discrepancy across interaction turns to guide both takeover and return. It reuses probabilities already available for distillation without training an auxiliary risk model.
DASH-OPD differs from AdaSwitch in switching granularity and direction. AdaSwitch (Peng et al., 2025) makes a single student-to-teacher switch at the token level within a response, after which the teacher generates all remaining tokens. In contrast, DASH-OPD supports repeated turn-level switches in both directions within an episode.
Changes in the revised manuscript (blue). Appendix A.3 cites ThriftyDAgger. Appendix A.21 adds explicit comparisons with ThriftyDAgger and AdaSwitch.
We thank you for recognizing DASH-OPD's flexibility and the convincing experimental results. Below, we provide detailed responses to your valuable suggestions and questions.