- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Distilling long-horizon task performance from a strong teacher to a weaker student multimodal agent is particularly complex. Decision-making at any given step relies not only on current input, but also on the full history of past steps and anticipated future trajectories. A behavioral analysis of teacher-student distillation highlights these challenges: SFT on teacher trajectories leads to premature termination because the student inherits stopping behaviors without learning how to finalize a trajectory. Conversely, OPD addresses premature stopping but introduces another issue, unnecessarily prolonging interaction trajectories without progressing toward the answer. To address these limitations, the authors propose -OPD. This approach improves upon standard OPD by applying supervisory signals only when the student’s interactions demonstrate meaningful progress toward the answer with reduced redundancy. Low progress or redundancy is probabilistically quantified via variations in answer-readiness, the likelihood of the student generating a correct answer immediately. Experiments across complex multimodal benchmarks demonstrate that -OPD narrows the performance gap between small multimodal agents and proprietary models, outperforming open-source baselines of similar scale. Furthermore, direct comparisons with existing OPD methods show that -OPD achieves the best average performance across multiple benchmarks.
Critical strengths
-
The proposed method yields meaningful performance improvements over baseline OPD methods and existing VLMs. Furthermore, the ablation study clearly demonstrates the specific contribution of each component technique.
-
The analysis of pathologies in SFT and OPD training, which forms the foundation of the proposed technique, is both clear and compellingly motivated.
Critical weaknesses
-
While the problem formulation targets multimodal agents, the proposed technique is not specific to multimodality in any respect and can be applied to text-only agents.
-
The paper lacks an in-depth analysis of how the key indicators are determined, as well as empirical verification of whether these indicators accurately detect and mitigate the intended pathologies.
Questions for authors
-
How were the specific thresholds and formulations for "answer-readiness" determined? Did you explore alternative indicators during development?
-
Have you conducted behavioral audits or qualitative case studies to verify that low-progress interactions identified by the model actually correspond to human-judged stalling or repetitive reasoning loops?
-
Is it possible to provide a baseline comparison against alternative supervision filters or progress-gating heuristics to demonstrate that the proposed -OPD formulation is optimal?
-
The related work section seems overly brief and lacks depth. Is it sufficient to omit a broader literature survey and conceptual comparison across general LLM/VLM knowledge distillation or trajectory-level imitation?
4 comments
Response to Reviewer xbQJ (Part 2/3)
We next address the reviewer's question regarding behavioral validation (Q2) through a blinded human audit, post-training behavioral analysis, and qualitative case studies.
Q2: Have you conducted behavioral audits or qualitative case studies to verify that low-progress interactions identified by the model actually correspond to human-judged stalling or repetitive reasoning loops?
A2: Yes. Our blinded human audit provides direct evidence that low-change interactions frequently correspond to human-judged stagnation, while post-training behavioral analysis shows that Δ-OPD substantially reduces prolonged low-progress tool use.
Human behavioral audit. We uniformly sampled 100 low-change interaction windows from 586 eligible windows across 55 tasks, using the criterion |Δt| < 0.05 (Section 4.4). Annotators examined each window and its preceding history, blinded to readiness scores, routing decisions, ground-truth answers, and subsequent turns. Meaningful progress required new task-relevant evidence, hypothesis narrowing, useful computation, or error recovery.
Of these windows, 80 were judged stalled: 54 redundant searches, 14 failed tool calls, 7 instances of repeated reasoning, and 5 unsupported guesses. The remaining 20 exhibited meaningful progress. These results show that readiness change captures diverse forms of low-yield behavior beyond exact repetition, while acknowledging that not every flagged interaction is stagnant.
Mitigation after training. Table 4 reports one rollout per policy on 358 OmniGAIA tasks. Compared with standard OPD, Δ-OPD reduces low-progress tool turns from 3,995 to 2,611, with 94.2% of the reduction occurring after turn five. Tool turns not flagged as low-progress increase from 1,345 to 1,534, while budget exhaustion decreases from 17.6% to 10.9%. In the main four-run evaluation, budget exhaustion similarly decreases from 15.3% to 9.8%, alongside an accuracy improvement from 40.8% to 45.4%. Together, these results indicate that Δ-OPD mitigates prolonged stagnation while improving task-solving performance.
Qualitative case studies. Appendix D illustrates repeated reasoning over resolved results, failed retrieval during verification, and unsupported inferences that constrain subsequent searches. A boundary case further distinguishes redundant repetition from potentially useful verification, providing a more nuanced understanding of low-progress behavior.
Response to Reviewer xbQJ (Part 3/3)
We further address Q3 on alternative supervision strategies and Q4 on related work, while clarifying the multimodal relevance of our method. Together with Parts 1–2, which address indicator design and behavioral validation, these responses directly cover both critical weaknesses raised by the reviewer.
Q3: Is it possible to provide a baseline comparison against alternative supervision filters or progress-gating heuristics to demonstrate that the proposed OPD formulation is optimal?
A3: Yes. We compare Δ-OPD against existing distillation methods, alternative supervision filters, and progress-gating strategies.
Three-benchmark comparison. Tables 2 and 3 report four-run average accuracy across OmniGAIA, VideoDR, and VisualSimpleQA.
| Method | Average accuracy |
|---|---|
| Standard OPD | 50.7% |
| TCOD | 52.4% |
| SOD | 52.1% |
| TurnOPD | 50.9% |
| Zero-loss replacement of FKL (filtering) | 52.8% |
| Proportion-matched Random FKL | 52.1% |
| Δ-OPD (Ours) | 55.5% |
Zero-loss filtering achieves 52.8% and proportion-matched Random FKL achieves 52.1%, versus 55.5% for Δ-OPD. Neither filtering nor frequency matching reproduces the gains of progress-conditioned supervision.
Alternative supervision and gating strategies. We further compare uniform KL objectives, fixed-ratio allocation, and signed gating on OmniGAIA (Table 4). These variants retain the same readiness-based masking and turn-level normalization as Δ-OPD, isolating the effect of supervision allocation.
| Strategy | Accuracy |
|---|---|
| Standard OPD | 40.8% |
| Uniform FKL | 42.5% |
| Uniform RKL | 43.3% |
| Fixed-ratio FKL/RKL | 43.3% |
| Signed gating | 44.0% |
| Δ-OPD (Ours) | 45.4% |
Δ-OPD achieves the highest accuracy among the evaluated strategies, outperforming uniform objectives, randomized allocation, and signed gating. Together, these results provide strong empirical evidence that both the placement and the form of supervision matter, validating progress-conditioned KL routing as a key design choice for long-horizon agent distillation.
Q4: The related work section seems overly brief and lacks depth. Is it sufficient to omit a broader literature survey and conceptual comparison across general LLM/VLM knowledge distillation or trajectory-level imitation?
A4: We appreciate this suggestion and have expanded Section 5 to broaden the literature review and further clarify our contribution. The additions cover general LLM/VLM distillation and divergence selection (MiniLLM, GKD, DistiLLM, AKL), trajectory-level imitation (behavior cloning and DAgger), and recent multimodal distillation methods, including Vision-OPD, SpatialOPSD, visual-manipulation self-distillation, and visual-grounding-aware objectives.
Conceptual distinction. Existing methods primarily focus on matching teacher distributions, selecting divergence objectives, transferring privileged visual information, or addressing distribution shift. In contrast, Δ-OPD tackles a distinct challenge: how to allocate teacher supervision across student-visited states based on realized progress. Specifically, answer-readiness changes determine forward/reverse KL routing, while a trajectory-specific reference enables selective masking of low-progress supervision. A3 further shows that divergence-based weighting (SOD), fixed or uniform KL objectives, and signed gating do not reproduce Δ-OPD's gains. Our key contribution is therefore not merely choosing a different distillation objective, but adapting teacher supervision to the student's realized progress throughout long-horizon interactions.
Multimodal Relevance and Scope. We respectfully note that multimodal relevance does not require a modality-specific architectural mechanism. Δ-OPD targets a concrete failure mode identified in long-horizon multimodal agent distillation: agents must gather and integrate visual, video, and other task-dependent evidence through multi-step interactions, yet locally reasonable teacher-supervised actions can repeatedly fail to advance the student's answer state. Consistent with this mismatch, readiness change is more predictive of downstream task failure than local teacher–student discrepancy (AUROC 0.65 vs. 0.47; A1 and Appendix C), motivating progress-conditioned KL routing and selective masking.
This is not merely a general method evaluated incidentally on multimodal tasks. Our behavioral analysis (Section 2) identifies the failure mode, while experiments across OmniGAIA, VideoDR, and VisualSimpleQA establish the effectiveness of Δ-OPD in multimodal settings, outperforming standard OPD on all three benchmarks. The contribution lies in addressing a demonstrated multimodal agent learning problem, not in imposing modality-specific architectural constraints. Its potential applicability to text-only agents broadens the scope of the principle rather than diminishing its contribution to multimodal learning.
Follow-up on Author Responses to Reviewer xbQJ
Dear Reviewer xbQJ,
Thank you for your careful review and thoughtful feedback.
As the discussion deadline approaches, we would appreciate your feedback on whether our responses have addressed your concerns. We believe our responses address the issues you raised, and the additional evidence further validates the original design choices and supports the central claims of Δ-OPD.
If any issues remain unresolved, we would be happy to provide further clarification. We would also be grateful if you could reconsider your assessment in light of this evidence.
Thank you again for your time and consideration.
Best regards, The Authors
Response to Reviewer xbQJ (Part 1/3)
We sincerely thank the reviewer for the thoughtful feedback and constructive suggestions. This part addresses Q1: the formulation, threshold selection, and alternative indicators. Responses to Q2–Q4 follow in subsequent comments.
A1: We fixed τ = 0.05 through offline calibration before Δ-OPD training and examined alternative progress indicators and gating strategies. Our empirical comparisons favor readiness-change magnitude over the alternatives considered.
Formulation. Answer readiness is the student's mean per-token log-likelihood of the reference answer at the current state (Eq. 2). It directly measures how strongly the current state supports the reference answer, reflects the student's own answer state rather than its local agreement with the teacher (Section 2.2), requires no learned critic, and is length-normalized across answers. We use its change rather than its absolute level because the latter is task- and state-dependent, whereas the change captures the effect of an individual interaction.
Magnitude rather than sign. Δ-OPD switches to forward KL to help the student escape stalled, self-reinforcing interaction patterns (Eqs. 9–10). A large change in either direction indicates substantial answer-state evolution rather than stagnation; we do not assume such changes are necessarily beneficial. Consistently, under identical masking and normalization, fully reverse-KL training achieves 43.3% on OmniGAIA, while signed gating that also routes large negative changes to forward KL achieves 44.0%, compared with 45.4% for magnitude-based gating (Table 4). Extending forward KL to large negative changes thus removes most of the gain from magnitude-based routing. Moreover, the signed variant yields fewer tool turns (3,652 vs. 4,145) yet lower accuracy, indicating that the gain of Δ-OPD is not explained by shortening interaction alone.
Threshold. We fixed τ = 0.05 before Δ-OPD training through offline behavioral calibration on multiple batches of student-generated trajectories collected during separate standard OPD training runs. We compared candidate thresholds {0.01, 0.025, 0.05, 0.1}, using exact tool-call repetitions as conservative indicators of stagnation rather than its complete definition. At τ = 0.05, the readiness-change criterion captured 73–81% of exact repetitions, while only 10–16% of flagged interactions exhibited clearly observable progress under conservative, coarse-grained rule-based checks, such as new evidence subsequently used, useful computation results, or successful recovery from tool failures. This offered a favorable balance between capturing repetitive behavior and limiting the inclusion of evidently productive interactions. The reported ranges reflect variation across separate standard OPD training runs rather than confidence intervals.
Since readiness is measured as mean per-token log-likelihood, τ = 0.05 corresponds to approximately a 5% relative change in the geometric-mean probability assigned to the reference answer. Subsequent analysis of 28,952 training tool-calling turns shows that 39% satisfy |Δt| ≤ 0.05 (Appendix C), and trajectories observed at these states fail more often than those at progressing states (72% vs. 58%). Threshold sensitivity experiments further show that tightening τ to 0.025 yields comparable OmniGAIA accuracy (45.0% vs. 45.4%), whereas loosening it to 0.1 reduces accuracy to 43.1% (Table 4). This is consistent with the design: a looser threshold may route interactions with non-negligible readiness changes to forward KL. Together, the offline calibration and subsequent sensitivity results support τ = 0.05 as a well-grounded operating point. For masking, we use the readiness at the state from which a verified correct answer is produced as a trajectory-specific reference, avoiding an additional global threshold.
Alternative indicators. We also examined several alternative indicators, which showed important limitations compared with readiness change. Local teacher–student discrepancy is substantially less predictive of downstream outcomes: its AUROCs for task failure, budget exhaustion, and future repetition are 0.47, 0.46, and 0.45, compared with 0.65, 0.64, and 0.56 for readiness change (Appendix C). Exact repetition accounts for only 1.3% of tool-calling turns, while 98% of near-zero-change turns are not exact repeats, so repetition alone misses most low-change interactions. Trajectory length alone is also insufficient: even after eight turns, 45% of subsequent turns still exhibit readiness changes above τ. Additional supervision-allocation baselines, including proportion-matched random routing, are compared in A3.