- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies on-policy distillation (OPD) for multimodal deep-search agents and identifies a failure mode called evidence-blind supervision (EBS): the per-token OPD signal is computed only from the pre-retrieval prefix and never reflects what a query actually retrieves. Regrouping the OPD objective by interaction turn, the authors expose two implicit priors: a budget prior that allocates supervision by token length, and a direction prior that pushes tokens toward the teacher’s pre-retrieval preference. A diagnosis on MMSearch shows both are misaligned with post-retrieval information gain. The proposed IG-OPD uses IGPO-style information gain, defined as the change in the teacher’s gold-answer log-likelihood after each observation, to correct both priors. ITC reweights per-turn supervision budgets by IG magnitude, and IQC adds a signed, cosine-ramped IG correction to the action-critical tokens of each tool call; the authors interpret IQC as distillation toward an answer-tilted “power posterior” of the teacher. With an 8B OpenSearch-VL teacher and Qwen3-VL 2B/4B students, IG-OPD improves the six-benchmark average over vanilla OPD by 3.4 and 1.7 points, and the 4B student surpasses its teacher on MMSearch and BrowseComp-VL. Training overhead is about 11% and inference cost is unchanged.
Critical strengths
- Clear, well-motivated problem. In search, a query’s value is only known after retrieval, while the OPD signal is fixed before it.
- Method is lightweight and practical. IG-OPD needs only one extra teacher-forced scoring pass per turn, with no additional rollouts. Training cost rises about 11.6% in time and 10.6% in memory, inference is unchanged, and the method drops into existing OPD trainers.
- Realistic setting with consistent gains. Experiments use a multimodal agent with a rich tool set in a live web environment, six benchmarks, and two student scales. Gains are largest on search-intensive benchmarks (MMSearch, BrowseComp-VL), which is consistent with the paper’s hypothesis.
Critical weaknesses
- Most of the gain is not from IG. Equal per-turn weighting alone (λ = 0) gives +2.6, the largest single gain in Table 2, and uses no IG. With λ = 0.1, the IG term changes turn weights by at most 1.1×, and Figure 4a shows a nearly uniform budget. ITC’s benefit therefore comes mainly from length normalization, which is close to existing turn-aware OPD work.
- IQC is essentially an IGPO reward added to OPD. Adding αₖgₜ to the advantage makes the method an OPD+RL hybrid; at α_max = 10, the correction is about half of the OPD signal. Comparing against IGPO alone does not isolate the contribution; baselines such as OPD + tuned IG reward or OPD + outcome reward are needed.
- The rollout and compute settings for GRPO and IGPO are not given, and the “mean across candidate rollouts” conflicts with using one rollout per prompt. Relevant OPD variants that the paper cites (SCOPE, TIP, SOD, SAGE-OPD, TurnOPD) are not compared. Beating the author-enhanced teacher on two benchmarks by 1–2 points, without variance estimates, is weak evidence.
- The OOD claim is inaccurate. Training includes the FVQA train split, yet FVQA test is presented as out-of-distribution. Overlap between the other training sources and the evaluation sets should also be checked.
Questions for authors
see above