- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PTD-PO, a multimodal RLVR framework that uses structured, answer-free privileged hints to guide a frozen reference model and distills its token-level distributions into the student on failed trajectories.
Critical strengths
- The paper addresses the genuine challenge of sparse outcome rewards in multimodal RLVR, particularly the lack of informative supervision for failed trajectories.
- PTD-PO achieves strong results across Qwen3-VL models from 2B to 8B parameters, outperforming several RLVR and self-distillation baselines on diverse multimodal reasoning benchmarks.
- The combination of structured privileged hints, failure-routed distillation, and Top-K JSD with tail compensation is intuitive and supported by several useful ablations.
Critical weaknesses
- The paper assumes that answer-revealing distillation induces shortcut behavior and harms reasoning exploration. However, the evidence in Figures 3 and 5 is not sufficient to establish this claim. Shorter responses, larger distribution shifts, and higher success rates under CoT+GT conditioning do not necessarily indicate undesirable shortcuts. Consequently, the central motivation for replacing answer-revealing supervision with way-revealing hints remains insufficiently validated.
- Although PTD-PO outperforms HDPO, these methods differ in more than just the privileged information provided to the teacher. A comparison within the same PTD framework, varying only the teacher context (e.g., GT answer, CoT+GT, regular hints, and WSPH), is necessary to establish whether the proposed hint design is genuinely responsible for the improvement.
- The paper claims that WSPH provides useful reasoning guidance without revealing answer-identifying information. However, it lacks quantitative analysis of hint quality, implicit answer leakage, and the reliability of the filtering process. Furthermore, the additional cost of generating and validating hints with large external models is not sufficiently quantified.
Questions for authors
- What evidence demonstrates that answer-revealing distillation actually harms downstream reasoning? In particular, could the shorter responses and higher success rates observed under CoT+GT conditioning simply reflect more informative teacher supervision rather than shortcut behavior? Can the authors provide direct evidence from the trained student policies?
- Does WSPH still outperform answer-revealing supervision when the distillation framework is held fixed? Specifically, what happens if the proposed failure-routed PTD objective is used with GT-answer or CoT+GT teacher contexts instead of WSPH?
Controlled Validation of Structured Privileged Guidance under Fixed Distillation Settings
W2 & Q2: Controlled Validation of Structured Privileged Guidance
We thank the reviewer for this valuable comment. Following the same experimental settings as those described in our response to W1 & Q1, we conducted additional experiments to further investigate this issue. The results are presented in Table C, followed by our analysis.
Controlled comparison within the PTD framework. Under the identical failure-routed PTD framework (Appendix E.4 protocol), WSPH achieves the highest mean accuracy (61.27%), outperforming CoT+GT (59.66%) by 1.61 pp and GT-only (56.68%) by 4.59 pp. These results directly confirm that WSPH provides additional benefits beyond the PTD optimization framework itself.
Cross-framework validation with HDPO. The advantage of WSPH also holds under HDPO, improving mean accuracy from 57.64% to 58.99% (+1.35 pp) over CoT+GT. Moreover, PTD consistently outperforms HDPO under both teacher contexts (+2.02 pp with CoT+GT; +2.28 pp with WSPH), further demonstrating the complementary contributions of structured guidance and the PTD framework.
Table C. Student accuracy under fixed-framework comparison.