- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PaRa-OPD reallocates token-level distillation supervision for pathology vision-language models with the help of an offline Pathology Reward Model (PRM). Specifically, the authors derive image/no-image probability-difference labels from answer-correct teacher trajectories and combine them with pathology-vocabulary labels to train a scorer that operates on trajectory text alone. The scorer’s predictions are then converted into normalized weights applied to the reverse-KL objective of the privileged OPD. The experiments compare PaRa-OPD against several alternative training methods, using student models of 2B and 4B parameters and evaluating on PathMMU and PathGen-Instruct.
Critical strengths
The intervention is well focused, and the main sources of improvement are at least partially disentangled. Conceptually, the PRM modifies only the supervision weights while retaining the teacher distribution as the semantic target, and the normalization controls the average loss scale. Empirically, Table 4 separates the contribution of privileged teacher information from that of PRM weighting, and the authors are commendably candid in acknowledging that privileged supervision accounts for much of the observed improvement. The evaluation includes strong and directly relevant baselines as well as several useful design controls. Privileged OPD and Med-OPD are compared across two student scales, supplemented by additional analyses of the weight scale, soft versus hard mappings, and the granularity of privileged information. Taken together, these experiments provide a reasonable basis for assessing the approach.
Critical weaknesses
The central motivation mischaracterizes the closest prior methods. Figure 1 argues from image-conditioned vs. no-image probability differences, whereas VA-OPD [1] (Eq. 2) and Med-OPD [2] (Eq. 4) use rectified log-probability differences between original and degraded images—and low absolute probabilities do not imply a small log-ratio (0.04 vs. 0.02: a difference of 0.02 but a log-ratio of log 2). The motivating example thus targets a different quantity and does not demonstrate the claimed failure mode; the baselines’ token grouping and trajectory weighting also need to be accounted for in their implementation [3]. [1] Liu R, Lv X, Li G, et al. Visual-advantage on-policy distillation for vision-language models[J]. arXiv preprint arXiv:2605.21924, 2026. [2] Qian Y, Yu J, Liu J, et al. Med-OPD: Improving Medical Vision-Language Models via Evidence-Aware On-Policy Distillation[J]. arXiv preprint arXiv:2607.16303, 2026. [3] Xu Y, Sang H, Zhou Z, et al. Tip: Token importance in on-policy distillation[J]. arXiv preprint arXiv:2604.14084, 2026.
The PRM’s value over existing token selection and lexical priors is not isolated. Receiving only text, with a token-ID prior and vocabulary-derived supervision, it can capture lexical/contextual importance but cannot verify image–statement consistency. Label-prediction AUROC on correct teacher trajectories does not certify reliable detection of visual errors in student rollouts. Missing controls include vocabulary weighting, token priors, separated visual/vocabulary labels, and TIP-style selection. The PathMMU gains over privileged OPD (~0.62 and 0.97 points) also lack repeated-training or paired statistical support.
Questions for authors
Were the VA-OPD and Med-OPD baselines run with their complete original configurations—rectified log-probability differences, degraded images, grouped KL, and trajectory weighting—or under budget-constrained adaptations? In particular, how is within-prompt trajectory weighting implemented given that the appendix reports only a single response per prompt? A clear distinction between faithful reproductions and simplified variants is essential for interpreting the comparison. Could online VA/MEA, the offline PRM, and simple token-importance rules be compared directly on erroneous student trajectories—trajectories containing visually incorrect tokens? Such a comparison would directly test the central claim that the PRM improves supervision precisely at visually incorrect positions, which is currently established only indirectly. What are the downstream results for the following controls: vocabulary-only weighting, scoring from the token-ID prior alone, visual-only labels, and vocabulary-only labels? Additionally, could repeated-seed results (mean ± std) and paired statistical tests against both privileged OPD and Med-OPD be provided to assess the reliability of the reported margins? Were the models in Table 3 trained on the 30,000 PathGen prompts listed in Appendix Table 6? If so, please clarify whether this constitutes training on the target domain, and specify the image- or case-level separation and deduplication enforced among the PRM training data, the validation trajectories, and the final VQA test sets—any overlap here could inflate the reported gains.
Clarification of PRM Motivation, Baseline Adaptations, and Additional Experimental Evidence
We thank the reviewer for recognizing the focused design of our method. We address the concerns regarding our motivation, baseline implementation, and experimental evidence below.
1. Motivation and Figure 1
We acknowledge that low absolute probabilities do not necessarily imply a small log-probability ratio. We will revise Figure 1 to distinguish probability differences from the rectified log-probability ratios used in VA-OPD and Med-OPD.
Our central motivation is that online visual gain and token-level supervision importance are related but distinct quantities. Online visual gain measures changes in teacher support for a generated token under different visual conditions, whereas supervision importance concerns whether the underlying content requires strong visual guidance.
For example, a non-positive original-versus-degraded log-probability ratio may be rectified to zero, even when the position involves a critical pathological feature. This illustrates a potential limitation rather than suggesting that all incorrect tokens receive low weights.
PRM addresses this distinction by learning importance patterns from correct teacher trajectories. PRM determines where supervision should be emphasized, while the image-conditioned teacher determines what the student should learn.
2. Online Baseline Implementation
We clarify that our Med-OPD implementation adopts a simplified configuration under a shared training setup. We retain token-level online visual weighting but do not apply trajectory-level weighting, as our experiments sample one response per prompt. Under this setting, within-prompt trajectory weighting reduces to a constant and cannot distinguish among alternative responses.
We will clarify these adaptations in the manuscript. Accordingly, our comparisons concern the evaluated online weighting variant under the shared experimental setting, rather than the full original Med-OPD configuration.
3. Incorrect-Trajectory Evaluation
We conducted additional experiments on PathMMU 4B using only student trajectories with incorrect final answers for distillation.
Across three runs, the evaluated online method decreased from 73.46% to 71.65% (−1.82 pp), whereas PaRa-OPD decreased from 76.04% to 75.01% (−1.03 pp).
PaRa-OPD exhibits a 0.79-percentage-point smaller performance decrease, indicating lower sensitivity to incorrect-only training and better downstream performance under this condition.
These findings support our motivation that learned PRM weighting can provide advantages over the evaluated online visual-weighting mechanism. We will also include a case study comparing their actual weights on the same tokens in incorrect student trajectories.
Detailed experimental settings and results are provided in our response to Reviewer 2.
4. Simpler Weighting Controls and Multi-Seed Evaluation
We additionally evaluated direct pathology vocabulary weighting. On PathMMU 2B/4B, vocabulary weighting improves standard OPD by 1.03/1.08 pp, but improves Privileged OPD by only approximately 0.21 pp.
PaRa-OPD further outperforms vocabulary-weighted Privileged OPD by 1.52/1.85 pp, suggesting that its benefits cannot be fully explained by direct vocabulary matching.
We also conducted three independent training runs across both datasets and student scales. Compared with Privileged OPD, PaRa-OPD improves mean accuracy by 1.28/1.70 pp on PathMMU 2B/4B and 2.21/3.46 pp on PathGen 2B/4B.
Moreover, PaRa-OPD achieves higher mean accuracy than the evaluated Med-OPD variant in all four settings. Without privileged information, PRM weighting also improves PathMMU 2B/4B accuracy by 1.15/1.41 pp.
These additional experiments provide consistent evidence of PRM's incremental contribution beyond privileged supervision and simple lexical weighting.
Detailed multi-seed comparisons and vocabulary-weighting results are reported in our responses to Reviewers 1 and 2.
5. Training Protocol and Data Separation
We clarify that Table 3 uses 30,000 PathGen training prompts and therefore represents target-domain training rather than zero-shot cross-dataset student transfer.
A shared PRM is constructed from the training splits of PathMMU and PathGen, while student models are trained separately on their respective datasets.
We verified image-level separation between training and test sets and case-level separation between PRM training and validation, keeping all trajectories from the same case within one partition. Benchmark test images are excluded from PRM construction and student training.
We will clarify these procedures and replace "cross-dataset transfer" with "evaluation on a second dataset". Further details are provided in our response to Reviewer 1.