acceptodds
Under review as a conference paper at ICLR 2027

PaRa-OPD: Pathology Reward-Aware On-Policy Distillation via Token-Level Supervision Reweighting

Abstract

Pathology vision-language reasoning requires models to connect fine-grained histologic evidence with specialized diagnostic concepts, while the task-critical information in a generated rationale is often concentrated in only a small fraction of tokens. On-policy distillation (OPD) provides an effective way to transfer reasoning behavior from a teacher on student-generated trajectories, but its standard token-level objective supervises visually grounded diagnostic tokens in much the same way as generic reasoning and connective text. This uniform treatment limits the effectiveness of OPD on pathology tasks. Existing online visual-gain weighting methods attempt to identify visually dependent tokens during OPD, but their signal can weaken when the sampled student token is already visually incorrect. To make OPD more effective for pathology reasoning, we introduce PaRa-OPD, a pathology reward-aware OPD framework for token-level supervision reweighting that directs distillation toward tokens that either depend on visual evidence or encode pathology-specific diagnostic information. PaRa-OPD trains a Pathology Reward Model (PRM) to identify these complementary token patterns in generated trajectories. The resulting PRM scores are converted into token-level weights that dynamically reweight the OPD objective, strengthening supervision where visual grounding and diagnostic knowledge are most consequential. On PathMMU, PaRa-OPD improves Qwen3.5-2B from 63.88 to 71.44 and Qwen3.5-4B from 66.96 to 74.96 over standard OPD, demonstrating strong and consistent effectiveness across model scales. Code and processed data will be released in the future.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
pathology vision-language models, on-policy distillation, token-level reward modeling, reward-aware distillation, pathology reward model, medical VQA, medical VLM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FbPpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PaRa-OPD reallocates token-level distillation supervision for pathology vision-language models with the help of an offline Pathology Reward Model (PRM). Specifically, the authors derive image/no-image probability-difference labels from answer-correct teacher trajectories and combine them with pathology-vocabulary labels to train a scorer that operates on trajectory text alone. The scorer’s predictions are then converted into normalized weights applied to the reverse-KL objective of the privileged OPD. The experiments compare PaRa-OPD against several alternative training methods, using student models of 2B and 4B parameters and evaluating on PathMMU and PathGen-Instruct.

Critical strengths

The intervention is well focused, and the main sources of improvement are at least partially disentangled. Conceptually, the PRM modifies only the supervision weights while retaining the teacher distribution as the semantic target, and the normalization controls the average loss scale. Empirically, Table 4 separates the contribution of privileged teacher information from that of PRM weighting, and the authors are commendably candid in acknowledging that privileged supervision accounts for much of the observed improvement. The evaluation includes strong and directly relevant baselines as well as several useful design controls. Privileged OPD and Med-OPD are compared across two student scales, supplemented by additional analyses of the weight scale, soft versus hard mappings, and the granularity of privileged information. Taken together, these experiments provide a reasonable basis for assessing the approach.

Critical weaknesses

The central motivation mischaracterizes the closest prior methods. Figure 1 argues from image-conditioned vs. no-image probability differences, whereas VA-OPD [1] (Eq. 2) and Med-OPD [2] (Eq. 4) use rectified log-probability differences between original and degraded images—and low absolute probabilities do not imply a small log-ratio (0.04 vs. 0.02: a difference of 0.02 but a log-ratio of log 2). The motivating example thus targets a different quantity and does not demonstrate the claimed failure mode; the baselines’ token grouping and trajectory weighting also need to be accounted for in their implementation [3]. [1] Liu R, Lv X, Li G, et al. Visual-advantage on-policy distillation for vision-language models[J]. arXiv preprint arXiv:2605.21924, 2026. [2] Qian Y, Yu J, Liu J, et al. Med-OPD: Improving Medical Vision-Language Models via Evidence-Aware On-Policy Distillation[J]. arXiv preprint arXiv:2607.16303, 2026. [3] Xu Y, Sang H, Zhou Z, et al. Tip: Token importance in on-policy distillation[J]. arXiv preprint arXiv:2604.14084, 2026.

The PRM’s value over existing token selection and lexical priors is not isolated. Receiving only text, with a token-ID prior and vocabulary-derived supervision, it can capture lexical/contextual importance but cannot verify image–statement consistency. Label-prediction AUROC on correct teacher trajectories does not certify reliable detection of visual errors in student rollouts. Missing controls include vocabulary weighting, token priors, separated visual/vocabulary labels, and TIP-style selection. The PathMMU gains over privileged OPD (~0.62 and 0.97 points) also lack repeated-training or paired statistical support.

Questions for authors

Were the VA-OPD and Med-OPD baselines run with their complete original configurations—rectified log-probability differences, degraded images, grouped KL, and trajectory weighting—or under budget-constrained adaptations? In particular, how is within-prompt trajectory weighting implemented given that the appendix reports only a single response per prompt? A clear distinction between faithful reproductions and simplified variants is essential for interpreting the comparison. Could online VA/MEA, the offline PRM, and simple token-importance rules be compared directly on erroneous student trajectories—trajectories containing visually incorrect tokens? Such a comparison would directly test the central claim that the PRM improves supervision precisely at visually incorrect positions, which is currently established only indirectly. What are the downstream results for the following controls: vocabulary-only weighting, scoring from the token-ID prior alone, visual-only labels, and vocabulary-only labels? Additionally, could repeated-seed results (mean ± std) and paired statistical tests against both privileged OPD and Med-OPD be provided to assess the reliability of the reported margins? Were the models in Table 3 trained on the 30,000 PathGen prompts listed in Appendix Table 6? If so, please clarify whether this constitutes training on the target domain, and specify the image- or case-level separation and deduplication enforced among the PRM training data, the validation trajectories, and the final VQA test sets—any overlap here could inflate the reported gains.

1 comment
AuthorsOpenReview ↗

Clarification of PRM Motivation, Baseline Adaptations, and Additional Experimental Evidence

We thank the reviewer for recognizing the focused design of our method. We address the concerns regarding our motivation, baseline implementation, and experimental evidence below.

1. Motivation and Figure 1

We acknowledge that low absolute probabilities do not necessarily imply a small log-probability ratio. We will revise Figure 1 to distinguish probability differences from the rectified log-probability ratios used in VA-OPD and Med-OPD.

Our central motivation is that online visual gain and token-level supervision importance are related but distinct quantities. Online visual gain measures changes in teacher support for a generated token under different visual conditions, whereas supervision importance concerns whether the underlying content requires strong visual guidance.

For example, a non-positive original-versus-degraded log-probability ratio may be rectified to zero, even when the position involves a critical pathological feature. This illustrates a potential limitation rather than suggesting that all incorrect tokens receive low weights.

PRM addresses this distinction by learning importance patterns from correct teacher trajectories. PRM determines where supervision should be emphasized, while the image-conditioned teacher determines what the student should learn.

2. Online Baseline Implementation

We clarify that our Med-OPD implementation adopts a simplified configuration under a shared training setup. We retain token-level online visual weighting but do not apply trajectory-level weighting, as our experiments sample one response per prompt. Under this setting, within-prompt trajectory weighting reduces to a constant and cannot distinguish among alternative responses.

We will clarify these adaptations in the manuscript. Accordingly, our comparisons concern the evaluated online weighting variant under the shared experimental setting, rather than the full original Med-OPD configuration.

3. Incorrect-Trajectory Evaluation

We conducted additional experiments on PathMMU 4B using only student trajectories with incorrect final answers for distillation.

Across three runs, the evaluated online method decreased from 73.46% to 71.65% (−1.82 pp), whereas PaRa-OPD decreased from 76.04% to 75.01% (−1.03 pp).

PaRa-OPD exhibits a 0.79-percentage-point smaller performance decrease, indicating lower sensitivity to incorrect-only training and better downstream performance under this condition.

These findings support our motivation that learned PRM weighting can provide advantages over the evaluated online visual-weighting mechanism. We will also include a case study comparing their actual weights on the same tokens in incorrect student trajectories.

Detailed experimental settings and results are provided in our response to Reviewer 2.

4. Simpler Weighting Controls and Multi-Seed Evaluation

We additionally evaluated direct pathology vocabulary weighting. On PathMMU 2B/4B, vocabulary weighting improves standard OPD by 1.03/1.08 pp, but improves Privileged OPD by only approximately 0.21 pp.

PaRa-OPD further outperforms vocabulary-weighted Privileged OPD by 1.52/1.85 pp, suggesting that its benefits cannot be fully explained by direct vocabulary matching.

We also conducted three independent training runs across both datasets and student scales. Compared with Privileged OPD, PaRa-OPD improves mean accuracy by 1.28/1.70 pp on PathMMU 2B/4B and 2.21/3.46 pp on PathGen 2B/4B.

Moreover, PaRa-OPD achieves higher mean accuracy than the evaluated Med-OPD variant in all four settings. Without privileged information, PRM weighting also improves PathMMU 2B/4B accuracy by 1.15/1.41 pp.

These additional experiments provide consistent evidence of PRM's incremental contribution beyond privileged supervision and simple lexical weighting.

Detailed multi-seed comparisons and vocabulary-weighting results are reported in our responses to Reviewers 1 and 2.

5. Training Protocol and Data Separation

We clarify that Table 3 uses 30,000 PathGen training prompts and therefore represents target-domain training rather than zero-shot cross-dataset student transfer.

A shared PRM is constructed from the training splits of PathMMU and PathGen, while student models are trained separately on their respective datasets.

We verified image-level separation between training and test sets and case-level separation between PRM training and validation, keeping all trajectories from the same case within one partition. Benchmark test images are excluded from PRM construction and student training.

We will clarify these procedures and replace "cross-dataset transfer" with "evaluation on a second dataset". Further details are provided in our response to Reviewer 1.

Reviewer_xPtjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PaRa-OPD to improve on-policy distillation (OPD) for pathology vision-language reasoning. Standard OPD assigns equal weights to all tokens, while online visual-gain methods may assign insufficient weights to incorrect tokens generated by the student. To address this issue, the authors train a Pathology Reward Model (PRM) using token-level labels constructed from correct teacher trajectories. These labels are obtained by comparing token probabilities with and without images and matching pathology-specific terms. During OPD training, the PRM predicts the importance of each token in the student-generated trajectory, and its predictions are used to reweight the distillation loss.

Critical strengths
  1. In pathology reasoning, a small number of tokens often carry the key visual evidence and diagnostic information. Standard OPD gives these tokens the same weight as the rest of the response, which may limit the effectiveness of distillation. Giving more attention to these tokens is a reasonable idea.

  2. Online visual-gain weighting may give incorrect student tokens very low weights, since the teacher can assign them low probabilities both with and without the image. The authors notice this issue and try to address it with an offline PRM. I think this is a reasonable direction.

  3. The PRM is only used to adjust the weight of each token in the distillation loss. It does not change the teacher supervision target or require any changes to the student's inference procedure. It can also be used with standard OPD, not just Privileged OPD.

Critical weaknesses
  1. The paper argues that online visual-gain weighting may give insufficient weights to incorrect student tokens. However, there is no direct comparison of the weights given to these tokens by PRM and the online method.

  2. PRM predicts token scores from the generated text alone, without seeing the image or question. Some pathology terms may sound correct but have little to do with what is actually shown in the image. PRM cannot directly check whether these terms are supported by the visual evidence.

  3. The improvement from PRM weighting is rather small on PathMMU. Compared with Privileged OPD, the gain is only 0.62 pp for 2B and 0.97 pp for 4B. Most of the improvement over standard OPD already comes from the privileged teacher.

Questions for authors
  1. Since the PRM only receives generated trajectories, how can it distinguish visually important tokens from pathology terms that are not supported by the image? Can the authors compare the token weights assigned by PRM and online visual-gain methods on incorrect student trajectories, and examine whether PRM provides more effective teacher supervision in these cases?

  2. How does PRM-based weighting compare with simpler methods, such as pathology vocabulary matching or frequency-based token weighting?

6 comments
AuthorsOpenReview ↗

PRM Supervision Mechanism and Performance on Incorrect Trajectories

We thank the reviewer for the constructive comments on the PRM supervision mechanism and its effectiveness on incorrect student trajectories. We address these concerns through methodological clarification and additional experiments.

1. PRM Supervision Mechanism

The PRM is designed to identify tokens whose underlying content requires visual evidence, rather than independently verifying whether they are supported by the current image. For example, assessing tissue morphology or cellular structures generally requires visual information, even when the student's specific description is incorrect. Such tokens may still benefit from stronger visual supervision.

Accordingly, the PRM estimates token-level supervision importance, while the image-conditioned privileged teacher provides the distillation targets. A high PRM score does not reinforce potentially incorrect student content. Instead, it increases the weight of the teacher-matching objective at that position, allowing the teacher's predictions to guide the student more strongly.

This design separates supervision allocation from target construction: the PRM determines where supervision should be emphasized, while the privileged teacher determines what the student should learn.

2. Evaluation on Incorrect Student Trajectories

To evaluate PRM weighting on incorrect trajectories, we conducted an additional experiment on PathMMU 4B, restricting the OPD distillation loss to student trajectories with incorrect final answers. We compared this setting with training on all trajectories, reporting three independent runs for each condition.

Table R1. Incorrect-Trajectory Training on PathMMU 4B (N = 1,138)

| Method | Training Trajectories | Seed A | Seed B | Seed C | Mean ± SD (%) | Change (pp) |

| Online / Med-OPD | All | 832 | 837 | 839 | 73.46 ± 0.32 | — |

| Online / Med-OPD | Incorrect only | 812 | 818 | 816 | 71.65 ± 0.27 | −1.82 |

| PaRa-OPD | All | 863 | 865 | 868 | 76.04 ± 0.22 | — |

| PaRa-OPD | Incorrect only | 851 | 856 | 854 | 75.01 ± 0.22 | −1.03 |

Note: Seeds A/B/C represent the numbers of correct predictions. Mean accuracy and sample standard deviation are calculated across three runs. Changes are relative to the corresponding all-trajectory training results.

Under incorrect-only training, Online / Med-OPD decreases from 73.46% to 71.65% (−1.82 pp), whereas PaRa-OPD decreases from 76.04% to 75.01% (−1.03 pp).

PaRa-OPD exhibits a 0.79-percentage-point smaller accuracy decrease, indicating lower sensitivity to the incorrect-only training condition.

This finding is consistent with our motivation. Online visual-gain scoring relies on teacher probability differences for student-generated tokens. When both image-conditioned and image-free teachers assign low probabilities to an incorrect token, its visual supervision importance may be underestimated. In contrast, PRM allocates supervision weights based on learned importance patterns without relying on these online probability gaps.

Furthermore, we will include a case study comparing PRM and online visual-gain weights on the same tokens in incorrect student trajectories, providing a more direct illustration of their different supervision allocation behaviors.

AuthorsOpenReview ↗

Comparison with Direct Pathology Vocabulary Weighting

3. Comparison with Simpler Token-Weighting Strategies

To address the reviewer's question regarding simpler token-weighting strategies, we conducted additional experiments using direct pathology vocabulary weighting. Specifically, we increased the distillation weights of tokens matching our previously extracted pathology vocabulary, without introducing a learned PRM.

We evaluated this strategy on PathMMU 2B and 4B with three independent runs and compared the results with PaRa-OPD.

Table R2. PathMMU 2B (N = 1,138)

| Method | Seed A | Seed B | Seed C | Mean ± SD (%) |

| OPD | 723 | 728 | 730 | 63.88 ± 0.32 |

| OPD + Vocabulary | 735 | 740 | 741 | 64.91 ± 0.28 |

| Privileged OPD | 802 | 807 | 809 | 70.83 ± 0.32 |

| Privileged OPD + Vocabulary | 805 | 808 | 812 | 71.03 ± 0.31 |

| PaRa-OPD | 824 | 824 | 829 | 72.55 ± 0.25 |

Table R3. PathMMU 4B (N = 1,138)

| Method | Seed A | Seed B | Seed C | Mean ± SD (%) |

| OPD | 759 | 763 | 764 | 66.96 ± 0.23 |

| OPD + Vocabulary | 771 | 775 | 777 | 68.04 ± 0.27 |

| Privileged OPD | 839 | 843 | 844 | 73.99 ± 0.23 |

| Privileged OPD + Vocabulary | 841 | 846 | 846 | 74.19 ± 0.25 |

| PaRa-OPD | 863 | 865 | 868 | 76.04 ± 0.22 |

Note: Seeds A/B/C represent the numbers of correct predictions. Mean accuracy and sample standard deviation are calculated across three runs. PaRa-OPD results are taken from our additional multi-seed evaluation.

Results and Analysis

Direct pathology vocabulary weighting improves distillation performance, but its incremental benefit is relatively limited under privileged teacher supervision.

For standard OPD, vocabulary weighting improves mean accuracy by 1.03 and 1.08 percentage points for the 2B and 4B models, respectively. Under Privileged OPD, the improvements are approximately 0.21 percentage points for both models.

In comparison, PaRa-OPD achieves 72.55% and 76.04% accuracy, outperforming vocabulary-weighted Privileged OPD by 1.52 and 1.85 percentage points, respectively.

Unlike predefined vocabulary matching, PRM learns token importance from trajectory context using both visual-dependence and pathology-vocabulary supervision. These results provide further evidence that context-dependent importance learning offers benefits beyond direct pathology vocabulary weighting, supporting the effectiveness of PRM-guided token-level distillation.

AuthorsOpenReview ↗

Incremental Contribution of PRM and Multi-Seed Evaluation

4. Incremental Contribution Beyond Privileged Supervision

PaRa-OPD combines privileged teacher supervision with PRM-guided token weighting. These components serve complementary roles: the privileged teacher provides richer visual supervision, while the PRM improves the allocation of supervision across student tokens.

To further evaluate the independent contribution of PRM and the consistency of our method, we conducted additional controlled comparisons and multi-seed experiments, which demonstrate that PRM-based weighting provides incremental performance gains under different supervision settings.

First, without privileged information, the additional multi-seed experiments show that PRM weighting improves mean accuracy on PathMMU by 1.15 and 1.41 percentage points for the 2B and 4B models, respectively. This indicates that token-level supervision weighting can improve distillation performance independently of privileged teacher information.

Second, PRM provides further improvements when combined with privileged supervision. In the original single-run results, PaRa-OPD outperforms Privileged OPD by 0.62/0.97 percentage points on PathMMU 2B/4B and 0.72/2.06 percentage points on PathGen 2B/4B.

Our additional multi-seed experiments demonstrate larger mean improvements than those observed in the original single-run results:

Table R4. Multi-Seed Evaluation Results

| Dataset | Student | Privileged OPD (%) | PaRa-OPD (%) | Gain (pp) |

| PathMMU | 2B | 71.27 ± 0.58 | 72.55 ± 0.25 | +1.28 |

| PathMMU | 4B | 74.34 ± 0.35 | 76.04 ± 0.22 | +1.70 |

| PathGen | 2B | 51.72 ± 0.50 | 53.93 ± 0.23 | +2.21 |

| PathGen | 4B | 51.00 ± 0.28 | 54.46 ± 0.23 | +3.46 |

Across all four settings, PaRa-OPD achieves higher mean accuracy than Privileged OPD, with improvements ranging from 1.28 to 3.46 percentage points, exceeding those observed in the original single-run experiments. Moreover, PaRa-OPD outperforms Privileged OPD in all three corresponding runs for each setting, with relatively low training-run variability.

These additional experiments provide further evidence of the effectiveness of PRM-guided weighting under different supervision conditions and the consistent performance advantages of PaRa-OPD across datasets and model sizes.

Overall, PRM does not replace privileged teacher supervision but complements it by learning token importance and emphasizing supervision on tokens relevant to visual dependence and pathology reasoning.

Reviewer_xefwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PaRa-OPD proposes token-level supervision reweighting for on-policy distillation in pathology vision-language models. The method builds on privileged OPD, where the teacher receives reference answers and explanations while the student receives only the image and question. An offline Pathology Reward Model (PRM) is trained on token labels derived from image/no-image teacher probability differences and pathology-vocabulary matches. During distillation, the trajectory-only PRM produces normalized weights for the token-level reverse-KL objective. Experiments use Qwen3.5-2B and 4B students with a 9B teacher on PathMMU and PathGen-Instruct.

Critical strengths
  1. The paper identifies a plausible limitation of online visual-gain weighting and proposes a targeted intervention. An incorrect student token can receive low probability from both image-conditioned and image-free teachers, yielding a weak weighting signal despite requiring correction. Separating importance estimation from online scoring is a reasonable response. The normalized weighting formulation also separates supervision allocation from overall loss scaling.
  2. The component comparisons help distinguish teacher-context improvements from token-weighting improvements. The paper evaluates PRM weighting both with and without privileged teacher information and compares against several relevant distillation methods. PRM weighting improves over privileged OPD in all four reported dataset–student settings, and the weight-scale ablation provides useful evidence favoring moderate, continuous reweighting.
Critical weaknesses
  1. The evidence does not yet establish the proposed mechanism or the necessity of a learned PRM. The PRM is trained on answer-correct teacher trajectories and receives only trajectory text, yet its motivation concerns identifying important errors in incorrect student trajectories. The reported AUROC measures agreement with automatically generated visual-credit/vocabulary labels rather than localization of genuine visual errors. No direct comparison against vocabulary-only weighting, token-prior weighting, or separately trained visual-only and vocabulary-only scorers is provided. Moreover, the incremental PathMMU gains over privileged OPD correspond to approximately seven and eleven additional correct predictions, without reported training-seed variation or paired significance tests. The substantially larger gains over standard OPD therefore should not be attributed primarily to PRM weighting.
  2. The PathGen evaluation protocol and auxiliary-data separation are insufficiently clear. Section 4.2.2 describes PathGen as cross-dataset transfer, whereas Appendix Table 6 lists 30,000 PathGen training prompts for OPD-family methods. It is unclear which training regime produced Table 3. The PRM also uses PathGen examples, so target-dataset exposure must be distinguished from student-level transfer. The manuscript does not clearly document case/image/source-level separation between PRM construction data and evaluation sets. These ambiguities limit interpretation of the transfer results and prevent a complete assessment of data leakage safeguards.
Questions for authors
  1. Were the Table 3 models trained exclusively on PathMMU, or additionally trained on the 30,000 PathGen prompts listed in Table 6? Please specify the training history for each reported checkpoint.
  2. How are PRM training and validation examples separated from both benchmark test sets? Are splits grouped by case, image, patient, or source, and are multiple trajectories from the same case kept together?
  3. Does PRM outperform vocabulary-only, token-prior-only, and visual-credit-only weighting under matched data and training budgets? On independently annotated incorrect student trajectories, does it identify consequential visual errors more accurately than online visual-gain scoring?
  4. Can you report multiple training seeds and paired confidence intervals or significance tests against privileged OPD and Med-OPD, together with the additional cost of PRM data construction, training, and scoring?
5 comments
AuthorsOpenReview ↗

Clarification of PRM Motivation and PathGen Evaluation Protocol

We thank the reviewer for the constructive feedback and careful assessment of our work. We appreciate the opportunity to clarify the motivation behind our PRM, its intended role in knowledge distillation, and the evaluation protocol.

1. PRM Estimates Token Importance Rather Than Detecting Reasoning Errors

We would first like to clarify a distinction central to our method: the PRM estimates token-level importance rather than directly identifying or localizing reasoning errors. Specifically, it identifies tokens associated with visual dependence and pathology-related reasoning, which are important targets for knowledge distillation.

Our discussion of incorrect tokens is intended to illustrate a potential limitation of online visual-gain scoring, rather than to define error detection as the PRM's objective. Online visual-gain scoring relies on the probability difference assigned to student tokens by image-conditioned and image-free teachers. When a student-generated token is incorrect, both teachers may assign it low probabilities, even if the underlying reasoning content requires visual evidence. The resulting small probability difference may therefore underestimate the importance of visually grounded supervision.

To address this limitation, we learn token-importance patterns offline from answer-correct teacher trajectories and apply the learned PRM to student-generated trajectories, independently of their online image-conditioned versus image-free probability gaps.

The PRM estimates token importance and allocates distillation weights, while the privileged teacher provides supervision targets. These components therefore serve complementary roles: the PRM determines where supervision should be emphasized, whereas the teacher determines what should be learned.

We also clarify that the reported AUROC measures agreement with automatically constructed importance labels, rather than accuracy in detecting visual errors. Since the PRM operates solely on trajectory text, it cannot independently verify image consistency. We have refined the manuscript to distinguish token importance, factual correctness, and error localization, and to avoid conflating these objectives.

2. PathGen Evaluation Protocol and Cross-Dataset Setting

We clarify that PathGen represents a separate student training and evaluation setting, rather than zero-shot transfer from a PathMMU-trained student.

Our experimental protocol is as follows:

  1. A shared PRM is trained using the training splits of both PathMMU and PathGen.

  2. Student models are trained independently on their respective datasets.

  3. Each student is evaluated on the corresponding test split.

Specifically, Table 3 uses 30,000 PathGen training prompts, consistent with Appendix Table 6. Therefore, the PathGen results do not represent zero-shot student transfer.

We acknowledge that the original term "cross-dataset transfer" was imprecise. We will revise it to "evaluation on a second dataset" and explicitly distinguish PRM-level knowledge sharing from student-level transfer.

The shared PRM uses training examples from both datasets, including PathGen, while each student is trained separately on its corresponding dataset. We will clarify these training-data exposures and checkpoint histories to ensure that the evaluation protocol and the interpretation of Table 3 are unambiguous.

AuthorsOpenReview ↗

Clarification of Data Separation and the Independent Contribution of PRM

3. Data Separation and Leakage Prevention

We further clarify the data separation procedures used in benchmark evaluation and PRM construction.

First, at the benchmark level, both PathMMU and PathGen have separate training and test splits. PRM construction and student training use only training data. We have verified that training and test images are disjoint, ensuring that test images are excluded from both PRM construction and student training.

Second, at the PRM training and validation level, we adopt case-level splitting. All reasoning trajectories associated with the same case are assigned to the same partition, preventing trajectories from the same case from appearing in both PRM training and validation and reducing potential dependencies between the two partitions.

Importantly, the verified separation covers image-level benchmark splits and case-level PRM training/validation splits. We do not claim patient-level separation without verification using the corresponding patient identifiers.

We will further clarify the data construction procedures, case-grouping rules, and evaluation-subset definitions in the revised manuscript to make the data usage protocol more transparent.

4. Independent Contribution of PRM and Component Ablations

We acknowledge that the overall improvement of PaRa-OPD over standard OPD should not be attributed entirely to the PRM. Our framework combines privileged teacher supervision with PRM-guided token weighting, which serve distinct but complementary roles: the privileged teacher provides richer visual supervision, while the PRM adjusts supervision weights according to token importance. Our existing ablations on teacher context and token weighting are designed to disentangle these contributions.

First, without privileged information, introducing PRM-based weighting improves PathMMU accuracy by 1.15 and 1.41 percentage points for the 2B and 4B students, respectively. These results indicate that the benefits of PRM are not solely attributable to additional privileged information; its token-level supervision allocation can independently improve distillation performance.

Second, PRM provides further gains when combined with privileged supervision. In the original experiments, PaRa-OPD improves over Privileged OPD by 0.62/0.97 percentage points on PathMMU 2B/4B and 0.72/2.06 percentage points on PathGen 2B/4B. These results demonstrate that PRM-based weighting offers additional benefits under the same privileged supervision setting.

To further assess the consistency of these improvements, we conducted three additional independent training runs for each compared method. The results show that PaRa-OPD improves mean accuracy over Privileged OPD by 1.28, 1.70, 2.21, and 3.46 percentage points across the four settings, respectively, exceeding the corresponding improvements observed in the original single-run results. Moreover, PaRa-OPD outperforms Privileged OPD in all three corresponding runs for each setting, with relatively low training-run variability.

These findings provide further evidence of the incremental contribution of PRM-based weighting beyond privileged supervision. We emphasize that we do not attribute the overall performance improvement entirely to the PRM. Rather, its contribution lies in learning token importance to improve the allocation and utilization of existing teacher supervision.

The complete multi-seed results, including individual runs, mean accuracies, standard deviations, and detailed analysis, are provided in our subsequent responses.

AuthorsOpenReview ↗

Multi-Seed Evaluation and Performance Consistency on PathMMU

5. Multi-Seed Evaluation on PathMMU

To further assess the consistency of our results, we conducted three independent training runs for each compared method. The following tables report the original manuscript results (Ref.) alongside three independent runs (Seeds A, B, and C). Mean accuracy and sample standard deviation are calculated from the numbers of correct predictions across the three runs.

The PRM is kept fixed across student-training runs. Thus, these experiments primarily characterize variability in student training under a fixed PRM, rather than variability arising from retraining the entire PRM pipeline.

Table R1. PathMMU 2B (N = 1,138)

| Method | Ref. | Seed A | Seed B | Seed C | Mean ± SD (%) |

| Base | 659 | — | — | — | N/A |

| SFT | 775 | 779 | 779 | 773 | 68.28 ± 0.30 |

| Rej-SFT | 719 | 716 | 712 | 720 | 62.92 ± 0.35 |

| GRPO | 684 | 694 | 683 | 687 | 60.46 ± 0.49 |

| OPD | 727 | 727 | 721 | 724 | 63.62 ± 0.26 |

| TRD | 753 | 752 | 762 | 757 | 66.52 ± 0.44 |

| VA-OPD | 791 | 785 | 789 | 793 | 69.33 ± 0.35 |

| Priv. OPD | 806 | 810 | 818 | 805 | 71.27 ± 0.58 |

| Med-OPD | 808 | 801 | 806 | 802 | 70.56 ± 0.23 |

| PaRa-OPD | 813 | 824 | 824 | 829 | 72.55 ± 0.25 |

Table R2. PathMMU 4B (N = 1,138)

| Method | Ref. | Seed A | Seed B | Seed C | Mean ± SD (%) |

| Base | 747 | — | — | — | N/A |

| SFT | 819 | 820 | 826 | 820 | 72.23 ± 0.30 |

| Rej-SFT | 783 | 781 | 776 | 786 | 68.63 ± 0.44 |

| GRPO | 781 | 785 | 786 | 778 | 68.80 ± 0.38 |

| OPD | 762 | 755 | 760 | 765 | 66.78 ± 0.44 |

| TRD | 788 | 796 | 788 | 795 | 69.68 ± 0.38 |

| VA-OPD | 799 | 802 | 789 | 794 | 69.86 ± 0.58 |

| Priv. OPD | 842 | 846 | 850 | 842 | 74.34 ± 0.35 |

| Med-OPD | 839 | 832 | 837 | 839 | 73.46 ± 0.32 |

| PaRa-OPD | 853 | 863 | 865 | 868 | 76.04 ± 0.22 |

Results and Analysis

Across both model sizes on PathMMU, PaRa-OPD achieves the highest mean accuracy among all compared methods.

For the 2B model, PaRa-OPD achieves 72.55 ± 0.25%, outperforming Privileged OPD (71.27 ± 0.58%) by 1.28 percentage points.

For the 4B model, PaRa-OPD achieves 76.04 ± 0.22%, compared with 74.34 ± 0.35% for Privileged OPD, yielding an improvement of 1.70 percentage points.

Notably, PaRa-OPD exhibits standard deviations of 0.25 and 0.22 percentage points for the 2B and 4B models, respectively, both lower than those of Privileged OPD. Moreover, PaRa-OPD achieves more correct predictions than Privileged OPD in all three corresponding runs for both model sizes.

Compared with the original single-run results, the mean improvements over Privileged OPD increase from 0.62 to 1.28 percentage points for the 2B model and from 0.97 to 1.70 percentage points for the 4B model.

These results provide further evidence that PRM-guided token weighting delivers consistent incremental benefits beyond privileged teacher supervision, rather than relying on a favorable outcome from a single training run.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.