- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PSMG addresses weakly supervised whole-slide image classification through slide-specific generated text, reinforcement-based region reviewing, and multi-scale hypergraph learning. SGRM converts textual descriptions into semantic guidance for low-magnification regions; RSR uses a PPO reviewer to enhance selected candidate regions; and CMHA integrates both magnifications with prediction-level, high-to-low distillation. The model follows a two-stage training procedure and reports competitive classification performance on Camelyon16, TCGA-Lung, and BRACS. The problem is relevant, but the evidence does not yet adequately establish that the method resolves its central challenges: unreliable local grounding of generated text, task-dependent region selection, and diagnostically meaningful consistency across magnifications.
Critical strengths
S1: Relevant problem formulation. The paper distinguishes semantic relevance from diagnostic utility: regions that match a textual description are not necessarily the most informative regions for classification. This provides a useful motivation for reviewing semantic candidates using task feedback.
S2: Reasonable integration of complementary information. Semantic modulation, spatial correspondence between magnifications, and hypergraph aggregation form a plausible framework for combining contextual and fine-grained evidence.
S3: Useful experimental coverage. The paper reports patient-level five-fold evaluation on three datasets, component and reviewer ablations, text-perturbation experiments, and qualitative visualizations. The full model consistently improves upon its Stage-1 variant in the reported AUC results.
Critical weaknesses
W1. The implementation does not convincingly establish the claimed task-feedback-driven policy-learning mechanism. In the inspected public PPO path, the same terminal reward is assigned to every step, and advantages are normalized within each trajectory. When normalization is enabled and the trajectory contains more than one step, centering (A_t=R-V_t) cancels the constant reward (R), removing its direct contribution to the actor’s surrogate objective. The value loss still uses the reward, and shared parameters may convey an indirect influence, so this observation does not establish that all task information is absent from learning. Furthermore, reviewer scores do not explicitly condition on the content of previously selected regions beyond action masking. The exact experimental configuration and a focused diagnostic are needed to establish how the reported reviewer learns task-dependent, subset-sensitive selection.
W2. Local grounding and correction of misleading text priors remain insufficiently demonstrated. A learned projection between independently pretrained visual and textual features is a valid design choice, but its diagnostic grounding requires evidence. The text-perturbation experiments support the usefulness of textual priors, and Table 6 provides localization results on two representative slides. However, these analyses do not directly connect generated descriptions to semantic weights, reviewer decisions, and the corresponding diagnostic regions. Reviewing is also restricted to semantic candidates, so regions outside that pool cannot receive additional reviewing enhancement. This differs from exclusion from final classification: the public soft-weight implementation retains all low-magnification regions with valid high-magnification children, whereas Equation (9) describes semantic Top-K retention. Clarifying the implementation used for the experiments is necessary to interpret how reviewing handles misleading or incomplete textual guidance.
W3. The evidence for diagnostically meaningful cross-magnification consistency is limited. CMHA aligns class-probability distributions using the high-magnification branch as a stop-gradient teacher. The classification objectives and alignment ablation provide evidence of task-level benefits, but agreement alone does not establish that the branches rely on corresponding, correct diagnostic evidence. An incorrect teacher prediction may also influence the low-magnification branch. The paper does not directly analyze branch disagreement, shared errors, or cases in which alignment changes correctness, leaving the mechanism behind the reported alignment gains insufficiently explained.
Presentation Comment: Figure 2 should distinguish feature modulation, parameter updates, and reward feedback more clearly. According to Section 3.7, Stage 2 freezes the scale-specific hypergraph encoders, SGRM, and CMHA while RSR modulates the low-magnification features. This procedure is logically consistent, but the figure should label the trainable components and feedback paths explicitly. In the public code, freezing depends on configuration, and freeze_main_for_ppo is disabled by default. The experimental settings should therefore specify how the stated freezing procedure was implemented.
Questions for authors
Q1. How does terminal reward directly influence the actor in the reported configuration, given its cancellation under within-trajectory advantage normalization for multi-step trajectories? Please provide the exact PPO and freezing settings.
Q2. Could you show representative generated descriptions alongside semantic weights, reviewer selections, and corresponding diagnostic regions, including a case with misleading textual guidance?
Q3. Using the existing alignment ablation, how do branch disagreement and prediction correctness change with alignment, particularly when the high-magnification teacher is incorrect?
6 comments
Thank you for your clarification. As I understand it from your response, the top-256 cutoff applies only to PPO review, and Stage 2 freezes the Stage 1 modules. The new branch-level analysis also addresses much of my concern about cross-magnification alignment by showing both corrections and errors introduced by alignment.
As a follow-up about W1, could you compare two runs of the corrected PPO implementation in which the actor uses the true terminal rewards in one run and rewards shuffled across trajectories in the other? It would be clearer to report paired per-fold AUCs for each dataset to help isolate the contribution of the restored reward signal to the actor.
We appreciate the reviewers' valuable suggestions. Based on these suggestions, we conducted control experiments using a modified PPO implementation. All other experimental configurations remained unchanged.
| Dataset | Fold | True Reward | Shuffled Reward | ΔAUC |
|---|---|---|---|---|
| Camelyon16 | 1 | 96.5 | 96.1 | +0.4 |
| 2 | 97.1 | 95.4 | +1.7 | |
| 3 | 97.9 | 98.0 | -0.1 | |
| 4 | 98.6 | 95.6 | +3.0 | |
| 5 | 99.3 | 96.8 | +2.5 | |
| Mean ± SD | 97.9 ± 1.1 | 96.4 ± 1.1 | +1.5 ± 1.3 | |
| TCGA_Lung | 1 | 95.8 | 94.3 | +1.5 |
| 2 | 96.5 | 95.1 | +1.4 | |
| 3 | 97.1 | 95.5 | +1.6 | |
| 4 | 97.7 | 96.2 | +1.5 | |
| 5 | 98.4 | 96.9 | +1.5 | |
| Mean ± SD | 97.1 ± 1.0 | 95.6 ± 1.0 | +1.5 ± 0.1 | |
| BRACS | 1 | 89.9 | 88.2 | +1.7 |
| 2 | 90.9 | 91.2 | −0.3 | |
| 3 | 91.8 | 89.4 | +2.4 | |
| 4 | 92.7 | 90.5 | +2.2 | |
| 5 | 93.7 | 90.7 | +3.0 | |
| Mean ± SD | 91.8 ± 1.5 | 90.0 ± 1.2 | +1.8 ± 1.3 |
The results show that correctly allocating task rewards leads to a sustained overall improvement across all three datasets. The Camelyon16 dataset exhibits some fluctuations across folds, while the TCGA_Lung dataset shows the most stable improvement, with positive growth across all five folds. The BRACS dataset shows the largest average improvement. Slight decreases in folds are observed in Camelyon16 and BRACS, possibly due to variations in the effectiveness of task feedback depending on the data partitioning. However, their overall contribution to policy learning remains positive.
We thank the reviewer for the careful reading of the public implementation and for the precise diagnosis.
W1-R1: We agree with this diagnosis, but we do not believe that all task information is completely missing during the learning process. In the published PPO path, the final reward is broadcast to all T steps, and the advantage value is z-fractionated within each trajectory; centralization cancels out the constant R in A_t = R − V_t, so the Actor's proxy objective does not directly contribute to the reward, while the value loss still consumes R, and the shared encoder may have indirect effects. We correct this problem by removing the centralization of each trajectory and using A_t = R − b with a buffer-level moving baseline, which preserves R in the Actor gradient. We also add a summary of the selected regions to the policy state, so the reviewer's scoring criteria are a subset of reviewed regions, not an action mask. The following diagnostic results isolate the reward path.
W2-R2: We have added a qualitative analysis in Appendix C.2, presenting representative cases that connect LLM-generated descriptions, semantic weights, PPO reviewer selections, and ground-truth tumor annotations, including a case with misleading textual guidance. We have also revised the description of SGRM to clarify that semantic weights are used for feature modulation rather than region exclusion. Specifically, we have corrected Eq. (9) to reflect the actual implementation, in which all low-magnification regions with at least one valid high-magnification child are retained for classification. Their corresponding high-magnification regions are retrieved through the parent-child mapping in Eq. (10), and both magnification levels are retained for final classification in Eq. (11). The top-256 semantic candidates are used only to construct the PPO review candidate pool, rather than to exclude regions from final classification. We have also revised the accompanying text to ensure consistency with the implementation.
W3-R3: We agree that consistency at the predictive level does not constitute common, accurate diagnostic evidence. Using existing alignment ablation methods, we now report branch inconsistencies, common errors, and alignment-altered predictive accuracy. These analyses further confirm the task-level effects of alignment across magnification, while also highlighting its limitations when high-magnification teachers produce incorrect predictions.
W4-R4: We modified Figure 2 to differentiate between feature modulation, parameter updates, and reward feedback, explicitly stating that RSR is the only trainable component in the second phase. Regarding the freeze configuration, although freeze_main_for_ppo is disabled by default in the published configuration, we explicitly enabled it using the checkpoints from the first phase in our reported second-phase experiments. We will clarify this setting in Section 3.7, providing the corresponding commands and configuration files.
Q1-A1: In the reported configuration it does not directly influence the actor; only the corrected configuration restores that path. The exact settings are below.
Q2-A2: For information on qualitative visualization, see R2 and Appendix C.2, including examples of misleading text.
Q3-A3: Please refer to R3. We acknowledge that consistency at the predictive level does not guarantee consistency of diagnostic evidence. The accuracy of teacher predictions may affect the low-metaphorical branch. We will discuss this limitation and consider cross-metaphorical calibration of reliability as a direction for future work.