- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SFDA-SPF, a source-free domain adaptation framework for weakly supervised object localization (WSOL) in histopathology. The method addresses prediction bias under severe domain shifts, where source-trained models tend to assign a disproportionate number of target samples to a small subset of classes. SFDA-SPF identifies prediction-dominant classes using the empirical distribution of target predictions. It then selects high-entropy samples from these classes to form a forget set, while the remaining samples constitute a retain set. A complementary forgetting loss reduces the probability assigned to the current predicted classes of forget samples, whereas a cross-entropy objective preserves the predictions of retained samples. The method also incorporates an auxiliary pixel-level localization loss using CAM-derived partial pseudo-labels from the frozen source model. Experiments on GlaS, CAMELYON16, CAMELYON17, and an additional EBHI corruption benchmark evaluate the method under cross-organ, cross-center, class-imbalance, and multi-class settings. The proposed method reports substantial localization improvements over several existing SFDA baselines.
Critical strengths
- A meaningful problem formulation for source-free adaptation under prediction bias.
The paper addresses an important challenge in source-free domain adaptation: unreliable pseudo-labels may reinforce dominant-class prediction bias under severe domain shifts.
Instead of relying exclusively on confident predictions or global diversity regularization, the proposed approach explicitly identifies uncertain samples from prediction-dominant classes and attempts to reconsider their current assignments. This provides a coherent motivation for addressing prediction collapse in histopathology localization.
- Encouraging localization performance across multiple WSOL architectures and domain shifts.
The experimental evaluation covers multiple histopathology datasets and three different WSOL architectures, including CNN- and transformer-based models.
The reported localization improvements are substantial in several challenging settings, particularly when conventional SFDA methods suffer from strong prediction bias. The paper also provides component ablations, analyses of dynamic set reconstruction, class-imbalance experiments, and supplementary evaluations under multi-class corruptions.
These experiments provide useful evidence that the proposed adaptation framework can improve localization under certain challenging domain shifts.
Critical weaknesses
- The evaluation relies on labeled target-domain validation, limiting the strength of the unsupervised adaptation claim.
Although the adaptation objective uses unlabeled target training data, Appendix B.2.2 explicitly states that hyperparameters and checkpoints are selected using a separate labeled target-domain validation set.
More importantly, Appendix A.1 reports that localization-validation images and their pixel-level annotations are used for localization model selection. This is particularly relevant because the proposed approach introduces multiple tunable components, including the forgetting ratio, the reconstruction period, and the weights of the classification and localization losses.
The authors clearly disclose this validation-assisted protocol, and the use of target validation data does not itself indicate test-set leakage. Nevertheless, the complete experimental procedure is not fully target-label-free, which substantially limits its applicability to settings where target-domain annotations are unavailable.
This distinction matters because source-free unsupervised domain adaptation is generally motivated by the absence of target labels. Moreover, using target localization annotations for model selection may favor configurations that improve pixel-level localization, making it difficult to determine whether similar improvements would be obtained under a fully unsupervised model-selection protocol.
A comparison using target-label-free hyperparameter and checkpoint selection would substantially strengthen the experimental evidence and clarify the practical applicability of SFDA-SPF.
- The specific contribution and generality of selective prediction forgetting are insufficiently isolated.
The proposed approach assumes that uncertain samples assigned to prediction-dominant classes are particularly likely to contain incorrect pseudo-labels. However, high prediction entropy does not necessarily imply misclassification, and prediction dominance does not necessarily indicate erroneous over-prediction.
The criterion based on prediction frequency exceeding 1/K cannot distinguish genuine target-domain class imbalance from prediction collapse without additional assumptions about the target label distribution. Consequently, the forgetting objective may suppress correct predictions in naturally imbalanced domains.
Although the paper includes component ablations, the experiments do not sufficiently isolate the benefit of selectively forgetting uncertain samples from dominant classes. In particular, comparisons against random forgetting within dominant classes, class-agnostic high-entropy forgetting, and confidence-based sample reweighting under matched training conditions would help determine whether the proposed selection principle is essential.
Furthermore, the auxiliary localization loss contributes substantially to the final localization performance. For example, Table 4 shows that combining the image-level forget and retain objectives yields only 13.1 PxAP, whereas adding the localization loss increases PxAP to 49.9. The comparisons would be more convincing if existing SFDA baselines were also evaluated with the same auxiliary localization supervision.
The generalization claims also require qualification. The proposed method does not consistently achieve the best classification accuracy, and its performance is less competitive in reverse cross-organ adaptation and certain multi-class corruption settings. On EBHI, it underperforms SHOT in average classification F1 and exhibits substantial degradation under severe blur and contrast shifts.
Finally, the absence of repeated-seed performance statistics makes it difficult to assess the stability of the reported improvements.
Together, these limitations weaken the evidence that the selective forgetting strategy itself is responsible for the reported gains and that its benefits generalize across different adaptation settings.
Questions for authors
Q1. Target-label-free model selection.
Appendix B.2.2 states that labeled target-domain validation data are used for hyperparameter and checkpoint selection, and Appendix A.1 indicates that pixel-level target annotations are available for localization model selection.
Could the authors clarify exactly which validation metrics are used to select each hyperparameter and checkpoint? How much performance degradation occurs when model selection is performed without any target-domain labels or localization masks?
Such an experiment would materially strengthen the paper's source-free unsupervised adaptation claims.
Q2. Validity of the dominant-class forgetting criterion.
How does the proposed method distinguish erroneous dominant-class prediction bias from genuine target-domain class imbalance?
Could the authors compare their entropy-based selection strategy against random forgetting within dominant classes and class-agnostic high-entropy forgetting, while keeping the forgetting ratio and other optimization settings identical?
These comparisons would help establish whether the proposed selection mechanism is necessary rather than merely the application of negative learning to uncertain target predictions.
Q3. Contribution of the auxiliary localization objective.
Table 4 indicates that the localization loss accounts for a substantial improvement in PxAP.
Could the authors evaluate representative SFDA baselines augmented with the same CAM-based pixel-level localization objective? This would allow a more controlled comparison and help distinguish the contribution of selective prediction forgetting from the benefits of additional localization supervision.
Q4. Robustness under severe multi-class shifts and repeated training runs.
The EBHI experiments show that SFDA-SPF is less effective under severe blur and contrast shifts, where some classes receive almost no source-model predictions.
Could the authors further analyze the failure mode in which classes disappear from the predicted target distribution? Additionally, could they report repeated-seed results with standard deviations or confidence intervals for representative cross-organ and cross-center experiments?
These results would help assess the robustness of the proposed method.
3 comments
W2 — Prediction dominance and target-class imbalance (Sec .3, Tab .3, 6 and 15 and Appendix E)
In practice, the true target class proportions are unknown, so SFDA-SPF relies only on observable prediction cues. As described in Sec. 3, the threshold identifies prediction-dominant classes without estimating the true target priors or explicitly enforcing uniform predictions. Entropy serves as an empirical indicator of unreliability, allowing SFDA-SPF to prioritize assignments that are more likely to be incorrect.
Empirical evidence on severe shifts. App. E.1–E.2 support this choice under both cross-organ and inter-center shifts, across target class priors from 50:50 to 10:90. Misclassified samples concentrate in the mid- and high-entropy bins, whereas low-entropy predictions are mostly correct (Fig. 24), and accuracy decreases with entropy in most settings (Fig. 25). The controlled-imbalance experiments are conducted on GlaS→C17-0, combining a substantial histological domain shift with varying target class proportions. Figure 6 further shows that retained pseudo-labels become more accurate during adaptation, while the forget set contains a higher proportion of incorrect assignments.
Robustness to genuine imbalance. Tables 3 and 15 evaluate GlaS→C17-0 under both Normal-majority and Cancer-majority target distributions, up to 10:90, with a balanced test set. Across these imbalanced settings, SFDA-SPF maintains 79.0–86.2 CL and 43.7–58.8 PxAP, supporting its robustness to varying target class proportions.
To isolate the selection criterion, we additionally compare SFDA-SPF against random forgetting within dominant classes and class-agnostic high-entropy forgetting on GlaS→C17-0 (PixelCAM), using the same losses, forgetting ratio (). Results are reported for different Cancer:Normal ratios of the target training set. Added in Sec .3 Tab. 6.
| Selection strategy | 50:50 | 40:60 | 30:70 | 20:80 | 10:90 |
|---|---|---|---|---|---|
| Class-agnostic high entropy | 79.0 / 28.9 | 55.7 / 25.7 | 50.7 / 25.7 | 50.7 / 14.4 | 77.9 / 23.2 |
| Random within dominant classes | 76.3 / 35.4 | 78.2 / 32.8 | 80.5 / 42.2 | 78.2 / 30.4 | 78.2 / 37.9 |
| SFDA-SPF | 86.2 / 49.9 | 79.0 / 43.7 | 82.8 / 55.1 | 79.4 / 46.0 | 82.8 / 58.8 |
Class-agnostic forgetting may suppress correct but uncertain predictions in less frequently predicted classes, potentially harming adaptation. Random forgetting within dominant classes can still select incorrect assignments when prediction errors concentrate in these classes. However, entropy-based selection performs better across all evaluated class ratios, suggesting more effective identification of unreliable assignments.
W1 Q2 — Entropy/diversity objectives combined with localization (Sec 4.3)
The complementary contributions of prediction correction and localization supervision are examined in Sec. 4.3 (Table 4 and Fig. 5(a)). To directly address the comment, we additionally added Table 5 to Sec. 4.3 of the main paper, applying the same (CAM-derived foreground/background pseudo-labels from the frozen source model, without target pixel-level annotations) to all five SFDA baselines on C17-0→C17-2 (PixelCAM). SHOT combines entropy minimization and prediction diversity, while ERL uses entropy-based regularization.
| Method | CL | PXAP |
|---|---|---|
| SHOT | 77.0 | 31.6 |
| SHOT + | 56.4 | 25.1 |
| CDCL | 74.7 | 30.6 |
| CDCL + | 72.1 | 35.8 |
| SFDA-DE | 50.0 | 18.7 |
| SFDA-DE + | 49.4 | 23.9 |
| ERL | 58.4 | 45.6 |
| ERL + | 71.5 | 42.2 |
| RGV | 50.0 | 41.2 |
| RGV + | 51.2 | 24.9 |
| Ours | 81.6 | 58.1 |
Overall, SFDA-SPF exceeds the strongest localization-augmented baseline by 9.5 CL and 15.9 PxAP points, considering the best baseline separately for each metric.
Contribution of localization supervision. Adding does not consistently improve the baselines. It increases localization performance for CDCL and SFDA-DE but degrades it for SHOT and RGV. This behavior is consistent with the dependence of on image-level pseudo-label quality: when a prediction is incorrect, the CAM of the wrong class may introduce misleading pixel-level supervision.
Table 4 shows that combining without localization supervision achieves 84.7 CL but only 13.1 PxAP, compared with 86.1 CL / 49.9 PxAP for the complete objective. This highlights the importance of CAM-derived localization in cross-organ adaptation, where substantial differences in tissue architecture and cellular morphology make accurate localization particularly challenging.
W2 Q4 — Failure mode under severe prediction collapse (Appendix C.4)
Failure analysis. Appendix C.4 (Tables 19–20) reveals two regimes. When the source model predicts all six classes (Target Clean, Brightness), SFDA-SPF achieves the highest F1. Under Defocus Blur and Contrast, predictions collapse: High-grade IN, representing only 8.2% of target samples, receives 80.2% and 92.1% of predictions, respectively, while Low-grade IN (28.6%) receives none.
Our forgetting objective suppresses the current dominant-class prediction but does not identify the correct alternative class. Moreover, classes absent from the initial predictions receive no positive pseudo-label supervision at initialization, making their recovery particularly challenging. SHOT may be less affected because its diversity objective discourages prediction collapse.
Additional diversity ablation. To further investigate this limitation, we added the same marginal prediction-diversity regularizer used in SHOT to SFDA-SPF. Results are reported as F1 / PxAP (%).
| Target shift | SFDA-SPF | SFDA-SPF + KL | SHOT |
|---|---|---|---|
| Stain-heavy | 57.9/82.7 | 66.0/83.15 | 70.29/86.74 |
| Dust | 72.4/86.7 | 81.0/81.12 | 79.39/82.93 |
| Air Bubble | 66.5/78.4 | 75.0/73.92 | 78.07/77.84 |
| Defocus Blur | 21.6/76.5 | 43.0/71.28 | 42.24/73.25 |
| Motion Blur | 29.3/72.9 | 35.0/70.97 | 21.13/70.68 |
| Gaussian Noise | 57.2/88.2 | 64.0/85.57 | 60.94/82.92 |
| Shot Noise | 52.6/86.8 | 57.0/86.41 | 46.51/84.29 |
| Brightness | 79.4/84.6 | 82.0/89.03 | 70.88/81.09 |
| Contrast | 12.1/62.6 | 25.0/66.70 | 30.80/71.10 |
Adding diversity improves F1 across all nine evaluated conditions, particularly under Defocus Blur (21.6 → 43.0) and Contrast (12.1 → 25.0), and outperforms SHOT in F1 on six of nine conditions.
To assess robustness, we report the mean ± standard deviation over five random seeds on GlaS→C17-0 and C17-0→C17-2. We used a fixed learning rate of and , following the same hyperparameter selection protocol as in the main paper, based solely on classification accuracy on the target validation set.
C17-0 → C17-2 (5 seeds)
| Method | CL | PxAP |
|---|---|---|
| SHOT | 75.4 ± 1.2 | 31.2 ± 0.4 |
| SFDA-DE | 48.8 ± 3.0 | 17.2 ± 3.1 |
| CDCL | 74.9 ± 1.5 | 41.9 ± 11.4 |
| ERL | 57.9 ± 1.9 | 47.8 ± 3.9 |
| RGV | 50.0 ± 0.2 | 41.3 ± 0.1 |
| Ours | 80.1 ± 1.1 | 50.4 ± 5.1 |
GlaS → C17-0 (5 seeds)
| Method | CL | PxAP |
|---|---|---|
| SHOT | 64.9 ± 15.6 | 18.8 ± 10.6 |
| SFDA-DE | 46.3 ± 5.1 | 23.9 ± 9.1 |
| CDCL | 50.0 ± 0.1 | 32.9 ± 0.2 |
| ERL | 44.2 ± 0.6 | 21.4 ± 1.2 |
| RGV | 50.0 ± 0.1 | 33.9 ± 0.1 |
| Ours | 84.2 ± 2.3 | 44.0 ± 4.8 |
Q1 — Hyperparameter selection and validation protocol (Sec. 4.1 and Appendix B.1–B.2.2)
We thank the reviewer for raising this important practical question. Our evaluation follows the validation-assisted protocol described in Sec. 4.1 and Appendix B.1–B.2.2. Hyperparameter selection remains an open practical challenge in SFDA [1,2,3,4]. Different approaches have therefore been explored. For example, Xu et al. [1, Appendix C.5] use a validation set with 300 manually annotated target samples; [2] report the best performance across different learning rates for each method on the test set. [3] also mentioned using a validation set based on classification for WSOL adaptation. To remain consistent with the state of the art, our main experiments follow the validation-assisted evaluation framework for SFDA of WSOL models introduced in prior work [3], with histology dataset construction following [4].
Selection metric: For our method and every baseline, all adaptation hyperparameters and checkpoints are selected with one criterion: image-level classification accuracy on the same held-out target validation split (Macro-F1 on EBHI). No target pixel-level annotation or PxAP score is used for hyperparameter tuning. The test set remains strictly independent and is never used for model selection, preventing data leakage between training, validation, and testing. This follows the validation-assisted protocol of [3] for SFDA of WSOL models. For the main experiments, the hyperparameter search ranges are reported in App. B.1–B.2. Configurations and checkpoints are selected using the image-level validation criterion described above. The five parameters are selected through grid search using the classification-based validation criterion: λR,λF∈{0.5,1.0}, L∈{0.5,1.0,5.0}, ρ∈{0.05,0.15,0.25} and m∈{5,15,35} for CAMELYON17 or {3,5} for CAMELYON16 (Sec. 4.1 and Appendix B.1–B.2.2). Reduced search spaces are used for the additional experiments, as detailed in Appendix B.1.
Validation-set sizes: The image-level validation splits contain 18 samples for GlaS, 8,850 for C16, and 38–146 for the C17 centers (Table 7). For EBHI, the validation set contains 223 images, with 6–79 per class (Table 8).
Additional evaluation without target-label-based selection. Selection without any target label. To assess the performance of our method against baselines assuming no validation set for hyperparameter setup, we reran GlaS→C17-0 (PixelCAM) with a single fixed configuration for all methods: all loss weights set to 1, learning rate 10−4 decayed by 0.9 every 20 epochs, batch size 32, 100 epochs. SFDA-SPF uses default values ρ=0.1 and m=5, not tuned on this task. Test results, evaluated every 10 epochs:
Classification (CL)
Localization (PxAP)
References