- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PURIST, a training framework for Positive–Unlabeled learning. It combines balanced P sampling with Self-Adaptive EMA Soft Labels (SAEM) to balance the training contributions of reliable positive and unlabeled examples while iteratively refining the soft labels of unlabeled examples. The authors further analyze the ranking properties under SCAR and SAR settings and conduct experiments on 21 datasets spanning six data types, demonstrating the effectiveness of the proposed method and the synergy between its two components.
Critical strengths
-
The authors conduct extensive experiments on 21 SCAR datasets spanning six data types, comparing PURIST against 25 non-oracle methods. They further evaluate three SAR labeling mechanisms on 14 datasets.
-
PURIST uses only a single classifier and does not require additional data augmentation. It refines the training targets of unlabeled examples through a simple EMA update. The method is straightforward, easy to implement, and practically useful.
-
The authors provide a relatively systematic theoretical analysis of how fixed P sampling ratios affect the optimal ranking under SCAR.
Critical weaknesses
-
The two core components of PURIST, balanced P sampling and EMA-based soft-label refinement, are closely related to existing research. In particular, SatPU and NcPU have already explored self-adaptive training or iterative label refinement mechanisms in PU learning. However, these methods are only discussed in the Related Work section, without direct experimental comparisons.
-
According to Table 5, PURIST generally outperforms the balanced BCE baseline without SAEM (i.e., DT) at lower labeling rates. However, when (c=0.3) and (c=0.5), PURIST achieves average AUCs of 96.2% and 96.5%, respectively, which are lower than DT's 97.0% and 97.5%. This suggests that SAEM does not consistently provide performance improvements across different labeling conditions.
-
The authors construct SAR-A, SAR-B, and SAR-C using a posterior proxy trained on fully labeled source data. However, these synthetic labeling mechanisms do not necessarily reflect the instance-dependent labeling bias encountered in real-world scenarios.
Questions for authors
-
What are the key differences between PURIST and existing PU label refinement methods such as SatPU and NcPU? Could the authors provide direct comparisons under a unified experimental setting to clarify PURIST's distinct contributions and performance advantages?
-
Table 5 shows that SAEM can degrade the performance of balanced BCE at higher labeling rates. Could the authors explain why this occurs?
-
Could the authors further evaluate the robustness of PURIST on PU datasets with labeling mechanisms that better reflect real-world scenarios?