acceptodds
Withdrawn from ICLR 2027

PURIST: PU Refinement via Iterative Soft-Label Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Positive–unlabeled (PU) learning trains classifiers from trusted positives and mixed unlabeled examples. Existing methods compensate for missing negatives through different assumptions, but their effectiveness varies across data types and labeling conditions. We investigate which design principles transfer across heterogeneous PU problems. PURIST builds on two structural properties: under selected completely at random (SCAR) labeling, population-optimal P–U binary cross-entropy preserves optimal ranking at any fixed sampling ratio; label noise is confined to unlabeled examples. It combines balanced P:U optimization with Self-Adaptive EMA soft labels (SAEM), balancing training on trusted positives and noisy unlabeled examples while refining only unlabeled targets. Factorial ablations reveal a significant positive interaction between balancing and target refinement. Training dynamics further show that balancing improves target quality early, providing a more reliable starting point for subsequent temporal refinement. The method uses one model without a supplied class prior or data augmentation. Under selected at random (SAR) labeling, we analyze ranking preservation, AUC-regret bounds under anti-monotone selection, and non-identifiability under zero-correlation selection. Across 21 SCAR datasets from six source-data types, PURIST achieves an average AUC of 0.919 and the best average rank of 5.0 among 25 methods; it also ranks best on average under three proxy-defined SAR constructions on 14 datasets.

Primary area:
learning theory
Keywords:
Positive–unlabeled learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LazxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PURIST, a training framework for Positive–Unlabeled learning. It combines balanced P sampling with Self-Adaptive EMA Soft Labels (SAEM) to balance the training contributions of reliable positive and unlabeled examples while iteratively refining the soft labels of unlabeled examples. The authors further analyze the ranking properties under SCAR and SAR settings and conduct experiments on 21 datasets spanning six data types, demonstrating the effectiveness of the proposed method and the synergy between its two components.

Critical strengths
  1. The authors conduct extensive experiments on 21 SCAR datasets spanning six data types, comparing PURIST against 25 non-oracle methods. They further evaluate three SAR labeling mechanisms on 14 datasets.

  2. PURIST uses only a single classifier and does not require additional data augmentation. It refines the training targets of unlabeled examples through a simple EMA update. The method is straightforward, easy to implement, and practically useful.

  3. The authors provide a relatively systematic theoretical analysis of how fixed P sampling ratios affect the optimal ranking under SCAR.

Critical weaknesses
  1. The two core components of PURIST, balanced P sampling and EMA-based soft-label refinement, are closely related to existing research. In particular, SatPU and NcPU have already explored self-adaptive training or iterative label refinement mechanisms in PU learning. However, these methods are only discussed in the Related Work section, without direct experimental comparisons.

  2. According to Table 5, PURIST generally outperforms the balanced BCE baseline without SAEM (i.e., DT) at lower labeling rates. However, when (c=0.3) and (c=0.5), PURIST achieves average AUCs of 96.2% and 96.5%, respectively, which are lower than DT's 97.0% and 97.5%. This suggests that SAEM does not consistently provide performance improvements across different labeling conditions.

  3. The authors construct SAR-A, SAR-B, and SAR-C using a posterior proxy trained on fully labeled source data. However, these synthetic labeling mechanisms do not necessarily reflect the instance-dependent labeling bias encountered in real-world scenarios.

Questions for authors
  1. What are the key differences between PURIST and existing PU label refinement methods such as SatPU and NcPU? Could the authors provide direct comparisons under a unified experimental setting to clarify PURIST's distinct contributions and performance advantages?

  2. Table 5 shows that SAEM can degrade the performance of balanced BCE at higher labeling rates. Could the authors explain why this occurs?

  3. Could the authors further evaluate the robustness of PURIST on PU datasets with labeling mechanisms that better reflect real-world scenarios?

Reviewer_Wp2hOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PURIST combines balanced sampling of labeled positives and unlabeled examples with EMA based soft target refinement, while keeping positive labels fixed. The paper analyzes population ranking under SCAR and three proxy defined SAR settings, and evaluates 25 non oracle methods on 21 datasets spanning six data types. The results show a positive interaction between balanced sampling and target refinement, although the overall AUC improvement over the strongest reported baseline is marginal.

Critical strengths
  1. The paper evaluates 25 non oracle methods across 21 datasets spanning six data types, providing a broad empirical comparison under a unified PU learning framework.

  2. The factorial ablation identifies a statistically significant interaction between balanced sampling and SAEM. The target quality analysis further provides empirical evidence for why the two components work better together.

Critical weaknesses
  1. The method primarily combines balanced sampling with EMA based per instance soft target refinement, both of which have been explored in related work. HolisticPU already employs balanced resampling, while SatPU and NcPU also investigate iterative target refinement for unlabeled examples. The paper does not sufficiently establish the substantive novelty of this combination over existing methods or provide direct comparisons with the most relevant approaches.

  2. Proposition 4.2 proves that SAEM preserves the initial model's ranking over finite iterations under idealized population optimization, but does not explain why it achieves higher AUC than DT in the experiments. The analysis relies on sufficient function expressiveness and exact population optimization, which differ substantially from practical neural network training with finite samples.

  3. There is a gap between the theoretical analysis and the construction of the SAR experiments. The theoretical results rely on specific relationships between the labeling propensity and the true class posterior, whereas the experiments construct SAR sampling mechanisms using a proxy score. In particular, SAR-C ensures empirical orthogonality between the labeling propensity and the proxy score, but does not strictly guarantee orthogonality with the true posterior. The current experiments therefore do not adequately validate the corresponding theoretical claims.

  4. The full-U protocol used in the experiments may affect the gains of SAEM. Since the unlabeled set contains the entire source dataset, every instance selected for the labeled positive set also appears in the unlabeled set as the same input. During soft target refinement, supervision received by the model from the corresponding labeled positive may influence the target update for that instance, thereby affecting the actual benefits of the proposed mechanism. The paper does not evaluate whether these gains persist when labeled positives and unlabeled instances are disjoint under a one-sample setting or independently sampled under a two-sample setting. The generality of the reported improvements across different PU sampling protocols therefore remains unclear.

Questions for authors

Please refer to the Weaknesses.

Reviewer_rDfoOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PURIST combines balanced positive–unlabeled BCE training with per-example EMA soft targets for unlabeled data, while keeping trusted positive targets fixed. It uses one model without a supplied class prior or augmentation. The paper characterizes population rankings under SCAR and SAR and proves finite-time ranking preservation for idealized target updates. The single-model design requires no supplied class prior and maintains one persistent target per unlabeled example. The 21-task evaluation spans six source-data types and reports unfavorable results. However, the presentation of this paper needs substantial improvement.

Critical strengths

1. A practical baseline with broad and transparent evaluation. The training recipe is simple and potentially useful in settings without a supplied class prior.

2. Sound analysis with explicit boundaries. The fixed-ratio ranking argument, SAR regret bound, non-identifiability example, and finite-time SAEM recurrence are sound under their stated assumptions. The paper makes useful distinctions: idealized population SAEM preserves the initial order at finite iterations and compresses scores in the limit; under general SAR, PU observations alone do not generally identify the true-class ranking.

Critical weaknesses

1. The closest target-update controls are missing. SatPU (2025) and NcPU (2026), two closely related PU target-refinement methods discussed in Section 2, are absent from the comparison inventory (Appendix G). The simplest controls, fixed-coefficient U-only EMA and latest-prediction targets on the same infrastructure, are also missing. Generic SAT differs in optimizer, warm-up, weighting, and the set of updated targets; Mean Teacher averages parameters rather than outputs.

2. The presentation needs substantial improvement. The main issue is organization and explanation rather than pervasive grammatical errors. In the abstract, PURIST is named before the method is clearly defined, and abstract references to design principles and structural properties precede a direct explanation of the algorithm. The introduction does state the two properties, but does not adequately bridge ranking invariance to the benefit of balanced sampling, or one-sided noise to the specific choice of EMA targets. Section 3.3 repeatedly restates U-only refinement across its motivation, design, temporal-refinement paragraph, and closing description. Meanwhile, the essential relationship between sampling rounds, epochs, and target refreshes is deferred to Appendix F. The transition in Section 4.2 from asymptotic score collapse to practical checkpoint selection also needs explanation: finite-time ranking remains unchanged in the stated idealized model. Finally, Sections 5–6 emphasize favorable aggregate summaries without adequately interpreting important failures. These issues obscure both the rationale and the scope of the contribution.

2. The design-principle explanation is not fully isolated.

  • Each balanced epoch contains ⌈|U|/n_neg⌉ rounds that reuse all of P. When the batch floor is inactive, this presents approximately 2|U| examples, compared with |P| + |U| for one natural-proportion pass (Appendix F). Target refreshes and early-stopping patience are counted in epochs. The ablation therefore does not isolate relative P/U weighting from optimization exposure between refreshes; the actual update counts should be reported.
  • Section 4.1 shows that changing a fixed P:U ratio changes the population-optimal output link while preserving ranking. Squared target error can therefore change substantially through the output scale even when ranking is unchanged. The early error reductions of 50.8% versus 7.3% (Table 25) do not, by themselves, establish better early ranking. Target AUC on U excluding selected-P copies, together with a mean-target-matched constant baseline, would help distinguish discrimination from scale effects.
  • When B_floor is active, the meaning of pos_weight = n_neg/n_P needs clarification. If it multiplies all positive BCE terms, it also reweights the positive part of soft U targets and no longer implements the group-mean objective in Eq. (2). Restricting the weight to observed P samples would avoid this mismatch; the actual implementation is not established from the manuscript alone.
Questions for authors
  1. How does SAEM compare with fixed-coefficient U-only EMA and latest-prediction targets on the same infrastructure, and what distinguishes it from the target-update mechanisms of SatPU and NcPU? A small representative subset would suffice for the controlled comparison.
  2. Can you report target AUC at each refresh on U excluding selected-P copies, alongside squared target error and a mean-target-matched constant baseline?
  3. When B_floor is active, is pos_weight restricted to observed P samples or applied to all positive BCE terms, including those from soft U targets? Please give the implemented loss and identify the affected configurations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.