acceptodds
Under review as a conference paper at ICLR 2027

Prior-Enlightened Positive-Unlabeled Learning

Abstract

Positive-Unlabeled (PU) learning trains models from limited labeled positive samples and unlabeled data with mixed positive and negative instances. A core challenge resides in extracting credible negative candidates from unlabeled data, a procedure that critically determines the ranking supervision quality of pairwise PU learning. Current sampling paradigms rely solely on intrinsic data statistics and real-time model outputs, while substantially neglecting informative external prior cues. In practical scenarios, such prior knowledge provides complementary evidence for identifying credible negative samples. This constitutes a critical research gap: whether and how external priors can be properly integrated to alleviate the intrinsic limitations of purely model-based sampling in PU learning? To tackle this issue, this work presents a Prior-Enlightened Positive-Unlabeled Learning (PEPU) framework for pairwise PU learning. PEPU integrates the external prior credibility of unlabeled negative candidates with model-derived pairwise utility to achieve adaptive sampling, and formulates a principled optimization objective that balances pairwise utility and prior knowledge. This work theoretically analyzes the trade-off between prior guidance and model optimization, and further investigates its effects on hidden positive contamination and downstream ranking performance. Extensive experiments on controlled PU studies and two real-data application domains validate that PEPU consistently outperforms state-of-the-art sampling strategies.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
PU learning, Negative Sampling, Pairwise Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ikD5OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem of selecting negative samples from unlabeled data in pairwise PU learning by proposing the PEPU sampling framework. PEPU introduces an external model as a prior, which evaluates the prior credibility of candidate samples as negative samples. The pairwise utility is measured by the BPR loss of the current PU model. Multiplying these two values yields the sampling distribution, thus selecting negative samples that are both credible and valuable during training. Experiments were conducted on two synthetic PU settings, CIFAR-PU, and three recommendation datasets, with PEPU achieving the best results among all compared methods.

Critical strengths
  1. The motivation behind the problem is clear. The difficult samples in U could be true negative samples or hidden positives. Using external priors to assess credibility while preserving the model's difficulty is a natural and practical approach. The entire paper revolves around "difficulty" and "credibility," with clear logic.
  2. The analytical experiments support the method. OVL auditing, prior degradation, and frontier experiments explain the inversion of Prior-only and Hard-only, and show that PEPU smoothly degenerates to Hard-only when the prior fails.
Critical weaknesses
  1. The innovation is limited. This paper combines "selecting difficult negative samples" (e.g., Robinson et al. 2021) with "using external information to determine the reliability of negative samples" (e.g., KGPolicy), but both approaches have already been studied. The authors need to explain why this combination is better than other combinations. The theoretical part also does not fully explain why this method allows the model to learn better.
  2. The experiments are insufficient to support the claim of a unified framework. The paper claims that PEPU is a general framework that can use various external priors, but each setting uses only one fixed prior and construction method, and only one training method, BPR. It lacks ablation studies of prior sources, construction methods, and downstream PU methods. Furthermore, the PU experiments mainly rely on data created by the authors, with improvements relative to the strongest ablation only ranging from 0.001 to 0.01 with no standard deviations reported.
  3. No comparison with the most relevant methods. The paper mentions several methods most similar to PEPU in concept, but none of them were compared in the experiments: methods that use external information to pick negative samples (such as KGPolicy), methods for sampling hard negative samples (such as DENS), and two-step methods for picking reliable negative samples within the PU (such as Spy). These are the most suitable competitors for evaluating PEPU. Additionally, if feasible, it is recommended to also compare it with PU methods such as nnPU. If comparisons are not possible due to differences in class priors or labeling settings, please explain the reasons.
Questions for authors
  1. When there is systematic prior bias, will PEPU be worse than Hard-only? How can harmful priors be identified in practice?
  2. According to Appendix Table 4, the contamination (hidden positive) rate of PEPU is higher than that of Prior-only (0.18 vs 0.10). So where does the benefit come from? Please compare the contamination rate and difficulty (BPR loss) of sampled negatives from prior-only, hard-only, and PEPU.
  3. On CIFAR-PU, the OVL of the DINOv2 prior is only 0.02 (Fig. 1e), indicating that it can already distinguish hidden positive samples and true negative samples very well. Under the same PU data partitioning, if DINOv2 is frozen and only one classification head is trained, what is the test AUC? This comparison helps to determine what additional benefits PEPU can bring compared to the simple approach of directly utilizing external model features.

I would be willing to raise my score if the suggested experiments are conducted.

2 comments

We would like to thank the reviewer for the thoughtful and constructive feedback. In the following, we address the questions and concerns raised in the review.

  1. Methodological contribution and related comparisons (Weaknesses 1 and 3): The cited methods address different aspects of negative selection. Robinson et al. (2021) use similarity to identify hard negatives; KGPolicy learns a knowledge-graph-guided sampling policy; DENS uses disentangled item factors to guide recommendation sampling; Spy identifies reliable negatives through two-stage PU selection; and nnPU addresses pointwise PU risk estimation. Although related in motivation, these methods differ from PEPU in their information sources, sampling mechanisms, or learning objectives. PEPU investigates how independently derived external credibility information can guide negative sampling while retaining model-dependent pairwise utility. Specifically, external information defines a fixed credibility reference, while the current BPR loss determines sampling utility. These signals are combined through an explicit reference-deviation constraint, rather than simply combining two sampling heuristics. Our comparisons include RNS, DNS, AOBPR, AHNS, and SRNS, together with Prior-only and Hard-only, under matched BPR protocols. The theoretical contribution also goes beyond exponential reweighting itself. Proposition 3.4 derives the optimal sampler, Theorem 3.5 establishes the complete utility–reference-deviation Pareto frontier, and Theorem 3.6 establishes optimization consistency with the ordinary BPR gradient.

  2. Unified framework and experimental scope (Weakness 2 and Question 3): PEPU's unification lies in its common reference-based sampling formulation, instantiated with task-specific external information: an auxiliary view in controlled PU learning, frozen DINOv2 representations in image classification, and LLM-derived item representations in recommendation. BPR is the intended scope of this work. Pointwise PU methods such as nnPU and contrastive objectives such as InfoNCE address different learning objectives. Our evaluation covers controlled PU benchmarks constructed by hiding positive labels, real CIFAR-10 images under a brightness-dependent SAR mechanism, and real-world implicit-feedback datasets (Amazon-book, Yelp, and Steam). PEPU achieves the best AUC across all six settings, improving over the strongest ablation by 0.0013–0.0106. Appendix Table 4 reports three-seed results for the controlled PU budget sweep; at the default budget, the standard deviations are 0.0002 and 0.0014 in the two settings. DINOv2 is used exclusively to construct the external prior, while a separately initialized ResNet-18 learns from the original images using BPR, without shared parameters or gradients. A frozen-DINOv2 classifier evaluates direct feature transfer rather than prior-guided sampling. Our matched Prior-only and PEPU comparison uses the same DINOv2 prior and downstream model, isolating the contribution of model-dependent utility. Moreover, OVL = 0.02 measures prior discrimination, not the test AUC of a separately trained classification head. We do not claim superiority over direct DINOv2 feature transfer.

  3. Robustness to systematically misleading priors (Question 1): Section 4.2 shows that PEPU approaches Hard-only as the prior loses its discriminative information through noise injection. A systematically reversed prior can misguide the reference and potentially make PEPU perform worse than Hard-only. In practice, prior reliability can be assessed using independent domain knowledge or a trusted validation set when available. Our framework assumes meaningful relative credibility guidance and does not claim automatic correction of arbitrary prior bias.

  4. Sampling quality and utility–contamination trade-off (Question 2): Table 7 reports expected BPR utility and hidden-positive contamination along the Prior-only-to-PEPU sampling path. At the default budget, utility increases from 0.113 to 0.234, while contamination increases from 0.098 to 0.182. Table 4 reports corresponding training results, with AUC improving from 0.8765 to 0.8840 and contamination increasing from 0.098 to 0.179. Table 1 further compares downstream AUC across Prior-only, Hard-only, and PEPU, although it does not provide a matched three-way contamination and utility comparison. These results illustrate the central mechanism of PEPU: model-dependent utility favors more informative pairs, while the deviation budget controls departure from the credibility reference. The objective is to balance negative credibility and pairwise utility, rather than independently minimize contamination.

We believe these clarifications help distinguish PEPU from related sampling approaches and clarify the scope of the proposed framework and its empirical validation.

I thank the authors for their thoughtful responses, which have partially addressed my concerns. However, the experimental results I previously requested, particularly comparisons with relevant methods and the frozen-DINOv2 baseline, have not yet been provided. Therefore, I will maintain my original score.

Reviewer_UfemOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors of this paper focus on the trade-off between negative credibility and sampling informativeness in pairwise Positive-Unlabeled (PU) learning. They propose PEPU, a Prior-Enlightened Positive-Unlabeled Learning framework for pairwise Positive-Unlabeled learning. PEPU integrates external prior knowledge and model-dependent pairwise utility through KL-regularized adaptive sampling, supported by theoretical analyses of sampling optimality and contamination. Experiments on controlled PU benchmarks, CIFAR-PU, and recommendation datasets demonstrate consistent improvements over existing sampling methods.

Critical strengths
  1. The paper identifies a meaningful trade-off between negative credibility and pairwise utility, providing a clear motivation for the proposed framework.
  2. The proposed sampling strategy offers a unified formulation for integrating external priors and model-dependent hardness, with theoretical characterization of its optimization properties.
  3. The experiments cover diverse settings, and the analyses of prior quality and sampling behavior provide useful empirical insights.
Critical weaknesses
  1. Heavy dependence on external prior quality. The effectiveness of PEPU relies on informative external priors, but their reliability is largely assumed rather than established by the method. For example, the CIFAR-PU prior already exhibits strong discrimination (OVL = 0.02), and Prior-only achieves performance close to PEPU. This raises questions about how much improvement originates from the proposed sampling mechanism.
  2. Limited connection between theoretical results and learning guarantees. The theoretical analyses mainly characterize the optimality of KL-regularized sampling and bound contamination relative to the reference distribution. However, they do not establish that PEPU reduces false-negative contamination or consistently improves the true PU ranking objective. In fact, the default sampler exhibits higher contamination than Prior-only.
  3. Insufficient methodological differentiation. The proposed exponential reweighting follows a standard Gibbs formulation. Its distinction from simpler approaches combining prior credibility and model hardness remains unclear, particularly without direct comparisons under identical prior information.
  4. Potential mismatch in the AUC analysis. The theoretical AUC characterization uses the positive distribution underlying the sampled anchors, whereas observed positives may be selection-biased under SAR. The connection between this theoretical objective and population ROC-AUC requires clarification.
Questions for authors
  1. How much of PEPU's improvement can be attributed to the proposed sampling mechanism rather than the strong external priors? Could the authors compare against simpler prior-weighted hardness sampling strategies?
  2. The contamination bound only controls deviation from the reference distribution, while PEPU increases contamination relative to Prior-only. How do the theoretical results substantiate the claimed improvements in negative supervision quality?
  3. Does the AUC guarantee in Corollary 3.9 characterize ranking against the full positive distribution or only the observed-positive distribution under SAR labeling?
  4. How robust is PEPU to systematically misleading or incorrectly ranked prior information, beyond the random prior degradation examined in Section 4.2?
2 comments

We thank the reviewer for reviewing our work and for the detailed feedback. We address the related concerns and questions below.

  1. Prior quality and contribution of the sampling mechanism (Weakness 1 and Question 1): PEPU exploits independently derived external credibility information. Definition 3.1 requires meaningful relative credibility guidance, rather than calibrated negative probabilities or an oracle-quality prior. Section 4.2 examines prior quality through OVL: 0.28 for the default controlled prior, 0.02 for CIFAR-PU, and 0.65–0.81 for recommendation. We further degrade the controlled prior through noise injection, increasing OVL from 0.22 to 0.98. PEPU gradually approaches Hard-only as the prior loses its discriminative information. The matched Prior-only and PEPU comparisons isolate the contribution of model-dependent utility under the same prior. On CIFAR-PU, PEPU improves AUC from 0.9648 to 0.9705. In recommendation, Hard-only outperforms Prior-only on all three datasets, yet PEPU consistently achieves the best results. These findings demonstrate the complementary contributions of external credibility and model-dependent utility. PEPU's contribution is not merely combining two scores, but integrating them through an explicit reference-deviation constraint.

  2. Theoretical guarantees and contamination (Weakness 2 and Question 2): Proposition 3.4 establishes the optimal sampler, Theorem 3.5 establishes the complete utility–reference-deviation Pareto frontier, Theorem 3.6 establishes optimization consistency, and Theorem 3.7 bounds hidden-positive contamination deviation from the reference. PEPU balances pairwise utility against reference deviation rather than minimizing contamination relative to Prior-only. At the default budget, Table 4 shows that AUC improves from 0.8765 to 0.8840, while contamination increases from 0.098 to 0.179. Table 7 further shows that expected BPR utility increases from 0.113 to 0.234, with contamination increasing from 0.098 to 0.182. Theorem 3.7 controls the magnitude of contamination deviation, not its direction. The observed increase is consistent with the positive covariance between hidden-positive status and BPR loss analyzed in Appendix B.8. Thus, the theoretical and empirical results characterize the intended utility–credibility trade-off, rather than guaranteeing lower contamination than Prior-only.

  3. Methodological differentiation (Weakness 3): Exponential reweighting is an established mathematical tool. PEPU's contribution lies in formulating a specific utility–reference-deviation optimization problem for BPR-based PU negative sampling: max⁡QHθ(Q)−τDKL(JQ∥JS).\max_Q H_\theta(Q)-\tau D_{\mathrm{KL}}(J_Q\|J_S). Here, JSJ_S is constructed from independently derived external credibility information, while the current BPR loss provides pairwise utility. Proposition 3.4 derives the optimal sampler, and Theorem 3.5 establishes the complete Pareto frontier. Thus, the temperature characterizes the marginal utility gained per unit of reference deviation, providing a principled alternative to arbitrary score fusion. Appendix B.8 further verifies the frontier using shuffled- and noisy-utility controls.

  4. AUC characterization under SAR (Weakness 4 and Question 3): In our algorithm, the shared positive marginal P+P_+ of JQJ_Q, JSJ_S, and J∗J^* corresponds to observed-positive anchors. Under SAR, this distribution may be selection-biased. Accordingly, Corollary 3.9 characterizes true-negative ranking under the observed-positive anchor distribution, not an unconditional AUC guarantee over the full latent-positive population. The result is additionally conditional on Reference Coverage.

  5. Robustness to systematically misleading priors (Question 4): Section 4.2 examines progressively less informative priors through independent noise injection. Systematic rank reversal is different: a misleading prior can bias the reference distribution and potentially make PEPU perform worse than Hard-only. In practice, prior reliability can be assessed using independent domain knowledge or a trusted validation set when available. Our framework assumes meaningful relative credibility guidance and does not claim automatic correction of arbitrary prior bias.

We hope these explanations clarify the theoretical guarantees of PEPU, their assumptions, and the intended interpretation of its empirical results.

Several of the concerns relate to how external prior information is defined and incorporated in PEPU, how its contribution is distinguished from model-dependent hardness, and how broadly the resulting framework has been examined. We would therefore like to further clarify the design principle underlying PEPU and the scope of our investigation.

  1. Prior formulation and methodological contribution: A key methodological contribution of PEPU is the systematic formulation of external prior credibility for pairwise PU negative sampling. Prior credibility represents external evidence about whether an unlabeled candidate is suitable as a relative negative for a given positive anchor. We encode this information in a fixed reference distribution, while the current model's BPR loss provides an evolving measure of pairwise utility. PEPU integrates these two sources through an explicit utility–reference-deviation optimization objective, in which the prior establishes the credibility reference and model-dependent utility guides the adaptation of negative sampling. This formulation is instantiated with an auxiliary view in controlled PU learning, frozen DINOv2 representations in CIFAR-PU, and LLM-derived item representations in recommendation. Although the prior sources and their construction differ across tasks, they share the same underlying reference-based formulation.

  2. Theoretical and empirical investigation: We have examined this formulation through complementary theoretical and empirical analyses. The theoretical results characterize the optimal sampler and the complete utility–reference-deviation Pareto frontier, establish optimization consistency, and analyze hidden-positive contamination and its conditional connection to true-negative ranking. Our experiments cover two controlled PU settings, CIFAR-PU under instance-dependent SAR, and three real-world recommendation datasets. We further investigate prior informativeness and its degradation, compare PEPU with established negative-sampling methods, and use Prior-only and Hard-only ablations to examine the respective contributions of prior credibility and model-dependent utility. Section 4.3 examines the empirical utility–reference-deviation frontier and contamination behavior, while Appendix B.8 provides the corresponding numerical analysis, including comparisons with shuffled- and noisy-utility controls.

  3. Ongoing extensions: The present work focuses on pairwise PU learning with BPR, providing a common downstream objective for investigating the proposed sampling mechanism. Building on this formulation, we are exploring its extensions to more diverse prior settings and broader learning objectives.

Reviewer_TBp7OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper attempts to address a core challenge in positive-unlabelled learning – the selection of high-quality negative samples from the mixed unlabelled set. To this goal, the authors use external knowledge to judge the ‘negativeness’ of a sample and introduces Prior-Enlightened Positive-Unlabeled Learning (PEPU) framework for pairwise PU learning. The authors evaluate the performance of the method on two datasets and show marginal improvement over some existing methods.

Critical strengths

None.

Critical weaknesses

The training epoch is chosen by ROC-AUC on a development uses true hidden labels, and the deviation budget was appeared to have set in the the same way. I think this is major flaw of the paper – in PU you never have access to such data even during training.

The choice of the datasets also questionable. There is no real PU data to actually evaluate the PU learning efficacy.

The third major drawback is choice of the baselines. There are several strong existing PU learning baselines available – nnPU, uPu etc. It appears to me that only the sampling efficacy is shown. But, it is much more important to show how this sample technique is actually helping us to select good-quality negatives. The paper is completely missing out this.

The model also uses strong external cues such as DINOv2 features, while the other models are not using any such thing. This weakens the claim of the paper and this also limits the novelty of the paper.

Additionally, the improvement of performance is marginal showing very little real efficacy.

Questions for authors

Please see my comments in weakness section.

2 comments

We thank the reviewer for the comments and address the concerns regarding our experimental design, methodological scope, and contributions below.

  1. Development labels and model selection in PU learning: The relevant distinction is between supervision used for model training and ground-truth labels used for benchmark-level model selection. As documented in Appendix B.5, our experiments follow a widely adopted practice in empirical PU learning. Models are trained exclusively with observed positives and unlabeled data, while ground-truth labels in a separate development split are used for checkpoint and hyperparameter selection. Specifically, the training epoch is selected by development ROC-AUC, and the deviation budget δtr=0.1\delta_{\mathrm{tr}}=0.1 is selected once and fixed across the controlled PU experiments. These labels never enter prior construction, negative sampling, or gradient updates. Wang et al. (2026, ICLR) also discuss this common practice and propose PU-only proxy-AUC selection. However, their theoretical guarantee does not directly extend to our instance-dependent SAR setting, as discussed in Appendix B.5. Our protocol therefore follows established PU benchmark practices, without introducing hidden-label supervision into the proposed learning mechanism.

  2. Choice of datasets: Constructing PU benchmarks from fully labeled datasets by hiding a subset of positive labels is a widely adopted experimental practice in PU learning. It preserves the underlying ground truth, enabling controlled evaluation of hidden-positive contamination and learning performance. Our experiments follow this practice. The controlled PU settings allow systematic analysis under known latent labels, while CIFAR-PU uses real CIFAR-10 images with a brightness-dependent SAR mechanism and 37.5% hidden positives in U. We further evaluate PEPU on real-world implicit-feedback datasets, including Amazon-book, Yelp, and Steam. Together, these settings cover controlled PU analysis, image-based PU learning, and real-world recommendation.

  3. PU baselines and negative-sampling quality: Methods such as uPU and nnPU primarily address pointwise PU classification through risk estimation from positive and unlabeled distributions, with nnPU introducing a non-negative risk correction. PEPU instead focuses on selecting unlabeled candidates for pairwise BPR updates. They therefore address different learning objectives and are not directly interchangeable baselines for evaluating negative-sampling strategies. Accordingly, we compare PEPU with RNS, DNS, AOBPR, AHNS, and SRNS, as well as Prior-only and Hard-only, while keeping the downstream BPR protocol fixed. Moreover, sampled-negative quality is directly examined in our paper. Figure 1 evaluates prior credibility through OVL, Section 4.2 studies prior degradation, and Appendix B.6 / Figure 7 measures hidden-positive contamination across deviation budgets. At the default budget, PEPU has a contamination rate of approximately 0.18, compared with 0.10 for Prior-only. This reflects the intended utility–credibility trade-off: PEPU does not simply minimize contamination, but balances it against pairwise utility. Theorem 3.7 further provides a theoretical bound on contamination deviation from the reference distribution.

  4. Use of external DINOv2 features: DINOv2 representations serve exclusively to construct the external credibility prior. They are neither used as downstream classifier inputs nor as initialization. A separately initialized ResNet-18 is trained on the original images using BPR, without shared parameters or gradients with DINOv2. Similarly, LLM-derived representations construct the recommendation prior, while LightGCN learns from user-item interactions using ID embeddings. This separation is intentional and central to PEPU. Our contribution lies in formulating independently derived external information as a fixed credibility reference and integrating it with model-dependent pairwise utility. The Prior-only and Hard-only ablations further isolate the contributions of these two information sources.

  5. Marginal performance improvements: Although the absolute gains are moderate in some settings, PEPU achieves the best results across all six PU metrics in Table 1, all three recommendation AUC results, and all twelve Recall and NDCG metrics in Table 2. Importantly, Prior-only and Hard-only exhibit a task-dependent performance reversal, while PEPU consistently outperforms both. This supports our central motivation that external credibility and model-dependent utility provide complementary information for negative sampling.

We thank the reviewer again for the comments and hope these responses clarify the experimental protocol, methodological distinctions, and contributions of PEPU.

We would like to further clarify the experimental protocol, particularly the distinction between benchmark-level model selection and PU model training.

Our experiments follow an established practice in empirical PU learning: ground-truth labels are retained in a separate development set for benchmark-level model selection, while model training uses only observed positives and unlabeled data. In particular, true hidden labels never enter the training procedure, prior construction, negative sampling, or gradient updates.

As detailed in Appendix B.5, our protocol consists of three steps:

  • Training: All models are trained exclusively using observed positives and unlabeled candidates.
  • Model selection: Development-set ROC-AUC is used for checkpoint selection and standard grid-search-based hyperparameter tuning. The selected hyperparameters are then fixed for the subsequent experiments.
  • Evaluation: All compared methods share the same initialization and model-selection protocol, and the selected models are evaluated on a separate test set.

The key distinction is that using ground-truth labels to select among trained checkpoints and hyperparameter configurations does not mean using those labels to train the PU model. This is a benchmark-level evaluation practice, not hidden-label supervision of the proposed learning mechanism. Wang et al. (2026, ICLR) also discuss the use of labeled validation data for model selection in empirical PU learning and propose PU-only alternatives. Their work further highlights the distinction between conventional benchmark evaluation and strictly PU-only model selection. In our instance-dependent SAR setting, the theoretical guarantees of their proposed proxy-AUC criterion do not directly apply, as discussed in Appendix B.5.

Thus, true hidden labels are used to evaluate candidate checkpoints and hyperparameter configurations, but not to provide supervision for learning the model parameters or selecting negative samples. This is the distinction between benchmark-level model selection and PU model training underlying our experimental protocol. We hope this clarifies how ground-truth labels are used in our experiments.

Comments on the paper

1 comment
Area_Chair_QkseOpenReview ↗

Reminder: Please Review the Author Responses

Dear Reviewer TBp7 and Ufem,

As the discussion period is coming to an end, please take a moment to review the authors' responses and update your ratings if your assessment has changed. If you have any remaining concerns, please feel free to share them in the discussion.

Thank you for your efforts!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.