acceptodds
Under review as a conference paper at ICLR 2027

Mole-PAIR: Improving Reliability of Molecular Foundation Models with Preference Optimization

Abstract

Molecular foundation models pre-trained on large chemical corpora have become powerful tools for scientific discovery, yet their behavior can be unpredictable when inputs deviate from the pre-training distribution, which severely hinders deployment in safety-critical settings such as biomedical discovery and materials design. To mitigate this issue, we propose Molecular Preference-Aligned Instance Ranking (Mole-PAIR), a plug-and-play module that can be attached to existing foundation models and improve their reliability w.r.t. out-of-distribution (OOD) data through cost-effective post-hoc preference optimization on frozen representations. The key idea of Mole-PAIR is to formulate OOD detection as a preference optimization problem whose objective aims at optimizing the model's “preferences” on in-distribution (ID) samples over OOD ones. Our analysis shows that this new objective adaptively prioritizes hard pairs throughout the training dynamics, which facilitates fast learning in practice. We demonstrate the practical efficacy of this approach with recently proposed molecular foundation models and challenging OOD benchmarks, achieving up to 9.7% improvement of AUROC and 48.5% reduction of FPR95 over powerful OOD detection approaches under shifts of scaffold, assay and molecular size.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Foundation model, Preference Optimization, Out-of-distribution Detection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_z1tfOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Mole-PAIR, a simple framework to solve the OOD test issue and improve the reliability of the molecular foundation models by OOD detection. It proposes using preference optimization, in other words, a pairwise loss, to determine whether the evaluation data is OOD or ID. This method improves AUROC and FPR95 under some OOD-setting datasets such as DrugOOD.

Critical strengths
  1. The topic of OOD detection for molecular foundation models is critical and useful.
  2. The proposed method is simple and efficient. The framework only trains a lightweight MLP on frozen molecular representations, making it easy to apply to existing foundation models.
  3. Mole-PAIR achieves better OOD detection performance than most baselines across DrugOOD and GOOD, including methods using the same auxiliary OOD data.
Critical weaknesses
  1. The contribution of this work is trivial. The main technique is a standard pairwise ranking approach applied to OOD detection, using a frozen molecular encoder and a lightweight MLP. The proposed loss is essentially a pairwise logistic ranking loss, and the hard-pair emphasis discussed in the theoretical analysis is also a natural property of this loss. The pairwise loss has already been commonly used in current molecular and drug discovery research. The overall workflow is also simple. Some other experiments in the main paper are parameter tuning. Therefore, it is not very convincing regarding the main methodological novelty and the contribution of this work.

  2. The paper claims that the method does not require prior knowledge of the test-time OOD distributions. However, the main experiments use auxiliary OOD samples from the corresponding OOD-validation split for each distribution shift. Although the validation and test domains are different, the auxiliary OOD data are still selected based on the same shift type as the test data. It largely diminishes the usage of this work. Although Table 9 provides some cross-shift evaluations, all transferred models are tested on Size shifts, which are easy to distinguish. It is still unclear whether the method can generalize to more challenging unseen OOD types, such as Scaffold or Assay shifts, without using shift-specific auxiliary OOD samples.

  3. Since the method already uses ID and auxiliary OOD samples for training, a simple, straightforward binary classifier would be a natural baseline. Although Appendix D6 compares pairwise ranking with BCE and MSE, it only provides a score distribution visualization on one dataset, without quantitative results. Also, Appendix D6 shows a pairwise hinge loss, different from the pairwise logistic loss in Eq.6. A more comprehensive comparison is needed to demonstrate the advantage of the proposed preference optimization.

  4. Some results appear inconsistent. For example, MiniMol on EC50-Assay reports an AUROC of 0.743 in Table 4 but 0.711 in Table 7. Similarly, Uni-Mol on EC50-Scaffold reports 0.976 in Table 4 but 0.965 in Table 12.

Questions for authors

If pairwise loss is the key contribution which is better than the simple binary classification, it is necessary to use comprehensive experiments or analysis to prove it.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Reviewer z1tf (1)

Thank you for recognizing the importance of molecular OOD detection, the efficiency of our framework, and its performance against exposure-based baselines. We have added the requested binary-classification controls, challenging unseen-shift experiments, and a correction of the inconsistent results.

W1: "The contribution of this work is trivial. The main technique is a standard pairwise ranking approach applied to OOD detection ... The pairwise loss has already been commonly used in current molecular and drug discovery research ... Some other experiments in the main paper are parameter tuning."

We thank the reviewer for the review. We respectfully disagree with this interpretation of our contributions. While pairwise ranking is an established technique, our contribution is not merely applying an existing loss or tuning its parameters. We address a distinct problem in molecular learning: equipping pretrained molecular foundation models with the ability to identify unfamiliar molecular inputs without downstream property labels or backbone retraining. We highlight three aspects:

  1. A new problem formulation for molecular foundation models. Existing molecular ranking approaches primarily learn preferences over molecular properties, compound activities, or medicinal chemistry choices [1]. In contrast, Mole-PAIR formulates OOD detection as learning the relative preference between ID and auxiliary background molecules. This provides a dedicated OOD affinity score independent of downstream property prediction. (Section 3.1)

  2. A lightweight and task-agnostic detection framework. Rather than deriving OOD scores from a supervised property classifier, Mole-PAIR directly learns a scoring head on frozen molecular representations. This design requires neither molecular property labels nor classifier logits and applies to both 2D and 3D foundation models. Our controlled experiments further demonstrate its practical effectiveness over OE classifier baselines under matched auxiliary-data budgets, particularly on challenging Assay shifts. (Table 1, Requirements columns; Table 4, best AUROC in 11 of 12 cases per backbone; Appendix D.6)

  3. Theoretical characterization and empirical significance. We establish the connection between the pairwise objective and Bayes-optimal AUROC ranking under the stated population assumptions, and characterize its adaptive emphasis on difficult pairs during optimization. While the underlying ranking principle is established, these analyses provide a principled understanding of the objective in our molecular OOD setting. Moreover, our extensive experiments validate its effectiveness across multiple molecular foundation models, datasets, and distribution shifts. (Proposition 5.1, Lemma 5.2, Proposition 5.3, Section 5)

Importantly, methodological significance should not be determined solely by whether an underlying optimization primitive is new. Our contribution lies in identifying and validating a practical, theoretically motivated approach to molecular foundation-model reliability, rather than proposing a new generic ranking loss. The hyperparameter analyses are supplementary evaluations, not the central contribution. Section 4.2 is now renamed "Sensitivity Analysis", with the two scans combined into Figure 2 and the joint grid moved to Appendix D.2 (Figure 5).

References:

[1] Extracting Medicinal Chemistry Intuition via Preference Machine Learning, Nature Communications 2023, https://doi.org/10.1038/s41467-023-42242-1

W4: "Some results appear inconsistent. For example, MiniMol on EC50-Assay reports an AUROC of 0.743 in Table 4 but 0.711 in Table 7. Similarly, Uni-Mol on EC50-Scaffold reports 0.976 in Table 4 but 0.965 in Table 12."

Thank you for carefully checking our results and identifying these inconsistencies. We have reconciled the affected results, and aligned the related tables and summary averages accordingly.

AuthorsOpenReview ↗

Response to Reviewer Reviewer z1tf (2)

W3: "Since the method already uses ID and auxiliary OOD samples for training, a simple, straightforward binary classifier would be a natural baseline. Although Appendix D6 compares pairwise ranking with BCE and MSE, it only provides a score distribution visualization on one dataset, without quantitative results. Also, Appendix D6 shows a pairwise hinge loss, different from the pairwise logistic loss in Eq.6. A more comprehensive comparison is needed to demonstrate the advantage of the proposed preference optimization."

Q1: "If pairwise loss is the key contribution which is better than the simple binary classification, it is necessary to use comprehensive experiments or analysis to prove it."

Thanks for suggesting this comparison, which improves our evaluation. We add a binary cross-entropy (BCE) detector that uses the same frozen embeddings and the same scoring head as Mole-PAIR. ID molecules have target 0 and auxiliary OOD molecules have target 1.

Comparison under different auxiliary-data budgets. We evaluate on the challenging DrugOOD EC50-Assay shift. All methods use the same nested auxiliary-molecule lists, the same frozen MiniMol embeddings, the same validation and test sets, and a common full-batch training protocol. Results are AUROC (mean ± SD over five seeds), reported in Appendix D.6:

Auxiliary OOD moleculesMole-PAIRBCEBest-OE Baseline
2000.683 ± 0.0020.634 ± 0.0050.596 ± 0.013
400.625 ± 0.0050.527 ± 0.0120.553 ± 0.011
200.617 ± 0.0070.531 ± 0.0150.562 ± 0.020

Mole-PAIR exceeds BCE by 0.049, 0.098, and 0.086 AUROC with 200, 40, and 20 auxiliary molecules.

Robustness to the choice of auxiliary molecules. We repeat the 20-molecule setting with four independently sampled auxiliary sets and five initialization seeds each. Mole-PAIR obtains 0.580 ± 0.046 AUROC, compared with 0.536 ± 0.042 for BCE and 0.552 ± 0.023 for MSP-OE, the base-OE baseline.

W2: "The paper claims that the method does not require prior knowledge of the test-time OOD distributions. However, the main experiments use auxiliary OOD samples from the corresponding OOD-validation split for each distribution shift. It is still unclear whether the method can generalize to more challenging unseen OOD types, such as Scaffold or Assay shifts, without using shift-specific auxiliary OOD samples."

Thank you for the review. We agree that the original Table 9 was incomplete, We supplement two new experiments.

Generic public background without shift-specific selection. We replace the benchmark OOD-validation pool with molecules sampled from PubChem without using the target shift type. We exclude structures appearing in any of the six DrugOOD subsets. This gives 2,000 training and 600 validation molecules. The ID data, frozen MiniMol encoder, architecture, and test sets are unchanged.

Target shiftOfficial auxiliary poolPubChem backgroundBest ID-only baseline
EC50-Scaffold0.970 ± 0.0000.914 ± 0.0010.685
EC50-Assay0.711 ± 0.0020.602 ± 0.0030.572

We found that a generic public background provides substantial value for the structural shift (0.914 vs. 0.685 for the best ID-only baseline).

Transfer to Scaffold targets. Models are trained and selected using source-domain data only, then evaluated once on the target test set (1,000 ID and 1,000 OOD molecules). The target-trained model is a separate reference with access to target-domain training and validation data. Full results are in Table 10, Appendix D.5.

Source → targetTransferred Mole-PAIRTarget-trained Mole-PAIR
EC50-Size → IC50-Scaffold0.909 ± 0.0010.982 ± 0.001

The direction retains strong detection without target-specific auxiliary data. We thank you again for pointing out this.

Dear Reviewer Z1tf,

Thank you for your review.

We have added all the experiments and discussions that you have asked for in your rebuttal. The discussion phase will end soon and we are hoping to hear your thoughts.

Regards,

Authors

Reviewer_Qw8oOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Building upon formulating OOD detection as a preference optimization problem, this paper proposes a plug-and-play module, Molecular Preference-Aligned Instance Ranking (Mole-PAIR), to improve the reliability of existing foundation models such as MiniMol and Uni-Mol. Ablation experiments are conducted to evaluate the effectiveness of different parameters of Mole-PAIR, and the proposed method is compared with several baseline approaches, achieving up to 9.7% improvement of AUROC and 48.5% reduction of FPR95. Overall, the paper is well designed, and the methodology is generally sound.

Critical strengths
  1. Mole-PAIR is a plug-and-play module that can be seamlessly integrated into existing pre-trained foundation models, enhancing their reliability through lightweight additional training.

  2. Mole-PAIR demonstrates strong practical effectiveness, consistently improving performance when integrated with recently proposed molecular foundation models, including MiniMol and Uni-Mol.

Critical weaknesses

None

Questions for authors
  1. The practical application scenarios of Mole-PAIR are not sufficiently clarified. Could the authors elaborate on the intended real-world use cases and explain how Mole-PAIR would be deployed in practice? This clarification would help readers better understand the practical relevance and applicability of the proposed method.

  2. The definition and construction process of out-of-distribution (OOD) molecules remain ambiguous. Could the authors provide a more precise definition of OOD molecules and explain how they are constructed or selected in the experiments? This would help avoid potential misunderstandings and improve the reproducibility of the study.

  3. The paper primarily reports threshold-independent evaluation metrics, such as AUROC. However, practical applications typically require a decision threshold (τ) to distinguish in-distribution from OOD molecules. How should τ be determined in real-world deployment, and how sensitive is the performance of Mole-PAIR to different threshold choices?

  4. The paper states that the OE variants and Mole-PAIR share the same auxiliary OOD set and exposure budget. Could the authors specify the exact number of auxiliary OOD molecules used and clarify how the exposure budget is defined and controlled across different methods? This information is important for assessing the fairness and reproducibility of the experimental comparisons.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Reviewer Qw8o

We sincerely thank the reviewer for acknowledging the plug-and-play nature of our approach and its practical effectiveness, as well as providing support for our work. Below we answer your questions.

Q1: "Could the authors elaborate on the intended real-world use cases and explain how Mole-PAIR would be deployed in practice?"

Thank you for raising this question. Mole-PAIR is designed as a lightweight, plug-and-play reliability module for molecular foundation models, with potential applications across scientific discovery workflows.

For example, in protein–ligand screening, Mole-PAIR can identify unfamiliar candidate molecules before their predicted binding affinities are used to prioritize experiments. In drug discovery and lead optimization, it can flag novel molecular scaffolds that differ substantially from previously studied compounds, helping researchers identify predictions that warrant additional validation. Similarly, in molecular property prediction, it can provide an additional reliability signal when evaluating compounds outside the chemical space represented by the available training data.

In practice, researchers can attach Mole-PAIR to an existing molecular foundation model without modifying its pretrained parameters. Using a reference collection of molecules and readily available auxiliary molecular structures, a lightweight scoring head is trained to estimate OOD affinity. New molecules with high OOD scores can then be flagged for further examination or experimental validation, providing a practical safeguard for downstream scientific decisions.

Q2: "Could the authors provide a more precise definition of OOD molecules and explain how they are constructed or selected in the experiments?"

Thank you for this suggestion. We adopt DrugOOD [1] and GOOD [2], two established benchmarks for molecular and graph out-of-distribution (OOD) learning. OOD is defined relative to the reference training domains, following the official benchmark partitions rather than an OOD definition introduced by our method.

Specifically, we consider three types of distribution shifts:

  • Assay (DrugOOD): Molecules are grouped by the experimental bioassay; held-out assays constitute OOD domains.
  • Scaffold (DrugOOD and GOOD): Molecules are grouped by molecular scaffold; held-out scaffold domains represent structural shifts.
  • Size (DrugOOD and GOOD): Molecules are grouped by atom count; held-out size domains represent shifts in molecular size.

These definitions and what each shift tests are now stated in Section 4 (Datasets) and Appendix C.1. For our experiments, we follow the official ID/OOD partitions and use a fixed sampling seed to select reproducible subsets. Auxiliary OOD molecules for training and validation are drawn from the benchmark's OOD-validation pool, while the OOD-test partition is reserved for final evaluation. The auxiliary and test sets therefore contain distinct samples. Section 3.1 now states explicitly that no OOD-test molecule or held-out test domain is used for training or model selection.

References:

[1] DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery — A Focus on Affinity Prediction Problems with Noise Annotations, arXiv:2201.09637

[2] GOOD: A Graph Out-of-Distribution Benchmark, NeurIPS 2022 Datasets and Benchmarks Track

Q4: "Could the authors specify the exact number of auxiliary OOD molecules used and clarify how the exposure budget is defined and controlled across different methods?"

Thank you for this suggestion. The auxiliary OOD exposure budget refers to the number of auxiliary OOD molecules available during training. As described in Appendix C.1, we use fixed ID/OOD partitions with sampling seed 42. Appendix C.1 has been revised to separate the full official pools (Tables 2 and 3) from the realized samples actually used. The actual training sample counts are summarized below:

DatasetTraining Auxiliary OOD
DrugOOD (EC50/IC50, each shift)2,000
GOOD-HIV-Scaffold2,878
GOOD-HIV-Size1,941
GOOD-PCBA (each shift)5,000
GOOD-ZINC (each shift)5,000

All methods are under a fair comparison. In our main experiments, Mole-PAIR and the OE baselines (MSP-OE and ODIN-OE) are assigned the same auxiliary OOD data budget. Different methods retain their respective optimization objectives and training procedures.

AuthorsOpenReview ↗

Response to Reviewer Reviewer Qw8o (2)

Q3: "How should τ be determined in real-world deployment, and how sensitive is the performance of Mole-PAIR to different threshold choices?"

Thank you for this practical question. Mole-PAIR allows the decision threshold τ\tau to be selected according to the application's acceptable false-positive rate, without requiring access to test-time OOD samples. We clarify the threshold selection procedure and provide additional experiments below.

Threshold selection. In practice, researchers can reserve a small set of known ID molecules for calibration. Given a target false-positive rate α\alpha, we set τ\tau to the empirical upper (1−α)(1-\alpha) quantile of the ID calibration scores. A test molecule is flagged as OOD when Eϕ(S)≥τE_\phi(S)\geq\tau. This allows practitioners to adjust the detection threshold based on their application requirements without retraining the model.

Empirical sensitivity analysis. We evaluate this procedure using 600 held-out ID molecules for calibration, independently of model training and selection. We consider three false-positive budgets, α∈{1\alpha \in \lbrace 1\\%, 5\\%, 10\\%\rbrace , and report the actual test false-positive rate (FPR) and OOD detection rate (recall).

DatasetTarget FPRActual FPROOD Recall
EC50-Assay1%0.9%4.6%
EC50-Assay5%4.5%17.6%
EC50-Assay10%8.6%28.7%
EC50-Scaffold1%4.0%87.8%
EC50-Scaffold5%9.1%92.2%
EC50-Scaffold10%15.2%94.3%
EC50-Size1%0.7%99.5%
EC50-Size5%2.9%100%
EC50-Size10%9.2%100%

The results demonstrate a controllable trade-off between false alarms and OOD detection. For example, on Scaffold, increasing the target FPR from 1% to 10% improves OOD recall from 87.8% to 94.3%. On Size, Mole-PAIR achieves 99.5% recall even at the strictest threshold. Assay remains more challenging, reflecting the difficulty of identifying subtle distribution shifts.

Again, we sincerely thank you for taking the time to review our paper and provide valuable suggestions to help us make the paper better.

Reviewer_j8ggOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Mole-PAIR is an architecture that can be used on pretrained molecular embeddings from molecular foundation models to judge molecules that were in the training distribution or outside the training distribution. It is trained using preference optimization, which empirically works well here.

Critical strengths

The method outperforms other architectures when it has sufficient OOD training data.

Critical weaknesses

Mole-PAIR needs OOD data to be trained on, which is not always accessible in sufficient quantities. There is little to no discussion of this flaw, which is vital because this method is only useful for ID/OOD qualification if it can be trained properly.

The organization of the paper felt scattered and hard to read at times, and some of the data was confusingly presented or hard to parse. Better use of graphs could help this.

Questions for authors

What is the motivation for choosing preference optimization to improve OOD detection? It is stated later in the paper, but an explanation on that would strengthen the introduction. I have an idea, but it would help to see that explicitly. The point iii of your contributions is good.

More discussion in the 2nd paragraph of 3.1 (lines 149-159) about why it is safe to draw OOD samples from a general distribution would be appreciated. This might be true in many cases of interest, but that’s not clearly stated.

Figure 2 is confusing. What is it adding to the paper, and why is a smooth curve fit (presumably spline interpolation) appropriate?

Section 4.2 isn’t really an ablation study, it’s just two hyperparameter scans. Why isn’t the graph for the lambda scan shown in the same figure (or at least visually nearby) as the beta scan? The conclusions drawn from these two scans are dubious since there is no grid scan of lambda and beta simultaneously. The behavior of the test AUC is clearly very sensitive to these parameters, so assuming that they could be optimized/checked separately is questionable.

Comparing tables 1, 4, and 7 makes it seem like the model performs poorly with scarce OOD data, potentially worse than the competitors. This is not discussed much; there is a line saying it out performs ID-only models, but I don’t think it outperforms the other models also trained on OOD data, though that’s never explicitly stated.

The organization of the appendices was a little scattered, which made it hard to verify data.

A little more discussion of the benchmark datasets would have been appreciated.

4 comments
AuthorsOpenReview ↗

Response to Reviewer j8gg (1)

Thank you for recognizing the effectiveness of Mole-PAIR and for the suggestions on data availability and presentation. Below we answer the weaknesses (W) and questions (Q), merging related points. All changes below are included in the revised PDF.

W1: "Mole-PAIR needs OOD data to be trained on, which is not always accessible in sufficient quantities. There is little to no discussion of this flaw, which is vital because this method is only useful for ID/OOD qualification if it can be trained properly."

Q2: "More discussion in the 2nd paragraph of 3.1 (lines 149-159) about why it is safe to draw OOD samples from a general distribution would be appreciated. This might be true in many cases of interest, but that's not clearly stated."

Thanks for the questions. Mole-PAIR needs neither samples from the test-time OOD distribution nor a large auxiliary set. It uses auxiliary background molecules, and unlabeled chemical structures are freely accessible through public databases such as PubChem, ZINC, and ChEMBL, without any task-specific property labels. We have rewritten Section 3.1 (Problem Formulation) to state this explicitly, and we add two new experiments.

A general public background is sufficient for structural shift. We train Mole-PAIR with 2,000 PubChem molecules, sampled without using the target shift type:

DatasetOfficial auxiliary OODPubChem backgroundBest ID-only baseline
EC50-Scaffold0.9700.9140.685
EC50-Assay0.7110.6020.572

We found that the PubChem background improves over the best ID-only baseline on both shifts, reaching 0.914 AUROC on Scaffold. The gain is largest for structural shift, where a chemically diverse background directly represents "what is not ID."

A small auxiliary set is sufficient. With only 20 auxiliary molecules, Mole-PAIR obtains 0.617 AUROC on EC50-Assay (Appendix D.6), above the best ID-only baseline (0.572) and above the OE baselines trained with the same 20 molecules (see our response to Q5).

Why this is safe. The objective only requires auxiliary molecules to rank below ID molecules. They do not need to resemble the test-time OOD molecules, so a general background can be used once structures overlapping with the ID set are removed.

Q5: "Comparing tables 1, 4, and 7 makes it seem like the model performs poorly with scarce OOD data, potentially worse than the competitors. This is not discussed much; there is a line saying it out performs ID-only models, but I don't think it outperforms the other models also trained on OOD data, though that's never explicitly stated."

Thank you for raising this point, which allows us to state the comparison explicitly. The cross-table comparison uses different data budgets: Table 7 reports Mole-PAIR with as few as 20 auxiliary OOD molecules (EC50-Assay AUROC: 0.617), whereas the OE baselines in Table 4 use 2,000 (MSP-OE: 0.685).

Under the same budget, Mole-PAIR outperforms the OE baselines at every level. We fix 2,000 ID molecules, use identical nested auxiliary subsets for all methods, and apply a common full-batch training protocol. Results are MiniMol AUROC on EC50-Assay (mean ± SD over five seeds), reported in Appendix D.6:

Auxiliary OODMole-PAIRBest-OE Baseline
2000.683 ± 0.0020.596 ± 0.013
400.625 ± 0.0050.553 ± 0.011
200.617 ± 0.0070.562 ± 0.020

Mole-PAIR exceeds the stronger OE baseline by 0.055–0.087 AUROC under every matched budget.

Q1: "What is the motivation for choosing preference optimization to improve OOD detection? It is stated later in the paper, but an explanation on that would strengthen the introduction. I have an idea, but it would help to see that explicitly. The point iii of your contributions is good."

We sincerely thank you for recognizing the contribution of our paper. The motivation is to learn the quantity that OOD detection evaluates: the relative ordering of ID and OOD scores. The pairwise logistic objective encourages Eϕ(Sin)<Eϕ(Sout)E_\phi(S_{\text{in}}) < E_\phi(S_{\text{out}}) and provides a smooth surrogate for the ranking measured by AUROC (Eqs. 4–6, Section 3.2).

Mole-PAIR performs this learning directly in score space, without updating the foundation encoder or requiring downstream property labels. Averaging over ID–OOD pairs also avoids weighting the objective by the observed group proportions, which is why it remains effective with scarce auxiliary molecules (Appendix D.6).

AuthorsOpenReview ↗

Response to Reviewer j8gg (2)

Q3: "Figure 2 is confusing. What is it adding to the paper, and why is a smooth curve fit (presumably spline interpolation) appropriate?"

Q4: "Section 4.2 isn't really an ablation study, it's just two hyperparameter scans. Why isn't the graph for the lambda scan shown in the same figure (or at least visually nearby) as the beta scan? The conclusions drawn from these two scans are dubious since there is no grid scan of lambda and beta simultaneously. The behavior of the test AUC is clearly very sensitive to these parameters, so assuming that they could be optimized/checked separately is questionable."

Thank you for these suggestions, which we have adopted in full. Section 4.2 is renamed "Sensitivity Analysis", and Figure 2 now places the β\beta and λ\lambda scans in one figure, with every point re-run under the main-table protocol over five seeds and measured points joined by straight lines, replacing the interpolation.

Joint grid (Figure 5, Appendix D.2). We agree that the two parameters cannot be checked separately, and we now evaluate the full 3×33\times3 grid β∈{0.1,0.5,5}\beta \in \lbrace 0.1, 0.5, 5 \rbrace, λ∈{0.01,0.05,1}\lambda \in \lbrace 0.01, 0.05, 1 \rbrace on all three EC50 shifts, with five seeds per cell. The figure reports validation AUROC (used for selection) and test AUROC separately.

W2: "The organization of the paper felt scattered and hard to read at times, and some of the data was confusingly presented or hard to parse. Better use of graphs could help this."

Q6: "The organization of the appendices was a little scattered, which made it hard to verify data."

Q7: "A little more discussion of the benchmark datasets would have been appreciated."

Thank you for these suggestions. Appendix C and Appendix D now each open with a short guide that states, in order, which question every subsection answers, so that each number can be located. The hyperparameter scans and the joint grid are now presented as figures (Figures 2 and 5).

Appendix C.1 now explains what each shift actually tests: Assay is a functional shift across experimental contexts and is not identifiable from structure alone, Scaffold withholds molecular cores, and Size changes the atomic-count distribution and is the easiest to detect. This distinction is also stated in Section 4 (Datasets) and is what our transfer and background results turn on. Appendix C.1 further separates the requested from the realized sample counts (Appendix C.1, GOOD Datasets).

We sincerely thank you for taking the time to review our paper and provide valuable suggestions to help us make the paper better.

Dear Reviewer j8gg,

Thank you for your review.

We have added all the experiments and discussions that you have asked for in your rebuttal. The discussion phase will end soon and we are hoping to hear your thoughts.

Regards,

Authors

Dear Reviewer j8gg,

Thank you for taking the time to review our paper. As we approach the end of the discussion period, we would greatly appreciate any further feedback. Please let us know if you have any remaining questions.

We also hope you might consider reassessing your rating in light of our clarifications and additional experiments. All the added contents are included in the new submitted version based on your review.

Best regards,
Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

Dear Area Chairs and Reviewers,

We thank the reviewers for their careful reading and constructive comments. We are glad the reviewers found the problem of molecular OOD detection important (Qw8o, z1tf), the method plug-and-play and lightweight (Qw8o, z1tf), and the gains over exposure-based baselines consistent (j8gg, z1tf).

Two main questions recurred, and we address each with new experiments:

1. Does Mole-PAIR need shift-specific or plentiful auxiliary OOD data? No. We add a shift-agnostic PubChem background that never touches the target shift type, and a matched-budget study down to 20 auxiliary molecules. (j8gg W1/Q2/Q5, z1tf W2)

2. Is pairwise ranking actually better than a binary classifier? We add the direct control z1tf asked for: an identical-capacity BCE detector on identical embeddings, splits, and budgets. Mole-PAIR leads by 0.049-0.098 AUROC, and the gap widens as data shrinks. (z1tf W3/Q1)

New experiments added in the revision:

  • Matched-exposure comparison against a BCE detector and the OE baselines at 200/40/20 auxiliary molecules, five seeds each (Appendix D.6)
  • A shift-agnostic PubChem background that never uses the target shift type (Appendix D.5)
  • An additional transfer direction with Scaffold targets (Table 10, Appendix D.5)
  • Joint β×λ\beta \times \lambda grid with validation-based selection (Figure 5, Appendix D.2)
  • Threshold calibration from held-out ID molecules at target false-positive budgets (reported in our response to Reviewer Qw8o below)

We have also reorganized the paper: Section 3.1 now states exactly which partitions are used for training and model selection; Section 4.2 is renamed "Sensitivity Analysis" with both scans in one figure; and Appendices C and D each open with a guide to their subsections so every number can be located. All revisions are marked in blue in the updated PDF.

We are happy to provide further clarification and additional experiments to resolve any further question or feedback during the discussion. Thank you again for taking the time to review our paper.

Dear Reviewers,

We’d like to thank you again for your thorough review and constructive feedback. In light of the comments, we have added an extensive set of new results (see general response for a summary and individual responses for detailed results) and revised our paper accordingly.

We are sincerely looking forward to your further feedback and hoping that you can re-evaluate the ratings considering our response and revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.