acceptodds
Under review as a conference paper at ICLR 2027

Multi-Anchor Latent Transduction for Out-of-Distribution Molecular Property Prediction

Abstract

Predicting molecular properties under out-of-distribution (OOD) shift remains a major bottleneck in drug discovery. Transductive extrapolation is attractive because it predicts a test molecule relative to known training analogues, but existing molecular transduction methods such as Bilinear Transduction (BLT) have mainly been validated in hand-crafted descriptor spaces, where differences between descriptor vectors can be interpreted as property-relevant molecular transformations. This assumption does not reliably transfer to modern molecular encoders: learned embeddings are distributed, task-shaped representations whose nearest neighbors and difference vectors need not define useful analogies for property extrapolation. We introduce Multi-Anchor Latent Transduction (MALT), a molecular OOD transduction framework that makes retrieval geometry a learned component of the property predictor rather than relying on a fixed descriptor or frozen embedding space. MALT jointly trains the encoder, support geometry, and anchor-conditioned prediction rule under the same property objective, turning retrieved molecules into property-relevant latent supports rather than passive nearest neighbors. At inference time, MALT retrieves multiple training anchors in the learned latent space and predicts from their relations to the query. We instantiate two complementary prediction heads: a label-free latent-fusion head that uses only query and anchor embeddings, and a label-conditioned latent-delta head that extends BLT to learned molecular embeddings through multi-anchor updates. Across matched molecular OOD comparisons, MALT reduces average OOD error by 8.2–16.1% across different molecular representation modalities, with 14.2–30.1% reductions under label-range extrapolation, while maintaining competitive in-distribution performance. Code released in https://anonymous.4open.science/r/iclr_malt

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
molecular property prediction, out-of-distribution generalization, transductive learning, retrieval-augmented prediction, multi-anchor attention, molecular representation learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qo6bOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Multi-Anchor Latent Transduction (MALT) for out-of-distribution molecular property prediction. Instead of predicting from a query molecule alone, MALT retrieves multiple training molecules in a learned latent space and uses them as anchors for prediction. Two variants are introduced: MALT-Fusion, which combines query and anchor embeddings through attention, and MALT-Delta, which further uses anchor labels and predicted property corrections. The framework is evaluated on MoleculeNet, BOOM, and Activity Cliffs, covering different types of molecular distribution shifts. The results show improvements over several inductive and transductive baselines.

Critical strengths
  • The paper addresses an important problem in molecular property prediction. Using training molecules as references for OOD prediction is an interesting direction, especially for property extrapolation and activity cliffs.

  • The evaluation covers several OOD scenarios, including structural shifts, target-value extrapolation, low-density property regions, and activity cliffs. The experiments also use different molecular encoders, which helps assess the applicability of the framework.

Critical weaknesses
  • The methodological novelty is somewhat limited. The main components of MALT are learned molecular embeddings, nearest-neighbor retrieval, multi-head attention, and anchor-conditioned prediction. MALT-Fusion in particular looks quite similar to existing retrieval-augmented prediction frameworks. MALT-Delta extends BLT by replacing descriptor differences with latent differences and aggregating multiple anchors, which is reasonable but also relatively straightforward.

  • The paper emphasizes jointly learning the encoder, retrieval geometry, and prediction head. However, retrieval itself is non-differentiable, and the memory bank is updated periodically as the encoder changes. It is not entirely clear what new learning mechanism is introduced beyond training a retrieval-conditioned predictor with a task-specific encoder.

  • The reported results are encouraging overall, particularly under Y-split label extrapolation. However, the gains are considerably less consistent under structural OOD shifts. For example, on the MoleculeNet X-splits, MALT-Fusion with Chemprop improves BACE and ESOL but performs worse on FreeSolv and Lipophilicity. The Activity Cliffs results also show a clear dependence on the underlying encoder. The improvements for GIN are relatively large, while those for UniMol are only around 1% in OOD RMSE. This raises some questions about when the proposed method actually helps. The strong results under label extrapolation are interesting, but they do not necessarily imply equally strong generalization to unseen molecular structures. I would like to see a clearer analysis of the settings where MALT is less effective.

  • A central motivation is that conventional embedding spaces may retrieve molecules that are structurally similar but not useful for property extrapolation. The proposed solution is to learn a more property-relevant retrieval space. While the retrieval ablations support the importance of anchor selection, the paper provides relatively little direct evidence about what the learned neighbors actually represent chemically. This is particularly relevant to activity cliffs, where highly similar molecules can have very different activities. Does the learned retrieval space avoid such misleading neighbors? Are the selected anchors more consistent in their properties, or do they capture meaningful molecular transformations? Some concrete examples and quantitative analysis of anchor quality would make the main motivation more convincing.

  • The two prediction heads have different operating regimes, but the choice between them is not sufficiently clear. MALT-Fusion and MALT-Delta behave differently across the experiments. In some settings, anchor labels provide useful local priors, but in others they appear to hurt performance. As the authors described, Section 4.5 reports worse aggregate OOD MAE for MALT-Delta than MALT-Fusion with UniMol (0.506 vs. 0.469). For a new dataset, it therefore seems unclear which head should be used. The authors also mention that Figure 3 uses earlier UniMol results without the matched fine-tuned initialization, while the main discussion refers to the matched setting. However, using the same experimental setup throughout seems to make the comparison easier to interpret.

  • The presentation of the proposed architecture could be improved. For a paper introducing a new retrieval-based framework, it is a bit surprising that there is no architecture figure in the main text. The only framework illustration appears in Appendix and it mainly shows a general encoder, memory bank, retrieval module, and prediction head. It does not explain the differences between MALT-Fusion and MALT-Delta very clearly, especially the latent correction and anchor-label mechanism of MALT-Delta. Since these two heads are central to the paper, a more detailed figure in the main text would help readers understand the method.

Questions for authors

See weaknesses.

3 comments

W1–W2 — Contribution and learning. Thank you for asking us to clarify the contribution. MALT extends BLT-inspired molecular extrapolation to task-adapted encoder representations and jointly trained multi-anchor prediction. The issue is a representation–inference mismatch: query-only embeddings need not encode useful query–anchor relations for extrapolation. New Figure 1 shows the shared encoder and detached bank (a), label-free Fusion (b), and Delta's anchor-label-plus-correction pathway (c), including the training and bank-refresh paths.

1) Relative to BLT [2,3]. Molecular BLT [3] selects a single anchor by matching descriptor differences to training-pair differences via an argmin. Its Discussion identifies metric learning for anchor selection and graph/3D representations as future directions. MALT develops these directions by adapting pretrained molecular encoders under a multi-anchor prediction loss. BLT already learns nonlinear bilinear factors; our additional change is to adapt the coordinates used for retrieval and displacement prediction. Fusion predicts from query/support embeddings without anchor labels. Delta predicts

\delta_a=f([z_q-\bar z_a;e_{qa}])^\top g(\bar z_a).$$ The correction couples a latent-displacement/pair-feature factor with anchor context. Full Delta includes RDKit pair features. **Loss is applied to the aggregate of corrected anchor-property estimates**, rather than adding multi-anchor averaging only at inference. Table 2 also compares with BLT adapted to GIN, Chemprop and UniMol: Fusion/Delta have lower OOD means in **20/24 and 21/24** corresponding comparisons. These are complete-configuration comparisons, not matched-budget component isolations. **2) Relative to retrieval-conditioned predictors.** We agree that retrieval-conditioned training and attention are established; **our contribution is the specific molecular extrapolation formulation**, not a new retrieval or attention primitive. TabR [1] already uses label embeddings and displacement-dependent corrections. Unlike an embedding-update readout, **Delta explicitly reconstructs a target-unit estimate from each measured anchor label and an anchor-context-dependent bilinear correction, then aggregates these estimates** (Figure 1c). This is a prediction inductive bias, not an expressivity claim. Fusion uses standard attention but provides a complementary retrieved-label-free pathway (Figure 1b); fixed-neighbor label-readout controls (Table 7) and anchor interventions (Table 20) support its reliance on informative supports. **Matched retrieval-conditioned controls (Table 5; Appendix G):** | Y-split OOD MAE | BACE | ESOL | FreeSolv | Lipo | |---|---:|---:|---:|---:| | TabR-style | .811±.061 | .490±.024 | .244±.019 | .640±.046 | | Set attention | .728±.019 | .469±.042 | .256±.062 | .679±.038 | | **MALT-Delta** | **.670±.030** | **.450±.017** | **.177±.029** | **.604±.015** | Five-seed mean±SD. All arms share an MSE-trained GIN warm start, 500 MAE epochs, optimizer/schedule, access to ten training-anchor labels and 14 pair features, and trainable parameter counts within 0.1%. All use exact Euclidean top-10 retrieval, detached training-only banks refreshed each epoch and self-exclusion. Encoders adapt separately, so retrieved identities can differ. These are adapted controls, not full original-system reproductions; the separate main benchmarks use MAE fine-tuning. One training-identical ESOL OOD query per split is excluded equally across arms. **Delta improves all four Y-split means** (mean task-relative MAE reductions **14.67%/13.53%** versus TabR-style/set attention, from unrounded means). This supports the complete prediction rule under label extrapolation, not bilinearity alone; X-split exceptions are discussed in W3. **3) Non-differentiable retrieval (Figure 1; Algorithm 1).** Each training molecule is predicted from other training supports. The property loss updates the query encoder and head, **not** hard top-k indices or stored bank embeddings. Epoch-wise bank re-encoding changes support coordinates and future neighbors. Thus, **the encoder-induced retrieval geometry adapts indirectly**, without backpropagating through retrieval or introducing a new metric loss.

W3 — Where gains diminish. Y-split gains are strongest; structural/cliff benefits depend more on the encoder. Fusion versus query-only reductions:

EncoderY MAEX MAECliffs RMSE
GIN19.7%9.1%14.9%
Chemprop30.1%2.0%8.6%
UniMol14.2%2.3%0.9%

Y/X average task-relative reductions; cliffs use target medians (Tables 2/4). Chemprop-Fusion has FreeSolv/Lipo-X exceptions; UniMol's cliff gain is small. On X-splits (Table 5), Delta lowers BACE/ESOL means versus both controls and Lipo versus set attention (.490 versus .495); TabR-style remains stronger on Lipo (.455 versus .490).

Coverage analysis: Chemprop reruns (30 targets/five seeds) group queries by maximum training ECFP4 similarity. Median RMSE reductions relative to the restored query-only checkpoint remain 3.88% [1.79,6.76] at lowest coverage, rising to 14.98% [9.99,19.05] at highest coverage (95% paired target-family/seed bootstrap CIs; Table 30). RMSE uses all benchmark test queries within each tertile, including both cliff-flagged and unflagged queries.

W4 — Anchor relevance. Chemprop: 30 targets/five seeds; query labels are used only for offline scoring (Appendix I).

Retrieved anchors have smaller activity gaps than non-retrieved training molecules matched within .025 ECFP4 similarity and the same benchmark similarity category (Table 28):

HeadActivity-gap reductionTarget winsSupports matched
Fusion18.52%30/3069.0%
Delta10.97%30/3072.3%

Effects apply only to matched supports. Activity-gap and cliff-rate comparisons average five seeds per target and weight all 30 targets equally. Fusion's cliff rate among matched structurally similar pairs is 4.38 percentage points lower [1.00,7.22] (95% target-bootstrap CI). This comparison is observational.

Fusion before/after training: structural matching of initial/trained supports covers 82.8%. Mean target-wise activity gaps fall 1.42% [0.85,2.05] (24/30 target means; 95% paired target-family/seed bootstrap CI). Initial geometry is task-fine-tuned; the cliff-rate change is inconclusive.

Example (Table 29): for a Cl query in CHEMBL214_Ki, fingerprints prefer an isopropyl analogue (similarity .8254; gap 1.396) over F (.8125; .097 log units). Fusion ranks F first and isopropyl 11th. The initial encoder also ranks F first: this illustrates latent versus fingerprint ordering; the separate before/after analysis measures training's added effect.

W5 — Head selection. Fusion is the conservative default. Consider descriptor-assisted Delta (Table 18) when simulated deployment shifts formed from training data support it; this policy remains unvalidated. UniMol's .469/.506 compares complete configurations (Table 2; Appendix E.5).

W6 — Presentation. We added the two-branch architecture (Figures 1/4), gradient/bank-refresh paths, and reorganized Tables 2–3. The unmatched-initialization anchor-label plot was removed; head comparisons use Table 2/Appendix E.5, with X/Y results separate.

AuthorsOpenReview ↗

Summary and Appreciation

We sincerely thank the reviewer for the detailed and constructive feedback. We have revised the manuscript and conducted additional experiments to address all six concerns raised in the review. We would like to briefly summarize the main revisions.

Methodological contribution (W1–W2). We clarify MALT's distinction from BLT and retrieval-conditioned prediction: a task-adapted molecular multi-anchor transductive formulation with two complementary prediction rules. The new Figure 1 illustrates both heads and the training mechanism, including non-differentiable retrieval and implicit adaptation of the encoder-induced retrieval geometry. New matched TabR-style and set-attention controls (Table 5; Appendix G) show 14.67%/13.53% mean task-relative OOD MAE reductions for Delta under label extrapolation.

Operating regimes (W3). We explicitly analyze where gains diminish, including structural shifts and strong pretrained encoders. New Activity Cliffs experiments quantify performance across analogue-coverage levels, showing positive but smaller gains when training analogues are less similar.

Anchor relevance (W4). New similarity-controlled analyses across 30 Activity Cliffs targets demonstrate 18.52%/10.97% smaller activity gaps for Fusion/Delta on matched supports, alongside cliff-rate analysis, before/after-training comparisons, and a concrete chemical example (Appendix I).

Head selection (W5). We clarify when Fusion and Delta are preferable, recommending Fusion as the conservative default while explicitly acknowledging that the proposed validation-based selection policy remains unvalidated.

Presentation (W6). We added a detailed main-text architecture figure, reorganized Tables 2–3 for clearer comparisons, and removed the earlier inconsistent head-comparison figure.

We hope these revisions address the reviewer's main concerns and make both the methodological contribution and its empirical evidence substantially clearer. We appreciate the feedback, which helped strengthen the manuscript.

References

[1] Netanyahu et al. (2023). Learning to Extrapolate: A Transductive Approach.

[2] Segal et al. (2025). Known Unknowns: Out-of-Distribution Property Prediction in Materials and Molecules.

[3] Gorishniy et al. (2024). TabR: Tabular Deep Learning Meets Nearest Neighbors.

[4] Lee et al. (2019). Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks.

[5] Kim et al. (2019). Attentive Neural Processes. ICLR.

Reviewer_DJCtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes MALT, a multi-anchor transductive framework for molecular property prediction under distribution shifts. The authors aim to address the mismatch between learned molecular representations and the relationships required for transductive prediction. The framework retrieves multiple molecular anchors from a learned latent space and uses either latent feature fusion or anchor-based corrections for prediction. Experiments across several OOD benchmarks show improvements over existing inductive and transductive methods.

Critical strengths
  1. The paper tackles an important problem, as OOD generalization remains a major challenge in molecular property prediction and drug discovery. Using multiple molecular anchors to support predictions is an interesting idea, and the proposed framework can be applied to different molecular encoders.

  2. The experimental evaluation is fairly comprehensive, covering different types of distribution shifts and a variety of molecular properties.

  3. The authors also conduct a number of ablation studies, including analyses of retrieval strategies and anchor quality. These experiments provide useful evidence for the effectiveness of the approach.

Critical weaknesses
  1. The paper identifies a representation–inference mismatch, but the Introduction provides limited discussion of related work. It remains unclear why existing representation learning and retrieval-based methods are insufficient for molecular extrapolation.

  2. The framework mainly combines established techniques, including nearest-neighbor retrieval, attention-based fusion, and bilinear prediction. The claim of jointly learning retrieval geometry also needs clarification, given the non-differentiable retrieval operation.

  3. The fairness of the experimental comparisons is a concern. MALT receives additional training on top of already fine-tuned encoders, potentially giving it an advantage over the inductive baselines. Comparisons with matched training budgets and model capacity would help establish the actual contribution of anchor-conditioned prediction.

  4. The presentation needs further polishing. Tables 2 and 3 are rather dense, and some methodological details remain unclear. For example, Equation (16) appears to weight the anchor labels but not the predicted corrections. The authors should clarify whether this is intentional.

Questions for authors
  1. Could the authors clarify what makes MALT fundamentally different from existing retrieval-based and transductive molecular prediction methods?

  2. How would MALT compare with inductive baselines given the same additional training budget and comparable model capacity?

3 comments

Thank you for the thoughtful review. We have revised the manuscript and added controls. All section, table and figure references below refer to that revision.

Q1 — Contribution relative to BLT and retrieval-augmented prediction

MALT builds on transductive extrapolation [1] by jointly training molecular encoders for multi-anchor prediction. The representation–inference mismatch motivates this formulation: query-only fine-tuning need not produce the query–anchor relations required for extrapolation. Figure 1 now shows the shared encoder/bank workflow and the complementary Fusion/Delta pathways. We have expanded the Introduction and Section 2.2 to clarify MALT's relationship to query-only representation learning, BLT and retrieval-conditioned prediction.

1) Extending molecular BLT [1,2]. Molecular BLT [2] selects one anchor through a descriptor-space difference-matching argmin. Its Discussion identifies metric learning for anchor selection and graph/3D-aware representations as future directions. Original BLT [1] already learns nonlinear factors and includes weighted variants; MALT additionally adapts the molecular coordinates used for retrieval and displacements under the multi-anchor prediction loss. It starts from task-fine-tuned encoders, including pretrained GIN and UniMol.

Table 2 also tests BLT adaptations with GIN, Chemprop and UniMol, not only descriptor-based BLT: Fusion/Delta achieve lower OOD means in 20/24 and 21/24 corresponding comparisons. These are complete-method comparisons, not isolated tests of anchor selection or matched training budgets. Q2 separately controls additional optimization and approximate parameter counts against a query-only GIN.

2) A specific prediction rule within retrieval-conditioned learning. We agree that MALT belongs to this broader family. Our distinction is the molecular multi-anchor formulation, not retrieval-conditioned training or attention alone. TabR [3] already uses label embeddings and displacement-dependent corrections in a representation update. Delta explicitly reconstructs a scalar query property:

\delta_a &= f_\psi([z_q-\bar z_a;e_{qa}])^\top g_\omega(\bar z_a),\\ \hat y_q &= \sum_{a\in\mathcal A(q)}\lambda_a(y_a+\delta_a). \end{aligned}$$ Here $\bar z_a$ is a detached training embedding and $e_{qa}$ contains RDKit pair features (Appendix H). The correction is an inner product of learned nonlinear factors: the anchor-context factor explicitly modulates the displacement/pair-feature factor. Measured anchor properties supply reference levels in target units; corrections permit predictions beyond their range. **Training minimizes property loss on the aggregated estimate, rather than adding multi-anchor averaging only at inference.** This is a particular parameterization, not a claim that generic retrieval networks cannot express similar functions. **Fusion provides the alternative without retrieved-label access (Figure 1b).** It uses standard attention; fixed-neighbor readout comparisons and test-time anchor interventions support its use of retrieved information beyond direct label aggregation or an effectively query-only predictor (Tables 7 and 20). **Learning the geometry (Figure 1; Algorithm 1).** Training molecules are predicted from other training supports. Property-loss gradients update the query encoder/head; stored embeddings and hard top-k indices receive no gradients. Epoch-wise re-encoding changes support coordinates and can change future neighbors. **The representation adapts for anchor-conditioned prediction; the hard top-k rule itself is not learned.** This uses standard optimization, without a separate metric loss or a new differentiable retrieval algorithm. **Matched retrieval controls (Table 5; Appendix G).** Delta, adapted TabR-style and set attention [4,5] share an MSE-trained GIN warm start, 500 MAE epochs, optimizer/schedule, access to ten anchor labels and 14 pair features, and parameter counts within 0.1%. All use Euclidean top-10 retrieval, epoch-refreshed detached training banks and self-exclusion. Encoders adapt separately, so anchors can differ. This study is separate from the main MAE-initialized benchmarks. | Y-split OOD MAE | BACE | ESOL | FreeSolv | Lipo | |---|---:|---:|---:|---:| | TabR-style | .811±.061 | .490±.024 | .244±.019 | .640±.046 | | Set attention | .728±.019 | .469±.042 | .256±.062 | .679±.038 | | MALT-Delta | **.670±.030** | **.450±.017** | **.177±.029** | **.604±.015** | Five-seed mean±SD. **Delta improves all four Y-split means**, reducing mean task-relative OOD MAE by **14.67%/13.53%** versus TabR-style/set attention (unrounded means). These adapted controls support the complete formulation, not bilinearity alone; they are not full reproductions of the original methods.

Q2 — Matched additional optimization and parameter counts

GIN–Delta improves all eight means over a query-only GIN with an MLP readout (Tables 26–27; Appendix G). Both restore the same MAE-trained encoder, selected on a fixed 10% training-support holdout, then receive 500 epochs on all training molecules with identical batches, optimizer and schedule. We evaluate final-epoch models; total trainable parameter counts differ by less than 0.1%.

Dataset / splitQuery-only GINMALT-Delta
BACE-Y.727±.026.705±.043
ESOL-Y.500±.017.448±.016
FreeSolv-Y.299±.027.178±.048
Lipo-Y.646±.029.591±.027
BACE-X.612±.007.503±.028
ESOL-X.501±.018.334±.035
FreeSolv-X.351±.021.258±.025
Lipo-X.535±.013.503±.015

Five-seed OOD MAE (mean±SD). Mean task-relative reductions are 15.56% on Y and 20.92% on X, using unrounded means. This controls additional optimization and approximate parameter counts for the complete GIN–Delta configuration; it does not match FLOPs, wall time or inference access, or establish the same result for Fusion or all encoders. Both new GIN studies exclude one training-identical ESOL query per split; benchmark tables retain supplied partitions.

Equation (16). The same anchor weight multiplies both its label and correction: y^q=∑aλa(ya+δa)\hat y_q=\sum_a\lambda_a(y_a+\delta_a). We have rewritten Equation (16) to show both weighted terms explicitly and added an explanatory sentence, consistent with Algorithm 1. This clarification does not change the prediction rule.

Presentation. We have added the two-branch architecture in Figure 1 and detailed workflow in Figure 4, reorganized Tables 2–3 by encoder/backbone while retaining the full results, and standardized encoder/parameter notation in the algorithms.

AuthorsOpenReview ↗

Summary and Appreciation

We thank the reviewer for the thoughtful feedback, which helped us strengthen both the experimental comparisons and the presentation. We summarize the main revisions addressing your questions below.

Contribution and learning mechanism (Q1). We clarified MALT's relationship to BLT and retrieval-conditioned predictors, distinguishing the molecular multi-anchor formulation from its established components. We also clarified that the encoder and prediction head are trained jointly, while retrieval geometry adapts through bank refreshes—not gradients through hard top-k selection. New adapted TabR-style and set-attention controls match initialization, additional optimization, approximate parameter counts and support-information access. Delta achieves lower mean OOD MAE on all four Y-split datasets against both controls (Table 5; Appendix G).

Fairness of additional training (Q2). We added a query-only GIN comparison sharing the same MAE-trained initialization, 500 additional epochs, optimizer and schedule, with trainable parameter counts differing by less than 0.1%. GIN–Delta improves all eight dataset–split means, reducing mean task-relative OOD MAE by 15.56% on Y-splits and 20.92% on X-splits (Tables 26–27). We explicitly limit this conclusion to the complete GIN–Delta configuration; the control does not match computation or inference access.

Equation and presentation. We rewrote Equation (16) to make explicit that each anchor weight multiplies both its label and correction, without changing the prediction rule. We added the main-text architecture in Figure 1, a detailed workflow in Figure 4, and reorganized Tables 2–3 for clearer comparisons while retaining the full results.

We hope these revisions clarify MALT's contribution and provide the controlled evidence requested. We appreciate the reviewer's suggestions, which helped improve the manuscript.

References

[1] Netanyahu et al. (2023). Learning to Extrapolate: A Transductive Approach.

[2] Segal et al. (2025). Known Unknowns: Out-of-Distribution Property Prediction in Materials and Molecules.

[3] Gorishniy et al. (2024). TabR: Tabular Deep Learning Meets Nearest Neighbors.

[4] Lee et al. (2019). Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks.

[5] Kim et al. (2019). Attentive Neural Processes. ICLR.

Reviewer_bHG6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper expands on transductive OOD generalization methods, describing a drop-in prediction head and finetuning procedure compatible with existing chemical property predictors. This head takes advantage of a so-called memory bank containing latent representations of molecules in the training set, which can be retrieved at inference time for the predictive head to use in transductive prediction. The authors demonstrate the procedure’s effectiveness by applying it to several existing predictors on established benchmarks. MALT does not improve performance in all evaluations, but demonstrates significant improvements in MoleculeNet nearly across-the-board. The authors also use an ablation study to verify the value of each of the procedure’s components.

Critical strengths

The method represents an innovation over BLT; using latent representations to select anchors is more suitable to the transductive prediction task, at least with pretrained predictors.

The explanation is thorough and concise, including inference and training procedures. The empirical validation has appropriate baselines and demonstrates significant improvement.

Critical weaknesses

Some clarity issues: Tables 2 and 3 are unclear: encoder order changes between methods and the highlighting scheme does not indicate all comparisons between the baseline, BLT, and MALT. The use of εθ/ θε for the encoder and its weights in pseudocode does not parallel the use of P and T.

Questions for authors

Were Euclidean distance and cosine similarity the only evaluated distance metrics?

Why does adding retrieval distance to the set of input features reduce accuracy on certain datasets? Is it a case of fitting to the in-domain training data?

1 comment
AuthorsOpenReview ↗

Q1-2 and weaknesses

Thank you for the positive assessment and helpful suggestions. We have added distance-input diagnostics and clarified the presentation. All section, table and figure references below refer to the revised manuscript.

Q1 — Were Euclidean distance and cosine similarity the only evaluated metrics?

Yes, for latent-space anchor retrieval (Appendix E.3.1; Table 13). Adaptive masking and temperature sampling are selection strategies using these measures, not additional metrics. We use deterministic top-10 Euclidean retrieval, which has the best overall rank across the evaluated ID/OOD settings. Separate fingerprint-based analyses use ECFP4 Tanimoto similarity.

Q2 — Why can adding distance hurt? Is this in-domain overfitting?

Our follow-up supports sensitivity to raw distance scale, but does not establish in-domain overfitting as the cause. Top-k selection depends on distance ordering, whereas supplying raw distances also lets the prediction head depend on their magnitudes. Default Fusion instead predicts from query/support embeddings without explicit distance inputs.

Controlled follow-up (Appendix J). We compared three GIN-Fusion variants: no distance inputs, ten raw anchor distances, and normalized distances, across four datasets, X/Y splits and five seeds. All use common MAE warm starts, 500 additional epochs and Euclidean top-10 retrieval. A fixed 10% training-support validation subset remains excluded from adaptation training and memory banks. These runs are separate from Table 13 and the other GIN control studies.

OOD accuracy. Table 31 reports the following OOD MAE reductions for default Fusion relative to raw-distance Fusion:

DatasetY-splitX-split
ESOL12.05%1.17%
FreeSolv8.83%6.35%

Each entry compares five-seed dataset means. These results confirm that explicit raw-distance inputs can hurt OOD accuracy in these settings.

Normalization. We also trained the distance-input head using:

normalized_distance = distance / (mean_anchor_distance + 1e-8)

Here, mean_anchor_distance is computed separately for each query over its ten retrieved anchors.

Compared with raw inputs, normalization reduces held-out validation MAE by 9.27% on Y-splits and 8.77% on X-splits, improving all eight dataset/split means. These percentages average task-relative reductions equally across the four datasets; they are validation results, not OOD improvements.

Rescaling diagnostic. With model weights and selected anchors fixed, doubling only the distance scores changes raw-input predictions; normalized-input predictions remain unchanged to numerical precision. Normalization largely cancels uniform rescaling, so this verifies sensitivity of the explicit distance-input channel, not that an OOD scale shift actually caused the errors.

Together, these findings support scale sensitivity as a possible contributor to the dataset-dependent effect. They do not establish training-distribution overfitting or universal OOD superiority of normalized distances.

Presentation and notation. Tables 2–3 now organize comparisons by encoder/backbone with consistent ordering, retaining the full ID/OOD results. Table 2 places query-only, BLT and MALT results within each encoder block; its caption defines query-only as the fine-tuned inductive predictor. Green identifies MALT rows rather than implying improvement. Bold/underline mark the best/second-best means within each encoder block in Table 2 and across methods in each column in Table 3, as the captions specify.

Algorithms 1–2 now consistently use EθE\mathcal E_{\theta_{\mathcal E}} for the encoder, distinguish encoder/head parameters from the fixed retrieval operator, and update encoder/head parameters only. Figure 1 illustrates both alternative heads and their training paths; Figure 4 provides the detailed workflow.

We appreciate the suggestions, which helped us clarify both the distance-input findings and the presentation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final Author Comment

We sincerely thank all reviewers for their constructive feedback, which has helped us strengthen the manuscript through additional experiments, clearer methodological explanations, and improved presentation.

We would like to emphasize the central novelty of MALT: extending transductive learning for out-of-support extrapolation to modern, powerful molecular encoders. Building upon prior work demonstrating that transductive reparameterization can enable extrapolation through analogical input–target relationships, MALT moves beyond fixed, hand-crafted molecular descriptors to learned molecular representations. Our key contribution is a multi-anchor transductive framework that jointly adapts molecular representations and anchor-conditioned prediction, retaining the expressive power of modern molecular encoders while enabling more effective extrapolation beyond the observed property range. Rather than proposing a new retrieval or attention mechanism, we address the representation–inference mismatch that arises when conventional molecular embeddings are used for transductive prediction.

Importantly, our extensive empirical evaluation demonstrates substantial improvements in OOD generalization, particularly in the challenging setting of label-range extrapolation. Across different molecular encoder families, MALT reduces average OOD prediction error by 8.2–16.1%, with particularly strong reductions of 14.2–30.1% when extrapolating to property values beyond the training distribution. These results highlight that transductive learning can complement powerful molecular representations and improve their ability to predict previously unseen property ranges, a critical capability for discovering molecules with extreme or desirable properties.

In response to the reviews, we added controlled comparisons with matched training budgets and model capacities, stronger retrieval-based baselines, quantitative analyses of anchor relevance, and further evaluations under different OOD conditions. These additional experiments reinforce our methodological contribution and provide stronger evidence that the improvements arise from the proposed transductive formulation rather than simply additional training or retrieval.

We hope the revised manuscript and additional evidence address the reviewers' concerns and clarify both the novelty and practical significance of MALT. We greatly appreciate the reviewers' time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.