acceptodds
Under review as a conference paper at ICLR 2027

MolPFN: Graph-Level Prior-Data Fitted Networks for Molecular Property Prediction

Abstract

Molecular property prediction is essential across chemical, biological, and materials applications, from molecular discovery to materials design. However, experimentally measured labels are often scarce, whereas existing molecular models often require large labeled datasets and task-specific training. Tabular foundation models based on Prior-Data Fitted Networks (PFNs) offer a promising alternative: they achieve strong predictive performance in small-data regimes and can adapt to new tasks through in-context learning without task-specific parameter updates. Yet, applying these models to molecules requires bridging the gap between graphstructured molecular inputs and the tabular representations expected by PFNs. We introduce MOLPFN, a framework for in-context molecular property prediction with a learnable molecular graph adapter, that transforms molecular graphs into PFN-compatible representations, via a synthetic graph prior designed for PFN pretraining. MOLPFN thereby combines molecular graph structure and atomic information with the in-context learning capabilities of PFNs, while supporting optional auxiliary molecule-level information, such as learned embeddings and molecular fingerprints, without task-specific retraining. We evaluate MOLPFN on nine molecular benchmarks covering regression and classification tasks. When trained only on tasks sampled from the synthetic graph prior, MOLPFN matches or outperforms established tabular foundation models consistently improving results across tasks. We further fine-tune MOLPFN across thousands of experimentally measured small-molecule bioactivity tasks, substantially improving fewshot prediction for previously unseen protein targets and outperforming most established few-shot molecular learning baselines, even when using only standard atomic features. Together, these results show how domain-specific adapters and structured priors can extend tabular foundation models to molecular graphs, enabling flexible in-context prediction across molecular tasks and representations.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Foundation Models, Meta Learning, Prior Fitted Networks, Molecular Property Prediction, Learning on Graphs.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Bb9MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MolPFN, a graph-level prior-data fitted network for molecular property prediction without task-specific fine-tuning. The method extends a pretrained tabular PFN, LimiX, to molecular graphs by introducing molecular tokenization and graph adapters. The graph adapters are interleaved with frozen Transformer blocks to incorporate molecular connectivity while leveraging in-context learning. To pretrain these modules, the authors construct a synthetic graph-level prior that generates molecular-like graphs, atom features, and target labels. Experimental results demonstrate competitive performance on molecular regression tasks and the potential to transfer to classification and few-shot prediction settings.

Critical strengths

1. Graph-aware Extension of Tabular PFNs

The paper presents an interesting approach to extending tabular PFNs to molecular graph prediction. By integrating graph adapters between Transformer blocks, the proposed method enables repeated interaction between molecular structural information and context-dependent representations. This architecture provides a meaningful alternative to approaches that compress molecular graphs into fixed-dimensional representations before PFN inference.

2. Competitive Performance Without Task-specific Fine-tuning

The proposed method achieves competitive performance on molecular property prediction tasks without updating model parameters for each new task. This is an important advantage because it allows the model to leverage labeled context examples for prediction without requiring additional task-specific fine-tuning. The use of a synthetic graph-level prior for pretraining without experimentally measured molecular property labels is also an interesting aspect of the approach.

Critical weaknesses

1. Insufficient Justification for the Synthetic Graph-level Prior

The proposed method generates synthetic graphs and labels using randomly constructed structural causal models (SCMs). However, it is unclear why the relationships learned from these synthetic tasks should transfer effectively to real molecular property prediction. Although the generated graphs are designed to resemble molecular structures, similarity in graph topology does not necessarily imply similarity in the relationships between molecular features and target properties. The paper would benefit from a clearer explanation or experimental analysis of why the proposed prior is appropriate for molecular prediction.

2. Insufficient Analysis of the Graph Adapter's Contribution

The proposed method repeatedly integrates graph adapters between LimiX Transformer blocks to incorporate molecular connectivity. Although comparisons with approaches using pretrained molecular encoders are provided, it remains unclear whether this interleaved architecture provides substantial benefits over simpler approaches that encode molecular graphs before PFN inference. A more direct ablation study would help clarify whether repeated graph processing is necessary and how much it contributes to the overall performance improvement.

Questions for authors

1. Transferability of the Synthetic Graph-level Prior

What characteristics of the synthetic feature–label relationships enable effective transfer to real molecular property prediction? Could the authors provide additional analysis demonstrating how the proposed synthetic prior relates to real molecular prediction tasks?

2. Classification Performance

Although MolPFN achieves competitive classification performance despite regression-only pretraining, could the authors clarify whether classification-specific graph adapter pretraining would be expected to further improve these results?

3. Necessity of Interleaved Graph Processing

How much does repeatedly inserting graph adapters between LimiX Transformer blocks contribute to prediction performance? Would a simpler architecture that applies graph encoding only before PFN inference achieve comparable results?

4. Sensitivity to Synthetic Data Generation

How sensitive is the performance of MolPFN to the design of the synthetic graph-level prior? In particular, how do different graph generation strategies or SCM-based label generation mechanisms affect downstream prediction performance?

3 comments
AuthorsOpenReview ↗

Response to questions 1 and 2

We thank the reviewer for their insightful comments and for highlighting the value of graph-aware in-context learning without task-specific fine-tuning. In response to their questions, we have further examined the transferability and sensitivity of our synthetic graph-level prior, investigated the contribution of interleaved graph processing through additional ablations, and clarified the potential role of classification-specific pretraining. We discuss these findings below.

Question 1: Transferability of the Synthetic Graph-level Prior

What characteristics of the synthetic feature–label relationships enable effective transfer to real molecular property prediction? Could the authors provide additional analysis demonstrating how the proposed synthetic prior relates to real molecular prediction tasks

We thank the reviewer for this question. The key characteristic of our synthetic prior is that it constructs graph-level labels from node features through graph-dependent computation. This reflects the structure of molecular property prediction, where atom attributes and molecular connectivity jointly inform a molecule-level target.

Specifically, our GNN-based SCM propagates information between connected nodes, while a virtual readout node connected to all nodes aggregates information to generate the graph-level label. Degree and PageRank values are also included as structural inputs in a sampled subset of synthetic datasets. These components provide a learnable relationship between node-level information, graph structure, and the target. The intended correspondence to molecular tasks is therefore in the structure of the prediction problem; we do not assume that the sampled SCMs reproduce specific physical or chemical mechanisms.

Our additional ablations (Appendix E) provide empirical support for this interpretation. Competitive downstream performance across different graph generators (Table A16) suggests that transfer does not require precisely matching molecular topology. Conversely, the substantially weaker performance of the edge-independent SCM variant (Tables A18-19) highlights the importance of preserving meaningful node-feature-to-graph-label relationships. Together, these findings support the relevance of the synthetic prior's graph-level prediction structure for transfer to real molecular tasks.

Question 2: Classification Performance

Although MolPFN achieves competitive classification performance despite regression-only pretraining, could the authors clarify whether classification-specific graph adapter pretraining would be expected to further improve these results?

Classification-specific graph adapter pretraining could potentially improve performance; however, our additional evaluation suggests that regression-only pretraining already learns representations that transfer effectively to classification. We believe that this transfer is plausible because threshold-based classification preserves the ordering of the underlying continuous targets, allowing a regression-trained model to distinguish classes without explicitly learning a classification objective.

Specifically, we sampled eight synthetic datasets from the training prior and converted their regression targets into binary classification labels. Without classification-specific adapter training, MolPFN achieved a mean ROC-AUC of 0.9139 across these datasets, with individual results reported in the table. This supports the ability of the regression-pretrained adapters to capture information useful for classification.

The table displayed below shows the calssification based synthetically generated datasets.

DatasetMoleculesROC-AUCQuery % positive
011270.998796.9%
110840.767015.2%
217860.951227.7%
314950.913336.5%
411780.930040.3%
510210.877285.4%
616890.976625.4%
716230.897383.4%
Mean0.913951.2%

We have included the experiment in the manuscript, please see section 5.2, end of "Evaluation results", Table A4.

AuthorsOpenReview ↗

Response to question 3

Question 3: Necessity of Interleaved Graph Processing

How much does repeatedly inserting graph adapters between LimiX Transformer blocks contribute to prediction performance? Would a simpler architecture that applies graph encoding only before PFN inference achieve comparable results?

We thank the reviewer for this important question. To directly investigate the necessity of interleaved graph processing and address Weakness 2, we have substantially extended our architectural ablation study. Specifically, we introduced a simpler architecture that retains the initial GNN and multi-view pooling but removes the interleaved graph adapters, encoding each molecule only once before inference with the frozen LimiX backbone. We additionally evaluated simplified message-passing and pooling configurations, resulting in six architectural ablations summarized in Table A9 of the revised manuscript.

Molecular-property prediction. Across nine benchmarks and three input configurations (atom features alone or combined with Mole-BERT or MolDeBERTa embeddings), removing interleaved adapters yields broadly comparable performance to the full MolPFN architecture, with generally modest, dataset-dependent differences and no consistent advantage for either approach (Tables A10–A12).

The additional ablations further clarify the role of individual architectural components. Simplifying message passing (variants A and B in Table A9) has relatively little impact on performance. For graph-level pooling, retaining mean, min, and max aggregation while removing attention largely preserves performance (variant E), whereas mean-only or attention-only pooling (variants C and D) generally performs worse, particularly on regression benchmarks (Tables A10–A12). These results highlight the benefit of combining complementary pooling operations, while suggesting that attention pooling is not essential for conventional molecular-property prediction.

FS-Mol few-shot prediction. The contribution of interleaving becomes more apparent when generalizing to previously unseen biological assays. Without assay-level meta-training, full MolPFN generally performs better than the encode-once variant, particularly with larger support sets, although the differences remain modest (Figure A2 left, Table A14). After meta-training across FS-Mol assays, removing interleaved adapters reduces ΔAUPRC by 0.054–0.084 across all five support sizes (Figure A2 right, Table A15). Further simplifications of message passing and pooling do not recover the performance of the full architecture, with removing message passing leading to additional deterioration.

Together, these results demonstrate that the contribution of interleaved graph processing depends on the prediction setting. For conventional molecular-property benchmarks, the synthetic prior may already provide a sufficiently strong learning signal for effective prediction using initial graph encoding alone. Our prior sensitivity ablations discussed below (Appendix E) support this interpretation: simplified architectures retain strong performance under the same graph-dependent synthetic prior, whereas disrupting the SCM-based feature–target relationship substantially degrades performance even with the full architecture. In contrast, interleaving provides a clear advantage for few-shot generalization to unseen assays, particularly after meta-training. This may reflect the ability to progressively refine molecular representations within the PFN using task-specific information from labelled support examples.

We have incorporated these additional experiments and their discussion into the revised manuscript (Appendix D, Tables A9–A14 and Figure A2), sections 5.2-5.3, subsections Architectural ablations and Evaluation results respectively, as well as section 6, fourth paragraph.

AuthorsOpenReview ↗

Response to question 4

Question 4: Sensitivity to Synthetic Data Generation.

How sensitive is the performance of MolPFN to the design of the synthetic graph-level prior? In particular, how do different graph generation strategies or SCM-based label generation mechanisms affect downstream prediction performance?

We thank the reviewer for raising this question. We investigated sensitivity to the synthetic prior through two complementary ablations: (i) changing the graph-generation strategy while retaining the SCM mechanism, and (ii) removing graph dependence from SCM-based feature and label generation. Across these experiments, we retained the MolPFN architecture and optimization procedure and selected checkpoints using ZINC validation performance.

1. Sensitivity to graph-generation strategy (Tables A16-A17). We compared the molecular-skeleton prior with two alternative graph families: random trees, which contain no rings or chemical valence constraints, and degree-corrected stochastic block models (SBMs), which generate heterogeneous connectivity and community structure without molecular valence constraints or calibrated ring-size preferences. The SCM-based feature and label generation mechanism remained unchanged.

All three priors achieve broadly comparable downstream performance, with modest, task-dependent differences and no consistently superior topology. Without a molecular encoder, the tree prior improves six of nine benchmarks, including ESOL (RMSE: 1.114 → 1.061), FreeSolv (RMSE: 2.130 → 2.024), and BACE (ROC-AUC: 0.827 → 0.839). Conversely, the baseline performs better on ZINC (MAE: 0.316 versus 0.319), ClinTox (ROC-AUC: 0.936 versus 0.918), and SIDER (ROC-AUC: 0.660 versus 0.652). The SBM prior similarly improves ESOL (RMSE: 1.057) but yields larger degradations on ZINC (MAE: 0.343) and ClinTox (ROC-AUC: 0.904). The preferred topology also depends on the molecular embeddings: trees achieve the best mean rank without an encoder and with Mole-BERT, whereas SBM achieves the best mean rank with MolDeBERTa (Table A16). These findings demonstrate that MolPFN transfers effectively across different synthetic graph families without requiring a particular molecularly calibrated topology.

2. Sensitivity to graph dependence in the SCM (Tables A18-A19). We constructed an edge-independent, purely tabular variant by disabling SCM message passing, removing degree and PageRank inputs, and assigning the virtual readout node its own non-zero sampled cause. This variant performs worse than the baseline in all 27 comparisons across nine datasets and three prediction pipelines. Without a molecular encoder, ZINC MAE increases from 0.316 to 1.583, and BACE ROC-AUC decreases from 0.827 to 0.558. Adding molecular embeddings does not eliminate the gap.

This comparison requires a specific interpretation: with these changes, the virtual-node label depends only on its own cause and is no longer predictable from atom features. Thus, the experiment demonstrates the importance of preserving a learnable relationship between node features and graph-level labels in our SCM construction, although it does not establish that graph-based SCMs outperform every possible tabular label-generation mechanism or isolate the contribution of each modified setting.

Together, these findings support flexibility in the choice of synthetic graph topology, but a strong dependence on a learnable graph-level supervision mechanism. Full experimental results are provided in Appendix E (Tables A16–A19). We have also amended the main text of the manuscript accordingly at the end of the sections 4.1 and 4.2, section 5.2 "Prior sensitivity", section 6, fourth paragraph.

Reviewer_53SeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission introduces MolPFN, a graph-level Prior-Data Fitted Network for molecular property prediction. MolPFN uses a frozen LimiX backbone, converts atomic features and molecular topology into Transformer tokens through molecular tokenization, and inserts graph adapters between Transformer blocks to enable repeated interaction between contextual information and atom representations. The authors also design a graph-level synthetic prior to train the added graph modules.

Critical strengths
  • Molecular prediction tasks often have limited labeled data, and adapting to new tasks through in-context learning without parameter updates is practically relevant.
  • MolPFN incorporates a purpose-built design for graph-level inputs. The combination of multiple pooling strategies, persistently maintained atom states, and layer-wise interaction between the virtual-node token and LimiX forms a relatively comprehensive graph-level adaptation framework.
  • The graph adapters trained only on synthetic regression tasks can be transferred directly to classification. Although the classification results are not consistently superior, this cross-task transfer is still an interesting finding.
Critical weaknesses
  • MolPFN largely combines the frozen LimiX backbone with the attention-based graph adapters introduced by GraphPFN. The main extension is adapting this framework from node-level to graph-level prediction through molecular tokenization and pooling.
  • The paper does not isolate the benefits of molecular tokenization, multi-view pooling, virtual-node interaction, or interleaved graph adapters. It is therefore difficult to determine which proposed component provides the improvement.
  • The graph-aware SCM follows GraphPFN, while the main molecular modification is the use of valence-aware trees and ring-closing edges. Comparisons with simpler graph priors are missing, so the value of this molecular prior remains unclear.
  • The experiments do not compare MolPFN with a simpler approach that encodes each molecule once and applies a frozen PFN to the resulting representation. Such a comparison is necessary to justify the proposed repeated graph–Transformer interaction.
Questions for authors
  1. How much does performance decrease when the layer-wise graph adapters are removed while retaining the input GNN and pooling?
  2. Does the improvement on ZINC remain after removing descriptors closely related to logP, such as the Crippen logP contribution?
5 comments
AuthorsOpenReview ↗

Response to weakness 3

We appreciate the reviewer's thoughtful assessment and their recognition of the strengths of our graph-level adaptation framework. Their comments motivated us to investigate the contributions of individual architectural components and the sensitivity to synthetic-prior design more systematically. We have also examined the role of the Crippen logP descriptor in the ZINC results. Our findings and the corresponding manuscript revisions are detailed below.

Reviewer Note (Weakness 3): The graph-aware SCM follows GraphPFN, while the main molecular modification is the use of valence-aware trees and ring-closing edges. Comparisons with simpler graph priors are missing, so the value of this molecular prior remains unclear.

We thank the reviewer for highlighting the need to compare our molecular-skeleton prior with simpler graph priors. We would also like to clarify that, while our prior builds on the graph-aware SCM framework of GraphPFN, its adaptation to molecular property prediction involves two modifications: a valence-aware graph generator and a graph-level target-generation mechanism. Unlike GraphPFN, which generates node-level targets, our prior uses a virtual readout node connected to all atoms to produce one target per molecular graph. To assess the importance of different components of the prior, we have added two sets of ablations in Appendix E.

Graph topology (Table A16). We compare our valence-aware molecular-skeleton generator with random-tree and degree-corrected stochastic block model (SBM) priors, keeping the SCM-based feature and label generation, MolPFN architecture, and optimization procedure fixed. Unlike our molecular prior, these alternatives do not impose chemical valence constraints or calibrated ring-size preferences. The results show task-dependent sensitivity, with no consistently superior topology. With atom features alone, the random-tree prior performs better on six of nine benchmarks, while the molecular-skeleton prior performs better on ZINC, ClinTox, and SIDER. Thus, molecularly calibrated topology is not necessary for competitive performance across the evaluated tasks.

SCM construction (Table A18). We additionally compare the baseline against an edge-independent variant, that removes SCM message passing and structural node attributes (degree and PageRank), while assigning the virtual readout node its own sampled cause. The MolPFN architecture and optimization remain unchanged. This variant performs substantially worse across all nine datasets and three input configurations. However, these modifications also disrupt the intended relationship between atom features and graph-level labels. Therefore, this comparison demonstrates the importance of preserving a meaningful feature-to-label relationship in the synthetic graph-level prediction tasks via the addition of the virtual node and GNN operations to the SCM.

Together, these experiments demonstrate the robustness of synthetic pretraining across different graph families and highlight the importance of meaningful SCM-based feature–label relationships, while showing that the benefit of molecularly calibrated topology depends on the downstream task. Full experimental details are provided in Appendix E. We have amended the main text of the manuscript accordingly at the end of the sections 4.1 and 4.2, section 5.2 Prior sensitivity, section 6, fourth paragraph.

AuthorsOpenReview ↗

Response to question 1

Question 1: How much does performance decrease when the layer-wise graph adapters are removed while retaining the input GNN and pooling?

We thank the reviewer for raising this important point. To address the raised Weaknesses 2 and 4 and directly answer Question 1, we have substantially extended our architectural ablation study. First, we evaluated a simpler architecture that retains the initial GNN and multi-view pooling but removes the interleaved graph adapters, encoding each molecule only once before inference with the frozen LimiX backbone, as specifically suggested by the reviewer. We then investigated the contributions of individual architectural components through further simplifications of message passing and graph-level pooling, resulting in six architectural ablations summarized in Table A9 of the revised manuscript.

Our results reveal an important distinction between conventional molecular-property prediction and few-shot prediction on FS-Mol, where the model must generalize to previously unseen biological assays from limited labelled examples.

Conventional molecular-property prediction. Across nine benchmarks and three input configurations (atom features alone or combined with Mole-BERT or MolDeBERTa embeddings), removing interleaved graph adapters does not consistently reduce predictive performance. Differences are generally modest and dataset-dependent, with no systematic advantage for either architecture (Tables A10–A12).

The additional architectural ablations provide further insight into the contribution of individual components. Simplifying message passing (variants A and B in Table A8) has relatively little impact on predictive performance. For graph-level pooling, retaining mean, min, and max aggregation while removing attention largely preserves performance (variant E), whereas mean-only or attention-only pooling (variants C and D) generally performs worse, particularly on regression benchmarks (Tables A9–A11). These findings highlight the benefit of combining complementary pooling operations, while suggesting that attention pooling is not essential for conventional molecular-property prediction.

FS-Mol few-shot prediction. Without assay-level meta-training, the full MolPFN generally outperforms the architecture without interleaved adapters, particularly at larger support sizes, although the differences remain modest (Figure A2, Table A13). After meta-training across FS-Mol assays, however, the contribution of interleaving becomes substantially more pronounced: removing interleaved adapters reduces ΔAUPRC by 0.054–0.084 across all five support sizes (Figure A3). Further simplifications of message passing and pooling do not recover the performance of the full architecture, with removing message passing leading to additional deterioration.

Together, these experiments demonstrate that the contribution of interleaved graph adapters depends on the prediction setting. On conventional molecular-property benchmarks, the synthetic prior may already provide a sufficiently strong learning signal for effective graph-level prediction using initial graph encoding alone, with limited additional benefit from interleaving. This interpretation is supported by our prior sensitivity ablations (Appendix E; see also our response to Weakness 3): simplified architectures retain strong performance under the same graph-dependent synthetic prior, whereas disrupting the SCM-based feature–target relationship substantially degrades performance even with the full architecture (Table A18). In contrast, interleaved graph processing provides a clearer advantage for few-shot generalization to unseen assays, particularly after meta-training. One possible explanation is that interleaving enables progressive refinement of molecular representations within the PFN, potentially allowing the model to better exploit task-specific information from labelled support examples.

Beyond clarifying the contribution of individual architectural components, these experiments establish the practical relevance of extending graph–Transformer interaction to graph-level in-context learning, particularly for few-shot transfer to unseen molecular assays.

We have incorporated these additional architectural ablations and their discussion into the revised manuscript (Tables A9–A14 and Figures A2–A3), sections 5.2-5.3, subsections Architectural ablations and Evaluation results respectively, as well as section 6, fourth paragraph.

AuthorsOpenReview ↗

Response to question 2

Question 2: Does the improvement on ZINC remain after removing descriptors closely related to logP, such as the Crippen logP contribution?

We thank the reviewer for raising this point. Removing the Crippen logP contribution from the atom features (leaving 39 features) increases ZINC MAE from 0.316 to 0.353 for the atom-features-only configuration, which still outperforms the evaluated tabular foundation models paired with pretrained molecular encoders. With auxiliary molecular embeddings, the effect is smaller: MAE increases from 0.343 to 0.353 with Mole-BERT and remains unchanged at 0.275 with MolDeBERTa (Table A8). Thus, while the descriptor contributes to performance with atom features alone, MolPFN's strong ZINC results do not depend on its inclusion.

Reviewer_pywDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MolPFN, a framework for graph-level molecular property prediction using in-context learning. The method extends a frozen LimiX backbone with trainable graph adapters and molecular tokenization, allowing it to incorporate molecular structure without task-specific fine-tuning. The authors also propose a synthetic graph prior for pretraining and further evaluate meta-training on FS-Mol. Experiments show competitive performance against tabular foundation models and several few-shot learning baselines.

Critical strengths
  • The integration of molecular graph representations into a pretrained tabular foundation model is an interesting direction. In particular, the ability to make predictions on new molecular tasks without task-specific parameter updates is practically relevant for low-data applications.
  • The method achieves competitive performance on several molecular benchmarks, particularly ZINC, and demonstrates meaningful improvements after meta-training on FS-Mol. These results suggest that graph-aware adaptations of tabular foundation models are worth investigating further.
Critical weaknesses
  • The proposed method closely follows GraphPFN, reusing its graph adapters and the general idea of graph-aware synthetic pretraining. The main differences lie in molecular tokenization, graph-level pooling, and the molecular prior. However, the paper does not provide sufficient ablations to establish the individual contributions of these components. In particular, comparisons with simpler graph-level adaptations of GraphPFN would help clarify the methodological novelty. This is my main concern regarding the paper's contribution.
  • The conventional molecular benchmarks mainly compare against tabular foundation models using pretrained molecular embeddings, without strong supervised molecular GNN baselines. Furthermore, the appendix indicates that ZINC validation labels are used for checkpoint selection, which complicates the interpretation of synthetic-only pretraining results on ZINC. In the FS-Mol experiments, the comparison between meta-trained MolPFN and frozen tabular foundation models also does not isolate the benefits of the proposed architecture from those of additional task-specific meta-training. These issues make it difficult to determine how much of the reported improvement comes from the proposed method itself.
Questions for authors
  • Could the authors provide ablations isolating the effects of molecular pooling, graph adapters, and the synthetic graph prior? In particular, how would MolPFN compare against a simpler graph-level adaptation of GraphPFN?
  • Appendix A.1 states that ZINC validation data are used for selecting the synthetically pretrained model. How much does this affect the reported ZINC performance? Would the conclusions remain unchanged if model selection relied exclusively on synthetic validation tasks?
  • Could the authors clarify why supervised molecular GNN baselines are not included in the conventional benchmark evaluation? Also, can the FS-Mol improvements be separated into gains from meta-training and gains from the proposed architecture?
6 comments
AuthorsOpenReview ↗

Response to question 1

We thank the reviewer for their constructive feedback and for recognizing the potential of our approach. To address the concerns raised, we have substantially extended our evaluation with additional architectural and prior ablations, checkpoint-selection analyses, and discussed the comparison with established molecular prediction methods. We address each question below.

Question 1: How much does repeatedly inserting graph adapters between LimiX Transformer blocks contribute to prediction performance? Would a simpler architecture that applies graph encoding only before PFN inference achieve comparable results?

Reviewer raises an imortant point. We therefore assessed the contribution of interleaved graph adapters by comparing full MolPFN with a simpler architecture that performs graph encoding and pooling only before LimiX inference. We then explored further simplifications of this architecture, including reduced message passing and ablations on the pooling strategies amounting to 6 additional ablations listed in Table A9 of the revised manuscript.

Our results show that the contribution of interleaved graph adapters differs between conventional molecular-property prediction and few-shot prediction on FS-Mol, where the model must generalize to previously unseen biological assays from limited labelled examples.

Molecular-property prediction. In this case, removing interleaved graph adapters did not consistently reduce predictive performance as compared to the full MolPFN architecture. Across nine molecular-property benchmarks and three input configurations (atom features alone or combined with Mole-BERT or MolDeBERTa embeddings), the performance differences were generally modest and vary across datasets, without a consistent advantage for either architecture (please see the Tables A10–A12 in the revised manuscript).

Further architectural simplifications reveal that reducing message passing (A and B in Table A9) has relatively little impact on predictive performance . For graph-level pooling, retaining mean, min, and max operations while removing attention largely preserves performance (E), whereas using mean-only (C ) or attention-only (D) pooling leads to substantially worse results, particularly on regression benchmarks (Tables A10–A12). This suggests that combining complementary statistical aggregation operations is sufficient for these tasks.

FS-Mol few-shot prediction. Even without FS-Mol meta-training, the full MolPFN generally outperforms the architectures without interleaved graph adapters, particularly at larger support sizes, although the differences remain modest (Figure A2 left, Table A14). After assay-level meta-training, this advantage becomes substantially more pronounced: removing interleaved adapters reduces ΔAUPRC by 0.054–0.084 across all five support sizes (Figure A2 right, Table A15). Further architectural simplifications do not recover the performance of the full model, with removing message passing leading to additional deterioration.

Together with the molecular-property benchmark results, these findings suggest that initial graph encoding is largely sufficient for conventional prediction tasks, whereas interleaved graph processing becomes more valuable for in-context prediction across diverse assays. One possible explanation is that interleaving allows molecular representations to be progressively refined within the PFN, potentially making better use of task-specific information from the labelled support set, particularly after meta-training.

Our prior ablations, investigating the roles of graph topology and SCM-based feature–target generation (Appendix E, Tables A16–A19), provide additional context for this distinction. Under the same graph-dependent synthetic prior, several simplified architectures retain strong performance on conventional benchmarks, suggesting that the prior provides transferable predictive information that does not necessarily require interleaved processing. Conversely, replacing the graph-dependent prior with the tabular-based variant substantially degrades performance even with the full architecture (Tables A18–A19). This supports the possibility that strong synthetic supervision reduces the need for interleaved processing in conventional prediction, while assay-level meta-training makes its benefits more pronounced.

We have added the corresponding descriptions and discussions to the revised manuscript, sections 5.2-5.3, subsections Architectural ablations and Evaluation results respectively, as well as section 6, fourth paragraph.

AuthorsOpenReview ↗

Response to question 2

Question 2: Appendix A.1 states that ZINC validation data are used for selecting the synthetically pretrained model. How much does this affect the reported ZINC performance? Would the conclusions remain unchanged if model selection relied exclusively on synthetic validation tasks?

We thank the reviewer for raising this concern. To assess the sensitivity to checkpoint selection, we evaluated two alternatives: (i) the last checkpoint from the original pretraining run and (ii) a checkpoint selected exclusively on eight synthetic validation tasks using mean R² as the selection criterion and no ZINC validation data and report our findings in the Appendix A and Table A1 of the revised manuscript.

ZINC performance. Using the last checkpoint improves ZINC test MAE by 2.5–3.8% across all three input configurations compared with the originally reported results, demonstrating that strong performance is achievable without ZINC-based checkpoint selection. Synthetic-only selection yields MAEs of 0.335, 0.373, and 0.320 for atom features alone, +Mole-BERT, and +MolDeBERTa, respectively. Importantly, all three results outperform the strongest tabular PFN baseline in Table 1 (MAE 0.457), confirming that our main ZINC conclusion holds even with exclusively synthetic checkpoint selection.

Transfer to other datasets. Across the remaining eight downstream datasets, synthetic-only selection yields broadly comparable performance to ZINC-based selection, with generally modest differences and no systematic deterioration: 12 of the 24 dataset–configuration results improve and 12 worsen (Table A1). These findings further support the robustness of synthetic pretraining across molecular prediction tasks.

AuthorsOpenReview ↗

Response to question 3

Question 3 (part 1): Could the authors clarify why supervised molecular GNN baselines are not included in the conventional benchmark evaluation?

We thank the reviewer for this suggestion. Our original evaluation focused on methods that can predict previously unseen molecular properties without task-specific parameter updates, reflecting the central setting addressed by MolPFN. Accordingly, we compared against tabular in-context learners and nearest-neighbour methods using pretrained molecular representations. Conventional supervised GNNs require separate optimization for each target dataset and therefore follow a different learning protocol.

Nevertheless, we agree that they provide an important reference for contextualizing MolPFN's performance. We have therefore expanded the section 5.2 in the revised manuscript to discuss the comparison with the established architectures (subsection "Performance in the context of supervised molecular GNNs"), including MPNN, D-MPNN, CMPNN, and AttentiveFP, as reported in published studies such as PharmHGT (2023; 10.1038/s42004-023-00857-x) and MvMRL (2024; 10.1093/bib/bbae298).

These published results show that MolPFN performs favorably against several supervised GNNs on individual benchmarks. On scaffold-split classification tasks, MolPFN outperforms the D-MPNN results reported in MvMRL on BACE (ROC-AUC: 82.7 vs. 79.8), ClinTox (93.6 vs. 85.7), and SIDER (66.0 vs. 56.8), although D-MPNN performs better on BBBP (86.7 vs. 74.3). On regression benchmarks, MolPFN achieves performance comparable to several established supervised GNNs. For example, the scaffold-split results reported in PharmHGT range from 1.833 to 2.430 RMSE on FreeSolv and from 0.845 to 1.541 on ESOL for MPNN, D-MPNN, CMPNN, and AttentiveFP. MolPFN achieves 2.130 and 1.114, respectively, outperforming MPNN on both datasets and D-MPNN on FreeSolv. On Lipophilicity, MolPFN achieves 0.760, compared with 0.658–0.730 for these supervised models. More specialized architectures achieve lower errors on some tasks.

These literature-based results should be interpreted with consideration of the evaluation protocols. MolPFN uses scaffold-based splits for the MoleculeNet benchmarks, whereas published supervised GNN evaluations employ both scaffold and random splits. Random splits generally present a less challenging generalization setting, and even scaffold-based evaluations may use different partitions. Thus, the reported results provide useful performance references rather than strictly controlled head-to-head comparisons.

Importantly, supervised GNNs are optimized separately for each molecular prediction task, whereas MolPFN predicts using labelled molecules as context without task-specific gradient updates. We have clarified this distinction and incorporated the discussion of supervised GNN performance and splitting protocols into the revised manuscript, section 5.2, Performance in the context of supervised molecular GNNs. Our contribution is therefore not to replace specialized supervised molecular predictors, but to enable competitive molecular graph prediction within a general in-context learning framework.

Question 3 (part 2): Also, can the FS-Mol improvements be separated into gains from meta-training and gains from the proposed architecture?

We thank the Reviewer for raising this issue. As detailed in our response to Question 1, our additional ablations help distinguish the benefits of meta-training from those of interleaved graph processing. Without FS-Mol meta-training, the full MolPFN shows modest improvements over the encode-once architecture (Figure A2 left, Table A14). After meta-training, this advantage becomes substantially larger, with removing interleaved adapters reducing ΔAUPRC by 0.054–0.084 across all five support sizes (Figure A2 right, Table A15). These results indicate that the FS-Mol improvements cannot be attributed to meta-training alone, but also reflect the contribution of the proposed architecture.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of performed experiments and revisions

We thank all reviewers for their constructive feedback and valuable suggestions, which have helped us strengthen the manuscript. In response, we have substantially expanded the experimental analysis and revised the manuscript to clarify the methodological contributions, experimental findings, and limitations of MolPFN.

In particular, we have added architectural ablations investigating the roles of interleaved graph processing, message passing, and molecular pooling, sensitivity analyses of the synthetic graph prior, including alternative topology generators and SCM configurationsm, additional experiments examining checkpoint selection, molecular descriptors, and classification transfer. We have also expanded the discussion of supervised molecular GNNs to better contextualize our results and clarified the interpretation of the ablation findings, particularly the setting-dependent benefits of interleaved graph processing, and updated the abstract and the manuscript to reflect these findings.

All corresponding changes have been incorporated into the revised manuscript. Revisions are highlighted in yellow. For entirely new sections (Appendix D, E), only the heading is highlighted in yellow to preserve readability, rather than highlighting the full text. Detailed responses to each reviewer, including references to the relevant sections, tables, and figures, are provided below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.