- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents G2T‑FM, a lightweight framework for adapting tabular foundation models to graph node‑level tasks. Instead of modifying the internal tabular foundation model, it augments node representations with graph topology information via three components: neighborhood‑feature aggregation (NFA), hand‑crafted structural features, and learnable PEARL structural encodings, converting graph‑node prediction into a tabular learning problem. Built upon off‑the‑shelf TFMs such as TabPFNv2 and LimiX, it supports both in‑context‑learning (ICL) and finetuning (FT) modes. Evaluated on multiple graph benchmarks including GraphLand covering classification and regression, G2T‑FM outperforms existing graph foundation models under ICL, and its finetuned version frequently surpasses well‑tuned vanilla GNN baselines by considerable margins. Ablation studies verify the necessity of each graph‑aware augmentation component; even when GNNs and LightGBM are fed with identical augmented features, G2T‑FM retains performance advantages. This work opens a new direction for solving graph‑learning problems using tabular foundation models, though it is currently limited to node‑level tasks and constrained by inherent limits of the underlying tabular foundation models on sample size and class number.
Critical strengths
- Propose a novel cross-modal migration approach: Pioneer research directions for using table foundation models for graph learning, without training graph foundational models from scratch, reusing mature table large models such as TabPFNv2 and LimiX. The framework features plug-and-play functionality, and future iterative upgrades of table foundation models will benefit G2T-FM directly without large-scale changes.
- Modular decoupled design, no changes to the table model kernel. All graph topology information is placed in the pre-feature augmentation stage, completed by three modules: NFA neighborhood aggregation, manual structure features, and PEARL encoding. The table base model section retains the original weights and inference logic; Supports two usage paradigms: Context-learned ICL and fine-tuning FT, offering flexible usage methods.
- Complete and fair experimental comparisons, with rigorous baseline configuration. Selected datasets such as GraphLand, which closely reflect the real distribution of industries, while also covering classification and regression tasks; The comparison objects include both traditional fully tuned GNNs (GCN, GraphSAGE, GAT, GT) and multiple public graph foundation models; Traditional GNN baselines perform complete hyperparameter searches, adding residuals and layer normalization enhancement components to avoid inflated effects caused by weak baselines; At the same time, I compared it with my work tag and also tested my speed.
- Sufficient ablation and controlled experimental evidence of the source of gain. Item-by-item ablation of the three graph augmentation modules proves that each component contributes to performance; Controlled experiment: feeding the full set of augmented features used by G2T-FM to GNN and LightGBM, it was found that even with identical features, G2T-FM still had the advantage, proving that the benefits do not come solely from feature engineering, but from the synergy between the table base model and graph augmentation components; It also validates the effect gains brought by integrated reasoning.
- Clarify common misconceptions in evaluating existing graph foundation models. Experiments have found that many published foundation graph models perform worse than fully tuned traditional GNNs on datasets with a higher proportion of real and training samples; It points out that some literature confuses the terminology between in-context learning and zero-shot, providing a reference benchmark for subsequent evaluation of graph foundation models.
- Support for node return tasks. Most public graph foundation models focus only on node classification. This framework, relying on the native capabilities of table foundation models, naturally supports node regression tasks, fills gaps in graph foundation models in regression scenarios, and better meets the diverse needs of industrial scenarios.
Critical weaknesses
- Graph information is injected only through prerequisite feature engineering, lacking a native deep interaction mechanism for the graph. The framework itself does not modify the table base model; the graph topology information relies entirely on feature augmentation (NFA, structural features, PEARL) during the preprocessing stage, and there are no native graph learning modules such as message passing or graph attention within the model; The upper limit of modeling capability for long-distance, dependent, complex high-order graph relationships is limited by manual/preprocessed features. The paper also mentioned that the initial experiment with multi-hop NFA did not yield stable returns.
- Strong performance depends on the inherent constraints of the underlying table foundation model. Inheriting the limitations of TFM itself: for example, TabPFNv2 natively has restrictions on the number of classes and the size of training samples, which can restrict graph tasks with many classes and a large number of nodes; The framework itself cannot solve these problems and can only wait for upstream table base model iterations.
- Task coverage is narrow, only supporting node-level tasks. Currently, only node classification and node regression have been completed, with no adaptation for edge prediction or full-map classification/regression tasks; The paper also acknowledges that migrating to graph-level tasks is not easy, lacking corresponding solutions and experimental validation, and that there is still considerable room for generalizability.
- Feature dimension expansion, some scenarios require PCA dimensionality reduction, introducing information loss risks. After encoding the original node feature concatenation with neighborhood statistics, degrees, PageRank, Laplace feature vectors, and PEARL, the feature dimensions increase significantly; Datasets such as avazu-ctr and city-reviews must use PCA compression to compress feature dimensions to avoid OOM. PCA compression loses some fine-grained feature information and may harm downstream performance.
- High inference overhead, and integrated inference further amplifies costs. By default, 10 forward-propagation rounds are used for integration to improve effectiveness; From the perspective of appendix running times, single-round inference on some datasets consumes a very long time; Although faster than TAG at the same time, compared to native lightweight GNN and LightGBM, inference speed is less disadvantaged, and deployment on large scales is under greater pressure.
- Insufficient interpretability and controllability of the PEARL module. In the ICL mode, PEARL encoding is generated directly by randomly initializing GCN, without downstream graph data training, yet it can generate partial dataset benefits; The mechanism behind this phenomenon is not clearly explained; the superparameters (GCN layers, output dimensions, number of random initializations) lack systematic mesh search tuning.
- Cross-map zero-sample generalization capability not fully verified. The paper mainly evaluates under same image forwarding settings (the test set and training set belong to the same graph), lacking true cross-dataset zero-sample/few-shot experiments across different graphs; ICL is in-context learning within the same graph, not cross-graph migration, making it difficult to prove that the framework can directly transfer to completely unseen new graphs.
- Insufficient testing of heterogeneous and extreme graph structures. Although the GraphLand benchmark is used, systematic testing of extreme scenarios is lacking, such as extremely sparse graphs, highly heterogeneous plots, and large-scale power-law distribution maps; It is unclear whether the NFA + manual structure feature augmentation scheme can still work stably when graph topology significantly deviates from the existing test set distribution.
Questions for authors
-
G2T-FM relies entirely on feature augmentation injection during the preprocessing stage, and there is no native graph messaging mechanism within the model. Since the initial experiments with multi-hop NFA have not brought stable returns, will they consider embedding lightweight, learnable messaging modules within table foundation models in the future, rather than just at the feature engineering level to further capture high-level, long-distance graph dependencies?
-
All current experiments are based on forwarding context learning settings within the same graph, without true cross-graph zero-shot generalization across different graph datasets. How do you distinguish between "ICL capability within the same map" and cross-map migration capability? How do you plan to design experiments to verify the G2T-FM migration effect on completely unseen new maps?
-
Framework performance largely inherits the limitations of the underlying table base model, such as upper limits on the number of classes and training sample size. When faced with graph tasks with an extremely large number of nodes and many categories, what feasible strategies can alleviate the performance bottleneck caused by sample and class count without replacing the upstream TFM?
-
Some high-dimensional datasets must use PCA to reduce dimensionality of stitched augmented features to prevent memory overflow, which can cause information loss. Is there a comparison of how different dimensionality reduction schemes (such as feature selection or sparse projection) affect the final metrics? Is there a risk that PCA compression could damage key topological features, leading to significant performance degradation?
-
In the ICL mode, random initialization without training with PEARL encoding can still improve some dataset performance, but its internal mechanism still lacks explanation. PEARL's hyperparameters such as GCN layer count, output dimension, and random sampling frequency are only fixed configurations and do not perform a full mesh search. Will there be further ablation to clarify the conditions for PEARL to take effect and provide more general guidance for selecting superparameters?
5 comments
Official Comment (Part 2)
W7, Q2. First, we highlight that we clearly state in our paper that G2T-FM works in the ICL (or FT) setting, and all relevant experiments follow this setting. We argue that the ICL setting is established and widespread in both the graph domain and other domains, including the tabular domain. We do not claim that our model works in a zero-shot setting.
Second, we note that true zero-shot transfer requires access to meta-information, such as a description of the task and target space in natural language. Without such meta-information or any labels, a model would not know what label "0" means compared to label "1", making predictions impossible. While this setup is valid, it falls far beyond the scope of our work.
Some confusion may arise from previous GFMs incorrectly claiming to work in a zero-shot setting, thus suggesting that it is a standard protocol for GFM evaluation. However, as we discuss in Section 2.2, these methods actually have access to labels from the same graph and thus cannot be called zero-shot. In fact, these methods use labels of training nodes from the same graph as the test nodes, matching the evaluation protocol we use. Thus, we believe our evaluation protocol is appropriate and common for GFM evaluation.
Finally, we note that the underlying TFMs have never seen the graphs considered here during pretraining, and the G2T-FM components are training-free. Therefore, we believe that our main experiments already show transfer to completely unseen graphs.
W8. We acknowledge that G2T-FM, like any other model, may fail under extreme conditions. However, we believe that, at present, GraphLand is the graph benchmark closest to real-world applications, and thus we focus on this benchmark in our evaluations.
Best regards,
Authors
Dear Reviewer HjET,
Since the discussion period is ending soon, please let us know if our response addresses your concerns, and if you have any further comments.
Thank you for your time and effort in reviewing our paper.
Best regards,
Authors
Official Comment (Part 1)
Dear Reviewer HjET,
Thank you for your detailed feedback and for the time and effort in reviewing our work. We address your concerns and questions below.
W1, Q1. First, we note that there is a follow-up work, GraphPFN, that implements your idea of adding native graph message passing to a TFM. Second, we argue that the lack of pretrained graph layers is both an advantage and a disadvantage for G2T-FM. On the one hand, G2T-LimiX performs worse than GraphPFN, which is based on the LimiX backbone. On the other hand, G2T-FM's plug-and-play design allows it to easily use any existing or future TFM, and using a more recent TFM, Kumo-Tabular, allows G2T-FM to outperform GraphPFN. In contrast, GraphPFN lacks such a plug-and-play capability, and migrating GraphPFN to a new TFM backbone would require adapting MPNN layers to the new architecture and pretraining them from scratch. Thus, we believe that reliance on feature engineering is an important trait of G2T-FM, which allows it to remain a relevant baseline even as GFMs develop further. We refer to our general response for further details on more recent TFM backbones and the comparison with GraphPFN.
W2, Q3. First, we note that the TFM field is rapidly developing. For example, more recent models like TabPFN-3.5 claim to handle datasets with up to a million training samples or natively handle more than 10 classes.
Second, we note that at least some limitations can be alleviated by our method. Specifically, G2T-FM supports error-correcting output codes, a common technique to extend TFMs beyond their native class limit. Below, we provide a comparison of G2T-FM with ECOC against supervised GNNs. We see that it performs competitively, matching or outperforming GNNs on
lastfm-asiaandroman-empire, though we also observe a substantial drop onhm-categories.Third, one can potentially alleviate a TFM's limits on the number of training samples or features by subsampling them, though we would expect a considerable drop in performance in this case. Since scaling to larger datasets is not specific to graphs, we believe exploring such approaches is outside the scope of this work.
W3. We clearly acknowledge this limitation in the Limitations section.
W4, Q4. We acknowledge that PCA may lead to a loss of information, though we note that the original features and NFA are compressed, while all other features (PEARL embeddings, LapPE, etc.) are not compressed because they have a fixed, relatively low dimension. Since PCA is a temporary workaround rather than a principled component of G2T-FM, providing a detailed analysis of different compression techniques is beyond the scope of our work, and we believe that more scalable TFMs would reduce the need for such workarounds.
W5. Scalability and efficiency remain important directions for future work. However, we note that G2T-FM significantly outperforms all prior GFMs considered in this study, justifying the runtime and memory overhead.
W6, Q5. First, we note that we deliberately do not optimize PEARL hyperparameters. Optimizing them for each dataset would substantially increase the runtime of our method and introduce a tuning stage that we try to avoid in the ICL setting. Optimizing them based on validation scores on a collection of datasets is possible, but would introduce a risk of overfitting to these datasets and would limit our ability to evaluate transfer to fully unseen graph datasets.
Second, we note that message passing already introduces certain inductive biases into a model. Thus, while this phenomenon is indeed interesting, it is not completely surprising. For example, even the original GCN paper (Kipf and Welling, 2017) shows in Appendix A.1 that embeddings from a randomly initialized GCN could carry some signal.