- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies feature preprocessing for graph foundation models (GFMs), where graphs may have different feature dimensions and distributions. The authors propose SoLIFT, which uses information from source graphs to guide feature preprocessing on target graphs, rather than applying PCA or SVD to each graph independently. The framework includes two designs SPLIT and FUSION. SPLIT combines source consensus with graph-local residual directions, while FUSION jointly selects source-informed and local feature directions, followed by Procrustes alignment. Experiments across seven GFM pipelines show that FUSION improves the average macro-F1 by 1.97 percentage points over exact SVD.
Critical strengths
- The paper addresses a practical problem in graph foundation models: how to handle node features with different dimensions and distributions across graphs. Using information from source graphs rather than preprocessing each graph independently is a reasonable direction.
- SoLIFT can be used as a feature preprocessing interface without changing the main architecture of the GFM encoder. This makes it relatively easy to integrate into different models.
- The experiments cover seven GFM pipelines, providing a relatively broad evaluation across different architectures. FUSION improves the average macro-F1 from 45.75% with exact SVD to 47.72%, with positive average gains across all seven pipelines.
- The paper provides theoretical analysis of the proposed spectral objectives and discusses several limitations, including the assumption of positional feature correspondence and the differences in performance across target datasets.
Critical weaknesses
- SoLIFT mainly builds on established techniques, combining source consensus with residual PCA. Although the paper cites relevant prior work, the technical novelty over existing shared/private subspace learning and projector aggregation methods remains unclear. In particular, given that the theoretical results mainly characterize the proposed spectral objectives, does the novelty lie in a new mathematical method, or in combining existing techniques and applying them to GFM feature interfaces?
- The paper aims to identify recurring feature directions across different graphs, but node attributes from different graphs may differ in dimensionality, origin, and even semantics. SoLIFT maps them into a common numerical coordinate space without establishing semantic correspondence between the features. Moreover, the paper does not evaluate the impact of this assumption through experiments with independent feature-column permutations across graphs.
- Some GFM baselines use different pretraining configurations and modified training protocols. For example, Appendix B.4 shows that GraphGlue uses a shorter, fast training protocol, while other baselines adopt different pretraining configurations. This limits the comparability of performance results across different GFM pipelines. The paper should clarify that the paired comparisons within each pipeline are the main evidence and provide a unified table summarizing training resources and key hyperparameters.
- Figure 3 presents t-SNE visualizations of different preprocessing methods on BRIDGE–Cora, but this is a single retrospectively selected case. It cannot be used to directly compare the global representation geometry of different methods quantitatively. The quantitative neighborhood-consistency statistics should be given more emphasis, and the key experimental settings should be included in the figure caption.
Questions for authors
- Once FUSION has already selected the feature subspace, why is Procrustes alignment still needed to rotate the basis of the selected subspace? Does this rotation actually improve the transfer performance of GFMs? What is the theoretical justification for this step?
- Why are the local subspace and source consensus always assigned equal weights when the relationship between source and target graphs may vary? Could this fixed design lead to negative transfer? The authors explain that both terms have directional support values in the range [0,1], which motivates using equal coefficients. However, having the same numerical range does not imply the same predictive reliability. A more detailed explanation of this design choice is needed.
- Although the paper provides relatively extensive performance comparisons across seven GFM pipelines, the existing experiments mainly focus on the final classification performance of different feature preprocessing methods. A more comprehensive ablation study of coordinate orientation is needed to evaluate its actual contribution across different GFM architectures and whether its effectiveness is consistent. In addition, a sensitivity analysis of the source–local fusion weights would help assess the reasonableness of the fixed equal-weight design in FUSION and its impact on model performance.
4 comments
Q2 / Q3(b). Equal weights and the possibility of negative transfer
We agree that equal numerical range does not imply equal predictive reliability. Both directional-support terms lie in [0,1], making λ=1 a fixed reference at a comparable numerical scale. Source informativeness can still differ across targets.
We varied λ over {0, 0.25, 1, 4} with orientation fixed. Relative to λ=1, changes at λ=0/0.25/4 are −0.58/−0.23/+0.59 pp in AnyGraph and +0.04/+0.05/−2.89 pp in GraphFM. AnyGraph shows mixed target effects; GraphFM declines on all five targets at λ=4. Stronger emphasis on source evidence can therefore hurt transfer, and its effect depends on the pipeline and target. We retain λ=1 and use the sweep to characterize this tradeoff. Appendix D.4.2 gives results at each weight for every target.
W2. Feature correspondence and independent column permutations
We have added theory distinguishing coordinate changes from correspondence changes. SoLIFT uses positional correspondence, and Procrustes does not infer feature semantics. Under exact spectral operators and strict selection gaps, a common permutation of the embedded feature coordinates preserves Fusion's node Gram matrix. For independent permutations, the bound relates subspace and Gram changes to consensus mismatch ε and fused eigengap δ: when ε<δ/2, the projector change is bounded by 2ε/δ. This is a conditional geometric guarantee; arbitrary independent permutations can alter correspondence and do not guarantee predictor invariance (Appendix H).
We tested source-only, target-only and combined permutations on five targets at 5-shot, using three seeds, three draws and paired episodes. Target-only tests reused the source checkpoints; source-only and combined tests refitted the interface and retrained the backbone. Fusion's signed mean changes are −0.12/+0.39/−0.33 pp in AnyGraph and −0.28/−0.59/+0.21 pp in GraphFM. All are within 0.6 pp, while its lead over matched SVD remains +5.45 to +6.38 pp and +2.00 to +2.80 pp, respectively. This supports average predictive robustness, alongside target variation: target-only permutation changes AnyGraph by +4.26 pp on Computers and −3.11 pp on Physics, and GraphFM by −3.32 pp on Computers. The experiment complements the conditional theory without establishing its small-mismatch premise for every graph (Appendix F.3).
W3. Within-pipeline comparisons, training policies and SVD correction
The primary evidence is the paired interface comparison within each pipeline. Each pair shares source graphs, training budget, checkpoint-selection policy, target adaptation and support/query episodes. GraphGlue's five-epoch schedule applies to both interfaces. Table 3 summarizes budgets, model selection and adaptation. Appendix B.4 describes implementation choices, and Appendix G gives computational resources. Cross-pipeline results assess whether the interface effect recurs across architectures, rather than rank the published systems.
With the corrected SVD implementation and matched re-evaluations, the mean Fusion–SVD gap is +2.64 pp, compared with +1.97 pp in the original submission. GraphGlue accounts for most of this change: its SVD mean changes from 37.04% to 32.52%, while Fusion remains at 37.47%. Fusion improves mean macro-F1 in all seven pipelines and, after averaging pipelines, at every shot budget. Accuracy, the original primary metric, also improves by +3.03 pp. Appendices C.1 and E provide the paired results and numerical implementation details.
W4. Quantitative neighborhood agreement and the role of t-SNE
We agree that independent t-SNE panels cannot quantify global geometry. We instead emphasize neighborhood agreement in the normalized 256-dimensional classifier-input space, before t-SNE: the same-class fraction among ten nearest query neighbors, averaged within class and then equally across classes. Fusion exceeds SVD in all three BRIDGE–Cora seeds, with mean agreement of 65.9% versus 62.1%.
This diagnostic uses the source-selected checkpoints from the original submission, including its original SVD implementation. Figure 3 now identifies the retrospectively selected 5-shot, seed-0 case and its 681 shared queries. The panels illustrate local organization; 10-NN quantifies consistency in the original representation space. This selected case concerns the complete trained model, rather than an isolated component or global geometry (Appendix D.6).
Response
I thank the authors for their detailed and thoughtful rebuttal, which effectively addresses most of my concerns and provides valuable clarifications. I will maintain my score.
We appreciate your careful consideration of our rebuttal and your acknowledgment that it addresses most of your concerns. In light of the additional evidence and clarifications, we would be grateful if you could consider whether the revised manuscript merits a more positive assessment and a higher score.
We would also appreciate knowing which remaining concern is most important to your assessment, so that we can provide a focused clarification.
Thanks for your comment!
We appreciate your careful assessment of SoLIFT's practical relevance and your detailed questions about its contribution and component effects. We have added controlled studies and clarified how the theory, cross-pipeline comparisons and representation analysis support our conclusions.
W1. Contribution beyond shared/private subspace learning
The contribution is a source–local GFM interface, its deployment protocol and analysis. SoLIFT fits spectral source evidence before backbone training, then combines it with each unseen graph's local variation within a fixed input width. Split explicitly reserves common capacity and fills the remainder with residual local directions; Fusion jointly allocates the full width. Projector averaging, residual PCA and Procrustes are established ingredients; we now state this explicitly.
In Knowledge Transfer across Multiple PCA Studies (Li et al., Algorithm 1), the target participates in the shared-space estimate alongside informative sources, followed by target-private residual PCA. Personalized PCA (Shi and Al Kontar, JMLR 2024) jointly optimizes shared and local reconstruction components for participating clients. SoLIFT estimates its common state from sources alone and reuses it across unseen targets without refitting the source state. This operational distinction makes simpler source-informed estimators relevant controls.
Our analysis characterizes the source–local tradeoff. At equal weights and matched rank, Fusion's gain in consensus overlap is at least its loss in local-projector overlap (Proposition 4); Theorem 5 bounds subspace change when source coupling is small relative to the block gap. These results characterize geometry, not prediction accuracy. Empirically, Fusion exceeds balanced covariance PCA with local residuals (48/16) by +1.29/+1.44 pp in AnyGraph/GraphFM. Split minus that capacity-matched covariance control is −1.80/+0.28 pp, so projector aggregation alone does not explain the gains. The earlier three-pipeline consensus-only variant loses 15.13 pp relative to Fusion, supporting the importance of retaining local information (Section 2; Appendices D.4 and F.2).
Q1 / Q3(a). Why orient the basis, and does orientation improve prediction?
Orientation fixes the coordinates supplied to the GFM after subspace selection. Procrustes maps the selected basis toward a common source reference while preserving its projector and within-graph Gram matrix (Proposition 6). Different graph-specific rotations can nevertheless produce different outputs from a shared encoder. GraphFM's graph-specific input MLP can absorb rotations at the function-class level, although training can differ. Orientation therefore supplies a coordinate convention, whose predictive value is an empirical question.
Our new 2×2 study crosses λ=0/1 with orientation off/on in AnyGraph and GraphFM: five targets, three seeds and five shot budgets. At λ=0, we retain Fusion's local PCA frame directly; orientation-on still uses the source reference. The four arms use the same interface conventions.
Selection and orientation each raise the mean when added to the local, unoriented interface. Their conditional gains shrink when the other component is present, producing the negative estimated interactions above. The two predictive effects therefore cannot be added independently. At λ=1, adding orientation also changes means by −0.39/−0.38/−1.52/−0.49 pp in MF-GIA/MDGFM/GraphGlue/BRIDGE; the earlier GraphLoRA study instead finds +1.16 pp. The six nonpositive mean effects have target-bootstrap intervals crossing zero. We therefore treat orientation as a common coordinate convention with architecture-dependent predictive effects, rather than a necessary source of Fusion's gains. The main configuration remains λ=1/on; these comparisons include source retraining (Appendix C.2, Tables 10 and 13, Appendix D.4.1).