- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies how depth is used in prior-fitted networks for node classification, mainly NodePFN (12 layers), with a repeat on GraphPFN. It borrows the mechanistic protocol of Balef et al. (2026) for tabular PFNs (CKA, probes, logit lens, layer skipping and self-repair) and adds branch-wise ablations, since each NodePFN layer has an attention branch and a message-passing branch. The main findings are:
- the first layer's attention carries the labels to the queries, and later attention is redundant and self-repairing;
- message passing matters mostly on homophilous graphs;
- so the depth at which the prediction is "ready" grows with homophily.
A two-class cSBM analysis is used to argue that no fixed depth is optimal across homophily levels. Based on this, the authors train a single NodePFN block that is looped (Looped NodePFN). The number of iterations is chosen per graph by holding out part of the labeled context (ICDS). With about a tenth of the parameters, the looped model gets close to the 12-layer model on average and is faster at prediction time once the depth has been selected.
Critical strengths
-
The mechanistic study is well designed. Zeroing one branch of one layer is a neat way to separate "refinement" from "propagation", and the self-repair experiment answers a question that plain ablation cannot.
-
The finding that the tuned pre-smoothing in the official NodePFN pipeline already does part of the propagation, and therefore hides the depth/homophily relation, is a nice and practically relevant observation. People evaluating graph PFNs should know about it.
-
The paper is very transparent. It has a threats-to-validity section, an explicit failure case for ICDS, oracle numbers shown separately and unranked, sensitivity tables for ICDS, released code and checkpoints, and a hypothesis stated before the experiments. This is good practice.
-
The negative control (a 1-layer model trained without looping collapses when looped at inference) is convincing and makes the point that looping must be trained for.
Critical weaknesses
My major concerns are:
- the method itself is quite simple (validation-based selection of one hyperparameter);
- the headline "no loss in accuracy" hides a clear loss on heterophilous graphs and large per-graph swings;
- the depth-follows-homophily story does not hold in the setting where the headline numbers are reported;
- the theory overlaps with known oversmoothing results on CSBMs that are not cited;
- the paper is not clearly positioned against existing low-cost GNN methods that already address homophily and over-squashing (e.g., DeltaGNN, spectral graph pruning, curvature- and community-based rewiring).
- The paper is hard to read because the text is overloaded with numbers and implementation details.
A detailed feedback is included below:
-
"No loss in accuracy" is only true on average.
- The looped models are better on homophilous graphs and worse on heterophilous ones, and the two roughly cancel in the overall mean. On heterophilous graphs the gap is about two points under ICDS, and the paper itself says even the oracle depth does not close it.
- The per-dataset table (Table 2) also shows very large swings in both directions.
- I would like the abstract and contributions to state the heterophilous loss clearly. The paper should also explain these outliers, in particular the below-chance result on questions.
-
The depth does not follow homophily in the headline setting.
- The main claim is that the right depth is a property of the graph and that ICDS picks it up. But under the standard protocol, the chosen depth is unrelated to homophily (the correlation is even slightly negative for Loop-1L).
- The relation only shows up without pre-smoothing, and there ICDS is below NodePFN-12L.
- So in the setting where the method matches the baseline, the motivation does not apply. In the setting where the motivation applies, the method does not match the baseline. The paper acknowledges this in Appendix G, but the main text and abstract read as if both hold at once.
-
ICDS is essentially validation-based model selection.
- The authors themselves describe ICDS as cross-validating one hyperparameter within the forward pass. That is a reasonable thing to do, but it is not much of a methodological contribution.
- Table 7 shows that a single global depth already recovers about half of ICDS's gain under the standard protocol. That global depth is chosen with test labels, so it is optimistic, but it shows how much of the gain comes simply from not using depth 12.
-
The efficiency claims need more context: fewer parameters, but not less compute.
- Training is not cheaper. The "10× fewer parameters" headline may suggest a cheaper model, but in floating-point operations pretraining Loop-1L costs practically the same as pretraining NodePFN-12L. The paper says so itself in Appendix E: a block applied twelve times costs as much computation as twelve layers. This needs to be clarified. What is the true gain here? This message should be consistent throughout the paper.
- Training memory drops only by about a quarter, not by ten times.
- The authors attribute the small remaining saving to updating fewer parameters, not to doing less computation.
- The savings are in storage only. The parameter reduction does reduce checkpoint size (12 MB against 116 MB). The ethics statement says looping reduces the memory needed "to store and train" a PFN; that holds for storage, but only marginally for training.
- Inference is not cheaper per iteration either. One looped iteration costs about the same as one layer. The prediction speed-up comes only from ICDS choosing fewer than twelve iterations. On the graphs where ICDS selects twelve or more iterations, there is no speed-up, and on two graphs the looped model is slower (Appendix E, "Exceptions").
- ICDS cost is excluded. The "up to 5.1 times faster" figure leaves out ICDS, which runs 20 × Tmax block applications once per graph. For a single prediction, Looped NodePFN is slower than NodePFN-12L.
- Suggested wording. The parameter reduction is real and useful for storage and deployment, but the abstract and contributions should say clearly that it does not reduce training or per-iteration compute. Any speed-up only applies when many predictions are made on the same graph.
- Training is not cheaper. The "10× fewer parameters" headline may suggest a cheaper model, but in floating-point operations pretraining Loop-1L costs practically the same as pretraining NodePFN-12L. The paper says so itself in Appendix E: a block applied twelve times costs as much computation as twelve layers. This needs to be clarified. What is the true gain here? This message should be consistent throughout the paper.
-
Proposition 1 is close to known results that are not cited.
- The idea that on a CSBM there is a finite optimal number of propagation steps, which depends on homophily and degree, and is zero when the graph signal is too weak, has been studied before. Examples are Wu et al. (ICLR 2023, "A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks") and Keriven (NeurIPS 2022, "Not too little, not too much: a theoretical analysis of graph (over)smoothing"). Baranwal et al. is cited, but only for the one-step case.
- Please position Proposition 1 against these works and say what is new.
- The proof is labeled a "proof sketch" and relies on several approximation steps. A full proof, or at least a clear list of what is rigorous and what is heuristic, would help.
- The result is also for two balanced classes, while most benchmarks have more. The multi-class case is only argued in a remark.
-
Positioning against cheap methods that already handle homophily and over-squashing.
The paper's central message is that how much a model should propagate depends on the graph's homophily, and that this can be decided cheaply per graph. That message is not new on its own. Several recent works study the same tension between homophily, over-smoothing and over-squashing, and propose solutions with low computational cost. Most of them are not cited or are only mentioned in passing:
- Spectral graph pruning (Jamadandi et al., NeurIPS 2024). Deleting edges can reduce over-squashing and over-smoothing at the same time, using a cheap spectral-gap criterion. It is evaluated on large heterophilous graphs.
- ComFy (Rubio-Madrigal et al., ICLR 2025). Rewiring guided by communities and feature similarity. It shows that the usual over-squashing fix (maximizing the spectral gap) hurts on homophilous graphs, which is close in spirit to this paper's "the right amount of propagation depends on h".
- DetlaGNN (e.g., Mancini et al., DeltaGNN TPAMI 2026). Oversmoothing & oversquashing detection and fixing across varied topologies.
- AFRC rewiring (Fesser & Weber, LoG 2023). A linear-time curvature measure that characterizes both over-smoothing and over-squashing, with heuristics that avoid expensive hyperparameter searches.
- Adaptive-depth and adaptive-propagation GNNs. Hevapathige et al. (AAAI 2026) select node-specific depths based on local homophily. GPR-GNN learns per-graph propagation weights. Both are cited, but only in one sentence, without discussion of how ICDS relates to them.
These methods differ from the paper because they are trained on the target graph or applied as preprocessing, whereas a PFN is not trained on the target graph. But that is exactly why they matter here. The paper itself shows that NodePFN's pre-smoothing already moves part of the propagation into preprocessing. A cheap, homophily-aware rewiring or edge-filtering step applied before NodePFN is therefore a natural alternative to looping plus depth selection, and should be discussed or tried as a baseline.
I would like the authors to state clearly what is unique to their work with respect to these methods. Is it:
- the mechanistic finding about NodePFN's branches;
- the observation that pre-smoothing hides the depth/homophily relation;
- the parameter reduction from looping; or
- the training-free depth selection from labeled context?
At the moment the paper mainly positions itself against tabular PFNs and looped transformers, and much less against the GNN literature on homophily and over-squashing, which is where most of the conceptual overlap lies.
-
The generality claim is broader than the evidence.
- The title and conclusion say a single block suffices for PFNs for graphs. But looping is only done for NodePFN, and GraphPFN is only analyzed, not looped.
- All results are single-pass without ensembling, while both models are normally used with ensembles. It is not clear whether the conclusions (in particular the parity in accuracy) hold in the ensembled setting people actually use.
-
Some reporting is confusing.
- The per-dataset averages in Table 2 mix accuracy and ROC-AUC (for the four majority-level graphs), and they do not match the split-0 numbers in Table 7 and Appendix E. It took me a while to figure out which numbers belong to which setting.
- Including four graphs where the model is at majority-class accuracy in the "All (23)" accuracy means also dilutes the comparison.
- I suggest reporting all averages with one metric over the informative graphs, and stating the setting (protocol, split, metric) in every caption.
-
Weight tying turns depth into repeated diffusion that the model cannot switch off.
In Looped NodePFN, each iteration is not pure refinement: it applies the same block, including its message-passing branch. Every extra iteration is therefore one more hop of diffusion with the same weights. The paper states this itself ("every iteration propagates", Section 5; "every iteration is also a hop of propagation", Appendix F).
This has two consequences that I think deserve more discussion:
(1) The looped model loses an ability the 12-layer model has. In NodePFN-12L, the later layers can effectively stop propagating. Their message-passing branches are nearly inactive (Section 3.3), and the Dirichlet energy rebounds after layer 2. In the looped model the energy falls at every iteration and never recovers (Fig. 14). Weight tying removes the freedom to refine without diffusing.
(2) I would like the authors to separate "how long to refine" from "how far to propagate" in the architecture itself, rather than only through test-time selection. Examples are a learned gate on the message-passing branch per iteration, or an initial-residual term as in APPNP/GCNII. They should also report whether this removes the heterophilous gap.
Questions for authors
- Can you report the main comparison with ensembling (as NodePFN and GraphPFN are normally used)? Does the parity in accuracy still hold?
- How does Proposition 1 differ from the CSBM oversmoothing analyses of Wu et al. (2023) and Keriven (2022)?
- Under the standard protocol, the chosen depth does not follow homophily. What is ICDS adapting to in that setting?
- Can you add training-seed variance for the looped models, at least for Loop-1L?
- What exactly is new with respect to spectral graph pruning, ComFy, DeltaGNN, AFRC rewiring and the adaptive-depth GNNs (Hevapathige et al., GPR-GNN)? Would a cheap homophily-aware rewiring or edge-filtering step before NodePFN give similar gains to looping plus ICDS?
- Since every looped iteration also propagates, can the block learn to switch off message passing in later iterations (e.g., a learned gate or an initial residual)? Does this close the gap with NodePFN-12L on heterophilous graphs?
- Could the branch schedule (message passing only in the first g iterations) be learned, or trained into the block, rather than selected at test time? The oracle gains suggest this is where the real improvement on heterophilous graphs is.
7 comments
Response to W2&Q3: The depth does not follow homophily in the headline setting
We agree that without pre-smoothing ICDS is below NodePFN-12L, by 1.5% for Loop-1L and 1.4% for Loop-2L (Table 7). The branch schedule reduces this difference to 0.9% and 1.0%. In this setting the depth selected by ICDS follows homophily, as the analysis predicts (Figure 8b).
Under the standard protocol the depth does not follow homophily, because NodePFN tunes its pre-smoothing for each dataset and this step performs part of the propagation before the model (Figure 10). The motivation still applies in this setting, since the depth a graph needs still depends on the graph. For Loop-1L the best depth for each graph is 1.8% above the best single depth for all graphs (Table 7). To answer Q3, ICDS adapts to this per-graph depth. The depth it selects follows the per-graph oracle depth, with a rank correlation of 0.38 for Loop-1L and 0.36 for Loop-2L (Appendix E).
We have removed from contribution 3 the claim that the chosen depth follows homophily on the benchmarks.
Response to W3: ICDS is essentially validation-based model selection
We agree that ICDS is a simple validation rule and that much of its gain comes from not using depth 12. Our contribution is that looping makes the depth a choice for each graph, and our analysis shows that this choice is necessary.
Table 7, which the reviewer cites, shows this under the standard protocol. With test labels, the best single depth for all graphs improves Loop-1L from 64.6% at the training depth to 65.9%, and the best depth for each graph improves it to 67.7%. Thus, a choice for each graph adds 1.8% beyond any single depth. Without test labels the best single depth is unknown. ICDS uses only the labelled context and reaches 66.3%, above this single depth. For Loop-2L, ICDS reaches 66.9% against 66.4%. We have marked the test-label rows of Table 7.
Response to W4: Fewer parameters, but not less compute
We agree that looping does not reduce training or per-iteration compute, and that training memory drops by only about a quarter. The true gains are a 10 times smaller model, from 116 MB to 12 MB, and a depth chosen for each graph after pretraining. We have corrected the ethics statement, the only place that suggested cheaper training, and the abstract states the speed-up only "once its depth is selected".
The one-off cost of ICDS can be significantly reduced. A large graph needs few hold-outs, so we set their number by the size of the graph.
| On questions, the largest benchmark | NodePFN-12L | Loop-1L with ICDS as in the paper | Loop-1L with hold-outs set by graph size |
|---|---|---|---|
| Mean score over the 23 benchmarks | 64.6% | 64.0% | 64.0% |
| ICDS once per graph | none | 87.0 s | 6.6 s |
| Time per prediction | 4.10 s | 1.04 s | 1.04 s |
With hold-outs set by graph size, Loop-1L is faster in total than NodePFN-12L from the third prediction on questions. Later predictions are faster than with NodePFN-12L on 21 of the 23 benchmarks. On air-europe and air-brazil ICDS selects more iterations than the training depth. The training cost can also be lowered. Truncated backpropagation through the loop (Geiping et al., 2025), which keeps the first iteration and the last four, cuts the time of a training step by about 40% and its GPU memory by about half. In our first untuned attempt it also lowers accuracy, and we leave its tuning to future work. We have added these results to Appendix H.3.
Response to W5&Q2: Proposition 1 and known results
We agree that a finite optimal number of propagation steps on the CSBM is known, and we now cite both works in Appendix D. Proposition 1 adds two results. First, it gives in closed form the boundary in homophily and degree between graphs where no propagation is best and graphs where propagation helps. Wu et al. (2023) give bounds on the variance and on the optimal depth and assume , so they do not cover heterophilous graphs. Keriven (2022) studies latent-space random graphs rather than the CSBM. Second, Proposition 1 states the consequence for a PFN. No fixed depth is optimal over a prior of graphs, and this is its role in the paper.
In our revised paper, Appendix D lists which steps of the proof are rigorous and which are heuristic, and it marks the multi-class remark as an argument rather than a proof. Section 5 tests the multi-class case on syn-Cora with 7 classes, where the selected depth rises with homophily as predicted (Figure 8(d)).
Response to W6&Q5: Positioning against cheap methods
We ran the suggested baselines. We placed each method in front of NodePFN-12L and chose its strength from the labelled context, in the way ICDS chooses the depth. None of them improves NodePFN-12L.
| Mean score (%) in front of NodePFN-12L | Strength chosen in context | Strength chosen on test labels |
|---|---|---|
| None | 64.6 | |
| One of three edge filters per graph (feature similarity / AFRC / predicted labels) | 64.2 | 66.6 |
| ComFy / FeaSt | 64.6 / 64.6 | 65.2 / 65.6 |
| Spectral pruning (ProxyDelete max / min) | 64.6 / 63.6 | 65.6 / 65.2 |
| DeltaGNN score without training | 64.5 | 66.0 |
Test labels would raise NodePFN-12L to between 65.2% and 66.6%, but hold-out selection in the context does not find these strengths. A rewiring step therefore does not give the gain of ICDS, which improves Loop-1L from 61.8% to 64.0% (Q5). What is new in our work is the explanation of how the two branches of a graph PFN use depth, and a depth selected inside a pretrained model from the labelled context, without changing the graph or training on it. Adaptive-depth GNNs such as GPR-GNN and the method of Hevapathige et al. learn the propagation by training on the target graph. We have added a related-work paragraph to Appendix F and these results to Appendix H.4.
Response to W7&Q1: The comparison with ensembling
The parity holds with the official ensembles. When ICDS selects the depth and the branch schedule for each member, the looped models are as accurate as NodePFN-12L on all 23 benchmarks and more accurate on the heterophilous graphs. Each member is a separate forward pass with its own permutation of features and classes and its own preprocessing, so ICDS can select a different depth for each member. In ourr paper, we use a single pass because each readout of the analysis must reflect one computation (Section 3).
| Mean score (%) with the official ensemble | Parameters | All 23 | Heterophilous 10 |
|---|---|---|---|
| NodePFN-12L | 29.0M | 67.5 | 59.5 |
| Loop-1L / Loop-2L with one depth for all members (paper) | 2.97M / 5.33M | 66.6 / 67.0 | 59.5 / 59.2 |
| Loop-1L / Loop-2L with ICDS per member and the branch schedule | 2.97M / 5.33M | 67.8 / 68.2 | 61.4 / 61.5 |
We have added all selection rules we ran to Appendix H.6.
Response to W7&W8: Scope and reporting
We agree that we loop only NodePFN, and we have limited the claim in the abstract to NodePFN. GraphPFN trains only graph adapters, 3.6M of its 20.1M parameters, on a frozen LimiX backbone, so looping it would mean pretraining that backbone itself. Thus, we use GraphPFN to test the mechanistic findings. It also cannot work without its first layer, and its prediction depth grows with homophily (Appendix C).
Table 1 follows NodePFN in reporting ROC-AUC on the four majority-level graphs. We have added a version to Appendix H.2 that reports ROC-AUC over these 4 graphs and accuracy over the other 19 graphs separately. In the revised manuscript, the averages of Table 2 are reported in this way. Table 7 and Appendix E use accuracy on all 23 graphs, which is why their numbers differ from Table 2. In the updated manuscript, eevery result table states the protocol, the split and the metric in its caption.
Summary and map of this response
We thank the reviewer for the careful and detailed review. Any changes or additions in reponse to your review are highlighted in blue in the revised paper. The table below shows where we answer each major concern. After the table, we answer every point of the detailed feedback and every question.
Response to W1 (a): "No loss in accuracy" is only true on average
We agree that "no loss in accuracy" in contribution 3 holds only on average. We have revised contribution 3 to "comparable performance on average" and stated the heterophilous loss in the abstract. The loss comes from questions and 3 small WebKB graphs. The branch schedule removes this loss on average. When ICDS also selects the schedule from the labelled context, Loop-1L reaches 59.7% and Loop-2L 59.3% on the 10 heterophilous graphs, against 59.5% for NodePFN-12L.
Our analysis predicts this loss, as W9 explains. The large per-graph differences in Table 2 are partly split noise, since NodePFN-12L itself ranges from 62% to 87% over the 10 splits of texas. We have added the per-graph and per-seed results to Appendix H.1.
Response to W1 (b): The below-chance result on questions
Selecting the depth and the branch schedule with hold-outs scored by ROC-AUC fixes the below-chance result on questions. On this graph each iteration pulls the positive nodes, which have a high degree, toward their negative neighbours. The paper's ICDS was unable to detect this because it scored the hold-outs by accuracy.
We have added the diagnosis to Appendix H.2.