- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper introduces Node Signal Atlas (NSA), a lightweight pre-training diagnostic framework that characterizes node-classification datasets through three marginal signal channels: node attributes and their local organization, unlabeled structural roles, and local training-label context. These channels are quantified by the Feature Signal Score (FSS), Topology Signal Score (TSS), and Label Smoothness Score (LSS), respectively. The authors interpret these coordinates as aligned with feature-centric, topology-aware, and diffusion-based model interfaces, respectively. The resulting three-dimensional profile is intended to characterize accessible predictive signal beyond homophily, with preliminary model-family triage serving as a secondary application. Experiments on 24 synthetic datasets and 20 real-world benchmarks examine how the scores relate to the performance of different model families. A separate cross-generator architecture-selection experiment further evaluates whether NSA, alone or combined with homophily measures, can help prioritize candidate architectures with low evaluation budgets.
Critical strengths
-
The paper presents an interesting perspective on node classification, particularly in heterophilic settings, by treating node attributes and graph topology as sources of predictive signals. It provides an intuitive framework for relating the signals accessible in a dataset to the interfaces of different model families, helping interpret differences in their performance across datasets. (However, I am not convinced by the proposed correspondence, for the reasons discussed in Critical Weakness.)
-
The paper provides a broad empirical evaluation, covering 24 synthetic datasets, 20 real-world benchmarks, and a separate cross-generator architecture-selection benchmark. These experiments examine both the behavior of the proposed scores and their potential utility for model selection.
Critical weaknesses
The proposed interpretation linking each score to a specific model family is not sufficiently justified.
-
FSS: The FSS descriptor includes feature statistics from k-hop neighborhoods and therefore incorporates graph-dependent information that a feature-only MLP does not access. A high FSS may consequently reflect the predictive value of neighborhood statistics rather than the discriminative power of the node's own attributes. This weakens its interpretation as evidence that MLPs should be competitive. The information accessed by FSS is more closely aligned with methods that combine graph-based feature preprocessing with an MLP, such as FAF [1].
-
LSS: LP is a more direct match in terms of access to observed training labels. APPNP, however, propagates feature-based predictions, whereas LSS directly uses local training-label distributions. Although label smoothness may benefit APPNP, this does not establish that APPNP accesses the information measured by LSS. Grouping models by their use of diffusion therefore does not adequately justify the proposed correspondence at the level of information interfaces. The paper should clarify which of the evaluated GNNs genuinely correspond to this label-context channel.
I am therefore not convinced that the proposed three-way grouping offers additional insight beyond the conventional distinction between graph-agnostic and graph-aware models. This weakens the paper’s central premise of using three scores to characterize datasets through their alignment with distinct model interfaces.
[1] Fixed Aggregation Features Can Rival GNNs. ICML 2026.
Questions for authors
- Could the authors address the concerns raised in the Critical Weakness section by providing further justification or explaining how the framework could be revised to resolve these issues?
- Could the authors clarify the substantive advance of NSA over Tri-Hom? Indeed, your Appendix C.4 distinguishes NSA’s separate coordinates from Tri-Hom’s composite scalar, but Tri-Hom also analyzes the individual and joint effects of its three homophily measures on graph-agnostic and graph-aware models (see Appendix E of its NeurIPS version). It therefore remains unclear what new insights NSA provides beyond different metric definitions and broader model evaluation. Could the authors provide concrete examples and direct comparisons demonstrating diagnoses or conclusions that existing frameworks do not adequately capture?
Response to follow-up Part 1
We thank the reviewer for the thoughtful follow-up; this question gets to the heart of the framework.
FSS and TSS are separated by input, not by model. FSS sees features (a node's own and its neighbors', averaged uniformly) but nothing about where the node sits in the graph. TSS sees where the node sits (degree, core number, structural roles) but no features at all. Graph-aware models read both, so a high FSS does suggest that they will perform well too. That is expected, and Table 5 confirms it.
An intuitive example. Consider two prediction tasks:
Both graphs can have similar homophily, but the signal lives in different places, and NSA separates them. Two corners of our synthetic cube show why the distinction matters (Table 1):
In both cases, NSA tells the user in advance where the signal lives, and therefore which kind of model can access it.
On the design space. We agree that it is large. FSS sits at its neutral point: every neighbor at hop k counts equally, with no weighting by degree, core number, or role. Once aggregation is weighted by such quantities, it reads structure, and that information belongs to TSS. This is the boundary between the two channels.
We do not claim that this boundary makes the channels perfectly isolated. Uniform aggregation still reflects how a node's neighborhood is organized, so some structural information reaches FSS; Table 13 quantifies this on noise-feature datasets. What matters for the user is that NSA still identifies the dominant source. On the T corner, TSS (0.711) is nearly three times FSS (0.257), and the same holds on every noise-feature dataset in our suite: the leading coordinate is always TSS or LSS, never FSS. When an attribute-only reading is needed, FSS₀ removes this component entirely (0.05 on the same dataset).
Revisions.
We hope this clarifies the distinction, and we are grateful for questions that made the paper sharper.