acceptodds
Under review as a conference paper at ICLR 2027

Node Signal Atlas: Mapping Predictive Signal Beyond Homophily

Abstract

Node classification benchmarks often favor very different model interfaces, yet these differences are commonly summarized by homophily. We introduce the *Node Signal Atlas* (NSA), a lightweight pre-training diagnostic that characterizes a dataset through three marginal signal channels: the*Feature Signal Score* (FSS), the *Topology Signal Score* (TSS), and the *Label Signal Score* (LSS). Each coordinate is computed using the same fixed logistic-regression probe on inexpensive annulus-based descriptors, without training a GNN. On a controlled 24-dataset synthetic suite, NSA responds systematically to feature, structural-role, and local label-context manipulations, captures mixed-signal regimes, and exposes an interaction-only boundary outside its marginal scope. Across 20 real benchmarks, NSA separates datasets with nearly identical homophily but substantially different signal profiles and model behavior. Finally, on a frozen cross-generator architecture-selection benchmark, NSA improves over scalar node homophily, while a richer homophily profile is a stronger standalone baseline; combining the two yields the best low-budget triage. NSA thus complements homophily by describing *where predictive signal is accessible*.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph learning, node classification, data-centric learning, dataset characterization, homophily, heterophily, predictive signal, architecture selection

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_YrbEOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper introduces Node Signal Atlas (NSA), a lightweight pre-training diagnostic framework that characterizes node-classification datasets through three marginal signal channels: node attributes and their local organization, unlabeled structural roles, and local training-label context. These channels are quantified by the Feature Signal Score (FSS), Topology Signal Score (TSS), and Label Smoothness Score (LSS), respectively. The authors interpret these coordinates as aligned with feature-centric, topology-aware, and diffusion-based model interfaces, respectively. The resulting three-dimensional profile is intended to characterize accessible predictive signal beyond homophily, with preliminary model-family triage serving as a secondary application. Experiments on 24 synthetic datasets and 20 real-world benchmarks examine how the scores relate to the performance of different model families. A separate cross-generator architecture-selection experiment further evaluates whether NSA, alone or combined with homophily measures, can help prioritize candidate architectures with low evaluation budgets.

Critical strengths
  1. The paper presents an interesting perspective on node classification, particularly in heterophilic settings, by treating node attributes and graph topology as sources of predictive signals. It provides an intuitive framework for relating the signals accessible in a dataset to the interfaces of different model families, helping interpret differences in their performance across datasets. (However, I am not convinced by the proposed correspondence, for the reasons discussed in Critical Weakness.)

  2. The paper provides a broad empirical evaluation, covering 24 synthetic datasets, 20 real-world benchmarks, and a separate cross-generator architecture-selection benchmark. These experiments examine both the behavior of the proposed scores and their potential utility for model selection.

Critical weaknesses

The proposed interpretation linking each score to a specific model family is not sufficiently justified.

  • FSS: The FSS descriptor includes feature statistics from k-hop neighborhoods and therefore incorporates graph-dependent information that a feature-only MLP does not access. A high FSS may consequently reflect the predictive value of neighborhood statistics rather than the discriminative power of the node's own attributes. This weakens its interpretation as evidence that MLPs should be competitive. The information accessed by FSS is more closely aligned with methods that combine graph-based feature preprocessing with an MLP, such as FAF [1].

  • LSS: LP is a more direct match in terms of access to observed training labels. APPNP, however, propagates feature-based predictions, whereas LSS directly uses local training-label distributions. Although label smoothness may benefit APPNP, this does not establish that APPNP accesses the information measured by LSS. Grouping models by their use of diffusion therefore does not adequately justify the proposed correspondence at the level of information interfaces. The paper should clarify which of the evaluated GNNs genuinely correspond to this label-context channel.

I am therefore not convinced that the proposed three-way grouping offers additional insight beyond the conventional distinction between graph-agnostic and graph-aware models. This weakens the paper’s central premise of using three scores to characterize datasets through their alignment with distinct model interfaces.

[1] Fixed Aggregation Features Can Rival GNNs. ICML 2026.

Questions for authors
  1. Could the authors address the concerns raised in the Critical Weakness section by providing further justification or explaining how the framework could be revised to resolve these issues?
  2. Could the authors clarify the substantive advance of NSA over Tri-Hom? Indeed, your Appendix C.4 distinguishes NSA’s separate coordinates from Tri-Hom’s composite scalar, but Tri-Hom also analyzes the individual and joint effects of its three homophily measures on graph-agnostic and graph-aware models (see Appendix E of its NeurIPS version). It therefore remains unclear what new insights NSA provides beyond different metric definitions and broader model evaluation. Could the authors provide concrete examples and direct comparisons demonstrating diagnoses or conclusions that existing frameworks do not adequately capture?
Reviewer_WALBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Node Signal Atlas (NSA), a dataset characterization framework that measures three types of predictive signal: feature accessibility, structural-role information, and local label context. Each score is obtained through fixed logistic-regression probes without training a GNN. Experiments on synthetic and real-world benchmarks examine the interpretability of these scores and their utility for architecture selection.

Critical strengths

Motivation: The idea of characterizing graph datasets based on where predictive information is accessible, rather than relying only on homophily is intriguing. This is a meaningful problem because datasets with similar homophily can favor different model families.

Evaluation: The controlled synthetic experiments are useful, particularly the mixed-signal settings and interaction-only boundary case. Authors acknowledge the limitations of NSA and compare it against stronger homophily-based descriptors rather than only scalar homophily.

Critical weaknesses

The three signal dimensions are not sufficiently disentangled. My main concern is whether FSS, TSS, and LSS can be interpreted as distinct sources of predictive information. FSS incorporates multi-hop neighborhood feature statistics, meaning that it already relies on graph structure rather than purely node-level feature accessibility. Similarly, LSS measures predictability from neighboring label distributions, which can also be informative under structured heterophily rather than conventional label smoothness. The synthetic experiments reveal some of these interactions, such as the simultaneous increase of TSS and LSS in the topology-dominant setting. Although the authors explicitly frame the scores as marginal diagnostics, I think their interpretation as separate signal channels requires additional justification. A more targeted decomposition of zero-hop features, neighborhood features, and structural descriptors would help clarify what each coordinate actually measures.

The diagnostic depends on supervised labels, limiting its practical applicability. Although NSA does not train a GNN, all three coordinates are obtained through supervised logistic-regression probes, and LSS additionally requires observed neighboring labels. Therefore, NSA is not an entirely label-independent dataset characterization method. I wonder how stable these scores remain under different training-label budgets, especially in low-label regimes where architecture selection can be particularly important. The paper describes cross-validation carefully, but it would be helpful to clarify whether NSA scores used for architecture selection are computed exclusively from labels available to downstream training. Without this analysis, I find it difficult to assess the practical cost and reliability of the proposed pre-training diagnostic.

The incremental contribution over existing graph characterization methods remains limited. NSA improves over scalar node homophily, but Homophily-4 is a stronger standalone architecture selector, achieving 1.62 versus 2.63 percentage points of regret. Combining NSA with Homophily-4 reduces regret to 1.31, which is encouraging. However, Appendix D.5 reports a comparable regret of 1.30 when Homophily-4 is augmented with five generic graph statistics. This raises a natural question about whether the improvement comes from the proposed signal decomposition or simply from adding more descriptive graph features. I would like to see stronger evidence that the three NSA coordinates provide complementary information that cannot be captured by simpler descriptors. The fact that the combination analyses were introduced after examining the standalone results also suggests that the reported improvements should be interpreted somewhat cautiously.

The architecture-selection results do not sufficiently establish general practical utility. The strongest evidence for NSA-guided architecture selection comes from a synthetic benchmark with a restricted bank of six architectures. While the cross-generator design is useful, I am not sure how well these results translate to modern graph learning settings with more flexible architectures. The authors' own real-world analysis shows that the selection advantage becomes less pronounced once graph transformers are included. Since the proposed framework is motivated partly by helping practitioners understand which architectures to evaluate, I think stronger real-world validation is necessary to establish its practical value beyond descriptive dataset analysis.

The marginal probes may incorrectly characterize difficult but learnable datasets. I appreciate that the paper explicitly investigates an interaction-only boundary case, where all three NSA scores are near chance while GPS achieves 87.95% accuracy. However, this also highlights a fundamental limitation: low marginal scores do not necessarily imply that a dataset lacks useful predictive information. They may simply indicate that the chosen descriptors and linear probes cannot access it. This distinction is important because the proposed "signal desert" interpretation could be misleading for datasets where graph-feature interactions are essential. I would be interested in understanding how frequently such cases arise in real benchmarks and whether the framework can distinguish limited probe expressiveness from genuinely weak predictive signal.

Questions for authors

See weakness

Reviewer_BHYoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Node Signal Atlas (NSA), a diagnostic for node-classification datasets with three scores measuring accessible feature, topology, and local label-context signal. Each score uses the same regularized logistic probe on a corresponding family of inexpensive descriptors, with chance-normalized macro-accuracy and fold-specific construction of label-dependent descriptors. Validation includes 24 controlled synthetic datasets, 20 real benchmarks, and a separate frozen cross-generator architecture-selection benchmark. NSA has lower mean selection regret than scalar node homophily. Homophily-4 has lower regret and higher Top-1 accuracy than NSA alone, while NSA has higher Spearman correlation and Top-3 coverage (Table 4). Combining the descriptions improves several selection measures.

Critical strengths
  1. The diagnostic contribution is supported by controlled validation and an explicit scope boundary. Section 3 applies a common probe and normalization across the descriptor families and rebuilds label-context descriptors within each fold. The synthetic experiments show how the scores respond to different signal constructions. Table 1 also reports near-chance marginal scores on Node Twins despite GPS reaching 87.95%, making clear that low NSA scores need not imply an absence of exploitable interactions. This combination of interpretable measurements and a demonstrated limitation makes the atlas useful as a diagnostic.

  2. The reported architecture-selection protocol separates benchmark calibration from descriptor evaluation. Appendix D describes a pilot–freeze–final protocol, fresh evaluation seeds, and different training and test generator families. It reports a missed calibration criterion and distinguishes exploratory concatenations from standalone comparisons that the authors describe as pre-registered. In my assessment, this reported protocol strengthens the evidence for utility beyond the pilot setting, even though the practical advantage is modest in some comparisons.

Critical weaknesses
  1. The distinctive practical benefit over inexpensive alternative descriptions is less established than the diagnostic benefit. Table 4 reports selection regret of 2.63 percentage points for NSA, 1.62 for Homophily-4, 1.31 for their combination, and 1.18 for NSA plus scalar node homophily. Appendix D.5 includes a relevant control: Homophily-4 plus five generic statistics achieves 1.30 points. The 1.30 and 1.31 means do not establish either a meaningful difference or equivalence. Table 23 also reports 99.3% of oracle performance after two evaluations for NSA, versus 98.3% for a constant ranking. These results support useful selection information, but warrant a narrower account of its practical advantage and of how much is specific to NSA. They do not undermine the descriptive contribution.

  2. Class count and imbalance may influence the real-data associations. Table 5 correlates normalized macro probe scores with downstream micro-accuracy across datasets; Section 4.1 and Appendix A.3 explicitly specify that downstream metric. Section 3.1 explains why raw accuracy baselines vary with class distributions. Correlating these quantities is legitimate, but the correlations and matched-homophily examples may reflect both accessible signal and differing evaluation baselines. This is a possible confound, not a demonstrated explanation of the results. A sensitivity analysis using normalized macro performance or appropriate controls would clarify the interpretation. This concern is separate from the balanced synthetic selection experiment.

Questions for authors
  1. Do the Table 5 associations and matched-homophily examples retain their interpretation when downstream performance uses the same chance-normalized macro-accuracy as NSA?

  2. Can you report paired regret differences with uncertainty for NSA + Homophily-4, Homophily-4 + Meta-5, and NSA + scalar homophily on the frozen test problems? This would clarify the operational claim.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to follow-up Part 1

Under your revised interpretation, FSS measures feature signal in graph context through neighborhood aggregation. However, graph-aware models also exploit neighborhood feature information. Following the same reasoning, shouldn't a high FSS also suggest that graph-aware models are likely to perform well?

We thank the reviewer for the thoughtful follow-up; this question gets to the heart of the framework.

FSS and TSS are separated by input, not by model. FSS sees features (a node's own and its neighbors', averaged uniformly) but nothing about where the node sits in the graph. TSS sees where the node sits (degree, core number, structural roles) but no features at all. Graph-aware models read both, so a high FSS does suggest that they will perform well too. That is expected, and Table 5 confirms it.

An intuitive example. Consider two prediction tasks:

  • Topic of a paper in a citation network. The words in a paper and in the papers it cites reveal its topic. The signal is in the features (high FSS), and averaging neighbors' features helps.
  • Activity level of an airport in a flight network. Whether an airport is a major hub or a regional airport is determined by its position in the network: many connections, central location, links to other hubs. Its features (and its neighbors' averaged features) say little, because hubs connect to regional airports and vice versa, so averaging mixes the two classes. The signal is in the structure (high TSS), and only models that encode structural roles can recover it.

Both graphs can have similar homophily, but the signal lives in different places, and NSA separates them. Two corners of our synthetic cube show why the distinction matters (Table 1):

  • F corner (informative features, random graph): FSS = 0.681, TSS = −0.012. MLP reaches 86.7%, while GCN drops to 35.7% because aggregating over a random graph mixes the classes.
  • T corner (noise features, labels encoded in structure): TSS = 0.711, FSS = 0.257. Models that only read and propagate features stay near chance (MLP 31.8%, APPNP 33.7%, SGC 36.5%), while models that encode structure succeed (GCN 68.1%, H2GCN 65.9%, GPS 75.0%).

In both cases, NSA tells the user in advance where the signal lives, and therefore which kind of model can access it.

On the design space. We agree that it is large. FSS sits at its neutral point: every neighbor at hop k counts equally, with no weighting by degree, core number, or role. Once aggregation is weighted by such quantities, it reads structure, and that information belongs to TSS. This is the boundary between the two channels.

We do not claim that this boundary makes the channels perfectly isolated. Uniform aggregation still reflects how a node's neighborhood is organized, so some structural information reaches FSS; Table 13 quantifies this on noise-feature datasets. What matters for the user is that NSA still identifies the dominant source. On the T corner, TSS (0.711) is nearly three times FSS (0.257), and the same holds on every noise-feature dataset in our suite: the leading coordinate is always TSS or LSS, never FSS. When an attribute-only reading is needed, FSS₀ removes this component entirely (0.05 on the same dataset).

Revisions.

  • Section 3.2: states the boundary explicitly. FSS aggregates uniformly, topology-dependent weighting belongs to TSS, and some structural information still reaches FSS.
  • Section 3.3: states that TSS captures signal that exists without features, illustrated by the T corner.
  • Appendix B.5: a new paragraph shows that this leakage never changes the diagnosis on noise-feature datasets.
  • Opening of Section 3: aligned with these definitions.

We hope this clarifies the distinction, and we are grateful for questions that made the paper sharper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.