acceptodds
Desk rejected at ICLR 2027

Class Pair Distance Calibration for Robust Graph Neural Networks

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Distance based graph defenses can suppress useful messages when edges between different classes have large representation distances. CLASP divides each edge distance by the median for its predicted class pair, then applies a robust penalty to the relative distance. Each pair thus has its own reference scale. If retained edges form a majority of a perturbed bucket and their distance drift is bounded, we bound the new median using their clean order statistics and the drift bound. We evaluate CLASP in a two layer GCN and in HARA, which combines ego, one hop, and two hop channels. With full training labels and five shared attack edits, matched GCN controls show mean accuracy gains of 7.7 percentage points on Cora and 4.0 on Actor, and losses of 1.0 point on Roman-Empire and Amazon-Ratings. The system comparison uses unequal tuning budgets, with HARA's search expanded after earlier test results and final parameters selected on validation data. Across seven datasets and four attack families, HARA exceeds the strongest recorded fixed baseline in 24 of 48 budget scenarios, mostly under shared and local attacks.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph neural networks, Adversarial robustness, Heterophily, Robust aggregation, Class-pair calibration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uQJ9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The motivation is clearly illustrated with intuition (Figure 1) and schematic diagrams (Figure 2). Mathematical formulations are well-defined, and experimental protocol details are clearly articulated.

Critical strengths

Standard distance-based pruning often destroys legitimate cross-class edges because inter-class distance distributions naturally differ from intra-class distributions. CLASP directly resolves this with a fine-grained, class-pair specific reference scale without requiring ground-truth labels during test time.

Critical weaknesses

HARA receives extended hyperparameter tuning (additional search budgets targeting specific dataset-attack combinations guided by earlier test outcomes), whereas fixed baselines use pre-recorded configurations. This protocol renders main system comparisons descriptive rather than strictly controlled.

CLASP relies entirely on a frozen preliminary 1-layer GCN to assign edge buckets. Under severe structural attacks or high heterophily, if the preliminary classifier produces inaccurate predictions or misclassifies noisy neighborhoods, edge buckets become corrupted (RR grows large), which degrades median calibration.

Questions for authors

How sensitive is CLASP's performance to the accuracy and architecture of the preliminary classifier? Have you evaluated replacing the 1-layer GCN with a stronger or more robust preliminary model?

4 comments
AuthorsOpenReview ↗

Response to W1: Controlled system and calibration comparisons

Thank you for distinguishing the performance of a selected system from a controlled test of its methodological contribution. We addressed this with two completed experiments that answer complementary questions. The first gives every system the same registered search opportunities. The second holds the training recipe fixed and changes only calibration grouping within a host.

ExperimentQuestion answeredMain controls
System comparisonHow do the methods perform after comparable search opportunities?Same candidate count, finalist schedule and validation selection procedure
Grouping comparisonWhat changes when class pair grouping replaces global grouping?Same host configuration, starting weights and preliminary classifier

The system comparison includes GCN, RUNG, GCN Global, GCN Pair, HARA Global and HARA Pair. Thus, the existing robust baseline and both Global controls receive a new search alongside the proposed variants. Each method receives the following schedule for every dataset.

StageCommon rule
ScreeningEight candidates evaluated with seed 0
FinalistsBest two candidates additionally evaluated with seeds 1 and 2
SelectionValidation results only, using the predefined ranking and tie rule
ConfigurationOne selected configuration per method and dataset across attacks
TestingAll selections sealed before the new test results were opened

This changes the comparison from newly tuned methods against previously fixed baseline settings to methods with the same scheduled search opportunities. Common epoch and early stopping caps also constrain training. Equal candidate counts refer to search opportunities, while the computation consumed by each fit can depend on the architecture and when training stops. The new test results were not used to revise the candidate lists, selected configurations or attack scope.

The grouping experiment addresses attribution more directly. Here, a host means the final network that predicts node classes. Global normalizes edge distances using one common median. Pair, our CLASP rule, uses a median for each predicted endpoint class pair. Both variants use the same frozen preliminary classifier to supply predictions, and those predictions are recomputed on each evaluation graph. The host starts from matched weights and follows the same training recipe. Each variant is then trained separately.

Within HARA, the node feature channel, one hop and two hop channels, mixer, refinement and anchor remain unchanged between Pair and Global. The mixer also retains the same preliminary predictions. Therefore, the experimental difference concerns the aggregation grouping rather than introducing HARA's extra components along with it. This is the appropriate comparison for identifying the contribution of CLASP at the fixed recipe, while the system experiment evaluates the independently selected complete methods.

Both studies use the existing benchmarks, whose earlier test results had already been observed. The prospective seal makes the new selection sequence explicit but does not remove that history or create a new independent test set. Thank you for highlighting this distinction. The additional evidence now separates comparisons of complete systems from tests of calibration grouping, so each conclusion can be assessed against the corresponding control.

AuthorsOpenReview ↗

Response to W2: Prediction error and bucket stability

Thank you for highlighting the role of preliminary predictions under heterophily and structural perturbations. We examined prediction accuracy and changes in bucket membership separately, because they measure different aspects of the concern. Our calibration rule operates on predicted groups, and the theoretical analysis explicitly allows those groups to change after an attack.

The preliminary classifier is fitted first and then frozen, meaning its learned parameters stop changing. Its predictions are nevertheless recomputed on every perturbed graph. For each directed edge, CLASP takes the predicted class of the sending node followed by that of the receiving node. Edges with the same ordered pair enter the same bucket. Their median representation distance provides the reference used to normalize each distance before robust weighting. The true test labels are not used to construct these buckets.

An incorrect predicted pair and a changed predicted pair are therefore different events. A pair can be incorrect on both the clean and perturbed graphs while remaining in the same bucket. Conversely, a changed prediction can move an existing edge between buckets even when the attack does not edit that edge. The former concerns agreement with true classes. The latter concerns changes to the set of distances used to compute a bucket's median.

We measured both quantities for the frozen one layer GCN on Cora and Actor. The following results use shared attacks with five edits.

DatasetOrdered pair accuracy (%)Surviving edge bucket migration (%)
Cora69.32 ± 0.140.0407 ± 0.0002
Actor9.67 ± 0.060.0231 ± 0.0014

Ordered pair accuracy requires both endpoints to be predicted correctly and is measured over all labeled directed edges other than self loops. Migration is the fraction of surviving edges whose ordered predicted pair changes from the clean graph. Inserted and deleted edges are excluded from this migration denominator. We average 100 attack graphs within each seed, then report the mean and sample standard deviation over three seeds. The two columns have distinct denominators and should not be interpreted as complementary percentages.

The measurements distinguish persistent prediction error from the additional reassignment induced by these attacks. Small migration over the whole graph describes membership stability at that scale. Correctness and behavior at an individual attacked target are separate quantities. We therefore evaluate the usefulness of the resulting groups through downstream prediction experiments as well.

Proposition 1 concerns a bucket defined by predicted classes. Its retained majority condition means that more than half of the edges in the perturbed bucket were already in the corresponding clean bucket. A bound on how much those retained distances move then bounds the new median using their clean distance ordering. The premise is about retained membership and distance drift, rather than requiring every predicted class to match its true label. Its conclusion concerns the calibration statistic, with final classification assessed empirically.

To address dependence on the original preliminary model directly, we also completed controls using an MLP, a two layer GCN and RUNG in addition to the one layer GCN, with both downstream hosts on clean and shared evaluations. Thank you for motivating these measurements. They assess the method using actual prediction errors and alternative preliminary architectures rather than assuming an ideal source of class labels.

AuthorsOpenReview ↗

Response to the Question: Sensitivity to the preliminary architecture

Thank you for suggesting a direct test with alternative preliminary architectures. We completed that comparison using a one layer GCN, a multilayer perceptron (MLP), a two layer GCN and RUNG. This tests the method beyond its original preliminary classifier and examines how the resulting calibration effect changes with the provider, dataset and downstream host.

The preliminary classifier supplies predicted node classes. CLASP groups directed edges by the ordered predictions at their endpoints and uses each group's median distance as a reference. The host is the final classifier that consumes the calibrated messages. We evaluate both GCN and HARA hosts. Within each provider and host setting, Pair uses the predicted class pair groups and Global uses one common group. Their starting host weights and training recipe are matched, and both use the same fitted preliminary classifier.

The completed panel covers Cora and Actor, clean graphs and shared attacks with 1, 3 and 5 edits, using three seeds. Preliminary models are selected by validation loss and frozen before downstream fitting. Predictions are recomputed on each evaluation graph. RUNG supplies an architecture with robust propagation and is trained using clean cross entropy in this experiment.

The table reports the largest shared attack budget for all four providers and both hosts. Each entry is Pair accuracy minus Global accuracy in percentage points. Positive values favor class pair grouping.

Preliminary modelCora GCN hostActor GCN hostCora HARA hostActor HARA host
One layer GCN+8.00 ± 1.73+1.00 ± 4.00+2.00 ± 3.00+1.33 ± 1.53
MLP−1.00 ± 2.65+1.00 ± 3.46−3.00 ± 4.58+0.00 ± 2.65
Two layer GCN+2.00 ± 3.61+4.33 ± 2.08−5.67 ± 5.13−2.67 ± 2.08
RUNG−9.33 ± 3.79+3.33 ± 3.06−11.00 ± 7.55+3.33 ± 3.06

All shared evaluations use the same 100 predetermined targets and stored attack graphs. We first calculate the difference within each seed and then report its mean and sample standard deviation across seeds. This includes all targets, including those initially classified incorrectly. The standard deviation describes the variability of the registered repeats.

The results show that gains occur with several preliminary architectures, while their direction and size depend on the host and dataset. Across the full panel, the attack budget also matters. For example, RUNG on Actor has positive GCN grouping gain at the displayed budget but negative gains at the smaller shared budgets.

We also measured preliminary quality directly. On Cora under the same attack budget:

Preliminary modelTest node accuracy (%)Ordered pair accuracy (%)
One layer GCN75.16 ± 0.1269.32 ± 0.14
RUNG81.30 ± 0.4976.19 ± 0.43

Node accuracy uses the full test mask. Pair accuracy requires both endpoints to be correct over all labeled directed edges other than self loops. Each quality metric first averages the 100 attack graphs within a seed. RUNG has higher preliminary accuracy here, yet its grouping gain is smaller. Thus, preliminary accuracy alone does not determine the observed calibration benefit.

These are sensitivity measurements of complete fitted pipelines. Architecture changes several properties at once, some GCN Global control models also learn different weights across preliminary architectures, and HARA uses preliminary predictions in its mixer as well as calibration. The controls therefore quantify observed sensitivity without isolating classifier accuracy as a single cause. Thank you for encouraging this broader evaluation, which establishes compatibility with alternative preliminary models and makes their measured effects explicit.

Dear Reviewer uQJ9,

Thank you for raising the question of comparable tuning budgets and for suggesting that we test alternatives to the preliminary GCN. In our supplementary replies, we describe how we gave all six methods the same number of tuning trials and report how the effect of calibration varied across four preliminary classifiers on Cora and Actor.

When you have a chance, we would be grateful to hear whether these results address your concerns. If any part of the comparison would benefit from further explanation, we would be happy to clarify it. Thank you for your time and thoughtful feedback.

Reviewer_XahpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes CLASP, a class-pair distance calibration method for improving the adversarial robustness of graph neural networks. The method first predicts node labels using a preliminary GCN and groups edges according to their predicted class pairs. It then normalizes representation distances by the median distance within each group before applying a robust message aggregation penalty. CLASP is evaluated within a two-layer GCN and a more complex HARA architecture that combines ego, one-hop, and two-hop information. The paper also provides a conditional theoretical bound on median changes under graph perturbations. Experiments across seven datasets and several attack settings examine the effectiveness of the proposed calibration strategy.

Critical strengths
  1. The paper considers a potentially meaningful limitation of distance-based graph defenses: legitimate cross-class edges may be incorrectly suppressed because of their relatively large representation distances.

  2. The authors evaluate the proposed approach under several attack settings and include matched comparisons between global and class-pair calibration, which are useful for examining the contribution of the calibration mechanism.

Critical weaknesses
  1. Insufficient motivation and limited justification of the proposed calibration mechanism. The central motivation is that legitimate cross-class edges may have larger representation distances and therefore receive excessively small aggregation weights. However, the paper provides little empirical evidence demonstrating how frequently this issue occurs in real datasets or how substantially it affects existing robust GNNs. Figure 1 is primarily a schematic illustration rather than an empirical characterization of this problem. More importantly, CLASP constructs its distance groups using predicted node labels from a preliminary GCN, which may themselves be unreliable under adversarial perturbations. Since these predictions are recomputed on the perturbed graph, attacks may alter both the group assignments and their distance statistics. The paper does not adequately investigate the reliability of these predicted class pairs or explain why the resulting calibration should remain effective under adversarial conditions. This weakens the justification of the core design.

  2. The experimental evidence does not convincingly establish the contribution of class-pair calibration. The matched comparisons in Table 2 are particularly concerning. Within the HARA architecture, replacing global calibration with class-pair calibration decreases mean attacked accuracy on Cora, Actor, and Roman-Empire, with only a small improvement on Amazon-Ratings. Although the GCN host shows improvements on Cora and Actor, the results are inconsistent across datasets and architectures. Meanwhile, the stronger performance reported for HARA in Figure 3 cannot be directly attributed to CLASP because HARA introduces additional architectural components, including multiple aggregation channels, feature mixing, and refinement. Consequently, the experiments do not convincingly demonstrate that class-pair calibration itself is responsible for the reported robustness improvements. This is a substantial concern because the calibration mechanism constitutes the paper's primary methodological contribution.

  3. Unequal and test-informed hyperparameter searches undermine the reliability of the experimental comparisons. Section 5 explicitly states that the proposed methods and existing baselines are evaluated using unequal tuning budgets. More concerningly, the authors conduct additional HARA hyperparameter searches targeting dataset-attack combinations identified from earlier test results. Even though the final configurations are selected using validation data, allowing previously observed test results to influence the subsequent search strategy introduces a potential selection bias. Furthermore, HARA is compared against previously evaluated fixed baseline configurations rather than methods receiving comparable tuning opportunities. Under these conditions, the reported improvements over the strongest recorded baselines are difficult to interpret as evidence of methodological superiority. The fact that HARA outperforms the strongest fixed baseline in only 24 of 48 attack-budget scenarios further limits the strength of the empirical claims.

Questions for authors
  1. Can the authors provide empirical evidence that legitimate cross-class edges are systematically suppressed by existing distance-based defenses? How accurate and stable are the predicted class-pair assignments under different adversarial attacks?

  2. Why does class-pair calibration provide limited or negative improvements within HARA in Table 2? Can the authors provide additional controlled experiments demonstrating its independent contribution across more datasets and attack settings?

  3. Can the authors conduct comparisons under equivalent hyperparameter tuning budgets and an evaluation protocol in which test results are not used to guide any subsequent model or search-space selection?

  4. Regarding Proposition 1, how frequently do the retained-majority and bounded-distance-drift assumptions hold under the evaluated attacks? How does the resulting median bound translate into improved final node classification robustness?

8 comments
AuthorsOpenReview ↗

Response to W1: Empirical motivation and reliability of class pair calibration

Thank you for identifying the need to connect the motivation to measured behavior. We have added direct measurements of message suppression and preliminary predictions across all seven datasets, using clean graphs and shared attack graphs. These diagnostics make the calibration operation explicit beyond the schematic in Figure 1.

A message is a neighbor's contribution to a node's representation. The robust aggregation rule reduces this contribution when the distance between the two node representations is large relative to a reference scale. Global calibration uses one median distance for all active edges. Pair calibration, or CLASP, instead uses a median for each ordered pair of predicted source and receiver classes. Thus, an edge is compared with other edges in its predicted group. The preliminary classifier supplies these predictions. Its trained weights remain fixed during evasion evaluation, while its predictions are recomputed on the perturbed graph.

We first isolate the calibration operation by holding the graph, layer and trained GCN Pair representation fixed, and switching only between Global and Pair scales. The table reports the change in the percentage of surviving original cross class messages assigned exactly zero weight under five shared edits. A negative value means that fewer such messages are removed by the weighting rule. The other columns measure the preliminary GCN used to form the groups.

DatasetZero weight change, ppPair accuracy, %Bucket migration, %
Cora−3.7269.340.0186
CiteSeer−2.6759.530.0377
Chameleon−5.6926.320.1608
Squirrel−6.6713.990.0552
Actor−0.319.670.0221
Roman Empire−9.1110.580.0176
Amazon Ratings−0.0718.820.0044

Each seed averages 20 attack graphs, followed by averaging across three seeds. Measurements refer to the first one hop aggregation call in layer 0. True labels are used only to identify cross class edges for this offline diagnostic. CLASP itself uses predicted labels. Pair accuracy requires both endpoint predictions to be correct across directed edges excluding self loops. Bucket migration is the fraction of surviving edges whose predicted pair changes relative to the clean graph.

The table separates correctness from sensitivity to the attack. A group assignment can remain unchanged even when its predicted class is inaccurate. These whole graph migration rates therefore measure assignment movement, rather than correctness or stability near every target. The suppression comparison also isolates weighting at fixed representations, whereas separately trained models may learn different representations. It measures how the proposed grouping changes message treatment. The matched classification experiment evaluates the overall effect of grouping, but it does not establish that preserving these particular cross class messages causes the accuracy gain.

The complementary matched classification experiment measures the predictive effect directly, holding the host architecture, initialization, preliminary classifier and training recipe constant. Its clearest benefit occurs for GCN under shared attacks. Thank you for encouraging this distinction between the calibration mechanism, assignment reliability and final prediction quality. It allows the motivation to be supported by three directly interpretable measurements.

AuthorsOpenReview ↗

Response to W2: Isolating the contribution of calibration within each host

Thank you for emphasizing the distinction between the contribution of class pair calibration and the performance of the full HARA system. We have completed additional matched experiments that test grouping within each host architecture. Here, the host is the network into which the aggregation rule is inserted. GCN provides a simpler host, while HARA also includes separate feature and graph channels, a learned mixer, refinement and anchoring.

Global calibration divides edge distances by one median computed across active edges. Pair calibration divides each distance by the median of its ordered predicted class pair. In every matched comparison, Pair and Global share the preliminary classifier, initial host weights, cutoff, training recipe and all remaining architectural components. In HARA, both versions retain the same channels, mixer, refinement count and anchor strength. Both versions use the same preliminary classifier and prediction procedure, including for the mixer. Each version is trained separately from the matched initialization, so the comparison measures the effect of choosing grouping throughout training and evaluation.

The expanded experiment covers every applicable attack family in the registered panel. Values below are Pair minus Global classification accuracy in percentage points. A positive value favors Pair. We first average scenarios equally within each seed, then report the mean and sample standard deviation across three seeds.

Attack familyScenariosGCNHARA
Shared21+3.254 ± 0.707+0.413 ± 1.502
Local9−1.074 ± 1.976−3.000 ± 1.176
Global12+0.008 ± 0.897−1.257 ± 1.438
Poisoning6+0.562 ± 0.704−0.943 ± 1.897
All attacks48+1.294 ± 0.574−0.814 ± 0.547

A scenario is a dataset, attack family and budget combination. Shared attacks use the same fixed perturbed graphs for both versions. Local and global attacks are generated against each evaluated model using the same attack protocol. Poisoning uses shared perturbed training graphs and retrains both versions. Accuracy includes all prescribed evaluation targets, including those initially classified incorrectly.

The strongest positive grouping result is in the simpler GCN host under shared attacks, with higher accuracy in 19 of the 21 scenarios. Because the other components are fixed, this comparison directly measures grouping within that recipe. The HARA columns show why a result for GCN should be interpreted with its host and attack setting specified. We use these matched results to support the contribution of the calibration operation, while comparisons between complete systems assess the combined architecture and selected configuration. The reported standard deviations describe variation across the three registered seeds.

We appreciate your request for clearer attribution. The added controls provide a direct answer by changing the calibration grouping while retaining the architecture, and by reporting the complete attack panel instead of inferring the contribution from HARA's overall system ranking.

AuthorsOpenReview ↗

Response to W3: Comparable search opportunities and complete system comparisons

Thank you for the careful assessment of the search protocol. We have completed a new comparison in which all six methods receive the same scheduled search opportunities, including fresh tuning of RUNG and both Global controls. The search schedule, candidate sets and evaluation scope were fixed before this round's test results were inspected. This directly addresses the concern about comparing an extended search for HARA with previously fixed baseline configurations.

For each dataset and method, we evaluated eight candidate configurations with seed 0. The two strongest validation candidates were then evaluated on seeds 1 and 2 as well. Final selection used validation results across the three seeds. One configuration per dataset and method was fixed for all attack settings, and all selected configurations were sealed before the subsequent test evaluation. Host training also used the same epoch and early stopping caps. Equal candidate counts establish comparable scheduled selection opportunities, while method specific candidate spaces and realized training costs are recorded separately.

The resulting comparison includes GCN, RUNG, GCN Global, GCN Pair, HARA Global and HARA Pair. Global and Pair denote a single distance reference scale and separate scales for predicted class pairs, respectively. Below we report HARA Pair against every comparator across the complete attack panel. Positive differences favor HARA Pair. Scenario counts use the sign of the mean difference, and the accuracy column first averages scenarios within each seed, then reports the mean and sample standard deviation across three seeds.

ComparatorPositive/negative/tied scenariosAccuracy difference, pp
GCN44/4/0+19.068 ± 1.028
RUNG15/33/0−4.224 ± 0.614
GCN Global35/13/0+8.426 ± 1.190
GCN Pair31/17/0+6.847 ± 1.247
HARA Global13/31/4−1.212 ± 1.253

Each row contains all 48 dataset, attack family and budget scenarios. Accuracy uses the prescribed evaluation population, including initial errors and cases with no accepted edit. The table identifies the supported comparative result precisely. HARA Pair has higher aggregate attacked accuracy than GCN and the two calibrated GCN systems, while the complete results also retain RUNG and HARA Global. These are descriptive comparisons under the registered search schedule, with the same reporting rule for every method.

The original 24/48 count remains a description of the historical comparison. The new experiment supplies a separate comparison with prospectively fixed search opportunities. Since these benchmark data had already been studied, the rerun does not create an independent new test set. This round's test outcomes were not used to revise the candidate sets, selected configurations or attack scope.

Thank you for prompting a more directly comparable evaluation. We also completed a separate experiment with a fixed training recipe and matched Pair/Global initialization. That experiment isolates grouping itself, while the new tuning experiment evaluates complete systems after comparable selection opportunities.

Reviewer_5saoOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CLASP, which normalizes the median distance of pairs by prediction class before performing robust aggregation. The authors validate this method on GCN and HARA and provide bounds on the variation of the median under certain conditions.

Critical strengths
  1. The method is intuitive: it calibrates the scale by category, aiming to reduce the false suppression of useful cross-category edges.
  2. Setting global median calibration as a control helps to examine the role of category grouping in isolation; it also clearly specifies the conditions under which the theory applies.
Critical weaknesses
  1. The benefits of calibration vary by category. As shown in Table 2, HARA performed worse than the global calibration version in three of the four datasets; it is therefore too early to conclude that this module provides consistent benefits.
  2. The hyperparameter tuning budgets used in the system comparison were inconsistent, and early test results influenced the planning of subsequent searches; therefore, performance differences cannot be entirely attributed to the methods themselves.
Questions for authors
  1. Why does categorization work well for calibration on some GCN datasets but often fails to improve performance in HARA?
  2. Do the main performance differences still hold under the same hyperparameter tuning budget and independent testing setup?
6 comments
AuthorsOpenReview ↗

Response to W1: Controlled evidence for class pair calibration

Thank you for asking us to clarify when class pair calibration is beneficial. We addressed this with an expanded controlled comparison covering the applicable attack settings across all seven datasets. The main contribution supported by this experiment is a measurable grouping effect within a fixed host, with its dependence on the host and attack made explicit.

Here, the host is the neural network that uses the calibrated messages to predict node classes. Global places all active edges in one group and uses their median distance as a common reference. Pair, our CLASP rule, groups edges by the predicted classes of their two endpoints and computes a median for each group. Thus, Pair judges a distance relative to other edges with the same predicted class pair. The preliminary classifier supplies these predicted classes and does not use true test labels to form groups.

Within each host, Pair and Global use the same starting weights, preliminary classifier, cutoff and training configuration. The preliminary classifier's parameters are frozen, while its predictions are recomputed on each evaluation graph. HARA's channels, mixer, refinement and anchor are also held fixed. Each model is then trained separately. This design changes the aggregation grouping without adding architecture or giving Pair an additional parameter search.

Attack familyDataset countScenariosGCN differenceHARA difference
Shared721+3.254 ± 0.707+0.413 ± 1.502
Local39−1.074 ± 1.976−3.000 ± 1.176
Global412+0.008 ± 0.897−1.257 ± 1.438
Poisoning26+0.562 ± 0.704−0.943 ± 1.897
All attacks748+1.294 ± 0.574−0.814 ± 0.547

Differences are Pair accuracy minus Global accuracy in percentage points, so positive values favor Pair. Shared attacks use the same stored perturbations for the compared models. Local and global evasion attack each fitted model under the same protocol. Poisoning perturbs the training graph and retrains the complete pipeline. Each family covers its applicable dataset subset.

For each of three seeds, we average the accuracy differences equally across the relevant scenarios. We then report the mean and sample standard deviation across those seeds. The standard deviation describes variation between the registered repeats.

HostPositive / negative / tied scenarios across all attacks
GCN31 / 17 / 0
HARA22 / 24 / 2

These counts use the sign of each scenario's mean difference. The clearest accuracy benefit occurs for GCN under shared attacks, where Pair improves the mean in 19 of 21 scenarios. The full panel shows how this benefit depends on the host and attack. We therefore attribute the controlled GCN improvement to grouping under the specified recipe and keep HARA's overall system performance separate. Thank you for prompting this broader test, which gives a more precise account of where calibration contributes.

AuthorsOpenReview ↗

Response to W2: Comparable search opportunities and sealed selection

Thank you for identifying the distinction between selecting configurations on validation data and providing comparable opportunities to find those configurations. We addressed the latter directly through a new registered comparison that retunes every included method under the same staged search schedule. This provides a common basis for comparing complete systems.

The methods are ordinary GCN, RUNG, GCN Global, GCN Pair, HARA Global and HARA Pair. Global and Pair use the same kind of distance calibration, but Global estimates one median across active edges while Pair estimates separate medians for predicted endpoint class pairs. Including both controls in both hosts allows the comparison to cover the calibration variants as well as the established baselines.

StageRule applied to every method and dataset
Initial screeningEight candidate configurations evaluated with seed 0
Finalist evaluationBest two candidates additionally evaluated with seeds 1 and 2
Model selectionValidation results only, with a fixed ranking and tie rule
Selected configurationOne configuration per method and dataset, used across all attack settings
Test evaluationStarted only after all selected configurations were sealed

This schedule matters because each method receives the same number of opportunities at both search stages. RUNG and the Global controls are retuned under this schedule, rather than being represented only by earlier fixed configurations. Selecting one configuration across attacks also prevents choosing a different final configuration after seeing each attack's test performance.

The validation ranking was specified in advance. Screening combines clean and shared validation evaluations. Finalists additionally use the applicable local and global validation evaluations. Scores first average budgets within each family and then weight the families equally. Ties are resolved by clean validation accuracy, validation loss and candidate identifier, in that order. The resulting choices were fixed before the new test results were opened, and those results did not trigger further changes to the candidates, selected settings or attack coverage.

Training componentMaximum epochsEarly stopping patience
Host classifier30060
Preliminary classifier, when required20050

The host is the final node classifier. The preliminary classifier supplies predicted labels for calibration. Common caps and candidate counts equalize the registered search opportunities, while actual training time can still differ with architecture and early stopping.

We also completed a separate experiment for the narrower question of attribution. There, Pair and Global start from matched host weights, share the same preliminary classifier and use an identical fixed training recipe. Only the calibration grouping changes. This distinguishes the performance of selected systems from the contribution of the grouping rule itself.

The new protocol is a prospectively fixed evaluation on the existing benchmarks. Earlier benchmark test results had already been observed, so the seal applies to this new selection and evaluation round and does not create a historically unseen test set. Thank you for raising both aspects of the protocol. The completed studies now report comparable search opportunities and controlled grouping effects as separate evidence.

AuthorsOpenReview ↗

Response to Q1: Why the effect differs between GCN and HARA

Thank you for asking how the same calibration operation can behave differently inside GCN and HARA. Our additional experiments provide evidence that its additional benefit depends on the processing around it. We examined this with predefined component changes, while retaining a matched Pair and Global comparison within each setting.

The host is the network that consumes the calibrated messages. Pair uses a separate median distance for each predicted endpoint class pair. Global uses one common median. In GCN, the resulting neighbor aggregation directly supplies the next layer. HARA combines a node's own representation with messages from neighbors at one and two hops. A learned mixer sets the contribution of these channels. Within each graph channel, refinement updates the representations and recomputes distances and weights. Anchoring mixes the aggregation result with the channel's initial projected representation.

Consequently, a change in message weights is processed through additional steps in HARA. These architectural differences motivate testing the grouping effect separately in each host. To examine this dependence empirically, we evaluated the following settings on Cora and Actor.

SettingRefinement stepsAnchor coefficient
Initial comparison10
Additional refinement20
Main HARA setting20.15

For each setting, Pair and Global share the preliminary classifier, host initialization and remaining training recipe. The preliminary classifier supplies predicted classes and stays frozen after fitting. Within a matched pair, HARA's mixer receives the same preliminary predictions. Models are trained separately for each setting. Evaluation covers clean graphs and shared attacks with 1, 3 and 5 edits, using three seeds.

To read the next table, first compute Pair accuracy minus Global accuracy in each setting. The interaction is how that difference changes when refinement or anchoring changes. A positive interaction means the relative Pair advantage becomes larger. It does not by itself mean that Pair's absolute accuracy rises.

Component changePositive / negative / tied interactions
Increase refinement from one to two steps, with no anchor1 / 7 / 0
Add the anchor with two refinement steps6 / 2 / 0

Each row covers all eight dataset and evaluation combinations, and each sign is determined from the mean over three seeds. Increasing refinement reduced the relative Pair advantage in every shared attack condition in this panel. On Actor, both Pair and Global became more accurate under these attacks, but Global improved more. This illustrates why a smaller Pair minus Global difference cannot be interpreted simply as refinement harming the classifier. The anchor changed the relative advantage in the other direction in most conditions, with dependence on the dataset and budget.

These observations give a concrete reason to assess calibration within each host. They show interactions between grouping and HARA's additional processing, while the separate training runs identify differences between complete settings rather than a unique cause at an individual message. We retain the original main setting for the primary comparison. Thank you for encouraging this analysis, which connects the architectural distinction to measured behavior without attributing HARA's entire performance to calibration alone.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.