- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The motivation is clearly illustrated with intuition (Figure 1) and schematic diagrams (Figure 2). Mathematical formulations are well-defined, and experimental protocol details are clearly articulated.
Critical strengths
Standard distance-based pruning often destroys legitimate cross-class edges because inter-class distance distributions naturally differ from intra-class distributions. CLASP directly resolves this with a fine-grained, class-pair specific reference scale without requiring ground-truth labels during test time.
Critical weaknesses
HARA receives extended hyperparameter tuning (additional search budgets targeting specific dataset-attack combinations guided by earlier test outcomes), whereas fixed baselines use pre-recorded configurations. This protocol renders main system comparisons descriptive rather than strictly controlled.
CLASP relies entirely on a frozen preliminary 1-layer GCN to assign edge buckets. Under severe structural attacks or high heterophily, if the preliminary classifier produces inaccurate predictions or misclassifies noisy neighborhoods, edge buckets become corrupted ( grows large), which degrades median calibration.
Questions for authors
How sensitive is CLASP's performance to the accuracy and architecture of the preliminary classifier? Have you evaluated replacing the 1-layer GCN with a stronger or more robust preliminary model?
4 comments
Response to W2: Prediction error and bucket stability
Thank you for highlighting the role of preliminary predictions under heterophily and structural perturbations. We examined prediction accuracy and changes in bucket membership separately, because they measure different aspects of the concern. Our calibration rule operates on predicted groups, and the theoretical analysis explicitly allows those groups to change after an attack.
The preliminary classifier is fitted first and then frozen, meaning its learned parameters stop changing. Its predictions are nevertheless recomputed on every perturbed graph. For each directed edge, CLASP takes the predicted class of the sending node followed by that of the receiving node. Edges with the same ordered pair enter the same bucket. Their median representation distance provides the reference used to normalize each distance before robust weighting. The true test labels are not used to construct these buckets.
An incorrect predicted pair and a changed predicted pair are therefore different events. A pair can be incorrect on both the clean and perturbed graphs while remaining in the same bucket. Conversely, a changed prediction can move an existing edge between buckets even when the attack does not edit that edge. The former concerns agreement with true classes. The latter concerns changes to the set of distances used to compute a bucket's median.
We measured both quantities for the frozen one layer GCN on Cora and Actor. The following results use shared attacks with five edits.
| Dataset | Ordered pair accuracy (%) | Surviving edge bucket migration (%) |
|---|---|---|
| Cora | 69.32 ± 0.14 | 0.0407 ± 0.0002 |
| Actor | 9.67 ± 0.06 | 0.0231 ± 0.0014 |
Ordered pair accuracy requires both endpoints to be predicted correctly and is measured over all labeled directed edges other than self loops. Migration is the fraction of surviving edges whose ordered predicted pair changes from the clean graph. Inserted and deleted edges are excluded from this migration denominator. We average 100 attack graphs within each seed, then report the mean and sample standard deviation over three seeds. The two columns have distinct denominators and should not be interpreted as complementary percentages.
The measurements distinguish persistent prediction error from the additional reassignment induced by these attacks. Small migration over the whole graph describes membership stability at that scale. Correctness and behavior at an individual attacked target are separate quantities. We therefore evaluate the usefulness of the resulting groups through downstream prediction experiments as well.
Proposition 1 concerns a bucket defined by predicted classes. Its retained majority condition means that more than half of the edges in the perturbed bucket were already in the corresponding clean bucket. A bound on how much those retained distances move then bounds the new median using their clean distance ordering. The premise is about retained membership and distance drift, rather than requiring every predicted class to match its true label. Its conclusion concerns the calibration statistic, with final classification assessed empirically.
To address dependence on the original preliminary model directly, we also completed controls using an MLP, a two layer GCN and RUNG in addition to the one layer GCN, with both downstream hosts on clean and shared evaluations. Thank you for motivating these measurements. They assess the method using actual prediction errors and alternative preliminary architectures rather than assuming an ideal source of class labels.
Response to the Question: Sensitivity to the preliminary architecture
Thank you for suggesting a direct test with alternative preliminary architectures. We completed that comparison using a one layer GCN, a multilayer perceptron (MLP), a two layer GCN and RUNG. This tests the method beyond its original preliminary classifier and examines how the resulting calibration effect changes with the provider, dataset and downstream host.
The preliminary classifier supplies predicted node classes. CLASP groups directed edges by the ordered predictions at their endpoints and uses each group's median distance as a reference. The host is the final classifier that consumes the calibrated messages. We evaluate both GCN and HARA hosts. Within each provider and host setting, Pair uses the predicted class pair groups and Global uses one common group. Their starting host weights and training recipe are matched, and both use the same fitted preliminary classifier.
The completed panel covers Cora and Actor, clean graphs and shared attacks with 1, 3 and 5 edits, using three seeds. Preliminary models are selected by validation loss and frozen before downstream fitting. Predictions are recomputed on each evaluation graph. RUNG supplies an architecture with robust propagation and is trained using clean cross entropy in this experiment.
The table reports the largest shared attack budget for all four providers and both hosts. Each entry is Pair accuracy minus Global accuracy in percentage points. Positive values favor class pair grouping.
| Preliminary model | Cora GCN host | Actor GCN host | Cora HARA host | Actor HARA host |
|---|---|---|---|---|
| One layer GCN | +8.00 ± 1.73 | +1.00 ± 4.00 | +2.00 ± 3.00 | +1.33 ± 1.53 |
| MLP | −1.00 ± 2.65 | +1.00 ± 3.46 | −3.00 ± 4.58 | +0.00 ± 2.65 |
| Two layer GCN | +2.00 ± 3.61 | +4.33 ± 2.08 | −5.67 ± 5.13 | −2.67 ± 2.08 |
| RUNG | −9.33 ± 3.79 | +3.33 ± 3.06 | −11.00 ± 7.55 | +3.33 ± 3.06 |
All shared evaluations use the same 100 predetermined targets and stored attack graphs. We first calculate the difference within each seed and then report its mean and sample standard deviation across seeds. This includes all targets, including those initially classified incorrectly. The standard deviation describes the variability of the registered repeats.
The results show that gains occur with several preliminary architectures, while their direction and size depend on the host and dataset. Across the full panel, the attack budget also matters. For example, RUNG on Actor has positive GCN grouping gain at the displayed budget but negative gains at the smaller shared budgets.
We also measured preliminary quality directly. On Cora under the same attack budget:
| Preliminary model | Test node accuracy (%) | Ordered pair accuracy (%) |
|---|---|---|
| One layer GCN | 75.16 ± 0.12 | 69.32 ± 0.14 |
| RUNG | 81.30 ± 0.49 | 76.19 ± 0.43 |
Node accuracy uses the full test mask. Pair accuracy requires both endpoints to be correct over all labeled directed edges other than self loops. Each quality metric first averages the 100 attack graphs within a seed. RUNG has higher preliminary accuracy here, yet its grouping gain is smaller. Thus, preliminary accuracy alone does not determine the observed calibration benefit.
These are sensitivity measurements of complete fitted pipelines. Architecture changes several properties at once, some GCN Global control models also learn different weights across preliminary architectures, and HARA uses preliminary predictions in its mixer as well as calibration. The controls therefore quantify observed sensitivity without isolating classifier accuracy as a single cause. Thank you for encouraging this broader evaluation, which establishes compatibility with alternative preliminary models and makes their measured effects explicit.
Dear Reviewer uQJ9,
Thank you for raising the question of comparable tuning budgets and for suggesting that we test alternatives to the preliminary GCN. In our supplementary replies, we describe how we gave all six methods the same number of tuning trials and report how the effect of calibration varied across four preliminary classifiers on Cora and Actor.
When you have a chance, we would be grateful to hear whether these results address your concerns. If any part of the comparison would benefit from further explanation, we would be happy to clarify it. Thank you for your time and thoughtful feedback.
Response to W1: Controlled system and calibration comparisons
Thank you for distinguishing the performance of a selected system from a controlled test of its methodological contribution. We addressed this with two completed experiments that answer complementary questions. The first gives every system the same registered search opportunities. The second holds the training recipe fixed and changes only calibration grouping within a host.
The system comparison includes GCN, RUNG, GCN Global, GCN Pair, HARA Global and HARA Pair. Thus, the existing robust baseline and both Global controls receive a new search alongside the proposed variants. Each method receives the following schedule for every dataset.
This changes the comparison from newly tuned methods against previously fixed baseline settings to methods with the same scheduled search opportunities. Common epoch and early stopping caps also constrain training. Equal candidate counts refer to search opportunities, while the computation consumed by each fit can depend on the architecture and when training stops. The new test results were not used to revise the candidate lists, selected configurations or attack scope.
The grouping experiment addresses attribution more directly. Here, a host means the final network that predicts node classes. Global normalizes edge distances using one common median. Pair, our CLASP rule, uses a median for each predicted endpoint class pair. Both variants use the same frozen preliminary classifier to supply predictions, and those predictions are recomputed on each evaluation graph. The host starts from matched weights and follows the same training recipe. Each variant is then trained separately.
Within HARA, the node feature channel, one hop and two hop channels, mixer, refinement and anchor remain unchanged between Pair and Global. The mixer also retains the same preliminary predictions. Therefore, the experimental difference concerns the aggregation grouping rather than introducing HARA's extra components along with it. This is the appropriate comparison for identifying the contribution of CLASP at the fixed recipe, while the system experiment evaluates the independently selected complete methods.
Both studies use the existing benchmarks, whose earlier test results had already been observed. The prospective seal makes the new selection sequence explicit but does not remove that history or create a new independent test set. Thank you for highlighting this distinction. The additional evidence now separates comparisons of complete systems from tests of calibration grouping, so each conclusion can be assessed against the corresponding control.