- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes OUGP, a method that learns which graph edges and model weights to keep or remove while training a GNN. To guide pruning, it compares the model’s internal representations under four settings: no pruning, edge pruning only, channel pruning only, and both together. These comparisons help measure how pruning changes the representations and how the two types of pruning interact. The method combines this information with training gradients and a memory module that retains feedback from earlier training steps. The paper also introduces ESPM for adapting the pruned model to new test data. It compares feature-energy statistics between the training and test data, then adjusts parameter scores to select which weights to retain. This requires no test labels and keeps the number of retained parameters unchanged, while allowing the selected weights to change.
Critical strengths
The paper addresses a practically relevant problem. It considers how to reduce graph edges and model weights while maintaining prediction accuracy and adapting the pruned model to new test data. Studying pruning and test-time adaptation together makes the work relevant to deployment settings with limited computational resources and changing data distributions.
Critical weaknesses
-
ESPM lacks a key control. Under the same distribution-shift conditions, Tables 3 and 4 do not report the performance of the same pruned model without adaptation, making ESPM’s gain over direct deployment unclear. The paper also provides insufficient detail on how the other TTA methods are adapted to the GCN and how their hyperparameters are selected, making the performance gap difficult to interpret.
-
LHCU’s interaction mechanism is not evaluated in isolation. Table 2 enables or disables LHCU as a whole, without a control that removes only the interaction term while keeping the other components unchanged. It is therefore unclear whether the gains come from the proposed interaction mechanism or could also be achieved with simpler hidden-representation feedback.
Questions for authors
-
In the linked code, I noticed that
hidden_coupling_utilitiesuses activation–gradient products from a single forward pass as feedback, but I could not find the four-state comparisons and interaction calculations defined in Equations (3)–(5). How are these two implementations related? Which implementation was used for the experiments, ablations, and Figures 5/6? If I overlooked the relevant implementation, could you point me to its exact location? -
I would like to clarify ESPM’s independent gains over directly deploying the same pruned model. Could you provide the accuracy of the same frozen model without adaptation under the distribution-shift conditions used in Tables 3/4? Please also explain how Tent, EATA, and SAR are adapted to this GCN, including which parameters are updated, the learning rates and update steps, and how these settings are selected.
-
Does the non-additive interaction term itself provide additional benefits? Have you conducted a comparison that removes or replaces only this term while keeping the memory module, other hidden-representation feedback, and pruning budget unchanged? This comparison would help demonstrate the advantage of the interaction mechanism over general hidden-representation feedback.
3 comments
Response to Reviewer qMvy (Part 2/2)
Q1: The released code and the four-state comparisons and interactions in Eqs. (3)–(5) are not correspond. How are they related, and which implementation was used for the experiments, ablations, and Figs. 5–6?
The Methods section describes an earlier four-state version, whereas the actual experiments used activation–gradient (Act×Grad) utility. We will revise the paper to reflect the implementation used in the reported experiments. Tables 1–4 and the ablation use single-pass activation–gradient utility in src/ougp/model.py::hidden_coupling_utilities, called after one backward pass: , which is a first-order Taylor proxy. Node, channel, and edge signals are aggregated, mixed with mask-gradient utility, and stored for score updates. Equations (3)--(5) below and Figures 5/6 in the paper were evaluated through the independent diagnostic runner scripts/run_hidden_state_diagnostic.py.
Equation (3) — Channel Utility
.
Equation (4) — Vertex and Edge Utility
,
.
Equation (5) — Layer-wise Aggregation
,
.
Response to Authors
Thank you for your response and revisions. I have no further questions. However, given the difference between the activation–gradient (Act×Grad) utility mechanism described in the revised manuscript and the four-state interaction mechanism described in the original submission, I will maintain my original rating.
Response to Reviewer qMvy (Part 1/2)
Dear Reviewer qMvy
Thank you for the detailed review. We have addressed these issues below and updated the manuscript, appendix, and anonymous release.
To show the gain of ESPM, we added results of Frozen OUGP in table 1, 6, 9, using identical source checkpoints, graphs, pruning budgets, shifted features, and test nodes for each shift, direction, and seed. ESPM uses no target labels or optimizer steps and only reselects fixed-size parameter support, leaving source-trained values unchanged.
The iid results:
For cross-domain transfer:
For Tent, EATA, and SAR, all three methods use the same frozen OUGP checkpoint, full deployment graph, 30% source parameter sparsity, 60% test-node feature masking, test split, and seeds 0--3. GCN weights, graph support, and parameter support remain frozen throughout adaptation. The details can be found in appendix, A.2 TEST-TIME ADAPTATION VARIANTS.
A fresh identity-initialized channel-wise affine controller is inserted after the hidden ReLU. It summarizes target-node hidden states using channel-wise mean, standard deviation, and mean absolute activation. A rank-8 projection and readout produce δa and δb, which are applied as H′=(1+0.10δa)⊙H+0.05δb. Only the statistics projection and readout are updated. Each method is reset for every dataset and seed and performs 40 full-graph adaptation steps without target labels.
Tent: Uses Adam with a learning rate of 10−3 and minimizes mean predictive entropy over all target test nodes.
EATA: Uses Adam with a learning rate of 10−3. It retains nodes with entropy at most 0.4logC and cosine similarity below 0.95 to a running prediction reference. An L2 anchor with coefficient 10−4 is added to the initial controller.
SAR: Uses SGD with a learning rate of 5×10−4 and momentum 0.9. It applies the same filtering and anchoring mechanisms with a two-step sharpness-aware update using ρ=0.05.
To disentangle the contributions of activation–gradient (AG) feedback and four-state non-additive interaction, we conducted a controlled ablation study under fixed settings of 30% graph and channel sparsity across four seeds. Results are provided in table 4, 11. We compare four variants: No hidden (without hidden-state feedback), AG only (using activation–gradient hidden utility), AG + four-state, no interaction (adding four-state perturbation terms while disabling explicit interaction), and AG + four-state, full interaction (enabling the non-additive term). The joint-mask forward computes HF and remains the only gradient-carrying path. Three additional no-gradient forwards compute HD, HG, and HC.
AG improves Cora accuracy by 0.74±2.38 percentage points over No hidden, but has negligible effect on PubMed (+0.02±0.29). Enabling the interaction term yields a further paired gain of 0.88±0.94 points on Cora, whereas PubMed shows a small decrease (−0.16±0.20). These results suggest complementary contributions from AG and non-additive interaction, with benefits varying across datasets.