acceptodds
Under review as a conference paper at ICLR 2027

Online Unified Graph Pruning for Compact Graph Neural Networks

Abstract

Graph neural networks (GNNs) have achieved strong performance on graph learning tasks, but their deployment is often limited by the computational cost caused by dense graph structures and high-dimensional hidden representations. Existing pruning methods usually focus on either graph sparsification or model compression, and often treat these two pruning dimensions independently. In this paper, we study Online Unified Graph Pruning (OUGP), which jointly learns graph-edge sparsity and parameter sparsity during GNN training. The key idea is to use hidden representations as feedback signals to couple graph pruning and parameter pruning, so that the compact model can preserve task-relevant structural and semantic information. The final model is materialized as a compact GNN with fixed graph and parameter masks. To improve deployment robustness under distribution shift, we further introduce an Energy-Shift Parameter-Mask adaptation module (ESPM), which adjusts parameter selection according to unlabeled target-domain statistics while preserving the compact structure. Experiments compare OUGP+ESPM with dense and existing unified pruning baselines, and evaluate its accuracy, sparsity, efficiency, and robustness under distribution shift. For instance, under 30% graph and 30% model sparsity, OUGP+ESPM improves accuracy by 0.92% on Cora, while reducing training FLOPs by 25% compared with the dense GCN. The code is anonymously available at [link](https://anonymous.4open.science/r/OUGP-release-E92A/).

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Learning, Sparsification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qMvyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes OUGP, a method that learns which graph edges and model weights to keep or remove while training a GNN. To guide pruning, it compares the model’s internal representations under four settings: no pruning, edge pruning only, channel pruning only, and both together. These comparisons help measure how pruning changes the representations and how the two types of pruning interact. The method combines this information with training gradients and a memory module that retains feedback from earlier training steps. The paper also introduces ESPM for adapting the pruned model to new test data. It compares feature-energy statistics between the training and test data, then adjusts parameter scores to select which weights to retain. This requires no test labels and keeps the number of retained parameters unchanged, while allowing the selected weights to change.

Critical strengths

The paper addresses a practically relevant problem. It considers how to reduce graph edges and model weights while maintaining prediction accuracy and adapting the pruned model to new test data. Studying pruning and test-time adaptation together makes the work relevant to deployment settings with limited computational resources and changing data distributions.

Critical weaknesses
  1. ESPM lacks a key control. Under the same distribution-shift conditions, Tables 3 and 4 do not report the performance of the same pruned model without adaptation, making ESPM’s gain over direct deployment unclear. The paper also provides insufficient detail on how the other TTA methods are adapted to the GCN and how their hyperparameters are selected, making the performance gap difficult to interpret.

  2. LHCU’s interaction mechanism is not evaluated in isolation. Table 2 enables or disables LHCU as a whole, without a control that removes only the interaction term while keeping the other components unchanged. It is therefore unclear whether the gains come from the proposed interaction mechanism or could also be achieved with simpler hidden-representation feedback.

Questions for authors
  1. In the linked code, I noticed that hidden_coupling_utilities uses activation–gradient products from a single forward pass as feedback, but I could not find the four-state comparisons and interaction calculations defined in Equations (3)–(5). How are these two implementations related? Which implementation was used for the experiments, ablations, and Figures 5/6? If I overlooked the relevant implementation, could you point me to its exact location?

  2. I would like to clarify ESPM’s independent gains over directly deploying the same pruned model. Could you provide the accuracy of the same frozen model without adaptation under the distribution-shift conditions used in Tables 3/4? Please also explain how Tent, EATA, and SAR are adapted to this GCN, including which parameters are updated, the learning rates and update steps, and how these settings are selected.

  3. Does the non-additive interaction term itself provide additional benefits? Have you conducted a comparison that removes or replaces only this term while keeping the memory module, other hidden-representation feedback, and pruning budget unchanged? This comparison would help demonstrate the advantage of the interaction mechanism over general hidden-representation feedback.

3 comments
AuthorsOpenReview ↗

Response to Reviewer qMvy (Part 1/2)

Dear Reviewer qMvy

Thank you for the detailed review. We have addressed these issues below and updated the manuscript, appendix, and anonymous release.

W1 & Q2: The independent gains of ESPM require comparison with the same frozen model without adaptation under the distribution shifts in Tables 3/4, along with details of the GCN adaptations of Tent, EATA, and SAR.

To show the gain of ESPM, we added results of Frozen OUGP in table 1, 6, 9, using identical source checkpoints, graphs, pruning budgets, shifted features, and test nodes for each shift, direction, and seed. ESPM uses no target labels or optimizer steps and only reselects fixed-size parameter support, leaving source-trained values unchanged.

The iid results:

MethodCoraPubMedCora-fullDBLPACM
Dense87.04 ± 1.2486.36 ± 0.4070.35 ± 0.5585.51 ± 0.3592.58 ± 0.49
OUGP87.89 ± 0.8786.73 ± 0.6866.11 ± 2.7784.81 ± 1.2092.74 ± 1.45
OUGP+ESPM88.04 ± 0.9986.80 ± 0.1366.83 ± 1.8085.33 ± 0.7192.80 ± 1.17

For cross-domain transfer:

MetricDBLPv8→ACMv9ACMv9→DBLPv8
Source acc (OUGP)98.27 ± 0.3883.69 ± 0.61
Frozen OUGP target69.20 ± 2.4163.38 ± 4.17
OUGP+ESPM target71.46 ± 1.8467.69 ± 3.41
ESPM gain (pp)+2.26+4.31

For Tent, EATA, and SAR, all three methods use the same frozen OUGP checkpoint, full deployment graph, 30% source parameter sparsity, 60% test-node feature masking, test split, and seeds 0--3. GCN weights, graph support, and parameter support remain frozen throughout adaptation. The details can be found in appendix, A.2 TEST-TIME ADAPTATION VARIANTS.

A fresh identity-initialized channel-wise affine controller is inserted after the hidden ReLU. It summarizes target-node hidden states using channel-wise mean, standard deviation, and mean absolute activation. A rank-8 projection and readout produce δa\delta_a and δb\delta_b, which are applied as H′=(1+0.10δa)⊙H+0.05δbH'=(1+0.10\delta_a)\odot H+0.05\delta_b. Only the statistics projection and readout are updated. Each method is reset for every dataset and seed and performs 40 full-graph adaptation steps without target labels.

Tent: Uses Adam with a learning rate of 10−310^{-3} and minimizes mean predictive entropy over all target test nodes.

EATA: Uses Adam with a learning rate of 10−310^{-3}. It retains nodes with entropy at most 0.4log⁡C0.4\log C and cosine similarity below 0.95 to a running prediction reference. An L2 anchor with coefficient 10−410^{-4} is added to the initial controller.

SAR: Uses SGD with a learning rate of 5×10−45\times10^{-4} and momentum 0.9. It applies the same filtering and anchoring mechanisms with a two-step sharpness-aware update using ρ=0.05\rho=0.05.

W2 & Q3: Does the non-additive interaction term itself provide additional benefits?

To disentangle the contributions of activation–gradient (AG) feedback and four-state non-additive interaction, we conducted a controlled ablation study under fixed settings of 30% graph and channel sparsity across four seeds. Results are provided in table 4, 11. We compare four variants: No hidden (without hidden-state feedback), AG only (using activation–gradient hidden utility), AG + four-state, no interaction (adding four-state perturbation terms while disabling explicit interaction), and AG + four-state, full interaction (enabling the non-additive term). The joint-mask forward computes HFH_F and remains the only gradient-carrying path. Three additional no-gradient forwards compute HDH_D, HGH_G, and HCH_C.

AG improves Cora accuracy by 0.74±2.380.74\pm2.38 percentage points over No hidden, but has negligible effect on PubMed (+0.02±0.29+0.02\pm0.29). Enabling the interaction term yields a further paired gain of 0.88±0.940.88\pm0.94 points on Cora, whereas PubMed shows a small decrease (−0.16±0.20-0.16\pm0.20). These results suggest complementary contributions from AG and non-additive interaction, with benefits varying across datasets.

MethodCoraPubMed
No hidden84.54 ± 1.8786.54 ± 0.80
AG only85.27 ± 1.8786.56 ± 0.70
AG + four-state, no I\mathcal I84.43 ± 1.8786.56 ± 0.49
AG + four-state, full I\mathcal I85.31 ± 1.3686.40 ± 0.52
AuthorsOpenReview ↗

Response to Reviewer qMvy (Part 2/2)

Q1: The released code and the four-state comparisons and interactions in Eqs. (3)–(5) are not correspond. How are they related, and which implementation was used for the experiments, ablations, and Figs. 5–6?

The Methods section describes an earlier four-state version, whereas the actual experiments used activation–gradient (Act×Grad) utility. We will revise the paper to reflect the implementation used in the reported experiments. Tables 1–4 and the ablation use single-pass activation–gradient utility in src/ougp/model.py::hidden_coupling_utilities, called after one backward pass: Al=∣Hl⊙∂L/∂Hl∣A_l=|H_l\odot\partial\mathcal L/\partial H_l|, which is a first-order Taylor proxy. Node, channel, and edge signals are aggregated, mixed with mask-gradient utility, and stored for score updates. Equations (3)--(5) below and Figures 5/6 in the paper were evaluated through the independent diagnostic runner scripts/run_hidden_state_diagnostic.py.

Equation (3) — Channel Utility

ul,jC=λC1∣V∣∑i∣ΔHC,ij∣+λR1∣V∣∑i∣ΔHG,ijΔHC,ij∣+λI1∣V∣∑i∣Iij∣u_{l,j}^{C} = \lambda_C \frac{1}{|\mathcal V|}\sum_i |\Delta H_{C,ij}| + \lambda_R \frac{1}{|\mathcal V|}\sum_i |\Delta H_{G,ij}\Delta H_{C,ij}| + \lambda_I \frac{1}{|\mathcal V|}\sum_i |\mathcal I_{ij}|.

Equation (4) — Vertex and Edge Utility

uiV=λG∥ΔHG(i,:)∥2+λR∥ΔHG(i,:)⊙ΔHC(i,:)∥2+λI∥I(i,:)∥2u_i^V = \lambda_G\Vert\Delta H_G(i,:)\Vert_2 + \lambda_R\Vert\Delta H_G(i,:)\odot\Delta H_C(i,:)\Vert_2 + \lambda_I\Vert\mathcal I(i,:)\Vert_2,

ueG=us(e)V+ut(e)V2u_e^G = \frac{u_{s(e)}^V+u_{t(e)}^V}{2}.

Equation (5) — Layer-wise Aggregation

rl=∥Il∥F∥ΔHG,l∥F+∥ΔHC,l∥F+ϵr_l = \frac{\Vert\mathcal I_l\Vert_F}{\Vert\Delta H_{G,l}\Vert_F+\Vert\Delta H_{C,l}\Vert_F+\epsilon},

ηl=λL∥ΔHC,l∥F+λIrl+λR∣cos⁡(ΔHG,l,ΔHC,l)∣\eta_l = \lambda_L\Vert\Delta H_{C,l}\Vert_F + \lambda_I r_l + \lambda_R|\cos(\Delta H_{G,l},\Delta H_{C,l})|.

Reviewer_qMvyOpenReview ↗

Response to Authors

Thank you for your response and revisions. I have no further questions. However, given the difference between the activation–gradient (Act×Grad) utility mechanism described in the revised manuscript and the four-state interaction mechanism described in the original submission, I will maintain my original rating.

Reviewer_sgUFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to jointly prune graph edges and GNN parameters while preserving prediction accuracy. It also considers how the resulting compact model can adapt to distribution shifts at deployment. The proposed method, OUGP, uses hidden representations to guide pruning decisions. It compares dense, graph-pruned, model-pruned, and jointly pruned hidden states to estimate pruning damage and interactions. These utilities are combined with gradient signals and low-rank memory to update pruning scores during training. After training, the scores are converted into binary masks. An additional module, ESPM, uses unlabeled target-feature energy statistics to reselect parameter support without gradient updates, while maintaining the parameter budget.

Verification is primarily empirical, supported by complexity analysis rather than theoretical guarantees of predictive performance. The main experiments use a two-layer GCN on five node-classification datasets. Baselines cover graph-only pruning, model pruning, and unified pruning. Additional experiments examine component ablations, varying sparsity budgets, test-time adaptation, and bidirectional cross-domain transfer. The method reports 22–25% lower source-training FLOPs than the dense GCN. Accuracy improves on three of the five main datasets and decreases on two. Transfer results are also mixed: OUGP+ESPM achieves the best target accuracy in one direction but trails UniFews in the other.

Critical strengths
  1. Practical and meaningful problem. Jointly reducing graph and parameter redundancy addresses the computational demands of GNN deployment. Considering distribution shifts also makes the problem relevant beyond clean test settings.
  2. Intuitively motivated design. Using hidden-state perturbations to guide joint pruning is reasonable, since graph and parameter pruning can interact through the learned representations. The four-way comparison provides an interpretable way to examine these interactions.
Critical weaknesses
  1. The evaluation has limited scope and reporting gaps. The main experiments focus on five citation/academic-network datasets using a two-layer GCN with hidden width 32. Larger or structurally different graphs would strengthen the generality claims. The main results omit variability across runs, making small improvements difficult to assess. Additionally, the GCNP baseline description does not match the cited method, leaving its implementation unclear.

    • The deployment state and complete computational costs are unclear. The paper states that discarded parameter entries are removed, yet ESPM subsequently reselects parameter support. It is unclear how reactivated weights remain available and whether a full weight bank must be stored. The reported training FLOPs also do not clearly account for hidden-state evaluations, memory operations, and pruning-score optimization, limiting the evidence for compactness and practical efficiency.pruning-score optimization. These ambiguities limit the evidence for compactness and practical efficiency.
Questions for authors

My concerns and questions are covered in weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer sgUF

Dear Reviewer sgUF

Thank you for your detailed review and valuable feedback. We have addressed these issues below and updated the manuscript, appendix, and anonymous release.

W1: The evaluation has limited scope and reporting gaps. Additionally, the GCNP baseline description does not match the cited method.

Thank you for pointing out the limit of our experimental coverage. We have added experiments on additional graphs and included the results in the revised paper. We also reran all main methods and report the mean ± sample standard deviation in the revised table 1, 6, 9. OUGP+ESPM achieves the best accuracy on Cora, improving over Dense by 1.00 pp, while also improving over Dense by 0.44 pp on PubMed. On Amazon Computers, OUGP+ESPM achieves the highest accuracy among the evaluated methods, reaching 74.59 ± 0.55, exceeding Dense by 5.95 pp. Regarding the inconsistency of GCNP citation, We corrected it to the channel-pruning GCNP method of Zhou et al., Accelerating Large Scale Real-Time GNN Inference using Channel Pruning (PVLDB 2021).

W2: ESPM's deployment compactness conflict and the completeness of the reported training FLOPs.

We distinguish parameter storage overhead from computational cost. ESPM retains the frozen full source-trained weight bank to support parameter reselection, while keeping the active parameter support fixed. This introduces a limited storage overhead but does not increase the active parameter budget. We further profile forward/backward computation, memory operations, and pruning-related updates, etc., to quantify their runtime overhead. The results are provided in appendix table 10. The primary benefit of pruning lies in reducing the parameters involved in computation, rather than eliminating the storage of inactive weights. For example, on DBLP, the peak GPU memory is 1,527.94 MiB, whereas the stored deployment artifact for DBLPv8→ACMv9 occupies only 3.124 MiB, corresponding to approximately 0.20% of the peak memory. The artifact for ACMv9→DBLPv8 is similarly small at 3.160 MiB. For a more detailed assessment of training costs, we conducted additional experiments. Compared with Dense, OUGP reduces materialized deployment artifact size by 12.3–23.8%, but requires 3.06–3.37× training-only wall-clock time and 1.06–9.20× peak GPU memory across five evaluated datasets.

Reviewer_EYKhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OUGP, which jointly optimizes graph-edge and parameter sparsity using hidden-state perturbations and their interactions. It further introduces ESPM, which adjusts parameter selection at test time using unlabeled target-domain statistics.

Critical strengths
  1. Jointly considering graph and parameter pruning is reasonable and a relevant direction for efficient GNN deployment.
  2. The paper evaluates multiple pruning baselines and includes component ablations and sparsity sensitivity analyses.
Critical weaknesses
  1. ESPM has a fundamental logical inconsistency. If mask re-selection is restricted to the retained parameters, keeping the same Top-K budget makes adaptation ineffective. If previously pruned parameters can be reactivated, their weights must remain stored, contradicting the claimed compact deployment model.

  2. The benefit of hidden-state coupling is weakly supported. Figure 5(b) shows relatively small interaction ratios. In Table 2, adding LHCU improves Cora accuracy by only 0.22 pp with memory, while slightly reducing Photo accuracy without memory. These results do not convincingly justify the proposed coupling mechanism.

  3. Hidden-state perturbation is not necessarily task-relevant importance. Large representation changes do not necessarily imply larger task loss. Moreover, Eq. (4) estimates edge importance from endpoint utilities rather than measuring each edge's actual contribution.

  4. The necessity of the memory mechanism is unclear. Pruning scores are already optimized using task gradients, yet the method introduces an additional memory module. A comparison with simple EMA-based importance accumulation is missing.

  5. The connection between OUGP and ESPM is unclear to me. ESPM does not use hidden-state coupling or recompute LHCU, relying instead on source parameter scores and target feature energy. It appears to be an independent adaptation module rather than an integral component of the proposed joint pruning framework.

  6. Similarly, Table 1 reports OUGP+ESPM without an OUGP-only baseline, while ESPM is not applied to competing pruning methods. The experiments therefore cannot establish whether ESPM's gains specifically depend on OUGP.

  7. Changes in channel-wise squared feature energy do not necessarily indicate changes in task-relevant parameter importance. Comparisons with frozen masks, random mask re-selection, and simpler importance-based adaptation are missing.

Questions for authors

Please refer to weakness.

1 comment
AuthorsOpenReview ↗

Response to Reviewer EYKh

Rebuttal for Reviewer EYKh

Thank you for the detailed review. We report matched experiments and the revisions made below.

W1: ESPM re-selects parameters from the full parameter set, conflicting with the claimed storage and computation reduction.

Compact refers to the active parameter budget during inference rather than physical storage savings. In the revised manuscript, we explicitly clarify that ESPM does not reduce storage footprint, and we clearly separate active parameter count and inference efficiency from storage space and adaptation latency.

W2: LHCU is weakly supported by Figure 5(b) and Table 2.

Modest gains do not show LHCU is ineffective. After rechecking the code, we found that the original Table 2 was produced using the previous four-state LHCU, which estimated coupling from perturbed hidden representations, and was not updated after the method revision. The repository uses single-pass activation–gradient LHCU to derive task-sensitive utilities without extra perturbation passes. We reran the 2×2 ablation using the repository code, the same budget, and four seeds. The results are reported below, and the corresponding table has been added to the revised manuscript.

MemoryLHCUCoraPhoto
✗✗86.08 ± 0.37%89.19 ± 0.36%
✗✓86.63 ± 0.28%89.27 ± 0.10%
✓✗86.41 ± 0.34%89.43 ± 0.33%
✓✓86.08 ± 0.47%89.64 ± 0.24%

LHCU gives +0.55 pp on Cora without Memory and +0.08/+0.21 pp on Photo without/with Memory. The only decrease is −0.33 pp on Cora with Memory, where both mechanisms refine the same scores. It helps in three of four settings, although gains are not uniformly additive. The small Figure 5(b) ratios indicate selective coupling rather than ineffectiveness. We revised the paper to report all settings and characterize LHCU as effective but configuration dependent.

W3: Hidden perturbation is not evidence that an edge is important to the downstream task.

We agree that endpoint perturbation alone cannot establish task importance. This analysis belongs to the original prototype. The current formulation no longer uses endpoint perturbation or Eq. 4 to construct pruning scores. Its single-pass utility is computed from activations and gradients during source training. We removed the outdated causal interpretation and distinguished the legacy diagnostic from the current scoring pipeline.

W4: The memory may be functionally similar to a simple EMA accumulator.

EMA applies fixed exponential smoothing, whereas our memory predicts a context-dependent residual from historical pruning states to modify current scores. Under the same protocol and four seeds:

DatasetSimple EMAFull OUGP memory
Cora86.60 ± 0.35%86.16 ± 0.71%
PubMed87.14 ± 0.09%87.24 ± 0.06%

EMA is stronger on Cora and our memory is stronger on PubMed. Historical smoothing is useful, but our context-dependent residual correction is not equivalent to fixed-decay EMA. We therefore describe Memory as a distinct module with dataset-dependent value.

W5: The relationship between OUGP and ESPM is unclear.

ESPM uses the scores, weight structure, source statistics, and keep budget learned by OUGP, then reselects parameter support from target statistics while preserving the budget. It uses no target labels, backward pass, or optimizer update. OUGP provides the source pruning basis and ESPM performs forward-only target-aware deployment adaptation. We made this one-way dependency explicit in the framework diagram.

W6: The main table does not isolate OUGP-only or compare ESPM with parameter-pruning baselines.

We added matched OUGP-only and OUGP+ESPM controls:

DatasetOUGP onlyOUGP + ESPM
ACM91.39 ± 1.12%91.95 ± 0.57%
Cora86.34 ± 0.69%85.60 ± 1.29%
Cora-full62.71 ± 2.35%63.51 ± 2.26%

ESPM yields an average gain of 0.21 pp over OUGP across the three datasets. Under the same four-seed IID protocol and keep budget, its changes with Magnitude and SNIP are +0.59/−1.40/−1.11 pp and +0.59/−0.11/−0.20 pp on ACM/Cora/Cora-full, respectively. Gains for both generic baselines occur only on ACM, suggesting that ESPM is more compatible with OUGP-derived scores and masks than with generic pruning baselines.

W7: Channel-wise feature energy is not necessarily related to task-relevant parameter importance.

We performed channel-occlusion calibration by masking each channel with the model, graph, and remaining parameter support fixed, then measuring the validation-loss increase. Spearman correlations across four seeds are −0.185 ± 0.194 on Cora and −0.290 ± 0.314 on PubMed. The correlation is not universally monotonic. We therefore treat energy as a target-shift signal for re-ranking an OUGP-derived mask, not as direct task importance, and added this calibration to the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.