acceptodds
Under review as a conference paper at ICLR 2027

Effective Depth: A Scalar Correction Closes the Oversmoothing Gap in Trained GNNs

Abstract

Asymptotic analyses of oversmoothing in graph neural networks assume linear dynamics, i.i.d. layer weights, and constant aggregation across depth. Trained networks violate all three, and these violations are widely cited as the reason the theory fails in practice. Across 324 training runs spanning three architectures, six depths (2–64), and homophilous and heterophilous benchmarks, we ask which violation matters. The i.i.d. assumption is violated severely – the top singular value of trained weights rises from 0.97× to 2.15× the Marchenko–Pastur bulk edge, pooled across architectures – yet trained networks sit no further from the linear i.i.d. prediction than randomly initialized ones (mean absolute slope error 0.071 against 0.188). What explains the gap is one scalar: the total branch gain∑︁ ℓ ∥hℓ∥/∥xℓ∥ in the residual stream, the network’s effective depth. Training collapses it from 49% of nominal depth at L = 8 to 14% at L = 64, following Leff ≈ cLk with k = 0.39±0.04. Supplying that scalar to an otherwise unchanged linear i.i.d. simulation cuts prediction error by 83% at depth 32 and 89% at depth 64. Its layerwise distribution carries no additional information: no profile of equal total predicts worse than the measured one – uniform, reversed, sorted, and three independent shuffles all fall within 0.80–1.08× (p > 0.18) – while halving the total more than doubles the error (2.16×, p < 10−5). Fitting the exponent on three depths predicts a held-out depth for the same dataset and architecture at about 1.1 times the oracle’s error. Finally we intervene: regularizing total gain in 32-layer GCNs sets it precisely (r = 0.992) and moves decay as predicted (r = −0.943), with unregularized networks on the same curve. Where the constraint binds, decay varies several-fold at no measurable accuracy cost on Cora and Chameleon; Texas, the smallest graph, pays a real cost, and at extreme targets accuracy tracks distance from a network’s own operating point at least as well as it tracks decay. Primary claims are established for GCN, the one architecture where propagation operator and energy metric coincide

Primary area:
learning on graphs and other geometries & topologies
Keywords:
graph neural networks, oversmoothing, Effective depth, Dirichlet energy

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_pEXpOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates oversmoothing predictions for trained residual GNNs using an effective-depth statistic, Leff=∑ℓ∥hℓ∥F∥xℓ∥FL_{\mathrm{eff}}=\sum_{\ell}\frac{\lVert h_{\ell}\rVert_F}{\lVert x_{\ell}\rVert_F}. It reports that effective depth grows sublinearly with nominal depth and that incorporating this scalar into a random-weight simulator improves predictions of normalized Dirichlet-energy decay, particularly for deep GCNs. The evaluation includes gain-profile perturbations, prediction across depths, and training interventions that regularize effective depth toward specified targets. The central empirical observation is promising, although the claimed connection to existing theory and the interpretation of the supporting evidence require clarification.

Critical strengths
  1. Potentially useful empirical predictor. Table 6 (p. 14) reports substantial improvements for residual GCNs: prediction error decreases from 0.071 to 0.012 at depth 32 and from 0.051 to 0.0055 at depth 64. Section 4 also reports improvement across all 12 GCN configurations. These findings suggest that accumulated relative branch magnitude captures useful information about trained dynamics.

  2. Complementary experimental evidence. The comparison of full and uniform gain profiles, prediction across depths, and regularization interventions goes beyond a single observational correlation. The intervention provides evidence of controllability within the tested range. The authors also explicitly discuss prediction bias, architecture-specific failures, and limitations of the mechanistic interpretation.

Critical weaknesses
  1. The theoretical attribution and mathematical justification do not support the central framing. Section 2.1 (p. 2) and Appendix A.4 (p. 11) attribute an i.i.d. weight assumption to Oono and Suzuki (2020). Their main contraction theorem instead uses layerwise spectral norms and does not require random weights. Although Appendix A.4 allows nonlinearities to be handled through contraction, it does not resolve this attribution problem. The manuscript needs to distinguish its simulator assumptions from those of the cited theorem and establish the connection to normalized-energy slopes. See Oono and Suzuki, Sections 3.2–3.3, pp. 3–4.

    Separately, Section 2.4 (p. 3) infers a magnitude-only explanation from predominantly nonnegative cross-terms. However,

    E^(x+h)=E(x)+E(h)+2B(x,h)∥x∥F2+∥h∥F2+2⟨x,h⟩F.\widehat{E}(x+h)=\frac{E(x)+E(h)+2B(x,h)}{\lVert x\rVert_F^2+\lVert h\rVert_F^2+2\langle x,h\rangle_F}.

    The sign of B(x,h)B(x,h) alone cannot determine the change in normalized energy. Branch energy and alignment remain relevant. This undermines the proposed justification, although it does not invalidate the empirical prediction improvements.

  2. The evidence supports narrower conclusions, and statistical reporting requires reconciliation. The initialization-versus-training comparison (Section 3, pp. 4–5) changes multiple properties simultaneously and therefore does not isolate the contribution of non-i.i.d. structure. The authors appropriately limit profile ablations to simulator predictions (p. 7), but nonsignificant differences do not establish equivalence; the introduction’s stronger exclusion of distributed mechanisms should be narrowed.

    Table 1 (p. 12) specifies four target-level means, yet r=−0.70r=-0.70 gives approximately p=0.30p=0.30 under a standard two-sided Pearson test at n=4n=4, rather than the reported 0.011. This may reflect a caption or aggregation mismatch. Section 6 (p. 8) also compares intervention error 0.013 against observational depth-32 P4 error 0.027, whereas Table 6 reports 0.012 for the latter. These discrepancies need resolution before the associated significance and predictability claims can be assessed.

Questions for authors
  1. Which precise theoretical result motivates the i.i.d. assumption, and how does it connect the residual simulator to the normalized-energy slopes evaluated here?
  2. After accounting for normalization, what existing evidence supports excluding branch alignment and spectral content from the explanation?
  3. What statistical unit was used in Table 1, and what analysis produced the 0.027 comparator in Section 6? Which conclusions remain supported after these discrepancies are reconciled?
Reviewer_XD5COpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the discrepancy between the theoretical and empirical oversmoothing rate in graph neural networks. It analyzes the violation of i.i.d. layer weights, one of three key assumptions the authors claim underlie the theoretical analysis of GCNs in particular, and concludes that it is not responsible for the observed gap in residual GCNs through a series of experiments that measure and control ‘effective depth’, a concept introduced by the paper to quantify the total contribution, in terms of norm magnitude, of the residual branch relative to the identity branch to oversmoothing, as commonly measured by the decay in Dirichlet energy.

Critical strengths
  • The paper makes an effort to further the understanding of oversmoothing in trained residual GCNs, which is a widely studied phenomenon in GNNs but often limited to the development of empirically validated but not theoretically well-understood mitigation strategies, of which residual links are a primary example.
  • The concept of effective depth and related experiments reveal some interesting insights.
Critical weaknesses
  1. The paper claims that three assumptions, i.e., linear dynamics (A1), i.i.d. weights (A2), and constant aggregation (A3), underlie the theoretical analysis of oversmoothing by Oono & Suzuki (2020), on which this paper is based. However, as far as I understand, A1 and A2 are not assumed explicitly by Oono & Suzuki. Their section 3.2 states use of ReLU as the activation function, and Assumptions 1 and 2 listed in section 3.2 do not necessitate iid weights, although they use iid Gaussian weights in their experiments. Furthermore, the theory is developed for non-residual GNNs, yet the experiments are conducted on residual GNNs. Why is this a valid comparison?

  2. The paper’s main aim is unclear. Even if A1 and A2 are correctly attributed assumptions, the research question posed in the introduction in ‘which violation actually carries the discrepancy for trained, fine-depth, residual networks?’. Yet only A2 is analyzed explicitly. I acknowledge the authors explain why it is still necessary for A3 to hold by construction, but there are no controlled experiments for A1. The (non)linearity of the anlaysis/experiments in the paper is also vague. For example, what do lines 164-167 mean? While I understand that the paper attempts to provide insights into oversmoothing and effective depth in trained GNNs, the implications of these are not discussed clearly. For example, what does it mean for model performance/accuracy?

  3. Since the oversmoothing rate is dependent on the spectrum of the propagation operator and thus the graph connectivity structure, the datasets used, although they vary in homophily levels, are all small. Both Cora and Chameleon have very small spectral gaps, and while Texas has a larger gap, the authors mention that its results are not conclusive. Also, the authors mention that they retain the duplicate nodes of Chameleon for comparison with prior work, but the experiments in the paper compare only their own simulation protocols. Wouldn’t an empirical oversmoothing analysis, as done in the paper, that takes into account more varying structural and task-related properties, such as long-range benchmarks, be more insightful?

Questions for authors
  1. Do the trained GNNs in the experiments use node features? If so, is it a fair comparison against the simulation references, which use Gaussian noise instead?

Also, the questions accompanying each weakness (1-3).

Reviewer_roqEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper examines why the asymptotic theory of GNN oversmoothing (Oono & Suzuki, 2020) does not match trained networks. The theory assumes linearity, i.i.d. weights, and fixed aggregation, all of which trained networks violate, and the authors ask which violation actually explains the gap. To answer this, the paper compares the layerwise oversmoothing rates of trained networks with those of linear simulations at the same depth. The results indicate that the i.i.d. weight assumption is not the cause. Instead, the authors define an effective depth that measures how much of its nominal depth a trained network actually uses. Adding only this value to the simulations greatly reduces prediction error, while the layerwise distribution provides no further information. An intervention experiment further suggests that regularizing effective depth during training changes the oversmoothing rate in the predicted direction. The main claims are restricted to trained GCNs.

Critical strengths
  1. The core question of how well standard theoretical assumptions (linear dynamics, i.i.d. weights, and fixed aggregation) hold in trained GCNs is straightforward, and the authors divided their analysis into detailed parts.

  2. The paper reports its limitations and failure cases transparently (e.g., the different trends in accuracy and LeffL_{\text{eff}}), which makes it easy to judge the scope of its claims.

Critical weaknesses
  1. The paper’s presentation needs to be clearer. The introduction lists many variables and numbers without explaining what they mean, making it difficult to judge the importance of the results or how they advance prior work. The analysis also reports many statistics that do not directly support the main claims. The authors should focus the presentation on the main results and analyses.

  2. The practical significance and scope of the findings are limited. Identifying which of the three assumptions (A1-A3) does not explain the gap is theoretically useful, but the paper does not explain how this finding could be applied in practice. For example, in Section 7, changing the decay rate has little effect on accuracy for Cora and Chameleon, making it unclear why accurately predicting that rate matters. The main claims are also limited to GCNs, and the narrow experimental setting (only three datasets and a single hidden width) makes it difficult to generalize the authors' conclusions.

Questions for authors
  1. In Section 2.4 (Eq. 3), B≥0B \ge 0 in 94.9% of layers is used as evidence that the branch does not counteract smoothing and affects decay only through its magnitude, motivating LeffL_{\text{eff}}. However, BB decomposes the unnormalized EE, whereas smoothing is measured using the normalized E^\hat{E}, so the sign of BB alone does not show whether the branch smooths or sharpens. Could you explain why the sign of BB is valid for this distinction, or directly report the fraction of layers where E^(xℓ+hℓ)>E^(xℓ)\hat{E}(x_\ell + h_\ell) > \hat{E}(x_\ell)? If this fraction is large, a magnitude-only measure like LeffL_{\text{eff}} may not be sufficient.

  2. Since many non-residual networks failed to train, the analysis only covers residual ones. However, the theory is for non-residual GNNs, and LeffL_{\text{eff}} is only defined for residual networks. Could you clarify whether the conclusions are expected to hold in the theory's original setting?

  3. The simulation starts from Gaussian noise rather than dataset features, but the fitted decay slope at finite depth may depend on the composition of the initial signal. Do the P1-P4 comparisons still hold when the simulation starts from the real input features?

  4. The main claims focus on GCNs, while some results combine GATs and Graph Transformers with GCN. Clearly separating these results, or adding more analysis and experiments for GATs and Graph Transformers, would make the contributions much clearer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.