- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper analyzes whether preserving pre-propagation node representations (w/ simplified JK readout) improves GNN robustness to noisy labels. The author theoretically characterizes the benefit using prediction deficit and recoverability.
Critical strengths
The proposed method is straightforward and easy to follow
The theorem shows the value of preserving information
Experimental analysis is broad
Critical weaknesses
Concatenating initial and final representations closely resembles existing JK and skip-connection designs. In addition, the theoretical results follow from feature concatenation and standard linear projection identities
The analysis assumes fixed representations. Thus, it is ambiguous how the proposed method improves test accuracy under noisy supervision
Empirical gain is marginal on Amazon-Ratings. GraphSAGE almost converges to 0. These results make it difficult to isolate noise-specific robustness
Questions for authors
Please see above
Response to Reviewer VKhS
Novelty. Our contribution is not a new JK architecture, but a theoretical and empirical characterization of representation preservation as a robustness mechanism under noisy supervision. First, JK differs from layer-wise residual connections: residuals combine representations during propagation, whereas JK guarantees direct access to H(0) at readout. Importantly, GCN*, GAT* and SAGE* already employ learned residual connections, yet JK provides substantial improvements for GCN* and GAT*. The limited gains observed for SAGE* further highlight that the benefit of JK depends on how each backbone preserves node-specific information. Second, while our theoretical results build on standard linear projection identities, their contribution is to characterize when representation preservation improves prediction. The exact decomposition ΔJK∗=D(ρ)Γ(ρ) distinguishes prediction deficit from recoverability, showing that losing predictive information alone is insufficient for JK to help. In particular, even a large prediction deficit yields no JK advantage when the missing signal is not recoverable from H(0), a distinction that cannot be established from predictor-class containment alone. Recoverability is approximately 0.97 on BlogCatalog, 0.55 on Flickr, 0.31 on Roman Empire, and 0.10 on Amazon-Ratings, matching the ordering of probe gains at every nonzero corruption rate. Notably, Amazon-Ratings exhibits the largest prediction deficit but negligible probe gains, demonstrating why information loss alone cannot explain JK's effectiveness. Unlike prior JK and skip-connection studies focused on heterophily, depth, or oversmoothing, our work connects representation preservation to noisy supervision and identifies the conditions governing its effectiveness.
Fixed representations. Representations are fixed only during the analysis, not during training. Both H(0) and H(T) are learned under noisy supervision at rate ρ, so noise affects D(ρ) and Γ(ρ). Our theory characterizes the clean predictive signal accessible to a linear readout, while linear probes measure its impact on classification accuracy. Table 3a reports increasing probe gains for GCN*. We repeated the analysis for GAT* under uniform noise (gain in accuracy points): | Dataset | ρ=0 | ρ=0.15 | ρ=0.30 | ρ=0.45 | |---|---:|---:|---:|---:| | BlogCatalog | +1.4 | +4.2 | +8.8 | +10.0 | | Flickr | +0.5 | +4.4 | +7.0 | +8.9 | | Roman Empire | +0.6 | +1.7 | +2.4 | +4.0 | | Amazon-Ratings | −0.3 | −0.8 | +0.2 | −0.2 |
These results confirm the GCN* trends: probe gains increase with noise on three datasets, while remaining negligible on Amazon-Ratings.
Overall improvements. JK provides substantial improvements across multiple architectures and datasets, particularly under stronger label corruption. Averaged across noise types and rates, GAT*, GCN*, and GIN improve by +9.3%, +11.0%, and +7.5%, respectively. On BlogCatalog, the average gains reach +20.4% for GAT*, +16.3% for GCN*, and +14.9% for GIN; on Flickr, they reach +13.6%, +24.6%, and +12.1%.
Amazon-Ratings and GraphSAGE. These cases help identify when representation preservation provides limited additional benefit.
Noise rate 0. Clean test accuracy at ρ=0 (JK / No JK, mean over three seeds): | Dataset | GAT* | GCN* | GIN | SAGE* | |---|---|---|---|---| | BlogCatalog | 0.972 / 0.933 | 0.974 / 0.960 | 0.956 / 0.795 | 0.967 / 0.971 | | Roman Empire | 0.888 / 0.885 | 0.880 / 0.870 | 0.865 / 0.854 | 0.896 / 0.894 | | Amazon-Ratings | 0.531 / 0.520 | 0.519 / 0.496 | 0.501 / 0.368 | 0.539 / 0.539 | | Flickr | 0.940 / 0.925 | 0.923 / 0.915 | 0.897 / 0.919 | 0.936 / 0.931 |
For GAT* and GCN*, the relative JK gain is much larger under noise than on clean labels: +4.2% → +20.4% and +1.5% → +16.3% on BlogCatalog, and +1.6% → +13.6% and +0.9% → +24.6% on Flickr. The gain is therefore largely noise-specific. GIN's No-JK variant is already unstable on clean labels (0.368 on Amazon-Ratings), so part of its gain reflects training stability. On Amazon-Ratings, the gain does not grow with noise, which is consistent with its low recoverability (Γ≈0.10). We also added these results to the updated PDF (Table 7).