Understanding Robustness to Noisy Supervision in Graph Neural Networks

Graph Neural Networks (GNNs) are vulnerable to noisy supervision, yet the role of architecture in mitigating this effect remains poorly understood. We study a modified version of Jumping Knowledge (JK) that preserves direct access to pre-propagation node representations by combining the initial and final representations at readout, while discarding intermediate propagation states. We theoretically show that this construction preserves information that message passing may make inaccessible to the prediction head, and characterize its advantage in terms of the prediction deficit of the propagated representation and the recoverability of this deficit from the pre-propagation representation. Across four GNN architectures, four datasets, three noise mechanisms, and three corruption levels, we find that JK improves accuracy under label noise by 4.3 points (around 7.0% relative) on average, with its benefit increasing with corruption in the regimes where information made inaccessible through propagation remains recoverable from the initial representation. Our analysis shows that label noise enlarges the prediction deficit across the corruption range considered, while recoverability remains comparatively stable, identifying recoverable prediction deficit as a key factor governing when representation preservation improves robustness to noisy supervision.