Abstract
The abstract is not self-contained and is missing some key information to be read as a standalone. "Depth degradation" is mentioned but never explained (a few words would be enough). The approach is stated as a "simple mean-field iterative scheme that outperforms GNNs" which does not say much (it should be stated that it is applied to label distribution). The second half of the abstract (from "Explicitly imposing" to the end) is very general and does not bring anything other than "regularizing makes training harder but generalizes better unless there is too much of it". The abstract should be rewritten to clearly explain the motivation, problem, method (succinctly) and conclusion so that a reader can judge whether the paper is of interest to their specific work.
Related work
The related works is more of a list of name-dropping a list of papers and concepts without explaining why they are relevant to the paper at hand. Each paper should have a takeaway that motivates the current approach, rather than a vague statement. For instance: "Existing deep GNN analyses distinguish among expressivity, optimization, generalization, propagation scope, and representation collapse. Cong et al. (2021) separate expressivity, optimization, and generalization; Deng et al. (2025) model variation in propagation scope across nodes; Zhang et al. (2026a) compare energy- and rank-based measurements of over-smoothing; and Arnaiz-Rodriguez & Errica (2026) separate representation collapse from predictive performance.". This does not state how the results of those papers connect to the current one, which should appear in the main text rather than in the appendix as is currently done; otherwise, the scope and the positioning of the work remain unclear. This issue is true at different degrees for the four paragraphs of the section, which should be overhauled to properly motivate the current work.
Unclear positioning
The main goal of the paper is unclear; if the paper intends to give general diagnostics and remedies for depth degradation, then the study is arguably insufficient. Experiments are carried over a single dataset (then other experiments are run on the City dataset), and a single architecture (GCN). The results are compared to a very minesweeper-specific baseline, but the GCN results are not convincingly related to those. While GCNs and belief-propagation approaches share the same constraints, how do they align with each other in terms of update dynamics?
Regarding minesweeper in particular, "Nearly independent neighboring labels" is a strong claim for minesweeper; the labels are quite strongly dependent conditioned on the clues, and the GNNs learns that conditional distribution.
Overall, the conclusion seems to be that "there is a tradeoff between regularization and model size", with the additional result that you can learn minesweeper with a rank-one, recurrent approach, and that enforcing those constraints on a GCN works well, without any indication that this works beyond that very specific, noiseless dataset.
Paper Clarity
The paper lacks a lot of clarity, and over-relies on the appendices as "extra main paper space", making several claims that are not even understandable without reading the appendix.
- the mean-field scheme is evoked several times in part 1 and 2 without being described. When it is finally introduced in part 3, the fact that this is the mean-field scheme is written nowhere explicitly. It should be clear from the start what the approach is doing so that the rest can be read with that information.
- In table 1, the numbers in the first row should be explained (number of iterations?)
- "Jointly increasing width and weight decay produces a deeper GCN": why does increasing width and WD increase depth?
- the notion of "rank-one" message passing is evoked many times but only explained late in the experimental section.
- Unclear sentence: "Deep GNNs can instead fail through optimization or generalization"
- "Minesweeper is an unusual benchmark as found by Rubio-Madrigal & Burkholz (2026): simple, task-agnostic aggregated features, followed by a graph-agnostic learned MLP, match learned GNNs on most standard node classification datasets—but fall substantially behind on Minesweeper, most likely due to depth-related task dependencies": This sentence should be rewritten as it is quite ambiguous.
- A lot of the relevant information is buried in the Appendix. While the size constraints force understandable tradeoffs, the main takeaway and ideas should be present in the main text. For instance:
** proper positioning with regards to the existing literature
** The theoretical part just lists vague principles and refers to the appendix. It should be self contained; while proofs can remain in appendix, the relevant lemmas and theorems should be written explicitly and rigorously. Otherwise this entire section provides neither insight nor definitive results.
** the dataset should be properly described in one place (features, classes, clues format, how they are treated by the GCN), especially since there is only a single benchmark for the paper. Paragraph 3 states that the features "encode neighboring mine counts", then 3.1 states xi indicates whether cell i contains a mine (which is the label if I follow correctly?).
** "To compare the reference with learned predictions, we fit effective (α, β) pairs to a GCN’s output probabilities at recurring training checkpoints (Fig. 7 in the appendix)." This is not understandable without the appendix, and should either be explained in the main text or entirely in the appendix.
Rebuttal for Reviewer h914
We thank the reviewer for the detailed feedback. We address their concerns below and would be happy to answer additional questions if they come up.
W1/Q1. We now report a broader set of experiments on City Networks highlighting similar mechanisms at play that we also observed for Minesweeper. When the network is narrow, limiting its width already regularizes the model, so adding weight decay makes little difference to performance. At width 128, the stable rank of the final message map is about 10 on every city dataset (Fig. 17). As we increase width, this rank rises, and the best-performing runs use weight decay. For Los Angeles and Shanghai, the rank is still close to 10 in the best wide models, while the other wide models have higher ranks. For Paris, the wide models retain higher ranks even with weight decay and all runs show overfitting. As on Minesweeper, training becomes less stable as we increase network width and depth (Fig. 18). Note that we do not have a reference solution for City Networks, so we do not know what level of performance is achievable on these datasets.
W2/Q2 (matching parameters/regularization). We agree that the same weight decay coefficient can regularize the two architectures differently, as we describe in appendix E. The matched runs in Table 2 illustrate how the models behave at the same depth, width, and weight-decay setting in order to study their representation geometry. But to compare how well each architecture can perform, we sweep all three hyperparameters and select weight decay separately by validation. Particularly, the advantage of the tied rank-one GCN at 32 layers persists when we do this. At width 512, the unrestricted GCN reaches 90.9% mean test ROC–AUC with a selected coefficient of 1e-5. The tied rank-one GCN exceeds that score even with weight decay turned off (96.5%) and reaches 98.3% with its selected coefficient of 1e-4. For a similar parameter count, we can also compare to the tied GCN but with rank two (90.5% for no weight decay and 97.8% for the selected one).
W2/Q2 (better training for failing runs). We have added a study of optimization for some failing configurations in appendix G, particularly for configuration C with smaller learning rates and for depth 32 with a cosine scheduler and different initializations. With a smaller learning rate, the run succeeds but still heavily overfits. This supports our main points: 1) When training succeeds, as depth increases, regularization needs to decrease the rank of the message maps, which helps with generalization; 2) As width, depth, and weight decay increase, the runs become harder to train (but the solution is not unreachable); 3) Single representation geometry measures, including ones for over-smoothing, do not mean much when training fails.