- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a statistical framework for separating the effects of network topology (wiring) and weight placement in rewiring null models. The authors introduce decompositions of the conventional rewiring statistic and its variance, along with a statistical test for weight placement. Experiments on neural network pruning and connectome reservoirs demonstrate that weight placement can substantially influence the interpretation of conventional rewiring results.
Critical strengths
-
The paper addresses a meaningful problem in existing rewiring null models, where topology and weight placement effects are mixed. The overall motivation and research objectives are clearly presented.
-
The proposed framework provides a systematic way to distinguish wiring and weight placement contributions, supported by mathematical derivations and statistical testing.
-
The authors evaluate the framework on both neural network pruning and connectome reservoirs. The results demonstrate the importance of separating topology and weight placement and provide evidence supporting the proposed decompositions.
Critical weaknesses
-
Although the motivation is clearly explained, an illustrative figure showing how rewiring changes topology and weight placement at the same time would help readers understand the problem and proposed solution more intuitively.
-
Figure 1: Important symbols, such as , , and , are introduced before their formal definitions. The authors should briefly explain these symbols in the caption or explicitly refer readers to the corresponding equations in Section 3.
-
Some parts of the methodology are difficult to follow, particularly the construction of matched draws and the variance estimation and confidence bound in Proposition 3. More detailed explanations or illustrative examples would improve readability.
-
Propositions 1 and 3 describe the numerator and denominator decompositions of the same statistic. Presenting them together in a unified framework could improve the logical flow and make their relationship clearer.
-
Several formulations rely on established mathematical or statistical principles, such as algebraic decomposition, permutation testing, and the law of total variance. The authors should provide appropriate citations where applicable and clarify which formulations are novel contributions rather than direct applications of existing methods.
-
Most pruning experiments focus on high sparsity levels (90%–99%). Given the relatively simple LeNet/MNIST setting, including moderate sparsity levels, such as 50%, would provide a more comprehensive understanding of how wiring and weight placement contributions change with sparsity.
-
Figure 3: The figure should be redrawn to improve readability, as some sparsity labels appear truncated at the bottom.
-
Section 6 contains substantial discussion and interpretation of experimental findings, but it appears after the Limitations section. These discussions should be incorporated into Section 4 or placed immediately after the Results section, with Limitations presented afterward to improve the paper's overall organization.
Questions for authors
Please refer to the concerns raised in the Weaknesses section, particularly regarding the novelty of the mathematical formulations, the clarity and validity of the proposed methodology, and the experimental settings. The remaining points suggest improvements to the paper's clarity and presentation.
4 comments
Thank you for the detailed response. However, I do not see the proposed changes reflected in the currently uploaded PDF. I look forward to reviewing the revised version once it is available.
Revised PDF uploaded
Thank you for your patience, and we are sorry the revision came late. The revised PDF is now on OpenReview, with revised text on a light-blue background. We hope it reflects your suggestions, and we would be glad to answer any further question.
Thanks for the revision. The paper has improved a lot.
Thank you for the positive assessment and the concrete suggestions. The general response lists all changes.
Weaknesses: All of the changes below are being made to the manuscript, and we will upload the revised PDF to OpenReview soon. (Note that points 1, 2 and 4 to 8 are also acknowledged in the general response.)
1) A new figure shows, on a small weighted graph, a null draw, in which wiring and weight placement change together, and a matched draw, which keeps the connections and places the weights by the null's own weight rule.
2) The caption of Figure 1 explains z_naive, z_wiring and z* and points to their equations in Section 3.
3) The new figure also shows how a matched draw is built. Before the variance decomposition (Proposition 3 of the submission), the revision explains in words how the variance is estimated and how the bound is obtained.
4) Propositions 1 and 3 are presented together, as the decompositions of the numerator and the denominator of the same statistic.
5) The revision now cites the standard tools used: Monte Carlo and permutation tests (Barnard, 1963; Hope, 1968; Phipson and Smyth, 2010), the law of total variance (Casella and Berger, 2002) and variance components (Searle et al., 1992). It states that the new part is the construction of the matched draws from the null's own weight rule, and the test, variance split and standard error built on it.
6) We added LeNet at 50, 70 and 80% sparsity, three initializations each, for all six methods. For IMP, the wiring term grows steadily with sparsity, from +0.12 to +0.16 at 50% to +3.8 to +4.2 at 95% and +10.5 to +11.9 at 99% (exactly paired family, accuracy readout), while the placement term is reproducible only from 95% sparsity. GraSP has a reproducible negative placement term at 70 and 80%, which corrects a statement of the submission (see the general response).
7) Figure 3 is redrawn with every label legible.
8) We are moving the Discussion directly after the Results, as in your second option, with the Limitations after it.
Would a worked example or a panel in the new figure help more for Proposition 3? We hope the revision reflects your suggestions and are happy to discuss any of them further.