acceptodds
Under review as a conference paper at ICLR 2027

Does the Mask Know Where to Cut? Separating Topology from Weight Placement in Rewiring Nulls

Abstract

Rewiring null models are widely used to test whether the pattern of connections in a weighted network matters. Yet rewiring changes both the connections and the weights assigned to them, so the resulting score combines topology with weight placement. We show that this score decomposes into wiring and placement terms, provide an exact finite-sample test for placement, and separate placement variance from rewiring variance. The construction applies the null’s weight rule to the original topology as well as to rewired topologies, allowing wiring to be compared under the same treatment of weights. In neural-network pruning, placement is significant in all 36 high-sparsity iterative-magnitude-pruning comparisons across three initializations. In connectome reservoirs, it accounts for up to 99.8% of the null variance; removing it from the score difference reduces the statistic’s magnitude, whereas removing its contribution to the variance increases it. A conventional rewiring score is therefore not a biased topology score but a different quantity that combines wiring and weight placement.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
null models, rewiring, weighted networks, pruning at initialization, permutation tests, variance decomposition, reservoir computing, connectomes

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wkVKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a statistical framework for separating the effects of network topology (wiring) and weight placement in rewiring null models. The authors introduce decompositions of the conventional rewiring statistic and its variance, along with a statistical test for weight placement. Experiments on neural network pruning and connectome reservoirs demonstrate that weight placement can substantially influence the interpretation of conventional rewiring results.

Critical strengths
  1. The paper addresses a meaningful problem in existing rewiring null models, where topology and weight placement effects are mixed. The overall motivation and research objectives are clearly presented.

  2. The proposed framework provides a systematic way to distinguish wiring and weight placement contributions, supported by mathematical derivations and statistical testing.

  3. The authors evaluate the framework on both neural network pruning and connectome reservoirs. The results demonstrate the importance of separating topology and weight placement and provide evidence supporting the proposed decompositions.

Critical weaknesses
  1. Although the motivation is clearly explained, an illustrative figure showing how rewiring changes topology and weight placement at the same time would help readers understand the problem and proposed solution more intuitively.

  2. Figure 1: Important symbols, such as znaivez_{\text{naive}}, zwiringz_{\text{wiring}}, and z⋆z^\star, are introduced before their formal definitions. The authors should briefly explain these symbols in the caption or explicitly refer readers to the corresponding equations in Section 3.

  3. Some parts of the methodology are difficult to follow, particularly the construction of matched draws and the variance estimation and confidence bound in Proposition 3. More detailed explanations or illustrative examples would improve readability.

  4. Propositions 1 and 3 describe the numerator and denominator decompositions of the same statistic. Presenting them together in a unified framework could improve the logical flow and make their relationship clearer.

  5. Several formulations rely on established mathematical or statistical principles, such as algebraic decomposition, permutation testing, and the law of total variance. The authors should provide appropriate citations where applicable and clarify which formulations are novel contributions rather than direct applications of existing methods.

  6. Most pruning experiments focus on high sparsity levels (90%–99%). Given the relatively simple LeNet/MNIST setting, including moderate sparsity levels, such as 50%, would provide a more comprehensive understanding of how wiring and weight placement contributions change with sparsity.

  7. Figure 3: The figure should be redrawn to improve readability, as some sparsity labels appear truncated at the bottom.

  8. Section 6 contains substantial discussion and interpretation of experimental findings, but it appears after the Limitations section. These discussions should be incorporated into Section 4 or placed immediately after the Results section, with Limitations presented afterward to improve the paper's overall organization.

Questions for authors

Please refer to the concerns raised in the Weaknesses section, particularly regarding the novelty of the mathematical formulations, the clarity and validity of the proposed methodology, and the experimental settings. The remaining points suggest improvements to the paper's clarity and presentation.

4 comments

Thank you for the positive assessment and the concrete suggestions. The general response lists all changes.

Weaknesses: All of the changes below are being made to the manuscript, and we will upload the revised PDF to OpenReview soon. (Note that points 1, 2 and 4 to 8 are also acknowledged in the general response.)

1) A new figure shows, on a small weighted graph, a null draw, in which wiring and weight placement change together, and a matched draw, which keeps the connections and places the weights by the null's own weight rule.

2) The caption of Figure 1 explains z_naive, z_wiring and z* and points to their equations in Section 3.

3) The new figure also shows how a matched draw is built. Before the variance decomposition (Proposition 3 of the submission), the revision explains in words how the variance is estimated and how the bound is obtained.

4) Propositions 1 and 3 are presented together, as the decompositions of the numerator and the denominator of the same statistic.

5) The revision now cites the standard tools used: Monte Carlo and permutation tests (Barnard, 1963; Hope, 1968; Phipson and Smyth, 2010), the law of total variance (Casella and Berger, 2002) and variance components (Searle et al., 1992). It states that the new part is the construction of the matched draws from the null's own weight rule, and the test, variance split and standard error built on it.

6) We added LeNet at 50, 70 and 80% sparsity, three initializations each, for all six methods. For IMP, the wiring term grows steadily with sparsity, from +0.12 to +0.16 at 50% to +3.8 to +4.2 at 95% and +10.5 to +11.9 at 99% (exactly paired family, accuracy readout), while the placement term is reproducible only from 95% sparsity. GraSP has a reproducible negative placement term at 70 and 80%, which corrects a statement of the submission (see the general response).

7) Figure 3 is redrawn with every label legible.

8) We are moving the Discussion directly after the Results, as in your second option, with the Limitations after it.

Would a worked example or a panel in the new figure help more for Proposition 3? We hope the revision reflects your suggestions and are happy to discuss any of them further.

Thank you for the detailed response. However, I do not see the proposed changes reflected in the currently uploaded PDF. I look forward to reviewing the revised version once it is available.

AuthorsOpenReview ↗

Revised PDF uploaded

Thank you for your patience, and we are sorry the revision came late. The revised PDF is now on OpenReview, with revised text on a light-blue background. We hope it reflects your suggestions, and we would be glad to answer any further question.

Thanks for the revision. The paper has improved a lot.

Reviewer_qAmaOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how weighted-network rewiring experiments combine the effects of topology and weight placement. The proposed framework applies the null model’s weight rule to the original topology, decomposing the conventional standardized score into wiring and placement terms. It also introduces a placement test and a decomposition of null variance. Experiments cover neural-network pruning and connectome reservoirs.

Critical strengths
  1. The research question of distinguishing topology from weight assignment is important for interpreting weighted-network experiments.
  2. The numerator decomposition and total-variance identities are intuitive.
  3. Several experiments illustrate meaningful changes in interpretation, including opposing effects of numerator and denominator correction and differences across rewinding settings.
Critical weaknesses
  1. The presentation needs clearer definitions. Core concepts such as null family, null ensemble, matched draw, rewiring rule RR and weight rule ω\omega have limited explanation.
  2. Several qualitative findings substantially overlap with prior work. Su et al. [1] already distinguish mask rearrangement from fixed-mask weight shuffling. The importance of signs is established and investigated by Zhou et al. [2] and Gadhikar and Burkholz [3]. The manuscript acknowledges these prior works, but should explain more clearly which new conclusions its decomposition enables.
  3. The robustness of the sampling procedure is insufficiently evaluated. Although Proposition 2 and Corollary 1 discuss the effects of KK, the experiments do not systematically evaluate sensitivity to KK, null-ensemble size, or the number of rewired supports.
  4. The main pruning experiment uses LeNet for MNIST, ResNet20 for CIFAR-10, and the connectome study covers five graphs on one task. This limited setting hinders the generality of the conclusions.
  5. The empirical novelty of Figure 3 is limited. Frankle et al. [5] already demonstrate that subnetworks obtained by pruning at initialization methods generally maintain performance after weight reinitialization. In addition, the original Lottery Ticket Hypothesis paper [6] establishes that randomly reinitializing winning tickets can substantially reduce performance, particularly at high sparsity levels.

[1] Jingtong Su et al. Sanity-Checking Pruning Methods: Random Tickets Can Win the Jackpot. NeurIPS 2020.

[2] Hattie Zhou et al. Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask. NeurIPS 2019.

[3] Advait Gadhikar and Rebekka Burkholz. Masks, Signs, and Learning Rate Rewinding. ICLR 2024.

[4] Leone Costi et al. The Drosophila Connectome as a Computational Reservoir for Time-Series Prediction. Biomimetics 2025.

[5] Jonathan Frankle et al. Pruning Neural Networks at Initialization: Why Are We Missing the Mark? ICLR 2021.

[6] Jonathan Frankle and Michael Carbin. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. ICLR 2019.

Questions for authors
  1. Are the conclusions stable across sampling budgets?
  2. What insights can we take away from the proposed decomposition to design pruning methods?
5 comments

Thank you for the detailed review and the precise pointers to prior work; your question on sampling budgets led to a new analysis (point 3). We answer each point below; the general response lists all changes.

Weaknesses

1) The revision defines each of these terms early, and adds a figure showing a null draw and a matched draw. These are part of a wider effort to make the paper easier to follow.

2) The revision will state, next to each citation, what the work establishes and what our decomposition adds. For the works cited in the review:

[1] runs the mask rearrangement and the weight shuffle as separate checks; the decomposition measures how much of the gap is due to wiring. Under learning-rate rewinding the wiring term is within two standard errors of zero on ResNet-20 (-0.16 to +0.29) and on VGG-16 (-0.14 to +0.20), so the mask's advantage over its rearrangements is placement.

[2] and [3] show that signs matter; we split the placement term into a sign and a magnitude part, and the split depends on the setting. The signs alone keep 97 to 106% of the placement term under learning-rate rewinding (ResNet-20 and VGG-16), but 15 to 28% on ResNet-20 rewound to iteration 1000 at 90 and 95% sparsity, and 62 to 73% on VGG-16 rewound to iteration 1000.

[4] finds that shuffling a connectome's weights affects performance as much as randomizing its topology; the decomposition separates the two within one rewiring null, and on 100 connectomes placement makes up 94.4 to 99.7% of the null variance on average.

3) and Question 1) We resampled the stored scores.

Matched draws K: with K = 20, 99 to 100% of the significant IMP results stay significant, and the spread of the wiring term matches Corollary 1 (ratio 0.98 to 1.00).

Null draws n: with half the null draws (n = 20 of 40), the wiring term under learning-rate rewinding moves by a median of 0.11, against 0.68 to 1.30 for the reported statistic, and stays within two standard errors of zero in 97 to 99% of subsamples.

Rewired supports T: from T = 15, the bound on the corrected statistic keeps its sign and lies beyond the wiring term in every subsample where it is finite.

4) We added VGG-16 as a second architecture (95% sparsity, three initializations), three initializations for ResNet-20 at 90, 95 and 98% sparsity, LeNet at 50, 70 and 80% sparsity, and 100 connectomes with a second task (see the general response). The image datasets remain MNIST and CIFAR-10.

5) We agree that the pattern in Figure 3 is known. [6] shows that winning tickets lose accuracy when reinitialized, and [5] that masks from pruning at initialization keep or even improve their accuracy when the mask is shuffled within each layer or the kept weights are reinitialized. The revision will say so in the related work. What the decomposition adds is where the effect sits. Reinitialization draws new weight values, so it changes the weights as well as their placement; matched draws permute the same values, so they isolate placement, with an exact test for each method, sparsity and initialization. At 70 and 80% sparsity GraSP has a reproducible negative placement term (3 of 3 initializations; see Question 2).

Questions

Question 1) See 3.

Question 2) The decomposition gives three insights for designing pruning methods.

  • The matched draws can find a better weight assignment for a given mask. On LeNet with GraSP at 70 and 80% sparsity, all 50 random permutations of the kept initial weights within each layer beat GraSP's own placement at every initialization, by 0.49 to 0.73 accuracy points on average. This agrees with [5] and shows that the gain comes from placement alone.
  • On ResNet-20 rewound to iteration 1000, the wiring term grows from about +1 at 90% sparsity to +3.6 to +5.7 at 98%, while the placement share falls from 83 to 92% to 40 to 63%. The choice of connections therefore matters most at extreme sparsity; below that, most of the advantage lies in the weights kept.
  • Comparing a method's masks with rearranged masks mixes wiring and placement (see 2); adding matched draws tells whether the method finds good wiring or good weights.

We hope these answers resolve your concerns and would be glad to clarify any remaining point.

[1] Jingtong Su et al. Sanity-Checking Pruning Methods: Random Tickets Can Win the Jackpot. NeurIPS 2020.

[2] Hattie Zhou et al. Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask. NeurIPS 2019.

[3] Advait Gadhikar and Rebekka Burkholz. Masks, Signs, and Learning Rate Rewinding. ICLR 2024.

[4] Leone Costi et al. The Drosophila Connectome as a Computational Reservoir for Time-Series Prediction. Biomimetics 2025.

[5] Jonathan Frankle et al. Pruning Neural Networks at Initialization: Why Are We Missing the Mark? ICLR 2021.

[6] Jonathan Frankle and Michael Carbin. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. ICLR 2019.

I thank the authors for considering my concerns. Unfortunately, I still cannot determine from the rebuttal how several concerns have been resolved.

  1. I have not seen the changes in the current PDF version.
  2. The new numerical results are difficult to assess from the response as presented. In particular,
    • Could the authors explain why there are very different figures for ResNet-20 and VGG-16 in the following sentence?

      “The signs alone keep 97 to 106% of the placement term under learning-rate rewinding (ResNet-20 and VGG-16), but 15 to 28% on ResNet-20 rewound to iteration 1000 at 90 and 95% sparsity, and 62 to 73% on VGG-16 rewound to iteration 1000.”

    • Regarding the sensitivity analysis with K,n,TK, n, T, it would help to show how the result varies with these hyper-parameters rather than only report a value for each.

    • I also appreciate the expanded experiments, but it remains difficult to judge how much they address generality without seeing their results and variability across runs.

  3. Regarding the insights for pruning methods, the novelty of the first two insights are limited. Specifically, the first one confirms observations in Frankle et. al. ([5] as in author response), while the second insight has already observed in [7, 8] where connectivity matters at extreme sparsity levels.

[7] Vysogorets, Artem, and Julia Kempe. "Connectivity matters: Neural network pruning through the lens of effective sparsity." JMLR 2023.

[8] Pham, Hoang, et al. "Towards data-agnostic pruning at initialization: What makes a good sparse mask?." NeurIPS 2023.

AuthorsOpenReview ↗

Response to follow-up

Thank you for the follow-up. The revised PDF is now uploaded.

Signs on ResNet-20 and VGG-16. The three figures come from three settings, and each is stable across seeds (Appendix Tables 8 and 12). In accuracy points, the advantage over the matched draws splits into the part the signs keep (draws that keep every weight's sign and re-deal the magnitudes) and the rest:

setting (95% sparsity unless stated)advantagekept by signsrest
ResNet-20, iteration 1000, 90%3.29 to 3.560.52 to 0.612.69 to 3.05
ResNet-20, iteration 10002.97 to 3.530.73 to 0.852.13 to 2.73
VGG-16, iteration 10001.10 to 1.280.76 to 0.940.32 to 0.47
ResNet-20, learning-rate rewinding5.17 to 6.045.32 to 5.85-0.14 to +0.19
VGG-16, learning-rate rewinding1.56 to 1.871.65 to 1.82-0.10 to +0.05

At iteration 1000 the signs keep about the same amount on both networks, 0.5 to 0.9 points; ResNet-20's magnitudes add 2.1 to 3.1 points and VGG-16's 0.3 to 0.5, so the share kept by the signs is lower on ResNet-20. Under learning-rate rewinding the magnitudes add nothing, consistent with [3]. We have no explanation for why the magnitudes carry more on ResNet-20 and report it as an observation (Section 5).

Sensitivity. Appendix Table 9 now reports every budget (K from 10 to 40, n from 10 to 50, T from 10 to 30) for every setting. With K = 20 or more, 99.1 to 100% of the significant IMP cases stay significant (none can at K = 10, where the smallest p-value is 1/11), and the spread of the wiring term matches Corollary 1 at every K. Fewer null draws move the wiring term much less than the reported statistic (ResNet-20 under learning-rate rewinding, n = 20 of 40: 0.11 against 1.30). The bound on the corrected statistic keeps its sign at every T from 10.

Generality and variability. Every cell is listed per seed in Appendix Tables 5, 6, 8 and 10 to 12, and Table 2 gives ranges over seeds. Under learning-rate rewinding the wiring term is within two standard errors of zero at every seed on both networks, and the placement test rejects in 6 of 6 cases on each.

Insights. We agree that the first insight confirms [5] and the second agrees with [7] and [8]; the revision cites [7] and [8] (Section 5). What the decomposition adds is the size of each part within one comparison, with a standard error for the wiring term and an exact test for placement. Below extreme sparsity, placement dominates (83 to 92% of the statistic at 90% sparsity on ResNet-20 rewound to iteration 1000), and the split above shows which part of the weights carries it.

Correction. For GraSP, the matched draws permute all of each layer's initial weights, not only the kept ones (Section 3.1); the numbers are unchanged.

[7] Artem Vysogorets and Julia Kempe. Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsity. JMLR 2023.

[8] Hoang Pham et al. Towards Data-Agnostic Pruning at Initialization: What Makes a Good Sparse Mask? NeurIPS 2023.

Reviewer_9wF2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework to disentangle the effects of network wiring and weight placement in rewiring-based null models. The authors decompose the conventional rewiring statistic into a wiring term and a weight-placement term, and further introduce a finite-sample test for the placement effect and a decomposition of the null variance. Experiments on neural network pruning and connectome reservoirs show that weight placement can substantially affect both the reported statistic and its variance, and in some settings explains most of the effect previously attributed to network topology.

Critical strengths
  1. The paper identifies a meaningful confounding factor in commonly used rewiring null models and gives a simple way to separate topology from weight placement.
  2. The decomposition is clean and easy to interpret. The finite-sample test for the placement effect is also useful and does not require distributional assumptions.
  3. The experiments show that this is not only a theoretical issue. In particular, weight placement explains a large fraction of the reported statistic for high-sparsity IMP and learning-rate rewinding.
  4. The paper discusses its limitations relatively clearly.
Critical weaknesses
  1. The main decomposition itself is quite simple, essentially coming from adding and subtracting the matched weight-placement expectation. The novelty therefore mainly comes from the interpretation and evaluation rather than the decomposition itself.
  2. The experimental scope is somewhat limited. Most pruning experiments use LeNet-300-100/MNIST, while the ResNet-20 experiments are less extensive. The connectome experiments also contain only five graphs and one task. The authors should show what state-of-the-art performance gain can be gained from the analysis.
  3. While the paper shows that weight placement can have a substantial impact on network performance, it remains unclear how this observation can be used to guide better network or pruning design. An experiment showing that the proposed analysis leads to improved topology or weight assignment would strengthen the practical significance of the paper.
Questions for authors

Listed in Weakness part.

3 comments

Thank you for the careful review, and for asking how the analysis can guide design, which led to the new experiments in point 3. We answer each point below; the general response lists all changes.

1) We agree that the decomposition itself is simple and that our contributions mainly come from interpretation and evaluation. However, two parts of the method are novel and are not interpretation or evaluation of the decomposition:

  • First, the matched draws apply the null's own weight rule to the observed topology, which pairs the rewired null with a control for weights; in the studies we surveyed, the two are run separately (Appendix B). This pairing also gives the placement term an exact test.
  • Second, placement also inflates the denominator of the reported statistic, so we split the null variance, estimate the part due to rewiring alone from nested draws, and bound the corrected statistic. Point 3 below adds a practical use of the analysis, as the third weakness asks.

2) We agree, and have extended each setting (see the general response). The larger connectome cohort also corrects one statement: for NARMA-10 under the annealed null, the placement test rejects for 33 of 100 graphs, not 4 of 5.

The paper proposes an analysis rather than a new pruning or reservoir method, so a state-of-the-art comparison does not apply directly; the gain the analysis gives in practice is shown by the experiments in point 3.

3) To show that the proposed analysis leads to improved topology or weight assignment, we ran two searches from each of the 100 connectomes, each proposing 1,000 swaps and keeping a swap only when it improved the score: one swaps the weights of two edges, the other swaps two edges together with their weights while keeping every node's degree. On reservoir and input seeds the searches did not use, the placement search gained more: +0.99 against +0.55 null standard deviations for NARMA-10 (larger in 141 of 200 paired searches), and +5.16 against +4.05 for memory capacity (84 of 100). This is the direction the variance split points to, since placement accounts for 98 to 100% of the null variance of these graphs. For pruning, the matched draws find a better weight assignment for a given mask. On LeNet with GraSP at 70 and 80% sparsity, all 50 random permutations of the kept initial weights within each layer beat GraSP's own placement at every initialization, by 0.49 to 0.73 accuracy points on average.

We would be glad to know whether the experiments in point 3 address the third weakness, and are happy to clarify anything further.

AuthorsOpenReview ↗

Correction to our earlier reply

Correction to our reply (point 3). For GraSP at 70 and 80% sparsity, the matched draws permute all of each layer's initial weights, not only the kept ones, so each draw places a random subset of the layer's initial weights on the mask (Section 3.1 of the revised PDF). The numbers are unchanged.

The rebuttal substantially addresses my concerns. The core decomposition remains mathematically simple, but the clarification of the matched-null construction, exact placement test, and null-variance decomposition makes the methodological contribution clearer. The expanded connectome evaluation also improves the empirical scope, although I would still like to assess the broader pruning experiments described in the general response. Most importantly, the new placement/topology search and pruning experiments directly address my concern about practical significance by showing that the analysis can identify actionable directions for improving network performance. I will raise my rating.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response: new results, clarifications and corrections

We thank the reviewers for their careful reading. All three find the problem meaningful. The concerns are novelty, clarity, scope and practical use, and we do our best to assess and fix all of them. The reply to each reviewer goes into more detail.

Novelty. The identity itself is elementary, and we will make this explicit in the revised PDF. The novel elements are the construction of the identity, which applies the null's own weight rule to the observed topology, and what that construction allows: an exact test for the placement term, a split of the null variance with a bound on the corrected statistic, and a standard error for the wiring term.

New experiments.

  • ResNet-20 at 90, 95 and 98% sparsity, rewinding to iteration 1000, three initializations each: the wiring term grows with sparsity.
  • VGG-16 on CIFAR-10: under learning-rate rewinding, placement accounts for the whole statistic at all three initializations, as on ResNet-20.
  • LeNet at 50, 70 and 80% sparsity: the placement term of IMP is reproducible only from 95% sparsity.
  • 100 connectomes instead of 5, and memory capacity as a second task. For memory capacity under degree-preserving rewiring, the negative sign of the reported statistic comes from placement.
  • Sampling budget: with 20 matched draws, 99 to 100% of the significant IMP results stay significant.
  • Design: from each connectome, a search over weight placement improves the reservoir more, on average, than a search over wiring with the same budget, on held-out seeds.

Clarity. The revision will:

  • Define each term early with an example from both case studies.
  • Add a figure showing a null draw and a matched draw.
  • Present Propositions 1 and 3 together.
  • Cite the standard tools used.
  • Explain the symbols of Figure 1 in its caption.
  • Redraw Figure 3.
  • Move the Discussion directly after the Results.
  • Revise the Methods and the Discussion.

Corrections.

  • On ResNet-20 at 98% sparsity, rewinding to iteration 1000, we attributed the placement term to the signs from one initialization. With three, the magnitude component is significant in 4 of 6 cases, so signs and magnitudes share the term.
  • For NARMA-10 under the annealed null, the placement test rejects for 33 of 100 connectomes, not for most of them (4 of 5 in the submission).
  • The absence of a reproducible placement term for the initialization-time methods holds at 90% sparsity and above; at 70 and 80%, GraSP has a reproducible negative one.

We hope these changes address the reviewers' concerns, and we are happy to clarify any of them further.

AuthorsOpenReview ↗

Revised PDF uploaded.

Revised PDF. The revised PDF is now uploaded, with revised text on a light-blue background. The changes announced in our general response are in place:

  • Clarity: the terms are defined in Section 3.1, and Figure 2 shows a null draw and a matched draw. The decompositions of the numerator and the denominator are presented together, with the standard tools cited (Section 3.2). Figure 1's caption explains each symbol, Figure 3 is redrawn, and the Discussion follows the Results. The related work states what each cited work establishes and what the decomposition adds. We are working to further improve clarity.
  • New results: VGG-16 and ResNet-20 with three seeds each (Section 4.1, Table 2, Appendix Tables 8 and 12); LeNet-300-100 at 50, 70 and 80% sparsity (Figure 3, Appendix Tables 10 and 11); 100 connectomes with memory capacity as a second task, and the placement and wiring searches (Section 4.2, Table 3, Figure 4); sampling budgets (Appendix Table 9).
  • Corrections: the three corrections announced in our general response are made in Sections 4.1 and 4.2.

We are happy to answer any further question.

AuthorsOpenReview ↗

Revised PDF and Supplementary Material Uploaded

We have uploaded the final revised PDF and the revised supplementary material.

  • PDF. The light-blue highlights are removed. The supplementary file changes_against_submission.pdf marks every change against the submitted version. In addition to the revisions described in our previous comment, we have made the following updates:
    • A new experiment trains ResNet-20's masks with the signs from iteration 1000 and the magnitudes from initialization. Table 13 compares the kept weights at that point with their initial values (Appendix A).
    • ResNet-20 rewound to initialization now has three seeds, none showing a placement term (Table 2 and Appendix Table 8). Table 9 includes the new seed.
  • Supplementary material. It includes the code and every results file behind the paper, including the VGG-16 masks, the 100 connectomes, the two reservoir scores and the scripts for every table and figure.

We would like to express our sincere gratitude to the reviewers and the area chair for the time, care, and thoughtful attention they have devoted to our work. Their insightful comments and constructive suggestions have been invaluable in helping us improve the paper. We truly appreciate their engagement throughout the review process and hope that the final revisions satisfactorily address their remaining concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.