- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors port the analysis of wide networks with Gaussian weights to SNNs. This requires a new setup due to sequence output, synaptic operators and LIF activations. They establish a descriptor of a layer (the mean spike train and second moment) that is sufficient to understand the behaviour of the next layer. They then look at the descriptor when spike trains are decomposed into the constant-in-time direction and its orthogonal complement and use this to describe under what conditions temporal (rather than just rate) information affects network outputs. They follow up this theoretical treatment with a number of experiments that test theory-inspired predictions.
Critical strengths
The paper appears to make mathematically sound progress that establishes a useful and well-motivated way to analyse SNNs. Theorems such as 3.1's exact finite-width sufficiency seem quite important.
A caveat is that I did not cover all of the theoretical work that extends into the appendix and seems important for the paper as a whole. Furthermore, I am relatively unfamiliar with this area and cannot comment confidently on whether the work is novel and well-placed in the literature.
Critical weaknesses
I suggest a weak reject as the main section (without appendices) does not present a self-contained coherent account of the work. I imagine this issue is relatively straightforward to address.
While I would guess that the concepts and results are likely interesting to many people who study SNNs, I think the paper does a poor job of communicating its ideas. Three issues are:
- It is not possible to read the main text alone and understand the paper.
- The consequences of the assumptions needed to create the theory should be made clearer.
- Although more of a minor issue, an improvement to the mathematical presentation could make reading the paper far easier.
I spent a reasonable amount of time trying to read this paper, yet I do not think I have been able to properly review the whole paper. This is partly due to the subject matter being relatively unfamiliar to me, and partly because the main text is so inextricable from the appendix. I wonder if the material would not be better split into two: wide-SNN kernel theory plus convergence experiments and a "when timing matters" investigation.
Below, are the three points above in more detail.
1. Main vs appendix
$ n_l, Q_{ab}^{l}, c_n^l, V_{\Lambda}, Q_{\text{out}} $ are examples of important objects that appear in the main text without full definition or reference to the appendix where many are defined. Such objects without sufficient definition appear throughout the main text and are later defined in the appendix. Other objects such as $ b_{l+1} $ and $ \mathcal{F}_l $ get terse and informal definitions in the main text, with much clearer definitions in the appendix. This makes it difficult to read the main text without reading the appendix.
Not just symbols, but I would argue that conceptual material needed to understand the main text is missing. For example, the Experiments section (L305-) investigates a list of predictions listed only in the appendix.
2. Consequences of assumptions
The antecedent (If $ B_l\Pi_{\perp} = 0 $ of Theorem 3.3's implication seems to preclude most causal synaptic operators including the Toeplitz filter used as the motivating example (L128). If so, this significantly undermines the usefulness of the theorem. Consider the equivalent non-block case: $ A_lP_{\perp} = 0$. We can say:
& A_lP_{\perp} = 0 \\\\ &\implies A_l(I - P_{0}) = 0 \\\\ &\implies A_l = A_lP_0 \\\\ &\implies A_l = A_lv_0 v_0^T \\\\ &\implies A_l = uv_0^T & \text{(for some vector } u \text{ )} \\\\ \end{align*}$$ But as all entries of \$ v_0 \$ are equal, \$ A_l \$ must have all equal columns. I think this rules out all causal operators except maybe columns of [0, 0, 0, ... 1]^T which accumulates all input into the final time. The causality of \$ A_l \$ is strongly suggested by the example given on L128. If I haven't made a mistake here, I think the authors should explicitly state which nontrivial class of temporal operators Theorem 3.3 covers. While I have not noticed any other such issue, I do think that assumptions that underpin the theory should be explored in more detail in order to convince readers that the theory isn't restricted to applying to a trivial situation. ### 3. Improve mathematical presentation Some suggestions: - Define types of objects. For example, \$ x_1, x_2, ..., X^{M} \$ are of unknown type. - Do not overload \$ S_i^l $. This symbol is used as two distinct vectors ($ S_i^l \in \mathbb{R}^T, S_i^l \in \mathbb{R}^D \$) and two different functions outputting those vectors. - The notation is lacking a stable semantic convention for superscripts and subscripts due to mixing indexing with tagging and superscripts and subscripts. The use of subscript \$ n \$ not as an index or coordinate, but as a marker of the finite case is confusing and not described well. Compare \$ Q_n^l \$ which is the \$ Q \$ for finite layer \$ l \$ versus \$ Q_{ab}^l \$ which is an \$ (a,b) \$ index into the \$ Q \$ for the infinite width layer, \$ l \$. Consider \$A_l, b_l, B_l, W_{ij}^l, S_i^l, X^l, \mathcal{F}_l, M_l, c_n^l \$, where \$ l \$ indexes the network layer. Similarly, block extension is inconsistent, with \$ A_l \to B_l \$ and \$ \Psi \to \Psi^{(M)} \$ being the block extensions. Addressing these issues would make the paper easier to read.Questions for authors
-
Were there no other uses of AI that fall under the required to disclose list? (https://iclr.cc/Conferences/2027/AIPolicyForAuthors)
-
Do you have thoughts on how this theoretical foundation could (or could not) be used in theoretical neuroscience, where there is a similar rate vs. timing question?
7 comments
2. "The consequences of the assumptions needed to create the theory should be made clearer." and "…the authors should explicitly state which nontrivial class of temporal operators Theorem 3.3 covers."
We partially agree. The derivation is correct, but it does not restrict the theory to a trivial situation, since Theorem 3.3 is only its exact case and three other statements cover the operators it leaves out, which we may not have emphasized and clarify here.
The class of Theorem 3.3.
- What it is. Exactly the last line of the derivation, with a free vector of gains (the reviewer's , renamed because is the membrane potential in the paper). It is a spike counter, which hands every time step a fixed gain times the number of spikes in the window.
- Its only causal member. The one that the reviewer identifies, the terminal accumulator with the last unit vector, which releases the count at the last step. Theorem 3.3 therefore does not cover the Toeplitz filter of line 128.
The other operators. The table lists the statement that covers each kind, and the Toeplitz filter of line 128 falls under Theorem 3.4. In the windowed counter is the indicator vector of the -th window of steps, and in conversion the neuron, and not the synapse, does the counting. What each kind of operator gets from the theory:
| Temporal operator | Causal | Exact rate factorization | Statement that covers it |
|---|---|---|---|
| spike counter , for example the time average | no | yes | Theorem 3.3 |
| terminal accumulator | yes | yes, the only causal case | Theorem 3.3 |
| windowed counter , each supported at or after the last step of its window | yes | yes, at resolution | Theorem 3.3 at resolution (Appendix A.2.12) |
| exponential filter | yes | no | Theorem 3.4 at resolution (Appendix A.2.12) |
| identity, with the integrate-and-fire neuron of conversion | yes | no | Proposition A.17, the rate readout closes up to when the terminal potential stays bounded |
The other assumptions. Theorems 3.1, 3.2 and 3.4 hold for every temporal operator, and each assumption excludes an explicit set. What each assumption is needed for and what it excludes:
| Assumption | Needed for | What it excludes | Size of the excluded set |
|---|---|---|---|
| Gaussian weights, independent, with mean and variance | Theorem 3.1, exact at any width | other weight laws | the modelling assumption of the paper |
| width to infinity at fixed , and depth | Theorem 3.2 | finite width | measured distance to the limit 0.173 at width 32 and 0.009 at width 8,192 |
| boundary-null (Assumption 1) | continuity of the layer map despite the hard threshold | a membrane coordinate of zero variance that sits exactly on a threshold | at most threshold values for a fixed descriptor and operator (Lemma A.31) |
| positive conditional variances of the rate-projected membrane potentials | the main form of the sharpened Theorem 3.4 | a potential that is an exact affine function of its temporal part, or deterministic | covered by the two further forms of the theorem (Theorem A.11a) |
| anti-concentration (Assumption 2) | the earlier form of the bound (Theorem A.11) | the same exceptional thresholds | holds with explicit constants elsewhere (Lemma A.32) and in 75 of 75 sampled configurations (Appendix A.5) |
| constants uniform along the trajectory | the finite-depth version of Theorem 3.4 (Theorem A.12) | families of descriptors that approach the exceptional set | an explicit hypothesis of that theorem |
| Theorem 3.3 only | every continuously updated causal filter | characterized above |
The reviewer's derivation gave the paper an exact statement of what Theorem 3.3 covers, for which we thank the reviewer. The revision states it as Proposition 3.3a and adds the table of assumptions as Appendix Table 10.
3. "…an improvement to the mathematical presentation could make reading the paper far easier."
We agree with each of the suggestions, and the revision applies them throughout. The rule is that indices are subscripts in the order layer, neuron, input, time, that a hat marks finite width, and that the superscript marks the extension to inputs.
These suggestions give the paper a single rule for its indices and make Section 3 easier to read, and we thank the reviewer for them. The revision adds a notation table to the appendix (Table 4).
4. "I wonder if the material would not be better split into two: wide-SNN kernel theory plus convergence experiments and a 'when timing matters' investigation."
We considered this and kept one paper, because the kernel theory is what makes the timing question decidable. The rate criterion is a statement about the recursion of Theorem 3.2, namely whether it factors through the time-averaged part of the descriptor, so the second part has no object to test without the first.
We thank the reviewer for the suggestion, which led to a clearer division. The revision marks the two parts in Sections 3 and 4.
5. "Were there no other uses of AI that fall under the required to disclose list?"
There is one more use, and the revision adds it to the AI use statement. Beyond polishing the text, searching for related work and drawing the figures, which the submission declared, we used generative AI tools to audit the proofs for correctness. This is laborious checking work that does not change the contributions of the paper, and as the statement says we have reviewed all AI-assisted work and take responsibility for the final content.
We thank the reviewer for the question, which made the statement more complete.
6. "Do you have thoughts on how this theoretical foundation could (or could not) be used in theoretical neuroscience, where there is a similar rate vs. timing question?"
We think it gives that question a testable form for feedforward pathways, and that it stops at recurrent circuits.
- What a population sees. In the theory a population with Gaussian weights sees the population before it only through the mean spike train over neurons, which is the population peri-stimulus time histogram, and the joint second moment across time bins (Theorem 3.1).
- Link to mean-field theory. The need for a second-order variable beside the rate matches the exact mean-field reductions of spiking populations, which close only when a synchrony variable is carried along (Montbrió et al., 2015).
What the framework offers and where it stops:
| Question in neuroscience | What the framework offers | Where it stops |
|---|---|---|
| does a downstream area read spike timing or only rates | the question becomes whether the non-constant part of the second moment survives the synaptic filter, moves threshold crossings and reaches the readout | a feedforward pathway with independent weights, since in a recurrent circuit the weights that act on a spike train also helped to produce it |
| at what temporal resolution is a code a timing code | the index , the coarsest window that leaves the readout kernel unchanged within a tolerance, which gives the encoding time window of Theunissen & Miller (1995) a definition in terms of the network | a fixed finite horizon |
| can equal firing rates hide a difference that matters downstream | Theorem 3.5, where populations with equal counts and equal histograms but different pairwise timing change the second moment of the firing rates one layer downstream | proved for the random-weight prior, a diagnostic for trained or biological circuits |
| how to test it in data | shuffles of spike times that preserve counts, with the prediction that their effect follows the transmitted difference and the alignment with the task and not the amount of timing structure | structured connectivity and plasticity are outside the theory |
We thank the reviewer for the question, which gave the paper a statement of what it offers to neuroscience and where it stops. The revision adds it to the discussion of neural coding in Section 2.
We are here for an in-depth discussion and further feedback. We would be glad to answer and address any issue you raise, and to keep improving the paper together with you.
We thank the reviewer for the careful reading and for working through the condition of Theorem 3.3. Each point below led to a change in the paper.
1. "It is not possible to read the main text alone and understand the paper."
We agree, and in the revision the main text stands on its own. The objects it uses, with their types, in the notation of point 3 and with the seven that the reviewer names first:
The predictions that Section 4 tests:
We thank the reviewer for this point and for naming the objects one by one, which gave the main text the definitions it lacked. The revision adds them where each object first appears.