- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes the Nonlinear-Readout Dendritic Neuron (NR-DN), a channel-preserving spiking neuron that can replace pointwise activations in sequence backbones. It has three components:
- Dendrite. Each branch holds K compartments with real-valued diagonal linear dynamics (S4D-Real), computed in parallel. The branches come from a shared library of B templates assigned cyclically to channels, with one active branch per neuron by default.
- Distal-Modulated Proximal Readout (DMPR). One compartment is designated proximal. Its state is multiplicatively modulated by a sigmoid-gated, tanh-squashed combination of distal-proximal state differences, loosely inspired by asymmetric NMDA-mediated interactions in pyramidal dendrites.
- Soma. A short learnable causal kernel (W=4, shared across the layer) followed by bounded integer firing.
The theory includes a proposition that DMPR prevents the dendrite-soma cascade from collapsing into a single linear convolution, plus standard bounds on membrane potential and short-window approximation error.
Experiments cover column- and pixel-level sequential CIFAR, SSC, and LRA. With NR-DN replacing GELU in S4, the model reaches 87.53% LRA average, versus 86.09 for S4 and 86.10 for S4-MMDEND. Ablations compare against a soma-only neuron, an MLP readout, an exponential somatic kernel, and varying B.
Critical strengths
-
Competitive results with a lightweight, drop-in design. S4-NRDN improves on both S4 and S4-MMDEND across all six LRA tasks with comparable parameter counts, and the neuron outperforms prior dendritic neurons on S-CIFAR10/100. Because the neuron is channel-preserving and uses real-valued dynamics with one active branch, it is easy to adopt. This is the main reason the paper is of interest to the dendritic/spiking sequence-modeling community.
-
A concrete structural proposal for intra-branch nonlinearity. Most state-space dendritic neurons use linear branch readouts. DMPR is a clearly specified, cheap alternative that keeps parallel computation of compartment states. Even if its biological grounding is loose, it is a reasonable design point to explore.
Critical weaknesses
- The core contribution is not isolated experimentally. DMPR is presented as the main novelty over linear-readout neurons such as PMSN and MMDEND. Yet no ablation keeps the soma, templates, initialization and training recipe fixed while replacing DMPR with a linear readout (w^T z) or the modulation-free proximal readout (gamma=0). The only readout ablation is against a 2-hidden-unit sigmoid MLP, and even there the text overstates the margin: it claims "over 2% on both datasets," but S-CIFAR100 shows +1.45.
This gap matters because gamma is initialized at 0.1 and |m| < tanh(1), so the modulation initially changes the proximal response by at most about +/-7.6%. Without learned gamma values or the missing ablations, the reported gains cannot be attributed to DMPR rather than to the soma, the template library, the tau initialization, or the training schedule.
- The method and appendix are inconsistent, and the theoretical claims are overstated.
- Mismatched formulation. The proof of Proposition 2 refers to an additive distal term s_j rho_j m that does not appear in Eq. 10. Its worked example uses a normalized difference delta_hat = delta / sqrt(delta^2 + eps) that is absent from Eqs. 7-9. Appendix A.2 initializes tilde-parameters whose mapping to alpha, theta, pi is undefined. Together these make it unclear which formulation produced the results.
- Unexplained ablation numbers. Fig. 3(c) reports 84.30 / 59.07 on S-CIFAR10/100 for the free-kernel model, far below the main-table 93.86 / 70.71, with no explanation.
- Modest theory presented as a contribution. Proposition 2 essentially shows that a pipeline with a nonzero nonlinear term is not linear, which is also true of MMDEND's pointwise activation or of integer firing. The soma bounds (Eqs. 16-17) are standard convolution and geometric-tail results, and Appendix B.3 acknowledges that the tail bound need not hold after training.
Questions for authors
- With all other components and training settings fixed, what do S4-NRDN and the S-CIFAR models achieve with (a) a linear readout w^T z and (b) gamma=0? Results on S-CIFAR100 and at least one LRA task would be sufficient.
- What values do gamma and |m| take after training, across layers?
- Which DMPR formulation was used in the reported experiments: with or without the rho_j m term, and with raw or normalized differences? How are the tilde-parameters mapped to alpha, theta, pi?
- What configuration produced the Fig. 3(c) numbers, and why do they differ so much from Table 1?
- What are the standard deviations across the three LRA seeds? How many seeds were used for the other datasets and the ablations? The SSC margin (+0.49 over MMDEND) in particular may fall within seed variance.
3 comments
Q4 (W2). Configuration of Fig. 3(c)
Fig. 3(c) compares the two somatic kernels in the soma-only setting (without the dendrite), so its results are not expected to match the full NR-DN in Table 1. It also differs from the soma-only results in Fig. 3(a), where the kernel length is instead of . All other settings are identical, and we have clarified this in the revised manuscript.
Q5. Seeds and standard deviations
LRA. All LRA results are averaged over three seeds (Table R3-3).
Table R3-3. S4-NRDN on LRA (mean ± std over three seeds).
| ListOps | IMDB | AAN | CIFAR | Pathfinder | Path-X | Avg. |
|---|---|---|---|---|---|---|
| 62.00 ± 0.58 | 89.32 ± 0.48 | 91.66 ± 0.54 | 89.08 ± 0.32 | 95.70 ± 0.35 | 97.42 ± 0.42 | 87.53 |
Other datasets. The S-CIFAR and SSC results in the submission were single runs. We now report the mean ± std over three runs. For SSC, binary firing (, ) works better than the submitted range (, ), improving the accuracy from 76.12 to 77.21 with the same seed, so we adopt it and update Table 1, Fig. 3(a) and Appendix C accordingly.
Table R3-4. NR-DN (mean ± std over three runs) vs. MMDEND (as reported by Wang et al. (2025), who do not report seeds or standard deviations).
| Model | S-CIFAR10 | S-CIFAR100 | SSC |
|---|---|---|---|
| MMDEND | 92.71 | 67.65 | 75.63 |
| NR-DN (ours) | 93.75 ± 0.10 | 70.62 ± 0.21 | 77.11 ± 0.15 |
NR-DN outperforms MMDEND on all three datasets by margins well beyond its run-to-run standard deviation(The ablations in this response are single runs).
W2. Theoretical claim on DMPR's unique advantage
We agree with the reviewer. Proposition 2 only shows that the dendrite–soma combination does not collapse into a single convolution, a property shared by MMDEND's pointwise activation and by integer firing. In the revision, we present it only as a basic property. Likewise, the soma bounds (Eqs. 16–17) are standard results, and we now present them as supporting analysis that motivates the short window rather than as a contribution. We add the following more convincing theoretical analysis on DMPR to distinguish it from other symmetric readouts.
Definition. A readout family is basis-invariant if, for every and every invertible , the map also belongs to . Linear, Linear + activation, MLP, GLU and multiplicative gating are all basis-invariant, since each begins with a weighted sum of .
Proposition. Let be basis-invariant and be diagonalizable. Then any model with can be equivalently realized by , where is defined by .
Consequently, for basis-invariant readouts, (i) relabeling the compartments has no effect, and (ii) intra-branch coupling between compartments can be absorbed into a diagonal system. The detailed proof is included in the revised appendix.
DMPR is not basis-invariant: it operates on fixed coordinates through , with a designated proximal compartment . Its advantage therefore lies in a structured, role-specific inductive bias that exploits compartment identity, rather than in a larger function class. We verify two predictions of this result.
(1) Role assignment. We assign the proximal role to the slowest compartment instead of the fastest one.
Table R3-5. DMPR with different proximal compartments.
| Proximal compartment | S-CIFAR100 | SSC |
|---|---|---|
| Fastest (default) | 70.71 | 77.21 |
| Slowest | 67.67 | 75.31 |
This change cannot affect basis-invariant readouts, but it clearly lowers the accuracy of DMPR (by 3.0% and 1.9%), confirming that DMPR exploits compartment roles.
(2) Intra-branch coupling. We replace the diagonal with a tridiagonal chain , where is the chain Laplacian with coupling , constrained so that all eigenvalues of lie in (i.e., real, stable and non-oscillatory).
Table R3-6. Diagonal vs. tridiagonal under different readouts on S-CIFAR100.
| Readout | Diag. | Tridiag. | (Tri. − Diag.) |
|---|---|---|---|
| Linear + GELU | 68.90 | 69.05 | +0.15 |
| GLU | 69.61 | 69.19 | −0.42 |
| DMPR | 70.71 | 71.51 | +0.80 |
Consistent with the proposition, coupling has little effect on Linear + GELU and GLU but improves DMPR by 0.8%, showing that DMPR can exploit intra-branch structure that standard readouts cannot.
We hope these responses address the reviewer's concerns, and we are happy to provide further clarification. If our responses have resolved your concerns, we would sincerely appreciate it if you could consider raising your score.
Dear Reviewer qkTo,
As the discussion period ends today, we would greatly appreciate it if you could take a look at our response and the revised manuscript. We would be glad to clarify any remaining questions before the deadline.
We sincerely thank the reviewer for the detailed and constructive review, and for recognizing the concrete structural proposal of DMPR. Since W1 and W2 largely overlap with Q1–Q4, we answer them together with the corresponding questions and address the theoretical concerns in W2 at the end. Important results have been incorporated into the revised manuscript.
Q1 (W1). Ablation of the readout method
We replace only DMPR with other readouts, keeping all other components and the training pipeline fixed. Since our Eq. (10) contains an additive term ρjm (see Q3), we obtain the modulation-free readout R=sjz0 by setting m≡0.
Table R3-1. Accuracy (%) with all other components and training settings fixed. DMPR: mean ± std over three runs (see Q5); others: single runs.
DMPR outperforms all other readouts on all three tasks by margins clearly larger than its run-to-run standard deviation, so the gains can be attributed to DMPR rather than to other components. More readouts and separate ablations of the soma and the template library are reported in Fig. 3(a), Fig. 3(d) and our response to Reviewer JXah (Tables R1-1, R1-4 and R1-5). We have also corrected the overstated margin over the MLP readout in Sec. 4.4 to 2.35% (S-CIFAR10) and 1.45% (S-CIFAR100).
Q2 (W1). Learned γ and ∣m∣
The small initial modulation is intentional: DMPR starts close to the proximal response sjz0 for stable early training and then learns how much modulation to use. Table R3-2 reports the statistics on the test set after training (mean over layers with the range across layers in parentheses).
Table R3-2. DMPR statistics after training.
After training, ∣γj∣ grows by an order of magnitude, and the multiplicative modulation changes the proximal response by about 40% on average, compared with at most 7.6% at initialization. The additive term ρjm is also clearly used, especially on SSC. Together with Table R3-1, this shows that the gains stem from the learned distal–proximal modulation.
Q3 (W2). Correct DMPR formulation used in all experiments
We apologize for the inconsistency. All results in the paper and in this response use the formulation below. Eqs. (7)–(10) in the submission omitted the normalization and the additive term, and we have corrected this in the revised manuscript. For template j and distal compartment k≥1,
δk=zk−z0,δ^k=meanc(δk2)+ϵδk,
where the mean is taken over the channels (and spatial positions) of the layer at the same sample, time step and branch slot, with ϵ=10−6. Then
pk=σ(αj,k∣δ^k∣−θj,k),m=tanh(∑k=1K−1πj,kpktanh(δ^k)),
R=sj(z0(1+γjm)+ρjm),I=ηR+ξX.
The parameters are mapped as αj,k=softplus(α~j,k), θj,k=∣θ~j,k∣ and πj,⋅=softmax(π~j,⋅) over k, while γj, ρj, sj, η and ξ are learned directly. They are initialized as α~=6, θ~=0.5, π~=0 (i.e., πj,k=1/(K−1)), γ=0.1, ρ=0, s=η=1 and ξ=0. Proposition 2 still holds for this formulation, since δ^ is odd in the input and the ρjm term cancels under sign reversal.