acceptodds
Under review as a conference paper at ICLR 2027

A Dendritic Neuron with Intra-Branch Nonlinear Modulation for Sequence Modeling

14 views

Abstract

Dendritic neurons enrich sequence models with multiple compartment states that encode temporal memory. However, common branch readouts based on a weighted sum of compartment states do not explicitly distinguish the different functional and computational roles of compartments, restricting how internal state relationships shape the branch output. Inspired by location-dependent, asymmetric interactions within cortical pyramidal dendrites, we propose the distal-modulated proximal readout, which assigns proximal and distal roles to the compartments within each branch and uses their state differences to modulate the proximal response. Building on this readout, we construct the Nonlinear-Readout Dendritic Neuron (NR-DN), which introduces a role-specific nonlinear branch readout while retaining parallel computation of compartment states. The soma of NR-DN complements this temporal processing with a short, learnable causal window and bounded integer firing, allowing diverse frequency responses to the dendritic output. NR-DN preserves channel dimensions and can be integrated into different sequence backbones. Experiments on image, speech, and text sequence tasks demonstrate competitive performance with only one active branch per neuron. S4 equipped with NR-DN achieves 87.53% average accuracy on the Long Range Arena benchmark, demonstrating that NR-DN is an effective neuron model for sequence modeling.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Dendritic Neuron, Sequence Modeling, Nonlinear integration, Brain-Inspired Computing, Spiking Neuron Network

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_qkToOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes the Nonlinear-Readout Dendritic Neuron (NR-DN), a channel-preserving spiking neuron that can replace pointwise activations in sequence backbones. It has three components:

  • Dendrite. Each branch holds K compartments with real-valued diagonal linear dynamics (S4D-Real), computed in parallel. The branches come from a shared library of B templates assigned cyclically to channels, with one active branch per neuron by default.
  • Distal-Modulated Proximal Readout (DMPR). One compartment is designated proximal. Its state is multiplicatively modulated by a sigmoid-gated, tanh-squashed combination of distal-proximal state differences, loosely inspired by asymmetric NMDA-mediated interactions in pyramidal dendrites.
  • Soma. A short learnable causal kernel (W=4, shared across the layer) followed by bounded integer firing.

The theory includes a proposition that DMPR prevents the dendrite-soma cascade from collapsing into a single linear convolution, plus standard bounds on membrane potential and short-window approximation error.

Experiments cover column- and pixel-level sequential CIFAR, SSC, and LRA. With NR-DN replacing GELU in S4, the model reaches 87.53% LRA average, versus 86.09 for S4 and 86.10 for S4-MMDEND. Ablations compare against a soma-only neuron, an MLP readout, an exponential somatic kernel, and varying B.

Critical strengths
  1. Competitive results with a lightweight, drop-in design. S4-NRDN improves on both S4 and S4-MMDEND across all six LRA tasks with comparable parameter counts, and the neuron outperforms prior dendritic neurons on S-CIFAR10/100. Because the neuron is channel-preserving and uses real-valued dynamics with one active branch, it is easy to adopt. This is the main reason the paper is of interest to the dendritic/spiking sequence-modeling community.

  2. A concrete structural proposal for intra-branch nonlinearity. Most state-space dendritic neurons use linear branch readouts. DMPR is a clearly specified, cheap alternative that keeps parallel computation of compartment states. Even if its biological grounding is loose, it is a reasonable design point to explore.

Critical weaknesses
  1. The core contribution is not isolated experimentally. DMPR is presented as the main novelty over linear-readout neurons such as PMSN and MMDEND. Yet no ablation keeps the soma, templates, initialization and training recipe fixed while replacing DMPR with a linear readout (w^T z) or the modulation-free proximal readout (gamma=0). The only readout ablation is against a 2-hidden-unit sigmoid MLP, and even there the text overstates the margin: it claims "over 2% on both datasets," but S-CIFAR100 shows +1.45.

This gap matters because gamma is initialized at 0.1 and |m| < tanh(1), so the modulation initially changes the proximal response by at most about +/-7.6%. Without learned gamma values or the missing ablations, the reported gains cannot be attributed to DMPR rather than to the soma, the template library, the tau initialization, or the training schedule.

  1. The method and appendix are inconsistent, and the theoretical claims are overstated.
  • Mismatched formulation. The proof of Proposition 2 refers to an additive distal term s_j rho_j m that does not appear in Eq. 10. Its worked example uses a normalized difference delta_hat = delta / sqrt(delta^2 + eps) that is absent from Eqs. 7-9. Appendix A.2 initializes tilde-parameters whose mapping to alpha, theta, pi is undefined. Together these make it unclear which formulation produced the results.
  • Unexplained ablation numbers. Fig. 3(c) reports 84.30 / 59.07 on S-CIFAR10/100 for the free-kernel model, far below the main-table 93.86 / 70.71, with no explanation.
  • Modest theory presented as a contribution. Proposition 2 essentially shows that a pipeline with a nonzero nonlinear term is not linear, which is also true of MMDEND's pointwise activation or of integer firing. The soma bounds (Eqs. 16-17) are standard convolution and geometric-tail results, and Appendix B.3 acknowledges that the tail bound need not hold after training.
Questions for authors
  1. With all other components and training settings fixed, what do S4-NRDN and the S-CIFAR models achieve with (a) a linear readout w^T z and (b) gamma=0? Results on S-CIFAR100 and at least one LRA task would be sufficient.
  2. What values do gamma and |m| take after training, across layers?
  3. Which DMPR formulation was used in the reported experiments: with or without the rho_j m term, and with raw or normalized differences? How are the tilde-parameters mapped to alpha, theta, pi?
  4. What configuration produced the Fig. 3(c) numbers, and why do they differ so much from Table 1?
  5. What are the standard deviations across the three LRA seeds? How many seeds were used for the other datasets and the ablations? The SSC margin (+0.49 over MMDEND) in particular may fall within seed variance.
3 comments

We sincerely thank the reviewer for the detailed and constructive review, and for recognizing the concrete structural proposal of DMPR. Since W1 and W2 largely overlap with Q1–Q4, we answer them together with the corresponding questions and address the theoretical concerns in W2 at the end. Important results have been incorporated into the revised manuscript.


Q1 (W1). Ablation of the readout method

We replace only DMPR with other readouts, keeping all other components and the training pipeline fixed. Since our Eq. (10) contains an additive term ρjm\rho_j m (see Q3), we obtain the modulation-free readout R=sjz0R = s_j z_0 by setting m≡0m \equiv 0.

Table R3-1. Accuracy (%) with all other components and training settings fixed. DMPR: mean ± std over three runs (see Q5); others: single runs.

Branch readoutS-CIFAR100SSCLRA ListOps
Proximal only (m≡0m \equiv 0)66.2472.9459.75
Linear, w⊤z+bw^\top z + b66.9670.0259.46
Linear + GELU, GELU(w⊤z+b)\mathrm{GELU}(w^\top z + b)68.9075.7060.57
GLU, (w1⊤z+b1)⋅σ(w2⊤z+b2)(w_1^\top z + b_1)\cdot \sigma(w_2^\top z + b_2)69.6171.0560.12
DMPR (ours)70.62 ± 0.2177.11 ± 0.1562.00 ± 0.58

DMPR outperforms all other readouts on all three tasks by margins clearly larger than its run-to-run standard deviation, so the gains can be attributed to DMPR rather than to other components. More readouts and separate ablations of the soma and the template library are reported in Fig. 3(a), Fig. 3(d) and our response to Reviewer JXah (Tables R1-1, R1-4 and R1-5). We have also corrected the overstated margin over the MLP readout in Sec. 4.4 to 2.35% (S-CIFAR10) and 1.45% (S-CIFAR100).


Q2 (W1). Learned γ\gamma and ∣m∣\lvert m \rvert

The small initial modulation is intentional: DMPR starts close to the proximal response sjz0s_j z_0 for stable early training and then learns how much modulation to use. Table R3-2 reports the statistics on the test set after training (mean over layers with the range across layers in parentheses).

Table R3-2. DMPR statistics after training.

StatisticS-CIFAR100SSC
∣γj∣\lvert \gamma_j \rvert (initialized at 0.1)1.421 (0.675–1.820)1.491 (1.109–2.100)
∣m∣\lvert m \rvert0.310 (0.264–0.363)0.332 (0.177–0.520)
∣γjm∣\lvert \gamma_j m \rvert0.400 (0.228–0.535)0.403 (0.241–0.656)
share of steps with ∣γjm∣>0.05\lvert \gamma_j m \rvert > 0.0581.0% (66.3%–89.9%)87.4% (80.2%–97.7%)
∣ρj∣\lvert \rho_j \rvert (initialized at 0)0.429 (0.112–1.020)2.623 (0.953–3.950)

After training, ∣γj∣\lvert \gamma_j \rvert grows by an order of magnitude, and the multiplicative modulation changes the proximal response by about 40% on average, compared with at most 7.6% at initialization. The additive term ρjm\rho_j m is also clearly used, especially on SSC. Together with Table R3-1, this shows that the gains stem from the learned distal–proximal modulation.


Q3 (W2). Correct DMPR formulation used in all experiments

We apologize for the inconsistency. All results in the paper and in this response use the formulation below. Eqs. (7)–(10) in the submission omitted the normalization and the additive term, and we have corrected this in the revised manuscript. For template jj and distal compartment k≥1k \ge 1,

δk=zk−z0,δ^k=δkmeanc(δk2)+ϵ,\delta_k = z_k - z_0, \qquad \hat\delta_k = \frac{\delta_k}{\sqrt{\mathrm{mean}_c(\delta_k^2) + \epsilon}},

where the mean is taken over the channels (and spatial positions) of the layer at the same sample, time step and branch slot, with ϵ=10−6\epsilon = 10^{-6}. Then

pk=σ(αj,k∣δ^k∣−θj,k),m=tanh⁡(∑k=1K−1πj,kpktanh⁡(δ^k)),p_k = \sigma(\alpha_{j,k} \lvert \hat\delta_k \rvert - \theta_{j,k}), \qquad m = \tanh\Big(\sum_{k=1}^{K-1} \pi_{j,k} p_k \tanh(\hat\delta_k)\Big),

R=sj(z0(1+γjm)+ρjm),I=ηR+ξX.R = s_j \big( z_0 (1 + \gamma_j m) + \rho_j m \big), \qquad I = \eta R + \xi X.

The parameters are mapped as αj,k=softplus(α~j,k)\alpha_{j,k} = \mathrm{softplus}(\tilde\alpha_{j,k}), θj,k=∣θ~j,k∣\theta_{j,k} = \lvert \tilde\theta_{j,k} \rvert and πj,⋅=softmax(π~j,⋅)\pi_{j,\cdot} = \mathrm{softmax}(\tilde\pi_{j,\cdot}) over kk, while γj\gamma_j, ρj\rho_j, sjs_j, η\eta and ξ\xi are learned directly. They are initialized as α~=6\tilde\alpha = 6, θ~=0.5\tilde\theta = 0.5, π~=0\tilde\pi = 0 (i.e., πj,k=1/(K−1)\pi_{j,k} = 1/(K-1)), γ=0.1\gamma = 0.1, ρ=0\rho = 0, s=η=1s = \eta = 1 and ξ=0\xi = 0. Proposition 2 still holds for this formulation, since δ^\hat\delta is odd in the input and the ρjm\rho_j m term cancels under sign reversal.


Q4 (W2). Configuration of Fig. 3(c)

Fig. 3(c) compares the two somatic kernels in the soma-only setting (without the dendrite), so its results are not expected to match the full NR-DN in Table 1. It also differs from the soma-only results in Fig. 3(a), where the kernel length is TT instead of 44. All other settings are identical, and we have clarified this in the revised manuscript.


Q5. Seeds and standard deviations

LRA. All LRA results are averaged over three seeds (Table R3-3).

Table R3-3. S4-NRDN on LRA (mean ± std over three seeds).

ListOpsIMDBAANCIFARPathfinderPath-XAvg.
62.00 ± 0.5889.32 ± 0.4891.66 ± 0.5489.08 ± 0.3295.70 ± 0.3597.42 ± 0.4287.53

Other datasets. The S-CIFAR and SSC results in the submission were single runs. We now report the mean ± std over three runs. For SSC, binary firing (L=0L = 0, U=1U = 1) works better than the submitted range (L=−4L = -4, U=4U = 4), improving the accuracy from 76.12 to 77.21 with the same seed, so we adopt it and update Table 1, Fig. 3(a) and Appendix C accordingly.

Table R3-4. NR-DN (mean ± std over three runs) vs. MMDEND (as reported by Wang et al. (2025), who do not report seeds or standard deviations).

ModelS-CIFAR10S-CIFAR100SSC
MMDEND92.7167.6575.63
NR-DN (ours)93.75 ± 0.1070.62 ± 0.2177.11 ± 0.15

NR-DN outperforms MMDEND on all three datasets by margins well beyond its run-to-run standard deviation(The ablations in this response are single runs).


W2. Theoretical claim on DMPR's unique advantage

We agree with the reviewer. Proposition 2 only shows that the dendrite–soma combination does not collapse into a single convolution, a property shared by MMDEND's pointwise activation and by integer firing. In the revision, we present it only as a basic property. Likewise, the soma bounds (Eqs. 16–17) are standard results, and we now present them as supporting analysis that motivates the short window rather than as a contribution. We add the following more convincing theoretical analysis on DMPR to distinguish it from other symmetric readouts.

Definition. A readout family F\mathcal{F} is basis-invariant if, for every f∈Ff \in \mathcal{F} and every invertible P∈RK×KP \in \mathbb{R}^{K \times K}, the map z↦f(Pz)z \mapsto f(Pz) also belongs to F\mathcal{F}. Linear, Linear + activation, MLP, GLU and multiplicative gating are all basis-invariant, since each begins with a weighted sum of zz.

Proposition. Let F\mathcal{F} be basis-invariant and Aˉ=PΛP−1\bar{A} = P \Lambda P^{-1} be diagonalizable. Then any model (Aˉ,bˉ,f)(\bar{A}, \bar{b}, f) with f∈Ff \in \mathcal{F} can be equivalently realized by (Λ,P−1bˉ,f′)(\Lambda, P^{-1}\bar{b}, f'), where f′∈Ff' \in \mathcal{F} is defined by f′(z)=f(Pz)f'(z) = f(Pz).

Consequently, for basis-invariant readouts, (i) relabeling the compartments has no effect, and (ii) intra-branch coupling between compartments can be absorbed into a diagonal system. The detailed proof is included in the revised appendix.

DMPR is not basis-invariant: it operates on fixed coordinates through δk=zk−z0\delta_k = z_k - z_0, with a designated proximal compartment z0z_0. Its advantage therefore lies in a structured, role-specific inductive bias that exploits compartment identity, rather than in a larger function class. We verify two predictions of this result.

(1) Role assignment. We assign the proximal role to the slowest compartment instead of the fastest one.

Table R3-5. DMPR with different proximal compartments.

Proximal compartmentS-CIFAR100SSC
Fastest (default)70.7177.21
Slowest67.6775.31

This change cannot affect basis-invariant readouts, but it clearly lowers the accuracy of DMPR (by 3.0% and 1.9%), confirming that DMPR exploits compartment roles.

(2) Intra-branch coupling. We replace the diagonal Aˉ\bar{A} with a tridiagonal chain Aˉ=I−diag(τ−1)−L(c)\bar{A} = I - \mathrm{diag}(\tau^{-1}) - L(c), where L(c)L(c) is the chain Laplacian with coupling c≥0c \ge 0, constrained so that all eigenvalues of Aˉ\bar{A} lie in (0,1)(0, 1) (i.e., real, stable and non-oscillatory).

Table R3-6. Diagonal vs. tridiagonal Aˉ\bar{A} under different readouts on S-CIFAR100.

ReadoutDiag.Tridiag.Δ\Delta (Tri. − Diag.)
Linear + GELU68.9069.05+0.15
GLU69.6169.19−0.42
DMPR70.7171.51+0.80

Consistent with the proposition, coupling has little effect on Linear + GELU and GLU but improves DMPR by 0.8%, showing that DMPR can exploit intra-branch structure that standard readouts cannot.


We hope these responses address the reviewer's concerns, and we are happy to provide further clarification. If our responses have resolved your concerns, we would sincerely appreciate it if you could consider raising your score.

Dear Reviewer qkTo,

As the discussion period ends today, we would greatly appreciate it if you could take a look at our response and the revised manuscript. We would be glad to clarify any remaining questions before the deadline.

Reviewer_HRUmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a biologically-inspired sequence model, the Nonlinear-Readout Dendritic Neuron (NR-DN). It uses the Distal-Modulated Proximal Readout (DMPR) to introduce assymetry into dendritic-inspired processing, and also proposes and ablates a free somatic kernel at the soma level. The model shows strong performance on sequence modeling tasks, including several tasks from the long-range arena, while being more computationally efficient than comparable models. Both main components are shown in ablation studies to actively contribute to performance.

Critical strengths

Overall, the paper is quite clearly written, the figures are useful for understanding, it makes its contributions and inspirations clear (which look novel according to my understanding of the literature), shows good performance, and links this improved performance in ablation studies to the methodological innovations it proposes.

Critical weaknesses
  • My main critique is with the tension between the framing as clearly biologically and neuroscientifically inspired, and the paper using a fair amount of neuroscientific jargon without too much explanation (NMDA, pyramidal neurons, dendrites, soma etc.), but the evaluation itself is purely functional in terms of accuracy and efficiency. If the paper is only about getting better/more efficient sequence models, this is fine, but it’s not as satisfying if your goal is to understand how biological principles interact with AI. It would significantly strengthen the scope of the paper if it were to investigate to what extent the model actually learns the intended neuroscientific mechanisms, specifically, the asymmetric, NMDA-like modulation of proximal responses by distal compartments. For instance, can we visualize the activation patterns across compartments to see if they resemble real dendritic integration (e.g., distal inputs lowering the threshold for proximal spikes)? And could we compare the resulting patterns to neuroscientific recordings etc.?

  • I concede that this is not necessary for an ML paper where the goal is to improve sequence modeling, which is clearly the author's intended framing, but these are points that would still strengthen the paper, and should be addressed in the discussion, which is currently very short.

Questions for authors
  • "At the inference stage, the soma follows the method of Luo et al. (2024) to convert integer values into binary spikes, thus reducing the total MACs introduced by integer spikes." This was hard to understand without knowing the method already, so this could be briefly clarified in the manuscript directly.
  • Could the authors perform a simple ablation where the distal-proximal difference mechanism is disabled (e.g., setting the modulation signal $ m = 0 $) to clarify whether the observed performance gains depend on the asymmetric, NMDA-like interaction, or if they arise from other components of the architecture?
  • Figure 1: the role of d could be clarified in the Figure (i.e. it looks like you per default only route to one branch). In the text, the primary focus is on d=1, but in the figure it still is slightly ambiguous.
3 comments

We sincerely thank the reviewer for the insightful suggestions. We address each point below. Important results and clarifications have been incorporated into the revised manuscript.


W1&W2. Balance between neuroscientific realism and computational modeling

We agree that fitting to neural data or mechanism would further strengthen the work, and we would first like to clarify our scope. As the reviewer pointed out, NR-DN is a brain-inspired neuron model: findings on biological dendrites guide its design, but our goal is a more expressive and efficient sequence model rather than a biophysically faithful neuron. For example, all compartments of a branch receive the same input, whereas biological synapses contact specific dendritic locations. A quantitative comparison with neural recordings is therefore beyond the scope of this work. We regard it as an interesting future direction and state it in the revised Sec. 5 (Discussion and Conclusion). We also add an appendix section that maps each biological concept to its model component (e.g., dendritic branch → branch template, proximal/distal locations → the responsive and modulatory compartment roles in DMPR, NMDA-spike-like local threshold → the gate pkp_k).

Nevertheless, following the reviewer's suggestion, we test whether the trained model exhibits the intended mechanism with an paired-stimulation experiment of Behabadi et al. (2012), in which distal excitation facilitates proximally driven responses. For each NR-DN layer of a trained model, we keep the layer input fixed and shift the distal compartment states of a random 10% of its neurons, zk←zk+Δσkz_k \leftarrow z_k + \Delta \sigma_k for k≥1k \ge 1, where σk\sigma_k is the standard deviation of zkz_k on test samples and Δ>0\Delta > 0 (<0< 0) corresponds to depolarization (hyperpolarization). The proximal state z0z_0 is unchanged. We then measure the somatic firing rate of these neurons, averaged over all NR-DN layers and five random subsets.

Table R2-1. Stimulation experiment. Firing rate is computed per virtual time steps after the integer-to-spike conversion (see Q1).

Distal offset Δ\DeltaS-CIFAR100 firing rateSSC firing rate
−1-10.390.24
000.430.25
+1+10.540.32
+2+20.570.31

With the proximal input unchanged, distal depolarization raises the somatic firing rate by up to about 30%, and hyperpolarization lowers it. The distal compartments thus modulate the proximally driven response, which is qualitatively consistent with the distal facilitation reported by Behabadi et al. (2012).


Q1. Conversion from integer values to binary spikes

Thank you for pointing this out. We have replaced this sentence with an explicit description.

During training, the soma emits an integer St=⌊clip(vt,L,U)⌋∈{L,…,U}S_t = \lfloor \mathrm{clip}(v_t, L, U) \rfloor \in \lbrace L, \dots, U \rbrace at each time step (Sec. 3.4). During inference, each time step is expanded into D=max⁡(∣L∣,U)D = \max(|L|, U) virtual steps u=1,…,Du = 1, \dots, D, and StS_t is emitted as ∣St∣|S_t| unit spikes carrying the sign of StS_t:

St,u=sgn(St)⋅I[u≤∣St∣]∈{−1,0,1},St=∑u=1DSt,u.S_{t,u} = \mathrm{sgn}(S_t) \cdot \mathbb{I}[u \le |S_t|] \in \lbrace -1, 0, 1 \rbrace, \qquad S_t = \sum_{u=1}^{D} S_{t,u}.

Since the layer following the soma is linear (a convolution or projection, with BatchNorm folded into its weights at inference), training and inference are equivalent:

WlSt=∑u=1DWlSt,u.W^l S_t = \sum_{u=1}^{D} W^l S_{t,u}.

Because St,u∈{−1,0,1}S_{t,u} \in \lbrace -1, 0, 1 \rbrace, each term only adds or subtracts a weight, so the MACs introduced by integer activations become ACs. A neuron emits only positive spikes when St>0S_t > 0 and only negative spikes when St<0S_t < 0; for L≥0L \ge 0 (e.g., PathX), all spikes are binary in {0,1}\lbrace 0, 1 \rbrace, and for (L,U)=(0,1)(L, U) = (0, 1) no conversion is needed.


Q2. Ablation on DMPR's asymmetric non-linear design

We set the modulation signal to m≡0m \equiv 0 in Eq. (10). This removes both the multiplicative modulation and the additive distal term, so the branch output reduces to the proximal response R=sjz0R = s_j z_0 and the distal compartments no longer affect the output, all other components are unchanged. We compare it with the full DMPR and with some other classic symmetric standard readouts that use all compartment states (more readouts are reported in our response to Reviewer JXah). We also report DMPR with reversed roles, i.e., the last(slowest) compartment as the proximal one.

Table R2-2. Accuracy (%) with the distal–proximal modulation disabled or replaced.

Branch readoutS-CIFAR10S-CIFAR100SSC
Proximal only (m=0m = 0)89.6566.2472.94
Linear, w⊤z+bw^\top z + b90.4366.9670.02
GLU92.4769.6171.05
DMPR, last compartment as proximal92.0567.6775.31
DMPR (ours)93.8670.7177.21

Disabling the modulation lowers the accuracy by about 4% on all three datasets, and adding the distal states linearly or through GLU does not close this gap (on SSC, it is even below the proximal response alone). Reversing the compartment roles also degrades DMPR. Hence, the gains depend on the asymmetric distal-to-proximal interaction rather than on nonlinearity alone.


Q3. Role of dd in Figure 1

Thank you for the suggestion. We have revised Figure 1 so that, for each neuron, only the routed branch template is highlighted and the others are grayed out, and we have added to the caption that all experiments use d=1d = 1 (one active branch per neuron), while d>1d > 1 is supported as described in Appendix A.1.


We hope these responses address the reviewer's concerns, and we are happy to provide further clarification. If our responses have resolved your concerns, we would sincerely appreciate it if you could consider raising your score.

Thank you for the updated results and the detailed responses. One reason I raised the neuroscience applications is that the primary area was listed as “applications to neuroscience & cognitive science”. This is potentially misleading, since the paper’s main contribution lies elsewhere. Beyond this point, I maintain my impression that this is a solid paper and well presented.

Reviewer_JXahOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the Nonlinear-Readout Dendritic Neuron (NR-DN), a novel dendritic neuron model tailored for sequence modeling. The authors argue that existing multi-compartment dendritic neurons typically aggregate compartmental states via linear weighting, thereby overlooking the nonlinear interactions among different compartments. To address this limitation, they introduce Distal-Modulated Proximal Readout (DMPR), which nonlinearly modulates the proximal response based on the state discrepancy between distal and proximal compartments. Furthermore, the authors employ a real-valued state space model (SSM) to enable parallel compartmental state updates, coupled with learnable somatic temporal convolutions and an integer spike firing mechanism to enhance temporal feature modeling. Experimental results demonstrate that the proposed approach achieves competitive performance across image, speech, and long-range sequence modeling tasks.

Critical strengths
  1. brain-inspired: The paper rightly points out that existing multi-compartment neuron models typically rely on linear aggregation, thereby overlooking the nonlinear interactions among different compartments. Drawing inspiration from the asymmetric modulation mechanism between proximal and distal dendrites in biological neurons, the authors propose a well-targeted improvement. Overall, the research motivation and design rationale are sound.

  2. efficiency: DMPR introduces nonlinear inter-compartmental interactions while preserving the parallelizability of state updates across the temporal dimension. Furthermore, employing a real-valued state space model circumvents the computational overhead associated with complex-valued operations, endowing the overall design with notable computational efficiency advantages.

  3. generalization: The authors evaluate the proposed method across multiple sequence modeling benchmarks, including Sequential CIFAR, SSC, and LRA, spanning diverse modalities such as vision, audio, and text. Notably, S4-NRDN achieves an average accuracy of 87.53% on the LRA benchmark, outperforming both the original S4 baseline and several existing dendritic neuron methods.

Critical weaknesses
  • The authors only compare DMPR against a small MLP. Why were common nonlinear interaction mechanisms, such as GLU or multiplicative gating, omitted from the comparison? Are the current empirical results sufficient to justify the necessity of DMPR?

  • There is a substantial gap between the S-CIFAR10/100 results reported in Table 1 and those shown in Figure 3(c). Could the authors clarify the underlying cause of this discrepancy and confirm whether the experimental configurations are consistent across both settings?

  • The paper only reports theoretical FLOPs, lacking practical metrics such as wall-clock training time, inference latency, and GPU memory footprint. Given that DMPR introduces additional nonlinear operations, could the authors provide a more comprehensive efficiency comparison?

  • NR-DN simultaneously introduces DMPR, a Free Somatic Kernel, and Integer-valued Firing. Could the authors provide a more thorough ablation study to disentangle and quantify the specific contribution of each component to the overall performance gain?

  • Considering that the real-valued state updates, temporal convolutions, and integer spike firing are closely tied to existing literature, what unique theoretical or empirical advantages does DMPR offer over standard nonlinear gating mechanisms?

Questions for authors

see section weakness.

4 comments

We sincerely thank the reviewer for the careful reading and constructive comments. We address each concern below. Important empirical results and clarifications have been incorporated into the revised manuscript.


W1. Comparison with linear and other standard nonlinear readouts

Thank you for this suggestion. We additionally compare DMPR with five other branch readouts applied to the KK compartment states z∈RKz \in \mathbb{R}^K of a branch: Linear, Linear + GELU, GLU, multiplicative gating (SwiGLU-type) and MLP. All other components and the training pipeline are fixed.

Table R1-1. Accuracy (%) with different branch readouts (K=4K=4).

ReadoutDefinition#Params / templateS-CIFAR10S-CIFAR100SSC
Linearw⊤z+bw^\top z + b590.4366.9670.02
Linear + GELUGELU(w⊤z+b)\mathrm{GELU}(w^\top z + b)592.5768.9075.70
GLU(w1⊤z+b1)⋅σ(w2⊤z+b2)(w_1^\top z + b_1)\cdot \sigma(w_2^\top z + b_2)1092.4769.6171.05
Mult. gating(w1⊤z+b1)⋅SiLU(w2⊤z+b2)(w_1^\top z + b_1)\cdot\mathrm{SiLU}(w_2^\top z + b_2)1093.2568.2269.87
MLPone hidden layer (width 2)1391.5169.2674.85
DMPR (ours)Eqs. (7)–(10)1293.75 ± 0.1070.62 ± 0.2177.11 ± 0.15

The purely linear readout, under which the dendrite and soma collapse into a single convolution (Proposition 1), is among the weakest. DMPR achieves the best accuracy on all three datasets with a comparable number of parameters, indicating that its benefit comes from the proximal–distal design rather than from nonlinearity alone. Our response in W5 further discusses what distinguishes DMPR from these readouts theoretically.


W2. Discrepancy between Table 1 and Fig. 3(c)

We apologize for the confusion. Fig. 3(c) compares two somatic kernels in the soma-only setting (i.e., without the dendrite), so its results are not expected to match the full NR-DN in Table 1. It also differs from the soma-only results in Fig. 3(a) because the kernel length is TT in Fig. 3(a) and 44 in Fig. 3(c). All other configurations are consistent.

Table R1-2. Settings behind the reported S-CIFAR results.

Neuron SettingSomatic kernelS-CIFAR10S-CIFAR100Source
Soma-onlyFree, length TT87.1663.43Fig. 3(a)
Soma-onlyFree, length 4484.3059.07Fig. 3(c)
Soma-onlyExponential81.2357.34Fig. 3(c)
Full NR-DNFree, length 4493.8670.71Table 1

We have clarified this in the captions and text of the revised manuscript. A kernel ablation within the full NR-DN is given in our response to W4.


W3. Practical efficiency

We measure the training time per iteration and the peak GPU memory on LRA ListOps and CIFAR (a single RTX 5090 GPU, batch sizes as in Table 4, averaged over 100 iterations after warm-up).

Table R1-3. Training cost on LRA.

ModelListOps time (ms/iter)ListOps memory (GB)CIFAR time (ms/iter)CIFAR memory (GB)
S4-GELU482.84856.62
S4-SPSN682.841367.14
S4-NRDN, linear readout1834.6745514.2
S4-NRDN (ours)2084.8949314.5

NR-DN does incur a notable overhead compared with point neurons. However, the linear-readout variant shows that DMPR itself accounts for only about 10% of the training time, so most of the cost comes from the FFT convolutions of the K=4K=4 compartment states and the activation recomputation in the backward pass. Since the other multi-compartment dendritic neurons (MMDEND, DRF) have no public implementation, a direct wall-clock comparison is not possible. In terms of FLOPs, NR-DN is about 15% and 31% cheaper than MMDEND and DRF, respectively (Table 3).


W4. Contribution of each component

The dendrite has already been ablated in two ways: removing it entirely (soma-only, Fig. 3(a)) and replacing DMPR with other readouts (Fig. 3(b) and Table R1-1). The effect of the number of branch templates is shown in Fig. 3(d).

We further ablate the integer-valued firing range and the somatic kernel within the full NR-DN using the same seed as the reported result in Table 1.

Table R1-4. Firing range (L,U)(L, U) on S-CIFAR100.

(L,U)(L, U)Output levelsS-CIFAR100
(0,1)(0, 1)binary67.25
(0,4)(0, 4)unsigned, 5 levels69.43
(−1,1)(-1, 1)ternary66.70
(−4,4)(-4, 4)signed, 9 levels70.71

Table R1-5. Somatic kernel within the full NR-DN.

KernelS-CIFAR100SSC
Exponential kernel70.0774.60
Free kernel70.7177.21

Multiple firing levels bring the main gain over binary firing, and signed levels improve further on top of them. The free kernel outperforms the exponential one on both datasets, especially on SSC. Together with Table R1-1, these results show that DMPR, the multi-level firing and the free kernel all contribute to the performance of NR-DN.


W5. Unique advantages of DMPR over standard nonlinear gating

We agree that the real-valued diagonal state update, the sliding-window somatic integration and integer-valued firing build on prior work (Gu et al., 2022; Fang et al., 2023; Luo et al., 2024), which we cite. DMPR is our core contribution and the free-kernel soma is a secondary one, which we now state explicitly in the revised manuscript. Below we characterize what distinguishes DMPR theoretically and verify it with two experiments.

Definition. A readout family F\mathcal{F} is basis-invariant if, for every f∈Ff \in \mathcal{F} and every invertible P∈RK×KP \in \mathbb{R}^{K \times K}, the map z↦f(Pz)z \mapsto f(Pz) also belongs to F\mathcal{F}. Linear, Linear + activation, MLP, GLU and multiplicative gating are all basis-invariant, since each begins with a weighted sum of zz.

Proposition. Let F\mathcal{F} be basis-invariant and Aˉ=PΛP−1\bar{A} = P \Lambda P^{-1} be diagonalizable. Then any model (Aˉ,bˉ,f)(\bar{A}, \bar{b}, f) with f∈Ff \in \mathcal{F} can be equivalently realized by (Λ,P−1bˉ,f′)(\Lambda, P^{-1}\bar{b}, f'), where f′∈Ff' \in \mathcal{F} is defined by f′(z)=f(Pz)f'(z) = f(Pz).

Consequently, for basis-invariant readouts, (i) relabeling the compartments does not change the functions they can represent, and (ii) intra-branch coupling between compartments can be absorbed into a diagonal system. The detailed proof is included in the revised appendix.

DMPR is not basis-invariant: it operates on fixed coordinates through δk=zk−z0\delta_k = z_k - z_0, with a designated proximal compartment z0z_0. Its advantage therefore lies in a structured, role-specific inductive bias that fixes the roles of the compartments a priori, rather than in a larger function class. We verify two predictions of this result.

(1) Role assignment. We assign the proximal role to the slowest compartment instead of the fastest one.

Table R1-6. DMPR with different proximal compartments.

Proximal compartmentS-CIFAR100SSC
Fastest (default)70.7177.21
Slowest67.6775.31

This change does not affect the functions that a basis-invariant readout can represent, but it clearly lowers the accuracy of DMPR (by 3.0% and 1.9%), showing that the performance of DMPR depends on its role assignment.

(2) Intra-branch coupling. We replace the diagonal Aˉ\bar{A} with a tridiagonal chain Aˉ=I−diag(τ−1)−L(c)\bar{A} = I - \mathrm{diag}(\tau^{-1}) - L(c), where L(c)L(c) is the chain Laplacian with coupling c≥0c \ge 0, constrained so that all eigenvalues of Aˉ\bar{A} lie in (0,1)(0, 1) (i.e., real, stable and non-oscillatory).

Table R1-7. Diagonal vs. tridiagonal Aˉ\bar{A} under different readouts on S-CIFAR100.

ReadoutDiag.Tridiag.Δ\Delta (Tri. − Diag.)
Linear + GELU68.9069.05+0.15
GLU69.6169.19−0.42
DMPR70.7171.51+0.80

Consistent with the proposition, coupling has little effect on Linear + GELU and GLU but improves DMPR by 0.8%, indicating that DMPR can benefit from intra-branch coupling.


We hope these responses address the reviewer's concerns, and we are happy to provide further clarification. If our responses have resolved your concerns, we would sincerely appreciate it if you could consider raising your score.

I thank the authors for the additional baseline experiments and ablation analyses, which have alleviated some of my concerns. However, the evidence supporting DMPR’s core advantages remains insufficient. The additional basis-invariance analysis clarifies differences in parameterization and inductive bias across readouts, but it does not establish that standard gating mechanisms cannot exploit compartmental structure. Moreover, the relevant baseline comparisons and key ablations are still based on single runs, which do not sufficiently establish the robustness of the reported performance gains. Therefore, my concerns about the core contribution remain unresolved, and I am maintaining my original score.

We thank the reviewer for the follow-up and are glad that the additional baselines and ablations have alleviated some of the concerns. We address the two remaining points below.


1. What basis-invariance shows, and comparison with standard gating over three runs

We agree with the reviewer and correct an minor overstatement in our previous response. The basis-invariance property of standard gating mechanisms does not show that they definitely cannot exploit compartmental structure. As pointed out in our revised manuscript, what Proposition 3 shows is that for such readouts, relabeling or coupling the compartments does not change the set of functions they can represent. The roles and structures of the compartments are therefore not specified in advance and have to be learned from data. DMPR instead fixes the roles a priori: the fastest compartment is proximal, and other distal compartments act through their differences to it. Our claim is thus that this prior is beneficial, not that DMPR has a larger function class (Sec. 3.3). We tested this claim in Table R1-6 and R1-7, where the results align with our claim. We therefore believe this claim is proven both theoretically and empirically. Now we repeat the two gating mechanisms in Table R1-1 over three runs:

Table R1-8. DMPR vs. standard gating mechanisms (accuracy, %, mean ± std over three runs).

ReadoutS-CIFAR10S-CIFAR100SSC
GLU92.44 ± 0.1269.45 ± 0.1770.87 ± 0.25
Mult. gating93.13 ± 0.1668.35 ± 0.2369.95 ± 0.10
DMPR (ours)93.75 ± 0.1070.62 ± 0.2177.11 ± 0.15

DMPR outperforms both gating mechanisms on all three datasets by margins clearly larger than their standard deviations, especially on SSC, supporting the benefit of its role-specific prior.


2. Robustness of the performance gains

We would like to argue that for comparisons with large gaps, the run-to-run variation of DMPR already indicates that the gains are robust. In Table R1-1, the gap between DMPR and the strongest baseline is 0.50, 1.01 and 1.41 on S-CIFAR10, S-CIFAR100 and SSC, i.e., about 5, 5 and 9 times the standard deviation of DMPR. Assigning the proximal role to the slowest compartment (Table R1-6) lowers the accuracy by about 12–14 times this standard deviation. The gating baselines in Table R1-8 show standard deviations (0.10–0.25) similar to those of DMPR, which supports this comparison.

The differences in the coupling ablation (Table R1-7) are of the same order as the standard deviation, so we repeat it over three runs:

Table R1-9. Diagonal vs. tridiagonal Aˉ\bar{A} on S-CIFAR100 (accuracy, %, mean ± std over three runs).

ReadoutDiag.Tridiag.Δ\Delta (Tri. − Diag.)
Linear + GELU69.11±0.1669.17±0.24+0.06
GLU69.45±0.1769.12±0.16-0.33
DMPR70.62 ± 0.2171.62±0.19+1.00

Coupling has little effect on Linear + GELU and slightly lowers GLU, whereas it improves DMPR by 1.0%, about five times its standard deviation. This is in line with Proposition 3, since coupling does not change the functions that a basis-invariant readout can represent, while DMPR can benefit from it.


We hope these clarifications and results address the remaining concerns, and we would sincerely appreciate it if the reviewer could reconsider the score.

Reviewer_ziCmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose NR-DN, a channel-preserving dendritic neuron model for sequence modeling. Its dendrite maintains a shared library of learnable branch templates, each containing K compartments updated by a real-valued diagonal state-space model (S4D-Real) that supports parallel FFT-based computation. The core novelty is the Distal-Modulated Proximal Readout (DMPR): one compartment is assigned the proximal responsive role, and the differences between distal and proximal states, passed through an NMDA-inspired sigmoid gate and tanh mixing, multiplicatively modulate the proximal response when forming the branch output. The soma combines a short, freely learnable causal window (W=4) with bounded integer firing trained via a straight-through estimator. Theoretically, the authors show that linear readouts reduce the whole dendrite–soma computation to a single convolution (Prop. 1), while DMPR provably prevents this degeneration (Prop. 2), and they establish boundedness and Lipschitz properties of the membrane potential. Experiments on S-CIFAR10/100, SSC, pixel-by-pixel S-CIFAR10, and LRA show competitive results, with S4-NRDN reaching 87.53% average on LRA.

Critical strengths

1.A precise and valuable observation. Proposition 1 identifies a structural issue that, to my knowledge, has been overlooked in this line of work: with linear compartment updates and a linear readout, the entire multi-compartment dendrite plus soma collapses into a single channel-wise convolution, so the multi-compartment structure adds no pre-activation expressivity beyond a learnable kernel. This is a genuinely useful clarification for the dendritic-neuron subfield, and Proposition 2 correctly shows that DMPR escapes this reduction.

2.Well-motivated and economical design. DMPR introduces role-specific, asymmetric intra-branch interaction at readout while keeping the compartment updates parallel. The design is cheap, channel-preserving, and the cyclic template routing gives heterogeneous dynamics across channels without extra memory.

3.Comparisons are largely controlled and comprehensive. The evaluation covers most classical and recent neuron models (LIF, GLIF, KLIF, PSN family, TC-LIF, PMSN, MMDEND, DRF, SpikingSSM) under matched backbones and parameter budgets, and LRA results are averaged over three seeds. This protocol is appreciably more rigorous than many submissions in this area.

4.Reasonable ablations and transparent efficiency analysis. The soma-only, readout-replacement, kernel-replacement, and template-count ablations address the main modules, and the FLOPS/parameter analysis (57TC vs. 66TC for MMDEND and 81TC for DRF) is clearly reported.

5.Honest positioning of the biological inspiration. The functional-abstraction statement in the Ethics section is appreciated, and the cited location-dependent asymmetric interactions (Behabadi et al., 2012) are an appropriate source of inspiration.

Critical weaknesses

1.The most direct competitor and the most direct ablation are both missing. (a) DendSN (Huang et al., 2024), which the related-work section itself credits with nonlinear intra-branch aggregation, appears in no experiment table; meanwhile Section 4.1 claims NR-DN is "the only dendritic model that introduces nonlinear interactions within a branch," and Table 1 marks only NR-DN as having a nonlinear branch readout.

2.This is internally inconsistent and the claim can only stand by excluding DendSN. (b) The central theoretical claim (Prop. 1) calls for a direct ablation of DMPR versus a plain linear readout (w^Tz followed by the same pointwise activation) within the identical NR-DN architecture; instead, the paper compares against a small MLP readout, which does not isolate the role-specific asymmetry factor.

3.Statistical reporting is incomplete. Only the LRA results are stated to use three seeds; the S-CIFAR and SSC results appear to be single runs without error bars. Some margins are small (e.g., 76.12 vs. 75.63 over MMDEND on SSC), so their significance cannot be judged. The superiority claim needs qualification. On pixel-by-pixel S-CIFAR10 (1024 steps), NR-DN (83.26%) is below DRF (84.30%). The text is carefully worded around this, but the abstract-level impression of uniform superiority is somewhat misleading.

4.Component-level validation of the role assignment is absent. There is no sensitivity analysis on which compartment is designated proximal, no ablation of the sigmoid gate or the tanh sign-preserving term, no sweep of K or W, and the promised d>1 results are not actually presented (Appendix A.1 contains theory only).

5.The spiking aspect is under-quantified. No firing-rate or sparsity statistics are reported, and the MAC reduction from integer-to-binary conversion at inference is only mentioned in one sentence. The firing bounds L, U are set per task by hand, and their effect is not analyzed.

6.The short-window bound is limited. Eq. (17) only characterizes approximation of the exponential kernel at initialization and, as the authors note, does not hold once beta is learned; it should be presented as motivation rather than guarantee. Minor: modern SSM references (S4D, S5, LRU) are absent from the LRA table, and code availability is not stated.

Questions for authors

1.Could the authors kindly add an experimental comparison with DendSN under the same backbone and parameter budget, and revise the "only nonlinear intra-branch model" claim accordingly?

2.Would it be possible to add the direct ablation of DMPR versus a linear readout (with the same pointwise activation) inside the same NR-DN architecture? This seems to be the most direct empirical test of Proposition 1.

3.How many seeds were used for S-CIFAR10/100, SSC, and pixel-level CIFAR? Could mean and standard deviation be reported?

4.Is the performance sensitive to which compartment is designated as proximal? What happens if the sigmoid gate or the tanh in Eq. (9) is removed?

5.Could the authors report firing rates and measured SOPs/MACs at inference, and explain how L and U are chosen for each task?

6.Where are the experimental results for d>1? How sensitive is NR-DN to K and W?

7.As minor suggestions: adding S4D/S5-style references to the LRA table would strengthen context, and releasing code would greatly help reproducibility.

3 comments

We sincerely thank the reviewer for the thorough review and for recognizing the observation of Proposition 1 and the economical design of DMPR. We group related weaknesses and questions and answer them below. Important results and clarifications have been incorporated into the revised manuscript.


1. Comparison with other readouts, including DendSN (W1, W2, Q1, Q2)

Claim. We agree that the statement in Sec. 4.1 was inaccurate, since DendSN (Huang et al., 2024) also introduces a nonlinear intra-branch aggregation. What distinguishes DMPR is that its interaction is role-specific and asymmetric: DendSN aggregates the compartments symmetrically, whereas DMPR assigns proximal and distal roles. We have revised the claim to "the only dendritic model with role-specific, asymmetric intra-branch interaction" and renamed the corresponding column of Table 1. Our response to W5 of Reviewer JXah analyzes this distinction theoretically and empirically: readouts that treat the compartments symmetrically are unaffected by relabeling them, whereas DMPR is not.

Experimental design. The DendSN code is not public, and DendSN was not evaluated on S-CIFAR, SSC or LRA. Moreover, a DendSN layer is not channel-preserving and uses an LIF soma, so a full-neuron comparison would change the backbone width, the soma and the training pipeline at the same time. We therefore implement the DendSN branch aggregation (Eqs. 5 and 7 in Huang et al.) as the branch readout of NR-DN, and keep all other components unchanged. Other standard linear/non-linear readouts are also included.

Table R4-1. Accuracy (%) with different branch readouts in the same NR-DN architecture. DMPR: mean ± std over three runs.

Branch readoutS-CIFAR10S-CIFAR100SSC
Linear, w⊤z+bw^\top z + b90.4366.9670.02
Linear + GELU, GELU(w⊤z+b)\mathrm{GELU}(w^\top z + b)92.5768.9075.70
GLU, (w1⊤z+b1)⋅σ(w2⊤z+b2)(w_1^\top z + b_1)\cdot \sigma(w_2^\top z + b_2)92.4769.6171.05
DendSN aggregation, κψ(∥(z−μ)/ζ∥2)\kappa \psi(\lVert (z - \mu)/\zeta \rVert_2)92.5367.1775.66
DMPR (ours)93.75 ± 0.1070.62 ± 0.2177.11 ± 0.15

The linear readout is the weakest on all three datasets, as expected from Proposition 1, and adding a pointwise activation narrows but does not close the gap to DMPR. DMPR also clearly outperforms the DendSN aggregation on all three datasets, indicating that the gain comes from the role-specific asymmetry rather than from intra-branch nonlinearity alone.


2. Statistical reporting (W3, Q3)

Seeds. In the submission, only the LRA results were averaged over three seeds. We now report the mean ± std over three runs for S-CIFAR10/100, SSC and pixel-level S-CIFAR10 in Table R4-2, and add the LRA standard deviations (0.32 to 0.58) to Table 2. For SSC, we also found that binary firing (L=0L = 0, U=1U = 1) works better than the submitted range (L=−4L = -4, U=4U = 4), raising the accuracy from 76.12 to 77.21 with the same seed, so we adopt it and update the paper accordingly.

Table R4-2. NR-DN (mean ± std over three runs) vs. the best prior result in Table 1 (accuracy, %).

ModelS-CIFAR10S-CIFAR100SSCPixel S-CIFAR10
Best prior92.71 (MMDEND)67.65 (MMDEND)75.63 (MMDEND)84.30 (DRF)
NR-DN (ours)93.75 ± 0.1070.62 ± 0.2177.11 ± 0.1583.16 ± 0.23

On S-CIFAR10/100 and SSC, NR-DN exceeds the best prior results by margins well beyond its standard deviation, including on SSC (1.48% vs. a std of 0.15). On pixel-level S-CIFAR10, NR-DN is below DRF. However, DRF uses a different backbone, so this single result does not establish a general ranking between the two neurons. To avoid an impression of uniform superiority, we now state this result explicitly in Sec. 4.1 of our paper.


3. Role assignment, DMPR components, and KK, WW, dd (W4, Q4, Q6)

DMPR components. We ablate each part of DMPR on S-CIFAR100 and SSC: making the slowest compartment proximal, removing the sigmoid gate (pk≡1p_k \equiv 1), removing the inner tanh of Eq. (9) (i.e., using δ^k\hat\delta_k instead of tanh⁡δ^k\tanh \hat\delta_k) and removing the modulation (m≡0m \equiv 0, proximal response only).

Table R4-3. Ablation of DMPR components (accuracy, %).

VariantS-CIFAR100SSC
DMPR (original)70.62 ± 0.2177.11 ± 0.15
Slowest compartment as proximal67.6775.31
w/o sigmoid gate (pk≡1p_k \equiv 1)70.1675.82
w/o inner tanh in Eq. (9)70.8178.08
w/o modulation (m≡0m \equiv 0)66.2472.94

Swapping the proximal role or removing the modulation clearly lowers accuracy (by about 2–3% and 4%, respectively), so performance depends on the role assignment. Removing the sigmoid gate also lowers accuracy, slightly on S-CIFAR100 and clearly on SSC. In contrast, removing the inner tanh slightly improves S-CIFAR100 and clearly improves SSC, since the outer tanh already bounds mm. Adopting this simpler form would require rerunning all experiments, so we keep the original formulation for consistency and report this as a promising simplification in the revised ablation study and will use it in future work.

KK, WW and dd. Appendix A.1 previously contained only the formulation for d>1d > 1. We have corrected the pointer in Sec. 3 and added the results below. We vary each of KK, WW and dd on S-CIFAR100 with the others at their defaults (K=4K = 4, W=4W = 4, d=1d = 1).

Table R4-4. Sensitivity to KK, WW and dd on S-CIFAR100 (accuracy, %, single runs; the default is the run in Table 1).

DefaultK=2K = 2K=6K = 6W=2W = 2W=32W = 32d=2d = 2
70.7170.6070.6470.4270.9370.34

All settings stay within 0.4% of the default, so NR-DN is robust to these hyperparameters. We keep K=4K = 4, W=4W = 4 and d=1d = 1 as a trade-off between accuracy and cost.


4. Spiking statistics (W5, Q5)

Integer-to-spike conversion. Following MMDEND (Wang et al., 2025), each integer output St∈{L,…,U}S_t \in \lbrace L, \dots, U \rbrace is emitted at inference as D=max⁡(∣L∣,U)D = \max(\lvert L \rvert, U) unit spikes St,u=sgn(St)I[u≤∣St∣]∈{−1,0,1}S_{t,u} = \mathrm{sgn}(S_t) \mathbb{I}[u \le \lvert S_t \rvert] \in \lbrace -1, 0, 1 \rbrace. Since WSt=∑u=1DWSt,uW S_t = \sum_{u=1}^{D} W S_{t,u}, the next linear layer performs only accumulations (ACs). Details are given in the revised Sec. 3.4.

Firing statistics and operations. Table R4-5 reports the firing rate after conversion, the synaptic operations (SOPs) of the layers receiving spikes, the remaining MACs (dendrite, soma and the non-spiking input layer), and the energy estimated with EAC=0.9E_{AC} = 0.9 pJ and EMAC=4.6E_{MAC} = 4.6 pJ as in MMDEND.

Table R4-5. Spiking statistics per test sample, computed with the models of Table 1.

StatisticS-CIFAR100SSC
Firing rate (spikes / neuron / virtual step)0.450.24
SOPs (M)4093.63
MACs (M)34.752.6
Estimated energy (μJ)528245

The firing rates are moderate, and all layers fed by spikes only perform ACs. On SSC, most of the MACs come from the long somatic window (W=T=250W = T = 250).

Choice of LL and UU. LL and UU were selected per task through preliminary trials. We do not yet have a theoretical criterion for them and do not claim that the selected values are optimal. Their effect on S-CIFAR100 is shown in Table R4-6.

Table R4-6. Firing range (L,U)(L, U) on S-CIFAR100.

(L,U)(L, U)Output levelsS-CIFAR100
(0,1)(0, 1)binary67.25
(0,4)(0, 4)unsigned, 5 levels69.43
(−1,1)(-1, 1)ternary66.70
(−4,4)(-4, 4)signed, 9 levels70.71

Signed multi-level firing works best on S-CIFAR100, whereas binary firing is better on SSC (77.21 vs. 76.12). We list the values used for each task in Appendix C and note this task dependence as a limitation in Sec. 5.


5. Minor points (W6, Q7)

Short-window bound. We agree. Eq. (17) holds only at initialization, before β\beta is learned. It is now presented as the motivation for the short window rather than as a guarantee.

LRA references. We have added the non-spiking S4D to Table 2 for context.

Code. Currently we provide code with checkpoints for evaluating the results of 4 datasets in the supplementary material. We will release the full code with instructions upon publication.


We hope these responses address the reviewer's concerns, and we are happy to provide further clarification. If our responses have resolved your concerns, we would sincerely appreciate it if you could consider raising your score.

Dear Reviewer ziCm,

As the discussion period ends today, we would greatly appreciate it if you could take a look at our response and the revised manuscript. We would be glad to clarify any remaining questions before the deadline.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary of Revisions

General Response and Summary of Revisions

We thank all reviewers for their careful reading and constructive comments. We have uploaded a revised manuscript in which all changes are marked in blue, and we provide the code and four trained checkpoints in the supplementary material. The main revisions are summarized below, and detailed answers are given in the individual responses.

1. What distinguishes DMPR (Reviewers JXah, qkTo, ziCm)

  • We add a basis-invariance analysis (Sec. 3.3, Proposition 3 in Appendix A.4). For readouts that start with a learnable linear map(including linear, linear+Act, GLU, Multiplying gating), relabeling the compartments does not change the functions they can represent, and intra-branch coupling can be absorbed into a diagonal system. DMPR is not basis-invariant, so its advantage comes from a role-specific inductive bias rather than a larger function class. Experiments on swapping the proximal role and adding intra-branch coupling support this prediction (Appendix A.4).
  • We compare DMPR with linear, Linear+GELU, GLU, multiplicative gating, MLP, DendSN aggregation and proximal-only readouts (Sec. 4.4, Fig. 3(b), Appendix E.1). DMPR performs best on S-CIFAR10, S-CIFAR100, SSC and ListOps. We also ablate each DMPR component (Appendix E.2).
  • We revise the claim in Sec. 4.1 and Table 1 to the role-specific, asymmetric interaction within a branch.

2. Formulation and theory (Reviewers qkTo, JXah, ziCm)

  • Eqs. (7)–(10) now include the RMS normalization of the compartment differences and the additive term ρjm\rho_j m. Both were used in all experiments but carelessly omitted in the original submission. The parameterization and initialization are given in Appendix A.2, and the formulation for d>1d>1 in Appendices A.1 and A.2.
  • Proposition 2 is presented as a basic property (Appendix A.3). The soma bound in Eq. (17) is presented as the motivation for a short window rather than as a guarantee.

3. Statistical reporting and experimental details (Reviewers ziCm, qkTo, JXah)

  • Mean ± std over three runs for S-CIFAR10/100, SSC and pixel-level S-CIFAR10 (Appendix C, Table 6), and LRA standard deviations and the non-spiking S4D baseline in Table 2.
  • The firing range (L,U)(L, U) of each task is listed in Appendix C. SSC uses binary firing (L=0L=0, U=1U=1), and Table 1 and Fig. 3(a) are updated accordingly. The task dependence of (L,U)(L, U) is noted as a limitation in Sec. 5.
  • We state that NR-DN is slightly below DRF on pixel-level S-CIFAR10 (Sec. 4.1), and we clarify that Fig. 3(c) uses soma-only neurons with kernel length 4.

4. Biological grounding and scope (Reviewer HRUm)

  • A new Discussion and Conclusion (Sec. 5) clarifies the brain-inspired scope of NR-DN and its core contribution (DMPR). Appendix D maps each biological concept to its model component.
  • The integer-to-spike conversion at inference is given in Eq. (18). The caption of Fig. 1 states that all experiments use d=1d=1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.