acceptodds
Under review as a conference paper at ICLR 2027

Learnable Membrane Time Constants for Structure Dynamics in Recurrent Spiking Neural Networks

Abstract

In recurrent spiking neural networks (RSNNs), adapting connectivity structure represents an important means of RSNN optimization. However, their computational capabilities depend not only on connectivity structure but also on intrinsic neuronal dynamics. Previous studies have shown that heterogeneous neuronal timescales can improve learning in SNNs. Although existing approaches have made neuronal dynamics learnable, such adaptation is typically performed within a fixed connectivity structure. This raises the challenge of jointly adapting connectivity structure and heterogeneous neuronal dynamics without relying on backpropagation. To this end, we propose Directional Credit guided Forward Optimization (DCFO), a more biologically inspired, backpropagation-free learning framework that jointly adapts connectivity structure and neuron-specific membrane time constants. DCFO constructs candidate directions separately in the connectivity parameter space and the neuron-specific membrane-time-constant space, providing directional credit for each. The resulting directional credit guides forward-gradient updates in the two parameter spaces, enabling structural–dynamical co-adaptation. We evaluate DCFO on three Brax continuous-control tasks, including the 17-DoF Humanoid. The proposed method outperforms the state-of-the-art Evolving Connectivity method across a range of tasks and achieves performance competitive with deep recurrent neural network baselines. The learned neuronal dynamics further exhibit task-dependent heterogeneous neuronal timescales. These results demonstrate that DCFO enables effective structural–dynamical co-adaptation in RSNNs without relying on backpropagation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Recurrent Spiking Neural Networks,Neural Network Structure Learning,Backpropagation-Free Learning

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_cF56OpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Backpropagation-free learning methods are pivotal for training SNNs on neuromorphic hardware. While most high-performance SNNs rely on surrogate-gradient-based backpropagation, recent black-box zeroth-order optimization approaches, such as Evolving Connectivity, have enabled training recurrent SNNs with binary connectivity without backpropagation. However, optimizing only coarse-grained discrete connectivity reconfigurations fails to fully exploit the computing capacity and adaptability of continuous fine-grained neuronal dynamics. To address this limitation, the paper proposes Directional Credit-guided Forward Optimization (DCFO), a framework that jointly adapts discrete connectivity structures and continuous neuron-specific time constants by reconstructing forward updates from directional credits estimated via zeroth-order evaluations. Evaluations across three continuous-control tasks demonstrate that DCFO outperforms existing backpropagation-free baselines in both task performance and generation efficiency.

Critical strengths

By jointly optimizing coarse-grained discrete connectivity and neuron-specific time constants, the proposed approach goes beyond a simple extension of prior works and effectively applies the unique characteristics of SNNs. Unlike conventional structural optimization methods that merely adapt connectivity, this paper novelly integrates the intrinsic neuronal dynamics of SNNs into the zeroth-order optimization.

The paper presents a principled and convincing framework to effectively apply black-box zeroth-order optimization to SNNs (which is a good research direction), and empirically demonstrates that the method improves both final task performance and efficiency.

Critical weaknesses

While the proposed framework bypasses the memory overhead of BPTT and surrogate gradient dependencies, it trades them for heavy sampling demands (large population sizes; 10240) and high-dimensional matrix solves (Ridge Reconstruction). This trade-off raises concerns regarding scalability to larger networks and makes direct deployment on neuromorphic hardware non-trivial.

The evaluation is restricted to three robot locomotion tasks, which share similar continuous action spaces and physics. The method is not tested on event-based neuromorphic benchmarks or long-range temporal processing tasks where SNNs typically excel.

Questions for authors

How scalable is the population sampling? The method requires a large population size (N=10240N=10240) even for small hidden layers (H=256H=256). As the network scales up, the variance of the zeroth-order credit estimates will drastically increase, necessitating an impractically large population.

Is the ridge solving scalable and neuromorphically implementable? Reconstructing credits requires solving a K×KK \times K ridge regression system. As the parameter dimension grows, increasing KK leads to an O(K3)O(K^3) computational bottleneck. Furthermore, high-precision matrix decomposition may not be executed natively on neuromorphic cores.

Can DCFO be generalized to temporal sequence learning or event-based classification benchmarks like DVS-Gesture or SHD? The current formulation relies on trajectory-level episode returns, which seems to limit its applicability to continuous control tasks.

Could you also list reference results from backpropagation/surrogate-gradient-based upper bounds (e.g., PPO with PLIF-RSNN), so that readers can contextualize the performance and efficiency trade-off of using backpropagation-free methods?

Could you further justify the biological implications? DCFO's main components like the Ridge solving and forward JVP reconstruction seems to lack biological plausibility. Besides being backprop-free, the method remains purely mathematical rather than biologically grounded.

Why is the title termed ‘structure dynamics’? Terms like ‘connectivity-dynamics co-adaptation’ or 'Co-adapting Connectivity and Neuronal Dynamics' may be less misleading.

2 comments

We thank the reviewer for the constructive feedback. We address the questions below.


Q1: How scalable is the population sampling?

Population sampling remains challenging in reward-driven optimization. DCFO reuses sampled reward feedback for directional credit assignment, while ridge reconstruction accounts for correlated candidates and regularizes updates, enabling more effective utilization of each sampled population.


Q2: Is the ridge solving scalable and neuromorphically implementable?

DCFO supports scalable computation through matrix-free iterative solvers such as CG and LSQR, which approximate ridge solutions using matrix–vector products without explicitly constructing the Gram matrix or performing dense factorization. Randomized preconditioning can further improve solver efficiency [3].

For neuromorphic deployment, DCFO can adopt a hybrid architecture in which neuromorphic cores execute RSNN forward computation, while an external CPU/GPU performs ridge reconstruction.


Q3: Can DCFO be generalized to temporal sequence learning or event-based classification benchmarks like DVS-Gesture or SHD?

We extended DCFO to MNIST and SHD using classification-derived reward feedback, obtaining:

MethodMNIST (%)SHD (%)
DEEP R + SG-BPTT98.4971.13
Grad R + SG-BPTT98.4879.89
EC98.1882.08
DCFO (ours)98.0384.70

DCFO requires only scalar reward feedback, rather than environment interaction trajectories. These results demonstrate its applicability to visual and event-based temporal classification.


*Q4: Reference results from backpropagation-based methods?

We added PPO baselines using surrogate-gradient LIF-RSNN and ANN policies:

MethodHumanoidWalker2dHopper
PPO-SG-LIF-RSNN11,8971,853846
PPO (ANN, 50M)7,9032,8811,782
PPO (ANN, 1B)14,9344,2611,822
DCFO (Ours)13,104.316,244.512,725.81

Q5: Could you further justify the biological implications?

Perturbation-driven exploration. DCFO explores the joint adaptation of connectivity structure and neuron-specific membrane time constants through parameter perturbations. Bernoulli connectivity sampling and antithetic perturbations of membrane time constants under the same sampled connectivity enable exploration of structural and neuronal dynamical variations. Changes in membrane time constants affect membrane-potential decay and temporal integration, while task rewards provide learning signals that guide structural–dynamical co-adaptation to task demands.

Reward-modulated learning. The reward-related neuromodulator dopamine can regulate activity-dependent synaptic plasticity [1]. Inspired by this principle, DCFO uses task rewards to evaluate perturbation-induced changes and guide the adaptation of connectivity structure and membrane time constants, without explicitly propagating error signals.

Population coordination and directional credit assignment. Neural population reinforcement experiments show that reward feedback can select and shape coordinated neural activity patterns [2]. DCFO represents coordinated parameter variations as candidate directions. JVPs evaluate how strongly these patterns are supported by reward-conditioned evidence, while ridge reconstruction accounts for their correlations to suppress redundant contributions and guide selective, coordinated parameter adaptation. This reflects the biological learning principle of reward-guided population coordination.

Q6: Why is the title termed ‘structure dynamics’?

We thank the reviewer for this valuable suggestion. Our original motivation was to incorporate learnable intrinsic neuronal dynamics, particularly membrane time constants, into connectivity optimization. We therefore used “Structure Dynamics” to emphasize this extension. We agree that “Connectivity–Neuronal Dynamics Co-Adaptation” more accurately reflects our contribution.


References

[1] Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. C. R., Urakubo, H., Ishii, S., & Kasai, H. (2014). A critical time window for dopamine actions on the structural plasticity of dendritic spines. Science, 345(6204), 1616–1620. DOI: 10.1126/science.1255514

[2] Athalye, V. R., Santos, F. J., Carmena, J. M., & Costa, R. M. (2018). Evidence for a neural law of effect. Science, 359(6379), 1024–1029. DOI: 10.1126/science.aao6058

[3] Frangella, Z., Tropp, J. A., & Udell, M. (2023). Randomized Nyström Preconditioning. SIAM Journal on Matrix Analysis and Applications, 44(2), 718–752.

Thank you for the detailed response and the additional experiments. The SHD and MNIST results and the PPO reference results partially address my questions on generality and on contextualizing backpropagation-free performance. However, the rebuttal has weakened rather than strengthened my original assessment.

  • Performance. My positive assessment relied on the claim that joint optimization improves task performance. In the new five-seed results, the gap between full DCFO and structure-only DCFO is within one standard deviation on Walker2d and Hopper, and as Reviewer g1Gu notes, it is not statistically significant even on Humanoid (Welch’s p ≈ 0.10). The central claim of structural–dynamical co-adaptation is therefore not yet supported.
  • Efficiency. DCFO reaches the target in fewer generations, but it is slower than EC in wall-clock time on Humanoid and Walker2d (Table 2), while the rebuttal argues for efficiency only in terms of generation counts. The PPO comparison also does not report matched sample budgets, although DCFO evaluates 10,240 episodes per generation.
  • Neuromorphic and biological motivation. The response to my scalability question proposes running ridge reconstruction on an external CPU/GPU, which implies that the update step is not performed natively on neuromorphic hardware, while the conclusion still claims suitability for neuromorphic devices. On biological plausibility, the response relies on analogies to reward-modulated plasticity and does not address the ridge solve and JVP reconstruction specifically. Moreover, Appendix B shows that the JVPs are directional derivatives of a constructed quadratic surrogate rather than forward-mode derivatives through the network.

The current manuscript also does not include the new results. Given these points, my original assessment overstated the empirical support for the method, and I am lowering my score to 2.

Reviewer_g1GuOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Directional Credit guided Forward Optimization (DCFO), a backpropagation-free optimization framework for recurrent spiking neural networks (RSNNs). The idea is to construct candidate directions separately in the connectivity and membrane-time-constant parameter spaces and assign reward-based directional credit to each candidate direction. The paper proves that the reconstructed direction is a local descent direction of the corresponding credit-evaluation function. The paper evaluates Humanoid, Walker2d, and Hopper in Brax against Evolving Connectivity (EC), structure-only DCFO, and ES-trained RSNN, GRU, and LSTM baselines.

Critical strengths

(+) The paper presents an interesting structural dynamical optimization problem. The problem is well motivated. (+) The ablation separates the core components.

Critical weaknesses

(-) The paper combines established techniques such as Bernoulli connectivity optimization, learnable membrane time constants, antithetic perturbations/evolutionary optimization, and directional/forward-gradient optimization. The paper combines these through candidate-direction credit assignment and ridge reconstruction. Therefore, the novelty is somewhat incremental. (-) The forward-gradient and JVP framing confusing. The credit-evaluation function is not defined. It seems that the directional credits are computed directly as reward-weighted inner products between the sampled parameter perturbations and the candidate directions (Eq. 10 and 11). There is no forward-mode differentiation through the network. It is therefore unclear what distinguishes DCFO from evolution strategies with a subspace projection. (-) Theoretical analysis is weak. For instance, Proposition 1 characterizes the algebraic reconstruction step. It holds for any positive-definite ridge system, and its proof assumes that the credits equal the directional derivatives of the credit-evaluation function, which is not established for the credits actually used in Eq. 10 and 11. It does not establish that DCFO is an unbiased/consistent estimator of the gradient of the underlying reward objective (Eq. 5) or that the algorithm converges. (-)The construction of the structural candidate directions is described in only one sentence in Sec. 5.2. Sec. 4.2 says it is described in Sec. 5. The candidate directions for the membrane time constants and their number are not specified. The reward-derived structural weights are not defined. This makes the method difficult to reproduce. (-) The initialization for joint optimization is inconsistent. Table 3 states the initial membrane time constant is 10 ms. Sec. 5.2 state that joint optimization starts from a uniform heterogeneous distribution. If full DCFO starts heterogeneous while EC and structure-only DCFO use a fixed membrane time constant of 10 ms, the main comparison in Table 1 mixes the learning the time constants with heterogeneous initialization. Please clarify. (-) Experimental scope is narrow. All primary experiments consist of only three Brax locomotion environments: Hopper, Walker2d, and Humanoid. This makes it difficult to determine whether the approach generally benefits temporal processing in RSNNs or is primarily effective for this particular class of continuous-control problems. (-) The benefit of joint optimization rests mainly on Humanoid. On Walker2d (6,244 vs 6,024) and Hopper (2,726 vs 2,696), DCFO and structure-only DCFO differ by less than one standard deviation with only three seeds. In Table 2, structure-only DCFO reaches the target in fewer generations than full DCFO on Humanoid (346 vs 390) and Hopper (260 vs 290). (-) Some results are confusing. The paper claims that DCFO reaches target rewards in fewer generations than EC. For Humanoid, DCFO requires 390 generations versus 725 for EC. However, the wall-clock times are 367.18 versus 352.45 minutes (Table 2). Walker2d shows the same pattern (324.32 vs 297.52 minutes), while Hopper is the only task where DCFO is faster. Per generation, DCFO is about twice as expensive as EC on Humanoid. So it means that fewer generations do not imply lower training cost. It is also unclear whether the 512 paired evaluations for the membrane time constants are drawn from or added to the population of 10,240, so the total number of environment steps per method should be reported. (-) Baseline comparison is limited. ES-LSTM outperforms DCFO on Humanoid in both final return and time-to-target (194 generations and 83.85 minutes vs 390 generations and 367.18 minutes), so the claim of being competitive with deep recurrent networks should be qualified. ES-GRU (hidden size 256) and ES-LSTM (hidden size 128) are not parameter-matched, and it is unclear whether their hyperparameters were tuned. No surrogate-gradient/BPTT-trained RSNN or standard RL baseline (e.g., PPO) is included to quantify the cost of being backpropagation-free. An EC baseline combined with ES-based time-constant learning would also help isolate the contribution of DCFO for the membrane time constants. (-) Scalability and sensitivity are not analyzed. Fig. 5 shows performance still improving at 4096 structural candidate directions, the largest value tested. Each generation requires solving a ridge system of that size, but this cost and its scaling with network size are not reported. No sensitivity analysis is given for the ridge coefficients, the perturbation magnitude, or the learning rate for the membrane time constants. (-) The neuromorphic hardware motivation is weak. The paper states that the backpropagation-free nature makes DCFO suitable for training RSNNs on neuromorphic devices. However, all experiments are JAX implementations running on an NVIDIA RTX 4090 GPU, and no neuromorphic implementation, energy analysis, memory analysis, or hardware mapping is provided. Moreover, the algorithm requires 10,240 parallel network instantiations, a global reward broadcast, and a centralized dense linear solve. These are neither local nor obviously hardware-friendly, and they also weaken the claim that the method is biologically inspired. (-) The learned timescale is superficial. Figure 3(c) shows that the initially uniform membrane-time-constant distribution becomes nonuniform and differs between Humanoid and Walker2d. This demonstrates adaptation, but not that the learned timescales correspond to meaningful temporal structure in the tasks. For example, the Walker2d distribution concentrates at shorter timescales and Humanoid at longer ones, which is consistent with Fig. 3(a,b), but the paper does not make this connection. It does not analyze how far individual time constants move from initialization or whether they relate to the learned connectivity.

Questions for authors
  1. Why are only three seeds used? Can the authors provide results over more seeds and statistical significance/confidence intervals, particularly for DCFO versus structure-only DCFO?
  2. Given that DCFO uses fewer generations but is slower than EC in wall-clock time on Humanoid and Walker2d, what exactly is the intended efficiency advantage?
  3. What is the credit-evaluation function? How does DCFO differ from evolution strategies with the gradient estimate projected onto the candidate subspace?
3 comments

We thank the reviewer for the detailed and constructive feedback. We have strengthened the theoretical analysis and clarified the candidate construction. We further emphasize that DCFO's central contribution is reward-guided co-adaptation of discrete connectivity and continuous neuronal dynamics without backpropagation through time.


Clarifications on Credit Evaluation and Candidate Construction

Theoretical analysis. We have added credit-evaluation functions for connectivity-structure and membrane-time-constant optimization in Appendix B, together with a theoretical analysis establishing their relationship to the original expected reward objective.

Candidate construction. Structural candidates comprise 50% variations with the largest absolute reward-derived weights (including high- and low-reward samples), 25% high–low structural differences, and 25% exploratory and historical directions. Structural weights use centered ranks.


Generalization Beyond Continuous Control.

We additionally evaluated DCFO on MNIST and SHD using classification-derived reward feedback, achieving a mean MNIST validation accuracy of 98.03% across three evaluations and 84.7% accuracy on the full official SHD test set.

DatasetTaskAccuracy
MNISTVisual classification98.03%
SHDTemporal classification84.7%

As the RSNN used in DCFO does not incorporate additional architectural enhancements, it achieves a relatively modest accuracy of 84.7% on SHD.


Q1: Statistical Reliability and Baseline Comparisons

We extended the evaluation of DCFO Joint and Structure-only DCFO to five independent random seeds. The results (mean ± standard deviation) are reported below:

MethodHumanoidWalker2dHopper
DCFO Our12,903.88 (±736.04)6,181.04 (±264.33)2,627.27 (±276.48)
DCFO Structure Only12,003.96 (±805.31)5,979.16 (±495.13)2,587.89 (±204.72)

Q2: Generation-Level Convergence Efficiency

DCFO's primary advantage lies in generation-level convergence efficiency, rather than per-generation computational speed.As shown in Table 2, DCFO reaches the target rewards in substantially fewer generations than EC-RSNN:

MethodHumanoidWalker2dHopper
EC-RSNN725752837
DCFO (Ours)390534290

This represents 46%, 29%, and 65% fewer generations, respectively.Under the same population size, DCFO reaches the target rewards with substantially fewer generations, indicating more effective utilization of reward feedback from sampled connectivity structures. This improves generation-level optimization efficiency and partially alleviates the dependence on repeated large-population sampling.


Q3: Difference from Projected ES

Difference from Projected ES. Our goal is to develop a biologically inspired RSNN that jointly adapts connectivity structure and intrinsic neuronal dynamics through reward feedback without BPTT. DCFO's key contribution lies in reward-guided candidate subspace construction and block-specific credit assignment for structural–dynamical co-adaptation.

First, DCFO constructs structural directions from high-absolute-weight connectivity variations, high–low reward differences, and exploratory directions, guiding updates toward reward-informed structural changes.

Second, DCFO evaluates antithetic membrane-time-constant perturbations under identical sampled connectivity, isolating dynamical reward differences from structural variations and enabling structure-conditioned credit assignment.

Third, correlation-aware ridge reconstruction integrates directional credits into regularized updates, mitigating redundancy and sensitivity to noise.

With the same connectivity population size, DCFO reaches target rewards in 46%, 29%, and 65% fewer generations than EC-RSNN, demonstrating more effective generation-level utilization of sampled reward feedback.

Thank you for the response and the additional experiments. The description of the structural candidate construction and the new results on MNIST and SHD are helpful. However, several of my main concerns remain.

The five-seed results weaken the claim that joint optimization improves over structure-only DCFO. The mean difference is about 900 on Humanoid, about 200 on Walker2d, and about 40 on Hopper, while the standard deviations are of the same order or larger. A Welch t-test on the reported values gives a p-value of about 0.10 on Humanoid, and the differences on Walker2d and Hopper are far from significant. Please report confidence intervals or significance tests.

The response to the efficiency question restates the generation counts but does not address wall-clock time. DCFO is slower than EC on Humanoid and Walker2d and costs about twice as much per generation. It is also still unclear whether the 512 paired evaluations for the membrane time constants are added to the population. Please report the total number of environment steps for each method. Without this, fewer generations do not show better use of reward feedback.

The response to the question on projected evolution strategies describes the components but does not provide evidence. Evaluating antithetic pairs under the same connectivity is a standard variance-reduction technique. An ablation comparing reward-guided candidates with random candidates, and ridge reconstruction with direct use of the evolution strategies estimate, under the same population, would show what DCFO adds.

The MNIST and SHD results are reported without baselines, so they do not show an advantage of DCFO. Please include EC, evolution strategies, and a BPTT-trained RSNN on the same tasks, and report test accuracy for MNIST.

The inconsistency in the membrane-time-constant initialization between Table 3 and Sec. 5.2 is not addressed. The same holds for baseline fairness, scalability, sensitivity, the neuromorphic motivation, and the analysis of the learned timescales.

For these reasons, I keep my score.

We thank the reviewer for identifying this ambiguity. In the main experiments (Table 1 and Fig. 2), DCFO, DCFO (Structure Only), and EC-RSNN all use homogeneous initialization at 10 ms, ensuring a consistent comparison. We have also revised Table 3 in the manuscript to explicitly distinguish the initialization settings of the main experiments and the uniform-initialization ablation. The uniform initialization in Section 5.2 refers to a separate ablation (Fig. 3(c) and Appendix E), where adaptive DCFO and fixed heterogeneity share identical initial membrane time constants sampled from (U([1,20]ms)). Only adaptive DCFO updates these parameters during training. Fig. 3(c) further reveals task-dependent adaptation: Humanoid tends toward larger membrane time constants, whereas Walker2d favors smaller ones. This is consistent with Figs. 3(a,b), where larger time constants improve final rewards on Humanoid, while smaller ones accelerate early learning on Walker2d. We have revised Table 3 and clarified the distinction between the two experimental settings.

Classification Baselines and Test Accuracy. We thank the reviewer for this suggestion. We clarify that the reported MNIST accuracy of 98.03% was evaluated on the test set, not the validation set,We further added comparisons with EC and two surrogate-gradient BPTT-based RSNN baselines on both MNIST and SHD:

DatasetDEEP R + SG-BPTTGrad R + SG-BPTTECDCFO (Ours)
MNIST98.49 ± 0.0698.48 ± 0.0398.18 ± 0.0598.03 ± 0.05
SHD71.13 ± 1.1879.89 ± 1.8682.08 ± 1.3084.70 ± 1.05
Reviewer_Ja3VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Directional Credit guided Forward Optimization (DCFO), a backpropagation-free learning framework that jointly optimizes connectivity structure and neuron-specific membrane time constants in recurrent spiking neural networks (RSNNs). DCFO constructs candidate directions in the structural and dynamical parameter spaces, assigns directional credit based on task rewards, and reconstructs parameter updates through regularized optimization. Experiments demonstrate improved performance over Evolving Connectivity (EC) and competitive results compared with conventional recurrent neural networks.

Critical strengths

S1. The paper addresses an interesting problem: jointly adapting connectivity structure and neuronal dynamics in RSNNs without relying on backpropagation. S2: DCFO provides a unified reward-driven optimization framework for discrete connectivity parameters and continuous membrane time constants. S3: The experiments demonstrate promising performance across continuous-control tasks.

Critical weaknesses

W1: The paper defines an expected reward maximization objective in Eq. (5), but Proposition 1 only establishes that the reconstructed direction is a local descent direction for an auxiliary credit-evaluation function. The specific definitions of (E_t^\rho) and (E_t^\tau) are not provided, and their relationship to the original reward objective remains unclear. Moreover, while reward-weighted structural perturbations can be related to variance-scaled reward gradients, the subsequent subspace projection does not necessarily preserve an ascent direction for the original objective. Therefore, the practical implications of the theoretical guarantee require further clarification.

W2: According to Eqs. (8) and (10), the reconstructed structural update can be interpreted as a regularized projection of reward-weighted perturbations onto the candidate subspace. This formulation is conceptually related to existing zeroth-order and subspace-guided optimization methods, such as Guided Evolutionary Strategies [1]. Although the paper analyses the effect of candidate-direction coverage, it does not directly isolate the contribution of the reconstruction mechanism from that of candidate selection.

W3: The construction of candidate directions is central to DCFO, but the paper provides only a high-level description of their sources, including high-reward structural variations, differences between high- and low-reward samples, and previous update directions. The precise procedures for generating, selecting, and combining these candidates are not clearly specified.

W4: Although joint optimization achieves higher mean final rewards than structure-only optimization across the three tasks, the improvements are relatively modest on Walker2d and Hopper. The fixed-heterogeneity and Tau-ES comparisons provide useful supporting evidence, but they do not fully distinguish the benefits of introducing additional trainable neuronal parameters from those arising specifically from structural–dynamical co-adaptation.

[1] Maheswaranathan, N., et al. "Guided Evolutionary Strategies: Augmenting Random Search with Surrogate Gradients." ICML, 2019.

Questions for authors

Q1. How are (E_t^\rho) and (E_t^\tau) defined, and how does Proposition 1 relate to optimization of the original reward objective? Q2. What advantages does directional credit reconstruction offer over directly using reward-weighted perturbation updates? Q3. How are candidate directions and reward-derived structural weights constructed in practice? Q4. Can the authors further clarify whether the performance gains arise from structural–dynamical co-adaptation rather than simply introducing additional trainable parameters?

1 comment

We thank the reviewer for the constructive feedback. We address the four questions below.

Q1: Credit-evaluation functions and their relationship to reward optimization

We thank the reviewer for pointing out these issues. The reward-conditioned credit-evaluation functions are now explicitly defined in Appendix B of the revised manuscript:

Structural credit-evaluation function:

Etρ(x)=12N∑i=1Nwi,tρ∥x−θi,t∥22\mathcal{E}_{t}^{\rho}(x)=\frac{1}{2N}\sum_{i=1}^{N}w_{i,t}^{\rho}\lVert x-\theta_{i,t}\rVert_2^2

Dynamical credit-evaluation function:

Etτ(x)=12M∑i=1MΔRi,tτ(∥x−τ~i,t+∥22−∥x−τ~i,t−∥22)\mathcal{E}_{t}^{\tau}(x)=\frac{1}{2M}\sum_{i=1}^{M}\Delta R_{i,t}^{\tau}\left(\lVert x-\tilde{\tau}_{i,t}^{+}\rVert_2^2-\lVert x-\tilde{\tau}_{i,t}^{-}\rVert_2^2\right)

Proposition 1 guarantees local descent of the auxiliary functions. Eq. (16) links expected credit gradients to reward gradients, accounting for structural residuals and dynamical approximation errors. Eqs. (17–19) establish sufficient conditions for local reward ascent along the reconstructed direction.


Q2: Advantages of directional credit reconstruction

Direct reward-weighted perturbation updates aggregate reward feedback from sampled network configurations into a single update estimate, which can be sensitive to sampling noise and coupled parameter variations. DCFO offers three advantages:

First, improved utilization of sampled evidence. DCFO evaluates task-related candidate directions using reward-conditioned directional credits and reconstructs updates within their candidate subspace, rather than directly aggregating perturbations. This improves generation-level optimization efficiency. As shown in Table 2, DCFO reaches the target rewards in fewer generations than EC across all three tasks.

MethodHumanoidWalker2dHopper
EC-RSNN725752837
DCFO (Ours)390534290

Second, coordinated adaptation of neuronal dynamics. In membrane-time-constant optimization, simultaneous perturbations of multiple parameters produce coupled reward feedback, making it difficult for direct reward-weighted perturbation updates to identify effective combinations of neuronal timescale changes. Directional credit reconstruction prioritizes reward-supported combinations of neuronal timescale changes, enabling more targeted and stable joint adaptation of neuron-specific membrane time constants.

Third, correlation-aware reconstruction. Unlike direct reward-weighted aggregation, DCFO explicitly accounts for correlations among candidate directions through the Gram matrix. Ridge regularization attenuates redundant and weakly supported directional components, potentially reducing noise amplification and improving update robustness.


Q3: Construction of candidate directions and reward-derived weights

DCFO constructs candidate directions by balancing reward-guided exploitation and exploratory diversity.

Candidate directions. 50% are selected from structural variations with the largest absolute reward-derived weights(ukρ=θi−ρtu_k^{\rho}=\theta_i-\rho_t), 25% from differences between high- and low-reward samples (ukρ=θiH−θjLu_k^{\rho}=\theta_i^H-\theta_j^L), and the remaining 25% from random exploration and historical update directions.

Structural reward weights. We use centered-rank weighting:

wiρ=s(ri−1N−1−12)w_i^{\rho}=s\left(\frac{r_i-1}{N-1}-\frac{1}{2}\right)

where rir_i is the reward rank, NN is the population size, and ss is the scaling factor. This reduces sensitivity to reward magnitude.

Dynamical reward weights. Membrane-time-constant perturbations are evaluated under the same connectivity using antithetic rewards:

ΔRiτ=Ri+−Ri−\Delta R_i^{\tau}=R_i^{+}-R_i^{-}

wiτ=sign⁡(ΔRiτ)rank⁡(∣ΔRiτ∣)Mw_i^{\tau}=\operatorname{sign}(\Delta R_i^{\tau})\frac{\operatorname{rank}(|\Delta R_i^{\tau}|)}{M}


Q4: Can the authors further clarify whether the performance gains arise from structural–dynamical co-adaptation rather than simply introducing additional trainable parameters?

We conducted three experiments to demonstrate the effectiveness of structural–dynamical co-adaptation.

First, jointly optimizing connectivity structure and membrane time constants achieves higher mean rewards than structure-only optimization.

Second, Appendix E shows that optimizing membrane time constants outperforms fixed uniform heterogeneity, indicating that the gains are not merely due to introducing neuronal heterogeneity.

Third, Appendix C shows that Tau-DCFO outperforms Tau-ES across all three tasks under the same joint optimization setting, suggesting that more effective coordination between connectivity structure and neuronal dynamics benefits network decision-making. Together, these results support the effectiveness of structural–dynamical co-adaptation.

Reviewer_RQqFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper the authors propose Directional Credit guided Forward Optimization (DCFO), a biologically inspired, backpropagation-free learning framework that jointly adapts connectivity structure and neuron-specific membrane time constants. In both synaptic parameter space and the neuron-specific membrane-time-constant space, DCFO provides directional credits which enables forward-gradient updates. The proposed approach performs better than compared baselines on three Brax continuous-control tasks.

Critical strengths

The paper explores a highly relevant research domain of developing backpropagation free learning framework for spiking neural networks. Furthermore the co-adaptation of synaptic parameters and neuronal dynamics is also a key area of exploration in the domain of neuromorphic computing.

Critical weaknesses
  1. The work claims to be more biologically plausible, but I want to understand how exactly the claim stands. How bio-logically plausible is the Vector Jacobian product used in the algorithm? While the work does not use backprop, but its certainly not like STDP-like bio-plausbible learning algorithms.

  2. Equilibrium propagation [1] is also a key backprop free learning algorithm. The authors should explore EP and other bio-plausible learning frameworks as baselines for comparison.

  3. Some work [2] have tried to implement backprop based algorithms on-chip. The work should compare to those approaches if they are proposing a solution that is focused for on-chip deployment. Furthermore, in that case results from actual on-chip deployed models will be more convincing.

References: [1] Scellier, Benjamin, and Yoshua Bengio. "Equilibrium propagation: Bridging the gap between energy-based models and backpropagation." Frontiers in computational neuroscience 11 (2017): 24.

[2]Béna, Gabriel, Timo Wunderlich, Mahmoud Akl, Bernhard Vogginger, Christian Mayr, and Hector A. Gonzalez. "Event-based backpropagation on the neuromorphic platform SpiNNaker2." In 2025 Neuro Inspired Computational Elements (NICE), pp. 1-10. IEEE, 2025.

Questions for authors
  1. What is the motivation of using control problems? Why not explore conventional baselines such vision, language tasks, etc.
  2. How does the paper compare to the methods described in the Weaknesses.
  3. How scalable is the approach? Can it be applied to deeper architectures, convolutional models, etc.
1 comment

We thank the reviewer for the constructive feedback and for recognizing the relevance of backpropagation-free learning and structural–dynamical co-adaptation. We address the key concerns below.


W1: Biological plausibility and the role of JVP

Perturbation-driven exploration. DCFO explores the joint adaptation of connectivity structure and neuron-specific membrane time constants through parameter perturbations. Bernoulli connectivity sampling and antithetic perturbations of membrane time constants under the same sampled connectivity enable exploration of structural and neuronal dynamical variations. Changes in membrane time constants affect membrane-potential decay and temporal integration, while task rewards provide learning signals that guide structural–dynamical co-adaptation to task demands.

Reward-modulated learning. The reward-related neuromodulator dopamine can regulate activity-dependent synaptic plasticity [1]. Inspired by this principle, DCFO uses task rewards to evaluate perturbation-induced changes and guide the adaptation of connectivity structure and membrane time constants, without explicitly propagating error signals.

Population coordination and directional credit assignment. Neural population reinforcement experiments show that reward feedback can select and shape coordinated neural activity patterns [2]. DCFO represents coordinated parameter variations as candidate directions. JVPs evaluate how strongly these patterns are supported by reward-conditioned evidence, while ridge reconstruction accounts for their correlations to suppress redundant contributions and guide selective, coordinated parameter adaptation. This reflects the biological learning principle of reward-guided population coordination.\


W2/Q2: Comparison with equilibrium propagation

We evaluated EP-PPO (EP Actor + BP Critic) using the same network architecture. DCFO achieves substantially better performance:

MethodHumanoidWalker2dHopper
DCFO (Ours)13,104.31 ± 680.566,244.51 ± 149.322,725.81 ± 208.47
DCFO (Structure Only)11,865.84 ± 533.926,024.31 ± 313.792,695.70 ± 92.14
EP-PPO7,226.76 ± 302.181,978.04 ± 578.491,949.52 ± 100.60

EP's sensitivity to equilibrium convergence and nudging strength may contribute to the slow or unstable gaits observed.


Q1: Motivation for continuous-control tasks

Biological neural systems adapt through perception, action, and environmental feedback. Continuous control therefore provides a suitable setting for evaluating task-driven co-adaptation of connectivity structure and intrinsic neuronal dynamics, which remains underexplored in RSNNs.

We additionally evaluated DCFO on MNIST (98.03%) and SHD (84.7%), demonstrating applicability to visual and temporal classification tasks.

MethodMNIST (%)SHD (%)
DEEP R + SG-BPTT98.4971.13
Grad R + SG-BPTT98.4879.89
EC98.1882.08
DCFO (ours)98.0384.70

As the RSNN used in DCFO does not incorporate additional architectural enhancements, it achieves a relatively modest accuracy of 84.7% on SHD.


Q3: Scalability to deeper and convolutional architectures

DCFO can, in principle, extend to deeper and convolutional architectures. Inspired by scalable forward-gradient learning with local losses [3], layer-wise candidate directions and block-wise credit reconstruction could reduce high-dimensional reconstruction costs while retaining shared task-level reward feedback. For convolutional models, spatially shared Bernoulli connectivity masks could be defined at the kernel or channel level.


Reference

[1] Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. C. R., Urakubo, H., Ishii, S., & Kasai, H. (2014). **A critical time window for dopamine actions on the structural plasticity of dendritic spines.** *Science*, 345(6204), 1616–1620. [DOI: 10.1126/science.1255514](https://doi.org/10.1126/science.1255514)

[2] Athalye, V. R., Santos, F. J., Carmena, J. M., & Costa, R. M. (2018). **Evidence for a neural law of effect.** *Science*, 359(6379), 1024–1029. [DOI: 10.1126/science.aao6058](https://doi.org/10.1126/science.aao6058)

[3] Ren, M., Kornblith, S., Liao, R., & Hinton, G. (2023). Scaling Forward Gradient With Local Losses. ICLR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.