acceptodds
Under review as a conference paper at ICLR 2027

Alignment Is Not Learning: Loop-Gain Control in Heterosynaptic Plastic Circuits

Abstract

Heterosynaptic plasticity (HSP) rules combine local activity-dependent potentiation with compensatory decay, providing a biologically and physically plausible substrate for learning. However, in a learning network the local plasticity rule is embedded in a closed loop: perturbations of plastic weights alter neural signals, which in turn alter the plasticity drive. This suggests that stability of networks built with these rules is not guaranteed. Linearizing an HSP motif around a balanced operating point yields a loop-gain condition: stability depends on the closed-loop sensitivity from weights back to their local learning signals. We apply this analysis to continuous-time Kolen–Pollack learning, a well-known HSP-style mechanism for aligning feedback weights with forward weights. Our analysis shows that while KP's symmetric updates exponentially contract the forward–feedback weight mismatch, they critically leave a weight scale mode governed by the generic HSP loop-gain spectrum. Thus feedback alignment is not sufficient for stability. Finally, we show that bounded activation functions limit the effective plastic loop gain and stabilize continuous-time KP learning in deep multilayer perceptrons, outperforming the alternative local mechanisms we test at every depth and enabling training at depths where they fail. Across mechanisms, stability follows loop-gain control: a circuit whose loop gain is left uncontrolled fails, by fast divergence or by unbounded weight drift, even when its feedback weights align and its update magnitudes are bounded. These results identify loop-gain control as a central requirement for stable heterosynaptic learning in biological and continuous-time substrates.

Primary area:
applications to neuroscience & cognitive science
Keywords:
biologically plausible learning, feedback alignment, local learning, stability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dFbUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The goal is to study the learning dynamics of heterosynaptic plasticity, specifically in terms of the stability properties of the associated dynamics. Test case is a form of feedback alignment, with strong emphasis on spatial and temporal locality of synaptic operations as important for the solution, motivated by biological plausibility. The key mathematical insight is the fact that synaptic changes have downstream consequences onto neural activity which, due to the nature of the feedback loops considered in their example, comes back to affect local neural responses and associated further plasticity. Saturating nonlinearities in the neural responses help keep this instability inducing circuit loop in check.

Critical strengths

Analysis of the modes evolution is pretty clear in mathematical terms and somewhat akin to weight stability analyses in other systems, like Oja (but see point 1 in weaknesses)

Figures are clear

The explanation of the instability coming from a degeneracy of scale is intuitive and helpful

The explanations of separation of timescales for the continuous KP was very clear

Critical weaknesses

Although the math is generally clear and relatively easy to follow, the mapping of the operation into a circuit level implementation is not always apparent, for instance it is not clear why it is sensible to assume da ~= A dW etc. it is hard to judge the generality of the results if the dependence on the architecture is obscured in this way. The core assumptions of the perturbative approach need to be spelled out more explicitly

Writing clarity and self-containment: a precise definition of stability materializes very late in the process, the explanation on the properties of KP in the introduction come with too little context to be easily followable on their own. Several papers that this paper builds upon need to be read first for a lot of the details to become understandable.

Although the big picture motivations emphasize the biological plausibility, etc the numerics focus largely on traditional ML considerations. Yet, from a ML algorithmic improvement perspective it is not clear how helpful the insight is in practice since alternative nonbiological approaches for enforcing stability exist already.

The solution being a clipped version of a ReLu instead of using a saturating nonlinearity in the first place seems like a way of creating problems to then fix.

Narrow view of the literature from the perspective of very closely related works like Ziyin 2025, Bancvanski 2025 Limited or incremental new knowledge relative to https://openreview.net/forum?id=kmJuMuHe4j

Questions for authors

Can you please explain the exact neural implementation of the 'local signals' a and b in Eq 1 in an example architecture from the beginning? It would really help with understanding the causal path for perturbation in weights to manifest at the level of a,b By the time the concrete KP architecture is described the main causal paths are lost among the details

1 comment

Thank you for a careful review. The revision addresses each of your points.

CW1, Q1. "it is not clear why it is sensible to assume da ~= A dW etc." … "Can you please explain the exact neural implementation of the 'local signals' a and b in Eq 1 in an example architecture from the beginning?"

Thank you; the revision opens Section 3 with exactly this. Eq. 1 is the general motif: a synapse changes with the product of two signals present at it, a and b, minus a decay. One can think of them as a forward activity and a backward activity, an error. The motif does not say how the error reaches the synapse; that is set by the topology of the circuit. KP is one example, with a highly structured backward pathway; there are many others, including randomly wired circuits that still learn (Ziyin et al. 2025). In KP, the forward synapse from unit j of layer l to unit i of layer l+1 pairs b = y_{l,j}, the presynaptic activity, with a = z_{l+1,i}, the error carried by the postsynaptic cell (Eq. 10); for the reciprocal feedback synapse the roles swap.

The causal path you ask about: a change in a forward weight changes the activities of the later layers and so the output error, which travels back through the feedback weights and changes the errors that reach this synapse and all others. δa = A[δW] is the first-order part of this change: A is the derivative of the map from weights to settled signals, and B likewise. So the architecture enters only through A and B, which must cover every weight on the loop. The approximation rests on two assumptions, which the revision states at the start of Section 3: the signals have settled, and this map is differentiable. Your question points at the assumption that matters most: with settled signals none of our networks diverged, so the divergence needs signals that take time to travel (Section 5.2, Appendix G.1).

CW2. "a precise definition of stability materializes very late in the process"

We agree. The revision defines stability at the start of Section 3, gives the introduction's account of KP more context (Section 1), writes out the KP rule and its continuous-time form (Section 4), and adds a worked example on a small circuit (Section 3, Appendix E), so that the paper can be read on its own.

CW3. "the numerics focus largely on traditional ML considerations" … "alternative nonbiological approaches for enforcing stability exist already."

These approaches work well in machine learning, and we make no claim to improve on them; the revision says so in the introduction (Section 1). Our question is what a circuit can do without them: no batch statistics, no optimiser state, and signals that take time to travel, as in the brain or in analog hardware. The new experiments therefore vary circuit quantities: signal speed and the plasticity and decay time constants (Appendices G.1, G.3). These experiments also show why the setting matters: the failure is absent when signals settle instantly, as discrete-time training assumes, and appears as soon as the signals take time to travel.

CW4. "The solution being a clipped version of a ReLu instead of using a saturating nonlinearity in the first place seems like a way of creating problems to then fix."

This is a fair point. Our remedy is to bound the units, and a saturating nonlinearity used from the start is exactly such a bound; the revision makes this clear (Section 5.3). We clipped the ReLU-type units of the continuous-time KP model we build on, so that the bounded network differs from the original in one thing only. Following your comment we tested saturating units: tanh, hard-tanh and the logistic unit all prevent the divergence, and tanh and hard-tanh train better at depth 10 than the clipped unit (MNIST accuracy 0.85 and 0.87, against 0.34). The revision adds these results (Appendix G.4, Table 2).

CW5. "Narrow view of the literature" … "Limited or incremental new knowledge relative to https://openreview.net/forum?id=kmJuMuHe4j"

As you suggest, the revision broadens the related work (Section 2) to network-level analyses of plasticity and stability (Zenke et al. 2013; Toyoizumi et al. 2014; Kozachkov et al. 2020), the analysis of weight mirrors (Kunin et al. 2020) and earlier uses of bounded units (Nøkland 2016; Akrout et al. 2019; Galtier et al. 2012). It also states more clearly what is new. Ziyin et al. (2025) assume that the weights reach a stationary state and state that "a simple decay term is sufficient to ensure stability"; we show that in a circuit the loop gain decides stability, not the decay alone. Bacvanski et al. (2025) study the timing constraints of the continuous-time model but do not analyse its stability. Relative to the linked workshop page, the experiments and analysis of this response are new.

Reviewer_dXvgOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies heterosynaptic plasticity as a closed-loop learning process in which changing weights reciprocally affects subsequent weight updates.

With local linearization, the author derived a stability condition and applied it to continuous-time Kolen–Pollack learning. The analysis distinguished the forward weight and feedback weight alignment condition from weight magnitude overgrowth conditions. And experiments demonstrated in KP, weight growth despite alignment, especially in deep network structure, which will deteriorate inference performance.

The author further established a bounded activation approach to suppress runaway behavior and outperform the tested normalization and update-clipping configurations in accuracy and weight magnitude stability.

Critical strengths
  1. The distinction between feedback alignment and stable learning is clearly presented. The mathematical decomposition provides an interpretable framework for understanding the close-loop dynamics. Several deep network failure modes are investigated in the Appendix as well. The math is quite solid and candid.

  2. The proposed approach is simple and clean and results show enhanced task performance as well as steady weight control. The authors also clarified their limitations faithfully.

Critical weaknesses

The theoretically derived condition and empirically probed result remains mismatched. Equation (17) concerns local stability near an equilibrium of the averaged, aligned system. However, stabilized runs have measured abscissae around 0.10.1–11, above the stated threshold of 0.010.01. The author did take the bounded activation effect into account, but still didn't explain why there's a mismatch. The experiments therefore demonstrate controlled versus runaway growth, but do not directly validate the derived threshold as the explanation for successful training. Although acknowledged in the discussion, this gap limits the stronger mechanistic claims.

Limited contribution. As mentioned above, a gap remains between this condition and the observed training behavior. Moreover, the analysis offers limited guidance for controlling the dynamics throughout learning. Its practical applicability therefore depends on how closely the evolving system satisfies the assumptions underlying the equilibrium analysis.

Questions for authors

I look forward to more detailed evidence that could explain the empirical divergence from theory, or to a relaxed condition with a clearer practical interpretation.

Figure 4 is quite interesting, though. The second column shows accuracy, and we can see that the bounded activation method learns the fastest. Meanwhile, the third column shows that alignment between {W} and {V} is somewhat delayed, which means that the weight updates are not accurate in the early stages. So perhaps the mean cosine similarity does not capture the whole picture of the error propagation and update process. I mean that maybe a proportion of feedback weights might contribute more to loss reduction, but this is not obvious when averaged? Or does the bounded activation approach also elicit "right direction but wrong size" gradient update?

2 comments

Thank you for the review and the careful questions. The revision answers them in Section 5 and Appendix G. (response part 1)

CW1. "The theoretically derived condition and empirically probed result remains mismatched." … "do not directly validate the derived threshold as the explanation for successful training."

You are right that the threshold does not predict quantitatively which runs train, and the revision says what (17) does show. (17) states the principle when the weights are aligned. KP's alignment removes the difference between forward and feedback weights but does not control their overall size, which is left to the loop gain: when the loop gain exceeds the decay rate, a small change in the weights can grow instead of dying out. So alignment does not by itself make learning stable.

(17) is exact for a network resting at aligned weights, and training is not at that point. First, most learning happens before the feedback has aligned: KP shrinks the mismatch only at the decay rate, and when accuracy reaches 90% of its final value the mean forward–feedback cosine is still 0.36–0.41 on digits (0.21–0.26 on MNIST; see Q2). Second, the weights are still moving, while (17) describes small changes around a resting point. Third, (17) is linear: a small change that starts to grow need not run away, and in the stabilised runs the bound on the units stops it. So we read (17) as the principle rather than a sharp boundary. Networks that train sit one to two orders of magnitude above the threshold, while in runs that diverge the abscissa escapes by many orders, and bounded units, which hold it down, train (Fig. 3). The revision states this in Section 5.1.

CW2. "the analysis offers limited guidance for controlling the dynamics throughout learning."

We agree that the paper gave little such guidance. The practical guidance is to bound the loop gain, and bounding the activations is the best way to do it among the local mechanisms we compare (Section 5.4). The revision adds a short argument (Appendix E) that this works throughout learning: with bounded activations every signal and weight stays bounded at all times, for any depth, feedback mismatch and signal delay (the argument of Galtier et al. 2012, run down the feedback pathway; the bound is loose).

(response part 2)

Q1. "I look forward to more detailed evidence that could explain the empirical divergence from theory, or to a relaxed condition with a clearer practical interpretation."

Thank you; the revision adds this evidence to Section 5.2, with the details in Appendix G. The theory assumes settled signals: every activity and error at its steady value for the current input and weights. The continuous-time model departs from this, because after each input change the signals take about two time constants per layer to settle while plasticity keeps acting. Section 5.2 argued that depth and continuous time amplify the loop; the new runs test this directly (digits unless stated, 20 seeds per condition):

  1. With settled signals no network with unbounded units diverges: 0 of 360 runs, at depths 3 to 10 on digits and 5, 8 and 10 on MNIST, with random or matched feedback. From the same initial weights with the paper's 1 ms signals, 96 of the 120 digits runs with random feedback diverge, and all 60 MNIST runs (Appendix G.1).
  2. At depth 8, 10 of 20 runs diverge with a signal time constant of 0.1 ms and all 20 from 0.2 ms to 2 ms (Fig. 8). A finer integration step makes them diverge sooner, so this is not a numerical artefact (Appendix F).
  3. Feedback matched to the forward weights from the start does not prevent it: all 20 depth-8 MNIST runs diverge (Appendix G.2).

So the divergence needs signals that take time to travel, and the practical interpretation is the one in CW2: bound the loop gain, best by bounding the activations, which keeps every signal and weight bounded whatever the lag or mismatch.

Q2. "perhaps the mean cosine similarity does not capture the whole picture" … "maybe a proportion of feedback weights might contribute more to loss reduction, but this is not obvious when averaged? Or does the bounded activation approach also elicit "right direction but wrong size" gradient update?"

Learning does not wait for alignment: KP shrinks the mismatch at the decay rate (Eq. 13), so the mean cosine between forward and feedback weights reaches 0.98 only after about three decay times, while accuracy rises much earlier: when it first reaches 90% of its final value, the mean cosine is only 0.36–0.41 on digits (0.21–0.26 on MNIST at depths 5 and 8). The update already points downhill: its cosine with the loss gradient is then 0.68–0.87 (0.60–0.78 on MNIST), and an update within 90° of the gradient is enough, as in feedback alignment (Lillicrap et al. 2016). As you suggest, the mean hides differences between layers: the pair next to the output aligns first and the pairs near the input last (about 0.75–0.90 against 0.2 at that moment, on digits). On size: bounded units make the error reaching the first layer 1.4–1.6 times smaller and keep its direction (cosine 0.82–0.87), at little cost in accuracy (Appendix G.5; reply to Reviewer ncm2, CW2). So the update has the right direction and a somewhat smaller size. The revision adds the cosine of each layer pair to Fig. 4 (fifth column) and gives these numbers in Section 5.1 and Appendix G.6.

Reviewer_ncm2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the stability of continuous plasticity dynamics that follow local Hebbian updates plus homeostatic weight decay (heterosynaptic plasticity - HSP abbreviated here). The authors make the point that alignment of feedback and forward weights is not a guarantee for stability of the learning dynamics, because the weight update equation might be too sensitive to perturbations to one of the local synaptic variables considered in the update equation.

The authors analyse the dynamics of continuous time Kollen-Polak (KP) equations, that is biologically grounded weight update equation that circumvents the weight transport problem. They seperate a common-weight mode and an alignment-error mode, where the former can be unstable while the latter has contractive dynamics.

They propose to use bounded activation functions as a local stabilisation mechanism and then compare them against root-mean-square normalisation (RMSNorm) and update clipping.

The authors perform experiments on MNIST and on a dataset with handwritten digits. Their results indicate that bounded activation functions improve learning stability and performance.

Critical strengths
  • The paper makes clear that alignment between forward and backward weights does not guarantee learning stability
Critical weaknesses
  • I see limited theoretical novelty in this contribution, but I leave it to the authors to make the novelty more explicit in the rebuttal (see also Q3)

  • I would like to see a more deeper explanation of the mechanism behind the boundness stabilisation, and how the saturation through the bounds does not create issues with suppressing the signals required for learning thereby preventing learning.

  • See also questions

Questions for authors
  1. Can you please explicitly write out the KP model in the main text or appendix? I had to google it to find out what exactly it is and it is supposed to be the central object you study.

  2. Can you explicitly state which structural properties of the heterosynaptic plasticity rule does the stability condition you employ exploit, because to me it seems that this is just generic Jacobian stability condition

  3. How does your KP results extend the results of Kunin et al. (2020) who identified the instability associated with the growth of the common symmetric weight mode?

  4. You mention in section 6 that the networks that train successfully violate the stability criterion you proposed. However you do not mention how reconcile this with your theory. Do you have any insight/intuition on why the networks train successfully in these cases?

  5. While in the theoretical part of the work you emphasises the ratio between potentiation and decay timescales, I didn't find anywhere in the manuscript systematic experiments that study this ratio and how changing these timescales might change the stability and the performance of the learning dynamics.

  6. Regarding the biological plausibility claim, although this does not pertain only to your work, is the symmetry between forward and feedback plasticity updates indeed biologically-grounded?

  7. Is it possible with your framework or with some extension to establish a quantitative predictions for the onset of instability as a function of network depth or plasticity timescales?

4 comments

Thank you for the detailed questions; the revision addresses each of them.

CW1. "I see limited theoretical novelty in this contribution, but I leave it to the authors to make the novelty more explicit in the rebuttal"

Thank you for the chance to make it explicit; the revision states these points in the introduction and Section 5. Plasticity rules of this kind are usually analysed one synapse or one neuron at a time, where the decay balances the Hebbian growth (Oja 1982; Ziyin et al. 2025). In a learning network, a change in one weight changes the signals that update the others, so stability is a property of the whole circuit. We show that:

  1. For any HSP circuit, stability is set by the loop gain: how strongly a change in the weights, passed through the rest of the circuit, changes their own updates (Section 3; its form is in Q2). The tool, linear stability, is standard, as you say. What it shows is that balancing growth and decay at each synapse does not make a circuit stable.
  2. In KP, the difference between the two weights of each reciprocal pair shrinks at the decay rate, whatever the weights (Eq. 13; in discrete time this goes back to Kolen & Pollack 1994 and Akrout et al. 2019). So any instability lies in the average of the two weights, which alignment does not control, and deep networks fail even when the feedback matches exactly from the start.
  3. In continuous-time KP the failure needs signals that take time to travel (new experiments; Section 5.2, Appendix G.1). With signals that settle instantly, as discrete-time training assumes, none of our networks diverged, even with plasticity running continuously alongside inference.
  4. Bounding each unit controls the loop gain locally. It keeps every signal and weight bounded at any depth (Section 5.3, Appendix E), and it trains best among the local stabilisers we tested.

CW2. "I would like to see a more deeper explanation of the mechanism behind the boundness stabilisation, and how the saturation through the bounds does not create issues with suppressing the signals required for learning"

Thank you; the revision makes this explanation explicit (Section 5.3) and adds the measurements below (Appendices G.4, G.5). What runs away in a blowup is the loop gain, and the loop gain grows with the size of the activities and errors that enter each update (Section 5.3). A bounded unit caps the activity it passes on, and a unit at its cap passes no error back, which damps the errors too. So the bound caps the loop gain: with bounded activations the measured abscissa stays at 0.1–1 for the whole run, while without them it runs away by orders of magnitude (Fig. 3b, c). This holds throughout learning: with bounded activations every signal and weight stays bounded, for any depth, feedback mismatch and signal delay (Appendix E). It is also why any bound works: every bounded unit we tried prevents the divergence (tanh, hard-tanh, the logistic unit, and the clipped leaky ReLU at caps from 0.25 to 64; none of 5,300 runs diverges).

And the bound does not prevent learning: saturation weakens the learning signal a little but does not block it (Appendix G.5, Table 3). At the paper's cap with 1 ms signals, 24–32% of the units in the last two hidden layers sit at the bound, and the error reaching the first layer is 1.4–1.6 times smaller than without saturation and keeps its direction (cosine 0.82–0.87). The cost in accuracy can only be measured where unbounded networks also train, that is with settled signals, where fewer units saturate (18–20%). There the cost is nothing at depth 8 and 1.4–3.5 points at depth 10, and learning takes 1.15–1.74 times as many samples; with a cap of 2 there is no cost. Over 100,000 samples saturation rises only from 26% to 31%, while accuracy keeps rising.

Q1. "Can you please explicitly write out the KP model in the main text or appendix?"

Yes, and thank you: the revision writes the model out in full (Section 4). In the model (Eqs. 8–10), activities relax towards y_l = φ(s_l) with s_l = W_l y_{l−1}, errors towards z_l = φ′(s_l) ⊙ (V_l z_{l+1}), with z_L = y* − softmax(s_L) at the output, and the weights follow Eq. 10. The revision adds the discrete rule it comes from (Section 4; Kolen & Pollack 1994; Akrout et al. 2019). In our notation, with the error sent backward through V_l instead of W_{l+1}ᵀ:

W_{l+1} ← W_{l+1} + η z_{l+1} y_lᵀ − λ W_{l+1}, V_l ← V_l + η y_l z_{l+1}ᵀ − λ V_l.

Both weights get the same update, transposed, so V_l − W_{l+1}ᵀ shrinks by the factor 1 − λ at every step, whatever the data, and V_l approaches W_{l+1}ᵀ. Eq. 10 is the same rule in continuous time, with η = dt/τ_pot and λ = dt/τ_dec.

Q2. "Can you explicitly state which structural properties of the heterosynaptic plasticity rule does the stability condition you employ exploit, because to me it seems that this is just generic Jacobian stability condition"

Thank you; the revision makes this explicit. The tool is standard linear stability; what the HSP rule supplies is the form of the Jacobian, and that form tells us how to fix the instability. Each update is the product of two local signals, a and b, so a change δW in the weights changes the update by T[δW] = A[δW] bᵀ + a B[δW]ᵀ, where A and B, set by the circuit's topology, say how the two signals respond to δW. The loop gain is built from the signals themselves and grows with their size, so bounding the signals bounds it (Section 5.3). Among the local ways to do this, bounded activations work best (Section 5.4).

Q3. "How does your KP results extend the results of Kunin et al. (2020) who identified the instability associated with the growth of the common symmetric weight mode?"

Thank you for the pointer; the revision discusses this link (Section 2). Kunin et al. found, for weight mirrors in a scalar, single-layer, linear model, a symmetric mode that grows when the decay is weaker than the input power. Our condition gives theirs when applied to both weights of the pair together: the loop gain is x², and the mode grows when x² exceeds the decay rate; the revision works this out on their example (Appendix E). They do not analyse the stability of KP; they report that it matches backpropagation on ResNet-18 and ResNet-50 but falls behind on deeper ResNets. We extend the analysis to matrices, depth and nonlinear units, with the loop closed through the backward error pathway, and measure the spectrum in deep networks. We also show that the divergence of continuous-time KP needs signals that take time to travel, and give a local stabiliser that keeps every signal and weight bounded.

Q4. "Do you have any insight/intuition on why the networks train successfully in these cases?"

Yes, and the revision adds it to Section 5.1 (see also our reply to Reviewer dXvg, CW1). (17) is exact for a network resting at aligned weights, and training is not at that point: most learning happens before the feedback has aligned (mean forward–feedback cosine 0.36–0.41 on digits when accuracy reaches 90% of its final value), the weights are still moving, and since (17) is linear, a small change that starts to grow need not run away. So (17) states the principle rather than a sharp boundary: networks that train sit one to two orders of magnitude above the threshold, while in runs that diverge the abscissa escapes by many orders (Fig. 3). In practice, whether a run diverges depends on whether the signals take time to travel and whether the units are bounded (reply to Reviewer dXvg, Q1).

Q5. "I didn't find anywhere in the manuscript systematic experiments that study this ratio and how changing these timescales might change the stability and the performance of the learning dynamics."

Thank you; we have now run them, and the revision adds them as a new figure (Fig. 9, Appendix G.3): for each ratio and depth, whether the runs learn, diverge or lose their weights to the decay. On digits we varied τ_pot/τ_dec from 0.001 to 0.3 at depths 3–10 (20 seeds), and on MNIST from 0.003 to 0.1 at depths 5 and 8 (10 seeds). Networks with unbounded units at depths 8 and 10 learn at no ratio: they diverge when the decay is weak (ratios up to 0.03), and their weights decay away when it is strong (0.1 and above). Bounded networks learn from 0.001 to 0.03 at depth 5 and up to 0.01 at depth 8, and unbounded networks with settled signals over the same or a wider range. The time to divergence follows τ_pot: making potentiation and decay ten times slower together delays it about tenfold, while making decay alone ten times slower changes it by less than a quarter. So no choice of the ratio lets deep unbounded networks learn.

Q6. "is the symmetry between forward and feedback plasticity updates indeed biologically-grounded?"

A fair question; the answer is partly. Reciprocal connections between cortical pyramidal cells are several times more common than chance, stronger than one-way connections and correlated in strength (Song et al. 2005; Perin et al. 2011; Cossell et al. 2015; Campagnola et al. 2022), which a symmetric rule with a common decay would produce. But between brain areas, feedback is reciprocal area to area rather than cell to cell (Lillicrap et al. 2020), the correlation is modest, and the exact symmetry of the update is an assumption of KP, which tolerates noisy updates (Kunin et al. 2020). Our results do not rest on KP being biological. KP is our case study; the loop-gain condition applies to any HSP circuit, symmetric or not. The brain has such circuits with a dedicated backward pathway: in the cerebellum, climbing fibres from the inferior olive carry an error signal to Purkinje cells in matched modules, and a parallel-fibre synapse weakens when its input coincides with that signal (Ito 2001; Apps & Garwicz 2005). The question also applies to analog hardware, where signals take time to travel and the symmetry can be built in.

Q7. "Is it possible with your framework or with some extension to establish a quantitative predictions for the onset of instability as a function of network depth or plasticity timescales?"

In part: we have measured relations, and the revision states them. The onset comes earlier at larger depth (Section 5.2, Fig. 3a). The time to divergence follows τ_pot (Appendix G.3): making potentiation and decay ten times slower together delays it about tenfold, and it depends little on τ_dec for ratios up to 0.01 (Q5). And since none of our networks diverged with settled signals (Appendix G.1), a prediction of the onset has to include the signal lag.

I would like to thank the authors for their detailed responses to my and to other reviewers' comments and concerns. The updated manuscript and their rebuttal resolve some of the issues I raised (the boundness issue and timescale and other parameters experiments). However, I am still not entirely convinvinced on whether indeed the proposed analysis provides sufficient explanation for the observed phenomenon. The new experiments (Fig. ) now that finite signal relaxation times play an important role in observing the divergence, yet both the proposed spectral criterion and the measured Jacobian rely on settled signals, or?

Moreover, the new theorem the authors provide guarantees boundedness under specified assumptions. Yet, as I understand, it does not directly establish local stability of the equilibrium.

I would therefore suggest the authors to clarify the distinction between these results, and to avoid the attribution of the observed instability too strongly to the reduced spectrum.

My main unresolved issue now is whether the proposed loop-gain analysis explains why the continuous-time learning system fails and whether it indeed predict when it will fail.

I will go through the points I raised in detail:

  • CW1: I still think that the explanatory role of the loop-gain condition remains limited. The revision indeed provides a useful boundedness argument and the controls for signal lag. I would suggest the authors to clearly state how these results differ from the standard equilibrium Jacobian condition, and to clarify which conclusions follow from the spectral analysis, as opposed to the bounded-drive argument or the empirical interventions.

  • CW2: ok

  • Q1: ok

  • Q2: The revision now correctly presents Eq.(7) as the standard Jacobian condition, but it is unclear whether the bounds on signal amplitudes also constrain the loop-gain spectrum. Could the authors provide explicit bounds on the signal sensitivities AA and BB, or clarify that bounded signal amplitudes alone do not necessarily establish control of the loop-gain spectrum?

  • Q3: ok

  • Q4: partly ok, but now the new experiments indicate that finite signal-relaxation times play an important role in the observed divergence. Yet, the Jacobian is computed using the relaxed signals. It is therefore unclear to me how well this reduced Jacobian captures the stability of the actual delayed learning dynamics, beyond its correlation with diverging trajectories. Can the authors provide further evidence for this connection and distinguish more clearly between the equilibrium stability criterion, the trajectory-based diagnostic, and the separate boundedness result?

  • Q5: 0k

  • Q6: fine, but please clarify in the updated manuscript that exact reciprocal update symmetry is a modeling assumption, and also which results remain valid under unequal potentiation or decay rates, and which rely on the cancellation in Eq. (13)?

  • Q7: Can the authors derive and test a lag-dependent stability condition in a simple scalar or two-layer model? Otherwise, they should clarify that the current analysis focuses on equilibrium stability and does not predict the onset of divergence.

Comments on the paper

1 comment

To all reviewers: additional experiments and analysis

Thank you for the thoughtful reviews. We have updated the manuscript to incorporate your feedback. Below we address common questions and what our revision changes.

  • What Condition (17) claims. (17) states the principle that alignment leaves stability to the loop gain: KP removes the difference between forward and feedback weights but not their overall size, so when the loop gain exceeds the decay rate, a small change in the weights can grow. It is exact for a network resting at aligned weights. Training is not at that point; most learning happens while the feedback is still far from aligned (mean forward–feedback cosine 0.21–0.41 when accuracy reaches 90% of its final value), and the weights are still moving. So the threshold states the principle rather than a sharp boundary: runs that train sit one to two orders of magnitude above it, while runs that diverge escape it by many orders (Section 5.1; Fig. 3).
  • Continuous time amplifies the loop, as Section 5.2 says. When every signal settles instantly, none of our networks with unbounded units diverged (0 of 360 runs, at depths up to 10). From the same initial weights with the paper's 1 ms signals, most deep networks diverge, and faster signals make fewer diverge (Appendix G.1, Fig. 8).
  • Alignment does not prevent the failure. Feedback that matches the forward weights exactly from the start still diverges in deep networks (Appendix G.2).
  • Any bound on the units works. Tanh, hard-tanh and the logistic unit prevent the divergence as the clipped unit does, and tanh and hard-tanh train better at depth 10 (Appendix G.4, Table 2). With bounded units every signal and weight stays bounded at any depth (Appendix E), and saturation costs little accuracy (Appendix G.5).

The revision keeps the paper's claims, experiments and remedy. Section 5.1 clarifies what (17) does and does not predict, and Appendix G adds the experiments above. The revision also writes out the KP model (Section 4), defines stability at the start of Section 3 with a small example, broadens the related work (Section 2), and states the initialisation correctly (He-uniform forward and Xavier-uniform feedback weights), along with the hidden widths, sampling with replacement, the integration step and the 1,000-image test subset (Appendix F). We thank the reviewers again for their comments and criticism that we believe made our paper better.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.