acceptodds
Under review as a conference paper at ICLR 2027

Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment

Abstract

We introduce Score Broadcast and Decorrelation (SBD), a principled framework for broadcast-based credit assignment for general families of differentiable losses. Error broadcast is a biologically plausible alternative to backpropagation that sends output information to hidden layers without weight transport. The Error Broadcast and Decorrelation (EBD) framework, recently introduced for the mean-squared-error (MSE) setting, grounded this mechanism in the stochastic orthogonality of optimal estimators, under which the error of the optimal estimator is orthogonal to functions of the input. We generalize that foundation by introducing an orthogonality principle between the output score (the gradient of loss with respect to the final-layer output) and hidden-layer activations, which holds whenever the optimal score has conditional mean zero. This single principle unifies broadcast-based credit assignment across the standard differentiable-loss families, including cross-entropy, Bregman divergences (with MSE as a special case), proper scoring rules through unconstrained links, and exponential-family negative log-likelihoods. The framework supplies a theoretical grounding for the three-factor learning rule under general losses, with the neuromodulatory factor derived as the broadcast loss score rather than postulated. We derive the cross-entropy case explicitly, characterize the admissible loss class, and introduce a score vector expansion technique that enriches the broadcast signal while preserving the orthogonality framework. Experiments on CIFAR-10 and Tiny ImageNet show that SBD substantially improves over existing broadcast approaches, with score vector expansion delivering further gains. Overall, this work identifies the loss score as the signal to broadcast, supplies the orthogonality theory and theoretical grounding for the three-factor learning rule from neuroscience, and shows how score vector expansion enriches the decorrelation directions of the resulting objective.

Primary area:
applications to neuroscience & cognitive science
Keywords:
credit assignment, biologically plausible learning, local learning, broadcast learning, score orthogonality, three-factor learning, score vector expansion

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zAUGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper seeks to develop neuro-plausible mechanisms for learning in neural networks, building on a recently introduced mechanism termed error broadcast and decorrelation (EBD). The key idea is as follows. Denoting the gradient of the loss function L\cal{L} with respect to the activations at the last layer (prior to the softmax, if one is used) by δ\delta, the authors write a first order optimality condition as E[δ∣X]=0E[ \delta |X] = 0, where XX denotes the input. This generalizes the idea behind the orthogonality principle for minimum mean squared error (MMSE) used in the EBD paper, where delta = Y – E[Y|X]. It follows that E[δf(X)∣X]=0E[\delta f(X)|X] = 0 for any well-behaved function f()f(). In particular, if f(X)=hk(X)f(X) = h_k(X) denotes (a function of) activations at an intermediate layer kk, we can use global feedback of δ\delta to drive layer-wise loss functions seeking to drive E[δhk(X)∣X]E[\delta h_k (X)|X] to zero. This provides an alternative to backprop.

The authors also point out that the approach can be expanded by setting f(X)=a(X)hk(X)f(X) = a(X) h_k(X), where a(X)a(X) is a function that is independent of layer index kk. They experiment with using multiple choices of a(X)a(X) together in an “expanded” scheme.

Evaluations are provided for classification on small vision datasets: CiFAR-10 and Tiny ImageNet. The proposed schemes are not competitive with standard backprop, but the proposed approach applied with cross-entropy loss is shown to be superior to the original EBD scheme, and the expanded scheme can yield some improvements.

Critical strengths

While the observation on first order optimality conditions is a straightforward extension of the concept in the EBD paper, its application to cross-entropy does lead to improved classification performance in the evaluation. It is also applied to a toy example (Appendix B) in which the loss function is a Poisson negative log likelihood.

Critical weaknesses

My main issue with the paper is regarding its potential for intellectual impact. It is unclear what the consequences are for either machine learning (not better than backprop) or computational neuroscience (what next?).

The evaluations are for small vision datasets, and the results are still inferior to backprop.

The paper as written might give an impression of optimality in training objective to the casual reader. While standard loss functions are convex in last layer activations, they are highly non-convex in the network parameters being learnt, hence all the framework is doing is providing a shortcut from the output to intermediate layers to drive reasonable training objective. While this is obvious, it should be clearly stated.

This is an observation, not necessarily a weakness: the derivations in the appendix are rather obvious. Not sure they are all needed, but they give the impression of more sophistication than is warranted for the rather simple idea the authors are bringing to the table.

Questions for authors

I would like for you to make a stronger case for intellectual impact in machine learning and/or computational neuroscience.

4 comments
AuthorsOpenReview ↗

Response to reviewer comments (Part I)

We thank the reviewer for their careful reading of the paper and thoughtful assessment. Below we provide our clarifications and provide information about relevant enhancements in the revised article.

Intellectual impact

We should first start with the observation that backpropagation (BP) remains as the accuracy reference, given the physical communication and computational resources are suitable for the generation of exact derivatives at individual nodes. SBD answers a question that BP's accuracy leaves open: when synapses are constrained to see only local activity and a broadcast teaching signal, what should that signal be for a given task loss, and why does plasticity driven by it assign credit? To answer this question, SBD establishes a principled framework for broadcast-based credit assignment and three-factor learning, connecting global task objectives to local synaptic plasticity. It identifies

  • Principled selection and justification of the broadcast signal: SBD provides a loss- and parameterization-aware theoretical foundation for broadcasting the score. Under the stated regularity conditions, an interior minimizer of the conditional risk in an unconstrained output parameterization yields a conditionally mean-zero score, establishing its orthogonality to input-measurable features with integrable score products. This covers Bregman divergences, proper scoring rules via unconstrained links, and exponential-family likelihoods. Under convex conditional risk, a conditionally mean-zero score also certifies population optimality. These results explain why the score is an appropriate teaching signal and provide an optimality-based rationale for local decorrelation objectives (Thms 3 and 5; Thm 6, App B).
  • How a synapse uses it: the global task objective is mapped to local activity - score decorrelation losses, whose optimization yields a three-factor rule with a loss-derived modulatory factor.
  • How to enrich it: score vector expansion (SVE) relaxes the output dimension based rank bottleneck through input dependent modulation(Sec. 6).
  • Why the local dynamics work: the new Lyapunov analysis (Theorem 4, App. A.7) proves convergence of moment-score correlations to zero even in the presence of partial gradient information.

Regarding the impact and future prospects for the specific fields:

Computational Neuroscience We can list

  • Three-factor plasticity: Three factor rule is experimentally supported. SBD provides a unified theoretical grounding for its task-dependent modulatory component across a broad family of losses. It specifies how the task objective determines a broadcast teaching signal and how local score–activity correlations transform that signal into neuron-specific modulation. This connects hypotheses about what a circuit learns to testable hypotheses about the signals that guide its synaptic plasticity.
  • Weight symmetry free credit assignment: The task score decorrelation based flexible credit assignment formulation eliminates the requirement about precise propagation paths with feedback weights transported forward weights, the fundamental critique about the bio-implausibility of BP.

What next:

  • Falsifiable predictions and analysis tools for experiments:This framework generates concrete, falsifiable predictions for experimental investigations:

    • Each candidate behavioral objective defines a specific teaching signal (neuromodulatory factor) and predicts that synaptic plasticity progressively extinguishes correlations between local neural activity and broadcast signals as task proficiency is reached. These correlations act as a signature or reflector of local learning performance.

    • Broadcast enrichment through input-dependent modulation offers a new theoretical lens for analyzing top-down signaling.

  • Further directions Further exploration of SVE and the choice of activation nonlinearities are potential future directions.

Machine Learning For machine learning, SBD provides a theoretical basis for designing learning algorithms under constraints on feedback communication and local computation. Once the output score is available, selected layers or modules can compute their updates without waiting for derivatives to traverse the intervening network. This supports concrete directions

  • Parallel updates Broadcast signal broadcasted to all network nodes simultaneously enabling parallel udpates.
  • On-chip learning. Some neuromorphic chips already use three-factor rules, but how to choose such rules in a principled way remains mainly open. SBD derives the rule from the loss and reaches hidden layers without weight transport.
  • Targeted credit delivery. The score can be routed to any subset of nodes, enabling fine-tuning of selected modules (e.g., MLP blocks of a transformer) under any differentiable objective with no full backward pass.
AuthorsOpenReview ↗

Response to reviewer comments (Part II)

small vision datasets, and the results are still inferior to backprop.

We agree that BP remains the strongest accuracy reference in the reported experiments and that larger-scale evaluation would strengthen the practical case. The contribution of SBD is a general, theoretically grounded framework for credit assignment under local learning constraints based on broadcasted credit information. It connects a broad family of task losses to appropriate broadcast signals and three-factor updates, without requiring a backward chain of derivatives or feedback weights matched to forward weights. SVE further provides a principled mechanism for enriching that feedback. The residual-network and online CorInfoMax experiments also demonstrate applicability under different architectural and learning conditions, including single-sample updates, lateral interactions, and unshared local connections.

The gap to BP is an important remaining challenge, which we explicitly report. It identifies the performance still to be recovered when replacing exact end-to-end gradient transport with the proposed local mechanism. We therefore position the current results as controlled empirical support for a flexible credit-assignment framework, with broader theoretical and architectural implications, while reserving claims of large-scale competitiveness for further evaluation.

On optimality and convergence

We thank the reviewer for highlighting this distinction. We have added an explicit clarification in Section 4:

"These results provide an optimality-based foundation for constructing local learning objectives. The convexity assumption in the sufficiency argument concerns the conditional risk as a function of the output a\mathbf a, whereas optimization over the network weights remains generally nonconvex, as in standard deep-network training. Accordingly, the population-optimality characterization does not itself imply global convergence of the parameter updates; convergence of the specified local decorrelation dynamics is established separately below under explicit assumptions."

In the revised article, we have also provided a convergence analysis for the update dynamics. Under the stated boundedness and step-size conditions, it establishes bounded weights and vanishing current-minibatch and EMA score–activation moments, even as inputs and scores change during training. This guarantees decorrelation for the specified local rule despite omitting output-score derivatives (Section 4 and Appendix A.7).

On the appendix derivations

Several derivations become straightforward once the appropriate score-orthogonality formulation is identified. We include these steps to make the assumptions, parameterization choices, and local approximations transparent, helping readers verify and build on the results. The central contribution lies in establishing the general formulation and developing its consequences for loss-dependent broadcast signals, three-factor learning, and orthogonality-preserving score expansion. The appendix provides a self-contained account of these connections, making the framework accessible to readers with different backgrounds.

I appreciate the authors' effort in preparing the response and revision. However, I remain sceptical about the impact of this work, and will leave my rating as is.

AuthorsOpenReview ↗

Closing remarks on intellectual impact

Thank you for considering our response and revised manuscript. As the discussion period draws to a close, we would like to reiterate the theoretical results and the evidence across diverse network architectures that bear most directly on your question about the paper’s intellectual significance.

SBD establishes a general theoretical foundation for broadcast-based credit assignment. We show that, across a broad family of losses, the output score in principally chosen output coordinates should be orthogonal to hidden activity at the population optimum. This explains why the score is a natural teaching signal and gives us a criterion for designing alternatives: a candidate broadcast signal should preserve the same orthogonality at the optimum. Score vector expansion is one example of how to use this criterion. Three-factor rule emerges naturally from this principle, and we show how to enrich the teaching signal through score vector expansion (SVE) while preserving orthogonality. Under the stated assumptions, our Lyapunov analysis shows that the unregularized local ReLU rule drives its decorrelation objective to zero.

We demonstrate learning in CNNs, a ResNet, fully and locally connected recurrent networks, and a Vision Transformer. We also showcase the clear benefit of SVE.

The proposed SBD framework has direct implications for both machine learning and computational neuroscience. For machine learning, SBD lets selected layers update in parallel from a shared score without waiting for gradients to pass through intervening layers. This opens a path to training individual modules and to on-chip learning. For computational neuroscience, SBD turns a hypothesis about what a circuit learns into a prediction about its local plasticity. Given a candidate objective and an output representation, the theory specifies a teaching signal whose variation across trials can be tested against measured modulatory activity and synaptic changes.

Reviewer_mpbyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Score Broadcast and Decorrelation (SBD), extending MSE-based EBD to differentiable losses through the output score. It characterizes population-optimal score orthogonality, derives local three-factor updates, and introduces score vector expansion (SVE). Evaluation focuses on image classification, with additional synthetic Poisson regression and experiments on residual and CorInfoMax networks.

Critical strengths

Detailed theoretical development. The manuscript provides extensive derivations and proofs, clearly states the assumptions underlying score orthogonality, and discusses the additional conditions required for sufficiency. The unified treatment of different loss families provides a useful conceptual extension of EBD.

Thorough experimental documentation. The appendices describe architectures, update equations, optimization settings, and hyperparameters in considerable detail. Multi-seed results, modulator ablations, update-alignment diagnostics, sensitivity analyses, and runtime and GPU-energy measurements provide useful supporting evidence.

Promising results within the evaluated settings. SBD improves over the evaluated broadcast baselines. The Tiny ImageNet comparison, where plain and expanded SBD share optimization settings, provides encouraging evidence for SVE, improving accuracy from 28.3% to 31.4%.

Critical weaknesses

Limited benchmark and task coverage. Real-data evaluation is restricted to CIFAR-10 and Tiny ImageNet image classification. The additional non-classification experiment is a synthetic Poisson regression task. These relatively small-scale settings provide useful proof-of-concept evidence, but limited validation of practical applicability across the broad loss families covered by the framework. A real-world regression or count-prediction task, together with controlled comparisons across suitable losses, would strengthen this aspect.

Limited coverage of model families. Although the residual and CorInfoMax experiments broaden architectural coverage, no Transformer, attention-based model, or language-modeling setting is evaluated. Applicability beyond the tested architectures therefore remains unclear. A small Transformer experiment would be a reasonable extension. Large-output settings also deserve attention because the correlation-state and broadcast costs grow with output dimensionality, which may become important for large-vocabulary prediction.

The connection between the theory and practical learning dynamics remains incomplete. The proofs primarily characterize population optima, while sufficiency relies on a sufficiently rich feature family. The practical algorithm uses finite features, running moment estimates, and local updates that omit derivatives through the output score. The authors acknowledge this distinction, but the reported alignment diagnostics do not fully explain the behavior of the complete training updates. A tractable analysis or controlled experiments connecting moment reduction, actual parameter updates, and task-loss improvement would substantially strengthen the paper.

The benefits and mechanism of SVE are not sufficiently isolated. On CIFAR-10, the plain-to-expanded comparison also changes batch size from 16 to 64 and initial learning rate from (10^{-4}) to (4\times10^{-4}). The matched Tiny ImageNet result is stronger evidence, but the contribution of expansion should be tested consistently under matched settings. Existing modulator ablations are useful; however, effective-rank diagnostics and dimension- and scale-controlled comparisons would better establish whether the gains arise from the proposed additional decorrelation directions.

Questions for authors

My main concern is that the experiments are too limited, in terms of task coverage, benchmark coverage and model coverage. Can the authors provide additional evidence on a real-world non-classification task or a small Transformer model to support broader applicability?

Can the authors more directly connect the population orthogonality results to the practical learning process, through a simplified theoretical analysis or diagnostics of the complete updates?

How do plain SBD and SVE compare under identical CIFAR-10 optimization settings? Do effective-rank or other mechanism-oriented measurements support the proposed explanation for SVE?

I am willing to raise my score if the rebuttal convincingly addresses these concerns through additional evidence, clearer analysis, or stronger controlled ablations.

4 comments
AuthorsOpenReview ↗

Response to reviewer comments (Part I)

We thank the reviewer for their detailed and constructive feedback. We have combined our responses to the weaknesses and questions under the following four headings to detail our clarifications and describe the relevant improvements to the revised manuscript.

A real-world regression or count-prediction task... would strengthen this aspect.

We thank the reviewer for this suggestion. Following this advice, we have added Appendix B.2, which evaluates hourly rental-count prediction on the real-world UCI Bike Sharing dataset with a chronological split.

Using the same MLP, split, paired test seeds, and validation-tuning budget, we compare BP across three admissible loss–head pairs, SBD with the two log-mean heads, and EBD, which is the squared/linear special case of SBD.

Test Poisson deviance (over five seeds; lower is better):

Hidden-layer ruleSquared / linearSquared / log-meanPoisson / log-mean
EBD22.81±1.2422.81\pm 1.24--
SBD-19.57±1.7719.57\pm 1.7718.27±1.1118.27\pm 1.11
BP21.38±2.1121.38\pm 2.1120.11±1.3020.11\pm 1.3017.33±0.9117.33\pm 0.91

The EBD baseline is constrained to the linear-head MSE formulation, whereas SBD supports a Poisson NLL loss with a log-mean output. In this experiment, SBD with that pair has 19.9% lower test deviance than EBD (better on all five seeds, paired tt-test p=0.0003p=0.0003) and 6.2% lower RMSE (70.5 vs. 75.2). Its deviance is within 5.4% of BP's (p=0.33p=0.33), with nearly equal RMSE (70.5 vs. 70.1). These results show that SBD can use a count-appropriate loss while performing close to BP on this real-world task.

beyond the tested architectures

Following the reviewer's suggestion, we have added a Vision Transformer experiment on CIFAR-10 in Appendix I, extending the evaluation to an attention-based architecture. We compare SBD with score vector expansion against end-to-end BP using a two-block ViT, five paired seeds, and 200 training epochs. Under the same fixed, test-selected recipe, final test accuracy is 69.62±0.5969.62 \pm 0.59% for SBD and 78.34±0.3478.34 \pm 0.34% for BP. Appendix I provides the architecture, component-wise local objectives and updates, training protocol, and full learning curves. These results demonstrate that SBD can train this attention-based model with a similar gap-to-BP as CNN architecture..

We would like to highlight the architecture-agnostic basis of the framework which is based on the score–feature orthogonality result in Theorem 3: it applies to any input-measurable feature satisfying the stated integrability conditions. Transformer representations fall within this scope. In our implementation, gradients are computed within each Transformer block and stopped between stages. We distinguish this general applicability of the objective from the convergence guarantees of Theorem 4, which concern the specific update rule and assumptions analyzed there. In summary, our experiments now demonstrate SBD across convolutional networks with weight sharing, ResNets with residual connections, biologically motivated CorInfoMax networks with lateral recurrence and locally connected pathways, and Transformers with attention blocks, providing empirical support for its applicability across diverse architectural mechanisms.

AuthorsOpenReview ↗

Response to reviewer comments (Part II)

The connection between the theory and practical learning dynamics ...local updates that omit derivatives through the output score... a tractable analysis or controlled experiments connecting moment reduction

We thank the reviewer for this suggestion. Indeed, in the revised article, we included a direct analysis of the local learning dynamics in Section 4 Theorem 4, with its proof in Appendix A.7.

The main difficulty is that SBD omits the gradient component arising from the derivative of the output score, so the standard gradient-descent argument does not establish convergence of the decorrelation objective. We address this through a Lyapunov function based on the squared Frobenius norm of the weights as the energy function. For local SGD updates in plain feedforward ReLU networks, bounded external inputs and broadcast scores, together with explicit constant-step conditions, ensure that the decorrelation objective converges to zero. This result holds with finite hidden features and does not require the omitted gradient component to be small.

The analysis therefore establishes a direct connection between the theoretical decorrelation criterion and the local update dynamics. Our finite-feature experiments additionally provide evidence of useful task learning, and we view the theoretical characterization of the finite-feature case as valuable research direction.

The benefits and mechanism of SVE are not sufficiently isolated. On CIFAR-10, the plain-to-expanded comparison also changes batch size from 16 to 64 and initial learning rate from (10^{-4}) to (4\times10^{-4}).... effective-rank diagnostics

We agree that the original CIFAR-10 comparison confounded score expansion with batch size. We reran the plain SBD and SBD+SVE experiments at the same batch size of 16. The SVE gain persists at both widths:

  • 1× width: test accuracy rises from 70.0±0.770.0\pm0.7% to 71.2±0.671.2\pm0.6% (+1.2 percentage points).
  • 4× width: test accuracy rises from $74.5$% to 74.9874.98% (+0.48 points).

We also reran 4×-width BP with batch size 16; its accuracy changed from 83.183.1% at batch size 64 to 83.0383.03%. We updated Section 7, Table 1, and Appendix E’s train- and test-accuracy curves and relevant information.

The plain SBD and SBD+SVE learning rates remain separately selected (10−410^{-4}) and 4×10−44\times10^{-4}), respectively, at their best-accuracy values. (All models were hyperparameter tuned for optimal accuracy). Thus, the revised comparison controls for batch size and shows that the SVE gain persists, while comparing each method at its selected learning rate rather than isolating expansion at a fixed learning rate. These accuracy results strengthen the evidence for SVE’s benefit.

We also performed effective rank diagnostics suggested by the reviewer for the CIFAR-10 experiment. We measured the effective ranks of the correlation matrices of plain (δ\boldsymbol{\delta}) and expanded score ($\tilde{\boldsymbol{\delta}} =\bigl[\boldsymbol{\delta};\mathbf{p}(X)\odot\boldsymbol{\delta};\mathrm{roll}_{k}(\mathbf{p}(X))\odot\boldsymbol{\delta}\bigr]$), over 33 seeds and obtained the following values:

Model Trained WithPlain Score E. RankExpanded Score E. Rank
Plain Scores8.67±0.018.67\pm 0.0119.82±0.0919.82\pm 0.09
Expanded Score8.69±0.028.69 \pm 0.0219.93±0.2219.93\pm 0.22

Here the effective rank is defined as re(R)=e−∑qilog⁡(qi)r_e(\mathbf{R})=e^{-\sum q_i \log(q_i)} where qi=σi∑σiq_i =\frac{\sigma_i}{\sum \sigma_i} and σi\sigma_i's are the singular values of R\mathbf{R}, which demonstrates effective dimension enrichment in broadcast through score vector expansion.

I thank the authors for their response and for the effort they put into the rebuttal. The response addresses some of my concerns. After considering the paper and the clarifications provided, I have decided to keep my original rating.

Thank you for following up. You mentioned that our response addressed some of your concerns. Could you specify which concerns remain and where you find the new evidence or analysis insufficient? We would appreciate the opportunity to respond directly before the discussion closes.

Reviewer_TDikOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces Score Broadcast and Decorrelation (SBD), a novel error broadcast framework that may provide a more biologically plausible alternative to standard backpropagation. SBD directly extends a recently proposed error broadcast framework, called Error Broadcast and Decorrelation (EBD), and generalizes the original MSE setting on which EBD was based to arbitrary differentiable losses. The generalization is achieved by introducing an orthogonality principle between the output and the hidden-layer activations, which holds whenever the population-optimal score has zero conditional mean.

Critical strengths
  1. Overall, this paper presents a direct extension of a previous error broadcasting framework, EBD, by generalizing it beyond MSE. The generalization for any arbitrary output score is certainly useful, and the orthogonality principle that enables the generalization is interesting.

  2. The technical and literary presentation of the manuscript is clear, logical, and relatively easy to follow.

  3. The proposed SBD framework has been evaluated fairly comprehensively based on various models, convincingly demonstrating its potential advantages.

Critical weaknesses
  1. As SBD directly extends EBD, there should be a more detailed discussion about how SBD builds on EBD, what are similar, and what the critical differences are. For example, it is not entirely clear what theoretical statements/developments are distinct and novel in this manuscript and what are restatement of what was presented in EBD.

  2. When comparing the performance between SBD and EBD, it is unclear what improvement is a result of changing the loss function from MSE to CE and what portion of the improvement may be attributed to the methodological differences. These should be clarified.

  3. As the performance of existing methods is often cited from previously published work, it is not entirely clear whether all comparisons are fair and consistent. This should be clarified as well.

  4. While it is mentioned that SBD may provide a more "biologically plausible" alternative to backpropagation, this claim seems to be a simple restatement of the error-broadcasting framework and is not fully substantiated in the current manuscript.

Questions for authors

Please see the comments in the weaknesses section and provide clarifications.

5 comments
AuthorsOpenReview ↗

Response to reviewer comments (Part I)

We appreciate the positive assessment and the positive constructive comments by the reviewer. Below we provide our clarifications and also information about relevant enhancements in the revised article.

  1. Positionining SBD relative to EBD

We have added an introductory paragraph to Section 1.2 that explicitly positions SBD as a general theoretical framework for broadcast-based credit assignment and three-factor learning across a broad family of losses, with EBD recovered as the particular case corresponding to MSE loss. The relationship and contributions are as follows:

i. Shared construction: SBD inherits EBD’s layerwise decorrelation construction and three-factor update structure; Section 2 reviews their MSE orthogonality foundation.

ii. General theoretical framework: SBD identifies the loss- and parameterization-dependent score as the principled broadcast signal and characterizes when it has zero conditional mean at optimality. This grounds broadcast learning and three-factor updates across a broad loss family, including cross-entropy, Bregman losses, and exponential-family likelihoods, with EBD as the MSE special case within Bregman losses (Sections 4–5).

iii. Score vector expansion: SVE augments the broadcast with input-dependent score weightings, enabling additional decorrelation constraints while preserving orthogonality at the same population optimum (Section 6).

iv. Convergence analysis added in the revision: A new Lyapunov proof establishes vanishing current-minibatch and EMA score–activation correlations and bounded weights, even as inputs and scores change. The guarantee covers the unregularized local ReLU rule with identity features, bounded inputs and scores, fixed EMA retention, and a sufficiently small constant scalar step. It applies despite omitting output-score derivatives and includes the MSE/EBD special case.

We would like to emphasize that these contributions did not exist in prior work, including EBD, and are solely additions by the SBD framework.

2. Performance Improvement by SBD relative to EBD

We would like to note that EBD framework is constrained to MSE loss only, and CE loss-derived signal cannot be broadcast without the generalized SBD formulation. There are two principal factors for increased performance of SBD relative to EBD

i. Broader loss choice: Unlike MSE-constrained EBD, SBD framework allows us to use relevant objective loss function for the given task, for example, for classification, CE is the appropriate choice; and newly added bike-sharing regression task example uses Poisson NLL.

ii. Score vector expansion: SVE is a novel mechanism derived from score orthogonality that enriches the broadcast while keeping the loss unchanged. On Tiny ImageNet, adding SVE under CE improves accuracy from 28.3±0.428.3 \pm 0.4% to 31.4±0.431.4 \pm 0.4% with the same architecture and optimization settings, demonstrating an additional 3.1-point gain beyond switching to CE.

AuthorsOpenReview ↗

Response to reviewer comments (Part II)

  1. Fairness and consistency of comparisons with previous work

We thank the reviewer for raising this important point. All Tiny ImageNet results (including BP (MSE), EBD (MSE), and NN-GEVB) were obtained in-house, as were the CE-based BP and DFA results on CIFAR-10. Within each dataset, our CE-based BP, DFA, and SBD experiments share the CNN architecture and evaluation procedure. For Tiny ImageNet NN-GEVB, we adapted the authors’ implementation to match the CNN layer widths and spatial layout and used the same principal training settings as SBD, including the training epochs, batch size, optimizer, initial learning rate, decay factor, weight decay, dropout, and training augmentation, while retaining GEVB’s defining architecture and learning rule.

The revision makes these shared conditions explicit and provides additional DFA and GEVB implementation details in Appendices E-F. Section 7 and Tables 1-2 also clearly identify the published CIFAR-10 MSE and NN-GEVB results and their sources.

  1. Biological relevance and generality of the framework

SBD provides a principled connection between a neural system’s computational objective and the teaching signals governing local plasticity. It identifies how these signals depend on the loss and output representation, and characterizes the conditions supporting their use in three-factor learning. This supplies a common theoretical foundation for biologically motivated models across diverse estimation and prediction objectives, with EBD’s MSE formulation recovered as one particular instance.

SVE proposed in this article is an additional mechanism, enriching shared feedback through input-dependent functions of activity or predictions. The expanded signals can expose score–activity correlations missed by the original broadcast while preserving orthogonality at the same population optimum. This provides a normative basis for investigating how neural populations could use shared teaching signals more effectively across different activity contexts.

These theoretical contributions are complemented by the online CorInfoMax experiments, which demonstrate learning with single-sample updates, recurrent lateral interactions, and unshared local connections in the locally connected model (Section 7 and Appendix G). We have added a concise clarification in Section 4 emphasizing this broader contribution to biological modeling and its connection to the computational evidence:

"For biological modeling, SBD provides a flexible normative framework linking diverse computational objectives to broadcast teaching signals and three-factor plasticity. The online CorInfoMax experiments illustrate compatibility with recurrent dynamics and, in the locally connected model, unshared weights (Appendix G)."

  1. Thank you. Adding this clarification is helpful for understanding the novel contributions in this work compared to the prior work, EBD.
  2. Thanks for the clarification. I understand that EBD can't use the CE loss function while SBD can, which might be the biggest factor in the performance gain. But it's good to know that SVE leads to additional improvements.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to reviewer comments and main revisions to manuscript

We thank all reviewers for their careful reading, detailed assessments, and constructive suggestions. Their feedback has helped us strengthen the theoretical analysis, broaden the experimental evidence, and clarify the contribution and scope of SBD. The manuscript and repository have been revised in accordance. Alongside our separate responses, we summarize the main revisions and response points below.

Main changes to the revised manuscript

  • New convergence analysis: We added a Lyapunov analysis establishing vanishing score–activity correlations and bounded weights for the specified unregularized local ReLU rule under explicit assumptions. Main obstacle in establishing such convergence analysis was due to the fact that SBD update omits the component of the gradient corresponding to the derivative of the score. Therefore, standard descent type convergence analysis was obscured. Our proof tracks a separate nonnegative “energy,” called a Lyapunov function, rather than requiring the decorrelation objective to decrease at every step. For each layer, we use V[m]=∣∣W[m]∣∣F2+κ∣∣R^[m]∣∣F2,V[m]=||W[m]||_F^2+\kappa||\widehat R[m]||_F^2, where the terms measure the squared sizes of the weights WmW_m and estimated score–activity correlations R^m\widehat R_m, and κ≥0\kappa\ge0 accounts for moving-average memory. For the specified unregularized ReLU rule, the stated boundedness and step-size conditions yield V[m]−V[m−1]≤−c∣∣R^[m]∣∣F2,c>0,V[m]-V[m-1]\le -c||\widehat R[m]||_F^2,\qquad c>0, meaning that the remaining correlations require a corresponding decrease in energy. Since this energy cannot fall below zero, the accumulated squared correlations must be finite, forcing ∣∣R^[m]∣∣F2→0||\widehat R[m]||_F^2\to0 as m→∞m\to \infty while keeping the weights bounded; extending the argument layer by layer establishes this result even when all layers learn simultaneously. The convergence theorem is provided in Section 4 and its proof is in Appendix A.7.

  • Broader experimental coverage: We added real-world hourly bike-sharing count regression, with comparisons across loss–output parameterizations, and a two-block Vision Transformer experiment, extending the evidence beyond image classification with convolutional networks. ( Section 7, Appendix B.2 for real-world count regression, Appendix I for the Vision Transformer. )

  • Additional evidence for score vector expansion: We added CIFAR-10 comparisons with matched batch sizes, explicitly disclosing the separately selected learning rates, and effective-rank diagnostics supporting the enrichment of the broadcast signal. ( Section 7, Table 1, and Appendix E.)

  • Clearer presentation and experimental reporting: We clarified the relationship to EBD, the biological relevance of the framework, the scope of the theoretical guarantees, and the provenance and experimental conditions of baseline comparisons. ( Appendix, Sections 1.2 and 4; Section 7, Tables 1–2, and Appendices E–F.)

Main points addressed in our responses

  • Contribution beyond EBD: SBD establishes a general, loss- and parameterization-aware foundation for broadcast credit assignment and three-factor learning, with EBD recovered as the MSE special case. SVE introduces a complementary mechanism for enriching feedback while preserving population-optimal orthogonality.

  • Meaning of the theoretical guarantees: We distinguish population-optimality conditions from convergence of the local decorrelation dynamics. Neither finite-feature decorrelation nor the new convergence theorem establishes global task-loss optimality for the complete training procedure.

  • Sources of empirical improvement: We distinguish the gains enabled by changing the loss from the additional gains from score expansion, including the 3.1 percentage-point Tiny ImageNet improvement under the same cross-entropy loss.

  • Intellectual impact and future directions: We explain how the framework connects wide range of task related objectives to models about modulatory plasticity and supports further algorithmic and analytical work on credit assignment with potential impact on computational neurosciance and machine learning fields. We also identify opportunities for parallel, modular, and neuromorphic learning, while acknowledging the remaining accuracy gap to BP.

AuthorsOpenReview ↗

Welcoming Feedback and Further Questions

Dear Reviewers,

We would like to thank you again for your efforts. With fewer than two days remaining in the discussion period, we would welcome any further questions or comments on our responses and revised manuscript. We would be happy to clarify any points or address any remaining concerns. For ease of reference, our general response summarizes the main revisions, with detailed replies provided under each review.

Thank you for your time and consideration of our submission and revisions.

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.