AI Insight
This paper introduces Score Broadcast and Decorrelation (SBD), a mathematical framework that generalizes biologically plausible learning algorithms beyond traditional backpropagation. The authors demonstrate that broadcasting the score (gradient of loss with respect to output) instead of errors provides a unified approach for credit assignment across multiple loss functions including cross-entropy, Bregman divergences, and proper scoring rules. The framework is grounded in an orthogonality principle and provides theoretical justification for three-factor learning rules observed in neuroscience, with experimental validation on CIFAR-10 and Tiny ImageNet showing improvements over existing broadcast-based methods.
Why it matters
This work could help bridge the gap between artificial neural networks and biological learning mechanisms, potentially leading to more energy-efficient and neuromorphic computing architectures. By eliminating the need for weight transport (a biologically implausible aspect of backpropagation), this framework may inform both the development of brain-inspired AI systems and our understanding of how biological neural networks actually learn.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: We introduce Score Broadcast and Decorrelation (SBD), a principled framework for broadcast-based credit assignment for general families of differentiable losses. Error broadcast is a biologically plausible alternative to backpropagation that sends output information to hidden layers without weight transport. The Error Broadcast and Decorrelation (EBD) framework, recently introduced for the mean-squared-error (MSE) setting, grounded this mechanism in the stochastic orthogonality of optimal estimators, under which the optimal residual is orthogonal to functions of the input. We generalize that foundation by introducing an orthogonality principle between the output score (the gradient of loss with respect to the final-layer output) and hidden-layer activations, which holds whenever the optimal score has conditional mean zero. This single principle unifies broadcast-based credit assignment across the standard differentiable-loss families, including cross-entropy, Bregman divergences, proper scoring rules, and exponential-family negative log-likelihoods. The framework supplies a theoretical grounding for the three-factor learning rule under general losses, with the neuromodulatory factor derived as the broadcast loss score. We derive the cross-entropy case explicitly, characterize the admissible loss class, and introduce a score vector expansion technique that enriches the broadcast signal while preserving the orthogonality framework. Experiments on CIFAR-10 and Tiny ImageNet show that SBD substantially improves over existing broadcast approaches, with score vector expansion delivering further gains. Overall, this work identifies the loss score as the signal to broadcast, supplies the orthogonality theory and theoretical grounding for the three-factor learning rule from neuroscience, and shows how score vector expansion enriches the decorrelation directions of the resulting objective.
Source: Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment