AI Insight
This paper introduces Deep Memory (DM), a mechanism that enables artificial neural networks to maintain stable memories even when their computational units are randomly replaced over time. The system uses a three-part process: recording specialized memory patterns, reintroducing stored information into new units, and continuously stabilizing these memories through re-entry, without requiring traditional gradient-based learning methods. Across over 1,000 simulation runs, the authors demonstrate that this approach achieves high memory retention (98.4% correlation) compared to systems without memory (38.5%), and that the combination of recording and seeding components is minimally necessary for function.
Why it matters
This work addresses a fundamental challenge in neuromorphic and energy-constrained computing systems where components may fail or be replaced. The mechanism offers a biologically plausible model analogous to hippocampal memory consolidation and could inform the design of robust artificial cognitive systems that maintain learning despite hardware instability or continuous component turnover.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Dissipative cognitive architectures maintain computation through continuous energy expenditure, where units that exhaust their energy are stochastically replaced with fresh random state. This creates a fundamental challenge: how can persistent, context-specific memory survive when all learnable state is periodically destroyed? Existing memory mechanisms — including elastic weight consolidation, synaptic intelligence, and surprise-driven gating — rely on gradient computation and are inapplicable to systems that do not perform it. We introduce Deep Memory (DM), a backpropagation-free persistent memory mechanism operating through a triple-loop consolidation cycle: (1) recording of expert-specific content centroids, (2) seeding of replaced units with stored representations, and (3) stabilization through continuous re-entry. Discrete expert routing via Mixture-of-Experts (MoE) gating is required, in the regimes tested, to prevent the centroid convergence that would render stored memories identical. We derive a Foster-Lyapunov drift bound for the full triple loop, showing that seeding rescales the turnover noise floor. Across $1{,}007$ simulation runs over thirteen blocks: (i) removing stable context-expert binding removes specialization ($mathrm{MI}=1.10$ vs. $0.001$; $n=91$); (ii) DM achieves $R=0.984$ vs. $0.385$ without memory ($n=16$); (iii) continuous seeding reconstructs representations after interference ($R_mathrm{recon}=0.978$; one-shot fails; $n=30$); (iv) the mechanism operates within a characterized $(K,p)$ envelope ($n=350$); (v) recording $times$ seeding is the minimal critical dyad ($n=40$); (vi) associative and reservoir baselines (Hopfield, ESN) are compared under matched turnover ($n=370$). DM is thus a falsifiable, bounded mechanism for persistent memory in backpropagation-free cognitive systems, with functional parallels to hippocampal consolidation.
Source: Persistent Memory Through Triple-Loop Consolidation Under Stochastic Unit Turnover