acceptodds
Under review as a conference paper at ICLR 2027

A Spike-Driven Hopfield Memory for Online Continual Learning

Abstract

Associative memory is inspired by the brain’s ability to recover stored content from partial cues while incorporating new experiences over time. Hopfield networks provide a computational form of associative recall, yet many variants realise memory through state-based operations rather than spike-mediated neural dynamics. We therefore propose the Spike-driven Hopfield Online Neural Architecture (SHONA), which couples membrane dynamics, timing-dependent plasticity, and spike-driven forgetting control to reformulate Hopfield associative memory as a continual process in which memories are retrieved, updated, and regulated. Experiments show that our approach sustains associative recall while embedding online adaptation directly into the memory process.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Hopfield networks, associative memory, spiking neural networks, online learning, continual learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AjZDOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SHONA, a spike-driven Hopfield associative memory framework for online continual learning. It combines membrane-potential-based retrieval, spike-timing-dependent plasticity for memory updates, and an adaptive forgetting-control mechanism. Experiments on image and event-based datasets evaluate associative retrieval under clean, noisy, and masked inputs, as well as memory preservation during online updates.

Critical strengths

The paper presents a coherent integration of spike-driven retrieval, online plasticity, and forgetting control within an associative memory framework. The evaluation covers multiple datasets and includes comparisons with established associative memory methods.

Critical weaknesses
  1. Limited methodological novelty. The main components—membrane-potential-based competition, STDP-inspired memory updates, and activity-dependent forgetting control—are largely established mechanisms. The paper does not clearly demonstrate a fundamentally new associative memory principle arising from their integration.

  2. Insufficient evidence of distinctive advantages. The empirical improvements are inconsistent across retrieval conditions, with simpler baselines outperforming SHONA in several settings. Moreover, the experiments do not establish clear computational or neuromorphic benefits, such as improved efficiency, spike sparsity, or energy consumption. Consequently, the significance of introducing spike-driven dynamics remains insufficiently demonstrated.

Questions for authors
  1. What specific computational property does SHONA introduce beyond combining existing spiking retrieval, STDP-inspired plasticity, and forgetting-control mechanisms?

  2. Can the authors clarify the advantage of spike-driven retrieval over simpler similarity-based methods, particularly given the weaker performance under Gaussian noise reported in Table 3?

  3. Does the proposed spike-driven formulation provide measurable computational or efficiency benefits compared with non-spiking alternatives?

2 comments
AuthorsOpenReview ↗

Response to Reviewer AjZD — Weaknesses 1–2

Response to Weakness 1

We respectfully disagree with the reviewer's assessment that SHONA lacks methodological novelty. SHONA is not a conventional SNN classifier, but an unsupervised, online Hopfield associative-memory architecture that uses spikes, an approach that is missing from the existing literature (Sections 1–2). SHONA draws upon established neural mechanisms but reformulates them for different computational purposes rather than simply applying them to another SNN classifier.

Each mechanism is specifically designed to perform a distinct task required for associative memory: memory selection (Equations 11–13), winner-conditioned consolidation of continuous-valued memory traces (Equations 14–15), and a newly developed spike-history-driven algorithm to regulate catastrophic forgetting (Equations 16–19). Importantly, catastrophic forgetting is an online learning phenomenon, not an existing algorithm copied into SHONA.

Our comparisons with related architectures, including SHONA's superior retrieval performance over another spike-driven associative-memory architecture, TPAM (Appendix A.7.4), further establish the differences in their intended operation and capabilities.

SHONA's methodological novelty lies in the specific mathematical reformulation of these principles and their application to an online Hopfield associative-memory architecture. The reviewer has not identified an existing method using the same learning and memory-control mechanisms, nor provided a specific comparison demonstrating that SHONA's proposed methodology is already established.

Response to Weakness 2

We respectfully disagree that SHONA's distinctive advantages are insufficiently demonstrated. The reviewer appears to evaluate SHONA primarily through reconstruction accuracy, overlooking its contribution as a unified, online spike-driven Hopfield associative-memory architecture (Sections 1–4). This distinction is also addressed in our responses to Reviewer KuNF (Weakness 1 and Question 1) and Reviewer eP1J (Weakness 1 and Question 2).

We included simpler retrieval methods—KNN, Manhattan L1, and cosine similarity (Appendix A.7.1)—to demonstrate where geometric similarity achieves lower MSE. These methods provide effective one-to-one image matching but are not equivalent to neural associative-memory architectures supporting online synaptic adaptation and forgetting control. By utilising geometric similarity, SQHN achieves more favourable retrieval accuracy. However, substituting such a method for SHONA's Hopfield retrieval solely to improve MSE would compromise its energy formulation and conditional exponential-capacity guarantee (Appendices A.2–A.3). Preserving these foundations is a deliberate design choice: our objective is to develop a generalised spike-driven Hopfield memory architecture, rather than another image-search algorithm.

SHONA outperforms MHN in all 24 larger-capacity grayscale comparisons (Section 5.2) and TPAM, an established spike-based associative-memory model, in 475/492 paired retrievals (Appendix A.7.4). Section 5.3 further demonstrates improved retention through our novel catastrophic-forgetting control, alongside masking robustness.

Regarding energy efficiency, latency, and sparsity, we refer to Reviewer eP1J (Weakness 1 and Question 4). These are not claimed advantages but future research directions (Section 6). The energy formulation in Appendix A.2 concerns mathematical Hopfield energy, not electrical consumption.

AuthorsOpenReview ↗

Answers to Reviewer AjZD — Questions 1–3

Answer to Question 1

As explained in our response to Weakness 1, SHONA's methodological novelty does not require introducing an additional mechanism beyond those identified by the reviewer. It lies in how these mechanisms are mathematically reformulated and integrated for distinct computational purposes within a unified online Hopfield associative-memory architecture (Equations 11–19). In particular, our newly formulated spike-history-driven catastrophic-forgetting control regulates memory modification during continual learning. These contributions are developed in Appendices A.2–A.5, with the forgetting-control mechanism experimentally evaluated in Section 5.3.

Answer to Question 2

As established in our response to Weakness 2, SHONA demonstrates retrieval advantages over MHN and TPAM, including under corrupted cues. Specifically, Table 3a shows that SHONA outperforms MHN at both Gaussian-noise levels, while Appendix A.7.4 demonstrates lower MSE than TPAM. These results show that SHONA's advantages are not limited to masking robustness.

Regarding simpler similarity-based methods, our response to Weakness 2 explains where they achieve lower reconstruction MSE and why substituting geometric retrieval would compromise SHONA's Hopfield foundations (Appendices A.2–A.3). The different effects of Gaussian noise on spike-driven retrieval and geometric similarity are further explained in our response to Reviewer eP1J (Weakness 2).

Answer to Question 3

As clarified in our response to Weakness 2 and Reviewer eP1J (Weakness 1 and Question 4), SHONA does not claim measurable advantages in energy efficiency, latency, or firing sparsity over non-spiking alternatives. These evaluations are explicitly identified as future research directions (Section 6).

SHONA's demonstrated contribution concerns computational capability rather than measured efficiency: binary spike-based representations support Hopfield-style associative retrieval, online memory adaptation, and catastrophic-forgetting control within a unified neural architecture (Sections 3–4). This provides a foundation for extending Hopfield associative memory to neural structures operating through spike-based communication, rather than relying solely on conventional continuous-valued retrieval operations. Experimental support is already detailed in our response to Weakness 2 (Sections 5.2–5.3; Appendix A.7.4).

Reviewer_eP1JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SHONA, which aims to extend traditional Hopfield associative memory into a spiking neural network architecture that supports online continual learning. Specifically, the main contributions of the paper are: 1.Extending Hopfield AM from a static cue-recovery structure to an online memory process that evolves as new inputs arrive. 2.Achieving adaptive memory storage through spike-mediated activity and time-dependent plasticity, thereby directly modifying selected memory traces. 3.Introducing a control mechanism based on spike history that protects stored traces from destructive modification, thereby mitigating functional alteration during continual updating.

Critical strengths

Closed-loop positioning + a design orientation of "using time windows for forgetting control." Retrieval, updating, and forgetting control are placed within the same spiking dynamics, and forgetting control truncates the plasticity window rather than adding an external penalty term or replay buffer, incurring no additional storage. This is precisely where the paper genuinely distinguishes itself from SQHN (online, non-spiking) and HIAM (spiking, without protecting old traces), and it also allows "Hopfield-style retrieval" to be reformulated under non-attractor dynamics. At the same time, it exhibits strong robustness to locally missing cues (masking): experimental results show that under 50% masking, SHONA's reconstruction MSE is markedly better than MHN, SQHN, and traditional KNN/Cosine methods, demonstrating the unique advantage of the spike accumulation mechanism in handling incomplete cues.

Critical weaknesses

1.The practical benefit is rather narrow: under clean cues, cosine/SQHN are often stronger, and SHONA mainly wins on masking robustness; however, whether the complexity brought by the spiking mechanism is justified by this limited benefit, and the energy, latency, and sparsity advantages repeatedly implied in the abstract/method are dismissed with a single sentence stating "not directly measured"—this question is not directly addressed.

2.Retrieval under Gaussian noise degrades significantly, and the paper provides no mechanism for this. In contrast to the advantage under masking, under σ=0.5 noise SHONA falls substantially behind on grayscale large-capacity settings. The paper's physical/biological explanation for this phenomenon is somewhat insufficient, lacking in-depth analysis of why spike membrane potential accumulation is relatively sensitive to continuous Gaussian perturbations.

3.The algorithm's dependence on short time windows and hyperparameter sensitivity: sensitivity analysis (Table 3b) shows that when the time window T increases from 300 to 350 and 400, reconstruction MSE and SSIM exhibit noticeable performance degradation. This runs counter to the conventional intuition in spiking neural networks that "the more fully time evolves, the richer the accumulated information." The authors fail to provide a satisfactory mechanistic explanation in the main text.

Questions for authors

1.Do the "expected margin grows linearly" and "dimensions are independent" assumptions in Theorem 3 actually hold for the periodic rate coding and image data you use? Can you provide empirical curves of margin versus d, or explain how the capacity conclusion degrades when the assumptions are not satisfied?

2.Under both clean and Gaussian noise conditions, SHONA is inferior to cosine/SQHN. Is the advantage under masking mainly due to the winner selection naturally ignoring missing dimensions? If an explicit cosine baseline that handles missing masks were used, would the gap be eliminated?

3.Regarding the mechanism by which increasing the time window T leads to performance degradation: experiments show that T=300 performs significantly better than T=400. What is the root cause of this degradation? In actual online continual learning streams, how can the optimal baseline T be adaptively determined?

4.Energy/latency/sparsity are all implied as neuromorphic value in the abstract and method, but the main text states they were not directly measured. At least reporting spike counts or end-to-end runtime would be more convincing; otherwise these claims can only be regarded as expectations.

2 comments
AuthorsOpenReview ↗

Response to Reviewer eP1J — Weaknesses 1–3

Response to Weakness 1

We respectfully disagree that SHONA's benefits are limited to masking robustness. It is an unsupervised, online Hopfield associative-memory architecture grounded in Modern Hopfield Networks (MHNs), not a conventional SNN classifier. It integrates spike-driven retrieval, synaptic adaptation and catastrophic-forgetting (CF) control as stored traces evolve (Sections 1–4).

Although cosine and SQHN can yield lower clean-cue MSE using geometric similarity, SHONA retains Hopfield-style retrieval rather than substituting those methods solely to minimise MSE. It outperforms MHN in larger-capacity grayscale clean, noisy and masked comparisons (Section 5.2), and achieves lower MSE than TPAM, an established spiking associative-memory model, in 475/492 paired retrievals (Appendix A.7.4). Its novel spike-driven CF control also improves retention and online retrieval during continual updates (Section 5.3, Appendix A.7.2).

We disagree that the abstract or methodology implies energy-efficiency, latency or spike-sparsity benefits; neither makes such claims. The paper's energy refers to mathematical Hopfield energy, not physical consumption. Traditional Hopfield retrieval is characterised by energy minimisation toward stored memories. SHONA preserves a conditional, event-specific energy-descent interpretation (Appendix A.2).

Regarding sparsity, SHONA's neurons correspond to stored memory traces, not hidden-layer neurons of a conventional SNN. For example, 300 neurons correspond to 300 memory traces, each associated with a competing neuronal response that determines retrieval and subsequent synaptic updates (Section 4). This explains their role, not a claim of sparse firing.

Section 6 identifies energy efficiency, latency and firing sparsity as future evaluation objectives, and deeper or hierarchical memory architectures as a separate direction. As MHN mechanisms have been incorporated into Transformers, SHONA might eventually extend to more complex neural structures. These remain future directions, not results claimed in this paper.

Response to Weakness 2

Under Gaussian noise, SQHN and cosine can achieve lower MSE using geometric similarity; KNN and L1 can perform even better (Appendix A.7.1). As explained in Weakness 1, we retain Hopfield-style retrieval rather than substitute geometric similarity solely to minimise MSE, preserving its energy formulation and conditional exponential-capacity guarantee (Appendices A.2–A.3).

Masking and Gaussian noise affect SHONA differently. Masked pixels produce no spikes, while remaining pixels retain their original intensities (Appendix A.1, Equation 21). These surviving inputs can generate discriminative membrane responses among competing memory-trace neurons sufficient to select a winner (Equations 11–13), explaining SHONA's masking robustness.

Gaussian noise instead perturbs pixel intensities without removing components. Under periodic rate coding (Equations 20–21), this alters spike frequencies and timing, potentially changing membrane accumulation and competition between traces. Cosine and SQHN can do better because noisy images may retain geometric similarity. Their lower MSE does not establish that membrane accumulation is intrinsically more sensitive to Gaussian noise; retrieval mechanisms respond differently to corrupted cues.

Response to Weakness 3

Degradation with increasing TT is consistent with SHONA's online associative-memory mechanism. FFW selects the earliest winning trace (Appendix A.6), so subsequent spikes do not change this selection. Under rate coding (Equation 21), pixel intensity sets firing frequency; extending TT permits more spikes at the same inter-spike interval without necessarily adding useful input information.

However, additional spikes can affect learning: winner-conditioned plasticity integrates cues into selected traces (Equations 14–15). A longer TT permits more updates, potentially over-modifying traces and distorting shape, contrast or image integrity. During continual learning, these effects may accumulate and interfere with stored information.

Conversely, T=300T=300 reduces repeated updates and may preserve general image structure while retaining fewer fine details, potentially explaining its lower MSE. Lower error does not necessarily mean better preservation of fine details.

We adopted T=350T=350 as a practical horizon, consistent with prior spiking literature (Appendix A.8.2), to preserve and continually adapt associative representations (Sections 1–2, 4), not merely minimise MSE. The optimal initial TT therefore depends on the learning objective.

AuthorsOpenReview ↗

Answers to Reviewer eP1J — Questions 1–4

Answer to Question 1

Theorem 3 establishes a conditional exponential-capacity guarantee for SHONA's winner selection under independent input coordinates and a linearly growing expected margin. As in MHN, this mathematical guarantee relies on sufficient assumptions.

Periodic rate coding does not inherently imply statistical dependence. Equation 20 calculates firing rates using shared image-wide minimum and maximum intensities, which can introduce dependence between spike coordinates, while Equation 21 determines their firing times. Thus, independence is not guaranteed, but neither is dependence inevitable.

Importantly, dependent inputs do not necessarily eliminate exponential capacity. We can extend Appendix A.3 using the dependent-variable concentration inequality of Kontorovich and Ramanan (2008). Let mdm_d denote the expected-margin lower bound, LL the bounded coordinate sensitivity, and KdK_d the dependence coefficient. Under the corresponding assumptions,

P(error)≤2Nexp⁡(−md22dL2Kd2).P(\mathrm{error})\leq 2N\exp\left(-\frac{m_d^2}{2dL^2K_d^2}\right).

For md≥αdm_d\geq\alpha d and Kd≤KK_d\leq K, with dimension-independent constants,

P(error)≤2Nexp⁡(−α2d2L2K2).P(\mathrm{error})\leq 2N\exp\left(-\frac{\alpha^2d}{2L^2K^2}\right).

Thus, bounded dependence preserves exponential capacity with a modified exponent; stronger dependence or slower margin growth weakens the guarantee.

Our objective, as in MHN, is a conditional mathematical capacity guarantee rather than empirical capacity scaling. Our experiments instead evaluate online retrieval, reconstruction, and memory preservation at fixed capacities. We will include this conditional extension and its assumptions in Appendix A.3 in our final version.

Answer to Question 2

As explained in Weaknesses 1–2, reconstruction MSE alone does not establish architectural superiority, and SHONA's masking robustness arises from spike-driven membrane accumulation and competition (Equations 11–13).

Incorporating geometric similarity could improve reconstruction accuracy, but replacing SHONA's Hopfield retrieval mechanism would compromise its energy formulation and conditional exponential-capacity guarantee (Appendices A.2–A.3). We deliberately preserve these foundations rather than minimise MSE alone.

We cannot comment on the performance of cosine baselines outside our core method. Developing such a method is beyond our objective. Our masking experiments evaluate SHONA against the tested baselines, with MHN, whose Hopfield framework SHONA extends through spiking dynamics, remaining our primary architectural comparator.

Answer to Question 3

Addressed in our response to Weakness 3

Answer to Question 4

As explained in our response to Weakness 1, we disagree with the reviewer's assertion that energy efficiency, latency, and sparsity advantages are implied in our manuscript. We make no such claims in either the abstract or methodology. These properties are explicitly identified as future research directions in Section 6.

Regarding the request for spike counts, our First-Firing Winner (FFW) algorithm (Appendix A.6) determines retrieval from the earliest winner-firing event. Subsequent spikes do not change the selected trace, so total spike count is not a measure of retrieval accuracy.

The relevance of these measurements should therefore be distinguished from SHONA's demonstrated associative-memory contributions.

Reviewer_KuNFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

Hopfield networks, their storage capacities, their explanations of how memory works have been put forward a long time with traditional implementations not discussing clearly what happens to memories over time. This paper takes a spiking neural network approach to explain how memory gets updated over time and how potentially controlled forgetting can happen. The basic approach seems to represent the stored objects as vectors of weights along their composed dimensions. For example, an image would be broken into its pixels and say their intensity values could reflect the weight. During retrieval, a partially specified image as a query would be converted into a spiking trace along those image pixels where it is known. Then doing a dot product per stored image within the composed dimensions and adding up essentially performs a correlation within a single vector. The vector with maximum correlation is then chosen as the winning retrieved object. Thus unlike modern Hopfield networks that are weighing across the vectors of different images this approach seems to be weighing within the composed dimensions of a single vector at a time and selecting a winner.

Specifically, it proposes an online associative memory in which stored pattern vectors compete through spike-driven membrane responses. An input image is converted into periodic spike trains, and each stored trace accumulates a dot-product response to those spikes, subject to leakage, firing thresholds, and inhibition. The default readout returns the first winning trace. Winning traces are modified through timing-dependent plasticity, while an adaptive processing horizon and dominance-based resets regulate further modification. The principal contribution is therefore the integration of temporal similarity-based competition, online updating, and forgetting controls, rather than partial-cue retrieval itself.

Critical strengths

The paper takes a refreshing approach to Hopfield style retrieval which may be interesting to present to the audience provided other concerns below are addressed.

The problem addressed of maintaining an adaptive memory while limiting destructive updates is important. Its experiments cover several datasets, memory sizes, corruption conditions, temporal readouts, and forgetting-control ablations. The results provide evidence of robustness to masked cues and indicate that restricting update exposure can improve retention of early memories. The timing-dependent competition and plasticity analysis is interesting. Equations and pseudo-code and parameter settings help expose their operation.

Critical weaknesses

The core matching computation is a dot product between each stored trace and the current query spikes, accumulated over time. Although spiking dynamics modify this competition, the experiments do not sufficiently isolate their benefit over direct correlation-based retrieval with equivalent updating and forgetting controls. Correlation based image selection is notoriously sensitive to image-specific variations, such as rotations and other transformations that don't alter the semantic content but could have component value variations.

The paper also doesn't mention how much of the stored object must be visible in the query for this approach to work, nor the effect of inherent similarity in the stored patterns that usually trip up modern Hopfield networks into their metastable states. The masking experiments do not establish how much discriminative content must remain visible or how performance changes with similar distractors. Evaluation primarily uses reconstruction MSE and SSIM, which do not directly establish whether the correct stored image was retrieved. Moreover, the tested memory sizes and conditional capacity analysis do not demonstrate that SHONA resolves large-bank interference: hard selection avoids a blended readout but can still select—and update—the wrong trace.

Finally, a discussion on improved retention under a shorter processing horizon must be balanced against the ability to acquire new memories; preserving old traces alone does not establish effective continual learning.

Questions for authors

What benefit do spike timing, thresholds, and inhibition provide over direct dot-product or normalized-correlation retrieval when the memory bank, plasticity budget, and forgetting controls are matched?

How does retrieval change with visible-image fraction, foreground versus background masking, geometric transformations, and distractors sharing substantial background content?

When an incorrect trace wins, how often does its subsequent update corrupt an existing memory? Are evaluation reconstructions measured before or after cue-driven plasticity?

2 comments
AuthorsOpenReview ↗

Response to Reviewer KuNF — Weaknesses 1–3

Response to weakness 1:

We respectfully disagree that these observations establish a weakness in SHONA's contribution. The criticism judges SHONA largely as an image-matching algorithm, rather than a unified, online, spike-driven Hopfield architecture.

The dot product is deliberately retained as a fundamental Hopfield computation. SHONA extends it through binary spike communication, integrating retrieval, synaptic adaptation, and forgetting control as stored traces evolve with new inputs (Sections 1–2, 4). These are interconnected neural operations, not independent additions to static image matching.

SHONA outperforms MHN in all 24 large-capacity grayscale MSE comparisons (Section 5.2). Its forgetting-control mechanisms are independently evaluated and improve retention (Section 5.3, Appendix A.7.2). Against TPAM, an established spiking associative-memory model, SHONA achieves lower MSE in 475/492 paired retrievals (Appendix A.7.4).

Geometric transformations test recognition of the same identity despite changes in image representation. Transformation invariance is not a claimed contribution or a prerequisite to demonstrating SHONA's integrated online memory. Generalising across changing representations is a wider challenge in online learning, not one unique to SHONA. Our results instead establish a mathematically supported architecture (Appendices A.2–A.3), validated experimentally for retrieval, memory adaptation, and forgetting control, rather than an image matcher optimised solely for one-to-one accuracy.

Response to weakness 2:

The reviewer partly applies a static image-matching criterion to SHONA, although its online memories evolve without fixed one-to-one mappings between inputs and stored traces.

Section 5.5 measures reconstruction as cues become less complete. MSE increases from 30% to 50% masking, but at 50% SHONA achieves the lowest average MSE and highest average SSIM against MHN, SQHN, KNN, and cosine similarity under matched conditions. These results show robustness to incomplete cues, not a universal minimum visible fraction.

Each qualifying winner event updates a trace, and different traces may win within one input (Section 4). A previously less-similar trace can become a better representation after learning; thus, comparing its identity with a fixed original image index is not sufficient to establish retrieval correctness. MSE and SSIM assess reconstruction quality; retention measures the preservation of learned information. Similar-distractor experiments would extend, not replace, these evaluations.

Large-bank interference is addressed both theoretically and experimentally. Theorem 3 gives a conditional bound P(error)≤Ne−γdP(\mathrm{error})\leq Ne^{-\gamma d} for instantaneous winner selection at exponential capacities (Appendix A.3). Theorem 5 bounds repeated dominant misassignments and trace corruption before reset (Appendix A.5). Across seven datasets, clean and masked MSE improve from the smallest to largest tested memory capacity. The MHN comparisons and Full-CF retention results provide further evidence (Sections 5.2–5.3; see Weaknesses 1 and 3). These results support robustness and interference mitigation without claiming to eliminate every misassignment.

Response to weakness 3:

Preserving memories and acquiring new information are not mutually exclusive. EWC (Kirkpatrick et al., 2017), SI (Zenke et al., 2017), probabilistic metaplasticity (2024), and AGMP (Dong and He, 2026) show that regulated plasticity can preserve knowledge while permitting continued learning.

SHONA's new spike-driven forgetting control leaves the input spike-train duration TT fixed. Its adaptive horizon TnewT_{\mathrm{new}} limits the remaining STDP update window based on winner activity; dominance pruning resets persistently dominant traces to prevent excessive updates. Similar cues can reinforce existing memories, while different traces can be updated during one input, incorporating new information (Section 4, Equations 14–19).

Section 5.3 directly tests whether this regulation merely preserves old traces: against No-CF, Full-CF improves retention in 50/63 settings, online retrieval MSE in 51/63, and SSIM in 53/63. These results show benefits to both preservation and ongoing retrieval. This independently evaluated forgetting-control algorithm is a contribution to continual associative learning, not a mechanism that simply prevents new learning.

AuthorsOpenReview ↗

Answers to Reviewer KuNF — Questions 1–3

Answer to Question 1:

The primary benefit of SHONA is enabling Hopfield associative memory to operate through binary spike-train communication within a unified, biologically inspired online neural architecture (Sections 1–2, 4). This extends modern Hopfield networks to support applications involving binary spike-based information.

The dot product is deliberately retained as the underlying synaptic computation. Spike timing, adaptive thresholds, and inhibition enable this computation through temporal neuronal responses, similar to biological neural communication. Firing events determine memory selection, drive synaptic adaptation, and provide the activity history required for catastrophic-forgetting control (Section 4, Equations 11–19).

Importantly, matching the memory bank, plasticity budget, and forgetting controls does not necessarily reproduce the same learning process. Spike timing and neuronal competition determine which traces are modified and how frequently they are selected, even within a single input's processing horizon.

SHONA's mathematical foundations (Appendices A.2–A.3) and experimental advantages over MHN and TPAM (see our response to Weakness 1) support this new architectural capability, rather than claiming that spiking dynamics guarantee superior retrieval accuracy under every comparison.

Answer to question 2:

SHONA does not explicitly distinguish foreground from background or impose semantic rules about which image regions should be preserved. Incoming information is encoded into binary spike trains, and associative retrieval is determined by the resulting neuronal responses and competition (Section 4). Consequently, sensitivity to particular image regions depends on the information contained in the input and the learned memory representations.

Section 5.5 evaluates retrieval under 30% and 50% masking. Reconstruction error increases as the masking fraction increases, yet SHONA achieves the lowest average MSE and highest average SSIM at 50% masking among MHN, SQHN, KNN, and cosine similarity (see our response to Weakness 2). These results demonstrate robustness to substantially incomplete cues, without establishing a universal minimum visible fraction.

Foreground-specific masking, geometric transformations, and distractors sharing substantial background content were not separately evaluated. As discussed in our response to Weakness 1, transformation invariance is not a claimed contribution. Such targeted tests would extend the evaluation of image-specific robustness rather than determine the validity of SHONA's architectural contribution.

Answer to Question 3:

An incorrect memory update cannot necessarily be identified by comparing the winning trace with a predetermined original image. SHONA follows Hopfield-style associative retrieval, not static one-to-one image matching. With finite memory capacity, new information is incorporated into evolving memory representations rather than preserving every input unchanged. At each qualifying spike event, the winning trace is updated, and different traces may win during a single input (Equations 11–15). Selecting a different trace therefore does not inherently constitute corruption (see our response to Weakness 2).

The relevant risk is excessive attractor dominance: one trace repeatedly attracts unrelated inputs, accumulates destructive updates, and interferes with previously learned information. SHONA limits this through adaptive-horizon control, which regulates the remaining plasticity window according to winner activity, and dominance pruning, which considers repeated selection and accumulated usage (Equations 16–19). Appendix A.5 establishes bounds on off-target update exposure, while Section 5.3 demonstrates improved memory retention (see our response to Weakness 3). Rather than counting mismatches against fixed image identities, we evaluate interference through these theoretical and experimental measures.

Regarding evaluation timing, training reconstructions are measured after cue-driven learning. Held-out test reconstructions use the evolving memory without learning from test cues. This tests whether previously learned associative representations can reconstruct unseen inputs, rather than retrieve identical images from unchanged memory slots, thereby assessing generalisation through online learning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.