acceptodds
Under review as a conference paper at ICLR 2027

Symmetry-Restoring Rectified Update Rules for Hopfield Neural Networks: Dynamics, Theorems, and Hamming-Distance Awareness

Abstract

Hopfield neural networks (HNNs) provide a classical framework for associative memory, but the binary dynamics are not fully specified when a neuron's local field is zero. Common fixed-sign tie-breaking conventions can introduce an artificial sign preference and break the global sign symmetry inherent in the energy. We show that retaining the neuron's current state at zero field restores this symmetry and yields a principled Rectified Update Rule. This symmetry restoration exposes a Hamming-distance geometry underlying the resulting dynamics. For a single stored message, we obtain an exact characterization of convergence and symmetric two-cycles. For two stored messages, we formulate a Hamming-Distance-Aware Rectified representation and characterize the complete set of dynamical regimes, including convergence, self-cycles, hetero-cycles, and symmetric-cycles. These results further provide exact state counts and convergence-domain characterizations as functions of the network size and inter-memory Hamming distance. Exhaustive enumeration and Monte Carlo experiments agree with the theoretical predictions across a range of network sizes and memory separations.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Hopfield neural networks, rectified update rule, Hamming distance, convergence, limit cycles

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5fLLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyzes synchronous binary Hopfield dynamics with state preservation at zero field. It derives Hamming-distance formulas for fixed points, cycles, basins and correction distances. It focuses mainly on one or two patterns and checks them through enumeration and Monte Carlo experiments

Critical strengths

The classification gives a clear account of fixed points, cycles and the role of parity at low load. Checks of the supplied code agree with direct updates in 16388 cases and with independent counts for 780 parameter pairs. This is good support for the core mechanism

Critical weaknesses
  1. State preservation at zero field is already explicit in Cheung (1987), Section IV (https://robertmarks.org/REPRINTS/1987_SynchronousVsAsynchonousBehavior.pdf). The remaining question is what the classification teaches us beyond the two-block reduction. The paper needs a stronger answer to it. This is not a claim that every formula has already appeared elsewhere.
  2. The experimental record is hard to reconcile. Tables 1(b) and 5 disagree, some ZZ signs do not match the stated formula, and the scripts label 51 repetitions as 50. More seriously, syn_update.py can produce zero instead of retaining the binary state; HDAR does not have this bug. Corollary 6 also fails at N=4,  r12=2N = 4,\; r_{12} = 2, and failure to recover the target is not necessarily a cycle. The effect of these problems on the reported results needs to be explained.
Questions for authors
  1. The known zero-field rule and the two-block reduction are accounted for. So what is the main new conclusion?
  2. Which code and run records produced the tables? Please explain the discrepancies, where sampling or full enumeration was used? Whether any reported result used syn_update.py.
  3. How would you revise Corollary 6 and the statements about recovery versus cycling? For example, at N=6,  r12=3N = 6,\; r_{12} = 3, all states converge to fixed points
3 comments

Dear Reviewer,

Thank you for the careful assessment. We address the two weaknesses and three questions separately. Our main result is the global sign symmetry of the synchronous dynamics and its consequences for the complete finite-size attractor landscape; the accompanying numerical revisions will make the experimental evidence traceable to a consistent protocol.

W1. Novel dynamical conclusions beyond the known tie convention

Among the classical binary Hopfield studies cited in our manuscript, the main questions are storage capacity, convergence, stable-state counts, and synchronous cycles. These works do not generally make zero-field state retention an explicit rule of study. Where a deterministic tie choice is specified, fixed-sign choices also appear: Abu-Mostafa and St-Jacques (1985) assign +1+1 at threshold equality, whereas Koml'os and Paturi (1988) assign −1-1. Thus, Cheung et al. provide an important explicit precedent, while state retention is not the standard explicitly analyzed tie convention across the classical studies cited here.

Cheung et al. (1987) explicitly discuss the zero-input problem in Section IV. For the synchronous case they consider, if neurons with nonzero input retain their states and the zero-input neurons also stay unchanged, the one-step energy change is zero. They then analyze complementation and state-hardening schemes for zero- and nonzero-autoconnection models, comparing their energy transitions, convergence, and oscillatory behavior. This is an important energy- and convergence-based treatment of zero-input handling. It does not, however, derive global sign equivariance or an exact classification of attractors, cycles, and convergence basins under the state-retaining rule.

Our contribution builds on this known rule by establishing its full dynamical consequences. For synchronous binary Hopfield networks with symmetric, zero-diagonal Hebbian weights, we prove that the rectified update map satisfies F(−s)=−F(s)F(-s)=-F(s), whereas a fixed-sign tie choice does not preserve this relation when zero fields occur. Consequently, every trajectory, fixed point, periodic orbit, and basin is paired with its globally sign-reversed counterpart.

For one and two stored patterns, HDAR expresses the dynamics in joint Hamming-distance coordinates and gives exact fixed-point and cycle conditions, parity-dependent counts of states entering cycles, and convergence-domain bounds. The two-block majority decomposition provides a useful structural reduction for two memories, but the reduction alone does not state the global sign-symmetry consequence or give the exact finite-size attractor and basin classification at tie-sensitive and parity-dependent boundaries. Our contribution is to derive these dynamical conclusions and make clear how the known update convention organizes the complete finite-size landscape. We will revise the related-work and contribution statements accordingly.

W2. Experimental relevance and reporting consistency

Appendix B.5 measures how often trajectories encounter zero fields; this occurrence rate is distinct from the fraction of outcomes changed by a tie rule. We therefore add paired comparisons under identical memory sets and initial states, reported in Q2. We also distinguish cycles that occur when zero fields are impossible: those cycles are properties of synchronous dynamics and are not attributed to the tie convention.

We agree that Table 1(b) and Appendix Table 5 should report results from the same experimental runs. During preparation of successive manuscript versions, we inadvertently populated them from different runs. This was an oversight in result collation and version checking; we apologize. In the supplied two-message driver, test_number is set to 51 and the loop therefore runs 51 trials, while the save routine records test_number-1, or 50. The reported sample statistics consequently use 51 trials even though the recorded count says 50. We will use a single 50-trial setting consistently for sampling, saved metadata, and statistical calculations. For the overlapping Monte Carlo rows, we will regenerate one shared set of run-level outputs and use it in both tables, then update the estimates, standard errors, and ZZ statistics. We will calculate Z=(xˉ−xth)/SEZ=(\bar{x}-x_{\mathrm{th}})/\mathrm{SE} from unrounded values; at the displayed precision, Table 5's ZZ should be negative at N=199N=199 (0.0557<0.05690.0557<0.0569) and positive at N=499N=499 (0.0359>0.03580.0359>0.0358). We will update these values from the regenerated data.

Q1. Main new conclusion

The principal conclusion is that a symmetry-consistent treatment of zero fields restores global sign equivariance of the synchronous transition map, which in turn pairs the complete attractor landscape, not only the stored memories. For one and two stored patterns, we further derive exact Hamming-distance criteria for fixed points and cycles, parity-dependent counts of initial states entering cycles, and bounds on convergence domains. These results turn the update convention into a precise description of which states converge, which cycle, and how the outcomes are organized by memory distance. The symmetry theorem extends to symmetric zero-bias binary networks, while the explicit HDAR counts are stated for the memory regimes analyzed here.

Q2. Code and run provenance

The supplied package contains two distinct distance-based paths: memmory_two_messages_test.py performs Monte Carlo sampling, while HDAR_test.py enumerates all 2N2^N initial states. Neither calls syn_update.py. Their entry points illustrate sampling at N=9N=9, r12=3r_{12}=3 and exhaustive enumeration at N=6N=6, r12=2r_{12}=2. The manuscript protocol is exhaustive enumeration for N≤20N\leq20 and, for N>20N>20, 50 Monte Carlo trials with 10,000 initial states per trial. The entry points do not by themselves document the origin of every historical table row; we will provide a row-level manifest linking each entry to its generating script and run record, identify whether it was enumerated or sampled, and regenerate any row that cannot be matched to the stated protocol. Tables 1(b) and 5 will be traced to the same run-level records; the trial count and random seed will be consistent, and the corresponding standard errors will be recomputed from those records.

Q3. Correcting the radius and attractor statements

We will clarify that rmax⁡r_{\max} is a target-specific bound: it limits the Hamming distances from which a state can converge to a specified memory (or its sign-reversed counterpart). It does not classify the global attractor of every state outside that target's convergence domain. Likewise, rccd<r≤rmax⁡r_{\mathrm{ccd}}<r\leq r_{\max} does not necessarily contain both convergent and cyclic trajectories. For N=6N=6 and r12=3r_{12}=3, Corollary 4 gives zero cycles because NN is even and r12r_{12} is odd; the exhaustive outcome atlas in Fig. 14 confirms that all 64 initial states converge to four fixed points, with 16 states in each basin. We will revise Section 4.4 so that cycling is asserted only when the cycle conditions in Theorem 3 and Corollary 4 are met, and distinguish failure to retrieve a particular memory from entry into a cycle.

We will revise Corollary 6 to state the smallest network size for which the strict inequality is guaranteed uniformly over all admissible memory distances. This minimum is Nmin⁡=5N_{\min}=5: for every 1≤r12≤N−11\leq r_{12}\leq N-1, rccd<rmax⁡r_{\mathrm{ccd}}<r_{\max} when N≥5N\geq5. The piecewise formula for rmax⁡r_{\max} remains unchanged. At smaller sizes, equality rccd=rmax⁡=0r_{\mathrm{ccd}}=r_{\max}=0 occurs for (N,r12)=(2,1),(3,1),(3,2),(4,2)(N,r_{12})=(2,1),(3,1),(3,2),(4,2); for N=4N=4 and r12∈{1,3}r_{12}\in\{1,3\} the inequality remains strict. We will state these small-size cases separately, avoiding an unconditional strict claim in the degenerate cases.

I would like to thank the authors for their efforts and responses. Although my comments have been partially addressed, I will leave my current score

Reviewer_2fNeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies synchronous binary Hopfield networks with a rule that retains a neuron’s current state when its local field is zero. For one or two stored memories, it characterizes convergence and cyclic behavior through Hamming distances. It also derives state counts and convergence-domain bounds, and compares theoretical predictions with exhaustive enumeration and Monte Carlo simulations.

Critical strengths
  • The paper addresses a meaningful issue in its setting. The zero-field convention affects the dynamics and the preservation of global sign symmetry.

  • The Hamming-distance formulation provides explicit, checkable predictions for convergence and cyclic states.

  • Appendix B.5 quantifies zero-field occurrence across network sizes and memory counts. It helps explain why zero-field handling deserves attention.

Critical weaknesses

The paper does not sufficiently justify the significance of its restricted scope.

The main analysis is limited to synchronous networks with Hebbian weights constructed from ±1\pm 1 patterns and binary states. It covers at most two stored memories. Although the authors acknowledge these limitations, they do not sufficiently explain why this setting is important to current associative memory research.

The significance of this setting is limited for two reasons. First, the zero-field frequency emphasized in the paper depends strongly on the structure of integer-valued weights and binary states. Ties are also possible with real-valued weights, but whether the same issue occurs frequently enough to matter in practice is not evaluated. In continuous-state, softmax-based Modern Hopfield networks, tied attention scores still yield a defined next state. The same sign-selection ambiguity therefore does not arise [1]. Meanwhile, binary dense associative memory models, where tie handling may remain relevant [2, 3], are outside the scope of the analysis.

Second, a restricted setting should yield clear, nontrivial insights. However, for two distinct, non-antipodal memories, the dynamics reduce to two independent majority blocks. They are explained by convergence to the unique nearest attractor and two-cycles when the nearest attractor is tied. It is unclear what additional understanding the extensive case analysis provides beyond this simple structure. The paper also does not explain what insights the proposed analysis offers for other models.

Appendix B.5 shows that trajectories frequently encounter zero fields in the classical setting. However, this measures the frequency of ties rather than how much their handling changes the final outcome. For example, in the regime M=αNM=\alpha N, the authors should compare the rectified and fixed-sign rules using identical memory sets and initial states. Relevant measures include the fraction of initial states leading to different final attractors and differences in retrieval accuracy. Such comparisons are needed to assess the practical significance of the proposed rule and its added value over existing work.

[1] Ramsauer, Hubert, et al. "Hopfield networks is all you need." arXiv preprint arXiv:2008.02217 (2020).

[2] Krotov, Dmitry, and John J. Hopfield. "Dense associative memory for pattern recognition." Advances in neural information processing systems 29 (2016).

[3] Demircigil, Mete, et al. "On a model of associative memory with huge storage capacity." Journal of Statistical Physics 168.2 (2017): 288-299.

Questions for authors
  • Why is the classical setting important to current associative memory research, and what nontrivial insight does the analysis provide beyond the two-block majority structure?

  • Beyond zero-field frequency, how does the rectified rule affect final attractors and retrieval accuracy compared with fixed-sign tie handling under identical memory sets and initial states, particularly when M=αNM=\alpha N?

4 comments

Dear Reviewer,

Thank you for your careful assessment. We address the three weaknesses and two questions separately. Our results establish the symmetry consequences of zero-field handling and provide exact finite-size attractor classifications, with paired experiments measuring how alternative tie rules affect outcomes.

W1. Significance and scope of the classical setting

The reviewer is right that our theorem-level analysis is restricted to synchronous binary Hebbian networks with at most two stored memories. We use this classical model as a controlled starting point for studying zero-field handling: binary states and discrete local fields make a zero-field event exact and expose a concrete choice in the transition rule, namely whether to retain the current bit or impose a fixed sign. With symmetric Hebbian weights and a fixed synchronous schedule, we can isolate this choice and derive finite-size consequences that are directly checkable by enumeration. The setting therefore provides a precise baseline for discrete associative-memory dynamics; it is not intended to represent every modern architecture. We will make clear that zero-field frequencies and quantitative effects are model-dependent and do not transfer automatically to other state spaces or update rules.

W2. Dynamical results beyond the two-block majority reduction

The agreement/disagreement partition indeed reduces the two-memory dynamics to two independent majority blocks. We use this factorization as a tractable structural starting point, not as the full contribution. HDAR tracks the joint Hamming distances to the two memories and how the relevant coordinate blocks change after each update. It yields explicit conditions for fixed points and for self-cycles, hetero-cycles, and symmetric cycles; exact parity-dependent counts of initial states entering cycles; and convergence-domain bounds. Together with the global sign-symmetry result, these are checkable finite-size descriptions of which initial states converge, which enter cycles, and how the corresponding regions and basins are organized. That trajectory-level classification is additional information not supplied by the statement that the two majority blocks evolve independently.

W3. Zero-field frequency, empirical consequences, and relation to cited models

Appendix B.5 measures how often trajectories encounter zero fields. We agree that occurrence frequency alone does not quantify the dynamical consequence. The paired M=αNM=\alpha N experiments in Q2 address that consequence by comparing the same memory sets and initial states under both rules. In the representative proportional-load settings in Table 1, the fraction of paired initial states ending in different attractor sets ranges from 18.1% to 42.3%; at α=0.10\alpha=0.10 it increases from 18.1% at N=40N=40 to 34.0% at N=160N=160, and at α=0.25\alpha=0.25 it ranges from 31.7% to 42.3%. Table 3 gives an exact higher-memory comparison: for M=5M=5, the attractor differs for 40.7% of initial states at N=11N=11 and 44.5% at N=13N=13, while at N=12N=12 the local fields cannot be zero and the two rules give identical trajectories. Table 2 shows that retrieval effects are also regime-dependent: at 10% corruption and α=0.25\alpha=0.25, rectification improves retrieval by 4.63 and 0.94 percentage points at N=40N=40 and N=80N=80, respectively, but gives no measurable change at N=160N=160. The low-load comparison in Q2 makes the distinction especially clear: at N=40,M=2N=40,M=2 (α=0.05\alpha=0.05), 33.5% of trajectories encounter a zero field, but only 0.10% end in different attractors. Thus, the added experiments separate how often ties occur from how often they change an outcome.

The three cited works concern related but distinct models. Ramsauer et al. [1] formulate Modern Hopfield updates with continuous states and softmax-weighted attention; the binary sign-selection ambiguity studied here therefore does not transfer directly to that update. Krotov and Hopfield [2] and Demircigil et al. [3] study dense, higher-order associative memories with binary spins and sign-based dynamics. \textbf{Because these models also use binary states, the zero-field question and the symmetry-based perspective are relevant and could in principle be extended to binary dense associative memories.} Such an extension would require a separate analysis of their higher-order interactions and asynchronous update rules; our exact theorems for pairwise, synchronous Hebbian dynamics do not automatically apply to them. The M=5M=5 enumeration in Table 3 provides empirical evidence that tie handling can matter beyond the two-memory regime in a binary network.

Our paper does not study or claim an improvement in storage capacity. Its contribution is dynamical: under the stated binary synchronous rule, retaining the current bit at zero field makes the transition map globally sign-equivariant, so convergent trajectories and cycles have sign-reversed counterparts. The theorem establishes this symmetry, and the exhaustive state classifications in Q2 are consistent with its predicted dynamics; the paired experiments measure attractor and retrieval changes, not capacity. We may have interpreted the requested comparison too broadly. If the reviewer has a particular binary dense-memory variant or capacity metric in mind, we would welcome that clarification so that we can address it precisely.

Q1. Why this setting matters and what it reveals beyond block majorities

The setting matters because it makes the zero-field ambiguity explicit while retaining enough structure for exact analysis. Beyond the block-majority reduction, the rectified update map satisfies F(−V)=−F(V)F(-V)=-F(V), pairing every trajectory, fixed point, cycle, and basin with its global sign-reversed counterpart. HDAR further resolves the two-memory transition landscape by (r1,r2,r12)(r_1,r_2,r_{12}): Theorem 3 gives conditions for three distinct cycle types, Corollary 4 gives exact cycle-entering state counts by parity, and the convergence-domain results give explicit distance bounds. For example, when NN is even and r12r_{12} is odd, no cycles occur; when both are even, all three cycle types are possible.These are exact finite-size dynamical statements that are not conveyed by noting block independence alone.

Q2. Paired outcome and retrieval comparisons

To measure outcome sensitivity rather than tie frequency, we ran paired simulations under (i) the rectified rule, which retains the current bit at zero field, and (ii) fixed-sign handling, which assigns +1+1 at zero field. For each memory set and each initial state, the same synchronous network was run under both rules. The local field was computed directly from the zero-diagonal Hebbian matrix,

h(V)=∑μ=1Mξμ((ξμ)⊤V)−MV.h(V)=\sum_{\mu=1}^{M}\xi^\mu\big((\xi^\mu)^\top V\big)-MV.

Here, α=M/N\alpha=M/N is the memory-load ratio. The tested network sizes were 40, 80, and 160. We used memory-load ratios of 0.05, 0.10, 0.25, and 0.50. Each grid point used 40 independently generated memory sets and 256 identical uniform initial states per set under both rules. The table reports representative loads. The 95% confidence intervals are percentile-bootstrap intervals over memory sets. We define the terminal-attractor mismatch fraction as the fraction of paired initial states whose terminal attractor state sets differ between the two rules. A fixed point is represented by its singleton state set and a cycle by the set of states in that cycle, so a phase shift within the same cycle is not counted as a mismatch. The denominator includes all paired initial states, whether or not a zero field was encountered.

Table 1: Paired attractor outcomes at proportional memory loads.

NMαDifferent terminal attractor set, % (95% CI)Rectified paths hitting zero, %Cycle fraction, rectified / fixed +1, %
4040.1018.1 [13.4, 22.7]52.92.8 / 3.8
8080.1024.1 [16.8, 31.5]52.22.9 / 5.7
160160.1034.0 [24.3, 43.8]55.09.7 / 15.2
40100.2531.7 [22.6, 40.8]57.39.8 / 16.5
80200.2535.3 [24.6, 45.7]52.415.1 / 23.3
160400.2542.3 [30.5, 54.0]55.024.5 / 31.9

At α=0.10\alpha=0.10, the fraction of initial states assigned to different attractors rises from 18.1% at N=40N=40 to 34.0% at N=160N=160; at α=0.25\alpha=0.25, it rises from 31.7% to 42.3%. At low load, tie occurrence need not produce a different attractor: at N=40,M=2N=40,M=2 (α=0.05\alpha=0.05), 33.5% of trajectories encountered a zero field, but only 0.10% ended in a different attractor (95% CI: 0 to 0.27%). This contrast shows why occurrence and consequence must be reported separately.

For retrieval, we generated 256 queries per memory set by flipping exactly 10% of the bits of a randomly selected stored memory. The same queries were evaluated under both rules. Retrieval counts only when the trajectory reaches the exact target memory as a fixed point; cycles and other fixed points count as failures. The table gives the α=0.25\alpha=0.25 results from 40 memory sets per network size. The intervals are 95% bootstrap intervals over memory sets, and Δ\Delta is the paired rectified-minus-fixed difference in percentage points.

Table 2: Paired target-memory retrieval for 10%-corrupted queries at α = 0.25.

NMRectified retrieval, % (95% CI)Fixed +1 retrieval, % (95% CI)Δ, percentage points (95% CI)
401032.99 [28.13, 38.01]28.36 [23.38, 33.57]+4.63 [2.99, 6.41]
80209.71 [8.13, 11.42]8.77 [7.15, 10.43]+0.94 [0.43, 1.53]
160400.97 [0.62, 1.37]0.97 [0.62, 1.35]0.00 [-0.08, 0.06]

The retrieval effect is measurable in the moderate-size settings where retrieval remains accessible and attenuates when both rules have very low retrieval at N=160N=160. It also depends on corruption level: at N=40,M=6N=40,M=6 (α=0.15\alpha=0.15) with 20% corruption, retrieval is 73.73% under rectification and 71.31% under fixed-sign handling (Δ=+2.42\Delta=+2.42 points; 95% CI: +0.96 to +4.04). At N=160,M=24N=160,M=24 (α=0.15\alpha=0.15) with 30% corruption, the rates are 9.59% and 9.71% (Δ=−0.12\Delta=-0.12 points; 95% CI: -0.35 to +0.10). The data therefore support a regime-dependent effect, not a uniform retrieval advantage.

Finally, to test outcome sensitivity beyond the two-memory theory, we enumerated every initial state for random binary networks with M=5M=5. For each memory set, all 2N2^N states were evaluated under both rules.

Table 3: Exact full-state paired enumeration for M = 5.

NMemory setsStates per setPaths hitting zero, %Different terminal attractor set, % (95% CI)Cycle fraction, rectified / fixed +1, %
11502,04885.940.7 [35.9, 45.6]20.6 / 31.4
12404,0960.00.0 [0.0, 0.0]28.1 / 28.1
13308,19288.944.5 [38.0, 50.8]17.6 / 32.3

At N=12,M=5N=12,M=5, every local field is a sum of M(N−1)=55M(N-1)=55 terms in {−1,+1}\{-1,+1\} and cannot be zero. The two rules consequently produce identical trajectories for all initial states, even though both have a 28.1% cycle fraction. At odd N=11N=11 and N=13N=13, zero fields are possible, and the terminal attractor differs for 40.7% and 44.5% of all initial states, respectively. For target retrieval at N=11,M=5N=11,M=5, exhaustive enumeration within Hamming radii 1 and 2 gives 34.0% versus 23.8% at radius 1 (Δ=+10.2\Delta=+10.2 points; 95% CI: +7.1 to +13.3) and 22.7% versus 16.1% at radius 2 (Δ=+6.7\Delta=+6.7 points; 95% CI: +4.5 to +8.8), for rectified versus fixed-sign handling.

Together, these paired results show that the tie rule can change attractor assignment for substantial fractions of initial states, while its effect on target retrieval depends on network size, load, and corruption level. The revision will foreground this distinction and make the scope of the exact theory and the paired empirical evidence explicit.

I thank the authors for their efforts in preparing rebuttal. I have revisited the other reviews, the rebuttal responses, and the manuscript.

While my concerns regarding W3 and Q2 have been addressed, the remaining concerns have not been fully resolved. Therefore, I would like to maintain my original score.

If the other reviewers or AC support accepting this paper, I would not oppose the decision.

Reviewer_M5QEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies synchronous binary Hopfield networks under an update convention that leaves a neuron's state unchanged when its local field is zero. The authors show that this convention preserves global sign symmetry and derive exact finite-size descriptions of fixed-point convergence and period-two cycles for networks storing one or two Hebbian patterns. Their results include cycle counts and convergence regions expressed through Hamming distances, and are compared with exhaustive enumeration and Monte Carlo simulations.

Critical strengths

S1. The paper draws attention to an important modeling detail: how zero local fields are handled in synchronous binary updates. Its symmetry-based formulation is clear, and the supplementary analysis of how often zero fields arise helps explain why the choice can matter in implementations.

S2. The authors provide explicit finite-size classifications and counts of convergent and cyclic states, including parity-dependent effects. The exhaustive checks for small networks offer useful verification and make the results potentially helpful as reference examples.

Critical weaknesses

W1. The theoretical novelty of the two-memory analysis appears limited by a simple decomposition. Partitioning the neurons according to whether the two stored patterns agree or disagree makes the Hebbian weight matrix block diagonal. The two blocks are independent one-memory Hopfield networks (up to an overall factor of two). As a result, the main two-memory cycle conditions and counts seem to follow by combining the one-memory results. Making this structure explicit, and identifying what the detailed HDAR analysis adds beyond it, may considerably strengthen the contribution.

W2. The contribution of the zero-field convention to the reported dynamics is not sufficiently separated from properties of ordinary synchronous Hopfield updates. For example, some of the highlighted cyclic regimes occur where zero local fields are impossible. The convention itself is also natural, so a clearer comparison with earlier tie-handling choices would help establish what is new. Since the dynamical theory covers at most two stored patterns, its implications for more general associative-memory models remain uncertain.

Questions for authors

Q1. Does the block-diagonal decomposition described above recover all the main results for two stored patterns? If not, which conclusions specifically require the HDAR analysis? Clarifying this would help assess the theoretical novelty.

Q2. Which of the reported dynamical results change when a different zero-field tie convention is used, and how does the proposed convention relate to prior treatments?

Q3. For fixed N and inter-memory distance, complete enumeration should produce the same cycle fraction for every memory pair.

3 comments

Dear Reviewer,

Thank you for your careful reading. We respond separately to the two weaknesses and three questions below. Our central contribution is the symmetry result for the full synchronous dynamics and the exact Hamming-distance classification of the resulting finite-size attractor landscape.

W1. What the two-memory analysis adds beyond block factorization

We agree that the agreement/disagreement partition exactly block-diagonalizes the two-memory Hebbian matrix into two independent one-memory-like subsystems. This is a useful structural reduction, and some total counts can indeed be simplified by combining block contributions. It is not, by itself, the full two-memory result: the full-network trajectory is the product of the two block trajectories, and its interpretation in the original memory space depends jointly on the block states and the inter-memory distance r12r_{12}. The resulting system can converge to a fixed point or enter one of three distinct cycle types—self, hetero, or symmetric—with different admissibility conditions and initial-state regions.

HDAR provides the joint memory-distance coordinates (r1,r2,r12)(r_1,r_2,r_{12}) needed to make this product dynamics explicit. Theorem 3 states the conditions for the three cycle types; Corollary 4 gives exact parity-dependent counts of initial states entering cycles; and the convergence-domain analysis gives distance bounds for guaranteed convergence. Even when a total count can be factorized, these results identify how the block outcomes correspond to full-network attractors and where they occur in memory space; they are not conveyed by the block-diagonal form or an overall factor of two alone. We therefore present the partition as the reduction that makes the analysis tractable and HDAR as the framework that yields a checkable finite-size classification. The two-memory setting is the first extension beyond one memory in which the inter-memory distance and parity organize multiple cycle types; more memories introduce a richer configuration space, while our exact claims remain limited to the regimes analyzed here. We will clarify this distinction and foreground the system-level conclusions.

W2. Separating tie-sensitive results from synchronous dynamics

We do not claim that every cycle reported in the paper is caused by the zero-field convention. Whenever a trajectory has no zero local field, the rectified rule and any fixed-sign tie rule make exactly the same updates; cycles in such regimes are therefore properties of synchronous dynamics that are independent of tie handling. We will identify these regimes as tie-independent controls.

The contribution specific to zero-field handling is different. Retaining the current state at a tie makes the update map sign-equivariant, F(−V)=−F(V)F(-V)=-F(V), so trajectories, fixed points, cycles, and basins are paired under global sign reversal. A fixed-sign convention does not have this property. The paper's N=3N=3 example makes the dynamical difference explicit: forcing a tied neuron to +1+1 creates a two-cycle, while retaining its current state sends the same initial condition to the stored pattern. Appendix B.5 separately quantifies how often ties arise: for M=10M=10, a trajectory encounters at least one zero field within 20 updates with probabilities 0.99310.9931 at N=499N=499 and 0.51040.5104 at N=500N=500. These are occurrence rates, not outcome-change rates; we will preserve that distinction and clarify which conclusions depend on ties.

Q1. Coverage of the block reduction and role of HDAR

Yes, the decomposition is exact at the transition-map level: every full state splits into agreement and disagreement blocks, and every full trajectory is reconstructed by combining their independent block trajectories. As the reviewer notes, some aggregate counts can also be combined from block-level counts. However, the block form alone does not classify the resulting attractors in the original memory coordinates.

HDAR builds on this reduction by tracking the joint distances (r1,r2,r12)(r_1,r_2,r_{12}). All exact two-memory results reported here are derived through this analysis: fixed-point conditions; conditions for self-, hetero-, and symmetric cycles; exact parity-dependent counts and fractions of cycle-entering initial states; and formulas for convergence and complete-convergence domains. \textbf{Thus, the factorization guarantees full-state coverage, while HDAR turns it into the explicit system-level classification.} We will clarify these roles.

Q2. Effect of alternative zero-field conventions

The proposed rule is the state-retaining convention used in earlier work; our novelty claim concerns its dynamical consequences. Under a fixed-sign rule, updates coincide with ours at nonzero fields and can differ only at zero-field coordinates. When a trajectory encounters a tie, its terminal attractor, cycle status, or target-retrieval outcome can change. For paired evidence under identical memory sets and initial states, see our response to Reviewer 2fNe, Question Q2: Table 1 reports terminal-attractor mismatch rates of 18.1%--42.3% across the listed proportional-load settings; Table 3 gives exact M=5M=5 mismatch rates of 40.7% at N=11N=11 and 44.5% at N=13N=13, with cycle fractions of 20.6% versus 31.4% and 17.6% versus 32.3%, respectively, for rectified versus fixed +1+1 handling. At N=12N=12, zero fields are impossible, the mismatch rate is zero, and both rules have the same 28.1% cycle fraction. Table 2 reports target retrieval: at α=0.25\alpha=0.25 with 10% corruption, rectification changes retrieval by +4.63+4.63, +0.94+0.94, and 0.000.00 percentage points at N=40,80,160N=40,80,160, respectively. These paired results show substantial changes in attractor assignment where ties occur, while retrieval effects depend on the regime

Under the state-retaining rule, the entire transition map obeys F(−V)=−F(V)F(-V)=-F(V); under a fixed-sign rule, tied coordinates generally violate this relation. We will state explicitly that zero-field-free cycles are not attributed to the proposed convention, while the symmetry theorem and tie-containing transitions distinguish it from fixed-sign handling. The symmetry argument itself requires only a symmetric weight matrix, zero bias, and the state-retaining tie rule, so sign-equivariance extends beyond two stored patterns. The explicit HDAR cycle counts and convergence-domain bounds use the one- and two-memory structure; we will state this distinction to clarify both the broader implication and the scope of the exact classification.

Q3. Invariance of the exact cycle fraction

Yes. For the rectified rule, any two memory pairs with the same fixed NN and r12r_{12} are related by a coordinate-wise sign change and a coordinate permutation. These transformations conjugate the transition maps and bijectively map all initial states and trajectories, preserving whether each trajectory reaches a fixed point or a cycle. Therefore, exhaustive enumeration gives exactly the same cycle fraction for every memory pair with those fixed values of NN and r12r_{12}. Monte Carlo estimates may differ because of sampling variability; an exact-enumeration discrepancy would indicate an implementation or bookkeeping inconsistency. We will state this invariance alongside the enumeration protocol.

Thank you for the clarifications and additional experiments. The paired comparisons provide useful evidence that zero-field handling can affect attractor assignments beyond the two-memory setting, addressing part of my concern about its empirical relevance. However, my main concern about the theoretical contribution remains. While the explicit finite-size classification is useful, I remain unconvinced that it provides substantial additional theoretical insight beyond combining the dynamics of the two independent one-memory networks and expressing the results in Hamming-distance coordinates. I therefore maintain my score, as my concerns about the novelty and significance of the contribution remain.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.