acceptodds
Under review as a conference paper at ICLR 2027

Intrinsic Associative Memory on Riemannian Manifolds: Curvature, Capacity, and Emergent Modes

Abstract

Geometry does more than constrain an associative memory: curvature determines what it remembers and which states it creates. We develop intrinsic dense associative memories on Riemannian manifolds by casting memory as Epanechnikov kernel-density mode seeking. We compare geodesic and volume-corrected energies and show that curvature separates their behavior. We prove that geodesic memory always retains an isolated pattern, while corrected memory obeys a sharp Ricci-curvature threshold: positive curvature can erase memories in high dimensions, while negative curvature reinforces them. We derive geodesic capacity scalings of for retaining every pattern and for a typical one, where is the pairwise kernel-overlap probability. We show how overlap creates novel memories: designed -pattern configurations realize all subset modes, but random data at the storage threshold yield only a Poisson number. We establish exact one-step recall using Riemannian mean shift. In simulations, we recover the predicted curvature transition and every designed mode. On WordNet's full noun hierarchy, we demonstrate that volume correction improves low-capacity retrieval. Together, our work shows that curvature is a design variable for associative memory, not merely a property of the data.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Riemannian Dense Associate Memory, Storage, retrieval, Creation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VxpyOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies geodesic and volume-corrected associative-memory energies on Riemannian manifolds. It is linking curvature to storage stability, capacity and additional attractor modes. Geometric experiments, WordNet, leaf-shape and EEG settings eare used for the examinationof the selected mechanisms and retrieval outcomes

Critical strengths
  1. The strongest part is the connection between curvature, storage and persistence of modes. The results explain how geometry changes memory behavior. How it going beyond simply using geodesic distance.
  2. The experiments help explain the mechanisms studied in the theory. All 18 supplied tests passed and selected leaf, WordNet and EEG calculations could be reproduced from the available files
Critical weaknesses
  1. The solver settings are given, but it is unclear how often runs stop at a limit. Can tighter settings change the reported differences? That matters when interpreting the size of the empirical gains
  2. The practical evidence is narrower than general retrieval performance. WordNet uses 16-memory banks and an exact-recovery criterion. The available NN identification numbers measure something different. So they don't allow a direct comparison. This limits the practical conclusion, but still leaves the theoretical contribution intact
Questions for authors
  1. For the main mode-persistence result how do the constants depend on dimension, curvature and configuration margins? Which bounds are uniform over subsets? A pointer to the relevant proof is enough if this is already covered.
  2. How often do WordNet runs hit iteration or backtracking limits? Do existing diagnostics or small sensitivity check show that the recovery differences are stable?
3 comments
AuthorsOpenReview ↗

response 1

We thank you for recognizing the contribution of our work and your thoughtful questions.

W1 — Stability of the empirical gains under tighter solver settings

“The solver settings are given, but it is unclear how often runs stop at a limit. Can tighter settings change the reported differences? That matters when interpreting the size of the empirical gains.”

Answer

We added a paired sensitivity check in Appendix E.3, Table 2 (p. 23), with a blue pointer in Section 5 (p. 9). The check holds the five embeddings, all 256 existing 16-memory banks per seed, queries, and radii fixed. It tightens hyperbolic and Euclidean stopping tolerances by a factor of 100, raises the iteration budget from 100 to 500, and raises the hyperbolic backtracking budget from 16 to 32 halvings.

Every one of the 552,960 paired exact-recovery outcomes is unchanged. No run exhausts backtracking, although some runs still reach the iteration limit. The reported recovery differences are therefore stable under these tested settings at the original radii; this is not a claim that every trajectory converges. Our answer to Q2 gives the termination counts. The new Appendix E.6 (p. 26) separately examines support-boundary effects and shows that the corrected benefit persists away from the rounding-scale retention boundary.

W2 — Comparable metrics and the practical scope of the result

“The practical evidence is narrower than general retrieval performance. WordNet uses 16-memory banks and an exact-recovery criterion. The available NN identification numbers measure something different. So they don't allow a direct comparison. This limits the practical conclusion, but still leaves the theoretical contribution intact.”

Answer

We now report a common hyperbolic identification task alongside exact state recovery in the new Appendix E.3, Table 3 (p. 23), with a blue pointer in Section 5 (p. 9). We decode the terminal states of the geodesic and corrected energies to a nearest stored key using the same hyperbolic distance. Geodesic NN uses that decoder on the original query. All three methods use the same banks and queries, without a retention or activity filter.

Emergent pair modes can be equidistant from two keys; naive numerical argmin labels can therefore change under tiny numerical perturbations. We use one explicit tie convention for all three methods: keys within 1e-4 h of the minimum distance are tied, and a target among k tied keys receives score 1/k, the expected accuracy under uniform tie resolution. This convention is independent of the target identity.

Table 3 compares geodesic NN with the two intrinsic energies. The corrected energy has higher mean identification than geodesic NN at all three evidence levels under both numerical settings. All paired 95% intervals for corrected-minus-geodesic-NN identification lie above zero under both settings:

EvidenceGeodesic NNGeodesic energyCorrected energy, original/tighter
25%.671.631.696 / .695
50%.805.772.839 / .838
75%.901.852.916 / .913

Geodesic NN is labeled NN geo. in Table 3; it uses the same hyperbolic distance as both intrinsic energies.

The largest identification change under tighter settings is 0.308 percentage points. Identification and exact recovery are therefore reported separately, with both numerical settings shown.

The matched comparison gives the corrected model a mean identification advantage of approximately 2.5, 3.4, and 1.5 percentage points under the original settings, and 2.4, 3.3, and 1.2 percentage points under the tighter settings, using differences of the displayed means. This identification benefit complements the exact-recovery comparison in Table 1 (p. 22).

The activity diagnostic clarifies what the exact-recovery benefit represents: initially multiple-active queries account for 91.2% of the net corrected-minus-geodesic gain on the same banks and queries (new Appendix E.6, Table 4, p. 26). Together with the distance-dependent corrected weights in the new Appendix E.4, Eq. (E.1) (p. 24), this supports a benefit in resolving competition between active memories. These conditional activity comparisons locate the sampled exact-recovery gains; the NN comparison above still includes all queries and measures target identification.

AuthorsOpenReview ↗

response 2

Q1 — Dimension, curvature, configuration margins, and subset uniformity

“For the main mode-persistence result how do the constants depend on dimension, curvature and configuration margins? Which bounds are uniform over subsets? A pointer to the relevant proof is enough if this is already covered.”

Answer

The designed-pattern part of Theorem 4 is proved in Appendix K (pp. 39–43). Its constants are uniform over every nonempty active subset and every 2≤N≤m+12\leq N\leq m+1, on a fixed normal neighborhood of a fixed manifold. They can depend on dimension and local geometry. We do not claim dimension-uniform constants for arbitrary manifolds or a bound based only on Ricci curvature.

The proof makes the dependence on the configuration margins explicit. For a subset of size kk, the rescaled squared distances from its Euclidean centroid are 1−1/k1-1/k to members and 1+1/k1+1/k to nonmembers. Thus the worst support margin is 1/N1/N. Averaging the active score, rather than summing it, gives a C2C^2 perturbation bounded by Ch2Ch^2 with the same CC for every subset size. The condition Ch2≤1/(64N)Ch^2\leq 1/(64N) ensures that the displaced critical point remains in the correct active region.

For novelty threshold τh\tau h, the other margin is gτ=1/2−τ2>0g_\tau=1/2-\tau^2>0. The proof supplies the explicit sufficient choice

\frac{h_0}{2}, \frac{1}{\sqrt{64C}}, \sqrt{\frac{1/2-\tau^2}{8C}} \right\\}, \quad h\leq \frac{c_{\tau}}{\sqrt{N}}.$$ Here $C$ is obtained from local derivative bounds for rescaled squared distance and inverse volume density, and $h_0$ also enforces the normal and convexity regime. On the unit sphere, Corollary K.1 makes the dimension dependence explicit: the sufficient radii are $h\leq c_\tau/\sqrt N$ for the geodesic energy and $h\leq c_\tau/\sqrt{mN}$ for the corrected energy, with dimension-independent $c_\tau$. These are sufficient bounds for the simplex construction, not necessary bandwidth conditions for arbitrary configurations. The isolated-pattern Ricci Hessian (Theorem 2, p. 5; new Appendix I.1, Eq. (I.3), p. 36) describes local stability at a key, whereas persistence requires the neighborhood and support-margin bounds above. The corrected two-key example in the new Appendix E.5 (p. 25) loses an emergent attractor outside the required component-concavity regime, so it does not contradict this small-bandwidth result. The new kernel comparison in Appendix I.1, Table 5 (p. 37), also explains why this Epanechnikov count does not automatically extend to other compact profiles. To make the existing proof easier to locate, we added a blue pointer in **Section 4.2 (p. 7)** and a blue clarification immediately before **Corollary K.1 (p. 42)**. The precise proof locations are **Eq. (K.1) (p. 40)** for the support margins; **Eqs. (K.2)–(K.3) (p. 40)** for the local geometric bounds; **Eq. (K.4) (p. 41)** for the subset-uniform averaged-score perturbation; and **Eq. (K.5) (p. 41)** for the sufficient margin condition. The explicit choice of $c_\tau$ is on **p. 41**, and Corollary K.1 supplies the separate spherical dimension bounds on **p. 42**.
AuthorsOpenReview ↗

response 3

Q2 — Iteration limits and sensitivity checks

“How often do WordNet runs hit iteration or backtracking limits? Do existing diagnostics or small sensitivity check show that the recovery differences are stable?”

Answer

We added Appendix E.3, “Solver sensitivity and comparable identification” (p. 23), including Table 2, and a pointer in Section 5 (p. 9). The check reruns all 256 archived banks for each of the five embedding seeds, with 16 memories per bank. Embeddings, clean keys, partial queries, bank membership, and radius selection are fixed.

The original hyperbolic solvers stop when the normalized logarithmic-update norm is at most 1e-5, or after 100 iterations, and allow up to 16 step halvings. The Euclidean displacement tolerance is 1e-8. The tighter setting uses tolerances 1e-7 and 1e-10, 500 iterations, and up to 32 hyperbolic halvings. The permitted kernel-sum decrease is reduced from 1e-12 to 1e-14. The solver operates on the kernel sum, so the additive epsilon in the logarithmic energy is not a solver parameter.

The new diagnostics distinguish tolerance termination, inactive support, iteration-limit exits, and exhausted backtracking. The iteration-limit counts and rates are:

SolverOriginal limit exitsTighter limit exits
Ambient Euclidean0 (0%)0 (0%)
Geodesic1,312 (0.712%)532 (0.289%)
Corrected694 (0.377%)853 (0.463%)

There are 184,320 runs per solver per setting. No run exhausts backtracking or reaches its configured minimum step. The remaining runs meet their stated stopping tolerance or return an inactive query.

Across 552,960 paired method/query/bank cases, every exact-recovery outcome is unchanged, so the headline paired gains are unchanged.

Thus, the reported exact-recovery gains are unchanged under the tested tolerances and budgets at the original support radii, while some iteration-limit exits remain. We do not equate this sensitivity result with convergence of every trajectory. The original success criterion remains a retained target and terminal distance at most 0.05 h, without adding a convergence-flag filter.

The endpoint diagnostic in the new Appendix E.6 (p. 26) further separates the gain from unfinished runs. All 16,461 corrected-only successes meet the corrected solver's stopping tolerance. In 15,707 (95.4%) of these cases, the geodesic solver also meets its tolerance but ends farther than 0.05 h from every stored key. Most of these gains therefore replace numerically converged non-key endpoints. This stopping classification does not certify each endpoint as a stable emergent memory.

A separate support-boundary check qualifies the scope of the stability claim. All 1,522 initially single-active cases with different recovery outcomes have a retained target that defines the radius, with only a rounding-scale retention margin. Contracting their radii by 10−610^{-6} in relative terms preserves initial active sets and retained-target masks; both solvers then remain single-active, converge, and recover every target. This is a diagnostic on selected discrepant cases, not a full benchmark at the contracted radius. At the original radii, the 136,017 cases with retained targets that do not define the radius still show a corrected gain of 6.17 percentage points [4.76, 7.45]. Thus the benefit also persists away from that numerical boundary.

Finally, we want to re-emphasize the main contributions of this work once more to you:

  • Two intrinsic memory energies, with exact local Hessians and a sharp Ricci-curvature stability threshold.
  • Riemannian mean-shift guarantees for one-step recall, descent, and local convergence.
  • Distinct all-pattern and typical-pattern capacity laws, including high-dimensional sphere rates.
  • An active-subset characterization of emergence and constructions realizing all 2N−12^N-1 subset modes.
  • A fixed-dimensional Poisson emergence limit conditional on retaining all originals, with explicit coexistence probabilities.
  • Numerical tests of curvature, capacity, and emergence, plus improved corrected retrieval in 16-memory WordNet banks.

In light of these contributions, the additional evidence, and our answers to your questions and concerns, we respectfully ask you to consider an increase of the score.

Reviewer_kWv9OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper extends Epanechnikov dense associative memory to Riemannian manifolds. The authors compare geodesic and volume-corrected energies and derive a Ricci-curvature criterion for the stability of stored patterns. They also establish retrieval guarantees, storage capacity scalings, and results on emergent memories. The theoretical predictions are supported by experiments on several manifolds, and the WordNet experiments demonstrate improved retrieval performance with volume correction.

Critical strengths

S1. The paper makes a significant theoretical contribution to understanding associative memory on curved spaces. The exact Ricci-curvature criterion reveals how geometry can affect memory stability, and the comparison between geodesic and volume-corrected energies provides a clear and interesting result. This goes beyond a straightforward extension of Euclidean associative memory.

S2. The paper develops a comprehensive theory supported by convincing experiments. The analysis covers retrieval guarantees, distinct storage capacity scalings, and the emergence of additional stable states. The results for designed and random configurations are particularly interesting. The experiments confirm the main theoretical predictions, while the WordNet results demonstrate the potential practical value of the approach.

Critical weaknesses

The following points are not critical, but addressing them could further strengthen the paper.

W1. The benefits of volume correction for associative memory could be explained more clearly. While its motivation from kernel density estimation is well established, its advantages for memory retrieval are less clear. In particular, volume correction can destabilize stored patterns under positive curvature. A clearer connection between the theoretical findings and the improved WordNet retrieval performance would strengthen the practical significance of the work.

W2. The role of kernel choice in the emergence of new memories could be discussed further. The coexistence of retained patterns and stable emergent memories appears to depend on both compact support and kernel shape. Clarifying which results extend to other kernels would help distinguish the effects of manifold geometry from those of the Epanechnikov kernel.

Questions for authors

Q1. What benefits does volume correction offer for associative memory, beyond its role in kernel density estimation? In particular, can it reduce attraction to emergent memories or enlarge the attraction basins of stored patterns?

Q2. What explains the improved retrieval performance of the volume-corrected model on WordNet? A comparison between queries with a single active memory and those with multiple active memories might help clarify the mechanism.

Q3. For retrieval of stored patterns, could alternative kernels suppress emergent memories while preserving exact storage and robust recall? How would such kernel choices affect the conclusions about curvature and memory stability?

7 comments
AuthorsOpenReview ↗

response 1

We thank you for recognizing the contribution of our work and your thoughtful questions.

Response to Reviewer kWv9

W1 — Benefits of volume correction for memory retrieval

“The benefits of volume correction for associative memory could be explained more clearly. While its motivation from kernel density estimation is well established, its advantages for memory retrieval are less clear. In particular, volume correction can destabilize stored patterns under positive curvature. A clearer connection between the theoretical findings and the improved WordNet retrieval performance would strengthen the practical significance of the work.”

Answer

We added Appendix E.4, “Retrieval benefit in the WordNet geometry” (p. 24), including Eq. (E.1), and a blue pointer in Section 5 (p. 9). The added explanation connects the curvature-dependent stability result, the corrected retrieval dynamics, and the observed WordNet gains.

Theorem 2 (p. 5) already characterizes how correction affects isolated stored patterns. WordNet uses a ten-dimensional hyperbolic embedding with Ric⁡=−9g\operatorname{Ric}=-9g, so correction adds 3g/[β(1+ϵ)]3g/[\beta(1+\epsilon)] to the isolated-pattern energy Hessian. This strengthens local attraction in the WordNet geometry and complements the positive-curvature destabilization established by the same theorem.

For corrupted queries, correction also changes the competition between active memories. The geodesic Epanechnikov update weights active keys uniformly. In hyperbolic space, the corrected score gradient instead has positive, distance-dependent weights that decrease with distance, as shown explicitly in the added Eq. (E.1). The normalized update therefore reduces the relative influence of more distant active keys. This changes retrieval dynamics while preserving the support gate.

The existing EvcE_{\mathrm{vc}}-versus-EgeoE_{\mathrm{geo}} comparison holds the embeddings, hyperbolic distances, queries, support radii, and retained-target sets fixed. The mean exact-recovery gains are 10.3, 9.3, and 7.2 percentage points at 25%, 50%, and 75% evidence, respectively. The paired 95% seed-bootstrap intervals are [8.3, 11.8], [7.5, 10.8], and [5.9, 8.3] percentage points, respectively (Table 1, p. 22). This provides evidence of a benefit from correction beyond the use of hyperbolic distance alone.

We distinguish the local stability result from this empirical retrieval result. In exact arithmetic, when the sole active key is retained, both normalized hyperbolic updates already recover that key; the larger isolated-key Hessian alone does not explain the observed gain. The new Appendix E.7, Proposition E.1 (p. 27) formalizes corrected one-step recall and damped contraction in a guaranteed singleton neighborhood. The added explanation connects the gain to the change in forces between competing keys, without claiming that the experiment establishes larger attraction basins. The new activity comparison and numerical boundary check in Appendix E.6, Table 4 (p. 26) examine this mechanism empirically, as detailed in our answer to Q2.

The practical benefit here concerns exact state recovery under the specified energy dynamics. The common hyperbolic identification comparison in the new Appendix E.3, Table 3 (p. 23) additionally finds higher mean corrected identification accuracy than direct hyperbolic NN at all three evidence levels under both solver settings. All paired 95% intervals for corrected-minus-geodesic-NN identification lie above zero under both settings. This supplies a separate identification comparison alongside the exact-recovery results.

The new solver check in Appendix E.3, Table 2 (p. 23) also shows that these exact-recovery outcomes are unchanged under tighter numerical settings at the original radii.

AuthorsOpenReview ↗

response 2

W2 — Compact support, kernel shape, and emergence

“The role of kernel choice in the emergence of new memories could be discussed further. The coexistence of retained patterns and stable emergent memories appears to depend on both compact support and kernel shape. Clarifying which results extend to other kernels would help distinguish the effects of manifold geometry from those of the Epanechnikov kernel.”

Answer

We expanded the explanation of the Epanechnikov choice in Section 2 (p. 3) and added Appendix I.1, “Kernel choice: storage, curvature, and emergence” (pp. 36–37), including the analytical comparison in Table 5 (p. 37). All of these additions are in blue. Compact support prevents sufficiently separated stored patterns from interacting. The profile inside the support determines the forces between active memories and the stability of their balance points. Epanechnikov's quadratic radial profile gives uniform geodesic mean-shift weights and, under our convexity assumptions, strongly concave active score components. These properties yield the intrinsic-mean equation and at most one candidate per active subset.

The isolated-pattern curvature result extends beyond Epanechnikov, as derived in the new Eq. (I.3) (p. 36). For a continuous compact profile ρ(d2/h2)\rho(d^2/h^2), positive and C2C^2 inside its support, with ρ(0)=1\rho(0)=1 and cρ=−ρ′(0)>0c_\rho=-\rho'(0)>0, an isolated stored pattern has

Hess⁡Eρ,geo=cρg1+ϵ,Hess⁡Eρ,vc=cρg−Ric⁡/(3β)1+ϵ.\operatorname{Hess}E_{\rho,\mathrm{geo}} =\frac{c_\rho g}{1+\epsilon}, \qquad \operatorname{Hess}E_{\rho,\mathrm{vc}} =\frac{c_\rho g-\operatorname{Ric}/(3\beta)}{1+\epsilon}.

Thus the kernel controls the restoring coefficient, while the same Ricci term describes the geometric effect of volume correction.

The family ρp(s)=(1−s)+p\rho_p(s)=(1-s)_+^p supplies concrete alternatives: Epanechnikov (p=1p=1), biweight (p=2p=2), and triweight (p=3p=3). All retain isolated stored patterns in flat space. For two keys separated by h<d<2hh<d<2h, the midpoint is a stable emergent memory throughout that interval for Epanechnikov, only when d<2h/3d<2h/\sqrt3 for biweight, and never for triweight. Biweight therefore demonstrates that coexistence extends to another familiar kernel, while the different stability ranges show that compact support alone does not determine emergence.

We distinguish this local extension from the counting results. The one-candidate argument extends when the chosen profile satisfies the required strong concavity on convex supports. Biweight and triweight do not satisfy that condition throughout their supports. The exact subset counts and Poisson emergence intensity are therefore not claimed for these alternatives; their support-boundary behavior also needs checking. The collision-based capacity argument carries over under the same sampling assumptions once almost-sure equivalence between retention and isolation is established for the new profile. Emergence is already possible in flat geometry, and curvature modifies the resulting kernel-dependent landscape.

For Epanechnikov, the designed-pattern proof in Appendix K (pp. 39–43) separates geometry from these profile assumptions quantitatively. On a fixed normal neighborhood, the averaged active score has a C2C^2 perturbation at most Ch2Ch^2, with the same CC for every subset. The sufficient bound Ch2≤1/(64N)Ch^2\le 1/(64N) controls the critical-point displacement relative to the worst rescaled squared-distance support margin 1/N1/N. This neighborhood-wide control is distinct from the isolated-key Ricci Hessian statement and does not by itself extend the count to other kernels.

The blue clarification before Corollary K.1 (p. 42) specifies the local geometric dependence and subset uniformity of these perturbation bounds; Eqs. (K.4)–(K.5) (p. 41) give the quantitative conditions.

AuthorsOpenReview ↗

response 3

Q1 — Emergent attraction and stored-pattern basins

“What benefits does volume correction offer for associative memory, beyond its role in kernel density estimation? In particular, can it reduce attraction to emergent memories or enlarge the attraction basins of stored patterns?”

Answer

Volume correction changes the retrieval landscape as well as the density estimate. In hyperbolic geometry it strengthens local stability at isolated stored patterns and reduces the relative influence of more distant active keys, as detailed in Theorem 2 (p. 5) and the new Appendix E.4, Eq. (E.1) (p. 24). The local Hessian result alone does not establish larger attraction basins.

To answer the question constructively, we have added an analytical two-key example in the new blue Appendix E.5, “Two-key example: reduced emergent attraction” (p. 25), Eqs. (E.2)–(E.4). Place two keys at distance 33 in H10\mathbb H^{10}, with support radius h=2h=2. Both stored keys are isolated and stable under both energies. The geodesic model also has a stable emergent memory at their midpoint. Volume correction turns this midpoint into a saddle while preserving both stored memories. Eq. (E.4) proves the strict radial-convexity bound kvc′′(r)/a(r)≥367/400>0k_{\mathrm{vc}}''(r)/a(r)\ge367/400>0 on 1≤r≤21\le r\le2; the corrected midpoint has one positive and nine negative score-Hessian eigenvalues, with the opposite signs for the energy Hessian.

Parameterize the joining geodesic by x∈[0,3]x\in[0,3]. Under negative-energy-gradient flow, every query with 1<x<21<x<2 converges to the midpoint in the geodesic model. With correction, queries with 1<x<3/21<x<3/2 converge to the left stored key, and those with 3/2<x<23/2<x<2 converge to the right stored key. The proof verifies transverse passage across the compact-support boundaries.

Thus correction can suppress an emergent attractor and extend the stored-pattern recall intervals along a geodesic. This is a result for the specified configuration and one-dimensional sections of its basins, not a general comparison of full-dimensional basin volumes. The example lies outside the component-concavity regime used by the small-bandwidth persistence proof: Eq. (E.4) gives kvc′′>0k_{\mathrm{vc}}''>0 on the overlap. Appendix K, Eqs. (K.4)–(K.5) (p. 41) instead controls C2C^2 perturbations and support margins to preserve the Epanechnikov modes. The two conclusions therefore concern different parameter regimes. The WordNet experiment separately establishes improved exact recovery for the sampled queries; it does not measure basin volumes or classify each failure as attraction to an emergent memory.

Reviewer_QMYkOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper builds associative memories that work directly on curved spaces such as spheres, rotation groups and hyperbolic space. Each memory is a mode of a compactly supported kernel density, so distant patterns do not interact and overlapping patterns can create new modes. The authors compare two energies that agree in flat space. The geodesic energy replaces Euclidean distance with geodesic distance, and the volume-corrected energy also compensates for how curvature distorts volume. They prove that the geodesic energy always keeps an isolated pattern, while the corrected energy keeps it only above a threshold set by Ricci curvature, so positive curvature can erase a memory and negative curvature can strengthen it. They also derive how many patterns survive random interference, describe the new modes that overlapping patterns create, and show that Riemannian mean shift recalls an isolated pattern in one step. Simulations recover the predicted threshold and the designed modes, and on the WordNet noun hierarchy the corrected energy improves retrieval in small memory banks.

Critical strengths

The paper shows that standard adjustments such as stretching distances and areas can make a memory system forget the very items it was built to store when the surface curves like a sphere, while the same adjustment makes memories more stable on saddle-shaped surfaces. It also gives the exact tipping point at which forgetting starts.

Critical weaknesses

1. The headline real-data result rests on the corrected energy, whose update has no retrieval guarantee, and a simpler explanation fits it.

  • The abstract says that on WordNet "volume correction improves low-capacity retrieval". The geodesic energy, the one with retrieval guarantees, has lower mean recovery than plain Euclidean memory on the same coordinates at every evidence level (Table 1), and no interval is reported for this difference. The main text compares the corrected energy with both but does not say this (p.9). It is visible only in Fig. 3a and Table 1.
  • Theorem 1 covers only the geodesic update. The paper proves neither one-step recall nor convergence for the corrected update used in the experiments, and it does not report the step size, stopping rule or ϵ used in any experiment.
  • Outside WordNet, the corrected energy is clearly worse than the geodesic one once the offset grows in the asymmetric-noise test (Fig. 7), and noticeably worse at the largest noise on the sphere (p.15). The manuscript reports no other setting where it is better, and no figure shows one.

2. The title and abstract claim more than the experiments show.

  • The abstract says that "curvature determines what it remembers and which states it creates" and that "curvature is a design variable for associative memory". No experiment treats curvature as a design choice. None selects, tunes or learns the curvature of a representation, and curvature differs only between benchmark manifolds (e.g., Fig. 4). Each comparison keeps the manifold fixed and switches the energy or the metric.
  • The abstract says random data at the storage threshold yield "only a Poisson number" of new modes. The mean of that Poisson count grows exponentially with the dimension (p.8), and the law is proved only at fixed dimension. "Only" holds for the dependence on the number of patterns, not for high-dimensional embeddings.

3. The experiments check the theorems only in configurations built to satisfy their hypotheses, and none of the probabilistic laws is tested.

  • The curvature-transition experiments place each pattern in its own bank (p.8). With a single pattern there is no interference. Recall then depends only on one kernel's radial profile, whose stability at the center is what Theorem 2 computes. These runs confirm that the implementation matches the algebra rather than testing memory under interference.
  • The capacity laws of Sec. 3 and the Poisson law of Theorem 4 are never compared with simulation.

Hallucinated reference:

Dongjie Wang, Meng Xiao, and Jimeng Sun. Curved embeddings for improving reasoning of large language models. arXiv:2602.13418, 2026.

arXiv number: arXiv:2602.13418 refers to an entirely different paper. It is actually “Text Has Curvature” by Grover et al.

Questions for authors

See Weaknesses

7 comments
AuthorsOpenReview ↗

response 1

W1 — WordNet comparisons, the scope of correction, and corrected recall

The main text emphasizes the corrected energy's WordNet gains but does not explicitly state that the geodesic energy has lower mean recovery than ambient Euclidean memory at every evidence level, or report intervals for that difference. Theorem 1 covers the geodesic update; the corrected update lacks a retrieval guarantee and numerical settings. Correction also performs worse than the geodesic energy in the asymmetric-noise experiment and at the largest spherical noise level.

Answer

We propose and comparatively study two Riemannian Epanechnikov memory energies. Their benefits depend on the geometry and retrieval task. The abstract's empirical improvement is explicitly scoped to WordNet. The cases where correction hurts are also part of this comparison: positive curvature can destabilize stored patterns, and the asymmetric-noise experiment favors the uncorrected intrinsic mean. We agree that the geodesic-versus-Euclidean WordNet result should be stated directly alongside the corrected gains.

The blue revision to the main WordNet paragraph in Section 5 (p. 9) now explicitly reports that the geodesic energy has lower mean exact recovery than ambient Euclidean memory at all three evidence levels. We added the missing paired differences and 95% intervals in blue to Appendix E.2, Table 1 (p. 22):

Ancestor evidenceGeodesic − ambient, percentage pointsPaired 95% interval
25%−2.24[−3.11, −1.03]
50%−3.60[−4.47, −2.24]
75%−3.10[−3.69, −2.37]

The difference is negative in all five seeds at each evidence level. These intervals enumerate all 55=31255^5=3125 bootstrap resamples of the five paired seed means, averaging the 256 overlapping banks within each seed before resampling. They use the archived 16-memory results and original numerical settings. A conditional retrieval theorem does not imply higher average recovery across this query distribution or against another metric.

For the corrected update, we added Proposition E.1 and its proof in the new Appendix E.7 (p. 27). It treats the normalized update used by the WordNet solver on Hm\mathbb H^m, including WordNet's m=10m=10. If the target key ξi\xi_i is isolated, with si=min⁡j≠id(ξi,ξj)>hs_i=\min_{j\ne i}d(\xi_i,\xi_j)>h, a query for which it is the sole active key is recalled in one full step. The positive corrected radial weight cancels under normalization, giving v^vc(x)=log⁡xξi\widehat v_{\mathrm{vc}}(x)=\log_x\xi_i. The step strictly decreases the corrected energy unless the query is already the key, so score-based backtracking starting at t=1t=1 accepts it.

In particular, every query in BΔ(ξi)B_\Delta(\xi_i) is recalled whenever 0<Δ<h0<\Delta<h and Δ≤si−h\Delta\le s_i-h. This ball is invariant under a fixed damped step 0<t≤10<t\le1, with the exact convergence rate

d(xn,ξi)=(1−t)nd(x0,ξi).d(x_n,\xi_i)=(1-t)^n d(x_0,\xi_i).

The proposition supplies a recall guarantee for the exact normalized corrected map in this isolated-memory region. It does not establish convergence or target recall for arbitrary queries with competing active keys. Those queries contribute to the empirical WordNet comparison; the new Appendix E.6, Table 4 (p. 26), analyzes them separately.

That activity analysis provides a concrete explanation for the WordNet gain. Initially multiple-active queries account for 91.2% of the net corrected-minus-geodesic gain. Across 45,372 such cases, mean exact recovery is 32.69% for the corrected energy and 0% for the geodesic energy, with a paired seed-bootstrap gain interval of [29.04, 35.73] percentage points. The new Appendix E.4, Eq. (E.1) (p. 24), shows how correction downweights more distant active keys; the experiment locates the benefit where those keys compete. In 95.4% of the corrected-only successes, the geodesic solver meets its stopping tolerance at a non-key endpoint. Thus the gain is primarily associated with different retrieval outcomes after numerical termination. The new Appendix E.5 (p. 25) separately gives an analytical configuration where correction removes a stable emergent midpoint while preserving both original memories. This supplies a constructive mechanism alongside the comparative empirical results.

The new Appendix E.3 (p. 23), including Table 2, documents the WordNet numerical protocol. Original hyperbolic updates start at t=1t=1, allow up to 16 halvings, and stop at normalized update norm 10−510^{-5} or 100 iterations. The accepted kernel-sum decrease is at most 10−1210^{-12}. The ambient Euclidean solver uses tolerance 10−810^{-8}, the same iteration budget, and no backtracking. These solvers operate directly on the kernel sum, so the additive ϵ\epsilon in the logarithmic energy is not a solver parameter.

AuthorsOpenReview ↗

response 1 (cont'd)

The new blue Appendix U, Table 7 (p. 61) completes the numerical documentation for the other retrieval experiments. It lists the original synthetic and leaf iteration budgets and update-norm thresholds, specifies their full geodesic steps and corrected backtracking rules, and documents the Euclidean/Hopfield controls and mode-verification tolerances. The iterative memory solvers evaluate kernel sums with no additive background (ϵ=0\epsilon=0 for score evaluation); a constant background does not affect these updates or acceptance tests. Figure 1 explicitly uses ϵ=1\epsilon=1 to display logarithmic energies. The appendix also distinguishes the synthetic corrected normalization from WordNet's radial weight normalization.

The paired sensitivity check in Appendix E.3 finds unchanged exact-recovery outcomes under tighter tolerances and larger budgets at the original radii. Some runs still reach the iteration limit; this numerical check is distinct from a convergence theorem. Appendix E.6 further documents finite-precision support-boundary effects, while Proposition E.1 states its exact-arithmetic scope.

Thus the revised presentation reports all three WordNet comparisons, provides a conditional corrected-recall guarantee, and documents the solver protocol. The poorer corrected performance in the other reported settings remains an explicit outcome of the comparative study.

AuthorsOpenReview ↗

response 2

W2a — Curvature on a given manifold versus learning curvature

The abstract states that curvature determines memory behavior and is a design variable, but the experiments use prescribed manifolds and do not select, tune, or learn their curvature.

Answer

Our claim concerns how the curvature of a given Riemannian manifold governs associative-memory behavior. This dependence is established theoretically and applies even when the geometry is fixed; see the existing Section 2.2, Theorem 2 (p. 5), and Section 4 (pp. 6–8). For example, Theorem 2 gives the exact isolated-pattern Hessians

=\frac{g_{\xi_i}}{1+\epsilon}, \qquad \operatorname{Hess}E_{\mathrm{vc}}(\xi_i) =\frac{g_{\xi_i}-\operatorname{Ric}_{\xi_i}/(3\beta)} {1+\epsilon}.$$ Thus, for a prescribed manifold and bandwidth, curvature determines whether an isolated pattern is stable under the corrected energy. The overlap analysis likewise characterizes memory formation through the geometry of interacting supports. The pattern configuration, kernel, and bandwidth also matter; curvature is not asserted to be the sole determinant of every memory state. The experiments examine these consequences on fixed manifolds and compare the two energy formulations. They do not evaluate curvature selection or learning. By “curvature is a design variable,” we mean that the choice of geometry has consequences for memory behavior and can inform the choice of energy and bandwidth. This is a design implication of the theoretical results. Combining metric or curvature learning with DAM is an interesting future direction: representation geometry could be optimized jointly with memory keys, queries, and bandwidths, using the stability and retrieval conditions as guidance. The present work provides an analysis for prescribed geometry; the benefits of such joint learning remain to be investigated. ## W2b — Fixed-dimension scope of the Poisson emergence law The abstract says random data at the storage threshold yield “only a Poisson number” of new modes. The limiting Poisson mean grows exponentially with dimension, and the theorem is proved at fixed dimension, so “only” does not imply few emergent memories in high-dimensional embeddings. **Answer** We agree that the abstract should state the asymptotic regime explicitly. Theorem 4, Eq. (22) (p. 7), treats independent patterns with a continuous density on a fixed manifold, with $h\to0$ and $N^2q_h\to2\lambda\in(0,\infty)$. Conditional on retaining all original patterns, the number of emergent memories at novelty scale $\tau h$ converges to a Poisson distribution with parameter $$\lambda_\tau=(2^m-a_\tau^m)\lambda, \qquad a_\tau=\max\\{1,2\tau\\},\qquad 0<\tau<1.$$ At fixed dimension, the conditional count therefore remains bounded in probability as $N$ increases in this scaling regime. This is not a dimension-independent bound. For $\tau\le1/2$, the limiting Poisson mean is $(2^m-1)\lambda$, which grows exponentially with dimension, as stated on page 8. We establish no emergence law for a joint limit in which dimension grows with $N$. The original “only” referred to the dependence on the number of patterns at fixed dimension; it did not justify a claim of few emergent memories in high-dimensional embeddings. We have removed that wording from the abstract and made the fixed-dimension regime and retention conditioning explicit in blue on **p. 1**: > At fixed dimension, we derive an explicit Poisson limit for random > emergence at the storage threshold, conditional on retaining all > originals. The blue revisions to the **Introduction's contributions paragraph (p. 3)** and **Section 6 (p. 9)** now use the same fixed-dimensional, conditional formulation. The positive contribution is an **explicit probabilistic characterization of the coexistence of storage and emergence**, for both the geodesic and corrected energies. The result identifies the full limiting distribution of the novel-memory count and gives the limiting probability $$e^{-\lambda}\bigl(1-e^{-\lambda_\tau}\bigr)$$ that all originals are retained and at least one novel memory exists (**Eq. (23), p. 7; proof in Appendix N, pp. 50–52**). It quantifies how dimension and the novelty threshold enter these predictions, providing a precise asymptotic description of when random data support new memories alongside the stored patterns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

Dear Area Chair,

We thank you and the reviewers for the careful evaluation. We would like to highlight the paper's main contribution and the additions made in response to the reviews:

  • Main contribution: a theory connecting curvature to associative-memory behavior. We compare geodesic and volume-corrected memory energies and characterize their stability, retrieval, capacity, and emergent modes. The results include a sharp Ricci-curvature stability criterion, memory guarantees for Riemannian mean shift, distinct all-pattern and typical-pattern capacity laws, constructions realizing all 2N−12^N-1 nonempty-subset modes, and an explicit Poisson emergence limit for random banks at fixed dimension, conditional on retaining all originals. These results explain when stored patterns and novel memory states can coexist.

  • Additional theoretical results. We added a proof of exact singleton recall and local contraction for the corrected hyperbolic update, a constructive example where correction suppresses an emergent attractor while preserving stored memories, and an analytical comparison of Epanechnikov, biweight, and triweight kernels. These additions clarify the retrieval benefit of correction and distinguish the effects of compact support, kernel shape, and curvature (Appendices E.4–E.7 and I.1).

  • Stronger WordNet comparisons and explanation. In 16-memory banks, the reported corrected exact-recovery gains over ambient Euclidean memory are 4.1–8.0 percentage points across three evidence levels. We now also report a common identification comparison: corrected retrieval has higher mean identification accuracy than geodesic nearest-neighbor retrieval at all three levels under both solver settings. The added activity analysis shows that queries with multiple active memories account for 91.2% of the net corrected-over-geodesic exact-recovery gain (Appendices E.3 and E.6).

  • Numerical robustness and reproducibility. All 552,960 paired exact-recovery outcomes are unchanged under the tested tighter tolerances and larger iteration budgets. We report termination and support-boundary diagnostics, document the numerical protocols, and include reproduction materials for the added experiments in the supplementary archive (Appendices E.3, E.6, and U).

  • New tests of the theoretical predictions. We added capacity experiments covering 109,000 independent random banks, a curvature-transition study using 2,000 random banks of 24 patterns, and experiments showing approach to the predicted limiting emergence mean. The revised presentation explicitly distinguishes these measurements and states the fixed-dimensional, conditional scope of the Poisson theorem (Appendices R–T).

We hope these additions help clarify the theoretical contribution and strengthen the evidence available for the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.