- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies retrieval metastability in continuous log-sum-ReLU dense associative memory with random spherical patterns. The authors propose that retrieval loss occurs through an entropic barrier into a two-pattern centroid state, followed by further centroid hopping. Using the exact spherical overlap distribution, they construct a multi-channel Kramers model. Within this model, the disorder-averaged escape rate is proportional to , up to subexponential factors, where is the large-system threshold for another pattern to lie within the target pattern’s kernel support. Monte Carlo simulations examine escape probabilities and finite-size effects. The authors interpret the apparent retrieval boundary as a finite-time kinetic crossover rather than a thermodynamic transition.
Critical strengths
S1. A useful distinction between retrieval stability and finite-time retention.The paper addresses a discrepancy between a theoretical retrieval threshold and finite-size simulations. The interpretation based on the lifetime of a metastable retrieval state is physically meaningful. It clarifies what a finite observation window measures. This insight remains useful even if the kinetic model is approximate.
S2. A geometric analysis that produces specific predictions. The barrier calculation and the two-pattern equilibrium analysis connect geometry with observable alignments. The saddle-point analysis balances the number of escape channels against their barriers. This gives a scaling prediction for the disorder-averaged escape rate within the proposed model. It goes beyond a qualitative explanation of metastability.
S3. Targeted numerical investigations. The authors compare all-pattern and nearest-pattern escape detection. They also test the sensitivity of the escape criterion and examine finite-size effects. A single calibrated prefactor is used across different conditions. Although the agreement is limited, these experiments provide useful support for parts of the proposed mechanism.
Critical weaknesses
Overall, I find the physical interpretation interesting and the geometric analysis useful. However, the current evidence does not sufficiently support the central dynamical claims.
W1. The dynamical exponent needs further justification. The free-energy calculation in Eq. (9) does not establish an exact one-dimensional reduction of the Metropolis first-passage problem. The escape-rate model in Eq. (18) requires additional dynamical assumptions. The union bound supports escape suppression if the per-channel estimates are valid. It does not establish the predicted exponent for the disorder-averaged escape rate of the full dynamics.
W2. The predominance of two-pattern states needs more direct evidence. Equation (10) describes equilibrium with only two active patterns. Agreement with the observed alignments does not establish that such states dominate the dynamics. Direct statistics of active-pattern counts and residence times would strengthen the centroid-hopping explanation.
W3. Quantitative agreement is limited. The model uses measured diffusion coefficients and one calibrated prefactor. Agreement is mainly obtained at low load, while escape is overpredicted at higher load. At and , the escaped fraction decreases approximately as , versus in the model over the tested sizes. The test at covers only three sizes and mainly supports slower escape as increases. These results do not rule out the asymptotic prediction, but they do not yet validate its exponent.
Questions for authors
Q1. What assumptions connect the support-entry barrier to the rate of confirmed escape? Please distinguish results established for the Metropolis dynamics from predictions of the rate model. What analytical or numerical evidence supports treating returns to retrieval and inter-channel correlations as prefactor corrections?
Q2. What numerical evidence supports the predominance of two-pattern states after escape? Active-pattern counts at a few representative settings would help assess how often states with three or more active patterns occur.
Q3. How well does the predicted survival curve match the existing escape-time data? A comparison at a few representative settings, with uncertainty estimates, would be useful.
3 comments
Thank you for the substantial revisions and for correcting the simulation issues. The added active-pattern statistics support the centroid-hopping mechanism. The revised finite-size tests and survival curves also substantially address my concerns about the quantitative evidence.
The new escape-probability bound strengthens the theoretical contribution, although it does not establish that the actual dynamics attains the predicted exponent. Given the revised numerical evidence, I consider it reasonable to assume that the remaining dynamical corrections are subexponential in . In this context, I do not regard the absence of a rigorous proof of this assumption or an analytical determination of as a major concern. Please state this assumption explicitly when applying Eq. (14) to the Metropolis dynamics at fixed load and temperature, while retaining the distinction between the bound and the rate-model prediction.
Overall, the geometric analysis and improved numerical validation provide a useful account of retrieval metastability. My main concerns have been sufficiently addressed, and I am raising my rating from 2 to 3.
Thank you for the reassessment and for raising the rating.
We will state this assumption in Section 3, after the bound: "Applying (14) to the Metropolis dynamics at fixed and assumes, beyond this bound, that the escape rate of the dynamics differs from the rate (13) by a factor subexponential in . The simulations support this assumption."
Response to Reviewer eX2K
We thank the reviewer for a careful and technically precise review. We agree with W1 to W3, and the revision addresses each of them.
W1. We agree that Eq. (9) does not reduce the Metropolis first-passage problem exactly to one dimension, and that the union bound with the Kramers rate of each channel, Eq. (18), rests on this reduction. The revision replaces the Kramers rate by the probability that one Metropolis step, taken in the Boltzmann equilibrium of the retrieval basin, brings a given spurious pattern into support. For a state started in that equilibrium, the escape probability within t steps is at most t times the sum of these probabilities (Levin and Peres 2017, proof of Thm. 7.4), whatever the correlations between channels. Each probability follows from the equilibrium distribution and the Metropolis proposal, with no fitted constant, and equals e−ΔF/T of Eq. (9) up to a factor polynomial in N. The disorder average of the sum is exact by linearity and has the exponent of Eq. (14). This exponent is therefore an upper bound for the Metropolis dynamics itself (App. B). The rate of Eq. (13) adds that escape reaches this exponent, with the prefactor A calibrated on the simulations.
W2 and Q2. App. C now gives these statistics for 1000 trials at each of (α,T)=(0.20,0.40), (0.22,0.325), (0.22,0.40) and (0.24,0.30). After the first escape, three patterns are in support during 0.2 to 4.1% of the time and four or more during at most 0.03%. Two patterns are in support during 33 to 75% of the time, and one pattern, the target or a spurious pattern, during the rest. In 99.4 to 100% of the hops the new two-pattern state shares one pattern with the previous one, so the transitions are pair-to-pair. The median residence in a two-pattern state is 2750 to 18400 steps, and the alignment there matches Eq. (10) with median deviation 0.002.
W3. The over-prediction at high load came mostly from the error in the confirmation test described in the general comment. The revision counts escape at the barrier crossing at all points of Fig. 3, with new runs, which is the event that the rate of Eq. (13) describes (see Q1). The measured diffusion coefficient sets only the time unit of the rate, and A is calibrated at one point and then kept fixed for Fig. 3, Table 1 and App. C. At α≥0.26 measured and predicted probabilities now differ by at most 0.03. Over the 25 points of Fig. 3 with predicted probability between 0.01 and 0.99, the median deviation is 0.016 and the largest 0.087. Table 1 also comes from new runs with the same count, read every 64 steps. The submitted table counted a trial as escaped when its target alignment at the end of the run was below 0.86, which also counts thermal dips below this level, and these dips flattened the measured decay at large N. Now the measured escaped fraction is 0.9 to 1.1 times the predicted one at all twelve entries, and 0.94 to 1.26 times with A calibrated at the same 64-step interval. Its decay with N is e−0.107N against e−0.111N predicted at T=0.40, and e−0.114N against e−0.119N at T=0.25. We agree that α=0.28 shows the direction of the N dependence rather than the exponent. Escape there takes at most one relaxation time, so it is not a rare event. The runs now cover six sizes, N=35 to 56, and the median time to the barrier crossing grows from 560 to 3420 steps (the submitted times were the first drop of the target alignment below 0.5). The decay with N predicted at these sizes is tested at α=0.20.
Q1. Escape is now counted at the barrier crossing ϕμ≥φc itself, so the count needs no assumption that connects the crossing to a confirmed escape. At α=0.20, counting escape instead when the spurious alignment first reaches the target alignment lowers the escaped fraction by at most 0.031 at every entry of Table 1 and changes its slope in N by less than 0.002. Returns therefore change the prefactor and not the decay with N. Correlations between channels do not enter the bound of App. B, and the recounted data agree with the model without a correction for them. Independent of the rate of Eq. (13) are the threshold αth (exact density), the barrier of Eq. (9) (Boltzmann equilibrium) and the upper bound of App. B (Metropolis dynamics). The rate itself, its prefactor A and the kinetic boundary come from the rate of Eq. (13) and are tested in Fig. 3, Table 1 and App. C.
Q3. App. C compares predicted and measured survival curves at eight settings, with bootstrap bands for the data and the calibration band of A for the model. Over the whole window the two curves differ by at most 0.05 at seven settings and by at most 0.10 at the eighth, (α,T)=(0.26,0.25).