- Rating
- 3(+1)
- Confidence
- 1(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies exact parametric memorization in llm. The paper proposes an empirical power-law relationship between LoRA rank, memory length, and loss reduction. It examines token-level confidence and autoregressive recall failures, then introduces MemFT, which concentrates optimization on tokens below a confidence threshold. The empirical evaluation shows strong fits for the proposed loss-reduction formula and improvements over standard baselines.
Critical strengths
1.The distinction between teacher-forced loss and recall fidelity is useful. The token-level analysis exposes local failure points that average loss can conceal.
2.MemFT is simple and meaningful in this settings. Average gains across two models and two benchmarks suggest that the approach warrants further investigation.
Critical weaknesses
-
Eq (6) and sec 3.3 demonstrate strong fits to observed loss reductions, but do not clearly establish prediction on unseen settings. And, Section 4.1 shows that low average loss can coexist with poor recall, weakening the interpretation of this as exact memory capacity law.
-
eq (7) provide a sufficient condition for greedy correctness, but do not establish the optimality of the threshold used in eq(9). Table 2 mainly compares MemFT with SFT; comparisons against alternative thresholds and difficult-token weighting are needed to isolate the specific contribution of the proposed threshold-guided token optimization strategy.
Questions for authors
-
Were the reported fitting metrics computed on the same configurations used to estimate (C,\alpha,\beta,b)?
-
A threshold sweep determine whether 0.5 has a specific empirical advantage.
Thank you again for your thoughtful review, and we apologize for another brief follow-up. With only 30 minutes remaining before the discussion deadline, we are available to respond promptly to any further questions.
We hope our responses have addressed your concerns and would be grateful if you could consider updating your rating in light of the clarifications and additional results.
Thank you sincerely for your time and contribution to the review process.