- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper focuses on studying memorization with a fine-grained look at training dynamics. In particular, the authors ask which learning contributions along the dimensions of parameters, data, and training process characterize memorization, and whether such understanding is actionable for prediction and intervention.
To this end, they resort to a recently proposed decomposition framework that maps a model's loss trajectory over sequences onto its influences along training, parameters, and data. Such decomposition derives an intuitive mathematical interpretation of how memorization is the result of these three aspects. Particularly, they focus on memorization of two categories of sequences: highly frequent training sequences, called recitation, and rare training sequences, called recollection. Prior work shows that recitation is the dominant mode of memorization, but it is still unclear why their memorization requires deduplication, while recollected sequences do not.
They find that a large fraction of memorization happens in very early pretraining before basic generalization has manifested. The decomposition suggests two aspects of memorization: a sequence's alignment with parameter updates from its own occurrences in training and its interaction with remaining training influences. Highly duplicated recited sequences exhibit stronger negative alignment with both other training data and weight-decay updates. Recollected sequences, in contrast, exhibit less negative alignment.
Critical strengths
The most interesting aspect of the paper is the decomposition framework (proposed by prior work) on studying memorization. The paper also has sufficient experiments.
Critical weaknesses
Although the paper looks at training dynamics to study memorization, the findings seem non-surprising. What are the implications of these results? The authors have shown that recited and recollected strings, when decomposed, yield different results—isn't it expected and could be derived without any decomposition?
Also, the central premise of using an extractable notion of memorization is questionable. Given a prefix of a string, its suffix can be extracted to some extent without having the same string in the training data (see counterfactual or contextual memorization). How do you expand your analysis to these measures where learning and not direct training can still leads to extraction?
References:
Questions for authors
What would be the implications of your results? Can we design training where memorization is reduced yet generalization is improved?
The paper is strong in the technical aspect. Where I feel it lacks is on motivation? It is not immediately clear why looking at training dynamics for memorization is needed. What results or misconceptions of the prior works does the new decomposition reveal? Is decomposition the only way to answer?
4 comments
My impression is the rebuttal is performed in an incomplete manner. There was plenty of time to address raised concerns (e.g., implications of the work, additional related work), and I find no action taken by the authors in updating the paper. I therefore decrease my score.
-
The motivation of a paper cannot be "many prior works have established the memorization problem, and hence we make an epsilon step towards it."
-
Verbatim memorization is highly criticized for exaggerating the threat of memorization. Many works show that LLMs can generate a text verbatim without having seen them during training. So, the choice of memorization measure should be well justified as the field becomes more mature in considering what definition of memorization makes sense or not.
Liu, Ken Ziyu, et al. "Language models may verbatim complete text they were not explicitly trained on." arXiv preprint arXiv:2503.17514 (2025).
I agree with Reviewer jz6X. I look forward to author's detailed response with full experimental results and justifications.
My impression is the rebuttal is performed in an incomplete manner.
We are sorry to hear that our rebuttal did not meet the level of detail you were expecting. We tried our best to stay brief as is encouraged by ICLR.
and I find no action taken by the authors in updating the paper. I therefore decrease my score.
We are sorry to hear this as well. Given the extensive amount of edits required to incorporate all the feedback, we decided against updating the paper and instead discussed them here.
The motivation of a paper cannot be "many prior works have established the memorization problem, and hence we make an epsilon step towards it."
This is a misrepresentation of our motivation and contribution.
Verbatim memorization is highly criticized for exaggerating the threat of memorization.
We would like to clarify that our motivation is not “memorization is a threat”.
Many works show that LLMs can generate a text verbatim without having seen them during training. So, the choice of memorization measure should be well justified [...].
Which we do as we explain above, building on a recently proposed taxonomy that arguably takes into account the phenomena you are describing. We are sorry the result of this careful deliberation did not convince you.
as the field becomes more mature in considering what definition of memorization makes sense or not.
We strongly agree that a discussion about the definition of memorization is necessary and we also discuss this in the paper. But we would like to raise that no such consensus on a definition for memorization has been reached in the community. We, therefore, argue this is part of a larger discourse on finding such consensus to which we think our work contributes
Thank you for the feedback.
We would appreciate more details on this. Which of our results did you not find surprising? What previous work has reported them or due to which rationale would one assume them?
How would one show that their decomposition is different without performing said decomposition? Again, we would appreciate which concrete results this refers to and which concrete arguments would derive them without our decomposition.
The problem we look at is well motivated by previous work such as Biderman et al. (2023) and Prashanth et al. (2025), published at NeurIPS 2023 and ICLR 2025 respectively. Our contribution is explaining its dynamics, rather than simply finding differences between frequency groups (line 133–135: “how the rest of the training influences work towards or against memorization solutions.”). Our analysis uncovers several aspects of verbatim memorization that are not reported in previous work and which, as we show, indicate actionable routes for prediction and intervention, both, a priori and post hoc.
We agree that extractability does not establish counterfactual or contextual memorization and will discuss this more explicitly. Our definition of verbatim memorization is explicit in lines 158–160 and follows Prashanth et al. (2025)’s taxonomy. They also include a third type they coin “reconstruction” which is defined to include repetitive or template cases, i.e. some common cases of contextual memorization. As we describe in lines 160-161, we did our best to exclude these examples.
We agree that the understanding we gained through this work should be used to improve models in practice. Prediction and memorization ablation, while we mainly use it to validate our insights, have practical value (see, e.g., Biderman et al. (2026)). Given that most verbatim memories are subject to forgetting, which we show to be a causal result in our new training experiments (described in the comment to reviewer cNrf), we show that deduplication is one of the avenues to avoid memorization of undesirable sequences. We will extend our discussion with these points.
We would appreciate your feedback on whether your concerns have been addressed. We are happy to provide any further clarifications. Thank you.