acceptodds
Withdrawn from ICLR 2027

How knowledge flows from memorization to utilization in LLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Humans can apply knowledge they have used before to new problems. Reasoning by analogy, they can also learn from this experience how to use other knowledge of the same kind, even knowledge they have memorized but never used. Do language models show the same forms of generalization? To study how models learn new knowledge without contamination from their pretraining data, we construct a synthetic world of fictional people, institutions, cities, and regions, described by 409 knowledge items that no model has seen before. We inject most of this knowledge through continued pretraining (CPT), teach models to reason with it through post-training, and track each knowledge item from injection to reasoning. We say that a model utilizes knowledge when it explicitly recalls the knowledge from memory as a step of its reasoning. We distinguish question generalization, utilizing knowledge seen in training questions to answer new questions, from knowledge generalization, utilizing knowledge that the model has memorized but never seen used in reasoning. Across four model backbones, CPT reaches 99.7% knowledge probe accuracy, yet CPT-only models explicitly recall less than 2% of the injected knowledge that reasoning questions require. Memorizing knowledge is thus not enough: models must also learn to recall internal knowledge during reasoning. Once models learn to recall, both forms of generalization emerge but remain limited. Recall also extends to knowledge the model was never taught, producing hallucination. We further propose Contextualized Utilization Encoding (CUE), which presents knowledge during CPT with diverse recall-style prefixes. Across four backbones and two synthetic worlds, CUE consistently improves knowledge generalization and also improves question generalization in three of four backbones. Our findings show that memorizing knowledge and learning to reason with it are distinct stages, and that how knowledge is injected shapes how it generalizes.

Primary area:
foundation or frontier models, including LLMs
Keywords:
New Knowledge Acquisition, Continual Pre-training, Knowledge Generalization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_iPeXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the flow from "knowledge injection" to "knowledge utilization" in LLMs. The main claim is that CPT allows the model to memorize injected knowledge almost perfectly, but does not automatically make the model recall/use this knowledge during reasoning; only SFT/OPD with explicit Recall demonstrations can open the "utilization gate." The authors further propose CUE, which adds recall-style prefixes to knowledge statements during CPT, improving KG and QG.

Critical strengths

The research question is important, and the idea is good. It attempts to study knowledge flow in LLMs by separating memorization and utilization.

Critical weaknesses

The experimental setup is not rigorous enough or somewhat strange. Details are in the Questions section.

Questions for authors
  1. Why choose an instruct model for CPT instead of a base model? How can one judge the impact of running CPT on a model that has already undergone post-training?
  2. Why is training conducted with a fixed number of updates? I understand that under this setting, different exposure rates correspond to different numbers of exposures per data point.
  3. Why is LoRA used for all training? The performance will certainly differ from full-parameter CPT/SFT. LoRA SFT is acceptable, but LoRA CPT is not a common practice. Is it possible that the model trained this way only supports probes with similar templates but does not truly internalize the knowledge?
  4. Could it be that the form of Rewrite SFT is somewhat strange, leading to poor results? What if one directly uses question-answer SFT without an intermediate reasoning chain, for example, "Q: What is the population of Maplewood? A: 3"? Based on my understanding of research in this area, after such training, for the soft-hidden knowledge defined in this paper, the model should be able to answer correctly.
  5. Could the effect of CUE simply be due to increased diversity in knowledge expression? It would be best to add a comparison experiment that rephrases knowledge without adding recall prefixes, or uses non-recall prefixes, so as to identify the true gain from the "recall" behavior.
  6. Is the evaluation of "utilization" overly dependent on explicit Recall? If the model internally uses knowledge but does not write out "Recall:", is it being underestimated? Can a measure of implicit utilization be added?
Reviewer_qzF4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how language models move from memorizing facts to explicitly recalling them during reasoning. It constructs a synthetic world with 409 knowledge items and separates knowledge introduced through continued pretraining (CPT) from knowledge demonstrated during post-training. This enables two evaluations: question generalization, involving previously demonstrated knowledge in new questions, and knowledge generalization, involving memorized knowledge absent from training-question dependencies.

Across four backbones, the authors find that near-perfect factual completion does not automatically produce explicit recall during reasoning. Recall demonstrations increase this behavior, but generalization remains limited and models also claim to recall withheld knowledge. The paper proposes Contextualized Utilization Encoding (CUE), which adds diverse recall-style prefixes during CPT and improves knowledge generalization across two synthetic worlds.

Critical strengths
  1. The experimental framework addresses an important question with useful control over knowledge exposure. Separating CPT exposure from post-training dependencies makes it possible to study transfer beyond the facts explicitly demonstrated during reasoning. Executable answer graphs also provide a concrete basis for identifying the knowledge required by each question.

  2. The evaluation goes beyond final-answer accuracy. Separating recall coverage, correctness, and relevance reveals meaningful differences between training methods. For example, demonstrations can increase recall while also increasing incorrect or irrelevant statements. These distinctions would be hidden by a single answer-accuracy metric.

Critical weaknesses
  1. Explicit recall does not establish actual knowledge use. SFT directly teaches models to verbalize remembered facts, so higher recall may partly reflect a learned response format. Likewise, low recall under search-enabled RL does not establish that answer rewards cannot teach internal knowledge use.

  2. CUE lacks controls that isolate why it works. The method changes prefix diversity, recall-related wording, and training token budgets. Comparisons with a fixed recall prefix, diverse neutral prefixes, and paraphrased facts under matched budgets are needed to establish its contribution beyond standard data augmentation.

  3. The exposure sweep does not isolate the effect of broader knowledge exposure. Increasing exposure also changes training-set size, repetition frequency, and which facts remain in the evaluation group. Moreover, the absence of a significant linear trend does not establish that additional exposure has little effect.

  4. The recall metrics need stronger validation. The LLM-based extractor lacks a reported manual accuracy assessment, and its handling of facts outside dedicated recall steps is unclear. Counting one correct set member as a correctly covered slot also limits how strongly QG/KG can be interpreted as successful knowledge recovery.

  5. The practical benefit of CUE remains unclear. The main comparisons do not systematically report final-answer accuracy, search usage, or incorrect recall alongside QG/KG. These outcomes matter because the paper itself shows that greater explicit recall does not necessarily produce better answers.

  6. Question in the title has a bigger scope of what is evidenced in paper. The paper partially answers its title by identifying training conditions that promote explicit recall of memorized facts. However, it does not establish whether this recall contributes to solving the task. It's therefore a useful behavioral hypothesis, while the broader interpretation of knowledge utilization requires stronger evidence.

Questions for authors
  1. How much of the CPT-to-SFT improvement survives changes to the response format? On questions requiring only CPT-visible knowledge, can you compare ordinary answering, explicit recall prompting, and search-disabled evaluation? Please also clarify how the extractor handles recalled facts outside dedicated Recall steps.
  2. Does CUE outperform simpler augmentation controls under matched budgets? A comparison with one fixed recall prefix and diverse neutral prefixes would help distinguish recall semantics, contextual diversity, and format alignment.
  3. Does the exposure conclusion hold on a fixed set of held-out knowledge items? Can you separate the effect of broader demonstrated knowledge from the reduction in repetitions caused by the fixed update budget?
  4. How reliable is the recall evaluator? Please provide a small, manually checked sample covering omitted facts, incorrect mappings, conflicting values, and set-valued knowledge, together with extraction precision and recall.
  5. What are CUE’s downstream benefits and costs? Across the existing paired runs, how do final-answer accuracy, search calls, and incorrect or hard-hidden recall change? These results would materially affect my assessment of the method’s significance.
Reviewer_AYZSOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how LLMs utilize knowledge acquired during pretraining. To avoid contamination from prior training data, the authors construct a synthetic knowledge graph and inject its facts through continued pretraining (CPT). They find that although models achieve nearly perfect accuracy on direct knowledge probes, they rarely explicitly recall the required facts during downstream reasoning. The authors evaluate several post-training methods, including SFT, on-policy distillation, and RL, finding that methods providing explicit recall demonstrations improve knowledge utilization, whereas RL with reward on final answers induces little recall. They also propose Contextualized Utilization Encoding (CUE), which augments CPT statements with diverse recall-style prefixes and improves correct recall during reasoning, particularly for facts never used in post-training questions.

Critical strengths

None

Critical weaknesses

Limited novelty of the experimental setup. Prior work has already used controlled synthetic datasets to study knowledge storage, extraction, and manipulation [1, 2]. The paper should clarify what its fact-level tracking and QG/KG distinction reveal beyond these existing frameworks.

Limited novelty of the memorization–utilization gap. The observation that LLMs can memorize facts yet struggle to retrieve or apply them is established in prior work [1, 2]. Moreover, prior work demonstrates that CoT during training and inference facilitates reasoning with stored knowledge [2]. The paper should distinguish its findings about explicit recall demonstrations from these earlier results.

Insufficient comparison with existing knowledge augmentation methods. Prior work shows that diversifying knowledge presentations during pretraining through paraphrasing and sentence shuffling improves subsequent extraction [1]. Synthetic continued pretraining similarly uses diverse generated text to improve parametric knowledge acquisition and downstream QA [3]. Although CUE introduces a specific recall-prefix strategy, comparisons against paraphrasing, diverse neutral prefixes, and a fixed recall prefix are needed to establish its incremental benefit and isolate the importance of recall-specific language.

References

[1] Zeyuan Allen-Zhu and Yuanzhi Li. 2024. Physics of Language Models: Part 3.1, Knowledge Storage and Extraction. ICML.

[2] Zeyuan Allen-Zhu and Yuanzhi Li. 2025. Physics of Language Models: Part 3.2, Knowledge Manipulation. ICLR.

[3] Zitong Yang, Neil Band, Shuangping Li, Emmanuel Candès, and Tatsunori Hashimoto. 2024. Synthetic Continued Pretraining. arXiv:2409.07431.

Questions for authors

In addition to the concerns discussed in the weaknesses section, I found several key terms and claims difficult to follow. For example, although “question generalization” and “knowledge generalization” appear throughout the paper, a concrete example explicitly contrasting the two would substantially improve clarity.

The paper would also benefit from substantial reorganization. Important quantitative evidence supporting central claims, such as the CPT-only results discussed in Section 4.1, is delayed to the appendix. While I recognize the page limit, the main text should prioritize the evidence needed to assess the paper’s core claims. Meanwhile, the introduction spans roughly two pages and interweaves motivation, experimental details, results, and contributions. Condensing and restructuring it could create space for essential evidence and make the paper easier to follow.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.