acceptodds
Under review as a conference paper at ICLR 2027

Learning a Fact Is Not Learning How to Retrieve It

Abstract

A model trained on “The capital of X is Y” may produce “Y” after “The capital of X is” but fail after “The capital of X:”. We call these different ways of eliciting the same fact request forms. To separate learning a fact from retrieving it, we train two models in two stages. In the first stage (request-form training), one model sees each fact in five forms and the other sees the same facts only as statements. In the second stage (target-fact training), both receive identical training on new facts, all as statements. Both then retrieve the new facts almost equally well from statements, but differ sharply on other request forms. Thus, a model can learn how to retrieve through a request form before it learns the facts. To understand this difference, we examine the hidden state immediately before the answer, which we call the context state. When given two different request forms for the same fact, the model trained on five forms in stage one produces more similar context states than the model trained on statements alone in that stage. Changing this state at retrieval time can enable or prevent retrieval of an already learned fact, and the same effect transfers across facts and factual relations, such as capitals and currencies. To test its role during learning, we change the context state only during target-fact training. This intervention changes later retrieval without intervention at test time. Together, these results show that later retrieval depends on earlier request-form experience and the context state during fact learning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
knowledge extraction, request format, training history, continued pretraining, format transfer, knowledge retrieval, prompt format sensitivity, instruction tuning, controlled experiments, language model fine-tuning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_pdRfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the separation between learning a factual association and learning how to retrieve that fact under different request forms. Through two-stage training with identical target-fact training but different earlier request-form exposure, the authors show that models can learn the same facts yet differ sharply in retrieval through held-out forms. They further identify a “context state” at the answer position as a link between earlier request-form training and later fact retrieval, and support this with context-state similarity analyses, patching, steering, and training-time state shifting across several models and factual relations.

Critical strengths
  • The two-stage design cleanly isolates earlier request-form exposure from target-fact training. Because the second-stage target-fact training is identical across conditions, the observed retrieval differences are easy to attribute to the earlier stage. This is the paper’s strongest conceptual contribution.

  • The paper validates the separation across several models, relations, and evaluation forms. The combination of correlational context-state analyses with intervention-style experiments strengthens the initial empirical claim beyond a single setting.

Critical weaknesses
  • The experiments use fictional country capital pairs, template based request forms, a fixed 2,000 candidate pool, and relatively small or open models. It remains unclear whether the conclusions transfer to natural text factual learning, larger models, or real fine tuning pipelines.

  • The paper shows that retrieval depends on the final words of the request, that the restoring direction overlaps with punctuation or suffix directions, and that transfer survives incorrect answers and different relations. This leaves open whether the context state encodes a general retrieval mode or mainly the request form and its wording. The residual support in Qwen also suggests additional components are not accounted for.

Questions for authors
  • Can you test this on natural text facts or on at least one larger model? If not, can you clearly state how far the claims are meant to apply?

  • Section 3.4 already shows that retrieval depends on the last few words of the request. Given that, can you show whether the context state direction still predicts retrieval when the last few words are held the same but the request type differs? This would help distinguish a format level state from a word level state.

  • Does the restoring direction remain effective when it is collinear with punctuation or suffix directions? In Qwen2.5-1.5B, removing it still leaves accuracy at 0.944. What explains this residual support? Are there settings where the direction fails or reverses?

  • Table 3 shows that transfer survives incorrect answers and different relations. To what extent does the context state mainly encode the request form/retrieval mode rather than the fact-answer binding? Could you add a stricter condition using the same five request-form templates but completely unrelated or randomly shuffled facts/answers, token-matched to the correct-unrelated condition, and test whether the same transfer and context-state similarity still appear?

3 comments
AuthorsOpenReview ↗

Response to Reviewer pdRf

W1 / Q1. Generalization to larger models and realistic factual learning.

We appreciate the reviewer's suggestion to test larger models and more realistic factual learning. In Qwen3.5-9B-Base, both training conditions learn the target facts almost perfectly (0.993 vs. 1.000), yet retrieval differs on unseen short key–value requests (0.052 vs. 0.604) and colon-ended declarations (0.687 vs. 0.998) (Appendix H, Table 7), across three seeds.

We also evaluated Qwen3.5-9B on CounterFact using 1,000 counterfactual edits involving real-world entities per seed across three seeds. Both conditions acquire the edited facts comparably well (editing-efficacy difference −0.001). Yet across six held-out non-declarative request forms, edited-answer preference accuracy improves from 0.600 to 0.885 (+0.285, positive in all three seeds) (Appendix H, Table 8), while greedy-generation accuracy improves by +0.365. In contrast, CounterFact's original declarative paraphrases show negligible improvement in edited-answer preference (+0.003).

These results extend the separation between factual acquisition and retrieval to a recent 9B model and real-entity counterfactual editing. The training inputs and evaluated request forms remain structured; we do not claim generalization to learning facts from unrestricted natural prose.

W2 / Q2. Evidence for a format-level retrieval mechanism.

We appreciate the reviewer's suggestion to distinguish broader request-form effects from the influence of the final tokens. We conducted a controlled experiment on Qwen3.5-9B, holding the final tokens (“capital of X:”) identical while varying the preceding context across eight request types, including questions, dialogue, instructions, lists, and quotations.

Despite identical endings, statement-only retrieval accuracy varies substantially across request types (0.000–0.780). Earlier diverse-form training raises mean accuracy from 0.408 to 0.948 (Appendix J); on the note, header, list item, and quotation wrappers, none of which begins with a prompt from earlier training, the mean rises from 0.640 to 0.998.

These results establish that the request ending, while influential, does not fully determine factual retrieval. Broader request structure and prior form experience also matter, even when the final tokens are held fixed. Together with our causal context-state interventions, this supports an account in which retrieval depends on the answer-position context state shaped by both local wording and broader request structure, rather than on the final tokens alone.

Q3. Residual support and bidirectional control of retrieval.

We appreciate the reviewer's question of whether retrieval restoration survives removing punctuation-related directions.

In Qwen2.5-1.5B, removing the original punctuation direction u from the restoring vector preserves substantial restoration (0.951, compared with 0.995 using the full vector, eight seeds). Thus, the causal effect is not confined to this single punctuation direction. Further analysis shows that Qwen's remaining effect is distributed across additional context-state directions: removing an eight-dimensional subspace derived from answer-free prompts with different endings reduces restoration to 0.395, comparable to a norm-matched random control (0.450) (Appendix N). These results distinguish the restoring effect from u alone, while showing that the additional directions overlap with a broader ending-related subspace.

The reverse intervention exhibits a clear form-dependent asymmetry. Subtracting the restoring vector abolishes retrieval on held-out list requests (0.603 → 0.000) and colon-ended declarations (0.482 → 0.000), but only partially suppresses the directly trained declarative form (0.999 → 0.559) (Appendix L). This is consistent with direct Stage 2 training making retrieval in the trained format more robust to perturbation, while retrieval through held-out forms depends more strongly on the transferable context-state component.

AuthorsOpenReview ↗

Response to Reviewer pdRf

Q4. Separating request-form learning from fact–answer bindings.

We appreciate the reviewer's suggestion to separate request-form learning from factual associations. Using the same five request forms and matched Stage 1 token budgets, we compared correct but unrelated fact–answer bindings, consistently incorrect bindings obtained by permuting answers across facts, and independently randomized answers on each training occurrence. All conditions received identical subsequent target-fact training.

Crucially, even with consistently incorrect bindings, context states for the trained declarative and unseen list forms remain strongly aligned (centered cosine 0.901), comparable to correct bindings (0.893). This alignment accompanies nearly identical retrieval accuracy on entirely new Stage 2 facts (0.990 vs. 0.994; Pythia-410M, three seeds) (Table 3). By contrast, independently randomized answers yield weaker state alignment (0.597) and retrieval (0.163), versus 0.383 and 0.000 under statement-only training.

Together with our causal interventions, these results provide evidence that prior request-form experience develops reusable internal conditions for retrieving new facts, rather than merely encoding previously learned factual associations.

Thank you for your response. It has resolved my concerns. I maintain the current score.

Reviewer_VxspOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether failure to retrieve a fine-tuned fact through a new prompt format reflects weak fact learning or a missing retrieval skill. It separates the two with a two-stage design. In stage 1, one model sees 1,000 facts as statements only and the other sees them in five request forms. In stage 2, both models receive byte-identical declarative training on 1,000 disjoint target facts. The main experiments use Pythia-410M with fictional country–capital pairs, with replications on Pythia-1.4B, Qwen2.5-1.5B, Llama-3.2-1B, five relations, and LoRA.

Critical strengths

The experimental design is clean. The causal interventions are well controlled.

Critical weaknesses
  1. Novelty relative to prior work is overstated. The central behavioral finding was already established: Allen-Zhu & Li (Physics of LMs 3.1) showed that facts can be stored but not extractable without augmentation, that mixed QA training generalizes extraction to held-out individuals, and that including augmented data for some individuals improves extraction for others. Jiang et al. (2024, PIT) used essentially the same two-stage ordering, QA exposure before document training, and showed it improves later knowledge absorption.

  2. Limited scale and realism. The primary model is Pythia-410M, all models are ≤1.5B base models, and facts are fictional. The effect visibly shrinks with stronger pretraining: Qwen2.5-1.5B already reaches 0.603 on the list form before any stage-1 training. Removing the key direction from the Qwen restoring vector still leaves 0.944, suggesting redundant pathways. This raises the question of whether the phenomenon matters at the scales and in the instruction-tuned models where knowledge injection is actually done.

Questions for authors

N/A

1 comment
AuthorsOpenReview ↗

Response to Reviewer Vxsp

W1. Novelty relative to prior work.

We appreciate the reviewer's comparison with Allen-Zhu and Li (2024) and Jiang et al. (2024). We agree that these studies established the behavioral importance of prior training formats. Our contribution is to identify and causally manipulate the answer-position context state linking earlier request-form experience to later factual retrieval.

Transferring this state restores retrieval on a held-out form (0.000 → 0.963) (Appendix L), even from an unrelated fact or relation, while transferring a non-retrieval state suppresses retrieval on the trained form (0.997 → 0.023). More importantly, intervening during factual learning improves subsequent unseen-form retrieval from 0.024 to 0.943 (Table 5), versus 0.100 for a norm-matched random shift, without inference-time intervention.

As an additional ablation, we compared five-form training with a token-matched statements-plus-QA condition. While prior work demonstrated the benefits of QA exposure for subsequent knowledge extraction, we examine whether this experience generalizes to unseen non-QA request forms. On the held-out list form, five-form training achieves substantially higher retrieval accuracy than statements-plus-QA training (0.994 vs. 0.242, three paired seeds) (Table 3). This indicates that QA exposure alone does not provide the same cross-format generalization as diverse request-form training.

New experiments show what earlier request-form experience teaches the model. Even with consistently incorrect Stage 1 fact–answer bindings, context states for the trained declarative and unseen list forms remain strongly aligned (centered cosine 0.901 vs. 0.893 with correct bindings) (Table 3). Retrieval of entirely new Stage 2 facts likewise remains nearly unchanged (0.990 vs. 0.994). Together with our causal interventions, this provides evidence that models learn reusable retrieval-supporting internal conditions beyond the correctness of earlier factual associations.

To make our contributions explicit, we have revised the Abstract, Introduction, and Related Work to emphasize three findings: the separation between factual acquisition and retrieval, a reusable and causally actionable context-state mechanism, and the ability to change future retrieval through interventions during factual learning without test-time intervention.

W2. Scale, realism, and residual retrieval.

We appreciate the reviewer's concern about scale and realism. In Qwen2.5-1.5B, list retrieval reaches 0.997 with diverse-form training, versus 0.603 with statement-only training. In Qwen3.5-9B, substantial differences persist on unseen short key–value requests (0.052 → 0.604) and colon-ended declarations (0.687 → 0.998) (Appendix H, Table 7), across three seeds.

The residual restoration after removing a single direction in Qwen (Section 4.3; 0.951) suggests retrieval support is distributed across multiple directions. We therefore formed an eight-dimensional subspace from hidden-state differences between answer-free prompts with different endings. It captures 97.9% of the restoring vector's norm. Removing the original direction alone leaves restoration at 0.951; removing the full subspace reduces it to 0.395, comparable to a norm-matched random intervention (0.450) (Appendix N). Thus, the causal retrieval component can be identified in a compact, low-rank subspace even when no single direction suffices.

We also evaluated Qwen3.5-9B on CounterFact, with 1,000 counterfactual edits involving real-world entities per seed across three seeds. Both conditions acquire the edited facts comparably well (editing-efficacy difference −0.001). Across six held-out non-declarative request forms, edited-answer preference accuracy improves from 0.600 to 0.885 (+0.285, positive in all three seeds) (Appendix H, Table 8), while greedy-generation accuracy improves by +0.365. In contrast, CounterFact's original declarative paraphrases show negligible improvement in edited-answer preference (+0.003, 95% CI [−0.012, +0.017]).

We additionally tested instruction-tuned Qwen3.5-9B using its chat template. Across nine request forms (one paired seed), the correct answer appears within 128 generated tokens in 0.073 of statement-only responses versus 0.518 after diverse-form training (Appendix I, Table 9); the first capital named is correct in 0.072 versus 0.518. Thus, earlier diverse-form training substantially improves retrieval under chat-template inference (Appendix I).

Together, these experiments extend the separation to recent base models and real-entity editing, and show a substantial retrieval gain under chat-template inference in an instruction-tuned model, while identifying low-rank retrieval-supporting components across models.

Reviewer_UY5JOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper analyses form-related generalisation of LLMs in the context of factual requests (e.g., requests related to capitals, currencies, populations) when models are trained on datasets with different levels of diversity. In particular, the authors propose a two-stage training setup and train one set of models on factual requests where the data form is aligned between training- and evaluation-time, and another set of models where training data is diversified. The authors experiment with four models in total: Pythia-410M and Pythia-1.4B, Qwen2.5-1.5B, and Llama-3.2-1B, and demonstrate the benefits of training on diversified data as part of a two-stage training setup.

The authors furthermore define the context state, which is the hidden state of a model right before providing the answer, and show that it transfers generalisability between the form-aligned and the form-diversified models.

Critical strengths
  • This paper provides a clear and focussed experimental setup, and is technically sound.
  • I appreciate that the authors mention implications and limitations of their work in the main manuscript.
Critical weaknesses

The paper's contributions are limited to a simple set of factual requests, and the authors experimented with outdated LLMs. It is unclear to what extent the proposed experimental design and findings are relevant for recent LLMs of similar size, which might generalise better to perturbations of form across such factual tasks. As such, I suggest to the authors to contextualise their experimental results into the state-of-the-art w.r.t. LLMs of comparable sizes.

Additionally, the experimental setup presented in this work is simple and straightforward, and topically limited to factual requests. To better demonstrate its robustness and applicability across domains, it'd be interesting to see similar experiments run on different topics and domains and on more relevant evaluation benchmarks of LLMs, to observe to what extent such data diversification during training might aid in evaluation task performance.

Questions for authors

N/A

1 comment
AuthorsOpenReview ↗

Response to Reviewer UY5J

W1. Generalization to recent language models.

We appreciate the reviewer's suggestion to evaluate recent models. We extended our experiments to Qwen3.5-9B-Base, Qwen3.5-2B-Base, and Ministral-3-3B-Base.

On Qwen3.5-9B, both training conditions learn the facts in the trained format (0.993 vs. 1.000), yet retrieval differs substantially on unseen short key–value requests (0.052 vs. 0.604) and colon-ended declarations (0.687 vs. 0.998) (Appendix H, Table 7), across three seeds. Similar gaps appear in Qwen3.5-2B and Ministral-3-3B.

We also replicated the causal mechanism in Qwen3.5-9B: transferring the answer-position hidden state from an unrelated, fact-free sentence (e.g., “The weather today is”) restores colon-ended retrieval (0.579 → 1.000) (Appendix L), whereas a random-state control provides virtually no recovery.

Together, these results demonstrate that factual knowledge and its retrieval remain separable in recent models, and that a fact-independent context state can causally restore access to learned knowledge.

W2. Generalization across factual domains and benchmarks.

We appreciate the reviewer's suggestion to examine broader domains and realistic benchmarks. Our original experiments cover five factual relations, capitals, currencies, populations, organization founders, and birth years, with retrieval accuracy on an unseen list format improving from 0.001–0.344 to 0.970–0.991 (Table 2) across eight seeds.

We further evaluated Qwen3.5-9B on CounterFact using 1,000 counterfactual edits per seed across three seeds. Both training conditions acquire the edited facts comparably well (editing-efficacy difference −0.001). Yet across six held-out non-declarative request forms, edited-answer preference accuracy improves from 0.600 to 0.885 (+0.285; positive in all three seeds) (Appendix H, Table 8), while greedy-generation accuracy improves by +0.365. In contrast, the improvement in edited-answer preference on CounterFact's original declarative paraphrases is negligible (+0.003).

These results extend the distinction between factual acquisition and cross-format retrieval to real-world entities, consistent with our account of a transferable retrieval mechanism rather than general paraphrase robustness.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank all three reviewers for their constructive feedback. We agree that prior work established that earlier training formats can improve subsequent factual knowledge extraction. Our contribution goes beyond this behavioral observation by identifying a causally actionable answer-position context state that governs access to learned facts. In particular, our interventions demonstrate not only that this state can restore or suppress retrieval at inference time, but also that manipulating it during factual learning changes the accessibility of newly learned facts without any intervention at test time.

We have substantially strengthened the empirical evidence by extending our experiments to recent 9B models. On Qwen3.5-9B, across three seeds, both training conditions learn the target facts almost perfectly, yet substantial retrieval gaps persist on unseen request forms. We further extend the finding beyond synthetic facts to real-world entities using CounterFact, where diverse-form training improves held-out cross-format retrieval by 28.5 percentage points (Appendix H, Table 8) despite comparable fact acquisition. Importantly, we also observe a substantial improvement in an instruction-tuned Qwen3.5-9B model under chat-template inference, with correct-answer retrieval increasing from 0.073 to 0.518 (Appendix I, Table 9). These results establish that the distinction between learning facts and learning to retrieve them remains relevant beyond the base models studied in the original submission.

Our additional mechanistic and controlled experiments also clarify what earlier request-form experience teaches the model. With matched training templates and token budgets, transfer remains nearly unchanged even when earlier fact–answer bindings are consistently incorrect (0.990 versus 0.994 with correct bindings). Diverse-form training also substantially outperforms a statements-plus-QA baseline on an unseen non-QA request form (0.994 versus 0.242) (Table 3). Furthermore, holding the final request tokens identical does not eliminate differences across request contexts, and our intervention analysis shows that the restoring effect in Qwen is not confined to the original punctuation direction. Together, these findings support reusable retrieval-supporting context states that cannot be explained solely by memorizing earlier fact–answer bindings or by a single punctuation direction.

We have incorporated these new results, experimental details, and clarified limitations into the revised manuscript and appendix. We believe they substantially strengthen the paper's novelty, mechanistic evidence, and relevance to contemporary language models.

AuthorsOpenReview ↗

A Note to All Reviewers

We sincerely thank all the reviewers for their thoughtful and constructive feedback. The suggestions and questions have led to a stronger and more general central argument. Looking at how much the paper has improved reminded us of the value of the review process.

We truly believe that the revised paper is much stronger now. We would greatly appreciate the opportunity to hear the reviewers’ thoughts again on how this work has evolved.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.