acceptodds
Under review as a conference paper at ICLR 2027

Fine-Tuning Dynamics of In-Context Factual Recall in Transformers

Abstract

In-context learning — performing tasks based on examples given in the prompt — is an important capability that has emerged in large language models and has received significant attention in both theory and practice. Existing theoretical work often focuses on settings where the learning uses information purely from the prompt. However, many practical instances of in-context learning require the model to retrieve factual knowledge stored in the model's parameters, with the context serving to identify which knowledge is relevant. In this work, we study how in-context learning leverages factual knowledge recall. We formalize this behavior by introducing the *in-context factual recall (IC-recall)* task, where a transformer is provided a context of (subject, answer) pairs generated from a hidden relation, along with a query subject, and must both infer this hidden relation and retrieve the corresponding answer. Factual knowledge is modeled by the transformer having access to a simple pre-constructed MLP associative memory storing (subject, relation, answer) triplets. We analyze the supervised fine-tuning dynamics of a one-layer transformer on IC-recall data and prove that the model successfully performs IC-recall by converging to a particular pairwise attention pattern. This fine-tuning stage requires a very small number of samples — only polylogarithmic in the number of stored knowledge triplets. Experiments verify our theoretical predictions and show that the pairwise attention pattern emerges even when the MLP layer is pretrained instead of constructed.

Primary area:
learning theory
Keywords:
transformer, fine-tuning, in-context learning, training dynamics, factual recall

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eRAGOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper formalizes in-context factual recall as identifying a hidden relation from subject–answer examples and then retrieving the answer to a new subject from an MLP associative memory. It analyzes fine-tuning dynamics in a one-layer transformer and reports synthetic experiments with constructed and pretrained memories.

Critical strengths

The paper isolates a useful question: how contextual examples select facts already stored in model parameters. The analysis gives a concrete attention mechanism and a sample-efficiency result under its stated assumptions.

Critical weaknesses

The main theoretical setting is far from the behavior the paper aims to explain. It assumes bijective relations, orthogonal token embeddings, an effectively complete frozen associative memory, and fine-tuning restricted to a positional attention block. The appendix reports a persistent generalization gap for a pretrained two-layer transformer. This substantially limits what the result establishes about factual recall in practical models.

The mechanism claim is stronger than the general theorem. The exact pairwise attention result comes from the three-subject analysis; Section 3.2 states that the near-optimal solution for general subject sets does not exhibit the exact pairwise pattern. The broader claim therefore rests largely on synthetic observations.

Questions for authors

Can the authors quantify when the pairwise pattern emerges in the general setting, and explain how the proposed mechanism accounts for the two-layer model’s generalization gap in Appendix A? Evidence on that point would materially affect my assessment.

2 comments

We thank the reviewer for the review.

Our theoretical claims concern a one-layer transformer, as stated in the abstract and theorem assumptions. Please see our global response for the scope of the paper. The two-layer experiments in Appendix A explore a possible extension; we do not claim that our theory explains their behavior.

The mechanism claim is stronger than the general theorem.

As explained in Section 1.1, we prove exact pairwise convergence in the three-subject setting and sample-efficient recall in the general setting. We also observe pairwise attention empirically beyond the three-subject setting.

I thank the authors for their detailed response. After considering their clarifications, I have decided to maintain my original score.

Reviewer_LvyVOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how a model infers a hidden relation from in-context examples and then retrieves facts already stored in its parameters. It constructs an MLP associative memory and analyzes fine-tuning a one-layer transformer under supervision that first predicts the relation and then the answer. The results include pairwise attention in a three-subject setting and a sample-efficient guarantee in a general low-temperature setting. Experiments also consider pretrained MLPs, additional demonstrations, and non-orthogonal embeddings.

Critical strengths

The question is meaningful. Many ICL tasks require both interpreting examples and accessing existing factual knowledge. Studying these processes together in a controlled task can help explain what fine-tuning changes.

The analysis of the symmetric saddle, subsequent symmetry breaking, and emerging pairwise attention provides substantive mechanistic insight. The pretrained-MLP experiments also make a useful attempt to test whether the phenomenon depends on the explicit construction. This is the paper’s strongest contribution.

Critical weaknesses

There is an important mismatch between the theoretical target and the experimental metric. Assumption 1 requires two demonstrations to identify a unique relation, whereas page 9 counts any predicted relation consistent with the demonstrations as correct. With multiple consistent relations, this need not recover the true relation or answer the query correctly. Moreover, the bijection setting implies |R|<= n(n-1) under this assumption, but Figure 5 uses n=8, |R|=64>56. The near-perfect experimental scores therefore do not yet establish success on the original target studied by the theory.

The scope of the sample-efficiency result and the advance over prior work need clearer positioning. The analysis assumes a fully populated factual memory, explicit relation supervision, and only a small number of trainable positional-attention parameters. These simplifications are understandable, but the result primarily concerns learning an access mechanism. Relative to existing contextual-recall work, the distinctive contribution appears to be pairwise attention and its optimization dynamics when relations share answers. The exact three-subject mechanism should also be distinguished from the weaker general low-temperature guarantee.

Questions for authors

How are the experimental relation sets generated? Is there a filtering step enforcing identifiability, and how should Figure 5’s relation count be reconciled with the assumption?

Could you separately report true-relation accuracy, context-consistency accuracy, teacher-forced second-step accuracy, and free-running end-to-end answer accuracy?

Which optimization difficulties or proof steps specifically arise from shared answers across relations, compared with existing contextual-recall analyses?

2 comments

We thank the reviewer for the review. Here are our responses.

Q1. We agree that our experiments do not strictly enforce Assumption 1. We generated relations as random bijections from S\mathcal S to A\mathcal A, without a filtering step. We therefore counted any relation consistent with all demonstrations as correct.

During the rebuttal period, we will rerun the experiments using relations sampled without replacement from the family of n(n−1)n(n-1) affine bijections over the finite field Fn\mathbb F_n, where nn is a prime power. This ensures that any two distinct subject–answer pairs identify at most one relation. In this setting, context-consistency accuracy coincides with true-relation accuracy. We expect the main findings to remain unchanged and will report the results.

Q2. Yes. Under the revised relation generation, true-relation and context-consistency accuracies coincide. We expect teacher-forced second-step accuracy to remain at 100%. If so, free-running end-to-end answer accuracy will be at least the true-relation accuracy. We will separately report them in the revision.

Q3. Shared answers allow a confusing relation that maps s1s_1 to a2a_2 and s2s_2 to a1a_1. Under symmetric attention, this relation receives the same logit as the correct relation. Stage 2 must break this symmetry to distinguish them, leading to the pairwise attention pattern analyzed in Section 3.1. Such a confusing relation cannot exist when different relations have disjoint answer sets.

I thank the authors for their detailed response. After considering their clarifications, I have decided to maintain my original score.

Reviewer_XSFmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces the in context factual recall (IC recall) task. The prompt contains two (subject, answer) pairs generated by a hidden relation r⋆r^\star and a query subject s3s_3. The model must first find out r⋆r^\star  and then the answer r⋆(s3)r^\star(s_3). Factual knowledge is modeled by a frozen, hand built MLP associative memory with one pair detecting neuron per (subject, answer) pair. Only the position to position block of WKQW^{KQ} is fine tuned, using a two step chain of thought loss with teacher forcing on r* and masking of irrelevant logits. Based on this construction, the author propose a theory of In-Context Factual Recall and validate it in a toy empirical investigation, showing that it provably learns to infer the hidden relation and retrieve the answer from very few fine-tuning samples.

Critical strengths

[S1] Understanding the inference process by which contextual information indexes relevant parametric knowledge is an important task, and developing a theory of this is a noble goal.

Critical weaknesses

[W1] The biggest weakness is that the authors don't provide much evidence for how these findings will relate to real LLM mechanisms, and many of their design decisions simplify their theoretical and experimental setting in such a way that limits their findings. As an example, the work assumes a bijection between all s and a. I'll note that this makes the examples in Figure 1 a bit misleading, since this implies that in their knowledge base, only Jack can be born in Boston, etc. While using natural language to represent entities and relations implies a broader scope, the reality is that the construction of the knowledge base is only testing whether models can learn to associate a subject and object according to an implicit relation. Moving beyond the simplification of the data, it's also difficult to map some of the design decisions and experimental setting (e.g., the choice of a quadratic activation function) to realistic implementation settings in modern LLMs. So, my concern is largely that while the authors may motivate their investigation with transformer-based LLMs and their in-context learning settings, I'm not sure the findings of their investigation generalize to much more than the specific experimental setting they've constructed. So while I appreciate the theory and the authors showing a toy empirical investigation that validates it, I don't think the authors have cleared the bar for convincing me the theory is a valid construct for how practical systems behave.

[W2] The construction of the MLP associative memory is not well motivated. During pretraining, language models are trained on next token prediction, so retrieving aa given (s,r)(s, r) is natural. The other two directions are much less intuitive, especially inferring the relation rr from (s,a)(s, a). This direction is exactly what the first decoding step of IC recall relies on, so the authors should explain why a pretrained model would be expected to store knowledge in this form.

[W3] The analysis depends on the explicit construction rather than on the property stated in Lemma 1. Lemma 1 only constrains the MLP’s behavior on sums of exactly two embeddings with unit weights. In IC recall, however, the MLP receives a weighted mixture of five or six tokens, and the property says nothing about such inputs. Every gradient computation relies on the closed form logits of the quadratic construction (Table 2, Eq. 18). The results therefore hold only for this one MLP, and the paper does not identify which of its properties are essential.

Questions for authors

[Q1] Does the pairwise pattern still emerge with ReLU or GELU activations, with a learned MLP, or when all of WKQW^{KQ} is fine tuned?

[Q2] What happens under nearly orthogonal embeddings, i.e., ∣⟨ϕ(u),ϕ(v)⟩∣≤ε|\langle \phi(u), \phi(v) \rangle| \leq \varepsilon for all u≠vu \neq v? How small must ε\varepsilon be relative to the margin 2(a−b)22(a-b)^2 for Theorems 2 and 3 to hold?

1 comment

We thank the reviewer for the review.

W1. We agree that our assumptions limit direct conclusions about practical LLMs. As explained in our global response, we study a controlled one-layer setting. We will make this scope and the illustrative role of Figure 1 clearer.

W2. Next-token prediction can supervise all three retrieval directions through different paraphrases. For example, “France’s capital is Paris,” “Paris is the capital of France,” and “France has Paris as its capital” place the answer, subject, and relation last, respectively. Our pretraining setup models this variation.

W3. We agree that Lemma 1’s retrieval property alone is insufficient for the dynamics analysis. We will present the construction first and use Lemma 1 to verify that it acts as an MLP associative memory. We will also state explicitly that the later theorems rely on this construction.

Q1. Yes. Figure 5 shows pairwise attention with pretrained MLPs. We also observe this pattern with ReLU activations and when fine-tuning the full WKQW^{KQ} matrix. We will add these results and specify the experimental settings in the revision.

Q2. We expect the two-stage dynamics and approximately pairwise attention to persist for sufficiently small ε\varepsilon. However, our proofs assume orthogonal embeddings, and we do not currently have a quantitative bound on ε\varepsilon. Figure 9 in Appendix A shows that approximately pairwise attention still emerges with non-orthogonal Gaussian embeddings.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global response: our scope and why we study a simplified model.

Dear Reviewers and Area Chair,

We would like to clarify our scope and explain why we study a simplified setting. The main contribution of our paper is theoretical. We study how a one-layer transformer learns to use context to retrieve stored factual knowledge on IC-Recall data. Even this simple setting presents nontrivial challenges. A rigorous end-to-end result needs to connect the optimization algorithm, initialization, and hyperparameters to the learned model and its test performance. In our work, this requires analyzing the two-stage fine-tuning dynamics, the Hessian near the saddle, and how attention weights evolve to escape the saddle and reduce the loss. To make this analysis possible, we focus on a one-layer model under simplifying assumptions.

Studying simpler models under idealized assumptions can reveal useful mechanisms and learning principles. For example, work on induction heads [1–3] helps explain how two-layer transformers gather information from context to complete sequences such as [A,B,…,A]→B[A,B,\ldots,A]\rightarrow B. In particular, Nichani et al. [3] show how the first attention layer learns the hidden causal structure of the data. In factual recall of subjects, relations and answers, Nichani et al. [4] analyze a one-layer linear attention model and identify an intermediate “hallucination” stage, where predictions depend only on the relation and ignore the subject. Similarly, studies of deep linear networks have helped explain learning plateaus and the role of initialization [5]. These studies provide insights by making the mechanisms and training dynamics precise in settings that can be analyzed rigorously.

Our work follows this philosophy. We study a one-layer transformer on the synthetic IC-recall task and identify a two-stage fine-tuning process. In the three-subject setting, the model initially approaches a symmetric saddle, where it cannot distinguish the correct relation from a confusing relation. It then escapes the saddle by learning a pairwise attention strategy that identifies the correct relation. For general subject sets, we establish high test accuracy with only polylogarithmically many fine-tuning samples.

Although our results do not directly apply to real LLMs, we hope our analysis helps the community gain useful insights into larger models. For example, attention may capture hidden causal structure not only through what to attend to, as highlighted by prior work on induction heads, but also through how much to attend. Our one-layer model illustrates this through equal attention weights within each subject–answer pair and different weights across pairs. At the end of the first stage, the model may predict a confusing relation, which resembles mixing up the associations among facts in context. Understanding how training resolves these mistakes through pairwise attention may help us study similar errors in larger models. Our results also suggest that the amount of fine-tuning data needs not grow strongly with the size of the knowledge base. This raises a possibility for larger models: fine-tuning data requirements may depend more on whether the data contain critical hard examples that teach the right strategy, such as the confusing sequences in our task, than on how much domain knowledge the model stores.

[1] Elhage et al. A Mathematical Framework for Transformer Circuits.

[2] Olsson et al. In-context Learning and Induction Heads.

[3] Nichani et al. How Transformers Learn Causal Structure with Gradient Descent.

[4] Nichani et al. Understanding Factual Recall in Transformers via Associative Memories.

[5] Saxe et al. Exact Solutions to the Nonlinear Dynamics of Learning in Deep Linear Neural Networks.

We have revised the paper and added some new experiments according to the reviews:

  1. We change the statement of lemma 1. Now the construction of MLP is proposed first, and lemma 1 verifies that the construction qualifies the MLP asociative memory.
  2. We enforce the identifiability assumption in experiments.
  3. We add experiment results on ReLU activations, fully fine-tuned attention layer WKQW^{KQ} to Appendix B.

All changes are in red for the ease of reading.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.