- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies contextual recall: inferring the queried attribute type from in-context examples, then retrieving the query subject's stored value of that type. A two-layer transformer pretrained on synthetic single-subject sequences (PT), where type-specific grammar precedes each attribute, learns the facts but fails on in-context prompts (Finding 1). Finetuning on half of the subjects, with plain in-context sequences (ICL) or with one to five grammar tokens before each attribute (ICL+Grm), generalizes to the other half (Findings 2-3); after ICL+Grm finetuning, layer-2 representations cluster by type (Finding 4). In a one-layer attention-only model with explicit relation tokens, one or two population gradient steps turn a factual-recall construction into a contextual-recall one.
Critical strengths
-
The question is well posed, and the held-out-subject test probes it cleanly. Retrieving a stored fact through a relation the context only implies ties in-weights knowledge to in-context inference, and held-out subjects separate learning an access mechanism from memorizing new pairs. The synthetic generator also makes the controls I request below cheap to run.
-
The constructions are checked against trained models. In the reported runs, the trained one-layer heads match the constructed subject and relation heads, so the constructions describe what training found.
Critical weaknesses
-
Format and the pretraining data confound the explanations of Findings 1 and 3. Pretraining places each attribute after a long grammar sequence, whereas in-context prompts ask for it right after the subject. The paper attributes Finding 1 to reliance on grammar statistics (p.4 l.174-175). Yet on ICL+Grm sequences that restore the type-specific grammar at its pretraining length, the model at iteration 0 almost always outputs an attribute but picks the right type no better than chance (Fig. 10, left; Q1). These sequences are over three times longer than PT sequences, which confounds the test. Finding 3's claim that short ICL+Grm sequences "serve the same purpose as ICL sequences" (p.5 l.224) also assumes, without quantifying it, that one to five grammar tokens cannot reliably encode the type (Q3). On the data side, no pretraining sequence contains same-type attributes of different subjects, whereas base LLMs solve knowledge ICL tasks such as country-capital (Hendel et al., 2023; Todd et al., 2024). The paper cites Allen-Zhu & Li (Part 3.1) for the same held-out-subject result but leaves out their condition that extraction depends on augmented pretraining data. This generator's fresh grammar and randomized attribute order resemble such augmentation, which may be why finetuning succeeds here. The title credits finetuning with enabling in-context reasoning, but the evidence does not yet separate missing type inference from a format mismatch, or a need for finetuning from a gap in this pretraining data. I would ask for three things: (i) evaluating the pretrained model on PT-length ICL+Grm sequences with type-specific versus type-uninformative grammar; (ii) finetuning on ICL+Grm with an independent type per demonstration, so only the format can be learned; and (iii) discussing the Allen-Zhu & Li condition, then either narrowing the title or showing that adding a small fraction of multi-subject, same-type sequences to pretraining does not yield contextual recall without finetuning.
-
The Sec. 4 theory covers a different setting from Sec. 3, and its gradient-step proof contains errors as written. By design (p.6 l.317), the theory models only Findings 1-2, not the ICL+Grm recipe of Finding 3. Prop. 2 also requires every attribute value to appear during finetuning. Sec. 3, however, has as many type-1 values as subjects; if they map one-to-one, the held-out subjects' type-1 values never appear in finetuning, yet held-out accuracy is near perfect (Fig. 3; Q2). The two-layer model may therefore use a route outside the construction. Finally, Lemma C.1 bounds every output probability below one over the vocabulary size, although the probabilities must sum to one, and the step at p.25 l.1332 treats their sum as much smaller than one. Because the theory is the paper's mechanistic explanation, these gaps leave the mechanism behind the main recipe unestablished. The lemma and that step look repairable, but until they are corrected, Theorem 1 is unverified. (I checked only this lemma and step.)
Questions for authors
-
Is iteration 0 in Figs. 3 and 10 the pretrained checkpoint? Restricted to attribute tokens, does it pick the right type above chance?
-
How are attribute values assigned to subjects, and what is held-out accuracy per type?
-
What Bayes-optimal type accuracy do the Finding 3 grammar tokens allow, pooled over all subsequences in the context?
-
Why does held-out accuracy under the Finding 3 recipe fall to about a quarter with eight layers and to near zero with four heads (Fig. 12)? Figs. 3, 4, and 12 also appear to be single runs; do the main results hold across at least five seeds, with confidence intervals?
-
With disjoint attribute vocabularies, one demonstration identifies the queried type, and held-out accuracy saturates after the first (Fig. 6, left). Does the model aggregate information across demonstrations when one is not enough? Sharing a controlled fraction of attribute tokens across types would test this; until then, I would narrow the claim that the model "aggregates information from multiple examples" (p.6 l.276-277).
3 comments
W2: Regarding the theoretical results in Sec. 4
We thank the reviewer for the careful examination of our theoretical analysis. We address the concerns below.
Revised Lemma C.1 and related steps. Thank you for pointing out the error in the probability bound in Lemma C.1 and related approximation steps in the proof of Theorem 2. We have revised App. C.2 to address these issues. Specifically, Lemma C.1 now establishes , which implies . We use this bound to control the softmax contribution in the relation-head gradient approximation, and have revised the subsequent gradient approximations to ensure consistency throughout the proof.
Scope of the theoretical results. As noted in Sec. 4, our theoretical analysis considers a simpler setting to understand the role of PT and FT in enabling contextual recall, rather than providing a guarantee for the ICL+Grm finetuning procedure in Finding 3. Our analysis shows that under certain conditions, a few steps of GD starting from the PT construction can enable contextual recall. This allows us to concretely see how the subject-head mechanism learned during PT is retained, while the relation-head mechanism is repurposed during FT to infer the relevant attribute type from the in-context examples. We present this as a candidate mechanism, without claiming that it is the only mechanism through which the model succeeds on contextual recall.
Q2: Assignment of attribute values and generalization across attribute types
Thank you for the question. In our main experiments, we use subjects and attribute types, with unique attribute values for type 1 and for the remaining types. For type 1, each subject is assigned a unique attribute value. We have included results for ICL accuracy evaluated separately for each attribute type, for both seen and held-out query subjects (Fig. 12 in the updated version). We find that the model achieves nearly perfect accuracy on held-out subjects even for type 1, where the corresponding attribute values are never observed during FT.
Q4: Effect of model scale and robustness across initialization seeds
Thank you for these questions. As shown in Fig. 12 (Fig. 14 in the updated version), while all architectures achieve good factual recall after PT, their generalization to ICL sequences after FT is not monotonic with model scale. While we do not have a definitive explanation for this behavior, one possibility is that different models may learn different representations of factual knowledge during PT, some of which are more amenable to generalization after FT. Indeed, Fig. 24 (Fig. 26 in the updated version) shows that models achieving perfect factual recall can learn different representations, leading to different generalization performance after FT. While this does not directly explain the model scale behaviour, it suggests that the representations learned during PT may play an important role beyond factual recall accuracy alone.
Thank you for the suggestion on including results across multiple initialization seeds, for the main experiments. We will include those in the updated version of the paper.
Q5: Role of multiple demonstrations in attribute-type inference
Good question. We agree that in our setting, where attribute vocabularies are disjoint, a single demonstration is sufficient to identify the attribute type. Our main observation in Fig. 6 is that increasing the number of demonstrations leads to stronger clustering of the learned attribute-type representations, even when the accuracy has already saturated. Thus, our results characterize how the representation geometry evolves with the number of examples, rather than establishing that multiple demonstrations are necessary for identifying the attribute type. We agree that studying settings with overlapping attribute vocabularies, where multiple examples are needed for disambiguation, would be an interesting extension.
We hope these clarifications address your concerns and that you would consider increasing your score.
Thank you for your responses. The revised probability bound is a sensible correction. After carefully considering the paper and your responses, I have decided to maintain my score.
Thank you for your careful reading and detailed feedback. We are encouraged that you find our research question well-motivated and the held-out-subject evaluation useful, and appreciate the empirical validation of our theoretical constructions. We respond to your concerns below.
W1(i), Q1: Performance of the pretrained model on ICL+Grm sequences
Thank you for the question. We verified that the ICL+Grm sequences in Fig. 10 have the same length as the PT sequences (not 3 times longer), we have clarified this in the App. B in the updated version. As you pointed out, in this case, at the start of FT, the model always outputs an attribute token, whereas it takes a few iterations to get attribute type and exact match correct.
Our statement in lines 174-176 refers specifically to PT sequences, where the model learns to use explicit grammar statistics to infer the attribute type and retrieve the corresponding fact. This does not necessarily imply that the same ability transfers directly to ICL+Grm sequences. Although the grammar length and subsequence structure are the same, the overall data distribution differs: PT sequences contain a single subject, whereas ICL+Grm sequences contain multiple subjects. Nevertheless, the two settings share the underlying grammar-based attribute retrieval task, and Fig. 10 (left) shows that the model requires only a few fine-tuning iterations to achieve high attribute-type and exact-match accuracy on ICL+Grm sequences. We have revised lines 174-176 in the updated version to mention that the pretrained model fails to generalize to grammar-free ICL sequences, but not attribute this failure specifically to its reliance on grammar statistics.
W1(ii), Q3: Informativeness of short grammar sequences
Thank you for the suggestion about better isolating the role of FT on sequences that encourage adaptation to format change vs implicit inference. We conducted the proposed experiment, where we finetune on ICL+Grm sequences with short grammar length, but with attribute types sampled independently rather than shared across demonstrations. We find that the model learns to adapt to the new sequence format and predict attribute tokens, but fails to implicitly infer the relevant attribute type from the in-context examples. Consequently, ICL accuracy improves only slightly. This highlights the importance of shared attribute types during finetuning for learning the implicit inference mechanism. We have included these results in Fig. 11 of the updated paper.
Next, regarding your question about the Bayes optimal accuracy for ICL+Gram sequences, We would like to highlight the results in Fig. 9 (left), where we evaluate the finetuned model on ICL+Grm sequences with Sft=1. In this case, each grammar token is sampled uniformly, independently of the attribute type. Thus, even when pooling information across all subsequences in the context, the Bayes-optimal attribute-type accuracy using grammar tokens alone is exactly 1/L=12.5%. Nevertheless, the finetuned model achieves high accuracy in this setting, demonstrating that it can infer the attribute type from the in-context attribute tokens rather than relying solely on grammar statistics.
More generally, during finetuning, we sample Sft∼Unif({1,…,5}). Even assuming perfect grammar-based type inference for Sft≥2, the average Bayes-optimal grammar-only accuracy is upper-bounded by 51⋅81+54⋅1=82.5%.
This is a conservative bound, whereas the finetuned model achieves near perfect accuracy, further indicating that it uses information beyond grammar statistics alone. We will clarify this in the updated version.
W1(iii): Role of PT augmentation and necessity of finetuning
Thank you for the question. Regarding your comment about the role of PT augmentation in Allen-Zhu and Li (2023), we have clarified this aspect of their findings in App. A in the updated version.
Regarding the suggested experiment, we agree that alternative PT distributions, such as mixtures containing multi-subject, same-type sequences, with shorter grammar lengths, could potentially induce contextual recall without a separate FT stage. Our main goal is to understand how factual knowledge acquired during PT can be leveraged for contextual recall after FT, and the role of different types of data, and we use stage-wise PT and FT to study this in a controlled setting. Thus, we do not claim that FT is universally necessary, but rather that the PT distribution with same subject sequences is insufficient, while FT on sequences requiring implicit attribute-type inference enables contextual recall. We will clarify the scope of this claim in the updated version.