acceptodds
Withdrawn from ICLR 2027

Same Knowledge, Different Use: How Training Pairings Shape Compositional Generalization in Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Fine-tuning can teach a language model to recall a fact without teaching it to apply that fact in a new problem. We study how the assignment of training examples contributes to this gap. Models first learn a recorded symbol for each of 384 fictional entities and learn to classify six-symbol sequences when all inputs are supplied. An application question supplies only the first five symbols and names the entity whose record specifies the sixth. The target is a category label denoting a range of the final-state probability under a fixed two-state hidden Markov model. We compare two training conditions using identical starting weights within each comparison. Each of 96 application-trained names receives eight different five-symbol contexts. In one condition, all eight contexts require the same answer for that name: SAME8 (8:0). In the other, four require one answer and four another: CROSS8 (4:4). We construct this difference by exchanging contexts between names with the same recorded symbol. The overall application material, labels, number of contexts per name, factual and explicit-task replay, and training updates remain matched. On contexts withheld from the compared application training, 4:4 improves accuracy by an average of 36.8 percentage points across three Qwen worlds, 38.4 points in Llama, and 39.8 points across two Phi assignments sharing one starting checkpoint and world. These gains do not accompany better factual recall. A subsequent Qwen comparison shows that 7:1 training already produces 29–36-point gains over 8:0. Its single minority-answer context is presented six times, and equivalence to 4:4 is not established. The original 4:4 advantage persists when both conditions' application labels uniquely identify the recorded symbol. Transfer to names without application examples is less consistent, and better ordinary application can coexist with worse performance when instructions require using a conflicting supplied symbol. Replacing selected feed-forward components between the original trained copies partially improves answers without identifying a unique composition algorithm. The results show that how the same available application material is assigned can substantially change generalization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, compositional generalization, supervised fine-tuning, knowledge use, training data design, shortcut learning, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_P55QOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how training examples affect language models' ability to apply learned facts. The authors construct a synthetic task that asks models to use a learned fact about an entity to fill in a missing symbol in a sequence, and then classify the completed sequence. They compare training settings in which each entity's questions are paired either with the same answer or with different answers. Across the evaluated models, pairing an entity's questions with different answers improves performance on held-out application questions.

Critical strengths
  • The paper describes its experimental setup and procedures in detail, supporting reproducibility.
Critical weaknesses
  • The paper has substantial presentation issues. The writing is difficult to follow, and the experimental design, results, and conclusions are not connected through a clear narrative.
  • The central idea appears insufficiently novel. Assigning the same answer to all application questions about an entity creates an obvious shortcut: the model can fit these examples by memorizing a fixed name-answer association. Showing that varying the answers improves generalization therefore seems unsurprising. Prior work has already discussed related issues [1,2,3]. I also find limited novelty in the experimental setting and methodology.
  • The paper reads more like an experimental report than a study offering meaningful new insights. Despite the extensive experiments, it is unclear what we learn beyond the familiar issue of shortcut learning or how the findings advance our understanding of generalization.

[1] Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models.

[2] Physics of Language Models: Part 3.1, Knowledge Storage and Extraction.

[3] Physics of Language Models: Part 3.2, Knowledge Manipulation.

Questions for authors
  • Are there any differences between the synthetic tasks in W1, W3, and W4? Are their examples drawn from the same distribution? What is the purpose for constructing separate worlds?
  • Regarding lines 197-198: Why are the BF16 checks introduced at this point? How are they related to the surrounding discussion?
  • Regarding 'Application improves without better recall': What relationship between application performance and recall is this comparison intended to establish? The improvement follows a change in training, while recall is another measured outcome rather than a demonstrated cause of the improvement. What insight does reporting these two outcomes together provide beyond describing the observed behavior?
  • What is the purpose of the internal intervention experiments in Section 5? Why replace MLPs rather than other components, such as attention heads? How do these experiments relate to the paper's main claim?
  • Regarding 'Better application can coexist with worse use of supplied inputs': This observation seems unrelated to the original motivation and main claim of the paper, but I find it interesting. Could the authors provide a deeper analysis of its underlying causes?
  • Beyond the familiar observation that models can learn shortcuts, what additional insights or contributions does the paper provide?
  • Why use an HMM rather than another rule-based method to construct the synthetic examples?
Reviewer_U9E9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the gap of performance between a language model recalling a fact, and applying that fact to a new question.

Critical strengths

None

Critical weaknesses

There is so much jargon in this paper, it's really hard to parse. I genuinely gave it a good shot to try and understand the intent behind this paper, but the presentation is really incoherent

Questions for authors
  • Why is this problem important? The abstract and introduction give a combined 4 sentences of discussion of what problem this work is trying to solve
  • Is there an example of the task you've designed? I'm assuming section 2.1 is intended to be the description of the task, but it's incredibly confusing
  • Line 78 ends the first paragraph with "We separately ask whether the model can recall Q, classify the sequence when Q is supplied, and classify it when the question only references the record". What is the "record", I didn't see that defined anywhere.
  • Some of the terminology used is very confusing. For example, Figure 1 refers to a "name" multiple times, what exactly is a name? I see it being used to describe "Same8" and "Cross8" in this sentence "Each numbered block is a different five-symbol context; each name keeps eight", but then the caption seems to refer to something different "Same8 (8:0) assigns one answer per name".
  • In Table 1 header: why are the headings called "All six inputs supplied" and "Recorded input omitted"? You've previously defined the capabilities as "EXPLICIT" and "DIRECT", and I'm assuming that's what these refer to, unless I'm wrong and this is actually a new concept?
  • I'm really struggling to parse a lot of the sentences in this paper. I've pulled out a few examples that are difficult. Maybe you can help clarify these?
    • Line 413 in the related work: "Correct factual answers need not support successful composition." Is the intent to say that successful composition can lead to an incorrect answer?
    • Line 462-464 of the discussion: "A large benefit appears before balance in this task, without establishing a universal exception threshold or an optimal allocation". This should be pretty important for a reader to understand since this is the discussion of your own results, but I genuinely have no idea what the intent behind this sentence is
Reviewer_TYN3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies why language models can recall a fact yet fail to apply it when solving a new problem. Models are fine-tuned to learn facts about fictional entities and to solve a task when all inputs are given, then tested on questions where one input must be filled in from a recalled fact. The authors show that whether models learn to combine recalled facts with the task depends on how fine-tuning examples are paired with entities. When every training question about an entity has the same answer, models learn an entity-to-answer shortcut and generalize poorly to new questions. When answers vary per entity, holding the overall training material fixed, accuracy on new questions improves by roughly 37–40 percentage points across Qwen, Llama and Phi, and a single differing example per entity already recovers most of this gain. The improvement partially transfers to entities never seen in application training for Qwen and Llama, but not for Phi. It also comes with a cost: models become less likely to follow a conflicting input supplied in the prompt. Internal interventions partially transfer the behavior but do not identify a specific mechanism.

Critical strengths

The experiment itself is neat, and the results are easy to understand. The control is clean, and the authors are also careful in reporting negative and unresolved results.

Critical weaknesses

The key issue with this paper is that it’s not clear.

(1) Confusing Presentation

The paper is filled with unexplained identifiers and internal terminology (W3, W1, W4, SAME8/CROSS8 vs 8:0/4:4 vs M0/M4, T/F/U/F+) that make it hard to read, parse, and make sense of the overall structure and argument.

(2) Unclear RQ and Contribution

The RQ (“can assigning the same available application material differently change whether a model uses its records successfully?”) is vague about what is being manipulated, and phrased in paper-specific terms that are hard to parse. It’s also not clear why this question is important.

(3) Supports Existing Findings, Lacks Depth

The experiment itself is neat, and it was pleasant to see the expected result pop out. However, this finding is not surprising given prior work (e.g., shortcut learning in Geirhos et al., 2020; knowledge manipulation in Allen-Zhu & Li, 2025). In SAME8, a constant name-to-answer mapping fits the training data perfectly, so it’s unsurprising that the model learns this shortcut instead of composing. CROSS8 prevents this shortcut, and we see that the model generalizes.

This is an interesting variation of fact-composition tasks (e.g., multi-hop composition in Press et al., 2023, and knowledge manipulation in Allen-Zhu & Li, 2025), where a recalled fact must be combined with a separately learned function over the supplied context. The transfer test on entities outside manipulation training is also interesting, though the results are inconsistent.

However, for an analysis paper, the work is quite shallow: it centers on a single manipulation, and most of the remaining evaluations are diagnostics of that one comparison rather than new questions. Since shortcut learning is already a known phenomenon, the contribution of this paper should focus on the nuances of this behavior in LLMs, rather than just confirming that it occurs. The few results that could add depth are inconsistent or unexplained, which makes it difficult to warrant a paper of its own.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.