acceptodds
Under review as a conference paper at ICLR 2027

Imprint of Experience: How Language Shapes Concept Representations from Pretraining to Continual Learning

Abstract

Language models learn about the world through linguistic experience. Different languages make different aspects of that world explicit, some routinely require speakers to mark when an event occurs or whether a referent is identifiable, while others allow such information to remain implicit in context. This raises a broader question: how does learning experience shape internal concept representations? We connect comparisons across five languages with controlled English training, tracing linguistic experience from pretraining to continual learning. English–Mandarin comparisons show that definiteness and classifier categories remain decodable without corresponding obligatory marking. Yet shared decodability conceals an experiential imprint. In late-layer representations, removing articles lowers full-space definiteness selectivity relative to matched random deletion while raising it in the leading four principal components. The influence of grammatical experience thus depends on which aspects of the representation are examined. Further training reveals a temporal dimension of this imprint. During article-free continuation, an earlier probe becomes less effective, whereas refitting extracts the distinction more successfully from the same updated representations. Together, these findings show that models can represent the same conceptual distinctions while carrying different imprints of linguistic experience. Experience shapes how concept information is organized, and continued learning changes how that information can be recovered.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_dp8HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examine how grammatical exposure affects the organization of information in LLM concept representations. It combines comparisons of independently trained Goldfish models across five languages and four data budgets with controlled English pre-training and continuation experiments using 125M-parameter models.

The paper finds that small classifiers can recover definiteness labels, such as the distinction between “a dog” and “the dog,” and a coarse numeral-classifier grouping in both English and Mandarin, despite differences in which distinctions their grammar requires speakers to mark.

In the experiments, removing “a/an/the” lowers classification performance using the full representation but improves it using its top four principal components, compared with matched random deletion. During further training on article-deleted text, an earlier classifier becomes less effective, while refitting the measurement pipeline gives approximately unchanged performance relative to the starting checkpoint.

Critical strengths
  • Experiments are well-designed. The article-deletion condition is compared with deletion of other non-article functional words, matched in deletion count and sentence-initial count. Architecture and tokenizer are fixed. The processed-token budgets for the two deletion conditions differ by at most 0.02%. Common-fork continuation restores the same training state before branching.
  • The authors include checks for shortcuts in the measurements and report unsuccessful behavioral experiments and other inconclusive analyses. This helps distinguish the supported findings from questions that remain open.
Critical weaknesses
  • The paper advertises a five-language investigation, but the main-text evidence focuses on English–Mandarin comparisons and English-only interventions. The results of other languages are mainly only in the appendix and receive little interpretation in the main text. Replication on another language or grammatical feature would strengthen the broader interpretation in terms of linguistic experience.
  • Predicting a word-related label is not the same as demonstrating a general concept. For example, the Mandarin evaluation uses explicit expressions such as "this dog" versus "a dog," rather than directly testing whether a bare word like "dog" refers to something already identifiable from context. The classifier cannot see the upcoming target, which is a useful safeguard, but familiar sentence patterns can still help predict its label. I would like a clearer separation between successful label prediction and the broader claims about conceptual knowledge.
Questions for authors
  • If I understand this correctly, this paper is trying to claim that even though Mandarin grammar does not require an English-style article system, a mandarin model’s internal states can predict the definite/indefinite labels used in this experiment?
  • Is there evidence that the controlled findings extend beyond English "a/an/the," or that they predict a difference in a task the model can successfully learn?
Reviewer_s5eyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how linguistic experience shapes internal representations in language models through cross-linguistic comparisons, controlled pretraining, and continual learning. The authors examine grammatical distinctions such as definiteness, numeral-classifier categories, tense, and evidentiality across five languages. They find that these distinctions remain linearly decodable even in languages without corresponding obligatory grammatical marking. Controlled pretraining experiments further show that removing English articles changes definiteness decodability differently in full hidden representations and low-dimensional PCA subspaces. Finally, continual-learning experiments suggest that previously trained probes can deteriorate while newly fitted probes recover comparable information.

Critical strengths

The paper addresses an interesting question about the relationship between linguistic experience and learned representations.

The controlled pretraining experiments, particularly the matched random-deletion baseline, provide a useful way to investigate the effects of grammatical exposure.

The distinction between the degradation of a fixed probe and the recoverability of information using a refitted probe is relevant to understanding representational changes during continued training.

Critical weaknesses
  1. Substantial overclaim regarding "concept representations."

My primary concern is the gap between the paper's broad claims about concept representations and the relatively narrow linguistic phenomena investigated. Although the paper includes additional analyses of tense and evidentiality, its central controlled experiments focus almost exclusively on English articles and definiteness, while the main cross-linguistic comparisons emphasize definiteness and numeral-classifier categories. These are specific grammatical distinctions rather than a representative set of general concepts.

It is unclear whether the observed effects generalize beyond these grammatical features to broader semantic or conceptual representations, such as objects, events, relations, causality, or abstract reasoning. Moreover, linear decodability of grammatical labels does not necessarily establish that a model has acquired an abstract concept or that the corresponding information is functionally used in semantic computation.

Consequently, the framing of the paper as a general investigation of how language shapes "concept representations" seems considerably stronger than what the experiments demonstrate. A more accurate characterization would be an investigation of how grammatical exposure affects the linear decodability and organization of selected linguistic features.

  1. Limited behavioral evidence connecting representational differences to meaningful language understanding.

The main conclusions rely heavily on linear probing and PCA-based analyses. Although the authors conduct activation interventions, these primarily concern preferences between grammatical continuations and do not establish whether the observed representational differences affect broader language understanding or generalization.

More importantly, the downstream behavioral experiments reported in Appendix A.8 are largely inconclusive. The paper therefore lacks convincing evidence that differences in grammatical-feature decodability translate into meaningful differences in model capabilities.

Can the authors demonstrate that these representational changes affect downstream tasks requiring genuine semantic understanding, rather than merely the prediction of grammatical markers?

  1. Unclear interpretation of the PCA-based decodability crossover.

One of the paper's central findings is that article removal decreases full-space definiteness selectivity while increasing selectivity in the top four principal components. While this observation is interesting, its conceptual significance remains unclear.

PCA identifies directions of high representational variance, which do not necessarily correspond to semantically meaningful dimensions. The comparison between a four-dimensional projection and the full representation also changes the dimensionality and effective capacity of the readout. Although the authors provide additional controls, the results do not establish a specific mechanism through which grammatical experience reorganizes conceptual information.

Why should increased decodability in the leading four principal components be interpreted as evidence about concept representation rather than a change in the statistical geometry of hidden states? What is the functional significance of this crossover?

  1. Limited novelty of the continual-learning findings.

The observation that a fixed probe deteriorates while a refitted probe retains its performance is closely related to established findings on representational drift and representation forgetting. The paper itself acknowledges this literature.

While applying this analysis to grammatical exposure is a reasonable extension, the conceptual contribution beyond existing observations about feature drift and decoder instability appears limited. The paper would benefit from a clearer explanation of what fundamentally new understanding of continual learning is provided by these experiments.

Questions for authors

The controlled experiments rely on 125M-parameter GPT-2-style models, with only three to five training seeds per condition. It remains unclear whether the findings extend to modern larger language models, different architectures, or other grammatical systems.

Furthermore, the cross-linguistic analyses involve heterogeneous corpora, tokenizers, and evaluation sets. Some comparisons also use relatively small samples, such as only 254 balanced examples for Mandarin definiteness. These factors make it difficult to draw strong conclusions about cross-linguistic representational similarities or differences.

Could you provide stronger evidence that the reported phenomena are robust beyond the specific model family and experimental settings.

Reviewer_jDenOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the hidden states of language models trained on corpus of different languages to analyze what is common across languages and what differs. They study one phenomenon : definiteness, the ability for a model to distinguish between "a" and "the" ; for languages that do have these markers (english) or don't have these markers (eg Mandarin). They show that a linear probe on hidden space recovers definiteness for LLMs trained on Mandarin only. this means besides the concept not being explicit for that language, a LLM is still able to internally learn the concept.

Critical strengths

Good ablations on how LLMs learn the same concept (definiteness) on different languages (EN vs Zh) with performance scaling with pretraining data.

Critical weaknesses

The paper's terminology is very hard to follow, it is full of NLP specific terms that are not explained and are not familiar for a ML audience.

The experimental results are hard to follow , eg one table reports IIA, there is no definition of interchange intervention accuracy in the paper. Same thing for definiteness which is the main concept studied in the paper. Selectivity is losely defined, without explaining what it means.

Overall very hard to follow, a lot of the figures use captions that go into too much detail, as if the reader was familiar with all bespoke notations introduced by the paper.

Experimental results are interesting but do not look novel. The observation that models learn abstract concepts (like tense, definiteness, or semantic class) even when a language lacks overt morphological markers has been shown repeatedly.

Questions for authors

NA

Reviewer_Q79wOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how linguistic features are represented internally in language models. Specifically, it first examines whether models trained on languages with different grammatical properties (e.g., the presence or absence of articles or numeral classifiers) internally represent the same conceptual distinctions. For example, it investigates whether English models encode numeral-classifier categories and whether Mandarin models encode definiteness information despite the absence of obligatory articles. The paper then examines how manipulating these grammatical features during training changes the internal representations of the corresponding concepts.

Critical strengths

The experimental results on how grammatical exposure (the presence or absence of articles during training) affects readout performance are interesting. It is intuitive that preserving articles would improve full-space selectivity (whether the concept exists), given the direct article-output direction discussed in Appendix A.9. However, the finding that removing articles actually leads to higher selectivity in the top four principal components (table 2) than preserving them is counterintuitive and particularly interesting.

Critical weaknesses
  1. Despite the interesting finding above, the paper does not sufficiently investigate the mechanism underlying why removing articles improves selectivity specifically in the top-4 PCs. To establish the broader significance of this result, it would be important to examine whether similar effects occur beyond English article removal. For example, removing numeral classifiers from Mandarin training data and analysing the resulting changes in internal representations, particularly in the top-4 PCs, would provide valuable evidence. As it stands, it is difficult to determine whether this finding is a statistical outlier (although the authors report variance across five seeds) or a phenomenon specific to a particular grammatical feature in a particular language.

  2. (Relatively minor) Although not directly overlapping, there are relevant prior studies that are not cited in the paper. For example, Zhang et al. (2025) [1] investigate internal representations and processing mechanisms when different languages employ the same or different morphosyntactic processes. Their experiments include past-tense processing, which is also discussed in the paper, where English and Mandarin express the same temporal distinction despite having different grammatical requirements. Given this overlap, I believe the paper's positioning of its contributions and its discussion of related work should be revised to better acknowledge and distinguish these prior findings.

[1] Zhang, Ruochen, et al. "The same but different: Structural similarities and differences in multilingual language modeling." ICLR 2025

Questions for authors

(All are about table 2) Would the grammatical-exposure results remain consistent if the number of training seeds were increased substantially beyond the current five? Also, would a similar decodability crossover be observed if numeral classifiers were removed from Mandarin training data? In particular, would classifier-category selectivity in the top-4 PCs increase relative to a corresponding control condition, as observed for English article removal?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.