Imprint of Experience: How Language Shapes Concept Representations from Pretraining to Continual Learning

Language models learn about the world through linguistic experience. Different languages make different aspects of that world explicit, some routinely require speakers to mark when an event occurs or whether a referent is identifiable, while others allow such information to remain implicit in context. This raises a broader question: how does learning experience shape internal concept representations? We connect comparisons across five languages with controlled English training, tracing linguistic experience from pretraining to continual learning. English–Mandarin comparisons show that definiteness and classifier categories remain decodable without corresponding obligatory marking. Yet shared decodability conceals an experiential imprint. In late-layer representations, removing articles lowers full-space definiteness selectivity relative to matched random deletion while raising it in the leading four principal components. The influence of grammatical experience thus depends on which aspects of the representation are examined. Further training reveals a temporal dimension of this imprint. During article-free continuation, an earlier probe becomes less effective, whereas refitting extracts the distinction more successfully from the same updated representations. Together, these findings show that models can represent the same conceptual distinctions while carrying different imprints of linguistic experience. Experience shapes how concept information is organized, and continued learning changes how that information can be recovered.