- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates the influence of lexical similarity on the semantic representations learned by large language models. The authors analyze how this influence varies across network layers, model architectures, and training regimes, using several benchmarks and probing techniques. They also investigate possible connections between lexical sensitivity, intrinsic dimensionality, and semantic performance, and explore the practical implications for summarization evaluation and model editing.
Critical strengths
The paper addresses an important research question: understanding how lexical similarity influences semantic representations in language models. This topic is relevant to a broad audience and has potentially important practical implications.
Some of the results, particularly those concerning the effects of lexical similarity on summarization evaluation and model editing, are potentially interesting and deserve further investigation.
Critical weaknesses
Weakness 1: Lack of a coherent scientific narrative. The paper presents a large number of experiments addressing different aspects of lexical and semantic representations, but their motivations and connections are often unclear. The analyses do not appear to be organized around a well-defined research hypotheses, and results are presented without explaining what specific conclusions can be drawn from them. As a consequence, the manuscript reads more like a collection of loosely connected observations than a systematic investigation.
Weakness 2: Insufficient evidence supporting the main conclusions. Several interpretations appear stronger than the numerical evidence. In particular, the proposed connection between lexical influence, semantic discrimination, and intrinsic dimensionality remains largely correlational, and the depth-dependent profiles are not always consistent across models and benchmarks. Moreover, failures in semantic discrimination are frequently interpreted as evidence of lexical influence, although other explanations are possible. Overall, the paper does not convincingly demonstrate the underlying mechanisms of the phenomena it describes.
Questions for authors
Could the authors formulate more explicitly the main scientific hypothesis of the paper, and explain which experiments provide the strongest evidence supporting it? In particular, what is the central new conclusion that emerges from combining the different analyses?
How can the authors distinguish failures caused specifically by lexical overlap from more general failures of semantic discrimination? Would it be possible to introduce controls in which lexical and semantic similarity are varied independently?
2 comments
Response to reviewer. (2)
Weakness 2: Insufficient evidence supporting the main conclusions. Several interpretations appear stronger than the numerical evidence. In particular, the proposed connection between lexical influence, semantic discrimination, and intrinsic dimensionality remains largely correlational, and the depth-dependent profiles are not always consistent across models and benchmarks. Moreover, failures in semantic discrimination are frequently interpreted as evidence of lexical influence, although other explanations are possible. Overall, the paper does not convincingly demonstrate the underlying mechanisms of the phenomena it describes.
How can the authors distinguish failures caused specifically by lexical overlap from more general failures of semantic discrimination? Would it be possible to introduce controls in which lexical and semantic similarity are varied independently?
The lexical influence experiments in § 3 are explicitly designed to examine how lexical similarity affects semantic discrimination. Each triplet contains an anchor, a meaning preserving paraphrase, and a semantically different distractor with substantial lexical overlap. Success requires the paraphrase pair to be closer than either pair involving the distractor. Thus, the evaluation specifically tests whether lexical cues distort semantic similarity rankings rather than interpreting low semantic task performance as lexical influence. Separately, § 4.2 evaluates broader semantic fidelity through MTEB, where several tasks also exhibit intermediate weakening. Appendix H.2 further examines the role of lexical overlap in MTEB retrieval, showing that it affects absolute performance while largely preserving the depth dependent trajectory. These evaluations therefore examine distinct but related aspects of representation quality.
To further isolate the effect of lexical overlap, we construct a controlled subset from 5,000 CounterFact records. For each anchor, we select two distinct paraphrases preserving the anchor entity and two neighborhood distractors referring to the same different entity. We retain records where both the paraphrase (P) and distractor (D) have High and Low lexical overlap variants, guaranteeing that P High has greater overlap than D Low and D High has greater overlap than P Low. These filtering criteria yield 1,929 eligible instances. This design preserves the semantic relationships within each pair while varying their lexical similarity to the anchor. The results for Gemma instruct are provided in the table below. Complete results across Gemma and Llama models are provided in a new Appendix § M.
Table: Success Rate under independently controlled paraphrase (P) and distractor (D) lexical overlap.
| Layer | (P High, D Low) | (P Low, D Low) | (P High, D High) | (P Low, D High) |
|---|---|---|---|---|
| 1 | 100.0 | 98.4 | 99.2 | 89.8 |
| 4 | 99.7 | 97.0 | 98.7 | 85.5 |
| 8 | 99.8 | 98.0 | 98.8 | 87.8 |
| 12 | 99.6 | 95.8 | 95.6 | 75.4 |
| 16 | 98.1 | 89.6 | 89.1 | 66.7 |
| 20 | 96.9 | 85.6 | 86.2 | 66.0 |
| 23 | 95.2 | 83.4 | 82.7 | 61.3 |
| 26 | 96.3 | 85.8 | 84.3 | 63.9 |
| 30 | 99.3 | 95.6 | 93.9 | 82.3 |
| 36 | 99.8 | 99.0 | 98.8 | 93.8 |
| 40 | 99.8 | 99.4 | 99.5 | 96.9 |
| 44 | 99.8 | 99.0 | 99.3 | 96.5 |
| 48 | 99.6 | 98.1 | 98.4 | 94.2 |
The results show a consistent effect of lexical overlap across depth, particularly at intermediate layers. At layer 23, success is highest (95.2%) when paraphrase overlap is high and distractor overlap is low, and lowest (61.3%) when these conditions are reversed. The two remaining conditions yield similar success rates of approximately 83%.
Differences in depth-dependent profiles are expected given variations in model architectures, training objectives, representation readouts, and benchmark characteristics. Our conclusions do not require identical trajectories across these settings. Rather, we establish the persistence of lexical influence across models and adaptation regimes, while identifying a recurring intermediate regime in the autoregressive models studied. In this regime, discrimination weakens despite progression through model depth, before recovering in later layers. This pattern is not confined to the adversarial stress tests. Several MTEB tasks exhibit similar intermediate weakening or stagnation (§ 4.2), while OLMo checkpoints show that the regime emerges early during pretraining and persists through subsequent training (§ 3). This provides evidence for a recurring depth-dependent phenomenon rather than an isolated benchmark effect. The manuscript explicitly documents differences across models, architectures, and benchmarks (§ 3, Appendices B and D).
Regarding geometry, we acknowledge that the correspondence between this regime and stalled or partially contracting intrinsic dimension is correlational, as already reported in § 5. The spectral interventions provide further evidence that representation geometry plays a role in semantic discrimination.
Response to reviewer. (1)
Weakness 1: Lack of a coherent scientific narrative. The paper presents a large number of experiments addressing different aspects of lexical and semantic representations, but their motivations and connections are often unclear. The analyses do not appear to be organized around a well-defined research hypotheses, and results are presented without explaining what specific conclusions can be drawn from them. As a consequence, the manuscript reads more like a collection of loosely connected observations than a systematic investigation.
Could the authors formulate more explicitly the main scientific hypothesis of the paper, and explain which experiments provide the strongest evidence supporting it? In particular, what is the central new conclusion that emerges from combining the different analyses?
Our objective is to systematically characterize how lexical influence persists and evolves across model depth and training regimes, along with its representational characteristics and downstream consequences.
Our findings show that lexical influence persists even when semantic discrimination improves through adaptation or learned readouts. The intermediate regime further reveals that reduced accessibility of token identity can coincide with weakening semantic discrimination, demonstrating that reduced lexical accessibility does not necessarily lead to semantic robustness. While [1] and [2] investigate linguistic abstraction and representation quality across depth, our work focuses on a distinct problem by systematically characterizing failures in semantic similarity arising from lexicality, their relationship with representation geometry, and their consequences for downstream applications.
The strongest evidence comes from § 3 and § 4, which establish the persistence of lexical influence across training regimes and readouts, and its relationship with lexical decodability and semantic fidelity. § 5 extends this investigation to representation geometry, identifying associated changes in intrinsic dimension and demonstrating that spectral interventions can modestly improve semantic discrimination. § 6 examines the downstream consequences of lexical influence in summarization evaluation and model editing locality.
The combined analyses establish that improvements in semantic representation quality do not necessarily translate into robustness to misleading lexical cues. The manuscript already develops these connections and their conclusions within the respective sections. We will further make the research questions and main findings clearer in the Introduction.
[1] Emergence of a High-Dimensional Abstraction Phase in Language Transformers
[2] Layer by Layer: Uncovering Hidden Representations in Language Models