acceptodds
Under review as a conference paper at ICLR 2027

Structured Dots in the Sentence Embedding Space

Abstract

Embedding spaces allow us to manipulate texts as mathematical objects, most frequently as points or vectors paired with distance metrics as proxies of linguistic properties, mainly similarity or relatedness. Viewed through this perspective, the embedding space appears anisotropic, and thus not able to encode linguistic differences accurately enough, despite empirical proof that the embeddings are useful for a variety of tasks. We propose that these observed shortcomings do not reflect properties of the embeddings themselves, but of the shallow measures such as cosine, which consider each dimension separately. By comparing the relative positions in the sentence embedding space of three sentence representation variations, and their performance on a variety of tasks, we show that sentence embeddings are complex objects with internal structure, which encodes linguistic structure in a systematic manner.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, sentence embeddings, syntactic structures

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nuv6OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper examines how geometric similarity relates to the linguistic information encoded in sentence representations. The main experiments compare the [CLS] embedding, mean-pooled token embeddings, and a randomly selected token embedding from four pretrained encoders. The evaluation combines cosine similarity analyses across six languages, 216 linguistic probing tasks in FlashHOLMES, and structural probing on the CN and SSR datasets. The main finding is that representations with low cosine similarity can support similar probing performance. In some settings, a structural encoder-decoder trained on one representation also transfers to another without retraining, with particularly strong results for BERT and ELECTRA. The authors interpret these findings as evidence that sentence embeddings have internal structure that raw cosine similarity does not adequately capture.

Critical strengths

The fixed-probe transfer in Section 5 is the most interesting observation. A structural encoder-decoder trained on one representation can work on another without retraining, even when their direct cosine similarity is low. The BERT and ELECTRA results in Figure 5 and Appendix Table 7 provide stronger evidence than similar scores from separately trained probes. They show that some of the tested linguistic information is accessible to the same learned mapping across representation types.

Critical weaknesses

1 The central explanation is not established by the experiments The Abstract, Introduction, and Conclusion attribute the apparent shortcomings of sentence embeddings to shallow similarity measures and argue for a particular account of their internal organization. These are central contribution claims. Section 5 defines encoding in a similar manner through transfer of a fixed probe, and the experiment does address that operational definition. The later inference about where information is encoded, and the broader explanation of why cosine gives a distorted picture, require evidence beyond successful transfer. A shared decoder can recover the same labels from inputs whose information is organized differently. For example, two representations could share a task-relevant linear subspace while differing in large background components orthogonal to it. Their direct cosine could then be low, while the same projection recovers the shared information. This is a simple alternative explanation that the experiments do not rule out. The t-SNE plots cannot resolve the issue either: they show the output of a learned compression, not the locations of features in the original embeddings. Appendix B.3 makes this distinction particularly important. On STS-B sentence pairs, the [CLS] and mean-pooled cosine scores have Spearman correlations of 0.70–0.81 for DeBERTa and 0.80–0.83 for ELECTRA. This suggests related geometric information despite low direct cross-representation cosine. The paper needs to test whether centering, standardization, dominant-component removal, or linear alignment explains the observations. The Euclidean-distance plots in Appendix B.4 do not isolate these possibilities. The broader limitation of raw similarity measures is also not new: Timkey and van Schijndel (2021), already cited in the paper, show how dominant dimensions can obscure representational quality. The present experiments would need to establish what they reveal beyond such effects. They currently support a narrower observation about shared accessibility of linguistic information, without establishing the proposed explanation or localization claim. This gap affects the main scientific contribution, rather than only the wording of the conclusions.

2 The remaining empirical contribution is insufficiently established The structural-probing method has direct precedents in the cited work of Nastase and Merlo (2023, 2024). The 2023 study already examines two-dimensional sentence embeddings, and the 2024 study uses the 32 × 24 CNN/VAE-style probe with a five-dimensional latent representation and contrastive structural examples. Section 5 appropriately acknowledges reuse of the earlier setup and data. The present comparison of [CLS], mean-pooled, and random-token representations, especially fixed-probe transfer, extends those studies. Its contribution therefore depends on what new and well-supported conclusion this comparison establishes. The narrower transfer finding still needs stronger validation. DeBERTa's SSR F1 scores in Figure 5 are 0.13–0.15, including when training and testing use the same representation type, despite its stronger morphology and syntax results in Section 4. The tasks differ, so this does not identify the cause. Section 5.2 suggests a mismatch between DeBERTa and the probe, but does not test it. Reshaping 768 coordinates into a 32 × 24 array imposes a particular neighborhood structure for the CNN. A simpler MLP-based encoder-decoder would help assess how much the results depend on that choice. The near-perfect BERT results on CN in Appendix Table 7, including transfer involving randomly selected tokens, also need a clearer generalization test. Contextualized tokens can contain sentence-level information, so high scores are plausible. However, the random split does not establish transfer to unseen lexical items or surface templates. The target structural categories are intentionally shared across splits; the concern is overlap in how those categories are instantiated. Without controlling that overlap, the results do not show how far the transfer extends beyond this dataset construction. The checkpoint selection further limits comparisons across models. Section 3 combines multilingual BERT and XLM-RoBERTa with English-pretrained DeBERTa and ELECTRA in the six-language analysis. Differences in language coverage, tokenization, and pretraining data can affect these comparisons. The discussion should separate these factors from effects attributed to architecture or pretraining objective. The model descriptions also need to match the checkpoints. Footnote 1 identifies microsoft/deberta-v3-base, but Section 3.1 describes the original DeBERTa training setup when contrasting it with ELECTRA. DeBERTa-v3 uses replaced-token detection rather than the original masked-language-modeling objective. On p. 3, the next-sentence-prediction discussion also cites Liu et al. (2019), although RoBERTa removes that objective, as the manuscript itself notes on p. 4. Please clarify which objectives apply to each model and use those distinctions when interpreting the representation differences.

Questions for authors
  1. Central explanation. What evidence distinguishes the proposed account of internal organization from representation-specific offsets, dominant components, or a shared linear task subspace? How do the results change after controls such as centering and standardization? Please clarify whether the location claim concerns the original embeddings or only the learned latent space, and what evidence supports it.
  2. Transfer validity. Does the cross-representation transfer persist with a simpler probe and splits that control lexical and surface-template overlap? What explains DeBERTa's low SSR performance, and which conclusions remain supported across models? These checks are needed to establish the scope of the transfer finding.
  3. Incremental contribution. Please distinguish the new findings from the structural information already recovered in Nastase and Merlo (2023, 2024). Once the conclusions are limited to what the current experiments establish, what additional scientific claim is supported by the cross-representation comparison?
5 comments
AuthorsOpenReview ↗

Central explanation, results across languages and contribution.

Thank you for the comments. We will address the questions in order below, and will add a few clarifications related to issues raised in the "Critical weaknesses" section, which we include with the question they are related to, or at the end:

1a. (central explanation) We use as input raw sentence embeddings obtained from the last layer of each model. We do not normalize or recenter them on purpose, as we aim to understand how and what information is encoded in these raw representations. The use the CNN architecture is specifically to go beyond linear probing to more complex patterns. A FFNN has a maximum field, so to speak, allowing for all combinations of dimensions, whereas the CNN is more restrictive, and rewards repetitive patterns. The latent clusters are presented as an added proof that structure was distilled from sentence embeddings. None of the experiment scores (F1s) are based on the latents. The F1 scores are computed based on the choices at the output of the VAE, among the out+ and the sentences in Out-

1b. (decoder recovering labels) The decoder does not recover a label. The output of the decoder is a sentence representation, that is then compared to the candidates. Through the use of contrastive examples, we assumed – and the experiments confirm it – that by using as a positive outcome a sentence that has the same pattern as the input sentence but is not the same, and use negative examples that have different patterns, will force the latent space to encode the shared information, i.e. the patterns.

2a. (Transfer validity) The aim in using the CNN architecture was to show that there are repetitive patterns in the sentence embeddings, which the high results confirm. Whether encoder&decoders using only FFNNs will lead to higher results or not does not invalidate the results obtained with the more constraining architecture. We would like to reiterate that pattern information is not provided to the system, which only manipulates sentence embeddings, both as input, and as the output produced by the decoder and the candidate answers.

2b: (DeBERTa) DeBERTa treats token and position embeddings differently than the other three models, and this may explain the difference in performance. It’s low performance though is useful, because it shows the results are not due to a very powerful probe – all experiments are done with a VAE that has the same architecture and the same hyperparameters. Probing how DeBERTa embeddings differ is the subject of a different investigation.

  1. (incremental contribution) The motivation of this work was to provide new insights into the geometric space of sentence embeddings. Without sections 3 and 4, that explore commonly held assumptions about the embedding space, the results in section 5 can be construed as reflecting the close position of CLS, AVG and T-Rand embeddings in this space. The cross-representation experiment is to show a profound result – that the shallow view of embeddings as point or vectors in a geometrical space hides the fact that these embeddings are actually complex objects with internal structure.

Additional explanations to issues raised in the "Critical weaknesses" section:

  • (recentering/filtering of embeddings) Our aim was to study the raw embeddings, without any kind of transformation, to understand what the model encodes in them and how they are similar or different. The progression of experiments and the methods used serve to show that, and to counter some alternative explanations (e.g. using both cosine and euclid to investigate the relative positions of the embedding variations in the space, using tasks grouped by the type of information they probe for).

  • (the tSNE plots vs location of features) The cross-representation testing does show that the same information is encoded in the same way in the different embedding variations. A model learned on one representation type is applied on all the others (each row on the heatmaps shows the results obtained by using a model trained on representations indicated on the y-axis and tested on the others with no modifications). Should the information not be encoded in the same manner across representation types, either or both the indicators – the F1 scores and the latent projection – would show different results. But that is not the case.

  • additional results: Section 5, a sentence that appears in the input in the training data does not appear as input in an instance in the test data. The results on a similar dataset for French shows the same profile as for English, with the F1 lower by approx 0.02. We have used gemma-4-31b-it to generate similar datasets in English and French with wider range of lexical variation. Unofficially (manual check of the datasets is ongoing), the F1 results profiles are similar (for French for Electra, all F1 are between 0.91 and 0.96 for all cross-testing set-ups, for English they are slightly lower 0.83-0.86), and further support our hypothesis.

Thanks for the response. The rebuttal clarifies that the author's intended claim is about raw embeddings and that Section 5 is meant as evidence of shared accessibility of structural information. However, it still does not establish the stronger claim about where/how information is organized in the representations, nor rule out simpler geometric explanations. It also does not adequately address robustness to probe architecture, the DeBERTa anomaly, novelty beyond prior probing work, or the checkpoint/objective-description issues. I would maintain the score.

Thank you for considering our answer. We respectfully disagree with some of the assertions:

robustness to probe architecture -- we disagree that the syntactic-semantic structure we probe for should be accessible through any probe. We show that it is accessible through a probe that relies on finding repetitive patterns. The fact that DeBERTa does not encode information in the same manner as the other three models does not invalidate our study and conclusions, but provides a starting point in understanding the impact of disentangling position and token embeddings within the architecture.

where/how the information is encoded -- our results show conclusively that the structural information we mine for is encoded the same way across variations. This is the important finding that in conjunction with the shallow (i.e. dimension-level) analysis shows the complex nature of sentence embeddings. This is an important step towards distilling, or making more overt, symbolic structures encoded in the continuous sentence embeddings. Localizing this type of information does not have a bearing on this particular investigation, although work on this specific topic has been done, but we suppress the citation for anonymity reasons.

Reviewer_qzzbOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper examines whether cosine similarity reliably reflects shared linguistic information in sentence embeddings. Using BERT, RoBERTa, DeBERTa, and ELECTRA, it compares CLS embeddings, mean-pooled token embeddings, and embeddings of randomly selected tokens. Results on FlashHOLMES reveal a mismatch between geometric similarity and task performance: even nearly orthogonal representations can perform similarly. Additional probing experiments show cross-representation transfer of probes targeting phrase structure, grammatical number, and semantic roles for some models. The authors take these results as evidence of shared internal structure that is not adequately reflected by cosine similarity.

Critical strengths
  • To my knowledge, the observation that CLS-based sentence embeddings are nearly orthogonal to AVG- and RAND-based embeddings in the evaluated DeBERTa and ELECTRA models is novel.
Critical weaknesses
  • I am not convinced that the limitations of cosine similarity demonstrated in this paper constitute a sufficiently novel contribution. Evaluating sentence embeddings beyond pairwise cosine similarity has been standard practice for years. For example, MTEB includes classifier-based classification and k-means-based clustering, while earlier work, including SBERT, evaluated embeddings on SentEval transfer tasks by training classifiers on top of fixed representations. Against this background, the mismatch between cosine similarity and task performance is not, by itself, particularly surprising. The cross-representation transfer results potentially go beyond this established understanding, but the paper does not clearly distinguish the additional insight they provide from the broader observation that cosine similarity is insufficient.
  • Although the near-orthogonality of CLS representations to AVG and RAND representations in DeBERTa and ELECTRA is an interesting observation, its implications for task performance are not clearly established. A rotation of an embedding space preserves all pairwise cosine similarities within that space, even though similarities between corresponding original and rotated embeddings may be low. Thus, low cross-representation cosine similarity does not, by itself, imply a difference in the geometry relevant to downstream tasks.
Questions for authors
  • Beyond showing that cross-representation cosine similarity does not necessarily correspond to similarity in downstream task performance, what new insight do the cross-representation probing results provide?
  • Please explain the rationale behind the expectation that the closer the CLS and AVG representations of the same sentence are, the more similar their respective downstream task performances should be.
3 comments
AuthorsOpenReview ↗

Insights into the embedding space, similarity vs. task performance

Thank you for the comments. We answer the questions in order below:

  1. (what insights do the results provide) Our investigation provides more insight into the sentence embedding space that goes beyond dimension-to-dimension comparisons. We show that at a shallow level, the CLS and AVG embeddings are very different, yet they can lead to similar results on a variety of tasks (by using the HOLMES benchmark), and furthermore, that they encode structural and semantic information the same way (through the cross-representation experiments in section 5). This serves to put in perspective how the embeddings, and the embeddings in conjuction with the cosine or other such measures is used, and as a starting point to distill symbolic structures from continuous sentence embeddings.

  2. (implications of cosine) As currently used in the field, the high cosine, reflected as closeness in the embedding space, indicates high similarity. High similarity means that the two representations encode similar information. This means that at least on some tasks – that rely on this shared information – their performance should be close. Our analysis is more extreme in this respect, as we do not compare the representations of sentences that have some degree of similarity (or relatedness), we compare different representations of the same sentence, which should encode the same information.

Thank you for your response. However, I do not think it adequately addresses the weaknesses and questions I raised. After also considering Reviewer nuv6’s comments and your responses to them, I have significant concerns about the paper and have therefore lowered my score.

If you can indicate what aspect of our reply was unsatisfactory, we can address that. We have replied to the concerns raised by reviewer nuv6, please consider our clarifications.

Reviewer_K8tXOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper analyses the similarities between different sentence representations by using 4 transformer encoder language models. The paper observes that these sentence representational spaces encode some structural linguistic information in similar manner although they have very dissimilar superficial representations. They argue that, because of the complex nature of sentence representation, simple and shallow cosine similarity-based analysis is limited.

Critical strengths

The paper contains an interesting study with insightful results. It critically analyses the limitations of the widely used cosine similarity metric in the sentence representation space. Also, it is good to see a small verification experiment to show similar behaviour of the decoder model. The paper is well written except in Section 5.

Critical weaknesses

Section 5 that describe one of the core experiments of the paper is not well written. Some experimental details and result descriptions are missing. Please see my ‘Questions for Authors’ for more details.

Questions for authors

Figure 1 and 2 shows the results from English and it would be great if you can say whether the observations from other languages are same or not (in the main text itself). In general, it would be nice if you could portray these results in a multilingual context by including or referring to your results on other languages with interpretation.

Section 5 experiments are not clearly explained. For example, how did you calculate the loss function in the variational encoder-decoder system? How were the contrastive examples utilised? Also, I am confused about how you construct out+ and out- samples. What is the exact procedure? It is very nice to see some examples in the paper. Can you include some examples of out+ and out- too?

Line 365: ‘BERT and Electra in particular show very high results, with results on S_T_{rand} even higher than S_{CLS}’. Are you talking about the diagonal entries of tables in Figure 5? Maybe you can first describe either the content of Figure 5 or the relevant results before starting the interpretation. The caption of Figure 5 also looks incomplete. It does not explain what the first half is and what the second half is.

Line 197: ‘According to the assumption that close points correspond to similar words and vice-versa, we must conclude that the optimizations over the BERT base model lead to more sharing of contextual information among the words in a sentence.’ According to your conclusion, this assumption may not be true, and therefore it is better to flag it here itself. Otherwise, it may misguide a reader to believe this potentially wrong statement.

Line 75: There is a missing reference.

2 comments
AuthorsOpenReview ↗

Results on other languages and loss function

Thank you for the comments! We include below the answers to the questions, in order:

  1. (Observations for other languages) As the results were so similar across languages, and for reasons of space, we decided to include only the English ones in the main paper. We will add a brief overview of all results in the main paper.

2a. (loss function) We use a max-margin loss function, which learns to score the positive answer above the negative ones:

  loss(in, out+, Out−) = max(0, 1−score(in, out+) + avg_{out−} \in Out− (score(in, out−))

The score is the dot product.

2b. (selection of out+, out-) For each dataset described in section 5.1, each sentence has a known pattern (as a chunk sequence or a syntactic-semantic pattern). For each pattern p and each sentence s exhibiting this pattern, we choose an out+ from all the other sentences that have the pattern p but are different than s, and then a set Out- of sentences out- that have different patterns (randomly selected from the set of patterns). The patterns are not provided to the system, only the sentence embeddings. We use the patterns to make a report of the errors made by the system (for us, not for the system), and to colour the plots of the latents

  1. (Line 365, Figure 5) We refer to the entire rows corresponding to the results on all sentence representation variations when trained on S_{T rand} and S_{AVG} (the last sentence in the caption). The top (heatmap + tsne plots) are the results on the CN dataset, and the bottom are the results on the SSR dataset. The heatmaps (point (i) in the caption) show F1 scores, and the tSNE plots (point (ii) in the caption) show the 2D projection of the latents.

  2. (Line 197): Thank you, we will clarify that this is an assumption that is invalidated by the analysis.

  3. (Missing reference): Apologies for the omission, it was due to a typo in the citation key. The reference is:

Samuel Marks, Max Tegmark (2024) The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets

Thank you for your responses and efforts to address my comments. I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.