acceptodds
Under review as a conference paper at ICLR 2027

Coupled Usage-Sense Processes: Temporal and Attributable Lexical Semantic Change

Abstract

Lexical semantic change is usually summarized by a scalar distance between independently sampled period distributions. This measures how much a word changed, but does not reveal when it changed, which mechanisms and component movements carried the change, or which usages support the attribution. We introduce Coupled Usage–Sense Processes (CUSP), which derives these answers from a single marginal preserving temporal process. A hierarchical coupling relates contextual distributions through latent usage components, while Markov composition makes adjacent and longer span correspondences compatible. Displacement operators quantify change magnitude and timing, split variation exactly between movement of component centers and reorganization within components, and attribute it to transported component pairs. Word-local modes resolve distinct directions of change and their activity over time, while representative passages from attributed components ground the analysis in text. Under a Gaussian mixture specialization, we prove parametric recovery of the operators and squared distances. Synthetic experiments support the predicted rate. CUSP remains competitive on English and German DWUG and recovers controlled Janus profiles while maintaining compositionally coherent transport. A large corpus of US court opinions demonstrates transition, mode, and passage attribution in unlabeled natural text. CUSP thus makes magnitude, timing, mechanism, movement, modes, and textual evidence compatible views of one lexical history.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
lexical semantic change, diachronic semantics, contextual embeddings, interpretable NLP, temporal distribution modeling, optimal transport, semantic change attribution

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ejUFOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Coupled Usage–Sense Processes (CUSP) for measuring and interpreting lexical semantic change. The authors fit Gaussian mixture models to contextual embeddings in each time period, match usage components across adjacent periods with hierarchical optimal transport, and chain these matches into a single multi-period process through Markov composition. On top of this process, they split the overall change into movement of component centers and variation within components, and attribute it to specific component pairs and distinct temporal directions. The paper gives formal proofs and reports experiments on synthetic data, DWUG, Janus, and a corpus of U.S. court opinions.

Critical strengths
  1. Intuitive and unified design. Representing a word's usages as latent components and using optimal transport to describe how their distribution shifts across periods is a natural and interpretable approach. The amount of change, its timing, and its attribution to components all come from one probabilistic process, which makes the framework mathematically coherent.

  2. A clean decomposition of change. Through conditional centering, the total squared displacement splits exactly into a part from moving component centers and a part from variation within components, and both parts can be assigned to specific component pairs. This makes the change score far more informative than a single number.

  3. Experiments at several levels. Synthetic data tests statistical recovery, DWUG tests whether the overall change score is plausible, Janus tests controlled temporal shifts, and the court corpus shows how the statistical output can be linked to real historical text.

Critical weaknesses
  1. The linguistic meaning of mass matching under balanced optimal transport needs more explanation.

    CUSP uses balanced optimal transport to match components across time periods. Consider the word plane, which can mean a geometric plane or an airplane. Suppose the shares of these two senses change from 90%/10% to 30%/70%. If these are the only two components, balanced optimal transport must move at least 60% of the probability mass from the geometric sense to the airplane sense. Yet both senses may have existed side by side throughout this period. The larger share of the airplane sense may simply reflect an increase in aviation-related texts.

    This mass flow could have different meanings. It might describe a statistical shift in the usage distribution, or it might represent a possible path of semantic change. The authors should clarify how they interpret this flow and how that interpretation supports their claims about historical semantic change.

  2. Component-level explanations need more direct evaluation.

    One of the paper's main goals is to explain which usages change, when they change, and which components contribute most. However, the current experiments mainly test overall change scores and the recovery of temporal change patterns. DWUG evaluates the correlation between the model's scores and human judgments of change. Janus tests whether the model recovers predefined change patterns over time. The court examples show the model's attributions through passages from real texts.

    These experiments provide useful evidence, but one important question remains: Do the main components and their links across periods match actual differences in usage and semantic change? For example, the passages in the court cases show different usages, but it would help to test whether the main component movements reliably capture those differences.

    Existing data could support a more direct test. Janus includes known sense labels. The authors could compare the fitted components with these labels and check whether the main component movements reflect the predefined changes in sense proportions. DWUG also provides sense clusters built from human judgments of semantic similarity. These clusters could help test whether the model's attributions agree with human judgments. This evaluation could allow several model components to correspond to the same sense.

    The authors could also refit the model on different subsets of the court corpus to check whether the main change periods and key component attributions remain stable. These tests would provide more direct evidence for the paper's claims about component-level explanations.

  3. The presentation and organization are unclear.

    The abstract uses many technical terms, such as "hierarchical coupling," "displacement operator," and "word-local modes," with little intuitive explanation. This makes it hard to understand the research question and main idea quickly.

    The mathematical properties introduced in the introduction also need more explanation. Temporal consistency mainly follows from composing Markov transitions, while preservation of the marginal distributions follows directly from the constraints of optimal transport. The authors present these properties as major contributions, but do not explain clearly why they matter in practice. The introduction would be clearer if it first explained the reasons for these modeling choices and then presented their properties.

    Finally, the paper has no separate Related Work section. Previous studies are covered in one dense paragraph in the introduction, which reads like a list of references. Table 3 in the appendix compares the capabilities of different methods, but the main text does not clearly explain how CUSP relates to or differs from prior work in terms of research questions and methods.

Questions for authors
  1. Why did the authors choose balanced optimal transport instead of unbalanced optimal transport, which allows probability mass to increase or decrease? Under this modeling choice, how should we interpret the flow of probability mass between components across time from a linguistic perspective, especially when new senses appear, old senses decline, or existing senses coexist while their usage proportions change?

  2. Could the authors use existing sense clusters based on human judgments in datasets such as DWUG to test more directly whether CUSP's main components and their cross-period links match human evidence? If the model is refitted on different subsets of a corpus, such as court opinions, do the main change periods and key semantic drivers remain stable?

  3. Could the authors make a clearer distinction in the introduction and discussion of prior work between modeling choices, properties guaranteed by the model's construction, and results established by theory? Concrete examples of why these properties matter for studying lexical semantic change would also be helpful.

Reviewer_aEaxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The Lexical Semantic Change (LSC) provides the scalar distance between the independently sampled period distribution, which measures the word change, which mechanisms, component movements carried the changes. In the paper, the author introduced the Coupled Usage-Sense Processes (CUSP), which answers from a single marginal preserving temporal process. The author evaluate the CUSP using the English and German DWUG dataset, controlled Janus profile, and US court opinions.

Critical strengths
  1. The CUSP provides more information than a single change scores that is, the framework combines the magnitude, timing, component contribution, temporal direction as shown in figure 1.

  2. The experiment shows results with synthetic data test recovery, DWUG tests agreement with human judgments, Janus tests controlled temporal changes, and cout opinions demontrate passage-based interpretation.

  3. The CUSP full trace achieves Spearman correlation of 0.746 for English and 0.815 for German on DWUG dataset and the authors shows it as competitive results rather than showing superiority (Table 1).

  4. The figure 2, shows that the estimated error decrease as the sample size increase (n∈{100,200,400,800,1600}n \in \{100, 200, 400, 800, 1600\}), supporting the theoretical recovery rate.

Critical weaknesses
  1. The experiment for figure 2 uses two-dimentional Gaussian setting with known number of components. However, testing with higher-dimensional data and different number of fitted component would help asses performance in more realistic conditions.

  2. The Table 1 does not show a clear improvement over the existing methods. The CUSP full trace performs best in English but not in German. Also, the overlapping confidence intervals leave uncertainty about whether the difference are statistically significant.

  3. The author used only one model (XL-LEXEME) to generate the embeddings. So it's unclear how it performs with other embedding models.

Questions for authors
  1. The Table 1 shows competitive performance. However, the statistical tests and results with random seeds would help establish whether the difference between methods are reliable.

  2. Figure 3 shows whether CUSP recovers when changes occurs. And comparing it with other methods would help asses whether CUSP connection better capture semantic changes.

Reviewer_AiohOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes a principled framework for lexical semantic change detection. It models the contextualized word embeddings in each time period as Gaussian mixtures and aligns the components by optimal transport. It also decomposes semantic change into mean shift and deviation, and quantifies the attribution of each component. Additionally, word-local temporal modes capture the semantic change and its transition in different directions. Experimental results provide empirical evidence that the proposed framework works well in both artificial and real settings.

Critical strengths
  1. The proposed framework is interesting and highly sophisticated. It constructs a temporally coherent process by composing couplings between adjacent periods.
  2. It theoretically shows that semantic change can be decomposed into mean shift and deviation, and gives a mathematical foundation for the quantitative analysis of semantic change. It also establishes theoretical properties, including statistical convergence rates, which are supported by the synthetic experiments.
  3. Extensive experiments, including artificially created datasets and real-world datasets, were performed to shed light on different aspects of the proposed framework.
Critical weaknesses
  1. Even though the mathematical foundation is elegant, it is not clear whether the proposed decomposition is appropriate in terms of the linguistic nature of semantic change. It would be better to use more complex controlled settings involving multiple simultaneous or temporally separated changes.
  2. Related to the above, it is not clear why the proposed components are necessary for the framework. According to the DWUG experiments, the full trace and mean trace achieve almost identical performance. Although the deviation component may provide additional interpretability, the experiments do not clearly demonstrate its practical benefits for analyzing semantic change.
Questions for authors
  1. Given that artificially created data allows you to control the semantic change generation process, we can quantitatively evaluate the contribution of each component transition and the accuracy of the temporal modes. Is it possible to directly assess the accuracy for attribution and temporal modes?
  2. Is it possible to conduct an experiment to show whether the attribution of CUSP gives a better explanation than simple clustering and pairwise OT? Particularly, I am curious about the correspondence between the estimated component transitions and the actual meaning change.
  3. Ablation of each component, such as hierarchical coupling and word-local temporal modes, would greatly help the understanding of the proposed framework in action. In particular, the DWUG dataset shows that the performances of full trace and mean trace are very similar; I am wondering if the deviation component is useful or not in a realistic setting.
2 comments

Thank you for the detailed responses and additional experiments. The controlled experiments and the DWUG cluster analysis largely address my concerns. I have update my score accordingly.

Thank you for revisiting the additional experiments. We are glad that they largely addressed your concerns. Your questions helped us provide more direct evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.