- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes a method for estimating the geometry of Language Models by modeling edits from high-dimensional cones and compositional paths. Using counterfactual edits, the authors build up a lattice of edits that would map to an underlying common geometry learned by the LLM. This geometry could be useful for interpretability applications and steering model behavior.
Critical strengths
The paper evaluates the methodology on 5 different models with a considerable set of examples on a large set of metrics, demonstrating a strong set of results that can be considered trustworthy. The approach for measuring the large-dimensional lattices with the walker formulation is also novel, providing a new way to compute approximate Shapley values in a sensible manner.
Critical weaknesses
There are theoretical errors to be addressed. In Def 2.3, the geometric interpretation of $ s=\mathbb E[\cos(\Delta,\hat{\gamma})]$ as the cos of the cone opening angle is false in general because you this is not a hard bound on how far individual effects can lie from the axis, it is an average directional concentration. Similarly, a general convex cone (Def 2.1) doesn't require one circular axis and opening angle. I think to get a cone one should define a genuine enclosing, like the smallest angle containing 95% of directions.
Also, the sentence "Two sentences that differ in one edit share all other cones, so their difference cancels them and isolates the edit" needs more careful analysis. This sort of behavior is valid for strong additive model, so every concept would contribute exactly the same vector before and after the edit, but this is not necessarily true for neural networks and LLMs even if polysemanticity or superposition hypothesis is disregarded simply because a single word can have different meaning in different contexts, so it would lead to different concepts and one edit could entirely change the understanding of a sentence and "activate" another meaning for a word that was not edited. For example, "The musician examined the organ" and "The surgeon examined the organ"; "The programmer found a bug" and "The entomologist found a bug", etc.
Then, there is a lack of proper comparison and discussion with prior literature. [1], [2], [3] also discuss textual edits as Shapley players and have several intersections with this work. [4], [5], [6] provide the basic tools of analyzing cones and concept geometry beyond individual directions, so it could be relevant for comparison and discussion.
- [1] Goto, Vasselli, and Watanabe, Improving Explainability of Sentence-level Metrics via Edit-level Attribution for Grammatical Error Correction, ACL 2025
- [2] Fern and Pope, Text Counterfactuals via Latent Optimization and Shapley-Guided Search, EMNLP 2021
- [3] Singhvi et al., Using Shapley Interactions to Understand How Models Use Structure, ACL 2025
- [4] Wollschläger et al., The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence, ICML 2025
- [5] Banerjee, Dhillon, Ghosh, and Sra, Clustering on the Unit Hypersphere Using von Mises-Fisher Distributions, JMLR 2005
- [6] Valois et al., Frame Representation Hypothesis: Multi-Token LLM Interpretability and Concept-Guided Text Generation, TACL 2025
Finally, the proof of proposition 3.3 seems to be wrong. The usage of the Hanson-Wright inequality does not directly apply to arbitrary jointly sub-Gaussian step vectors. As far as I know, it requires independent components for correlated vectors, but independent samples of a sub-Gaussian vector do not mean independent coordinates. Also, the proof is organized in a way that is not so clear even though it is in the appendices where one would expect the space could be used more freely, so the steps needs a bit more clarity. Please check this issue carefully.
Questions for authors
The interpretation of the results as the defined geometry and cones is weak.
- What evidence shows the proposed semantic decomposition is different from a simple projection onto the empirical mean and its orthogonal complement?
- Why is context-related variation always orthogonal to the shared direction?
- Could the authors compare the suggested cone representation with a low-rank linear or affine model? Singular value spectra (principle angle stability) or angular error could clarify the nature of the variability (directional, low dimensional, etc).
3 comments
Author Response (2/3)
We will also make the context interpretation explicit:
a_v(c)=\langle\Delta_v(c),\widehat\gamma_v\rangle,\qquad \langle r_v(c),\widehat\gamma_v\rangle=0.$$ **Both** $a_v(c)$ and $r_v(c)$ can depend on context. Orthogonality defines the two components; their variation is measured from the data. This separation of geometry from functional interpretation is consistent with [Wollschläger et al. (2025)](https://proceedings.mlr.press/v267/wollschlager25a.html), who explicitly distinguish orthogonality from independence under intervention. **[W2] How are interactions handled?** Fix the measured sentences for the allowed edit combinations, let $m_S$ be their mean representations, and let $\mu=m_N-m_\varnothing\ne0$. The game in section 3.2 is $$v(S)=\frac{\langle m_S-m_\varnothing,\mu\rangle}{\|\mu\|^2}.$$ For any allowed order, adding adjacent differences gives $m_N-m_\varnothing$ exactly. Therefore, the path marginals sum to $v(N)-v(\varnothing)=1$, and averaging each edit's marginal over allowed orders gives its Shapley allocation. These are algebraic identities for the measured game and hold when edits interact. The treatment of required edit order follows the framework of [Faigle and Kern (1992)](https://doi.org/10.1007/BF01258278). For example, the interaction between two edits can be measured as $$m_{S\cup\{i,j\}}-m_{S\cup\{i\}}-m_{S\cup\{j\}}+m_S.$$ The Shapley calculation averages marginal contributions in the presence of this interaction. Directional consistency helps explain the observed accuracy of a few paths; it is separate from the identities that define the allocation. This is also complementary to the pairwise linguistic interactions studied by [Singhvi et al. (2025)](https://aclanthology.org/2025.acl-long.1011/). **[W4] Revised proof of Proposition 3.3.** We can address coordinate dependence using the distinct-context estimator described in Proposition 3.3 and Appendix A. For one step $A_i$ and the total effect $B_i$ in context $i$, define $$U_N=\frac1{T(T-1)}\sum_{i\ne j}\langle A_i,B_j\rangle,\qquad U_D=\frac1{T(T-1)}\sum_{i\ne j}\langle B_i,B_j\rangle, \qquad \widehat w_U=U_N/U_D.$$ Independent contexts are sufficient; the step and total effect within one context, and their coordinates, may be dependent. Write $\nu=\mathbb EA$, $\mu=\mathbb EB$, $n=\|\mu\|>0$, and $w=\langle\nu,\mu\rangle/n^2$. With centered variables $a=A-\nu$, $b=B-\mu$, define $z=a-wb$ and $L=\langle\mu,a\rangle+\langle\nu-2w\mu,b\rangle$. We prove $$\operatorname{Var}(U_D)= \frac{4\mu^\top\Sigma_b\mu}{T}+ \frac{2\operatorname{tr}(\Sigma_b^2)}{T(T-1)}=:V_D,$$ $$\operatorname{Var}(U_N-wU_D)\le \frac{\operatorname{Var}(L)}{T}+ \frac{2\operatorname{tr}(\Sigma_z\Sigma_b)}{T(T-1)}=:V_R,$$ where $\Sigma$ denotes covariance. These expressions use finite second moments and allow arbitrary coordinate dependence. They follow by separating the linear sample-mean term from centered products of different contexts, using the order-two U-statistic decomposition of [Hoeffding (1948)](https://doi.org/10.1214/aoms/1177730196). For failure probability $\alpha$, if $d_\alpha=\sqrt{2V_D/\alpha}<n^2$, Chebyshev's inequality and a union bound give, with probability at least $1-\alpha$, $$|\widehat w_U-w|\le \frac{\sqrt{2V_R/\alpha}}{n^2-d_\alpha}.$$ This supplies an explicit bound for the ratio. With covariance operator norms bounded by $\sigma^2$, its terms have orders $1/\sqrt T$ and $\sqrt{\operatorname{tr}\Sigma_b}/T$, with the mean-effect lengths and denominator condition stated in the full proof. The argument uses independence across contexts rather than the independent-coordinate assumption in Theorem 1.1 of [Rudelson and Vershynin (2013)](https://arxiv.org/abs/1306.2872). Across eight experiments, 360 orders, and six shares, the maximum difference from the original readout is only 0.000395 (Table 3), and the shares sum to one numerically. **Table 3. Effect of using inner products from different contexts** | Model | Experiment | Largest change in one path's share | Largest change after averaging all orders | | --- | --- | --- | --- | | pythia-410m | Judicial, natural contexts | 0.000299 | 0.000235 | | pythia-410m | Judicial | 0.000287 | 0.000262 | | pythia-410m | Literary | 0.000145 | 0.000137 | | pythia-410m | Medical | 0.000212 | 0.000143 | | Qwen2.5-1.5B | Judicial, natural contexts | 0.000395 | 0.000305 | | Qwen2.5-1.5B | Judicial | 0.000258 | 0.000196 | | Qwen2.5-1.5B | Literary | 0.000258 | 0.000212 | | Qwen2.5-1.5B | Medical | 0.000358 | 0.000263 | We compared the original method with the distinct-context estimator using saved activations from eight experiments, each covering 48 contexts and all 360 allowed edit orders. The results are nearly identical, with a maximum difference below 0.0004. The estimated contributions also sum to one. This confirms that the choice of estimator has little effect on our experimental results.Author Response (3/3)
[Q3] Comparison with low-rank linear and affine models.
Our angular enclosure describes the spread of edit directions across contexts, while low-rank models examine whether this variation can be approximated by a few directions.
We compared these descriptions using 27 fixed word pairs and 331 test vectors per model, varying only the natural context. All texts from each test source were excluded from fitting.
Table 4. Angular reconstruction error across contexts (degrees; lower is better).
| Model | Edit type | Mean axis ↓ | Linear 1 ↓ | Affine 1 ↓ | Linear 4 ↓ | Affine 4 ↓ |
|---|---|---|---|---|---|---|
| pythia-410m | Adjectives | 41.8 | 43.8 | 41.6 | 39.1 | 39.6 |
| pythia-410m | Phrases | 39.0 | 41.9 | 37.1 | 36.0 | 35.9 |
| pythia-410m | Tense | 30.3 | 31.9 | 30.6 | 28.5 | 29.1 |
| pythia-410m | Verbs | 44.7 | 46.6 | 44.2 | 41.4 | 40.5 |
| Qwen2.5-1.5B | Adjectives | 42.8 | 45.3 | 43.5 | 41.7 | 42.0 |
| Qwen2.5-1.5B | Phrases | 37.7 | 40.7 | 36.1 | 35.6 | 35.4 |
| Qwen2.5-1.5B | Tense | 37.0 | 39.3 | 37.6 | 35.3 | 35.8 |
| Qwen2.5-1.5B | Verbs | 43.8 | 47.5 | 43.7 | 41.8 | 41.2 |
Mean axis uses the average training effect; linear models fit directions through the origin, while affine models add an offset. Results use layer 6 for Pythia and layer 7 for Qwen. The results show structured variation beyond the mean direction, partially captured by low-rank models.
We further examine this variation through singular value spectra after removing each word pair's mean effect.
Table 5. Structure of context-dependent variation after removing the mean effect.
| Model | Edit type | First 2 directions | First 4 directions | Effective rank |
|---|---|---|---|---|
| pythia-410m | Adjectives | 68.4% | 89.7% | 3.51 |
| pythia-410m | Phrases | 63.3% | 84.8% | 4.03 |
| pythia-410m | Tense | 69.5% | 79.6% | 2.63 |
| pythia-410m | Verbs | 47.1% | 67.5% | 6.38 |
| Qwen2.5-1.5B | Adjectives | 68.9% | 90.2% | 3.47 |
| Qwen2.5-1.5B | Phrases | 65.8% | 86.2% | 3.76 |
| Qwen2.5-1.5B | Tense | 66.3% | 77.7% | 3.44 |
| Qwen2.5-1.5B | Verbs | 53.4% | 71.4% | 5.54 |
The first four directions explain 67.5%–90.2% of the observed residual variance, revealing structured context-dependent variation. These values describe the observed samples rather than an intrinsic dimensionality. Together, the experiments show that angular enclosures and low-rank approximations offer complementary descriptions of edit-effect geometry.
[W3] Related work and presentation.
We thank the reviewer for the six suggested references and compare them in Table 6. Our contributions are as follows. First, we measure text-edit geometry across contexts and word replacements, distinguishing context concentration from replacement coherence using same-position controls. Second, we study how edits combine, measuring vector interactions and assigning Shapley contributions through a concept network. We validate efficient path-based estimation against an exhaustively evaluated network of 65,536 sentence variants.
Table 6. Connections to the six suggested papers.
| Reference | Main focus | Distinction from our work |
|---|---|---|
| [1] Goto et al. (2025) | Shapley attribution of grammatical edits to sentence quality. | We attribute representation changes and study their composition. |
| [2] Fern and Pope (2021) | Shapley-guided counterfactual text search. | We measure edit effects and allocate contributions on a sentence network. |
| [3] Singhvi et al. (2025) | Linguistic interactions between input components. | We measure vector interactions and compositional attribution across edits. |
| [4] Wollschläger et al. (2025) | Intervention-based refusal directions and concept cones. | We analyze context-dependent text-edit effect clouds and their composition. |
| [5] Banerjee et al. (2005) | Directional distributions and concentration. | We measure concentration and coherence of text-edit effects empirically. |
| [6] Valois et al. (2025) | Multi-token representation frames and concept-guided generation. | We analyze variation across contexts, replacements, and edit combinations. |
We will incorporate these comparisons into the related-work section and clarify the distinction between established tools, theoretical assumptions, and empirical findings.
Author Response (1/3)
Thank you for recognizing the empirical measurements and the walker evaluation. We response with explicit arguments, additional analyses, and comparisons with the suggested papers.
[W1] Concentration and the angle containing the effects. We will state the geometric quantities separately. For nonzero measured effects with a nonzero mean, let
u_i=\frac{\Delta_i}{\|\Delta_i\|},\qquad s=\frac1n\sum_i\langle u_i,\widehat\gamma\rangle.$$ Here, $s$ measures average directional agreement. We will additionally report $$\theta_{.95}=\inf\\{t:\frac1n\sum_i\mathbf1[\arccos\langle u_i,\hat\gamma\rangle\le t]\ge.95\\}$$ This is the smallest angle **about the reported axis** containing at least 95% of the measured directions. This states both the observed geometry and its behavior on new text. The circular axis-and-angle description will be distinguished from the definition of a general convex cone. The distinction has an established statistical basis: [Banerjee et al. (2005)](https://www.jmlr.org/papers/volume6/banerjee05a/banerjee05a.pdf) treat mean direction and concentration as separate quantities for directional data. We use directly measured edit vectors rather than requiring a fitted directional distribution. **[W2] What does subtracting two representations measure?** We define $$\Delta_v(c)=e(\delta_v x,c)-e(x,c)$$ as the **total representation response to the specified text edit**. This includes changes in the contexts. The organ and bug examples therefore describe interactions that the measurement includes. Our experiments also measure interaction explicitly: the submitted six-edit experiments report a relative interaction magnitude of 0.29–0.54 and mean cross-node direction cosines of 0.87–0.95. Thus directional agreement and interaction magnitude are measured separately. This distinction allows edits to have related directions while their magnitudes depend on preceding edits. **[Q1, Q2] What does the projection describe?** Projection separates each edit effect into a component along its mean direction and a perpendicular remainder. While this decomposition is mathematical, the distribution of effects across contexts and word replacements is measured empirically. Effects can have the same mean direction but differ in their consistency across contexts or replacements. Our experiments test this distinction using same-position, same-part-of-speech word controls (Table 2). Across 15 model–sentence combinations, adjective synonyms show lower replacement agreement than the reference (mean difference −0.12), while action verbs show higher agreement (+0.07). These differences are empirical observations, not consequences of orthogonal projection. Both the parallel and perpendicular components can vary with context. Orthogonality defines the decomposition, but does not determine the observed variation. **Table 2. Agreement between replacements compared with same-position word controls** | Replacement family | Above reference range | Within reference range | Below reference range | Mean difference from reference | | --- | --- | --- | --- | --- | | Adjective synonyms | 0 | 0 | 15 | −0.12 | | Object nouns | 1 | 3 | 11 | −0.07 | | Human → machine subjects | 5 | 9 | 1 | +0.03 | | Action verbs | 14 | 1 | 0 | +0.07 |