- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper provides an interesting perspective on activation steering by examining the geometry of per-example displacement vectors rather than focusing only on class separability or directional consistency. The analysis of standardization effects and the answer-polarity axis is informative and helps explain why different layer-selection criteria can produce different rankings. However, the paper provides limited insight into why the proposed connectivity measure predicts successful steering, rather than simply capturing another form of displacement dispersion. A more direct empirical connection between displacement geometry and steering effectiveness would substantially strengthen the paper's informativeness.
Critical strengths
- This paper tackles a real problem in activation steering: how to choose which layers to intervene on without expensive held-out steering tests. LayerNavigator relies on discriminability and directional consistency. TopoSteer instead looks at the geometry of per-example displacement vectors. That gives a useful alternative view of what information layer selection can use. The setup in Section 3.2 is simple, and the theory in Section 5 helps show how their score differs from existing criteria.
- The experiments cover five instruction-tuned models, six persona-steering tasks, several layer budgets, and both simultaneous and sequential steering. The comparisons with LayerNavigator and W2S-Multi are relevant, especially because W2S-Multi needs extra evaluation-based selection. The big gains from the unit-sphere variant on Qwen-3B are encouraging. I also like the answer-polarity axis analysis and the bridge share idea, which helps diagnose cases where MST length is dominated by one long edge. The limitations in Appendix O also show the authors are aware of possible failure modes.
Critical weaknesses
-
A central motivation of the paper is that the extent and connectivity of the displacement cloud provide useful information for selecting effective steering layers. However, the theoretical analysis mainly establishes how the proposed score differs from discriminability, rather than explaining why larger connectivity should indicate better steering effectiveness. In particular, Theorem 1 shows that total (H_0) persistence increases while discriminability decreases under a controlled variation of example-level residuals. This does not establish that the resulting increase in connectivity corresponds to an improvement in actual steering performance. Similarly, the geometric analyses in Section 5 characterize how different scoring criteria respond to changes in representation geometry, but do not directly establish which geometric properties predict successful steering. Although Table 1 and Figure 3 demonstrate that TopoSteer can select better-performing layers in several settings, these results do not isolate whether the improvement is driven by displacement extent, connectivity structure, or other properties of the selected layers. This matters because the proposed geometric interpretation is central to the paper's contribution. I would encourage the authors to report layerwise correlations between MST-based scores and actual single-layer steering performance across models and tasks. An analysis of layers with high connectivity scores but poor steering performance would also help clarify the conditions under which the proposed criterion is reliable.
-
The distinctive benefit of MST-based connectivity over simpler geometric statistics remains unclear. The proposed total (H_0) persistence score is equivalent to the total length of the Euclidean MST. While the paper establishes that this measure captures information different from discriminability and directional consistency, it does not sufficiently demonstrate that MST connectivity provides useful predictive information beyond simpler measures of displacement dispersion. For example, centered RMS displacement, mean pairwise Euclidean distance, and mean nearest-neighbor distance could also characterize important aspects of the displacement cloud without relying on persistent homology. The sum-of-norms analysis in Appendix O is useful, but it does not fully address this concern, particularly because centered dispersion measures share the translation invariance emphasized in the paper. This concern is reinforced by the substantial difference between the original and unit-sphere variants in Table 1. On Qwen-3B, the original TopoSteer score underperforms LayerNavigator at K=2 and K=3, whereas the unit-sphere variant achieves substantial improvements. Since the unit-sphere projection removes individual displacement magnitudes, these results raise an important question about whether the gains arise primarily from displacement extent or from the relative geometry of normalized displacement directions. Furthermore, although the bridge-share diagnostic can identify cases where a single long MST edge dominates the score, it does not directly influence the layer-selection rule in Algorithm 1. Appendix O also identifies settings in which MST connectivity can favor an undesirable mode-split geometry, without establishing how often this occurs in real-model experiments. These issues matter because the paper's distinctive contribution depends on the value of the proposed connectivity measure, rather than on displacement dispersion in general. I encourage the authors to compare MST length with simple translation-invariant geometric baselines under identical steering protocols and to analyze cases where these criteria produce different layer rankings. A clearer explanation of the original versus unit-sphere results and the practical role of bridge share would substantially strengthen the contribution.
Questions for authors
Q1. Could the authors report the Spearman rank correlation between total (H_0) persistence and actual single-layer steering performance across models and tasks? In particular, are there layers with high connectivity scores but poor steering effectiveness, and what geometric properties characterize these cases?
Q2. How does MST-based connectivity compare with centered RMS displacement, mean pairwise Euclidean distance, and mean nearest-neighbor distance under the same extraction data, standardization, and steering protocol? Are there cases where MST and these simpler measures select different layers, and does MST more reliably identify the better-performing layers?
Q3. Table 1 shows that the unit-sphere variant substantially outperforms the original TopoSteer score on Qwen-3B, particularly at K=2 and K=3. What explains this difference? Does the improvement primarily come from the geometry of normalized displacement directions rather than variations in displacement magnitude? How does this finding relate to the paper's motivation concerning displacement-cloud extent?
Q4. Does a high bridge-share value predict poor steering performance in the real-model experiments? Have the authors examined whether selected layers with high MST scores are dominated by long inter-cluster edges or outliers? Could incorporating bridge share into the selection procedure improve reliability without requiring additional validation data?
14 comments
Response 2/12
Weakness 1.2: Although Table 1 and Figure 3 demonstrate that TopoSteer can select better-performing layers in several settings, these results do not isolate whether the improvement is driven by displacement extent, connectivity structure, or other properties of the selected layers. I would encourage the authors to report layerwise correlations between MST-based scores and actual single-layer steering performance across models and tasks.
Response: We thank the reviewer for the insightful suggestion. We report these correlations below. They cover the MST scores and scores that isolate each property the reviewer names: extent, connectivity, and other properties of the layers (separability and depth). We give both the rank (Spearman) and the linear (Pearson) correlation and they lead to the same conclusions.
Setup. We steered each of the 1,128 layers of the five models on its own, with the same vectors and as in Table 1 of the paper, and measured the gain in aligned-answer probability over the unsteered model on the 100 test prompts. For each of the 30 model–task settings, we correlate each score with the gain across that setting's layers, and we report the mean over the 30 settings.
Table R1. Correlation between each score and single-layer steering gain (mean over 30 settings; "Positive in" = number of settings with a positive correlation).
| Score | Property it isolates | Spearman | Positive in | Pearson | Positive in |
|---|---|---|---|---|---|
| (Ours) | connectivity, with magnitudes | 0.59 | 28/30 | 0.51 | 27/30 |
| on unit displacements (Ours$_{\ell_2}$) | connectivity, magnitudes removed | 0.63 | 28/30 | 0.52 | 27/30 |
| Sum of displacement norms | extent only | −0.53 | 1/30 | −0.44 | 3/30 |
| Centered RMS displacement | global dispersion | −0.56 | 2/30 | −0.51 | 3/30 |
| Mean pairwise distance | global dispersion | −0.51 | 2/30 | −0.45 | 3/30 |
| Mean nearest-neighbour distance | local spacing | 0.59 | 28/30 | 0.51 | 27/30 |
| LayerNavigator | separability + direction | 0.52 | 27/30 | 0.47 | 27/30 |
| Discriminability | separability | 0.42 | 27/30 | 0.38 | 26/30 |
| Layer index | depth | 0.53 | 28/30 | 0.42 | 27/30 |
Table R2. Spearman correlation by model (mean over six tasks).
| Model | (Ours) | Ours$_{\ell_2}$ | LayerNavigator | Discriminability | Centered RMS |
|---|---|---|---|---|---|
| Llama-3B | 0.87 | 0.88 | 0.88 | 0.84 | −0.67 |
| Llama-8B | 0.72 | 0.82 | 0.66 | 0.42 | −0.73 |
| Qwen-3B | 0.42 | 0.54 | 0.27 | 0.14 | −0.55 |
| Qwen-7B | 0.42 | 0.41 | 0.47 | 0.46 | −0.38 |
| Qwen-32B | 0.49 | 0.51 | 0.34 | 0.24 | −0.49 |
Response 3/12
Table R3. Mean single-layer gain (points) and share of layers with positive gain, by quarter of each score within each setting (Q1 = lowest 25%, Q4 = highest 25%; 282 layers per quarter).
| Score | Q1 | Q2 | Q3 | Q4 |
|---|---|---|---|---|
| (Ours) | +0.04 (58%) | +0.51 (62%) | +1.80 (82%) | +2.56 (85%) |
| Ours$_{\ell_2}$ | +0.02 (57%) | +0.45 (62%) | +1.50 (78%) | +2.94 (90%) |
| LayerNavigator | +0.02 (57%) | +0.68 (68%) | +1.76 (84%) | +2.44 (78%) |
| Discriminability | +0.09 (58%) | +1.01 (73%) | +1.86 (84%) | +1.95 (72%) |
| Centered RMS | +2.53 (89%) | +1.77 (79%) | +0.58 (62%) | +0.02 (57%) |
First, the sum of displacement norms, which measures only the size of the cloud, is negatively related to steering gain (Spearman −0.53), and so are the two global dispersion measures. Removing the magnitudes from our score does not weaken it; it makes it slightly stronger (0.63 vs 0.59; paired Wilcoxon over the 30 settings, p = 0.006).
Secondly, the connectivity can track the gain. Both MST scores are positive in 28 of the 30 settings (27 under Pearson), and the gain rises steadily from the lowest to the highest quarter of each score. In the top quarter, 85% (Ours) and 90% (Ours$_{\ell_2}$) of layers improve the behavior. predicts the gain better than discriminability (0.59 vs 0.42, p < 0.001) and better than the full LayerNavigator score (0.59 vs 0.52, p = 0.050). Discriminability is least reliable at the top of its ranking, which is where a selector picks layers: its top quarter steers worse than its third quarter (72% vs 84% of layers helpful). Other properties of the layers do not explain the gain either. The mean nearest-neighbour distance gives the same correlation as (0.59), as expected, because every nearest-neighbour edge is also an MST edge. Together, these results show that the useful property is the local spacing of the displacements. Their global spread, which is dominated by the answer-polarity gap (Sec. 5.3), is negatively related to the gain.
Finally, the relation is monotone rather than linear. For every score, the Spearman correlation is larger in absolute value than the Pearson correlation, and most of the gain is in the top half of the ranking (Table R3). The values are also stable: on 200 separate validation prompts, the mean Spearman correlation of is 0.61, and the per-layer gains on the two prompt sets correlate at 0.92.
Response 1/12
We sincerely thank the reviewer for their time and careful evaluation of our submission. We greatly appreciate the reviewer's recognition of the real problem of choosing intervention layers without expensive held-out steering tests, the useful alternative view offered by per-example displacement geometry, the simple setup and the theory distinguishing our score from existing criteria, the breadth of the experiments (five models, six tasks, several layer budgets, simultaneous and sequential steering), the relevant comparisons with LayerNavigator and W2S-Multi, and the answer-polarity and bridge-share analyses. We address each concern below. If our clarifications and additional results satisfactorily resolve the reviewer's concerns, we respectfully ask the reviewer to reconsider their assessment. We will incorporate all of these discussions into the revised version.
Response: We agree with the author comment on Section 5. However, we would like to clarify that the Theorem 1 was not written to show that one ordering is better. It shows that our score and discriminability can rank the same configurations in opposite ways, so which one to trust must be tested on real models. Sections 5.1 and 5.3 play a similar role. They explain why the two criteria disagree (class separation affects our score through the standardization, and the answer-polarity axis affects it through a single bridge edge), not which one is right. To further address the reviewer concerns. We steered each of the 1,128 layers of the five models on its own and measured, across layers, how well each geometric property predicts the steering gain (full tables in Weakness 1.2). The result show that connectivity predicts steering. Specifically, the mean Spearman correlation is 0.59 for L and 0.63 for Ours$_{\ell_2}$ (the same score on unit-length displacements), and both are positive in 28 of the 30 model–task settings. The mean gain rises steadily from the lowest to the highest quarter of L (+0.04 to +2.56 points), and the share of layers that improve the behavior rises from 58% to 85%. Additionally, connectivity also predicts steering better than separability. Discriminability gives 0.42 (paired Wilcoxon over the 30 settings, p < 0.001), and the full LayerNavigator score gives 0.52 (p = 0.050). The size of the cloud does not explain the result: the sum of displacement norms, which measures only extent, is negatively related to the gain (−0.53).
So, across real layers, a larger L goes with better steering, and what matters is how the displacements are connected, not how large they are. This evidence is a correlation across layers, not a mechanism. It holds on all five models (the correlation of L is positive on each). It is stable between validation and test prompts, whose per-layer gains correlate at 0.92.