acceptodds
Under review as a conference paper at ICLR 2027

Riemannian Manifold Steering Stays on a Data-Informed Track

Abstract

Activation steering aims to control a model's behavior by moving activations along a linear direction or interpolating between concepts. However, steered activations can lie far from those the model naturally produces, degrading its output. Prior manifold-aware steering (Wurgaft et al., 2026) addresses this with cubic splines, but requires labeled concept centroids and assumptions about the manifold's topology. Instead, we model the activation space as a Riemannian manifold whose metric is pulled back from the output distributions. We consider geodesics (shortest paths) under that metric with a regularizing term that penalizes paths moving away from observed activations. We call this unsupervised method Riemannian Manifold Steering (RMS). We first examine path recovery and surface drift on synthetic surfaces with known geometry, isolating the roles of the metric and data support regularizer. We then show that the resulting paths produce outputs closer to the natural output distributions of the model across a bunch of tasks ranging from toy to real world LLM tasks.

Primary area:
interpretability and explainable AI
Keywords:
non linear steering, manifold steering, steering and model editing, manifold learning, geometry of representations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TibuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Riemannian Manifold Steering (RMS), an unsupervised method for activation steering and interpolation in language models. To keep steering trajectories on the data manifold without requiring labeled intermediate concepts or pre-specified topology, the authors combine a pullback metric induced by the output token distribution (measured via Hellinger distance) with an activation support penalty based on the distance to observed activations. In practice, trajectories between activation endpoints are parameterized by a small neural bridge network and optimized via finite-difference approximations of the pullback length. The method is evaluated on 2D synthetic surfaces, toy concept interpolation tasks in Llama-3.1-8B (weekdays and ages), and behavioral steering datasets in Llama-2-7B (hallucination, corrigibility, and myopia).

Critical strengths
  1. The geometric motivation combining output pullback metrics with an input support penalty is intuitive and well-analyzed on synthetic manifolds. As shown in Figure 1 and Table 1, neither the pullback metric alone nor the support penalty alone is sufficient to keep trajectories both on the data manifold and away from high output distortion regions. The synthetic surface ablations clearly demonstrate how the two terms complement each other.

  2. The paper is written with commendable transparency regarding its approximations and failure modes. The appendices thoroughly document the practical relaxations, hyperparameter sensitivities, and negative results, which makes it much easier to understand what the method actually does under the hood.

Critical weaknesses
  1. In-sample toy LLM evaluation where test metrics mirror the training loss. In Section 5.2 (Table 2), each neural bridge is trained to minimize pullback length and k-NN distance to training activations, and then evaluated using the exact same metrics on that same training reference set (Appendix C.1, C.2.6). Showing that a path optimized to stay close to training points achieves a smaller distance to those exact points than unoptimized Linear or Spline baselines is trivial by construction and does not validate better semantic steering.

  2. Real-world steering requires ground-truth test answers during optimization, cannot generalize to unseen prompts, and fails to beat straight lines. In Section 5.3 (Table 8), the algorithm trains a separate neural bridge per prompt and optimizes its endpoint using the ground-truth positive and negative answer log-likelihoods for that specific prompt (Equation 6), then reports that same log-likelihood gap as the test metric. Because finding the path requires knowing the contrastive answers in advance, this setup cannot steer unseen prompts at inference time. Even in this in-sample setting, curved RMS trajectories fail to consistently outperform a simple optimized straight line on Corrigibility and Myopia (Table 8).

  3. Failure to achieve ordered concept traversal on the toy LLM benchmarks. Although weekday interpolation is meant to show that RMS can naturally traverse intermediate concepts in order without labels, Table 5 (Appendix C.4) shows that RMS achieves 0 out of 21 pairs with ordered dominant concept visits (matching the 0/21 failure rate of Linear interpolation, versus 7/21 for Spline). Only 12.7% of interior peaks along RMS paths ever become the dominant weekday, with a tiny median prominence of 0.72%.

  4. Disconnect between the theoretical guarantee and the LLM implementation. Lemma B.1 proves that the support penalty bounds manifold deviation only under a constant Riemannian speed constraint; otherwise, a path can linger near support points and jump rapidly across empty regions. While the synthetic experiments approximate constant speed by minimizing quadratic energy, the LLM implementation minimizes a reparameterization-invariant piecewise-linear length without any speed constraint, invalidating the theoretical bound.

  5. Heavy test-time compute overhead and single-token evaluation scope. Unlike standard activation addition, RMS trains a 404K-parameter network for every prompt by decoding 51 path points and 8 subdivided graph connections through the upper transformer layers and LM head, requiring over 4,500 partial forward and backward passes for just 10 Adam steps. Moreover, all experiments only inspect the probability distribution of the first generated token rather than full multi-token generation.

Questions for authors
  1. In Section 5.3, since optimizing the endpoint in Equation 6 requires access to the target positive and negative answer tokens for each individual prompt and trains a separate bridge per prompt, how can RMS be applied to steer unseen test prompts where ground-truth answer pairs are unavailable? Can the learned steering trajectories transfer to held-out prompts, and how does the method affect full multi-token autoregressive generation beyond the first token distribution?

  2. Given that Table 5 shows 0 out of 21 weekday pairs achieve ordered dominant intermediate visits under RMS, and Table 8 shows that curved bridges do not outperform optimized straight lines on Corrigibility and Myopia, what concrete downstream behavioral benefit does optimizing a curved trajectory provide in LLMs beyond simply optimizing the endpoint or applying standard activation addition?

  3. Why did the LLM implementation minimize the piecewise-linear length functional instead of the quadratic energy functional used in the synthetic experiments, which would naturally encourage constant metric speed and align with the assumptions of Lemma B.1?

6 comments
AuthorsOpenReview ↗

Response 1/3: Summary + W1

We thank the reviewer for the careful reading, and for noting the synthetic ablations and our transparency about failure modes. We agree with several points and ran new experiments to address them. In summary:

  • Held-out evaluation (W1). With a clean train/test split, RMS stays closer to the model's natural outputs than linear and spline paths on held-out prompts. A baseline that directly fits the training activations is closest on training prompts, but on weekdays this does not carry over to held-out prompts; on ages it matches RMS.
  • Unseen prompts and multi-token generation (W2). On MAxBench, we use paths fit once per concept country pair to steer 100 unseen instructions in full generation. RMS has the highest combined score of the steered methods.
  • Our claim (W3). We agree that RMS does not visit weekdays in order, and we do not claim a consistent benefit of curvature. We will make this explicit in the main text. Our claim is that RMS stays closer to the model's natural outputs, which keeps steered outputs more natural, with a trade-off against steering strength.
  • Lemma B.1 (W4). We agree that Lemma B.1’s constant-speed assumption is idealised and does not hold in the LLM runs, so the bound does not cover them and will state it as a limitation in the main text.
  • Compute (W5). RMS costs more than activation addition by design. When the endpoints are concept centroids, this cost is paid once per concept pair and can be applied to held out prompts without additional cost.

W1 In-Sample Evaluation

In-sample toy LLM evaluation where test metrics mirror the training loss. [...] Showing that a path optimized to stay close to training points achieves a smaller distance to those exact points than unoptimized Linear or Spline baselines is trivial by construction and does not validate better semantic steering.

  • We agree that the original evaluation was in-sample, and we now evaluate on held-out prompts. Support graphs, endpoints and spline knots use training prompts only; every score below uses held-out prompts. On held-out prompts, RMS is closer to the model's natural outputs than linear and spline on both tasks (weekdays 0.126 vs 0.131 and 0.130; ages 0.258 vs 0.284 and 0.272; lower is closer). The weakest margin is weekday output against the spline (11/21 pairs). The full split and table, including activation proximity, are included in our response to Reviewer vSue (W11/Q9).

  • Our main score is not a training term. Output proximity is the Hellinger distance between the outputs along the path and the 5 nearest natural outputs (previously measured on the training points but now also measured on held-out prompts). As stated in Appendix C.2.6, it was never part of the training objective; it was, however, computed on the training references, which we now fix. Activation proximity has the same form as the support term, so we report it as secondary and it is more of a sanity check of the optimisation working as intended.

  • Is proximity trivial by construction? We test this with a baseline that directly optimises closeness to the training activations (B1, suggested by Reviewer opUL; full table in our response to opUL, W2). If staying close to training points were enough, B1 would win on held-out prompts. On weekdays it does not: B1 is the closest method on training prompts (0.079, vs 0.080 for RMS at the same budget) but the farthest on held-out prompts (0.133 vs 0.126), with RMS closer on 15/21 pairs. On ages, following the activation density is enough: B1 beats RMS at the paper's setting, and RMS matches it with the same 500 updates (0.228 vs 0.237, RMS closer on 14/21).

  • Semantic steering. We agree that proximity is not semantic steering. Our claim is that RMS paths stay close to the outputs the model naturally produces. We will qualify the sentence in Section 5.2 that RMS outputs "peak at intermediate concepts" (see W3), and state the claim as proximity throughout.

AuthorsOpenReview ↗

Response 2/3: W2 + W3

W2 / Q1 Answer Pairs, Unseen Prompts and Multi-Token Generation

Real-world steering requires ground-truth test answers during optimization, cannot generalize to unseen prompts, and fails to beat straight lines. [...]

Q1: [...] how can RMS be applied to steer unseen test prompts where ground-truth answer pairs are unavailable? Can the learned steering trajectories transfer to held-out prompts, and how does the method affect full multi-token autoregressive generation beyond the first token distribution?

The reviewer is right that Section 5.3, as written, has these limitations. They come from its per-prompt setup, not from the method. We ran a new experiment on MAxBench Countries that removes them (full table and setup are in our response to Reviewer opUL, See Responses 3/3 W3):

  1. No test answers. The endpoints are two continent centroids, not per-prompt answers: the same supervision as activation addition is needed. The training reward uses labelled Countries sentences only and test prompts need no answers (no groundtruths are needed). Since our centroids are unordered, splines do not work.
  2. Unseen prompts. We fit 15 paths once (one per continent pair; Llama-3.1-8B-Instruct, layer 19) and apply them unchanged to 100 AlpacaEval instructions that are not used to fit the paths and are held out.
  3. Multi-token generation. Steering acts at the last prompt token and at every generated token. MAxBench's LLM judge scores 30 full responses per instruction (up to 128 tokens) from 0 to 2 for concept, fluency and instruction following.

Result. RMS has the highest combined score of the steered methods within the steerable strength range. At α=0.6\alpha = 0.6 (steering magnitude), it raises the concept score from 0.39 (unsteered) to 1.08, with fluency and instruction following on par with difference-of-means steering (combined 0.99 vs 0.98). At α=0.65\alpha = 0.65, its fluency and instruction gains become significant (+0.17 [+0.08, +0.26], +0.16 [+0.07, +0.25]; combined 0.68 vs 0.50). At equal α\alpha, RMS expresses the concept less than difference of means (−0.17 and −0.13).

Section 5.3. We will present it as a per-prompt study of how the support term trades steering strength for output proximity, and move its limitation (Section 6, Appendix E.6) into the section itself. Against tuned mean-difference steering, RMS has lower output distance on all three behaviours and higher strength on two (Table 8; ten prompts, one seed).

W3 / Q2 Ordered Traversal and What the Path Adds

Failure to achieve ordered concept traversal on the toy LLM benchmarks. [...] RMS achieves 0 out of 21 pairs with ordered dominant concept visits [...]

Q2: [...] what concrete downstream behavioral benefit does optimizing a curved trajectory provide in LLMs beyond simply optimizing the endpoint or applying standard activation addition?

Ordered traversal. We agree: RMS does not visit the intermediate weekdays in order as the top answer. The spline is given the class labels and their order, and passes through the centroids by construction; RMS uses neither. Ordered visits are also not what RMS's objective targets. We will remove any wording that suggests ordered traversal, and qualify the sentence in Section 5.2 about peaks at intermediate concepts.

Q2: what the path adds. We do not claim a consistent benefit of curvature. Our claim is that RMS paths stay closer to the outputs the model naturally produces, which keeps steered outputs more natural. Curvature is the freedom that lets the path follow the data; it is not the goal in itself.

  • Closer to natural outputs. On held-out prompts, RMS paths stay closer to the model's natural outputs than linear and spline paths on both toy tasks (W1).
  • More fluent generations. On MAxBench, RMS is the most fluent and most on-task steered method at both strengths, against both difference-of-means steering and the optimised straight path (W2).
  • Strength, in some cases. RMS is stronger than tuned mean-difference steering on hallucination and myopia (Table 8), and curvature improves strength over the straight path (optimised linear method) on hallucination (0.749 vs 0.733). It does not on corrigibility and myopia.
  • A trade-off. The support term trades strength for proximity: raising its weight from 0 to 10 lowers output distance on all three behaviours at a small cost in strength (Table 8). On MAxBench, RMS expresses the concept less than difference of means at equal strength. We will state this trade-off explicitly.
AuthorsOpenReview ↗

Response 3/3: W4 + W5

W4 / Q3 Lemma B.1 and Length Versus Energy

Disconnect between the theoretical guarantee and the LLM implementation. Lemma B.1 proves that the support penalty bounds manifold deviation only under a constant Riemannian speed constraint [...]

Q3: Why did the LLM implementation minimize the piecewise-linear length functional instead of the quadratic energy functional used in the synthetic experiments [...]?

  • We agree. Lemma B.1 needs constant speed, and the LLM runs do not enforce it: path length is invariant to reparameterisation, but our support and proximity averages, taken at evenly spaced tt, are not. We will state in Appendix B and in the main text that the bound does not cover the LLM runs.
  • Why length (Q3). The support graph measures distance as shortest-path length, so we minimised the same quantity along the path. We had not tested energy in the LLM tasks; we now have.
  • What we tested. On held-out prompts (21 pairs per task): (a) training with energy, as in our synthetic experiments, which halves speed variation; (b) re-scoring the same paths at constant speed, weighting each stretch by its length instead of each tt-step equally. The full table is in our response to Reviewer vSue (W10/Q8).
  • Weekdays: no change. RMS stays closer than the spline under both changes (output difference −0.005 by default, −0.004 with energy, −0.005 at constant speed).
  • Ages: the reviewer's concern holds. RMS covers half its length in 20% of its tt-steps, exactly where the output switches age: the uneven speed the reviewer describes. At constant speed, RMS still beats linear (−0.015, 17/21 pairs) but no longer the spline (+0.011, 7/21), and training with energy does not close this gap (+0.005, 8/21).
  • We will report constant-speed scores alongside the default ones and state this limitation.

W5 Compute and Single-Token Evaluation

Heavy test-time compute overhead and single-token evaluation scope. Unlike standard activation addition, RMS trains a 404K-parameter network for every prompt [...] Moreover, all experiments only inspect the probability distribution of the first generated token rather than full multi-token generation.

  • We agree that RMS costs more than activation addition, by design. Activation addition estimates one direction. RMS optimises a whole path, and every point on it is a candidate intervention chosen to stay near the data, rather than a rescaling of one fixed direction. Optimising a path through the representation space, under the model's output geometry, is a harder problem than estimating one vector, and it costs more. We will state this cost in the main text as highlighted by the reviewer.
  • We note that the cost is paid per concept, not per prompt, when the endpoints are concept centroids. On MAxBench, the 15 paths are fit once and reused for every new instruction (W2); steering a new prompt then costs the same as activation addition. Only the per-prompt setting of Section 5.3 fits a path for each prompt.
  • Single-token evaluation. The toy tasks and the output-proximity scores use the next-token distribution. For free multi-token generation, we now show that MAxBench scores full responses of up to 128 tokens with an LLM judge (W2). We introduce this experiment in the rebuttal.
Reviewer_vSueOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes RMS, an interpolation method between activations that combines a pullback metric from the model’s output map with a penalty for staying near observed activations. It is evaluated on synthetic surfaces, toy weekday/age LLM tasks, and real-world steering tasks with a free endpoint.

Critical strengths
  • The paper identifies a real issue: steering paths can leave the region of naturally observed activations.
  • The synthetic experiments cleanly separate pullback-only and support-only failure modes.
  • The appendices contain substantial implementation detail.
  • The authors acknowledge that the real-world results are preliminary and that optimization and evaluation share prompts.
Critical weaknesses
  • Main concern: The paper is motivated by steering but essentially proposes an interpolation scheme. It does not motivate why interpolating—and hence why the intermediate points—matters. If the target behavior is known, why is the path between start and target useful?
  • The free-endpoint setting is handled somewhat ad hoc by optimizing the endpoint together with the path. This does not substitute for a numerical scheme based on the exponential map or geodesic shooting. Without such a scheme, I do not see how the method can deliver what it promises, even with major revisions.
  • The paper is hard to follow: the contribution is stated in a roundabout and verbose way, and datasets are introduced before the method.
  • The method is not implementable from the main-text description alone; the reader must assemble it from the metric, support estimate, neural bridge, and optimization details in the appendices.
  • The synthetic surfaces are not motivated as plausible models of LLM activation geometry. Why are these the right test cases?
  • In Section 2.2, it is unclear where the activations are extracted: which layer, token position, and prompt context? Are these activations or outputs?
  • The pullback metric g=J_F^\top J_F+\epsilon I may make directions in or near \ker J_F cheap. Could the optimizer exploit these directions to leave the manifold? Is F assumed full-rank or injective?
  • The Hellinger embedding appears convenient rather than statistically motivated. Why is it the right way to measure output change or naturalness?
  • The support penalty relies on observed activations, but the paper does not address how many are needed or whether coverage is realistic in high-dimensional activation space.
  • The practical objective averages support uniformly in t, so reparameterizations of the same curve can have different costs. The theory assumes constant metric speed, but the implementation does not enforce it; minimizers are therefore not unique and the returned path is arbitrary.
  • The toy-task evaluation uses the same training references for support estimation and proximity evaluation, so it is not an independent test.
Questions for authors
  1. Why is interpolation useful for steering? What role do the intermediate activations play beyond reaching the endpoint?
  2. In the free-endpoint setting, why not use an exponential-map/geodesic-shooting scheme? What addresses the associated numerical instability?
  3. Where exactly are activations extracted in the toy tasks?
  4. What justifies the synthetic surfaces as models of LLM activation geometry?
  5. What happens to paths that move through directions in or near \ker J_F?
  6. Why is Hellinger distance the appropriate output-space metric?
  7. How sensitive are results to the number and distribution of observed activations used for support estimation?
  8. How sensitive are the paths and metrics to path reparameterization?
  9. Can the proximity evaluation be performed on held-out activations or prompts?
6 comments
AuthorsOpenReview ↗

Response (1/3): Summary of Rebuttal and Addressing W1-W4

We thank the reviewer for the detailed questions. Here is a summary of changes:

  • New held-out evaluation (W11,Q9): with a clean train/test split, RMS is closest to held-out natural activations and outputs against both baselines.
  • Support-set ablation (W9,Q7): we find no consistent loss with a quarter of the references or with three days missing.
  • Reparameterization (W10,Q8): we find that reparametrisation affects our results, for instance our results hold on weekdays but on ages, RMS no longer beats the spline at constant speed. We will report this.
  • Presentation (W3, W4): We commit to have an algorithm block in our main text.

W1/Q1 Why the path matters if the target is known

Why is interpolation useful for steering? What role do the intermediate activations play beyond reaching the endpoint?

  • Steering already defines a path. Linear steering moves along one direction with growing strength, and every strength is an intermediate intervention.We fix the endpoints and study the path between them.
  • What we inspect is how these intermediate points behave: does the model's output at each intermediate activation stay close to the outputs it naturally produces on real inputs? If not, steering pushes the model into behaviour it never shows on data, and intermediate interventions break its outputs. A straight line in activation space can do this when representations lie on curved structures.
  • Staying in distribution can also matter for reaching intermediate concepts. Wurgaft et al. (2026) found that on labelled concept manifolds (for example, weekdays) the intermediate points of a path that stays on the manifold correspond to intermediate concepts; their spline steering reaches them by passing through the labelled centroids.

W2/Q2: Free endpoint and geodesic shooting

In the free-endpoint setting, why not use an exponential-map/geodesic-shooting scheme? What addresses the associated numerical instability?

In the free-endpoint setting, the start is fixed and the endpoint is chosen by the optimizer. We solve this as a direct method: we parameterize the path, discretize it, and minimize the full objective ( the path length, support penalty minus the endpoint reward) over the path parameters and the endpoint together. Shooting is an indirect method for a slightly different problem: it solves the geodesic equation of the pullback metric alone, which our objective does not reduce to. In particular:

  • Shooting finds geodesics of the pullback metric alone. Our objective adds an activation-support penalty and, in the free-endpoint case, an endpoint reward. Its solutions are therefore not plain geodesics, and shooting does not solve it directly.
  • Our synthetic results show what plain pullback geodesics do: without the support term, paths cut through the torus hole and recover the reference route on 0/6 pairs on the torus and the hyperboloid, while the full objective recovers 6/6 on every surface.
  • Shooting also requires second derivatives of the model (Christoffel symbols) in 4,096 dimensions; our optimization needs only first derivatives.
  • We do not claim that shooting is unstable in LLMs, however we expect it to have similar flaws as the plain pullback geodesics without the support penalty.

W3 and W4: Presentation and Algorithm in main text

We will (i) state the contribution in one sentence in the introduction, (ii) present the method before the datasets, (iii) move both algorithms into the main text (W4), and (iv) shorten the motivation.

Both algorithms will appear in the main text:

Algorithm 1: both endpoints fixedAlgorithm 2: free endpoint
InputGiven prompt cc; endpoints x0,x1x_0,x_1; observed activations XobsX^{\rm obs}; weight λ\lambdaGiven prompt qq; start xqx_q; steering vector vv; XobsX^{\rm obs}; weights λ,β\lambda,\beta
1Build graph GG on XobsX^{\rm obs} (pullback edge lengths)Build graph GG on XobsX^{\rm obs} (pullback edge lengths)
2γψ(t)=(1−t)x0+t x1+4t(1−t)Bψ(t)\gamma_\psi(t)=(1-t)x_0+t\,x_1+4t(1-t)B_\psi(t)x1←xq+vx_1\leftarrow x_q+v; γψ(t)=xq+t(x1−xq)+4t(1−t)Bψ(t)\gamma_\psi(t)=x_q+t(x_1-x_q)+4t(1-t)B_\psi(t)
3for τ=1,…,T\tau=1,\dots,T:for τ=1,…,T\tau=1,\dots,T:
4waypoints γi=γψ(i/N)\gamma_i=\gamma_\psi(i/N), i=0,…,Ni=0,\dots,Nwaypoints γi=γψ(i/N)\gamma_i=\gamma_\psi(i/N), i=0,…,Ni=0,\dots,N
5L=∑i<Nℓ(γi,γi+1)L=\sum_{i<N}\ell(\gamma_i,\gamma_{i+1}), R=1N+1∑irG(γi)R=\frac{1}{N+1}\sum_i r_G(\gamma_i)same, and Δb=bq(x1)−bq(xq)\Delta b=b_q(x_1)-b_q(x_q)
6Adam step on L+λRL+\lambda R w.r.t. ψ\psiAdam step on L+λR−β ΔbL+\lambda R-\beta\,\Delta b w.r.t. ψ,x1\psi, x_1
7return γψ\gamma_\psireturn the lowest-loss path found
AuthorsOpenReview ↗

Response (2/3) Addressing W5- W9

W5/Q4: Why Synthetic Surfaces?

What justifies the synthetic surfaces as models of LLM activation geometry?

  • They are controlled tests, not models of LLM activation geometry. LLM geometry is hard to evaluate on throughly as the true activation manifold is unknown, so we cannot measure how far a path leaves it; on known surfaces we can.
  • A known deformation changes which path is shortest in output space. This separates a path that follows the data but ignores the output (support only) from one that follows the output but leaves the data (pullback only).The surfaces cover positive curvature (sphere), negative curvature (hyperboloid) and holes (torus, genus-2).
  • The LLM tasks test the method where the geometry is unknown. We will state this appropriately. This purpose directly highlights the need to validate our methods on cases where ground truths exist compared to LLM tasks where the surface of the manifold is not known, but the data support is.

W6/Q3: Where are activations extracted?

Where exactly are activations extracted in the toy tasks?

  • We collect activations at the residual stream at the last prompt token using Llama-3.1-8B-Instruct, at the output of layer 28. We will make sure to clarify that in the main text. The outputs are the next-token probabilities over the concept tokens plus an "other" bin.

W7/Q5: Paths near ker J_F?

What happens to paths that move through directions in or near \ker J_F?

  • We do not assume F is full-rank or injective; that is why the metric is g=J⊤J+ϵIg = J^\top J + \epsilon I.
  • A direction in kerJF\mathrm{ker} J_F leaves the output unchanged to first order but costs ϵ∥v∥2\epsilon \|v\|^2. In general a direction with singular value σ\sigma costs σ2+ϵ\sigma^2+ \epsilon.
  • On top of that, the support penalty charges any move away from observed activations, so the optimizer is disincentivized to drift off the data. Without the support, paths are allowed to drift from the data distribution in the activation space more freely.
  • "Unchanged output" is local. A finite move preserves F only if the whole path stays in null directions.

W8/Q6: Why Hellinger?

Why is Hellinger distance the appropriate output-space metric?

Hellinger is a deliberate choice, motivated by information geometry and by prior manifold-steering work.

  • It is the Fisher geometry. The map p↦p/2p \mapsto \sqrt{p}/\sqrt{2} sends distributions to a sphere on which Euclidean distance is the Hellinger distance, and its local metric is exactly 1/8 of the Fisher information metric: ∥dF∥2=(1/8)∑idpi2/pi\|dF\| ^2 = (1/8) \sum_i dp_i^2/p_i. Our pullback metric is therefore the pulled-back Fisher metric up to a constant, and our path length is the Fisher–Rao length of the output trajectory. By Čencov's theorem, the Fisher metric is the unique Riemannian metric on distributions, up to scale, that is invariant under sufficient statistics (Amari, 2016).
  • It matches prior manifold-steering evaluation. Measuring an activation path by the length of its induced output trajectory follows Wurgaft et al. (2026), who evaluate steering paths with a Bhattacharyya coefficient based path energy. Hellinger distance satisfies H2=1−BCH^2 = 1-BC, and Fisher–Rao distance is 2arccos⁡(BC)2 \arccos(BC), so all three are monotone in the same coefficient and rank nearest neighbours identically.
  • Practical properties: It is symmetric, bounded in [0, 1], and finite when supports differ, unlike KL divergence, which matters for output distributions with near-zero bins.
  • Not unique: Any output metric in place of the Hellinger embedding (the pullback GFG_F = JF⊤gyJFJ_F^\top g_y J_F), and we do not claim Hellinger is the only valid choice.

W9/Q7: How many observed activations are needed?

How sensitive are results to the number and distribution of observed activations used for support estimation?

We see no consistent loss on results from using a smaller reference set for the training split which indicates low sensitivity to the number of training points.

We retrained RMS on weekdays with smaller and uneven support sets and scored every path on the 206 held-out prompts (See response to Q9). We used 3 endpoint pairs, drawn at random before running.

  • Support sets: all 823 training references; random subsets of 411 and 205 (3 draws each); and 411 references with three of the seven days removed entirely (3 draws).
  • Each reduced set against the full set on the same pair (mean change, lower is better; worse in how many of 9 = 3 pairs × 3 draws): Support set vs all 823 | Output | Activation | |---|---|---| | Random 411 | −0.001 (worse 6/9) | −0.001 (worse 4/9) | | Random 205 | −0.004 (worse 4/9) | +0.002 (worse 6/9) | | 411, three days missing | 0.000 (worse 4/9) | −0.001 (worse 5/9) |
  • We see no consistent loss from using a quarter of the references (about 29 per day) or from missing days. Changes go both ways and vary more between random draws than between set sizes.
  • With every one of the 10 support sets, activation proximity beats both baselines.
AuthorsOpenReview ↗

Response (3/3): Addressing W10–W11

W10/Q8: Sensitivity to Reparametrization?

How sensitive are the paths and metrics to path reparameterization?

We found that the paths can be sensitive to reparametrisation as the reviewer pointed out. While the path length is invariant to reparameterization, our support and proximity averages, taken at evenly spaced t, are not.The LLM runs do not enforce the constant speed Lemma B.1 assumes. We will state this clearly in our theoretical section in the appendix and the main text. We tested the effect two ways, on held-out prompts (21 pairs for each task):

  1. We train with energy, as in our synthetic experiments, which halves speed variation;
  2. We re-score the same paths at constant speed, weighting each stretch by its length instead of each t-step equally.
Output, RMS − baseline (wins/21)Default(a) Energy(b) Constant speed
Weekdays vs linear−0.005 (14)−0.005 (14)−0.006 (18)
Weekdays vs spline−0.005 (11)−0.004 (12)−0.005 (14)
Ages vs linear−0.026 (20)−0.007 (15)−0.015 (17)
Ages vs spline−0.015 (14)+0.005 (8)+0.011 (7)
  • On weekdays neither changes the result.
  • On ages the reviewer's concern holds. RMS covers half its length in 20% of its t-steps, where the output switches age, and once speed is even it still beats linear but no longer the spline. Activation proximity, where computed, shows the same pattern. We will report constant-speed scores alongside the default ones and state this limitation. However, we are not sure this is problematic in itself. It can be interpreted as the path lingering near more meaningful states, but also not entirely teleporting between them. Arguably, this may be a desirable property.

W11/Q9: Proximity Evaluation on Held Out?

Can the proximity evaluation be performed on held-out activations or prompts?

We thank this review for pointing out held out evaluations. We re-ran both toy tasks with a clean split and found that held-out proximity of RMS increased relative to all baselines. We observe that the support graph generalises to prompts it never saw.

  • Support graphs, endpoints and spline knots use training prompts only.
  • Weekdays: 823 train / 206 test prompts.
  • Ages: 827 train / 1,092 test prompts (12 per age).

We extended the test split with new prompts from the dataset's own generator, so that every age has enough held-out neighbours for a 5-nearest-neighbour score. No test prompt is used for fitting.

  • All 21 endpoint pairs per task. Proximity is the mean distance to the 5 nearest held-out natural outputs and activations; lower is closer.
Held-outRMSLinearSplineRMS − linear (wins)RMS − spline (wins)
Weekdays, output0.1260.1310.130−0.005 ± 0.002 (14/21)−0.005 ± 0.003 (11/21)
Weekdays, activation0.1180.1280.130−0.011 ± 0.002 (15/21)−0.013 ± 0.003 (16/21)
Ages, output0.2580.2840.272−0.026 ± 0.004 (20/21)−0.015 ± 0.005 (14/21)
Ages, activation0.2410.2690.252−0.028 ± 0.004 (20/21)−0.011 ± 0.005 (15/21)
(± is the standard error over the 21 pairs,)
  • On Weekdays, activation proximity is 8–9% better than both baselines; output proximity is 3–4% better, with a smaller margin against the spline.
  • Ages: RMS is closer than both baselines on both metrics, and beats linear on 20/21 pairs
Reviewer_opULOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds upon work on manifold steering and argues for a more unsupervised way to do steering while respecting the model's representation geometry: finding the steering path by pulling back output distribution points into activation space. The authors combine this pullback procedure along with a regularizing term (for straying off activation points) and term the method Riemannian manifold steering (RMS). They evaluate the method across synthetic settings, toy settings from previous studies, and tasks closer to real-world settings, and find that RMS produces more natural output distributions.

Critical strengths
  • The paper discusses a topic that has been of major interest to the mechanistic interpretability community: finding methods for controlling LLMs that generalize beyond the classic linear representation hypothesis (which treats concepts as lines). Importantly, the paper also focuses on how such a method can be made to be more unsupervised, which is certainly of interest to the community as well.
  • The paper is clearly written and presented, and its arguments were easy to follow despite the mathematical subject matter.
  • The paper's use of datasets from Rimsky et al. (2024) which are closer to real world setting is a great way to show that the method indeed allows for more unsupervised steering, since that is a substantial limitation of the spline-based method. It's synthetic settings, which examine the case of shrinking or expansion in the map between activation space and output space, were quite interesting as a case study.
Critical weaknesses

Given the fact that this work builds heavily on Wurgaft et al. (2026), I think the paper does not interface with its arguments directly enough.

  • First, Wurgaft et al. (2026) also defined and implemented pullback steering (which was not mentioned in the main text I believe). Furthermore, they made the argument that using activation manifolds vs. pullback is very similar in terms of the results, and hence relying on the easier activation-based method is preferred. If you disagree (for example, since spline fitting is too supervised) that is reasonable, but I believe it needs to be stated directly. Also, you fail to cite other concurrent work at several points, for example "RIEMANNIAN-MANIFOLD STEERING: GEOMETRY-AWARE GENERATIVE AUTOENCODERS FOR LABEL-FREE STEERING" which takes a different approach yet has many similarities.
  • Second, it might be worth noting that your metrics for evaluating closeness in both activation space and output space rely on nearest neighbor distance to actual points, while I believe Wurgaft et al. used distance to fitted manifolds which passed through the centroids. It is fair to say that your choice is less constrained, but it should be stated that this choice is partially what explains the superiority of the method.
  • Third, the paper somewhat suffers with respect to novelty given that Wurgaft et al. (2026) also used pullback, and there are other relevant concurrent works such as the one mentioned above. Highlighting and deepening the steering in the wild section would strengthen its novelty.

Finally, while I see the "steering in the wild" section as the strongest result, I think that section is somewhat briefly and confusingly described and the paper would benefit from its expansion.

Questions for authors

Clarification:

  • Why to use the pullback graph in evaluating distances to nearest observed activation?

Suggestions:

  • using MAxBench: A Multinomial Concept Recovery Benchmark (2026) would strengthen the "steering in the wild" section substantially, since the authors used manifold steering with splines but were only able to apply the method in constrained cases due to its supervised nature.
  • Wurgaft et al.'s argument was that following activation density (in their case via a fitted curve) is sufficient to get outputs near the natural manifold. Given that you are evaluating closeness in both activation and output space via a nearest neighbor metric (rather than closeness to a curve), it might be worthwhile to include an activation manifold fitting method that relies on minimizing nearest neighbor distance in activation space and compare it to both the spline fitting and RMS.
4 comments
AuthorsOpenReview ↗

Response 1/3: Summary + W1

We thank the reviewer for the close reading and the amazing suggestions! We ran both suggested experiments: the nearest-neighbour fitting baseline, and MAxBench (Suggestions 1 and 2). In summary:

  • Relation to Wurgaft et al. We state the link directly. We will also cite the concurrent work.
  • Does the nearest-neighbour metric explain our results? Partly, and we will say so. The reviewer's suggested activation density-following baseline is the closest method on training weekday prompts but the farthest on held-out ones, where RMS is closest. On ages, following the density is enough, as the reviewer suggests: this baseline beats RMS at the paper's setting and matches it with the same training budget. On a new synthetic manifold with corners, RMS follows the corner while the spline through the centroids cuts it.
  • Steering in the wild: RMS works in the MAxBench setting on an unordered concept (Countries), where the spline method cannot be applied. Paths fit on this dataset, transfer to unseen instructions, and RMS has the highest combined score of the steered methods at both strengths we test. It is more fluent and more on-task than a straight path trained with the same objective, consistent with staying near natural activations keeping outputs natural.

W1: Earlier Pullback Steering and Concurrent Work

First, Wurgaft et al. (2026) also defined and implemented pullback steering (which was not mentioned in the main text I believe). Furthermore, they made the argument that using activation manifolds vs. pullback is very similar in terms of the results, and hence relying on the easier activation-based method is preferred. If you disagree (for example, since spline fitting is too supervised) that is reasonable, but I believe it needs to be stated directly. Also, you fail to cite other concurrent work at several points, for example "RIEMANNIAN-MANIFOLD STEERING: GEOMETRY-AWARE GENERATIVE AUTOENCODERS FOR LABEL-FREE STEERING" which takes a different approach yet has many similarities.

  • We agree with the reviewer's link and will state this link directly. Wurgaft et al. define the pullback metric we use. Their pullback paths are computed as an inverse problem: a target path is fixed in output space, as a spline through labelled centroids, and we trace an activation path whose outputs reproduce it.

  • We relax this setting. Without labels, their order or a fitted output manifold, there is no output path whose pre-image we could compute. Instead, we estimate the pullback length directly on each interval of the activation path and minimise the total.

  • We find that neither term is enough alone. Without an output target, nothing keeps the path near natural activations, so we add a penalty for leaving the support of observed activations. We find that neither term is enough alone. On our synthetic surfaces, the support term alone keeps paths on the surface but never finds the route that is shortest in output space (0/6 pairs on all four surfaces). Pullback length alone finds that route on the sphere and genus-2 (6/6) but leaves the surface on the torus and hyperboloid (0/6). Only the combination recovers it on every surface (6/6; Table 1).

  • We agree that activation-based and pullback paths give similar results when labelled, ordered centroids are available. However, without them, following the activation density alone is not enough and here are some avenues where we disagree and we will state them clearly: (i) a spline cannot be built for an unordered concept, as on MAxBench Countries, where RMS needs only the two endpoint centroids (W3); (ii) on our synthetic surfaces, the support term alone, which follows activation density with no output geometry does not find the route shortest in output space (0/6 pairs on all four surfaces); (iii) more importantly the activation density-following baseline the reviewer suggested (B1) fits the training activations best but is farther than RMS from the model's outputs on held-out weekday prompts; on ages it matches or beats RMS (W2).

  • Concurrent work: We thank the reviewer for pointing this paper out and will cite the paper in the main text. It also studies pullback metrics, including a Hellinger-defined geometry, on low-dimensional representations. It differs from ours in three ways: it has no activation-support term, it does not commit to the output geometry, and it does not evaluate LLM steering in the wild. We also introduce synthetic surfaces with known geometry, which make the ablations in Table 1 possible.

AuthorsOpenReview ↗

Response 2/3: Addressing W2/S2

W2 and Suggestion 2: Does the Nearest-Neighbour Metric Explain the Results?

Second, it might be worth noting that your metrics for evaluating closeness in both activation space and output space rely on nearest neighbor distance to actual points, while I believe Wurgaft et al. used distance to fitted manifolds which passed through the centroids. It is fair to say that your choice is less constrained, but it should be stated that this choice is partially what explains the superiority of the method.

Suggestion: Wurgaft et al.'s argument was that following activation density (in their case via a fitted curve) is sufficient to get outputs near the natural manifold. Given that you are evaluating closeness in both activation and output space via a nearest neighbor metric (rather than closeness to a curve), it might be worthwhile to include an activation manifold fitting method that relies on minimizing nearest neighbor distance in activation space and compare it to both the spline fitting and RMS.

  • We agree, and will say so early in the paper. Our metric is less constrained than distance to a spline fitted through the centroids: it measures proximity to the observed distributions directly. This choice favours paths that stay near the data, and partly explains our advantage over the spline.

  • Our motivation: We want a path to pass near the centroids when they are dense and within the data's support. But the smoothness a cubic spline assumes (C²) need not be backed by the data; for instance, a piecewise-linear paths between the centroids, for example, could equally fit the data. Motivated by this, we therefore define the manifold more generally, as the support of the observed data distributions. This makes no particular smoothness assumption, and is likely to adapt better where the activation data has discontinuities or irregularities. (new experiment below)

New synthetic experiment: A manifold with corners.

A spline through class centroids is C², so it must round every sharp turn in the data while RMS has no such constraint and approaches the manifold's boundary well. We test this on the simplest manifold with corners: a noisy square.

Result: We find that RMS follows the corner of the square and hugs the manifold while avoiding the discontinuity at the corner. The spline avoids the corner. We observe this on all our tested pairs of centroids. See Figure: https://anonymous.4open.science/r/Riemannian-Manifold-Steering-Stays-on-a-Data-Informed-Track-rebuttal-E72A/synthetic_corner_iclr.png for a visual illustartion.

  • Data: we sample 5,000 points uniformly along the perimeter of a square of side 1 and add isotropic Gaussian noise (σ = 0.01). The direction of the square is discontinuous at each corner.
  • Classes and centroids: each point is labelled by the side it lies on, so there are four classes. A class centroid is the mean of its points, which lies at the middle of its side.
  • Paths: we connect the centroids of two sides (4 adjacent pairs and 2 opposite pairs). The spline is the periodic cubic spline through the four centroids in order, as for weekdays. RMS gets only the two endpoint centroids and the unlabelled points.

New baseline: nearest-neighbour fitting (B1)

To test how much of our advantage the metric explains, we ran the suggested baseline (B1): a path whose length is Euclidean and the activation penalty term is the Euclidean 5-nearest-neighbour distance to the training activations. It uses no output geometry in the form of the pullback metric. If the metric alone explained our results, B1 would match or beat RMS. B1 is trained for 500 updates ; we report RMS at the paper's setting (10 updates) and with the same 500 updates. We score output distance to the model's natural outputs, which neither method optimises directly (lower is closer, we use 21 centroid pairs):

Output distanceRMS, paper settingRMS, 500 updatesB1, 500 updatesLinearSpline
Weekdays, training prompts0.0850.0800.0790.0970.097
Weekdays, held-out prompts0.1260.1260.1330.1310.130
Ages, held-out prompts0.2580.2280.2370.2840.272
  • Fitting the training activations does not necessarily carry over to new prompts. On weekdays, B1 is the closest method on the training prompts but the farthest on held-out prompts. RMS is closer than B1 on held-out prompts at either budget (paper setting −0.007 ± 0.003, 500 updates −0.006 ± 0.004; RMS closer on 15/21 pairs in both), although on weekdays all held-out differences are small.
  • On ages, staying close to the activation density is what matters, as the reviewer suggests: B1 is closer than linear on all 21 held-out pairs, and closer than RMS at the paper setting (RMS closer on 5/21). With the same 500 updates as B1, RMS matches B1 (−0.009 ± 0.008, RMS closer on 14/21) and is closer than linear and spline on 21/21 and 20/21 pairs.
AuthorsOpenReview ↗

Response 3/3 : Clarification and W3/S1

Clarification:

Why to use the pullback graph in evaluating distances to nearest observed activation?

  • Raw Euclidean distance counts every direction equally, including directions the model's output ignores. The pullback graph measures distance by how much the output changes, plus a small Euclidean floor (ϵ\epsilon), which is the geometry steering acts through.

W3 and Suggestion 1: Novelty and Steering in the Wild

Third, the paper somewhat suffers with respect to novelty given that Wurgaft et al. (2026) ... to its supervised nature.

What is new in our method:

  1. Our method is only requires two endpoints and no intermediate labels/topology.
  2. A pullback length combined with an activation-support penalty, which neither term achieves alone (first point).
  3. Controlled synthetic surfaces with known geometry, which isolate what each term does.
  4. A free-endpoint formulation: a path is fit once per concept pair and transfers to unseen prompts and full generation (MAxBench).

Changes to the paper. We agree that steering in the wild is our strongest result and that it was too brief and thank the reviewer for their sugggestion. We will expand Section 5.3 with the MAxBench experiment, state the per-prompt limitation of the original setup in the section itself.

MAxBench with a Free Endpoint

We thank the reviewer for his suggestion, we ran RMS on MAxBench Countries (6 continents, 30 countries). The countries centroids are unordered, so MAxBench's spline method cannot be applied.

Result. We found that the RMS paths, steer unseen instructions and give the best balance of concept, fluency and instruction following: RMS has the highest combined score of the steered methods at both steering magnitude we test (Table 1).

  • The paths tend to be most fluent and most on-task. At both strengths, RMS is the most fluent and the most instruction-following steered method. At α=0.6\alpha = 0.6 (steering magnitude), it is on par with difference-of-means steering (fluency +0.07 [−0.04, +0.17], instruction +0.06 [−0.05, +0.17]. At α=0.65\alpha = 0.65, the gain becomes significant: fluency +0.17 [+0.08, +0.26], instruction following +0.16 [+0.07, +0.25] (combined scores are 0.68 vs 0.50).
  • Trade-off. At equal α\alpha, RMS expresses the concept less than difference of means while still raising it well above the unsteered model. We notice a trade-off between strength and fluency suggesting that higher concept strength is often traded off with fluency of generations.

Table 1. MAxBench Countries, scores 0–2 (higher is better; bold = best steered method at each strength). α=0.6\alpha = 0.6 and α=0.65\alpha = 0.65 are steering magnitudes selected based on a validation set. Combined = harmonic mean of the three averages; AxBench score = harmonic mean of each response's three scores, averaged over responses.

α\alphaMethodConceptFluencyInstructionCombinedAxBench score
—Unsteered0.392.002.000.840.41
0.6Difference of means1.250.830.940.980.28
0.6Optimised straight path1.480.620.740.830.26
0.6RMS, free endpoint1.080.901.000.990.24
0.65Difference of means1.430.330.440.500.12
0.65Optimised straight path1.560.250.350.400.13
0.65RMS, free endpoint1.300.500.610.680.15

Setup. We use Llama-3.1-8B-Instruct, layer 19, as in MAxBench. We use the free-endpoint formulation of Section 5.3. The path starts at one continent's centroid. The path and its endpoint are optimised for pullback length, activation support, and a reward which consists of the next-token log-probability of the target continent's countries minus that of the sentence's own country. We fit one path per continent pair (15 paths total) once and reuse it for every instruction.

We test on 100 unseen AlpacaEval instructions, generating 30 full responses per instruction and method (greedy decoding, up to 128 tokens); MAxBench's judge scores each response from 0 to 2. Steering acts throughout generation: at layer 19, every method replaces the activation hh with h′=(1−αj)h+αjsh' = (1-\alpha_j)h + \alpha_j s at the last prompt token (α0=α\alpha_0 = \alpha) and at every generated token jj, αj=0.9jα\alpha_j = 0.9^j\alpha, where ss is a point on the method's path. Each response uses a randomly drawn continent pair and position on the path, and the same draws are used for every method and both strengths, so the methods differ only in the target point ss. MAxBench's data is not released, so we regenerated Countries with their recipe. The path hyperparameters and α\alpha values were chosen on 5 validation instructions based on the values that showed highest harmonic mean of fluency, concept (steering strength), instruction following and diversity of responses on ALL methods.

I thank the authors for their detailed response to my review and for engaging with my suggestions! I think that proper attribution of credit and benchmarking against previous methods, as well as distinguishing the study from prior work (particularly through the MAxBench experiment), significantly strengthens the paper. I will maintain my weak acceptance score since I am not sure the paper is quite at the level of a 4, but I will advocate for its acceptance at the end of the review stage.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.