acceptodds
Under review as a conference paper at ICLR 2027

A Manifold-Aware Activation Steering Method for Large Language Models via Geometric Safety Control

11 views

Abstract

Inference-time activation steering enables low-cost, plug-and-play behavior control by directly modifying the hidden states of large language models (LLMs). Existing methods steer hidden states in the high-dimensional Euclidean space, making them vulnerable to task-irrelevant perturbations. They also rely on overly strong and directionally coarse single-step vectors that can push trajectories away from semantically dense regions, impairing representation acquisition and generation quality. To address these issues, this paper proposes GSCSteer (Geometric Safety Control Steering), a manifold-aware activation steering method grounded in geometric safety control. GSCSteer formulates single-layer activation intervention as a control barrier function-based quadratic program (CBF-QP) over a low-dimensional semantic manifold. To suppress task-irrelevant perturbations, the Kernel-Based Semantic Control Barrier Function (KS-CBF) models a nonlinear semantic boundary, while Orthogonal Tangent Subspace Projection (OTSP) confines the induced state-dependent corrections to a low-rank proxy tangent space. GSCSteer then replaces coarse, overly strong one-shot updates with multi-step integration, using Retraction-like Radial Correction (RRC) to limit norm drift and produce finer trajectories that more closely follow the empirical semantic geometry. Experiments on four open-source LLMs show that, compared with existing activation steering methods, GSCSteer improves the win rate on helpfulness tasks by 1.5%–3.6%, improves on truthfulness tasks by 2.9%–16.0% while reducing perplexity by 22.1%–44.0%, and reduces toxicity on detoxification tasks by 2.7%–24.1%.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, activation steering, representation geometry, control barrier functions, geometric safety control

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_Y1vMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GSCSteer, a test-time activation steering method that integrates a nonlinear semantic boundary, low-rank projection of its gradient, and norm correction during iterative hidden-state updates. Grounded in a control barrier function formulation, the method is practically instantiated through normalized projected gradient steps scaled by a fixed step size. Experiments across four open-source model families evaluate the method on helpfulness, truthfulness, and detoxification benchmarks. Additionally, the paper provides modular ablation studies, rank sensitivity analysis, and inference throughput measurements to assess efficiency trade-offs.

Critical strengths
  • Broad evaluation with encouraging behavioral improvements: The experiments cover four model families and three alignment tasks, with results reported over three random seeds. The improvements in truthfulness alongside lower perplexity provide encouraging evidence that the method can improve target behavior without worsening the measured fluency.

  • Useful ablations and practical cost analysis: Table 2 supports the contributions of the nonlinear boundary, low-rank projection, and norm correction. The appendix further examines projection-rank sensitivity, inference overhead, and the effects of adding the geometric modules to existing steering methods, helping clarify the approach’s benefits and practical limitations.

Critical weaknesses
  1. The CBF interpretation is not fully supported by the implementation: Appendix F.1 establishes directional agreement under active constraints, but the practical update uses a fixed normalized gain followed by radial correction. Constraint satisfaction and forward invariance are therefore not established for the complete algorithm, limiting the scope of its safety-control interpretation.

  2. The geometric interpretation lacks direct empirical support: The ablations show that low-rank projection is useful, but do not distinguish general regularization benefits from approximating semantic tangent spaces. Appendix F.2 states conditions for the latter interpretation without empirically assessing whether they hold in the evaluated settings.

Questions for authors
  1. With zero nominal control, the exact CBF-QP can leave already-safe states unchanged. Why does the implementation continue steering, and have you compared it with the closed-form multiplier in Appendix F.1?

  2. How does radial correction in Eq. (15) affect barrier scores and constraint satisfaction? Measurements before and after correction would help assess the complete update.

  3. Could you specify the kernel feature approximation and classifier training settings? Does the linear ablation use the same data and a comparable training protocol, with the rest of the pipeline unchanged?

2 comments
AuthorsOpenReview ↗

Controller Analysis, Stepwise RRC, and Kernel Implementation

Thank you for recognizing the behavioral results and practical analysis. We have added proofs and controlled comparisons addressing the multiplier, radial correction, and classifier settings.

Q1. Why continue steering in the desired region, and how does it compare with the exact multiplier? App. F.1 and the new App. I.1 compare the laws analytically. With p=Pk∇B≠0p=P_k\nabla B\ne0 and zero nominal control, the exact QP uses λ∗=max⁡{0,−α(B)/∥p∥2}.\lambda^*=\max\{0,-\alpha(B)/\|p\|^2\}. It can return zero once the boundary constraint is satisfied. Activation steering additionally aims to strengthen the target attribute, so GSCSteer follows the positive projected correction with a bounded normalized gain. It retains B˙=η∥p∥2/(∥p∥+ϵg)≥0.\dot B=\eta\|p\|^2/(\|p\|+\epsilon_g)\ge0. For B≥0B\ge0, this satisfies the continuous CBF inequality; at B<0B<0, the condition is B˙≥−α(B)\dot B\ge-\alpha(B). The analytical comparison specifies the retained direction and ascent property. The added empirical controls separately test boundary expressivity and state dependence (App. H.2, Table 14):

VariantT×I (%) ↑
Linear-CBF50.0
Frozen KS-CBF direction59.5
Random equal-rank subspace41.0
Final norm matching60.5
GSCSteer64.0

These are a common 200-example LLaMA/TruthfulQA diagnostic, distinct from the three-seed main evaluation.

Q2. What happens to the barrier after radial correction? RRC acts after each Euler step (Eq. 15, Algorithm 2). The classifier normalizes its activation input before sketching. Under ideal normalization, for nonzero proposal zz and c>0c>0, B(cz)=B(z)B(cz)=B(z). RRC therefore restores radius while preserving the proposal's directional barrier score. App. I.2 proves this and shows that the effective first-order barrier ascent is also retained because h⊤∇B(h)=0h^\top\nabla B(h)=0. It further bounds the score change due to the 10−1210^{-12} input-normalization stabilizer.

App. I.3 supplies the complete finite-step bound, with a=∇B(h)⊤u~a=\nabla B(h)^\top\widetilde u: B(hs+1)−B(hs)≥Δτa−L2Δτ2∥u~∥2−Erad.B(h_{s+1})-B(h_s)\ge\Delta\tau a-\tfrac L2\Delta\tau^2\|\widetilde u\|^2-E_{\mathrm{rad}}. Here LL bounds local gradient variation and EradE_{\mathrm{rad}} is the bounded stabilizer error; a nonnegative right side is sufficient for stepwise ascent. The same appendix proves cumulative radius preservation/bounded contraction. These analytic results address the score change directly. Table 2 evaluates its downstream role: removing RRC raises TruthfulQA PPL on all four models, e.g. 35.651→50.430 on Mistral.

Q3. Kernel approximation, fitting, and linear controls. New App. G.2 specifies normalized polynomial CountSketch/TensorSketch, fixed random hash/sign maps, and FFT products. The kernel on normalized activations is (0.1x⊤y+1)2(0.1x^\top y+1)^2, approximated with 8,000 features. Positive/negative activations are labeled 1/0. Fitting uses LogisticRegression(max_iter=1000) with L2, C=1, lbfgs, intercept, tolerance 10−410^{-4}, and no class reweighting. The fitted readout and sketch are frozen at inference; the gradient uses an analytic vector–Jacobian product.

The w/o KS-CBF row is a controller-level ablation using single-step linear steering. App. G.2 now states this explicitly and separates it from the added Linear-CBF and frozen-direction controls above. The latter controls examine boundary modeling and direction recomputation on the common diagnostic, avoiding conflation of a controller replacement with a classifier-only comparison. The table supports the joint geometric-control design through complementary, targeted interventions.

The revised manuscript also adds COPA and AxBench (Secs. 5.4–5.5, Tables 3–4), paired bootstrap intervals (App. H.1), and component/step-count costs (App. B.1–B.2), extending both behavioral and practical validation.

Thank you for the clarifications and additional experiments. The implementation details and normalization-based RRC analysis substantially alleviate my concerns. However, the new controls support the usefulness of the learned subspace more directly than its semantic tangent-space interpretation, and their limited scope leaves some uncertainty. On balance, I remain borderline but retain my rating of 2.

Reviewer_f7d3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The proposed steering method consists in:

  1. using as steering direction the gradient of a classifier over activations (barrier function). The classifier distinguishes positive activations (carrying the property) from negative ones (not carrying the property),
  2. filtering this direction by restricting to the top-k highest-variance principal directions of calibration activations.
  3. approximating the steering trajectory with a piecewise linear trajectory, recomputing the direction and applying a proper normalizing factor at each step. Results are evaluated on 4 models (all 7B size) and 3 tasks: Helpfulness (UltraFeedback), truthfulness (TruthfulQA) and Detoxification (RealToxicityPrompts), claiming improvements on the target steering metrics while maintaining or improving (in the truthfulness case) the control metrics given by a reward model, PPL + Informativeness and PPL + Dist-2 (a bigram preservation metric) respectively.
Critical strengths
  1. The idea of projecting the steering segment with PCA is simple and, to my knowledge, novel in this specific setting.
Critical weaknesses
  1. The most relevant similar method is discussed only en passant as a baseline. The two main ideas of using the classifier gradient as the steering direction and piecewise linearly approximating the steering trajectory were already present in [1] (ODESteer). The main novelty is the PCA filtering component, but the paper presentation gives the idea that the other two components are also introduced in the current paper.
  2. PPL main table and ablation numbers for the truthfulness task are unusually high (up to 215) for the unsteered model, and the PPL computation procedure is not described at all, making it difficult to trust and analyze. Given PPL is the main control metric, understanding the procedure is very important for evaluating the results.

[1] H. Zhao, H. Sun, J. Kong, X. Li, Q. Wang, L. Jiang, et al. ODESteer: A unified ODE-based steering framework for LLM alignment, 2026.

Questions for authors
  1. What procedure have you used to compute PPL for the truthfulness and detoxification tasks?
2 comments
AuthorsOpenReview ↗

Geometric Control Contributions and PPL Evaluation Protocol

Thank you for highlighting the contribution and requesting the PPL protocol. We have made the evaluation fully explicit and added controlled evidence for the geometric-control design.

1. The contribution is geometric control of the steering trajectory. GSCSteer formulates activation intervention through a learned semantic constraint coupled to an empirical low-rank geometry. KS-CBF supplies the state-dependent semantic boundary, OTSP constrains the proposed direction, and stepwise RRC stabilizes representation scale throughout integration. The low-rank CBF-QP yields an analytic correction direction; Apps. F.1 and I now establish its relation to the practical controller, score ascent, radial correction, and finite-step behavior. Our contribution concerns this coupled control formulation and its realization in activation space.

Its value is tested by interventions on the mechanisms, not only by comparing final scores. The added 200-example LLaMA/TruthfulQA diagnostic in App. H.2, Table 14 reports:

VariantT×I (%) ↑
Linear-CBF50.0
Frozen KS-CBF direction59.5
Single-step linear steering59.5
Multi-step linear steering58.0
Random equal-rank subspace41.0
Final norm matching60.5
GSCSteer64.0

The results separate nonlinear boundary modeling, state dependence, data-informed geometry, and norm control. Repeating a fixed linear direction does not reproduce the full result. App. D/Table 9 also shows transfer of the geometric operators to other steering methods: for example, CAA+GM improves all four reported metrics, while Linear-AcT+GM reduces PPL by 35.12%.

2. Exact PPL procedure for both tasks. New App. G.1 specifies a common, independent GPT-2-XL evaluator (openai-community/gpt2-xl) and its matching tokenizer for TruthfulQA and detoxification. It scores saved postprocessed completions only, without prepending the generation prompt. No steering is applied to the evaluator. Tokenization uses padding and truncation to the tokenizer's supported length; EOS is used for padding, excluded by the shifted attention mask.

Let mitm_{it} denote the mask of predicted token xitx_{it}. We shift logits/labels by one position, use unreduced next-token cross-entropy, and compute PPLi=exp⁡(−∑t=2Limitlog⁡p(xit∣xi,1:t−1)∑t=2Limit).\mathrm{PPL}_i = \exp\left(-\frac{\sum_{t=2}^{L_i} m_{it}\log p(x_{it}\mid x_{i,1:t-1})}{\sum_{t=2}^{L_i} m_{it}}\right). The first token is unscored. Each completion is exponentiated separately, then the dataset statistic is the arithmetic mean of valid per-completion PPL values (np.nanmean). Table 1 reports the mean and SD of the seeded runs. This specifies the checkpoint, context, masking, truncation, and aggregation consistently for every method.

3. Why the absolute PPL can be high. This is external-evaluator perplexity of sampled completions, not the backbone's own perplexity. The external distribution, completion-only context, and averaging after exponentiation determine its scale. Its interpretation therefore comes from common-protocol comparisons together with task metrics. For Falcon/TruthfulQA (Table 1; mean ± SD across three seeds):

MethodT×I (%) ↑PPL ↓
Original28.3±0.946215.244±24.836
ODESteer39.2±1.32995.331±6.982
GSCSteer42.5±2.04153.348±1.804

The lower PPL accompanies higher truthfulness/informativeness. The added COPA and AxBench evaluations in Secs. 5.4–5.5 also assess objective causal accuracy and separately judged concept/instruction/fluency scores, respectively (Tables 3–4), broadening the evidence beyond perplexity.

I thank the authors for the rebuttal. The PPL protocol is specified, and Table 14 and the new evaluations are useful. However, the specified protocol (completion-only scoring, arithmetic mean of per-completion PPL) makes TQA-PPL very sensitive to answer length and to a few short outliers. This plausibly explains both the unsteered value of 215 and the large drops under steering, which do not appear on RealToxicityPrompts, where continuation length is fixed. I therefore still do not see TQA-PPL as evidence of fluency. I will keep my score.

Reviewer_aAghOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes GSCSteer, an inference-time activation steering framework that introduces geometric constraints inspired by control barrier functions (CBFs). It is motivated by the concern that conventional steering interventions in high-dimensional Euclidean activation space may introduce task-irrelevant perturbations and drive representations away from semantically dense regions.

It contains three modules: (1) Kernel-Based Semantic Control Barrier Function that learns a nonlinear semantic decision boundary from positive and negative contrastive activations and provides state-dependent steering gradients, (2) Orthogonal Tangent Subspace Projection that uses truncated SVD to estimate a low-rank proxy tangent subspace and constrains steering updates to this subspace, and (3) Retraction-like Radial Correction that applies norm correction after each integration step to mitigate activation-scale drift.

Critical strengths
  1. The paper addresses an important problem in inference-time activation steering and presents an interesting geometric perspective by integrating nonlinear semantic boundary modeling, low-rank projection, and norm stabilization. The paper identifies two important limitations of conventional activation steering: fixed global steering directions and unconstrained updates in the ambient activation space. Reformulating activation intervention as a geometric control problem provides a potentially useful perspective for addressing these limitations.

  2. The three components target complementary problems: KS-CBF models nonlinear semantic boundaries, OTSP constrains intervention directions, and RRC stabilizes activation norms. This decomposition makes the framework relatively interpretable and facilitates component-wise analysis.

  3. The paper investigates the effects of removing KS-CBF, OTSP, and RRC, and includes projection-rank sensitivity analysis, activation-space visualizations, and plug-and-play experiments. These analyses offer some insight into the individual contributions of the proposed components.

Critical weaknesses
  1. OTSP estimates a fixed low-rank subspace from calibration activations using truncated SVD. However, PCA primarily captures directions of high empirical variance, which do not necessarily correspond to the local tangent directions of a semantic manifold. Although the supplementary material provides theoretical conditions for exact tangent-space recovery and bounds the approximation error using principal angles, these results depend on assumptions such as local affinity, approximately shared tangent spaces, and sufficient calibration coverage.

In practical LLM representations, semantic structures may exhibit substantial input-dependent curvature and local variation. Consequently, a globally estimated PCA subspace may fail to capture the appropriate local geometry for individual hidden states.

  1. RRC rescales each intermediate activation to approximately preserve its previous L2 norm. However, preserving the activation norm does not necessarily imply preserving its semantic information or keeping it within a high-density region of the representation space.

Moreover, the proposed correction is applied independently at each integration step, and its cumulative effect on activation norms and representation geometry is not sufficiently investigated. The authors should provide more direct evidence that norm preservation improves semantic consistency and generation quality, rather than simply introducing an additional normalization operation. It would also be useful to compare RRC with simpler normalization strategies.

  1. Although the paper evaluates six baselines across four LLMs, the improvements are not consistent across all tasks and models. For example, GSCSteer does not achieve the best helpfulness result on Falcon-7B, and its detoxification performance is slightly worse than ODESteer on LLaMA3.1-8B and Qwen2.5-7B.

This suggests that the proposed geometric constraints may provide task-dependent benefits rather than consistently improving activation steering.

  1. The paper reports that GSCSteer reduces inference throughput by approximately 10.8%–14.8% compared with the original models. Although the additional overhead relative to ODESteer is relatively small, this result indicates that the proposed method is not computationally negligible. The evaluation would also benefit from reporting latency under different integration steps and projection ranks to demonstrate the practical scalability of the method.
Questions for authors

See weakness

2 comments
AuthorsOpenReview ↗

Geometric Evidence, Statistical Validation, and Efficiency

Thank you for recognizing the geometric motivation and complementary roles of the components. The revision adds controlled comparisons, paired statistics, cost decomposition, and two further tasks.

1. Why the empirical geometry matters. OTSP selects task-specific directions from centered calibration activations. App. C.1 now combines the variance concentration in Table 8 with overlapping positive/negative residual distributions (reported mean magnitudes 4.5–5.1). This supports shared coarse geometry with different semantic regions. Fig. 3 and App. F.2 connect this proxy to locally smooth geometry and derive separate estimation/curvature error terms.

The new controlled study directly tests direction selection and simpler normalization. On 200 TruthfulQA examples with LLaMA, Table 14 reports:

ControllerT×I (%) ↑
Random equal-rank subspace41.0
Linear-CBF50.0
Frozen KS-CBF direction59.5
Final norm matching60.5
GSCSteer64.0

The 23-point difference from random equal-rank projection shows that generic rank reduction alone does not explain the result. The final-norm control and frozen-direction control further identify the value of coordinating geometry, state dependence, and scale stabilization.

2. RRC and cumulative effects. RRC acts after each Euler step. App. I.3 proves exact radius preservation when the stabilizer is zero; with proposal norms bounded below by qmin⁡q_{\min}, the relative contraction is at most Nϵr/qmin⁡N\epsilon_r/q_{\min}. Thus repeated corrections prevent cumulative radius expansion. App. I.2 additionally proves that positive radial scaling preserves the proposal's barrier score for ideal normalized features and bounds the stabilizer error. This connects RRC to the learned semantic score, beyond norm equality itself.

Its downstream contribution is visible in Table 2: removing RRC raises TruthfulQA PPL on all four models. For Mistral, PPL changes from 35.651 to 50.430 and T×I from 64.6% to 62.7%. The added final-norm comparison above complements this component ablation.

3. Task/model differences and paired evidence. We assess target behavior and output quality jointly. GSCSteer has the highest TruthfulQA T×I on all four models. Even on Falcon helpfulness, Win improves over ODESteer from 46.0% to 51.2%. App. H.1/Table 13 adds 10,000 paired bootstrap resamples. Positive differences favor GSCSteer; Win/T×I differences are percentage points.

ModelΔWin [95% CI]Δ(T×I) [95% CI]
LLaMA+6.70 [3.77,9.53]+1.80 [-0.16,3.76]
Falcon+5.20 [2.24,8.14]+3.30 [1.16,5.41]
Mistral+3.00 [0.12,5.87]+8.90 [6.68,11.10]
Qwen+7.10 [4.15,10.03]+3.20 [1.18,5.23]

Helpfulness gains are supported on all four models and truthfulness gains on Falcon/Mistral/Qwen. For the small LLaMA/Qwen Toxic differences, the paired intervals include zero; the generation-quality gains in Table 1 remain part of the comparison. Secs. 5.4–5.5 further add COPA (87.4% versus ODESteer's 74.0%) and AxBench, where GSCSteer has the highest three-seed mean C/I/F (Tables 3–4).

4. Practical cost and operating choices. New App. B.1/Table 6 decomposes steering CUDA time: KS-CBF features/gradient 46.6%, OTSP 38.2%, and RRC 0.5%. Additional static storage is 4.10 MiB against approximately 30.68 GiB peak memory. App. B.2/Table 7 gives the fixed-T=5T=5 step trade-off:

Euler stepsT×IController evaluations vs. N=10
50.6350.5
100.6401.0

Five steps retain closely comparable observed quality with half the controller evaluations. This measures integration work, not a claimed twofold end-to-end speedup. App. B reports only 1.9%–3.5% additional throughput reduction relative to ODESteer. Together with the rank sweep in App. A and O(dk)O(dk) projection cost, these results give concrete controls for balancing quality and computation.

Thanks for the response and clarification. The author’s additional information was helpful, so I have increased my score.

Reviewer_uH3sOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a noval activation steering method that uses gradients of a nonlinear classifier, orthogonal low-rank projection, and norm correction to update hidden states over multiple steps. Experiments on four open-source language models show slightly better overall performance on helpfulness, truthfulness, and detoxification tasks, while substantially reducing output perplexity (especially on truthfulness).

Critical strengths
  1. The idea of formulating activation steering as a CBF-QP within a low-rank representation subspace is interesting. This provides a useful control-theoretic perspective on the relationship between semantic constraints and steering directions.

  2. The empirical results are promising across the three selected alignment tasks and four language models. The method achieves competitive or improved task performance while reducing perplexity, with particularly substantial reductions on truthfulness. The experiments also cover a reasonably broad set of activation steering baselines. Together, these results support the practical value of the proposed approach.

Critical weaknesses
  1. The connection between the CBF-QP formulation and the implemented algorithm remains unclear. The implementation sets the nominal control to zero and replaces the optimal state-dependent multiplier with a fixed-gain normalized gradient. Although this can preserve the correction direction when the constraint is active, it is not generally equivalent to solving the QP. The paper should clarify which properties of the CBF-QP formulation carry over to the actual algorithm.

  2. The manifold interpretation is not sufficiently supported by the experiments. While Appendix F.2 provides conditional approximation results, the paper does not establish through experiments that the required assumptions hold in practice. It therefore remains unclear whether the improvements arise from alignment with manifold geometry or from a more general low-rank regularization effect.

Questions for authors
  1. How should the CBF-QP formulation be interpreted in relation to the implemented algorithm? In particular, which properties of the QP solution are retained when the optimal multiplier is replaced with a fixed-gain normalized gradient, and do they continue to hold after radial correction?

  2. Could the authors provide results on AxBench, including a comparison with ODESteer? Reporting concept control, instruction following, and fluency would help me assess whether the method’s benefits extend beyond the three selected alignment tasks.

  3. Could the authors discuss whether the shared or slowly varying tangent-space assumptions are supported by the activation distributions in their experiments?

2 comments
AuthorsOpenReview ↗

Control Properties, AxBench Results, and Empirical Geometry

Thank you for recognizing the control perspective and the cross-model results. We have added AxBench, COPA, geometric diagnostics, and proofs to address your three questions directly.

Q1. Properties retained from the CBF-QP and after RRC. With p=Pk∇B(h)p=P_k\nabla B(h), the active zero-nominal QP correction and our normalized controller share the positive direction pp. The practical field retains B˙=η∥p∥2/(∥p∥+ϵg)≥0.\dot B=\eta\|p\|^2/(\|p\|+\epsilon_g)\ge0. This satisfies the continuous CBF inequality for B≥0B\ge0; at B<0B<0, the condition is B˙≥−α(B)\dot B\ge-\alpha(B). The normalized gain controls intervention strength while continuing to enhance the target attribute. App. F.1 and the new App. I.1 distinguish this attribute-steering objective from the QP's minimum-intervention objective.

RRC is applied after every Euler step. Because the kernel features normalize their input, positive radial scaling preserves the proposal's barrier score under ideal normalization: B(cz)=B(z)B(cz)=B(z) for c>0c>0. App. I.2 proves this and bounds the effect of the numerical stabilizer. App. I.3 derives a sufficient finite-step ascent condition for the complete update, including radial error, and the cumulative radius bound. These results specify how the control interpretation carries through normalization and discretization.

Q2. Added AxBench comparison and broader applicability. Sec. 5.5, Table 4 reports LLaMA3.1-8B-Base, layer 14, FP32, ten fixed concepts, and 20 test instructions per concept in each of three complete seeded runs. Methods use identical instructions and paired generation seeds within each run. A common DeepSeek deepseek-flash judge scores C/I/F on 0–2 scales. Entries below are mean ± sample SD.

MethodConcept C ↑Instruction I ↑Fluency F ↑
Original0.0700±0.03281.7333±0.04371.5417±0.1729
ODESteer0.0750±0.02181.6933±0.14581.6150±0.0492
GSCSteer0.0783±0.01531.7817±0.05011.6300±0.0541

GSCSteer has the highest mean on all three dimensions; I and F improve over ODESteer in every paired seed. App. H.3 gives each run, frozen validation-selected strengths, and sample variances.

Sec. 5.4, Table 3 additionally evaluates objective causal reasoning on COPA. Components are fitted on 500 development examples and evaluated on 500 separate test examples, transferring TruthfulQA hyperparameters without COPA-specific tuning. Accuracy is in percent.

MethodOverallCauseEffectChoice 1Choice 2
Original67.662.472.835.2100.0
ODESteer74.070.078.049.698.4
GSCSteer87.484.890.086.888.0

The 13.4-point gain over ODESteer covers both relation types, with a 1.2-point answer-position gap.

Q3. Do the activation distributions support shared or slowly varying tangent spaces? They support the task-specific shared approximation used by OTSP. Table 8 gives:

TaskVariance explained by top 32 PCs
TruthfulQA33.75%
Toxicity33.31%
UltraFeedback48.44%

Positive/negative subspace-residual distributions strongly overlap, with reported mean magnitudes in the 4.5–5.1 range (new App. C.1), consistent with a shared coarse skeleton and semantic differences within it. Crucially, the 200-example diagnostic gives 64.0% T×I with the empirical subspace versus 41.0% with a random equal-rank subspace (Table 14): its orientation matters beyond dimension reduction.

For slowly varying geometry, the continuous curved point clouds in Fig. 3 are qualitatively consistent with local smoothness. We interpret this condition over short state-dependent updates; App. C.1 connects these observations to the estimation/curvature error decomposition in App. F.2. Together, the distributional evidence and matched-rank control support a useful data-informed geometric proxy.

I thank the authors for the detailed response and the effort put into the additional experiments and proofs. After reading through the revisions however, I still feel that my main concerns remain unresolved.

First, on Q1 regarding the CBF-QP formulation, the new appendix confirms that the implemented algorithm inherits only the ascent direction from the quadratic program. While Eq. 72 holds for any projected gradient ascent and guarantees non-decreasing values once B≥0B \ge 0, steering in practice begins from unsafe states where B<0B < 0. In that region, Eq. 73 goes unverified. Furthermore, a CBF guarantees only forward invariance, whereas driving an unsafe state into the safe set is a reachability property. The appendix also acknowledges that the method drops the QP's minimum-intervention behavior and that RRC moves updates outside the projected subspace. Because the algorithm behaves differently from an approximation of the QP, the gap between the control-theoretic framing and what the method actually does still leaves my first question unanswered.

Second, on Q2 regarding AxBench and COPA, I appreciate the added evaluations, though they leave me unconvinced about broader applicability beyond the three alignment tasks. On AxBench, concept relevance sits at only 0.078 for GSCSteer and 0.075 for ODESteer compared to 0.070 for the unsteered model on a 0 to 2 scale, which falls well within seed variation. In addition, when steering barely shifts concept relevance, preserving fluency and instruction following becomes expected and offers little evidence of a real advantage. On COPA, the unsteered model favors the second choice almost every time (35.2% versus 100% accuracy split by answer position), which suggests that the gain probably only reflects position-bias correction instead of real improvement in causal reasoning.

Third, on Q3 regarding the manifold interpretation, narrowing the claim to a "task-specific low-rank proxy" aligns closely with the alternative explanation I raised in my initial review, and the evidence provided fails to separate this proxy effect from genuine manifold alignment. In d=4096d = 4096, a random rank-128 subspace keeps roughly 3% of the gradient energy, so its failure is well expected under either view. Meanwhile, the variation of the tangent space remains unestimated, and Proposition 3 guarantees exact recovery only when the data lie in a flat affine subspace, where the manifold reduces to a standard PCA subspace.

It seems to me that the KS-CBF barrier and integration settings here appear very similar to those of ODESteer, making this work effectively ODESteer combined with PCA gradient projection and per-step norm rescaling. Over ODESteer, the empirical gains look quite modest, as the LLaMA truthfulness interval includes zero while toxicity increases slightly on LLaMA and Qwen. Since the control-theoretic and manifold framing adds substantial conceptual complexity without matching guarantees or clear empirical gains, my decision is to keep the original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.