acceptodds
Under review as a conference paper at ICLR 2027

Concept Control in Language Models via Activation Straightening and Clamping

Abstract

Representation intervention aims to control concepts in LLMs by intervening on internal activations. While promising, existing methods often compromise general model utility or fail to generalize under distribution shifts. We argue that these failures of faithful control stem from two structural obstacles in latent space. First, concepts are redundantly distributed across layers. Sparse interventions allow the model to bypass the intervention (the "hydra effect"), whereas naively intervening at many layers can over-correct activations and degrade utility. Second, the activation patterns that encode a concept can vary with context, so a simple linear subspace often fails to capture the geometry of the concept and can perturb unrelated capabilities, especially on out-of-distribution prompts. We propose Straighten-and-Clamp (SAC). SAC learns an invertible straightening map, which concentrates concept-related variation, followed by a projection clamp to enforce a target concept code in the straightened coordinates. Because the clamp is idempotent at each site, it becomes inactive once the target code is already satisfied, enabling stable multi-layer deployment. We train SAC in two stages: first, we learn a straightening map amortized across concepts; second, we learn lightweight per-concept clamps via pairwise preference optimization. Experiments on concept induction and suppression of harmful concepts suggest that SAC improves the control–utility trade-off and robustness to distribution shifts.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM post-training, representation intervention, faithful concept control

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_uRFMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SAC introduces a representation steering framework that pairs a learned invertible coordinate transformation with a concept-specific projection clamp. Training proceeds in two stages: first, an invertible mapping shared across concepts is learned to concentrate concept-related variation into low-dimensional subspaces; second, this transformation is frozen while concept-specific clamping parameters are optimized. Under stated assumptions, the intervention is provably idempotent at each intervention site and leaves representations already satisfying the target coordinate constraints unaffected. This approach is evaluated across concept induction and harmful-concept suppression tasks, with component ablations validating the synergistic benefit of coupling the learned transformation with projection clamping.

Critical strengths
  1. Clear Design with Useful Mathematical Properties: SAC sets concept coordinates to a target value while leaving representations that already meet the constraint unchanged. The paper formally establishes fixed-point and idempotence properties at each intervention site, making the intervention easier to understand.

  2. Component Ablations Support the Combined Approach:
    Appendix D compares full SAC with native-space clamping and straightened additive interventions under a matched training protocol. Full SAC performs better on concept induction across both models, supporting the value of combining the learned transformation with clamping.

Critical weaknesses
  1. The control–utility advantage needs stronger evidence: ACE achieves a lower average OOD ASR than SAC in Table 1 (10.7% vs. 14.5%). The paper argues that SAC causes less over-refusal, but Figure 2(c) does not compare it with ACE, making the trade-off difficult to assess. AxBench’s instruction-relevance and fluency scores also offer limited evidence that general task capabilities are preserved.

  2. The evaluation scope is limited: Concept induction is evaluated on two models of similar size, while the main harmful-concept suppression results are reported only for Qwen-2.5-7B. This leaves the generality of the claimed robustness and multi-layer stability unclear.

Questions for authors
  1. How does SAC compare with ACE on benign-prompt over-refusal? Since ACE has lower OOD ASR in Table 1, this comparison would help assess their control–utility trade-off.

  2. Could you report the separate Concept, Instruct, and Fluency scores for SAC and the strongest baselines in Table 2? This would help clarify whether the improvement comes mainly from stronger concept induction or also from better preservation of instruction relevance and fluency.

Reviewer_ehVeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses concept control in LLMs through representation intervention. The authors argue that current methods fail for two reasons. First, concepts are spread across many layers, so sparse edits are bypassed (the hydra effect), while dense additive edits pile up and hurt utility. Second, concept geometry depends on context, so a single linear subspace is only locally valid. The authors propose Straighten-and-Clamp (SAC). Experiments on AxBench Concept500 and harmful-concept suppression on Qwen-2.5-7B show better harmonic scores and lower average OOD ASR than ReFT, LoFiT, SV-1, LinEAS and several training-free methods.

Critical strengths

The main idea is a clean one. Instead of asking “which vector should we add,” the paper asks “which constraint should the state satisfy,” and it picks an operator whose fixed points are exactly that constraint. Putting this projection inside an invertible learned coordinate system is a sensible combination. It keeps the simplicity of linear clamping while allowing curved concept geometry. I also like the separation between a shared geometry map and cheap per-concept parameters. It is a practical design for serving many concepts.

Critical weaknesses

The first concern is how much the idempotence argument really buys in the multi-layer setting. Proposition 3.1 is a per-site statement. Each layer has its own glg_l, UlU_l and blb_l, and the transformer blocks between sites change the state. So idempotence at layer ll says nothing about how the clamps at layers ll and l+1l+1 interact. For induction with b≠0b \neq 0, every layer still forces its own target code, and these effects can still compound downstream. The claim that dense deployment “avoids repeated over-correction” is therefore an intuition, not a guarantee.

The natural test is the “Straightened Additive” variant from Appendix D, run across 4 to 24 layers. That would directly show whether the projection prevents degradation as density grows. Also, in Figure 2(c) SV-1’s ASR drops with depth almost as fast as SAC’s. The over-refusal gap is only a few points. This is weaker evidence than the text suggests.

Questions for authors

For induction on AxBench, is the clamp applied only to prompt tokens or also during decoding? Table 4 lists zero decode overhead. If decoding is not intervened, how does the concept persist over 512 generated tokens?

Can you report the “Straightened Additive” variant in the layer-density sweep? This is the cleanest way to show that idempotence, rather than straightening, is what prevents degradation under dense deployment.

Reviewer_fgvFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Straighten-and-Clamp (SAC) for concept control in LLMs. SAC learns an invertible map to transform hidden states into a straightened space. It then clamps the concept coordinates to a target value and maps the result back. The authors claim this allows stable intervention across many layers. Training has two stages: a shared straightening map is learned first, then a concept-specific clamp. Experiments cover concept induction and harmful concept suppression on two models.

Several method and data details are not clearly described. The generalization of the method is uncertain. The results also do not support some of the paper's claims.

Critical strengths
  1. The paper studies concept control through the lens of concept manifolds. This is an interesting direction.
  2. The experimental design is comprehensive.
Critical weaknesses
  1. The suppression results are overclaimed. Section 5.2 says SAC "comprehensively outperforms training-based defense methods," but the numbers do not support this. The average OOD ASR of SAC is 14.5, only 2 points lower than LoFiT at 16.5. LoFiT also beats SAC on Code Attack at 45.0 vs. 45.5. The training-free method ACE achieves a lower average OOD ASR of 10.7. The paper reports no variance. Some test sets are also small, for example, PAIR appears to have about 64 prompts. So a 2-point gap may not be significant. The authors should report the mean and standard deviation over several seeds, and the size of each test set.

  2. The over-refusal claim has no support. The paper says SAC has lower over-refusal than ACE, but reports no over-refusal rate for ACE. Table 1 should include the over-refusal rate for every method and for the base model.

  3. The induction gain depends on a per-concept strength sweep. Without Code Sweep, SAC scores 1.034 on Gemma-2-9B, which is lower than HyperSteer at 1.091. The clamp is meant to set a learned target value, so needing to tune its strength per concept weakens the claimed advantage. It also undermines the argument that SAC is more stable than additive methods under dense steering. The paper also does not say whether the baselines in Table 2 use the same tuning protocol. The authors should describe each baseline's tuning protocol and compare all methods under the same conditions.

  4. The probe experiment does not show that gg straightens the concept geometry. The map gg is non-linear, so a linear probe on g(h)g(h) is actually a non-linear probe on hh. A higher AUC is therefore expected regardless of whether the geometry is straightened.

  5. The paper claims the straightening map generalizes across concepts, but this is never tested. The test concepts in the induction task appear to be the same ones used to train gg. The suppression task involves only one harmful concept, so the shared map is not tested there either. The suppression results also cover only Qwen-2.5-7B. It is unclear how many AxBench concepts are used in total and how many are in the training set. The authors should report the concept train/test split, evaluate gg on held-out concepts, and report suppression results on Gemma-2-9B.

Minor Issues

  1. Figure 1 labels the Induction Clamp with b=0b = 0 and the Suppression Clamp with b≠0b \neq 0. Section 4 states the opposite. Please fix the labels.
Questions for authors
  1. Each clamp sets the concept coordinates to a fixed target, so a later clamp overwrites what an earlier one set. Where does the benefit of multi-layer clamping come from? Can the authors add an ablation that removes clamps at selected layers?

  2. How are the suppression training quadruples built, and which LLM generates the synthetic responses? What is DsftD_{sft}, how large is it, and is it shared between the two tasks? Section 4 points to Appendix G.2 for data examples, but G.2 has only two prompt templates. Please add one concrete example each for DindD_{ind}, DsuppD_{supp}, and DsftD_{sft}. Also, in Stage 1, xiCx_i^C is longer than xˉi\bar{x}_i because of the template prefix. How are the two prompts matched across token positions? How do the authors ensure the learned subspace captures the concept rather than the template structure?

  3. How is L\mathcal{L}_{reg} computed in practice? At which token positions and layers is hh replaced, and are concepts sampled in each batch? What is λ\lambda, and what happens when λ=0\lambda = 0? Since L\mathcal{L}_{reg} is a direct SFT loss on DsftD_{sft} rather than a distance to the original model, it may give SAC extra fine-tuning that the baselines do not receive. It also only tests the ablation case where b=0b = 0. What preserves utility when b≠0b \neq 0?

  4. The paper motivates the two-stage design as a way to decouple geometry from concept control. How does SAC perform when gg, UU, and bb are trained jointly?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.