acceptodds
Under review as a conference paper at ICLR 2027

Beyond Linearity: The Contextual Geometry of Concepts

Abstract

Representations of concepts are commonly read as directions within neural activations, assuming that a concept is mediated by a single, context-free vector. We prove that this assumption fails in general because modern architectures inherently encode contextual interactions. Across fifteen models and three modalities, the directions corresponding to the same concept exhibit high variability across contexts. Furthermore, measuring their effective dimension reveals that concepts correspond to multidimensional structures in the activation space. This motivates our novel contextual representations of concepts, which are able to capture these structures across different contexts. Under causal intervention on LLMs, or steering, they allow consistent reductions in perplexity over single directions, highlighting more faithful representation of concepts, and improved control over the behavior of models.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Explicability, Steering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eaWUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper challenges a common assumption: that concepts can be represented as global linear directions in neural network activations. The paper theoretically argues that if a concept is actively involved in computations by the attention mechanism, it typically cannot remain linear at all times within the Transformer module. The empirical section quantifies the degree of alignment and effective dimension of concept directions, measured by the mean difference, under various contextual conditions across 15 models, 9 datasets, and three modalities: images, text, and audio. Based on the observed directional changes, the paper proposes CACTUS: a method that models concepts by considering the union of all contextual concepts representing the target attribute, or using it as context. On language and sentiment concepts across three large language models, CACTUS achieved lower confusion scores compared to global linear steering and several baselines.

Critical strengths

1.This work tests the hypothesis from mechanistic interpretability and activation steering that concepts can be represented by a single global linear direction, and suggests that concept representations may exhibit significant context dependence.

2.Within the framework of the global linear representation hypothesis, this paper derives the constraints that must be satisfied for a concept vector to maintain a globally fixed linear output after passing through a Transformer block. This work establishes a mechanistic link between the empirical question of whether a concept possesses a linear representation and the attention-driven mechanisms of information routing and representation transformation. Although the scope of applicability of these theoretical conclusions remains to be further clarified, the formalization of the problem and the analytical perspective presented in this paper are highly insightful.

3.The empirical section spans 15 models, nine datasets, and three modalities (image, text, audio), encompassing both Transformer and non-Transformer architectures. Its conclusions therefore enjoy stronger evidence for generalizability compared with observations obtained solely from a single model or task.

Critical weaknesses

1.The theoretical section discusses the conceptual directions in which attention mechanisms actually participate in computations; however, the empirical section analyzes the conditional mean difference directions constructed based on externally supervised labels. The paper does not verify whether the directions derived from these labels indeed correspond to the concepts actually used by the attention mechanisms within the model. Therefore, the observed cross-context changes in direction may also stem in part from label definitions, conditional data distributions, covariance structures, or the direction estimation process, and do not necessarily represent the nonlinearity of the model’s intrinsic conceptual representations.

2.The effective dimension metric d_eff suffers from interpretive ambiguity. Computed over the Gram matrix of context wise concept discriminant vectors, it measures dispersion among context specific directions rather than the intrinsic dimension of the concept manifold in raw activation space. The main text does not draw a clear distinction, risking misinterpretation by readers.

3.The ablation and practical usability analysis of CACTUS is insufficient. Although the paper compares the Single-Context baseline and discusses the performance of certain directions when used individually, it lacks a systematic ablation analysis of each component, making it difficult to distinguish whether performance improvements stem from the two types of context directions, the max aggregation mechanism, or the larger set of candidate directions. Furthermore, the context-oracle relies on true context labels; the paper does not explain how this information is obtained in unlabeled real-world scenarios or how errors in this information affect performance.

4.The formalized proposition in the paper is derived exclusively for Transformer architectures. Nevertheless, empirical results show that context-dependent concept directions also exist in convolutional models. The paper does not discuss the potential mechanisms within convolutional layers that give rise to this phenomenon. In the absence of such discussion, readers cannot determine whether this contextual variability is an artefact specific to Transformers or a general property of discriminative representation learning.

Questions for authors

1.All concept directions are constructed from external supervision labels. How do the authors validate or support that these label-derived directions genuinely correspond to intrinsic concept representations that are actually utilized by the model’s attention mechanism, rather than primarily reflecting statistical correlations induced by dataset labels, conditional data distributions, or the probe estimation procedure?

2.Experiments have observed that CNN models also exhibit context-dependent conceptual directions, but your theory applies only to Transformers. Please supplement your manuscript with a discussion of which candidate mechanisms within convolutional networks could cause conceptual directions to vary with context; even if you do not provide a new formal proof, an analysis should be included.

3.The component ablation of CACTUS remains incomplete. Could the authors separately quantify the performance contributions of the two types of context directions, the max aggregation mechanism, and the size of the candidate direction set? In addition, please clarify whether CACTUS’s performance gains persist under baselines matched in parameter count or the number of candidate directions.

4.In the CACTUS implementation, gradient directions are computed only once from the original activations and are not recalculated during steering. Please discuss the potential biases introduced by this approximation and identify the conditions under which this approximation degrades steering performance.

Reviewer_JiqkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper challenge the assumption that concepts in neural network are linear and independent, and attempt to propose new geometry structures for concepts in LLM representations. There are three main contributions. First, this paper argues that the concepts cannot remain linearly represented through a transformer block if attention compute them. Second, this paper examines contextual concept directons across 15 models, 9 datasets, and 3 modalities. It finds the directions associated with the same concept can vary a lot across contexts. Third, this paper proposes context-aware concept union steering, a method selects an input-dependent steering direction from multiple context-conditioned concept directions. The steering experiment uses Gemma4-E4B, Llama-3.1-8B, and Qwen3.5-9B. This paper reports lower perplexity compared with linear steering on average.

Critical strengths
  1. Broad empirical study. This paper shows experiments on different foundation models and modalities. It quantifies the variability of concept directions using pairwise cosine similarity and estimate the dimensionality of contextual representations. I
  2. A practical steering method with meaningful baseline comparison. The proposed CACTUS method is relatively simple and training-free. The authors compare it with linear steering, quadratic steering, and a single-context steering, and show lower perplexity metric from CACTUS on multiple LLMs and modalities.
Critical weaknesses
  1. Insufficient comparison with prior steering methods. Context-dependent steering has been explored in previous work, such as HyperSteer and Learning to Steer. The paper mainly compares CACTUS against linear, quadratic, and single-context baselines, without demonstrating its advantages over existing input-dependent steering approaches. [1] Sun, J., Baskaran, S., Wu, Z., Sklar, M., Potts, C., & Geiger, A. (2025). HyperSteer: Activation Steering at Scale with Hypernetworks. [2] Parekh, J., Khayatan, P., Shukor, M., Dapogny, A., Newson, A., & Cord, M. (2025). Learning to Steer: Input-dependent Steering for Multimodal LLMs.
  2. Limited evaluation of steering effectiveness. The experiments primarily measure perplexity reduction on sequences already showing the target concept. However, lower perplexity does not necessarily imply successful semantic steering. The paper lacks systematic evaluation of target-concept generation, fluency, and preservation of unrelated semantic attributes.
Questions for authors
  1. Comparison with existing context-dependent steering methods: Context-dependent steering has been explored in prior work. Could the authors provide a comparison with these approaches or other representative input-dependent steering methods? This would help clarify the novelty and practical advantages of CACTUS.
  2. Evaluation of semantic steering effectiveness: The current experiments primarily evaluate steering through perplexity reduction. However, lower perplexity does not necessarily indicate successful control of the target concept. are there more evaluations of semantic steering the authors can provide?
Reviewer_FAw5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether semantic concepts in neural network activations can be adequately represented by a single, context-independent direction. It argues that this assumption is generally insufficient because neural architectures, particularly Transformers, use contextual interactions that cause concept representations to vary across contexts.

Critical strengths
  1. The paper addresses an important assumption in mechanistic interpretability. Representing a concept as a single direction is convenient for both interpretability and activation steering, but this representation may fail when the same semantic attribute appears in different contexts. The paper directly investigates this limitation rather than treating linear representations as universally adequate.

  2. The empirical characterization spans multiple modalities and architectures. The evaluation covers 15 models, nine datasets, and three modalities, including both Transformer and non-Transformer architectures. This breadth is valuable because it allows the authors to investigate whether contextual variability is confined to a particular architecture or data type.

  3. The proposed method CACTUS uses contextual information to construct a union of directions rather than introducing a separate, opaque steering network. Its maximum-over-directions formulation offers a relatively straightforward way to incorporate context-dependent representations into activation interventions.

  4. The authors evaluate steering at both intermediate and final layers, investigate multiple steering magnitudes, and compare against a context-oracle variant. The appendix reports that intermediate-layer steering generally performs better than final-layer steering for the evaluated methods.

Critical weaknesses
  1. Theoretical claims need more clarity: The paper makes strong claims about non-linearity in Transformers, but the assumptions behind the proof and the d/Hd/H bound need better explanation.

  2. Limited statistical validation: The contextual directions may be affected by sample size, class imbalance, and noisy labels. More analysis is needed to show that the findings are stable.

  3. Perplexity is not enough: Lower perplexity does not always mean better concept steering. The authors should also evaluate generated text using classifiers or human evaluation.

  4. Limited comparison with existing methods: CACTUS is compared with few baselines. Comparison with stronger context-aware steering methods would make the results more convincing.

  5. Limited scope of steering experiments: Although the paper studies 15 models across three modalities, CACTUS is tested only on three language models and three datasets. Its generalization to other settings remains unclear.

  6. Context selection is not clear: The paper does not fully explain how the relevant contextual direction will be selected during inference when the actual context labels are not available.

Questions for authors
  1. Can the theoretical proposition be strengthened or more precisely scoped?

  2. How robust are contextual directions to limited samples and changes in context distribution?

  3. Does CACTUS improve actual concept expression rather than only perplexity?

  4. How does CACTUS compare with existing context-dependent steering methods?

  5. How much inference-time context information does CACTUS require?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Paper Revisions & New Experiments

We thank the reviewers for their detailed feedback and challenging questions. They allowed us to clarify our arguments, and widen our analysis to more baselines, models, tasks, datasets and metrics. From those, our article is now more robust empirically and theoretically. To summarize:

  • We clarified our theoretical claims, and revised Sec.3.2, and discussed their applicability beyond transformers
  • Included an analysis of statistical robustness of steering on 372 additional datasets, making our experimental setting larger than any article presented at ICLR2026 (App.H)
  • Evaluated CACTUS steering on multimodal LLMs, on the setting defined in [1], outperforming the Learning-to-Steer method on hallucination and safety-related tasks (POPE, MMSafetyBench). Details in Sec.6
  • Evaluated CACTUS steering against HyperSteer [2] on their instruction-following setting (AxBench). Showing that CACTUS is able to insert concepts while following instructions and preserving fluency. Details in App.G.
  • Introduced CACTUS-U, a variation of CACTUS that does not require explicit label information, by using unsupervised activation clustering (Sec.6, App.F)
  • Added precisions on the inference of CACTUS (that does not require inference-time oracle information, nor iterative gradient recomputation).

After revisions, our analysis is now substantially larger-scale, more varied, and theoretically stronger. Therefore, our claims are now more robust, and our analysis is richer. We remain open to further discussion if any concerns remain.

[1] Parekh, Jayneel, et al. "Learning to steer: Input-dependent steering for multimodal llms." Advances in Neural Information Processing Systems 38 (2026): 159799-159834.

[2] Sun, Jiuding, et al. "Hypersteer: Activation steering at scale with hypernetworks." arXiv preprint arXiv:2506.03292 (2025).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.