Harmfulness in Large Language Models Lies on Curved Manifolds

Existing studies on how LLMs recognise harmfulness in textual inputs often rely on the linear representation hypothesis, which assumes that concepts are encoded as global linear directions in activation space. We show that this assumption does not fully capture the geometry underlying harmfulness judgements and can be unreliable for steering model behaviour. Using principal component analysis (PCA), we find that activations associated with harmfulness judgements lie along a curved, low-dimensional manifold rather than a fixed linear direction. This geometry resembles the horseshoe effect, where a single latent variable induces sinusoidal structure across principal components. Building on this connection, we characterise such manifolds in LLM activation spaces using sinusoidal parametric functions. Across diverse datasets, the latent variable recovered from the fitted manifold predicts model harmfulness decisions more accurately than projections onto linear directions. Steering along the manifold reliably shifts model judgements and outperforms linear steering, including beyond the region covered by the observed data. These effects generalise to arbitrary, unstructured instructions. Our findings highlight the value of geometry-aware modelling for understanding and evaluating how instruction-tuned models internally represent harmfulness.