Structure-Aligned Steering: Task-Space Operations for Activation Control

Activation steering usually linearly adds a vector to a hidden state, imposing translation that disregards the inherent structure of the task (e.g., cyclical structure of weekdays). We introduce Structure-Aligned Steering, an encoder-decoder interface that maps hidden states into task-structured coordinates, applies transformations there, and returns model-readable states. With prescribed weekday coordinates, one rotation advances the output towards the next day, composes fractional steps with smooth output transfer between neighboring days, and closes the cycle, which can not be achieved by linear steering. Without clear coordinate targets, matching geometry-derived output distributions still yields compatible geometry: sparse number mixtures unfold into a continuous circle, and color encodings preserve RGB-cube neighborhoods and approximate distances, which the decoder carries into hidden states. Perturbing these coordinates or supervising with task-inconsistent geometry reduces accuracy, linking this organization to steering. To show practical value beyond mechanistic analysis, we build a steerer–aligner system for modular arithmetic that outperforms parameter-matched PEFT baselines across eight models. Task-space alignment thus makes steering structured and compositional rather than merely directional.