Ranking Is Not Control: Reference-Grounded Steering for Distinct Behaviors

Representation steering provides lightweight control over frozen language models, and many optimization-based methods learn steering vectors from pairwise preferences. We argue that this formulation confuses ranking with control: objectives on ranking determine relative preference, but not the direction or selectivity of behavioral change. Unconstrained neutral or irrelevant outcomes mean that pairwise advancement does not guarantee intended uplift: the same positive margin can reflect intended uplift, joint suppression, or broad co-amplification. Thus, we formulate representation steering as reference-grounded constrained behavioral control and instantiate it as Reference-Grounded Steering (RGS), a soft-penalty objective that promotes intended uplift, penalizes grounding and separation violations, limits competing-behavior leakage, and regularizes intervention energy. Together, these terms constrain the behavioral movements that pairwise preferences leave unspecified. Controlled experiments validate the predicted failure mode: pairwise-only objectives attain large positive relative margins while decreasing support for the intended behavior, and additional endpoint flexibility amplifies this failure when the corresponding behavioral constraints are absent. Once reference-grounded constraints are introduced, the same flexibility becomes beneficial. Across nine models from three families, RGS achieves the strongest bidirectional control, ordered bracketing of the unsteered policy, and control against the prompt-designated orientation across held-out behavior-pair combinations that were never jointly trained or calibrated. Along interpolated endpoint chords, RGS also provides a wider control range with fewer and smaller local reversals than the baselines. These results suggest that effective representation steering requires specifying not only which behavior should rank higher, but the structured pattern of behavioral change relative to the unsteered model.