ChemSteer: Controllable Molecular Generation via Incoherent Subspace Steering in Chemical Language Models

The Linear Representation Hypothesis (LRH) holds that concepts are encoded along linear directions in the activation space of pretrained models. It was formulated for semantic concepts in natural language, where a concept and its negation define a natural axis. Chemical properties are more complex because they are determined by valence, ring systems, scaffold symmetries and synthesizability constraints. As such, a single global direction cannot control these properties across the entire expanse of chemical space. We argue that property control in chemical language models is better described as locally linear because the representation manifold is curved, so a linear direction is valid only within a limited neighborhood, and different structural families require different directions. We test this directly and find that the angle between local steering directions increases systematically as the chemical similarity between their molecular neighborhoods decreases, across datasets and properties. This motivates ChemSteer, which uses spectral co-clustering to partition molecules into locally linear regions and latent dimensions into incoherent subspaces simultaneously, extracts a steering direction within each region, and superposes them into a single global vector with exactly zero interference. We validate on QM9, ZINC250K, MOSES and ChEMBL. ChemSteer increases QED to 0.92–0.95, reduces synthetic accessibility to $\approx 1.5$, and steers effectively toward higher LogP, without retraining or fine-tuning.