acceptodds
Under review as a conference paper at ICLR 2027

ChemSteer: Controllable Molecular Generation via Incoherent Subspace Steering in Chemical Language Models

Abstract

The Linear Representation Hypothesis (LRH) holds that concepts are encoded along linear directions in the activation space of pretrained models. It was formulated for semantic concepts in natural language, where a concept and its negation define a natural axis. Chemical properties are more complex because they are determined by valence, ring systems, scaffold symmetries and synthesizability constraints. As such, a single global direction cannot control these properties across the entire expanse of chemical space. We argue that property control in chemical language models is better described as locally linear because the representation manifold is curved, so a linear direction is valid only within a limited neighborhood, and different structural families require different directions. We test this directly and find that the angle between local steering directions increases systematically as the chemical similarity between their molecular neighborhoods decreases, across datasets and properties. This motivates ChemSteer, which uses spectral co-clustering to partition molecules into locally linear regions and latent dimensions into incoherent subspaces simultaneously, extracts a steering direction within each region, and superposes them into a single global vector with exactly zero interference. We validate on QM9, ZINC250K, MOSES and ChEMBL. ChemSteer increases QED to 0.92–0.95, reduces synthetic accessibility to , and steers effectively toward higher LogP, without retraining or fine-tuning.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Incoherent Subspaces, Steering Vectors, Chemical Language Models, Representation Learning, Drug Discovery, Molecular Design

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vqZHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present analysis justifying a geometric framing of chemical property space as inherently curved, and stating that the linear representation hypothesis is only valid within a small step relative to the radius of curvature. Using this framing, the authors then go on to present a post-hoc intervention for steering chemical language models based on spectral co-clustering to determine local steering vectors within clusters that are then aggregated into a global steering vector used for steering hidden layer activations.

Critical strengths
  • The presentation and framing of the method's geometric motivation is very strong, and clearly delineates the problem for the remainder of the paper. This helps contextualize the later steering methodology and results
  • The method demonstrates good performance across a variety of models, making a compelling development for tool for steering chemical language models
  • The writing is easy to follow and not too equation or text-dense, so the authors' contributions come across clearly
Critical weaknesses
  • More characterization on the generated molecules would be useful for understanding the feasibility of the designs generated by the model under steered and non-steered conditions, using a metric such as Tanimoto to source would be helpful for understanding the generative diversity
  • While there are many baseline methods for comparison, the authors should consider using fairer comparisons in some cases since some of the gaps between models themselves are larger than any of the steering benefits observed (e.g. ChemSpace versus SmileyLLama on QM9 QED)
  • The authors should try additional datasets such as PubChem and Enamine
Questions for authors
  • There are three steps to the process, where are ablations testing the individual contributions of each part of the three-stage approach (e.g., what happens if you ablate clustering with a simpler method like random partitions?)
  • How does performance vary with k and what is the distribution of cluster sizes?
  • Some better reporting on some more descriptive statistics of the generated molecules would help characterize the things the model generates (e.g. how big are the molecules, what are their similarity to source between steered output and input?)
  • What happens if you look at less restrictive statistics such as the full average or basically things beyond the Top-100 mean?
7 comments
AuthorsOpenReview ↗

Response to Reviewer vqZH part 1 of 3

(Response to Reviewer vqZH part 1 of 3)

We thank the reviewer for the positive assessment, and especially for the comment that the geometric motivation delineates the problem and motivates the methodology. We are glad the authors found our writing quality and presentation to be strong, and for the acknowledgement of the breadth across backbones. We address each weakness and question below. We have marked all updates in blue text within the manuscript.

W1/ Q3

We agree that more detailed molecular characterization would strengthen the evaluation of the feasibility and structural properties of generated molecules. Table 5 reports validity, uniqueness, and novelty, while Table 7 examines chemical coherence within the spectral co-clustering by comparing within-cluster Tanimoto similarity against random assignments. However, these analyses do not directly characterize the structural similarity between generated molecules and molecules in their respective source datasets.

To address this point, we conducted an additional analysis computing Tanimoto similarity between source and generated molecules under steered, non-steered, and global-steering conditions, for each of the three optimization objectives (LogP, QED, and SA). The results show that ChemSteer consistently achieves higher source-to-generated Tanimoto similarity across all four datasets and all three objectives. The table below provides the tanimoto similarity within each source dataset to give context for the results table which follows it.

DatasetTanimoto Similarity
ChEMBL0.1114 ± 0.0389
MOSES0.1363 ± 0.0544
QM90.0746 ± 0.0577
ZINC0.1200 ± 0.0432
DatasetConditionMean Tanimoto (LogP)Mean Tanimoto (QED)Mean Tanimoto (SA)
ChEMBLSteered0.84230.84640.8258
ChEMBLNon-steered0.70950.60650.7466
ChEMBLGlobal steering0.71460.72420.6993
MOSESSteered0.13410.13730.1250
MOSESNon-steered0.11390.11110.1067
MOSESGlobal steering0.11710.10980.1081
QM9Steered0.04810.04640.0469
QM9Non-steered0.03700.03670.0417
QM9Global steering0.04440.03330.0357
ZINCSteered0.14390.14820.1444
ZINCNon-steered0.12340.11690.1177
ZINCGlobal steering0.12290.11900.1160

These results indicate that ChemSteer generates molecules that retain greater structural similarity to their source datasets than the two comparison conditions. This suggests that the proposed local steering approach better preserves source-associated chemical structure while enabling property-directed optimization, in contrast to non-steered generation and global steering. Combined with the validity, uniqueness, novelty, and cluster-coherence analyses, this provides a more comprehensive characterization of the generated molecules. We have added these results and discussion to Appendix U and reference it at the end of Section 5.

W2

We agree that differences in model architectures can confound comparisons between methods evaluated using different pretrained backbones. We have used the baselines within the paper because they are the current best methods we found in literature for molecular property optimization. None of them, or to the best of our knowledge, any other published work, uses chemical language model steering methods for optimization. We include controlled comparisons that directly address the reviewer's concern by holding the underlying generation backbone fixed. In Table 8, we compare ChemSteer against both unsteered generation and naive global steering using the same SmileyLlama backbone, thereby isolating the contribution of the steering mechanism from differences in model architecture. ChemSteer consistently improves molecular property optimization over both controls. For example, on ZINC250K, the top-100 mean QED increases from 0.88 with unsteered generation and 0.89 with naive global steering to 0.94 with ChemSteer. Similarly, the mean SA score decreases from 2.05 and 1.94, respectively, to 1.54 with ChemSteer. These improvements cannot be attributed to the choice of backbone, since all three conditions use the same pretrained model and generation framework. Rather, they demonstrate the additional benefit of ChemSteer's cluster-local steering over both the underlying model and a simpler global steering alternative. We also provide additional GPT-2 experiments in Table 9 to provide additional evidence that the steering approach is not intrinsically tied to a single architecture. We further examine structural preservation through row-wise Tanimoto similarity (W1/Q3) between generated molecules and their corresponding reference molecules. Across all four datasets and all three optimization objectives, ChemSteer achieves higher mean Tanimoto similarity than both unsteered generation and global steering. This provides complementary evidence that the improvements arise from the steering strategy and are not simply differences in the underlying generation backbone.

AuthorsOpenReview ↗

Response to Reviewer vqZH part 2 of 3

Q1

We do have extensive ablations of the methods components in the Appendix. Table 8 compares no steering, naive-global steering, and ChemSteer, all on the SmileyLlama backbone. ChemSteer has the best performance over these ablations. Table 6 compares held-out separation under global, row-wise, coordinate-restricted, and random partitions. We compare the following settings: (1) Sep*(1), a single global CAA direction (no partitioning); (2) Sep*~\tilde{\text{Sep*}}(k), row-piecewise CAA, full dimensionality per cluster (isolates the value of row-wise local specialization alone); (3) Sep*(k), the coordinate-restricted optimum of equation 4, approximated via alternating optimization initialized from the deployed SCC partition; (4) Sep(πChemSteer\pi^{\text{ChemSteer}}), the actual CAA vectors, evaluated for held-out generalization; and (5) Sep(πrand\pi^\text{rand}), a random-partition null. We show that amongst these settings, ChemSteer partitioning performs the best. Appendix H.3 compares sparse–dense alignment under random clustering. Appendix K ablates the steering magnitude, α\alpha, and the steering layer. We believe that these cover the components of ChemSteer. If there are other ablations the reviewer would like us to perform concerning any aspects of ChemSteer, we would be happy to conduct these experiments.

Q2

The main experiments use (k=300). Appendix E reports the cluster-size distributions and their averages: ChEMBL 57 molecules, MOSES 71, QM9 52, and ZINC 56 per cluster. The corresponding average subspace ranks are 10, 7, 10, and 10, respectively. The average numbers of unique scaffolds per cluster are 53, 65, 32, and 55, respectively.

Our theory does not derive an optimal number of clusters. kk specifies the granularity of the paired molecule-dimension partition. For any feasible kk, disjoint coordinate supports guarantee exact orthogonality of the corresponding steering subspaces, while the quality of the resulting local approximations depends on how well the partition captures representation and property heterogeneity.

To assess sensitivity to this choice, we conducted a kk-sweep over k∈{5,10,50,100,200,300,400,500}k\in\{5,10,50,100,200,300,400,500\}. The results are visualized in Appendix W, Figure 9, and we report the tables here.

k Ablation on LogP (Top-100 Average)

kChEMBLMOSESQM9ZINC
54.14±0.773.12±0.361.40±0.613.40±0.39
104.20±0.803.41±1.071.23±0.283.83±1.50
504.45±2.314.27±0.571.52±0.523.90±1.68
1005.04±0.774.25±0.511.31±0.214.37±0.42
2005.27±1.104.68±1.521.43±0.334.49±0.55
3005.24±1.196.68±0.642.70±0.166.82±0.71
4005.25±1.384.30±0.661.39±0.165.52±4.98
5005.53±2.234.51±1.401.30±0.215.14±1.83

k Ablation on LogP (Overall Average)

kChEMBLMOSESQM9ZINC
52.18±1.611.52±1.050.12±1.181.77±1.25
102.14±1.551.52±1.180.16±0.761.87±1.47
502.14±2.002.24±1.050.16±1.041.87±1.55
1003.03±1.612.25±1.070.20±0.732.36±1.23
2003.22±1.652.27±1.270.24±0.972.37±1.30
3002.46±2.632.30±0.740.24±0.932.45±1.59
4003.15±1.712.51±1.120.27±0.893.06±2.55
5003.30±1.952.57±1.210.25±0.952.99±1.62

The results demonstrate that the effect of kk is dataset- and evaluation-dependent. This behavior is consistent with the geometric motivation for local steering. When kk is small, clusters contain more molecules and cover broader regions of the representation manifold. If the manifold is curved, the local tangent spaces and projected property gradients can vary across these regions, such that a single cluster-level steering direction becomes a less accurate approximation to the molecule-specific property direction. Increasing kk can improve this local approximation by producing finer partitions with less within-cluster geometric variation.

However, increasing kk also introduces a competing constraint. The corresponding steering subspaces become more localized in the molecule-coordinate partition, potentially restricting the intervention to fewer representation dimensions. While this can reduce irrelevant perturbations, an overly restrictive subspace may omit dimensions needed to express the desired property direction. Furthermore, finer partitions reduce the number of molecules available for estimating each local steering vector, potentially increasing estimation variance. The relationship between kk and steering performance thus reflects a trade-off between local geometric fidelity and the expressive capacity of the steering subspace.

k=300k=300 achieves the highest top-100 average LogP on MOSES (6.68), QM9 (2.70), and ZINC (6.82), while ChEMBL achieves a comparable value at k=300k=300 (5.24) to its highest value at k=500k=500 (5.53). For the overall generated distributions, the best-performing kk varies across datasets.

AuthorsOpenReview ↗

Response to Reviewer vqZH part 3 of 3

(Response to Reviewer vqZH part 3 of 3)

Q4

We thank the reviewer for raising this important distinction. We agree that evaluating only the top-100 generated molecules measures best-candidate optimization rather than the typical quality of the entire generated distribution. These are complementary evaluation objectives, and we intended the top-100 results to measure the former, as is standard among our baselines, not to serve as an unbiased estimate of overall generation quality.

For each dataset, method, and target property, we rank the generated candidate pool according to the requested optimization objective, that is, descending LogP, descending QED, or ascending synthetic accessibility (SA). We then report the mean and standard deviation of the 100 highest-ranked candidates. Thus, the selected molecules are property-specific, making it such that the top-100 for LogP need not be the same molecules as the top-100 for QED or SA. We adopted this presentation because the molecular optimization baselines we compare against emphasize their best optimized candidates, often reporting only a small number (top 1–3) of top results. Reporting the top-100 mean and standard deviation provides a more statistically informative comparison than reporting only one to three individual best molecules.

We do not intend the top-100 results to imply that all generated molecules achieve these property values, nor do we use them as a substitute for evaluating generation quality. We therefore also report the averages and standard deviations across the full 15,000-molecule generation sets in Appendix J (Tables 15–16), alongside validity, uniqueness, and novelty in Table 5.

The full-set results provide a more representative view of the generated distributions and broadly support the effectiveness of ChemSteer, while also revealing where its advantage is less pronounced. For example:

  • ChEMBL: Across all 15,000 generated molecules, ChemSteer achieves LogP (2.46 ±\pm 2.63), QED (0.73 ±\pm 0.15), and SA (3.01 ±\pm 0.82). The corresponding Gradient Flow results are (-10.17 ±\pm 13.18), (0.41 ±\pm 0.19), and (4.82 ±\pm 1.04), respectively. This shows that the improvement is not restricted to the best 100 candidates.

  • MOSES: ChemSteer achieves LogP (2.30 ±\pm 0.74), QED (0.47 ±\pm 0.13), and SA (4.26 ±\pm 0.40). The full-set QED result is comparable to, rather than better than, Gradient Flow and Wave, which both report (0.54 ±\pm 0.19). We agree that it is an important qualification that ChemSteer's top-100 QED advantage does not translate into the same advantage across the entire MOSES distribution.

  • QM9: ChemSteer achieves LogP (0.24 ±\pm 0.93), QED (0.53 ±\pm 0.15), and SA (3.88 ±\pm 0.95), compared with Gradient Flow's (-2.52 ±\pm 2.06), (0.46 ±\pm 0.08), and (4.00 ±\pm 1.02). The full-set results therefore show improvements in LogP and QED and a more modest difference in SA.

  • ZINC: ChemSteer achieves LogP (2.45 ±\pm 1.59), QED (0.81 ±\pm 0.16), and SA (2.45 ±\pm 0.69). In comparison, Gradient Flow achieves (-9.27 ±\pm 11.44), (0.43 ±\pm 0.19), and (5.06 ±\pm 0.99). These results indicate substantial full-distribution improvements in all three reported property metrics.

These full-set results are important because they show that ChemSteer's gains are not solely a consequence of selecting a small number of high-performing candidates.

Reviewer_saQHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ChemSteer, a post-hoc method for controlling target chemical properties for pretrained chemical language models. Concretely, it jointly partitions molecules and latent representation dimensions using spectral co-clustering, yielding locally linear molecule clusters and coordinate-restricted subspaces, computes a contrastive steering direction within each coordinate-restricted cluster, and combines these directions into a single inference-time intervention. The paper motivates this design through locally linear property representations and proves exact recovery of an idealized block-structured affinity, with conditional optimality for a class-separation objective. Experiments on four molecular datasets report improvements in LogP, QED, and synthetic accessibility, primarily using SmileyLlama without model adaptations.

Critical strengths
  • The work provides a lightweight (e.g., training- and finetuning-free) approach for controllable molecule generation. It converts complex chemical properties into locally separable directions and combines them into a fixed vector in a simple way, and is reported to achieve good performance across several datasets. The reported improvements over unsteered and globally steered versions of the same backbone are the most relevant evidence for its potential utility.

  • The linear representation hypothesis in molecular spaces is an interesting view to investigate. The dense local-direction analysis in Section 3.3 avoids trivially obtaining orthogonal directions from disjoint coordinate supports. These explorations provide evidence of the link between chemical representations and steerable directions, even if the stronger geometric interpretation remains unproven.

Critical weaknesses
  • The theoretical demonstration does not convincingly establish the paper's core claim. Section 3.2 claims PkG=vkP_kG=v_k by assigning disjoint coordinates to different vectors. This identity is correct, but Step 3 injects the entire normalized vector into every generation state without applying PkP_k or routing the intervention according to the current molecule. Thus, preserving a vector under a hypothetical projection does not show that other blocks have no effect on generation in region kk. The abstract's “exactly zero interference” is therefore justified only as a coordinate-projection statement, not as a claim about molecular property control.

  • The main contribution is limited, as this essentially applies existing LRH/local-linearity and activation steering to molecules. CAA, local linear steering, spectral co-clustering, and latent molecular property control are already established. The work mainly combines them on chemical language models. It is no more than an engineering transfer rather than a new theory or control mechanism.

  • The curvature explanation is stronger than the evidence supports. Section 3.1 treats changing local directions as evidence of manifold curvature. However, gradients can change direction on a flat manifold when the property is nonlinear. For instance, y(x1,x2)=x1x2y(x_1,x_2)=x_1x_2. Thus, the correlations in table 2 do not identify curvature as the cause, nor establish that dataset-specific optimal steering coefficients measure a local radius of curvature. As a result, I am suspicious about the mathematical soundness of the "molecular representations and controllability" claim.

  • The paper controls only three simple RDKit-computable 2D metrics, which are not representative of real drug-design objectives such as docking, toxicity, selectivity, or ADMET. These metrics are easy to compute and largely independent scalar targets, so the results at best demonstrate a proof of concept for a few simple properties. This is insufficient to support broad claims about controllable molecule generation or complex property control, which would require more complex and potentially conflicting objectives.

  • The top-100 evaluation is problematic because it introduces selection bias and is disconnected from overall results. Reporting only the 100 best molecules per method measures best-case performance rather than overall generation quality, favoring methods that produce a few excellent samples amid many poor ones. This is inconsistent with Appendix J's full-set averages and hides ChemSteer's lower validity on ZINC250K. As a result, the protocol might overclaim ChemSteer's advantage and cannot support a general claim of superiority.

  • Baselines are not compared under the same backbone or generation framework, making the comparison unfair. ChemSteer uses SmileyLlama, while baselines like ChemSpacE and ChemFlow use entirely different architectures. Since the paper’s own table 9 shows huge performance differences across backbones, the reported gains may come from SmileyLlama rather than ChemSteer’s steering mechanism.

Questions for authors
  • What ensures that adding vectors from other coordinate blocks does not affect generation for a molecule in cluster kk? Can you compare the composite intervention against a molecule-routed local intervention and directly measure cross-block effects?

  • How are the sparse and dense vectors actually constructed, and why is their retained-coordinate cosine below 1? What evidence connects the Sep\mathrm{Sep} objective to causal steering performance rather than classification?

  • How does ChemSteer differ fundamentally from prior combinations of CAA, spectral co-clustering, and latent molecular property control? Beyond applying these components to chemical language models, what new theoretical insight or control mechanism does it contribute that existing methods could not already provide?

  • Why is the main evaluation based only on the top-100 generated molecules rather than the full generated set? As Appendix J reports full-set averages that differ dramatically from the top-100 results, could you clarify the selection criteria for the top-100, whether invalid molecules were filtered out, and how the reported gains would change under a full-set or unbiased evaluation protocol?

  • How much of ChemSteer's improvement comes from the method itself versus the SmileyLlama backbone? Can any additional empirical studies to be offered to illustrate the point?

  • How were the steering coefficients α\alpha selected, and is there any independent validation or ablation studies to verify this critical choice?

  • What is the core theoretical claim that distinguishes ChemSteer from existing steering methods? Since PkG=vkP_kG = v_k is only a coordinate-projection identity, but Step 3 injects the full normalized vector into every generation state without applying PkP_k, what actually guarantees that interference remains zero during generation?

6 comments
AuthorsOpenReview ↗

Response to Reviewer saQH part 1 of 5

(Response to Reviewer saQH part 1 of 5)

We thank the reviewer for a careful and technically precise review. We are glad the lightweight, training-free nature of the approach is highlighted, and we particularly appreciate the observation that the dense local-direction analysis in Section 3.3 avoids trivially recovering orthogonality from disjoint coordinate supports. That was the reason for computing those directions in the full 4096-dimensional space, and it is the measurement we would most want a reader to weigh. We also agree with the framing that the same-backbone unsteered and globally steered comparisons are the most relevant evidence for utility, and we expand exactly those below. We address each weakness and question individually below. We have marked all updates in blue text within the manuscript.

W1/ Q1/ Q7

The exact zero-overlap guarantee concerns the coordinate supports of the constructed subspaces. Each local vector vkv_k belongs to a disjoint coordinate block SkS_k, so PkG=vkP_kG=v_k by construction. Thus, every local steering direction is exactly preserved and recoverable within the composite vector. Importantly, the joint molecule-coordinate partition associates each molecular cluster with its corresponding property-relevant coordinate block. This association is supported by our held-out separation experiments (Table 6, Appendix C.7), while the random-clustering ablation (Appendix H.3, Table 12) demonstrates that the alignment between dense local directions and their coordinate-restricted counterparts degrades under arbitrary partitions. Finally, Table 8 (Appendix G) demonstrates that the resulting composite intervention improves generation over both unsteered and naive globally steered controls.

To further examine the effect of this composition during generation, we added a matched generation ablation comparing two intervention strategies: (i) applying the individual cluster-local steering vector mapped to each molecule's assigned cluster, restricted to its corresponding latent-coordinate subspace, and (ii) applying the full ChemSteer composite vector, which superposes the cluster-local directions into a single global intervention. This comparison directly evaluates whether the full composite intervention incurs a substantial performance penalty relative to applying only the relevant local direction.

DatasetEvaluationSteeringMethodLogPQEDSA
ChEMBLTop-100Mapped Individual5.48±3.410.93±0.001.55±0.12
ChEMBLTop-100ChemSteer Composite5.24±1.190.94±0.101.55±0.18
ChEMBLOverallMapped Individual3.08±1.690.67±0.172.62±0.62
ChEMBLOverallChemSteer Composite2.46±2.630.73±0.153.01±0.82
MOSESTop-100Mapped Individual6.46±0.430.95±0.001.50±0.09
MOSESTop-100ChemSteer Composite6.68±0.640.95±0.001.50±0.08
MOSESOverallMapped Individual2.56±1.030.79±0.112.48±0.51
MOSESOverallChemSteer Composite2.30±0.740.47±0.134.26±0.40
QM9Top-100Mapped Individual2.38±0.490.87±0.022.34±0.32
QM9Top-100ChemSteer Composite2.70±0.160.92±0.011.61±0.17
QM9OverallMapped Individual0.49±1.090.52±0.123.86±0.89
QM9OverallChemSteer Composite0.24±0.930.53±0.153.88±0.95
ZINCTop-100Mapped Individual6.83±1.020.94±0.001.59±0.10
ZINCTop-100ChemSteer Composite6.82±0.710.94±0.001.54±0.10
ZINCOverallMapped Individual2.69±1.300.74±0.142.77±0.67
ZINCOverallChemSteer Composite2.45±1.590.81±0.162.45±0.69

The results above show broadly comparable top-100 optimization performance between the two strategies. Across the full generated distributions, neither approach uniformly dominates: the mapped individual vector achieves higher overall LogP on ChEMBL, MOSES, and ZINC, while QED and synthetic accessibility exhibit dataset-dependent trade-offs. These results suggest that the composite intervention retains competitive optimization behavior relative to the corresponding individual local interventions, although they do not establish the absence of all cross-block effects. The composite construction also has a practical methodological advantage because it produces a single global steering intervention incorporating the cluster-local directions, rather than requiring a separate vector to be selected and applied according to each molecule's cluster assignment. Thus, the ablation evaluates the empirical consequences of this global composition while distinguishing its behavior from that of isolated local steering. We have added the above table and this discussion to Appendix V.

AuthorsOpenReview ↗

Response to Reviewer saQH part 2 of 5

W2/ Q3

We agree that CAA, spectral co-clustering, and latent-space molecular optimization are established techniques. However, we respectfully disagree that ChemSteer is merely an engineering transfer of these methods. The contribution is not any individual algorithm, but a new formulation of the Linear Representation Hypothesis (LRH) for molecular property control, the piecewise-linear LRH, in which property-relevant directions are locally linear within regions of chemical representation space, but vary systematically across those regions. To our knowledge, this formulation has not previously been established for chemical language model steering, or steering in any domain.

Existing activation steering approaches typically assume that a single global direction captures a target property, while approaches using state-dependent vector fields recompute the intervention based on the current representation. ChemSteer instead identifies the regions over which linear steering is valid, jointly discovers the molecular regions and their associated latent coordinate subspaces, and composes their locally valid directions into a single static inference-time intervention.

Importantly, this is supported by a theoretical result, not solely an empirical combination of existing techniques. Theorem 1, previously presented in the appendix and now moved into the main text, establishes that spectral co-clustering exactly recovers the paired molecule–dimension block partition under three explicit structural assumptions (block structure of the nonnegative affinity matrix, connectivity and positive degree within each block, and non-degenerate block sizes). The proof uses the Perron–Frobenius theorem and the multiplicity-(k) singular structure of the normalized block-diagonal affinity matrix to show that the joint degree-scaled spectral embedding collapses each true bipartite component to a single point. Consequently, joint k-means recovers the molecule clusters and their corresponding latent-dimension clusters, up to label permutation. This provides a formal recovery guarantee for the specific joint coordinate construction on which ChemSteer depends.

Corollary 1 then connects this recovery result to molecular property control. Under the additional aligned block LRH condition (that the affinity-induced structural partition coincides with the partition maximizing the defined property-separation objective) the recovered partition is CAA-optimal among feasible partitions. Theorem 1's recovery guarantee is unconditional with respect to the target property. The additional alignment assumption is required only for the optimality result. We test this alignment empirically on our four chemical datasets, finding that local steering directions diverge systematically with decreasing molecular similarity (Section 3.3), and that the deployed partition achieves stronger held-out separation than alternative partitions (Appendix C.7).

Furthermore, the resulting subspaces have disjoint coordinate supports, yielding exactly zero operator-norm interference between local steering directions. Thus, ChemSteer does more than simply apply existing local steering independently because it provides a principled construction for recovering the local regions, associating each region with its property-relevant coordinates, and composing all local directions into one global intervention without interference. This enables piecewise-linear property control without retraining, fine-tuning, or recomputing a state-dependent steering field.

We have revised the main text to foreground Theorem 1 and clarify the distinction between the recovery guarantee, the aligned block LRH optimality condition, and the empirical evidence supporting their applicability to chemical language models.

AuthorsOpenReview ↗

Response to Reviewer saQH part 3 of 5

W3

We thank the reviewer for identifying an important distinction that was insufficiently established in the original manuscript. The counterexample the reviewer gives is correct.

We analyze the possible sources of direction variation. Let M\mathcal{M} denote the manifold of molecular representations and ThMT_h\mathcal{M} its local tangent space at representation hh. A local property-steering direction can be expressed as v(h)=PThM∇y(h),v(h)=P_{T_h\mathcal{M}}\nabla y(h), where PThMP_{T_h\mathcal{M}} projects the property gradient onto the local tangent space. Variation in v(h)v(h) can arise from either (i) changes in the tangent space itself, as the representation manifold changes orientation, or (ii) changes in the property gradient within a fixed tangent space, due to property nonlinearity. The former is a geometric property of the representations and can be investigated without reference to the optimized property. We directly measure local tangent-space variation across all four molecular datasets.

We estimate a local tangent basis for each cluster using local PCA with q=10q=10 dimensions, matching the mean within-cluster rank reported in Appendix H. We then quantify the angular separation between cluster tangent spaces using the mean principal angle. To distinguish geometric variation from finite-sample estimation error, we introduce two reference quantities.

First, we estimate a split-half noise floor by randomly dividing each cluster into two subsets and independently estimating their tangent spaces. Second, we compute the mean principal angle between random 10-dimensional subspaces in the 4096-dimensional ambient representation space. We additionally examine whether tangent-space separation increases with the distance between cluster centroids. We use the same Mantel permutation procedure as in Table 2.

Random 10-dimensional subspaces in 4096 dimensions have a mean principal angle of 87.62∘±0.18∘87.62^\circ \pm 0.18^\circ. The results across the four datasets are:

DatasetSplit-half noise floorBetween-cluster angleExcess over floorFraction of floor-to-random rangeMantel rrMantel pp
ChEMBL71.84±4.20∘71.84 \pm 4.20^\circ76.56∘76.56^\circ+4.72∘+4.72^\circ30%+0.1230.0014
MOSES64.01±5.87∘64.01 \pm 5.87^\circ73.35∘73.35^\circ+9.33∘+9.33^\circ40%+0.2190.0001
QM959.47±5.50∘59.47 \pm 5.50^\circ76.36∘76.36^\circ+16.89∘+16.89^\circ60%+0.5010.0001
ZINC68.11±4.51∘68.11 \pm 4.51^\circ76.75∘76.75^\circ+8.64∘+8.64^\circ44%+0.3770.0001

The estimated tangent spaces are substantially more aligned than random subspaces, indicating shared low-dimensional geometric structure across molecular clusters. At the same time, their between-cluster angular separation exceeds the corresponding split-half estimation floor in every dataset. Moreover, tangent-space separation increases significantly with the distance between cluster centroids across all four datasets. This distance dependence is particularly pronounced for QM9 (r=0.501r=0.501) and ZINC (r=0.377r=0.377), and remains significant for MOSES (r=0.219r=0.219) and ChEMBL (r=0.123r=0.123).

These results provide evidence that local representation geometry varies systematically across molecular clusters, independently of the optimized property. Unlike variation in property gradients alone, this tangent-space variation cannot be explained only by the nonlinearity of the property being optimized.

As an additional internal consistency check, we measure how much of a cluster's property-steering direction lies outside another cluster's estimated tangent space, calibrated against its own split-half null. The resulting excess closely tracks the independently measured tangent-space excess across datasets (r=0.9996r=0.9996 across the four dataset-level measurements). For example, the corresponding excesses are 4.7∘4.7^\circ versus 4.7∘4.7^\circ for ChEMBL, 8.8∘8.8^\circ versus 9.3∘9.3^\circ for MOSES, 16.6∘16.6^\circ versus 16.9∘16.9^\circ for QM9, and 8.3∘8.3^\circ versus 8.6∘8.6^\circ for ZINC. This agreement provides an additional consistency check connecting the observed property-direction mismatch to independently estimated local geometric variation.

Finally, the central motivation for ChemSteer does not require curvature to be the sole source of local direction variation. Both variation in representation geometry and nonlinear property landscapes can make a single global steering direction suboptimal. ChemSteer addresses this problem empirically through cluster-local steering directions. Our additional tangent-space analysis now provides independent evidence that the representation geometry itself varies across molecular clusters, strengthening the geometric motivation while keeping the claims within what the experiments establish. We have included this entire discussion and analysis in Appendix T (referenced in Section 3.1).

Reviewer_dAqHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ChemSteer proposes inference-time control of molecular generation without fine-tuning the underlying chemical language model. Spectral co-clustering jointly partitions molecules and representation coordinates; property-contrastive directions are estimated within the assigned coordinate blocks and composed into one static steering vector. The paper motivates this construction through locally varying property directions and provides conditional block-recovery and separation results. Experiments separately optimize LogP, QED, and synthetic accessibility on QM9, ZINC250K, MOSES, and ChEMBL, primarily with SmileyLlama and with secondary GPT-2 results. Evaluation includes top-100 candidate scores, whole-set averages, generation-quality metrics, and representation analyses.

Critical strengths
  1. The static intervention is simple and potentially useful: it requires neither base-model fine-tuning nor repeated inference-time direction selection. Same-backbone unsteered and global-steering controls show promising candidate-level gains across several settings. For example, ZINC250K top-100 QED increases from 0.88/0.89 to 0.94, while SA decreases from 2.05/1.94 to 1.54 (Table 4). Four molecular datasets and a secondary GPT-2 backbone provide useful breadth.

  2. The paper investigates its representation hypothesis beyond the masking construction. Dense local-direction analyses relate chemical similarity to directional similarity, using Mantel tests for dependent cluster pairs and controlling for cluster size (Table 2). Held-out separation comparisons provide additional evidence that the grouping captures useful property information (Table 6). The explicitly disclosed top-100 evaluation is complemented by whole-generation averages and steering-strength ablations.

Critical weaknesses
  1. The incremental methodological and scientific contribution is not yet sufficiently isolated. Molecular property control, contrastive activation steering, and local/state-dependent steering are established directions, so the novelty rests mainly on the joint molecule-coordinate partition and static composition. Disjoint masks guarantee zero coordinate overlap by construction, but do not establish that the original model contains naturally noninterfering chemical subsystems or that the full composite intervention avoids cross-block effects during generation. The conditional theory and representation diagnostics are useful, but a matched generation ablation is needed to distinguish the value of the proposed grouping from sparsity and local direction estimation alone.

  2. The empirical advantage is promising but not yet dependably characterized beyond the selected candidates. Appendix J reports whole-set averages, but these tables omit the same-backbone unsteered/global controls, and the reported advantage is not uniform (e.g., MOSES mean QED is 0.47 versus 0.54 for some comparators). Moreover, Tables 16 and 22 both describe whole-15,000 QM9 averages but report ChemSteer LogP/SA as 0.24/3.88 versus 3.37/3.12. The supplied generation and vector-construction scripts also contain incompatible function calls. These unresolved reporting and artifact issues limit verification of the broad superiority claims. Please clarify whether these reflect packaging errors or different experimental settings.

Questions for authors
  1. Top-100 selection protocol: Are candidates ranked solely by the requested property for each dataset-method-objective pair? Please specify the candidate pool size, whether counts are measured before or after validity filtering/deduplication, canonicalization and other filters, treatment of unchanged inputs, and whether all methods receive comparable generation/scoring budgets. How was alpha selected, and was that selection separate from the final evaluation?

  2. Performance beyond top-100: I appreciate the whole-set averages in Tables 15-16. Using the existing generation outputs, can you add the same-backbone unsteered and naive-global controls and show whether the benefit persists across the candidate distribution, for example through medians/quantiles, the fraction of valid outputs improving over their own input, and a top-k curve? Please provide the underlying per-candidate scores or anonymized evaluation records and clarify averaging denominators and variability across independent runs, where available. In particular, how should the strong MOSES top-100 QED be interpreted alongside its lower whole-set mean?

  3. Distinctive contribution and mechanism: Can a focused generation ablation compare the proposed partition/composition against a size- and sparsity-matched random partition and a local-direction control, with matched perturbation norms and tuning budgets? Which benefit specifically requires the joint molecule-coordinate construction, and which conclusions follow from the conditional theorem rather than empirical evidence?

  4. Reporting and executable artifact: Please reconcile the QM9 whole-set LogP/SA in Tables 16 and 22 and identify any differences in settings, filtering, or populations. Please also correct generate_from_steer_new.py, whose steer_generate_similar signature omits zsteer while callers pass both a positional vector and orig_smiles=, and crossvalidation_unified.py, which passes unsupported subspace_rank to compare_sparse_dense_steering. A minimal runnable example and table-producing configurations/reference outputs would help resolve these concerns.

4 comments
AuthorsOpenReview ↗

Response to Reviewer dAqH part 1 of 3

(Response to Reviewer dAqH part 1 of 3)

We thank the reviewer for a detailed and accurate review. We appreciate the recognition that the intervention is static and requires neither fine-tuning nor per-molecule direction selection, and that the same-backbone controls show candidate-level gains, with ZINC250K top-100 QED rising from 0.88/0.89 to 0.94 and SA falling from 2.05/1.94 to 1.54. We are also glad the representation analyses beyond the masking construction were included as a strength. We address each weakness and question individually below. We have marked all updates in blue text within the manuscript.

W1/ Q3

The exact zero-overlap guarantee concerns the coordinate supports of the constructed subspaces. Each local vector vkv_k belongs to a disjoint coordinate block, SkS_k, so we have PkG=vkP_k G = v_k by construction. This establishes the exact recovery under projection, and not the absence of functional cross-block effects during generation, as mentioned by the reviewer. We inject the full normalized composite vector. It does not have molecule-dependent routing besides this construction guarantee, and it is also the case that subsequent nonlinear transformer computations may mix information across coordinates. We do not have guarantees on the downstream non-linear processing, we focus on the injection of steering at a single layer, and its effects on property optimization.

To further examine the effect of this composition during generation, we added a matched generation ablation comparing two intervention strategies: (i) applying the individual cluster-local steering vector mapped to each molecule's assigned cluster, restricted to its corresponding latent-coordinate subspace, and (ii) applying the full ChemSteer composite vector, which superposes the cluster-local directions into a single global intervention. This comparison directly evaluates whether the full composite intervention incurs a substantial performance penalty relative to applying only the relevant local direction.

DatasetEvaluationSteeringMethodLogPQEDSA
ChEMBLTop-100Mapped Individual5.48±3.410.93±0.001.55±0.12
ChEMBLTop-100ChemSteer Composite5.24±1.190.94±0.101.55±0.18
ChEMBLOverallMapped Individual3.08±1.690.67±0.172.62±0.62
ChEMBLOverallChemSteer Composite2.46±2.630.73±0.153.01±0.82
MOSESTop-100Mapped Individual6.46±0.430.95±0.001.50±0.09
MOSESTop-100ChemSteer Composite6.68±0.640.95±0.001.50±0.08
MOSESOverallMapped Individual2.56±1.030.79±0.112.48±0.51
MOSESOverallChemSteer Composite2.30±0.740.47±0.134.26±0.40
QM9Top-100Mapped Individual2.38±0.490.87±0.022.34±0.32
QM9Top-100ChemSteer Composite2.70±0.160.92±0.011.61±0.17
QM9OverallMapped Individual0.49±1.090.52±0.123.86±0.89
QM9OverallChemSteer Composite0.24±0.930.53±0.153.88±0.95
ZINCTop-100Mapped Individual6.83±1.020.94±0.001.59±0.10
ZINCTop-100ChemSteer Composite6.82±0.710.94±0.001.54±0.10
ZINCOverallMapped Individual2.69±1.300.74±0.142.77±0.67
ZINCOverallChemSteer Composite2.45±1.590.81±0.162.45±0.69

The results above show broadly comparable top-100 optimization performance between the two strategies. Across the full generated distributions, neither approach uniformly dominates: the mapped individual vector achieves higher overall LogP on ChEMBL, MOSES, and ZINC, while QED and synthetic accessibility exhibit dataset-dependent trade-offs. These results suggest that the composite intervention retains competitive optimization behavior relative to the corresponding individual local interventions, although they do not establish the absence of all cross-block effects. The composite construction also has a practical methodological advantage because it produces a single global steering intervention incorporating the cluster-local directions, rather than requiring a separate vector to be selected and applied according to each molecule's cluster assignment. Thus, the ablation evaluates the empirical consequences of this global composition while distinguishing its behavior from that of isolated local steering. We have added the above table and this discussion to Appendix V.

Our contribution is the joint molecule-coordinate partition in which molecule clusters identify local regions of representation space, while coordinate clusters identify the corresponding property- relevant features. Theorem 1 (previously in the Appendix, now in the main text) establishes exact block recovery under the stated block structure assumptions. CAA optimality also requires aligned block LRH (Corollary 1). Table 6 provides the held-out evidence that the partitioning we use captures useful property separation and it outperforms row-wise and random experiments, showing that ChemSteer’s partitioning leads to increased performance.

AuthorsOpenReview ↗

Response to Reviewer dAqH part 2 of 3

(Response to Reviewer dAqH part 2 of 3)

In Table 6, we compare the following settings: (1) Sep*(1), a single global CAA direction (no partitioning); (2) Sep*~\tilde{\text{Sep*}}(k), row-piecewise CAA, full dimensionality per cluster (isolates the value of row-wise local specialization alone); (3) Sep*(k), the coordinate-restricted optimum of equation 4, approximated via alternating optimization initialized from the deployed SCC partition; (4) Sep(πChemSteer\pi^{\text{ChemSteer}}), the actual CAA vectors, evaluated for held-out generalization; and (5) Sep(πrand\pi^\text{rand}), a random-partition null. We show that amongst these settings, ChemSteer partitioning performs the best.

Theory versus evidence clarification.

From the construction alone, no theorem needed: ϵ=0\epsilon = 0 and PkG=vkP_k G = v_k. These follow from each coordinate being assigned to exactly one block, and they concern vector composition, not generation. From Theorem 1, unconditionally given A1–A3: spectral co-clustering recovers the block partition of the affinity A. The theorem makes no reference to the property y. From the corollary, conditional on aligned block LRH: that the recovered partition maximises the separation objective. Aligned block LRH is a definition we state and test, not something we prove, and it is a property of the tuple (A, H, y) rather than of the algorithm. From empirical evidence only: that the clusters are chemically meaningful (Table 1, Appendix H); that local directions vary systematically with chemical structure (Table 2); that the deployed partition exceeds alternatives in held-out separation (Table 6); and every generation result. We add that separation-optimality would not by itself imply steering quality. An unpartitioned global direction attains the highest separation in every row of Table 6 while steering worst in Tables 3–4, degrading QM9 LogP from 1.50 unsteered to 0.54. Sep is a representation diagnostic; the generation ablation above is the evidence that bears on the contribution.

W2/ Q2/ Q4

We have added the same backbone controls in the whole set average results in Tables 15 and 16. The reviewer is correct to point out this minor case where ChemSteer does not outperform the other methods (MOSES QED for the whole-set average is 0.47, compared with 0.54 for Gradient Flow and Wave (unsupervised)). This is a modest difference in one metric and one dataset, rather than a systematic limitation of ChemSteer. Importantly, ChemSteer improves QED on the other three datasets and achieves competitive or superior results across the remaining property–dataset combinations. We would like to point out that on all twelve dataset x chemical property configurations, ChemSteer has the best top 100 performance, and ChemSteer has the top performance on the other 11/12 dataset x chemical property configurations for the whole set averages. In the case of MOSES and QED, the performance is similar, and this is the only case of underperformance of ChemSteer. For the top 100 (Table 3), ChemSteer increases QED to 0.95 on MOSES whereas all other methods are less than 0.92 (the aforementioned gradient flow is 0.92, and wave (unsupervised is 0.91).

We have added the table for the overall averages for the naive global and unsteered results in Table 9, and provide it below.

OverallDatasetsLogPQEDSA
No SteerChEMBL3.62±1.550.68±0.172.53±0.62
MOSES2.38±1.190.78±0.092.48±0.51
QM90.91±1.300.57±0.164.04±0.92
ZINC2.85±1.410.74±0.132.84±0.92
GlobalChEMBL3.52±1.120.69±0.162.59±0.57
MOSES2.54±1.090.79±0.112.41±0.43
QM93.42±5.490.42±0.184.08±0.96
ZINC2.67±1.350.74±0.142.67±0.58
ChemSteerChEMBL2.46±2.630.73±0.153.01±0.82
MOSES2.30±0.740.47±0.134.26±0.40
QM90.24±0.930.53±0.153.88±0.95
ZINC2.45±1.590.81±0.162.45±0.69

Concerning the QM9 values in Table 16 versus Table 22, this was an oversight on our part. The 3.88 and 3.12 values in Table 22 were from a previous run in which we were not enforcing the QM9 dataset constraint that the molecule can only have at most nine heavy atoms. The correct and updated runs yield the values in Table 16. We thank the reviewer for pointing out this discrepancy.

Concerning the function calls, we have updated the supplementary material to have the correct and up to date version of our code.

AuthorsOpenReview ↗

Response to Reviewer dAqH part 3 of 3

(Response to Reviewer dAqH part 3 of 3)

Q1

For each dataset, method, and target property, we rank the generated candidate pool according to the requested optimization objective, that is, descending LogP, descending QED, or ascending synthetic accessibility (SA). We then report the mean and standard deviation of the 100 highest-ranked candidates. Thus, the selected molecules are property-specific, making it such that the top-100 for LogP need not be the same molecules as the top-100 for QED or SA. We adopted this presentation because the molecular optimization baselines we compare against emphasize their best optimized candidates, often reporting only a small number (top 1–3) of top results. Reporting the top-100 mean and standard deviation provides a more statistically informative comparison than reporting only one to three individual best molecules.

We do not intend the top-100 results to imply that all generated molecules achieve these property values, nor do we use them as a substitute for evaluating generation quality. We therefore also report the averages and standard deviations across the full 15,000-molecule generation sets in Appendix J (Tables 15–16), alongside validity, uniqueness, and novelty in Table 5.

The full-set results provide a more representative view of the generated distributions and broadly support the effectiveness of ChemSteer, while also revealing where its advantage is less pronounced. For example:

  • ChEMBL: Across all 15,000 generated molecules, ChemSteer achieves LogP (2.46 ±\pm 2.63), QED (0.73 ±\pm 0.15), and SA (3.01 ±\pm 0.82). The corresponding Gradient Flow results are (-10.17 ±\pm 13.18), (0.41 ±\pm 0.19), and (4.82 ±\pm 1.04), respectively. This shows that the improvement is not restricted to the best 100 candidates.

  • MOSES: ChemSteer achieves LogP (2.30 ±\pm 0.74), QED (0.47 ±\pm 0.13), and SA (4.26 ±\pm 0.40). The full-set QED result is comparable to, rather than better than, Gradient Flow and Wave, which both report (0.54 ±\pm 0.19). We agree that it is an important qualification that ChemSteer's top-100 QED advantage does not translate into the same advantage across the entire MOSES distribution.

  • QM9: ChemSteer achieves LogP (0.24 ±\pm 0.93), QED (0.53 ±\pm 0.15), and SA (3.88 ±\pm 0.95), compared with Gradient Flow's (-2.52 ±\pm 2.06), (0.46 ±\pm 0.08), and (4.00 ±\pm 1.02). The full-set results therefore show improvements in LogP and QED and a more modest difference in SA.

  • ZINC: ChemSteer achieves LogP (2.45 ±\pm 1.59), QED (0.81 ±\pm 0.16), and SA (2.45 ±\pm 0.69). In comparison, Gradient Flow achieves (-9.27 ±\pm 11.44), (0.43 ±\pm 0.19), and (5.06 ±\pm 0.99). These results indicate substantial full-distribution improvements in all three reported property metrics.

These full-set results are important because they show that ChemSteer's gains are not solely a consequence of selecting a small number of high-performing candidates.

We agree that validity must be considered alongside property optimization. Table 5 reports the fraction of RDKit-parsable molecules across the complete generated sets. ChemSteer's validity is 100% on ChEMBL, 93.2% on MOSES, 96.4% on QM9, and 87.1% on ZINC250K. The properties reported for the optimization performance are only computed over the valid set of molecules because the property descriptors are calculated using RDKit, hence, invalid SMILES cannot be assigned LogP, QED, or SA values. All metrics are computed over the valid set that has been canonicalized before computing the respective properties. All molecules are generated using the same generation scheme as detailed in Section 4.2 (ChemSteer Step (3)). Novelty is measured against the source dataset, which contains every input molecule, so ChemSteer's 100% novelty across all four datasets implies that no generated molecule is identical to its input. The values on α\alpha are tuned per dataset and property pair, and these values, along with the α\alpha ablation are given in Appendix K, as described in Section 4.2 (ChemSteer Step 3).

Thank you again for your time and for the detailed review you provided on our paper. As the discussion period is ongoing, we wanted to check in and see if you have any follow-up questions or if there is any part of our response we could clarify further for you. We are ready to provide any additional information that might be helpful.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response to All Reviewers and the Area Chair (Part 1 of 2)

General Response to All Reviewers and the Area Chair (Part 1 of 2)

We thank all three reviewers and the Area Chair for their thoughtful and constructive feedback. We are encouraged that the reviewers recognized several central aspects of ChemSteer. Reviewer dAqH highlighted the practical value of a static intervention that requires neither fine-tuning nor per-molecule direction selection, and noted that our same-backbone controls demonstrate candidate-level gains. The reviewer also recognized our efforts to investigate the representation hypothesis beyond the coordinate-masking construction, including the use of Mantel tests to account for dependence between cluster pairs and controls for cluster size. Reviewer saQH appreciated the lightweight, training-free nature of the approach and found the investigation of the Linear Representation Hypothesis (LRH) in molecular representation spaces interesting. In particular, the reviewer noted that our analysis of dense local directions provides evidence beyond the trivial orthogonality induced by disjoint coordinate supports. Reviewer vqZH found the geometric motivation and problem framing compelling, and viewed the method as a promising development for steering chemical language models.

In response to the reviews, we conducted additional experiments and analyses, clarified the scope of our theoretical claims, and substantially reorganized the presentation of the theoretical framework. New results and revisions are highlighted in blue in the revised manuscript. The new analyses appear in Appendices T–W, and Theorem 1 has been moved from the Appendix into the main text.

Theoretical reorganization and clarification of contributions. A central change is that we now present the theoretical contribution as a framework for joint molecule–coordinate partitioning, rather than as an orthogonality result arising from coordinate masking alone. The partition jointly identifies local molecular neighborhoods and the corresponding representation-coordinate subspaces in which local property contrasts are estimated. This structure allows ChemSteer to combine locally estimated steering directions into a single static intervention, without retraining the model or selecting a separate direction for each generated molecule.

To make the guarantees precise, we now distinguish two theoretical results that were previously presented less explicitly. First, Theorem 1, now in the main text, establishes exact recovery of the paired molecule–coordinate affinity partition by bipartite spectral co-clustering under explicit structural assumptions on the affinity matrix. This recovery result does not depend on the target property. Second, we state the aligned block LRH condition separately: when the recovered affinity partition is aligned with the block structure that maximizes aggregate local contrastive separation, the resulting partition is optimal for that separation objective. This second result is conditional on the alignment assumption; it is not implied by exact recovery alone. Together, these results provide a theoretical foundation for identifying local representation subspaces and composing their corresponding steering directions.

We have also clarified the interpretation of orthogonality. Disjoint coordinate supports guarantee zero coordinate overlap between the composed local directions by construction. This is a property of the intervention, not evidence that the pretrained model contains intrinsically independent chemical subsystems or that nonlinear downstream computations cannot produce cross-block interactions during generation. We distinguish this construction-level guarantee from our empirical analyses of dense local directions and generation outcomes.

AuthorsOpenReview ↗

General Response to All Reviewers and the Area Chair (Part 2 of 2)

General Response to All Reviewers and the Area Chair (Part 2 of 2)

The new results in Appendices T–W provide additional tests of the geometric motivation, intervention design, structural correspondence, and partition granularity:

Appendix T: Independent analysis of local representation geometry. We measure variation in local tangent spaces using the representations, independently of the optimized molecular property, to distinguish between changes in tangent-space orientation and changes in the property gradient within a fixed tangent space. Using local PCA with q=10q=10, we compare between-cluster principal angles against a split-half estimation floor and a random-subspace reference of 87.62∘±0.18∘87.62^\circ \pm 0.18^\circ. In all four datasets, between-cluster angles exceed the corresponding estimation floor, while remaining substantially more aligned than random subspaces. Tangent-space separation also increases with centroid distance under the Mantel procedure used in Table 2 (ChEMBL r=+0.123r=+0.123, p=0.0014p=0.0014; MOSES r=+0.219r=+0.219, QM9 r=+0.501r=+0.501, and ZINC r=+0.377r=+0.377, each p=0.0001p=0.0001). An independent comparison of cross-cluster property-direction mismatch, calibrated against the corresponding split-half null, closely tracks the tangent-space measurements across the four datasets. These findings provide evidence that local representation geometry varies systematically across molecular clusters, independently of the optimized property.

Appendix U: Source-to-generated structural similarity. We measure Tanimoto similarity between generated molecules and their corresponding source molecules under steered, unsteered, and global-steering conditions. ChemSteer achieves the highest source-to-generated similarity across all four datasets and all three objectives. This provides an additional measure of structural correspondence under steering, complementing our existing validity, uniqueness, and novelty analyses.

Appendix V: Composite versus molecule-mapped local interventions. We compare the composite intervention against applying only the local direction associated with each molecule's assigned cluster. The two approaches achieve broadly comparable top-100 optimization, with neither uniformly dominating across datasets and properties. This comparison clarifies the practical role of composition: ChemSteer consolidates cluster-local directions into one static intervention rather than requiring molecule-specific direction selection.

Appendix W: Sensitivity to partition granularity. We evaluate eight values of kk, ranging from 5 to 500, to examine how the number of clusters affects performance. k=300k=300 achieves the highest top-100 LogP on MOSES, QM9, and ZINC, and is comparable to the best-performing value on ChEMBL. This is evidence that the selected k=300k=300 is competitive across datasets.

We also added same-backbone unsteered and naive global-steering controls to the whole-set comparisons in Tables 9, 15, and 16. These comparisons help distinguish the effects of ChemSteer's local composition from gains attributable simply to applying a steering intervention. Finally, we replaced the supplementary code with a corrected version whose function signatures are consistent with their call sites.

Clarification on novelty and conceptual contribution. ChemSteer’s contribution is a new formulation of the Linear Representation Hypothesis (LRH) for molecular property control, namely, our piecewise-linear LRH. Under our formulation, a molecular property is locally linear within regions of chemical representation space, but its associated linear direction varies systematically across different regions of the vast space. Thus, a single global direction need not adequately capture a property whose representation is locally linear. ChemSteer identifies the regions over which local linear steering is appropriate, jointly discovers the corresponding molecular regions and latent coordinate subspaces, and composes their locally estimated directions into a single static inference-time intervention. To our knowledge, this formulation has not been previously developed or empirically investigated as a framework for activation steering in chemical language models.

The novelty therefore lies in connecting three elements: (1) a piecewise-linear formulation of property representations, in which local directions vary across chemical regions; (2) a joint molecule–coordinate partition that identifies where those local linear relationships hold and which representation coordinates support them; and (3) a composition procedure that consolidates the resulting local directions into one global intervention, without per-molecule direction selection, retraining, or fine-tuning.

We believe these revisions make the paper's methodological contribution, theoretical guarantees, and empirical scope substantially clearer. We thank the reviewers and Area Chair again for helping us strengthen both the analysis and presentation of the work.

AuthorsOpenReview ↗

Update for all reviewers and the AC

ChemSteer Performance on PubChem and Enamine

We have also now added Appendix H: Performance of ChemSteer on Enamine and PubChem datasets. We thank reviewer vqZH for their request for results of ChemSteer on the Enamine and PubChem datasets. We wanted to bring the results to attention because they demonstrate impressive performance of ChemSteer on both datasets, demonstrating high performance on molecules from across this heterogeneous chemical space (please see our comment to Reviewer vqZH, and Appendix H for full results and analysis). The results on PubChem, LogP are particularly notable because ChemSteer achieves a higher overall LogP average (6.97 vs. 3.53) and substantially improves the bottom-100 average (1.11 vs. -5.89), while also producing a higher top-100 average (24.99 vs. 23.86). The increase of the bottom-100 average LogP molecules is important because it shows that the overall increase does not come merely from the high-end values pulling the mean. Specifically relevant to the points raised by Reviewer dAqH Q2 and Reviewer saQH W5, this is a strong example of ChemSteer performing well on molecules from across the distribution of property values, not merely top-100 bias. This demonstrates the power of ChemSteer’s method and the piecewise linear, local steering which effectively handles steering across a diverse space.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.