- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors present analysis justifying a geometric framing of chemical property space as inherently curved, and stating that the linear representation hypothesis is only valid within a small step relative to the radius of curvature. Using this framing, the authors then go on to present a post-hoc intervention for steering chemical language models based on spectral co-clustering to determine local steering vectors within clusters that are then aggregated into a global steering vector used for steering hidden layer activations.
Critical strengths
- The presentation and framing of the method's geometric motivation is very strong, and clearly delineates the problem for the remainder of the paper. This helps contextualize the later steering methodology and results
- The method demonstrates good performance across a variety of models, making a compelling development for tool for steering chemical language models
- The writing is easy to follow and not too equation or text-dense, so the authors' contributions come across clearly
Critical weaknesses
- More characterization on the generated molecules would be useful for understanding the feasibility of the designs generated by the model under steered and non-steered conditions, using a metric such as Tanimoto to source would be helpful for understanding the generative diversity
- While there are many baseline methods for comparison, the authors should consider using fairer comparisons in some cases since some of the gaps between models themselves are larger than any of the steering benefits observed (e.g. ChemSpace versus SmileyLLama on QM9 QED)
- The authors should try additional datasets such as PubChem and Enamine
Questions for authors
- There are three steps to the process, where are ablations testing the individual contributions of each part of the three-stage approach (e.g., what happens if you ablate clustering with a simpler method like random partitions?)
- How does performance vary with k and what is the distribution of cluster sizes?
- Some better reporting on some more descriptive statistics of the generated molecules would help characterize the things the model generates (e.g. how big are the molecules, what are their similarity to source between steered output and input?)
- What happens if you look at less restrictive statistics such as the full average or basically things beyond the Top-100 mean?
7 comments
Response to Reviewer vqZH part 2 of 3
Q1
We do have extensive ablations of the methods components in the Appendix. Table 8 compares no steering, naive-global steering, and ChemSteer, all on the SmileyLlama backbone. ChemSteer has the best performance over these ablations. Table 6 compares held-out separation under global, row-wise, coordinate-restricted, and random partitions. We compare the following settings: (1) Sep*(1), a single global CAA direction (no partitioning); (2) (k), row-piecewise CAA, full dimensionality per cluster (isolates the value of row-wise local specialization alone); (3) Sep*(k), the coordinate-restricted optimum of equation 4, approximated via alternating optimization initialized from the deployed SCC partition; (4) Sep(), the actual CAA vectors, evaluated for held-out generalization; and (5) Sep(), a random-partition null. We show that amongst these settings, ChemSteer partitioning performs the best. Appendix H.3 compares sparse–dense alignment under random clustering. Appendix K ablates the steering magnitude, , and the steering layer. We believe that these cover the components of ChemSteer. If there are other ablations the reviewer would like us to perform concerning any aspects of ChemSteer, we would be happy to conduct these experiments.
Q2
The main experiments use (k=300). Appendix E reports the cluster-size distributions and their averages: ChEMBL 57 molecules, MOSES 71, QM9 52, and ZINC 56 per cluster. The corresponding average subspace ranks are 10, 7, 10, and 10, respectively. The average numbers of unique scaffolds per cluster are 53, 65, 32, and 55, respectively.
Our theory does not derive an optimal number of clusters. specifies the granularity of the paired molecule-dimension partition. For any feasible , disjoint coordinate supports guarantee exact orthogonality of the corresponding steering subspaces, while the quality of the resulting local approximations depends on how well the partition captures representation and property heterogeneity.
To assess sensitivity to this choice, we conducted a -sweep over . The results are visualized in Appendix W, Figure 9, and we report the tables here.
k Ablation on LogP (Top-100 Average)
| k | ChEMBL | MOSES | QM9 | ZINC |
|---|---|---|---|---|
| 5 | 4.14±0.77 | 3.12±0.36 | 1.40±0.61 | 3.40±0.39 |
| 10 | 4.20±0.80 | 3.41±1.07 | 1.23±0.28 | 3.83±1.50 |
| 50 | 4.45±2.31 | 4.27±0.57 | 1.52±0.52 | 3.90±1.68 |
| 100 | 5.04±0.77 | 4.25±0.51 | 1.31±0.21 | 4.37±0.42 |
| 200 | 5.27±1.10 | 4.68±1.52 | 1.43±0.33 | 4.49±0.55 |
| 300 | 5.24±1.19 | 6.68±0.64 | 2.70±0.16 | 6.82±0.71 |
| 400 | 5.25±1.38 | 4.30±0.66 | 1.39±0.16 | 5.52±4.98 |
| 500 | 5.53±2.23 | 4.51±1.40 | 1.30±0.21 | 5.14±1.83 |
k Ablation on LogP (Overall Average)
| k | ChEMBL | MOSES | QM9 | ZINC |
|---|---|---|---|---|
| 5 | 2.18±1.61 | 1.52±1.05 | 0.12±1.18 | 1.77±1.25 |
| 10 | 2.14±1.55 | 1.52±1.18 | 0.16±0.76 | 1.87±1.47 |
| 50 | 2.14±2.00 | 2.24±1.05 | 0.16±1.04 | 1.87±1.55 |
| 100 | 3.03±1.61 | 2.25±1.07 | 0.20±0.73 | 2.36±1.23 |
| 200 | 3.22±1.65 | 2.27±1.27 | 0.24±0.97 | 2.37±1.30 |
| 300 | 2.46±2.63 | 2.30±0.74 | 0.24±0.93 | 2.45±1.59 |
| 400 | 3.15±1.71 | 2.51±1.12 | 0.27±0.89 | 3.06±2.55 |
| 500 | 3.30±1.95 | 2.57±1.21 | 0.25±0.95 | 2.99±1.62 |
The results demonstrate that the effect of is dataset- and evaluation-dependent. This behavior is consistent with the geometric motivation for local steering. When is small, clusters contain more molecules and cover broader regions of the representation manifold. If the manifold is curved, the local tangent spaces and projected property gradients can vary across these regions, such that a single cluster-level steering direction becomes a less accurate approximation to the molecule-specific property direction. Increasing can improve this local approximation by producing finer partitions with less within-cluster geometric variation.
However, increasing also introduces a competing constraint. The corresponding steering subspaces become more localized in the molecule-coordinate partition, potentially restricting the intervention to fewer representation dimensions. While this can reduce irrelevant perturbations, an overly restrictive subspace may omit dimensions needed to express the desired property direction. Furthermore, finer partitions reduce the number of molecules available for estimating each local steering vector, potentially increasing estimation variance. The relationship between and steering performance thus reflects a trade-off between local geometric fidelity and the expressive capacity of the steering subspace.
achieves the highest top-100 average LogP on MOSES (6.68), QM9 (2.70), and ZINC (6.82), while ChEMBL achieves a comparable value at (5.24) to its highest value at (5.53). For the overall generated distributions, the best-performing varies across datasets.
Response to Reviewer vqZH part 3 of 3
(Response to Reviewer vqZH part 3 of 3)
Q4
We thank the reviewer for raising this important distinction. We agree that evaluating only the top-100 generated molecules measures best-candidate optimization rather than the typical quality of the entire generated distribution. These are complementary evaluation objectives, and we intended the top-100 results to measure the former, as is standard among our baselines, not to serve as an unbiased estimate of overall generation quality.
For each dataset, method, and target property, we rank the generated candidate pool according to the requested optimization objective, that is, descending LogP, descending QED, or ascending synthetic accessibility (SA). We then report the mean and standard deviation of the 100 highest-ranked candidates. Thus, the selected molecules are property-specific, making it such that the top-100 for LogP need not be the same molecules as the top-100 for QED or SA. We adopted this presentation because the molecular optimization baselines we compare against emphasize their best optimized candidates, often reporting only a small number (top 1–3) of top results. Reporting the top-100 mean and standard deviation provides a more statistically informative comparison than reporting only one to three individual best molecules.
We do not intend the top-100 results to imply that all generated molecules achieve these property values, nor do we use them as a substitute for evaluating generation quality. We therefore also report the averages and standard deviations across the full 15,000-molecule generation sets in Appendix J (Tables 15–16), alongside validity, uniqueness, and novelty in Table 5.
The full-set results provide a more representative view of the generated distributions and broadly support the effectiveness of ChemSteer, while also revealing where its advantage is less pronounced. For example:
-
ChEMBL: Across all 15,000 generated molecules, ChemSteer achieves LogP (2.46 2.63), QED (0.73 0.15), and SA (3.01 0.82). The corresponding Gradient Flow results are (-10.17 13.18), (0.41 0.19), and (4.82 1.04), respectively. This shows that the improvement is not restricted to the best 100 candidates.
-
MOSES: ChemSteer achieves LogP (2.30 0.74), QED (0.47 0.13), and SA (4.26 0.40). The full-set QED result is comparable to, rather than better than, Gradient Flow and Wave, which both report (0.54 0.19). We agree that it is an important qualification that ChemSteer's top-100 QED advantage does not translate into the same advantage across the entire MOSES distribution.
-
QM9: ChemSteer achieves LogP (0.24 0.93), QED (0.53 0.15), and SA (3.88 0.95), compared with Gradient Flow's (-2.52 2.06), (0.46 0.08), and (4.00 1.02). The full-set results therefore show improvements in LogP and QED and a more modest difference in SA.
-
ZINC: ChemSteer achieves LogP (2.45 1.59), QED (0.81 0.16), and SA (2.45 0.69). In comparison, Gradient Flow achieves (-9.27 11.44), (0.43 0.19), and (5.06 0.99). These results indicate substantial full-distribution improvements in all three reported property metrics.
These full-set results are important because they show that ChemSteer's gains are not solely a consequence of selecting a small number of high-performing candidates.
Response to Reviewer vqZH part 1 of 3
(Response to Reviewer vqZH part 1 of 3)
We thank the reviewer for the positive assessment, and especially for the comment that the geometric motivation delineates the problem and motivates the methodology. We are glad the authors found our writing quality and presentation to be strong, and for the acknowledgement of the breadth across backbones. We address each weakness and question below. We have marked all updates in blue text within the manuscript.
W1/ Q3
We agree that more detailed molecular characterization would strengthen the evaluation of the feasibility and structural properties of generated molecules. Table 5 reports validity, uniqueness, and novelty, while Table 7 examines chemical coherence within the spectral co-clustering by comparing within-cluster Tanimoto similarity against random assignments. However, these analyses do not directly characterize the structural similarity between generated molecules and molecules in their respective source datasets.
To address this point, we conducted an additional analysis computing Tanimoto similarity between source and generated molecules under steered, non-steered, and global-steering conditions, for each of the three optimization objectives (LogP, QED, and SA). The results show that ChemSteer consistently achieves higher source-to-generated Tanimoto similarity across all four datasets and all three objectives. The table below provides the tanimoto similarity within each source dataset to give context for the results table which follows it.
These results indicate that ChemSteer generates molecules that retain greater structural similarity to their source datasets than the two comparison conditions. This suggests that the proposed local steering approach better preserves source-associated chemical structure while enabling property-directed optimization, in contrast to non-steered generation and global steering. Combined with the validity, uniqueness, novelty, and cluster-coherence analyses, this provides a more comprehensive characterization of the generated molecules. We have added these results and discussion to Appendix U and reference it at the end of Section 5.
W2
We agree that differences in model architectures can confound comparisons between methods evaluated using different pretrained backbones. We have used the baselines within the paper because they are the current best methods we found in literature for molecular property optimization. None of them, or to the best of our knowledge, any other published work, uses chemical language model steering methods for optimization. We include controlled comparisons that directly address the reviewer's concern by holding the underlying generation backbone fixed. In Table 8, we compare ChemSteer against both unsteered generation and naive global steering using the same SmileyLlama backbone, thereby isolating the contribution of the steering mechanism from differences in model architecture. ChemSteer consistently improves molecular property optimization over both controls. For example, on ZINC250K, the top-100 mean QED increases from 0.88 with unsteered generation and 0.89 with naive global steering to 0.94 with ChemSteer. Similarly, the mean SA score decreases from 2.05 and 1.94, respectively, to 1.54 with ChemSteer. These improvements cannot be attributed to the choice of backbone, since all three conditions use the same pretrained model and generation framework. Rather, they demonstrate the additional benefit of ChemSteer's cluster-local steering over both the underlying model and a simpler global steering alternative. We also provide additional GPT-2 experiments in Table 9 to provide additional evidence that the steering approach is not intrinsically tied to a single architecture. We further examine structural preservation through row-wise Tanimoto similarity (W1/Q3) between generated molecules and their corresponding reference molecules. Across all four datasets and all three optimization objectives, ChemSteer achieves higher mean Tanimoto similarity than both unsteered generation and global steering. This provides complementary evidence that the improvements arise from the steering strategy and are not simply differences in the underlying generation backbone.