- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies property encoding and steering in SELFIES-based Transformer VAEs. It compares autoregressive and non-autoregressive variants using linear and MLP probes, examines sequence-level confounds through residualization and directional alignment, and evaluates probe-derived directions by decoding latent traversals and recomputing RDKit descriptors.
Critical strengths
-
The paper addresses a relevant question: whether property information in molecular representations translates into control over decoded molecules.
-
Steering is evaluated using recomputed molecular properties, including raw and residual directions across 50 held-out seeds.
-
Cross-fitted nonlinear nuisance models and held-out residual checks provide useful controls for the measured SELFIES confounds.
Critical weaknesses
1. Figure 1 contains substantive errors and is difficult to read:
In SELFIES 2.1.1, [C][C][=O][O] decodes to acetaldehyde (CC=O), not the acetic acid (CC(=O)O) shown in the figure. The aromatic example also uses the unsupported [c] token. The direct connection from the latent vector to “SELFIES decoding” conflates the neural decoder with the string parser. Crowded text, small labels, and inconsistent arrow alignment and connection points further obscure the pipeline. As the main methodological overview, this figure requires both a correctness check and substantial redesign.
2. Table 20 is inconsistent with the reported probe results: All 30 AR entries in Table 20 disagree with the corresponding differences computed from Tables 3, 4, and 19. For example, the reported values give raw gaps of 0.0869 for FractionCSP3 and 0.1401 for TPSA, versus 0.067 and 0.078 in Table 20. The monotonic-group means likewise become 0.0909/0.2135 for raw/residual targets, versus 0.061/0.152. These discrepancies exceed rounding error and cannot all be explained by substituting the Ridge baseline. Appendix A.1 describes several probe configurations without specifying which produced each table. The results need to be reconciled before the quantitative interpretation of these gaps can be assessed.
3. The interpretation conflates predictability, monotonicity, and steerability: Sections 3.5–3.6 infer approximate global linearity from successful monotonic traversal and use larger MLP–linear gaps to argue against stable global directions. Neither implication follows. For example, is nonlinear but strictly monotonic along the fixed direction . An MLP improvement therefore does not establish that local steering is necessary, and successful traversals on sampled seeds do not establish a globally affine property map. Figure 1’s implication that high raw and residual establishes steerability is similarly too strong. Appendix F.1 proposes gradient-based local steering but provides no decoded results to validate this interpretation.
4. The comparisons do not isolate the source of the AR model’s advantage: Both Non-AR baselines use an explicit length-prediction loss (Table 7), directly targeting a quantity treated as a confound, whereas the AR model does not. The models also differ in depth, latent dimensionality, and KL scheduling. Traversal protocols introduce further differences, including fixed versus free decoding length and standardized versus original latent coordinates. These experiments compare particular systems, but do not establish that autoregression explains their differences in latent organization or controllability. Matched evaluation protocols and targeted ablations of the length objective and model capacity are needed to support that attribution.
5. Decoded steering quality is insufficiently characterized: Table 16’s median per-seed Spearman scores support overall property ordering, but do not establish fine-grained or structurally coherent control. A 100-point path containing only three repeated molecular states can achieve . Although Appendix E.1 lists additional diagnostics, the steering results do not adequately report path diversity, monotonicity violations, or similarity to the seed. The nonlinear nuisance check is useful, but addresses only the measured confounds; reduced directional alignment also does not establish reduced confound variation in decoded molecules. Per-path statistics, representative molecular edits, decoded confound trajectories, and matched random/confound-direction controls are needed to substantiate the stronger steering claims.
Questions for authors
Please see the Weaknesses section above.