- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose BOLEK - a language model for molecular properties prediction along with a suite of molecular alignment tasks and the synthetic reasoning preparation routing aimed at pretraining this language model to provide a good start for downstream molecular prediction tasks (the subset of TDC tasks in this case).
From my understanding the whole approach is transferable to any decoder-only LLM that supports taking as an input user custom embeddings. In this case, the authors pass the molecular embedding retrieved after 2 layer MLP feeded with molecular Morgan description. From my point of view, the requirement to take user custom embeddings in input is not that restrictive since many modern LLMs support this, but not well justified and even if briefly but make the training and inference more engineerically difficult than just plain text-only LLM.
The molecular alignment tasks resemble the predictive tasks on synthetic molecular descriptors and substructure detecting routine. This part is not novel and many approaches use the molecular encoder and embedding projector pretraining, for example to learn meaningful 3D structure or molecular graph input embedding. Still in this paper this step looks unnecessary and redundant because the embedding MLP learns to basically predict extended molecular fingerprints based on some base fingerprint. Also ablation study doesn’t provide any evidence that this part is essential - can this projector pretraining be omitted and the projector be trained along with LLM backbone on synthetic reasoning + downstream tasks instead of this separate step.
The moderately interesting part of the paper is synthetic reasoning traces generation. While the synthetic reasoning generation is not novel for general LLM training and the whole pipeline resembles general synthetic reasoning generation with chemical domain adaptation, this topic is underrepresented in the chemical LLM area. The main drawback of the proposed synthetic reasoning is that it’s designed mainly for predictive tasks and the transferability of the other chemical applications is not clear.
The experiment part is somewhat normal and supports the quality assessment of the model on predictive tasks and also includes the ablation. Still, while predictive tasks are important for the chemical domain, the only predictive tasks experiments are not enough for the current state of the chemical LLM field. Chemical LLMs [a, b] cover different applications like retrosynthesis, molecular optimization and molecular generation given a set of conditions.I recommend to think about adaptation of FGBench [c] as an additional set of predictive tasks on property change prediction given the molecular pairs.
The text of the paper is written in an easy-to-follow and clear way. The only problem is the experiments section, where authors introduce low-level metrics values comparison (practically, just describe the results in the table with plain text) instead of giving the high level analysis of success and fail cases along with some conclusion. It’s hard to read and this resembles how AI typically writes a text describing experiment results, please, rework this part and make it less low level and more human readable.
Overall, the paper doesn’t introduce any novel ideas except synthetic reasoning traces in the chemical LLM domain which is also not that novel in the general LLM domain. The experiment part is strong enough for predictive tasks, but does not provide evidence of the applicability and usefulness of this approach for other downstream tasks and does not provide any interesting analysis or conclusion.
The paper in current state is better suited for a workshop, not the main conference.
a.Galactica: A Large Language Model for Science
b.MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery
c. FGBench: A Dataset and Benchmark for Molecular Property Reasoning at Functional Group-Level in Large Language Models
Critical strengths
As stated in Summary, the only interesting and moderately novel part of the paper is synthetic reasoning generation. I would recommend authors to reprioritize the paper and make the main focus on this contribution and also show the applicability of this reasoning generation for other downstream tasks.
Critical weaknesses
Except synthetic reasoning traces, other contributions are not novel and the applicability is strongly limited by predictive tasks.
As a critical weakness I see the unjustified use of molecular embeddings instead of just SMILES representation of the molecule. The use of molecular embeddings complicates both training and inference of LLM while not providing any reasonable advantages - even if to talk about generation speed, 20-30 tokens for SMILES prefill/generation are not that slow in comparison to 1 molecular embedding. Still Ablation studies [Table 2] do not show any significant performance difference between SMILES and molecular embeddings. The use of molecular embeddings makes BOLEK model adaptation for molecular generation task more difficult since it’s just molecular encoder without decoder that can reconstruct the molecule back from the embedding. Overall, I recommend authors to get rid of these redundant molecular embeddings and molecular alignment parts and focus on synthetic reasoning and expansion of the model to other downstream tasks.
Questions for authors
The main question that can drastically change my assessment is:
- Could authors provide the use case that justify the necessity of molecular embeddings and molecular alignment contribution of the paper?
- It’s not clear how the synthetic reasoning generation is transferable to other downstream tasks. Could it be used for molecular generation/retrosynthesis. Since this question requires a wide set of experiments and the rebuttal stage is limited, I ask authors to provide some results on just one downstream task different from property/change of the property prediction.
Response to Reviewer g77H
Thank you for a concrete review — your questions name what would change the assessment. We take your recommendation on the experiments section, and we correct one premise that is doing a lot of work in your assessment.
The alignment stage is not projector pretraining.
You describe stage one as projector pretraining and ask whether the projector could instead be trained alongside the LLM backbone. Alignment trains the full backbone together with the projector, under the same next-token objective used downstream — Section 3 states this, and the configuration unfreezes all transformer layers, the token embeddings and the final norm alongside the molecular projection.
The proposed alternative is measured in Table 2, as the single-joint-stage and the no-alignment rows. The configuration you took alignment to be — projector only, with the language model frozen — is there too, and loses 9 to 14 p.p. in every column.
The single-stage row is competitive. We report the two-stage recipe because one alignment checkpoint can be reused across downstream ablations, which is what made the rest of Table 2 affordable.
Q1 — What justifies the molecular embedding.
The point of the embedding interface is that the model can be aligned to any fixed-size molecular representation and made to use its latent features, and that even the simplest of them, a Morgan fingerprint, outperforms SMILES. Table 2 swaps in two learned encoders without changing anything else, which is what makes the representation an experimental variable rather than an architectural commitment. Our later experiments (performed after this submission) show that using a structural embedding outperforms both SMILES and the Morgan fingerprint.
Generation speed is not the argument, and we agree it would be a weak one — 20 to 30 tokens of SMILES prefill is not slow.
Three of the six cells separate, all in the fingerprint's favour, and SMILES separates in its own favour nowhere. You are right that the other three show no significant difference, ADMET among them. Both held-out cells are among those that do separate, which is where we would put the weight: the representations come apart on endpoints the model was never trained on.
Q2 — Transfer to generation or retrosynthesis.
Our reasoning pipeline is built around single-molecule property prediction — it decomposes one molecule, computes task-relevant descriptors, and retrieves assay context for one endpoint — and nothing in it produces or scores a synthetic route or a target molecule.
The current architecture would need a representation from which the SMILES can be reconstructed, with an embedding-to-SMILES reconstruction task already present in the alignment phase. An alternative is to give the model both the feature-rich vector and the SMILES itself, either as raw text or in a chemistry-specific tokenization such as SMIRK.
The one piece of evidence we can offer is a change of answer type rather than task family: three TDC regression endpoints, none trained on (Appendix C.2). BoLek is the only model that ranks plasma protein binding at all, 0.340 Spearman against 0.001 for Qwen3 and −0.001 for TxGemma-9B-Chat. We are aware that this does not directly answer the question, as it is a different property-prediction format rather than an entirely new task.
On the experiments section writing.
We accept this criticism, and you are right that it reads that way: parts of the paper were edited with AI tools for readability, as our AI Use Statement records. Being obviously familiar with the high-level claims ourselves, we failed to anticipate that a reader coming to the section fresh would find the low-level analysis far less informative than we did. We are rewriting it around where the model succeeds and fails — where chain-of-thought helps and why and where the supervised specialists stay ahead.