acceptodds
Under review as a conference paper at ICLR 2027

Bolek: A Multimodal Language Model for Molecular Reasoning

Abstract

AI models and platforms for molecular science underpin high-stakes applications in drug discovery, yet the systems delivering them are largely opaque: they expose either a score and a binary answer with no rationale, or fluent prose rarely anchored in molecular structure. We introduce Bolek, a compact multimodal language model that injects a molecular embedding into an instruction-tuned decoder to predict molecular properties and explain them in auditable terms. Bolek improves over its Qwen3 base on every task group of a 30-endpoint TDC panel and outperforms the chemistry-specialist TxGemma-9B-Chat on 11 of 17 ADMET endpoints and on four of the five endpoints held out from its training. Together, these results suggest that targeted modality injection paired with reasoning supervision tied to verifiable features can match domain-specialist systems at a fraction of the parameter count while producing auditable explanations for downstream decisions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
molecular language models, multimodal learning, molecular reasoning, drug discovery, grounded reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_g77HOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose BOLEK - a language model for molecular properties prediction along with a suite of molecular alignment tasks and the synthetic reasoning preparation routing aimed at pretraining this language model to provide a good start for downstream molecular prediction tasks (the subset of TDC tasks in this case).

From my understanding the whole approach is transferable to any decoder-only LLM that supports taking as an input user custom embeddings. In this case, the authors pass the molecular embedding retrieved after 2 layer MLP feeded with molecular Morgan description. From my point of view, the requirement to take user custom embeddings in input is not that restrictive since many modern LLMs support this, but not well justified and even if briefly but make the training and inference more engineerically difficult than just plain text-only LLM.

The molecular alignment tasks resemble the predictive tasks on synthetic molecular descriptors and substructure detecting routine. This part is not novel and many approaches use the molecular encoder and embedding projector pretraining, for example to learn meaningful 3D structure or molecular graph input embedding. Still in this paper this step looks unnecessary and redundant because the embedding MLP learns to basically predict extended molecular fingerprints based on some base fingerprint. Also ablation study doesn’t provide any evidence that this part is essential - can this projector pretraining be omitted and the projector be trained along with LLM backbone on synthetic reasoning + downstream tasks instead of this separate step.

The moderately interesting part of the paper is synthetic reasoning traces generation. While the synthetic reasoning generation is not novel for general LLM training and the whole pipeline resembles general synthetic reasoning generation with chemical domain adaptation, this topic is underrepresented in the chemical LLM area. The main drawback of the proposed synthetic reasoning is that it’s designed mainly for predictive tasks and the transferability of the other chemical applications is not clear.

The experiment part is somewhat normal and supports the quality assessment of the model on predictive tasks and also includes the ablation. Still, while predictive tasks are important for the chemical domain, the only predictive tasks experiments are not enough for the current state of the chemical LLM field. Chemical LLMs [a, b] cover different applications like retrosynthesis, molecular optimization and molecular generation given a set of conditions.I recommend to think about adaptation of FGBench [c] as an additional set of predictive tasks on property change prediction given the molecular pairs.

The text of the paper is written in an easy-to-follow and clear way. The only problem is the experiments section, where authors introduce low-level metrics values comparison (practically, just describe the results in the table with plain text) instead of giving the high level analysis of success and fail cases along with some conclusion. It’s hard to read and this resembles how AI typically writes a text describing experiment results, please, rework this part and make it less low level and more human readable.

Overall, the paper doesn’t introduce any novel ideas except synthetic reasoning traces in the chemical LLM domain which is also not that novel in the general LLM domain. The experiment part is strong enough for predictive tasks, but does not provide evidence of the applicability and usefulness of this approach for other downstream tasks and does not provide any interesting analysis or conclusion.

The paper in current state is better suited for a workshop, not the main conference.

a.Galactica: A Large Language Model for Science

b.MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery

c. FGBench: A Dataset and Benchmark for Molecular Property Reasoning at Functional Group-Level in Large Language Models

Critical strengths

As stated in Summary, the only interesting and moderately novel part of the paper is synthetic reasoning generation. I would recommend authors to reprioritize the paper and make the main focus on this contribution and also show the applicability of this reasoning generation for other downstream tasks.

Critical weaknesses

Except synthetic reasoning traces, other contributions are not novel and the applicability is strongly limited by predictive tasks.

As a critical weakness I see the unjustified use of molecular embeddings instead of just SMILES representation of the molecule. The use of molecular embeddings complicates both training and inference of LLM while not providing any reasonable advantages - even if to talk about generation speed, 20-30 tokens for SMILES prefill/generation are not that slow in comparison to 1 molecular embedding. Still Ablation studies [Table 2] do not show any significant performance difference between SMILES and molecular embeddings. The use of molecular embeddings makes BOLEK model adaptation for molecular generation task more difficult since it’s just molecular encoder without decoder that can reconstruct the molecule back from the embedding. Overall, I recommend authors to get rid of these redundant molecular embeddings and molecular alignment parts and focus on synthetic reasoning and expansion of the model to other downstream tasks.

Questions for authors

The main question that can drastically change my assessment is:

  1. Could authors provide the use case that justify the necessity of molecular embeddings and molecular alignment contribution of the paper?
  2. It’s not clear how the synthetic reasoning generation is transferable to other downstream tasks. Could it be used for molecular generation/retrosynthesis. Since this question requires a wide set of experiments and the rebuttal stage is limited, I ask authors to provide some results on just one downstream task different from property/change of the property prediction.
1 comment
AuthorsOpenReview ↗

Response to Reviewer g77H

Thank you for a concrete review — your questions name what would change the assessment. We take your recommendation on the experiments section, and we correct one premise that is doing a lot of work in your assessment.

The alignment stage is not projector pretraining.

You describe stage one as projector pretraining and ask whether the projector could instead be trained alongside the LLM backbone. Alignment trains the full backbone together with the projector, under the same next-token objective used downstream — Section 3 states this, and the configuration unfreezes all transformer layers, the token embeddings and the final norm alongside the molecular projection.

The proposed alternative is measured in Table 2, as the single-joint-stage and the no-alignment rows. The configuration you took alignment to be — projector only, with the language model frozen — is there too, and loses 9 to 14 p.p. in every column.

The single-stage row is competitive. We report the two-stage recipe because one alignment checkpoint can be reused across downstream ablations, which is what made the rest of Table 2 affordable.

Q1 — What justifies the molecular embedding.

The point of the embedding interface is that the model can be aligned to any fixed-size molecular representation and made to use its latent features, and that even the simplest of them, a Morgan fingerprint, outperforms SMILES. Table 2 swaps in two learned encoders without changing anything else, which is what makes the representation an experimental variable rather than an architectural commitment. Our later experiments (performed after this submission) show that using a structural embedding outperforms both SMILES and the Morgan fingerprint.

Generation speed is not the argument, and we agree it would be a weak one — 20 to 30 tokens of SMILES prefill is not slow.

CellMorganSMILESStatistically significant
ADMET CoT0.7600.746no
ADMET YN0.7430.749no
Tox21 CoT0.7620.742yes
Tox21 YN0.7470.760no
Held-out CoT0.7990.777yes
Held-out YN0.7700.736yes

Three of the six cells separate, all in the fingerprint's favour, and SMILES separates in its own favour nowhere. You are right that the other three show no significant difference, ADMET among them. Both held-out cells are among those that do separate, which is where we would put the weight: the representations come apart on endpoints the model was never trained on.

Q2 — Transfer to generation or retrosynthesis.

Our reasoning pipeline is built around single-molecule property prediction — it decomposes one molecule, computes task-relevant descriptors, and retrieves assay context for one endpoint — and nothing in it produces or scores a synthetic route or a target molecule.

The current architecture would need a representation from which the SMILES can be reconstructed, with an embedding-to-SMILES reconstruction task already present in the alignment phase. An alternative is to give the model both the feature-rich vector and the SMILES itself, either as raw text or in a chemistry-specific tokenization such as SMIRK.

The one piece of evidence we can offer is a change of answer type rather than task family: three TDC regression endpoints, none trained on (Appendix C.2). BoLek is the only model that ranks plasma protein binding at all, 0.340 Spearman against 0.001 for Qwen3 and −0.001 for TxGemma-9B-Chat. We are aware that this does not directly answer the question, as it is a different property-prediction format rather than an entirely new task.

On the experiments section writing.

We accept this criticism, and you are right that it reads that way: parts of the paper were edited with AI tools for readability, as our AI Use Statement records. Being obviously familiar with the high-level claims ourselves, we failed to anticipate that a reader coming to the section fresh would find the low-level analysis far less informative than we did. We are rewriting it around where the model succeeds and fails — where chain-of-thought helps and why and where the supervised specialists stay ahead.

Reviewer_cW7tOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BOLEK, a multimodal language model for molecular property prediction and language explanation. The model uses MLPs to map a Morgan fingerprint to a molecular token embedding. Training proceeds in two stages: an initial alignment stage using tasks such as substructure recognition, descriptor prediction, and molecular description, followed by downstream fine-tuning that combines direct label supervision with synthetic CoT reasoning for molecular property prediction. The model is evaluated on a set of downstream binary classification tasks.

Critical strengths

Some downstream task scenarios are meaningful.

Critical weaknesses
  1. The method requires a customized inference pipeline to inject molecular embeddings into Qwen’s input sequence, preventing direct deployment through standard chat APIs. This introduces additional integration requirements and may limit compatibility with existing deployment infrastructure.
  2. The independent benefit of feature alignment, presented as a core contribution, lacks direct and sufficient empirical support. Moreover, the reliability of the model’s CoT reasoning is constrained by the teacher model, whose reasoning is not guaranteed to be reliable. The paper also appears to lack explicit measures to address this limitation.
  3. In practical applications, this model configuration requires separate two-stage sft for each molecular property, resulting in limited generalizability and substantial computational overhead.
Questions for authors

See the weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer cW7t

Thank you for your review.

W1 — Custom inference pipeline.

BoLek needs no bespoke inference stack. It is registered as a native vLLM multimodal model through vLLM's public vllm.general_plugins entry point, so the molecular embedding enters the sequence exactly as an image does for a vision-language model. The supplementary ships that plugin, the checkpoint, and two runnable entry points against it: predict.py for free-form questions and evaluate_task.py for a full endpoint evaluation.

W2 — Independent benefit of alignment; teacher reliability.

The alignment block of Table 2 isolates the stage. Under yes/no-only downstream supervision, where CoT traces cannot substitute for it, the aligned model reaches 0.716 on ADMET against 0.683 without alignment, beyond the 95% paired interval. Training the projector alone with the language model frozen collapses to 0.654 / 0.650 / 0.660, so the gain is not simply that a projector has been fitted — the language model itself has to adapt to the molecular input.

On teacher reliability we agree, and state it as a limitation. Our CoT supervision keeps traces only when the teacher reaches the ground-truth label: rejection sampling against the answer, not verification of the reasoning, and nothing checks whether an intermediate claim about the molecule is true. Appendix D.4 shows why it matters — the answer follows the rationale it is given. Rewards that check the rationale rather than the final label are the natural next step, and our GRPO arm, which optimizes the final label alone, trades one answer format for the other without improving either.

W3 — Per-property training.

BoLek is not trained per property: it is one model from one two-stage run, trained jointly on 25 endpoints and evaluated on 30. The five held-out endpoints are supplied only as an assay description at inference, and BoLek reaches zero-shot 0.799 mean ROC AUC on them in CoT — 6.1 p.p. ahead of TxGemma-9B-Chat, which was trained on those endpoints, and not beaten by any supervised specialist, each of which was refitted on those endpoints' own train splits. Table 2 shows supplying the assay description at inference is worth 2.6 p.p. on held-out yes/no and nothing on the trained groups, which is the signature of the model reading it rather than recalling the endpoint.

Thanks for the authors' responses, which have addressed my concerns and corrected my misunderstandings. I have raised my score.

Reviewer_4ZqkOpenReview ↗
Rating
2(−1)
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BOLEK is a molecular language model that projects a 2048-bit, radius-2 Morgan fingerprint into one token of Qwen3-4B-Instruct. It first learns molecular features through alignment tasks covering substructures, descriptors, naming, and descriptions. It then receives supervised training on direct answers and synthetic reasoning for 25 TDC classification endpoints. Teacher rationales use molecular decomposition, task-relevant descriptors, and literature-derived assay context; five additional endpoints are held out from downstream training. On the 30-endpoint panel, BOLEK achieves mean CoT ROC AUCs of 0.760, 0.762, and 0.799 for ADMET, Tox21, and held-out endpoints, respectively. It improves over the Qwen3 baseline in all three groups and exceeds TxGemma-9B-Chat on 11 of 17 ADMET endpoints and four of five held-out endpoints, while remaining below TxGemma-9B-Predict in group means. Ablations examine representations, supervision, training schedules, and assay context. Further analyses explore regression transfer, alignment quality, and sensitivity to substituted rationales.

Critical strengths

BOLEK improves over its Qwen3 backbone across the three evaluated task groups and shows encouraging transfer to endpoints excluded from downstream training (Table 1). These results suggest that a simple fingerprint-based interface, combined with molecular alignment and reasoning supervision, can be useful for molecular property prediction. The ablations in Table 2 help clarify the contributions of several training choices and indicate that the benefit of generating a rationale varies across endpoints.

Critical weaknesses

The factual accuracy of explanations generated by the final model is not directly evaluated, as Table 16 reports feature recovery at the alignment checkpoint. The counterfactual test in Appendix D.4 demonstrates sensitivity to supplied rationales, but does not establish whether the model’s own explanations are chemically reliable. Since the no-alignment variant receives fewer training steps, the observed difference may partly reflect the additional training budget rather than the alignment objective alone. In addition, retaining only label-matching rationales and using label-conditioned fallback generation for Tox21 may introduce selection bias into the CoT supervision, whose extent is not quantified. Differences in rollout counts, prompting, and scoring protocols across BOLEK, GPT-6-Sol, and TxGemma-Chat complicate interpretation of the reported performance margins. The smaller parameter count does not by itself establish an inference-efficiency advantage, particularly under the 50-rollout evaluation protocol. The absence of explicit overlap statistics across alignment data, downstream training data, and test sets also leaves uncertainty about the extent to which held-out endpoint performance reflects generalization to previously unseen molecules.

Questions for authors
  1. Have you assessed the factual accuracy of the final model’s generated explanations, particularly their structural and numerical claims, beyond the alignment-checkpoint metrics in Table 16?

  2. How should the near-complete loss of accuracy with opposite-label donor rationales in Appendix D.4 be interpreted in relation to molecular grounding, given that the molecular input remains unchanged?

  3. To what extent do held-out test molecules overlap with the alignment pool or other endpoints’ training data, and does the transfer performance persist on molecules absent from both?

  4. Do the available experiments distinguish the benefit of feature-based alignment from that of additional training or caption-based supervision under a comparable training budget?

  5. How much of the CoT training data is retained by label matching or obtained through label-conditioned Tox21 fallback generation, and what evidence is available about the effect of these choices on the results?

  6. Does BOLEK retain its advantage over GPT-6-Sol when both use five rollouts on the same test molecules with the final model configuration?

  7. How sensitive is the comparison with TxGemma-Chat to its prompting setup, particularly the difference between the direct-question evaluation and the published 10-shot protocol?

  8. What inference latency and token usage are required by the 50-rollout protocol, and how do these compare with the evaluated baselines?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 4Zqk

Thank you for your detailed review. Several of your questions needed new analysis of the stored predictions rather than an argument.

Q1 — Factual accuracy of explanations

We have not assessed this at the fine-tuning checkpoint; Table 16 reports feature recovery at the alignment checkpoint and does not stand in for it. The claims differ in how checkable they are. Numerical ones are exact: a stated descriptor either matches RDKit or it does not. Structural ones are easy to verify once extracted, but extracting them reliably from free text is the harder half of the problem. Claims about mechanism or assay behaviour cannot be checked this way at all, and scoring them with another model would be bounded by the judge.

Q2 — Counterfactual rationales

This result demonstrates that Bolek’s final answer actually does rely on the reasoning. What we feared was that Bolek’s reasoning would just imitate the training examples of chain-of-thought, and give a final answer that would only depend on the input molecule. Yet, the counterfactual experiment confirmed that BoLek indeed treats the rationale as the primary source of truth for the answer. With molecule and question fixed and only the rationale swapped, the answer follows the donor in 93.8% of single-donor cases, and accuracy tracks the donor's label, 99.8% vs 1.8%.

Q3 — Overlap with alignment and training data

New analysis. Overlap between the held-out test sets, the alignment pool and every trained endpoint's training split is uneven: 43% for Pgp Broccatelli, 16% for PAMPA NCATS and 10% for M1 antagonist, but 95% for CYP2D6 Veith and 99% for Tox21 PPARγ, which are drawn from the same screening libraries as their trained siblings. This follows from the benchmark rather than our splitting: we reuse the TDC splits so the TxGemma comparison is like for like, and molecules recur across endpoints.

The Veith panel is one compound library assayed against several CYPs, and Tox21 is one plate assayed many ways, so those two endpoints share nearly all their molecules with trained siblings. The overlap follows from the benchmark rather than from our splitting: we reuse the TDC splits so the comparison with TxGemma is like for like, and molecules recur across TDC endpoints. A corpus-wide split would remove the overlap and the comparison with it.

Molecule overlap is not label overlap. The alignment stage carries no assay labels at all, and no trained endpoint supplies a held-out label. Using a sibling's training label to predict the held-out test label gives 0.645 on CYP2D6 Veith, 0.643 on Tox21 PPARγ and 0.460 (anti-predictive) on M1 antagonist, against BoLek's 0.78–0.81 on those same molecules, so sibling leakage cannot account for the result. On molecules absent from both stages, BoLek's CoT mean is unchanged at 0.806. We will add this audit to the paper.

Q4 — Alignment vs training budget

You are right about the asymmetry: every variant in that block gets the same 1,750 downstream steps, and alignment adds 8,250 on top. We have not run the matched control. In its current state, adding further steps to the fine-tuning phase leads to overfitting and degrades the results, so a no-alignment arm given the full 10,000 downstream steps would mostly measure that rather than isolate the alignment objective.

Q5 — CoT retention and Tox21 fallback

New analysis. Label matching retained 84.9% of teacher attempts panel-wide, and 73.6% on the 13 ADMET endpoints where the figure is exactly recoverable. For Tox21, blind generation retained 58.7% of attempts; the label-conditioned fallback supplies 48.2% of Tox21 CoT rows, 37.8% of its molecules, and 13.7% of the CoT corpus. It was used for Tox21 and nowhere else.

Q6 — Matched rollout budget

This is a fair point. We were limited to five rollouts for GPT-6-Sol on cost grounds, but we should have held the comparison scenario the same. BoLek's metrics at 5 rollouts are in Table 12; head-to-head against GPT-6-Sol five of the six wins persist, some with reduced magnitude — ADMET YN +5.2, Tox21 CoT +12.5, Tox21 YN +14.7, held-out CoT +3.5 and held-out YN +4.5 p.p. — while the ADMET chain-of-thought scores become equal, +3.0 falling to −0.1.

Q7 — TxGemma-Chat prompting

The analysis is described in Appendix C.5: across the 15 tasks TxGemma reports at task level, our direct-question evaluation runs 2.8 p.p. below the published 10-shot numbers, so our TxGemma-9B-Chat column should be read as a lower bound under that model's own prompting.

Q8 — Inference cost of the 50-rollout protocol

The 50-rollout protocol is not how we intend BoLek to be run. As stated in the manuscript, it estimates the probability of an answer conditioned on the question alone, marginalising over the generated reasoning rather than conditioning on one sampled rationale. We used 50 rollouts only to compare AUC fairly against the other models, not as a deployment setting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.