acceptodds
Under review as a conference paper at ICLR 2027

ChemoGrad: Improving Chemical Reasoning through Test-Time Latent Optimization

Abstract

The latent space of chemical LLMs encodes the underlying characteristics and principles of chemical structures, making it possible to generate chemical compounds by reasoning directly within this space. Yet existing latent-reasoning methods for chemical generation lack a clear direction: the latent state is updated by re-forwarding it through the model, without a reward signal or verifier to steer the search. We propose ChemoGrad, which optimizes molecules by a reward-guided search over a frozen model's latent states: it scores each candidate with a property evaluator, nudges the hidden states to perturb the emitted prefix, then re-decodes and keeps a candidate only when the evaluator confirms an improvement. A likelihood-sharpening step proposes candidates around the model's own trace, a verifiable reward selects which to pursue, and a task-oriented initialization points the search in the right chemical direction — shifting task adaptation from training-time parameter learning to inference-time search. On ChemCoTBench, ChemoGrad consistently outperforms the state-of-the-art baseline across all six molecule-optimization tasks, gaining +51% in mean property improvement and +175% on JNK3 activity, with no additional training; it also transfers to editing, reaction prediction, and captioning.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
chemistry large language models, test-time reasoning, latent optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_h5nJOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

ChemoGrad is a training-free, per-instance test-time procedure for a frozen chemistry LLM (LatentChem's Qwen3-8B checkpoint, CoT mode). Last-layer hidden states of the first ρ=50% of generated tokens are treated as free variables and updated by a REINFORCE step (running-mean baseline, Adam, lr 0.05). The reward is an external oracle: an RDKit/TDC property or functional-group rule, or, for reaction/caption, similarity to the ground truth. States are decoded through lm_head, the suffix is regenerated, and a candidate is accepted only if the reward improves. Add-ons are hand-written task hints appended to the prompt and a learning-rate-doubling "deepening". On ChemCoTBench molecule optimization, mean Δ rises from 0.60 (published LatentChem) to 0.90 (+51%) with 94–100% success. Gains on editing, reaction prediction and captioning are also reported.

Critical strengths
  1. A simple, cheap, practical idea (oracle-in-the-loop search on a frozen model). Verifier types are labelled (avail vs gold), and limitations are candidly disclosed: init-to-final pairs favour the final answer, the add-ons cannot be separated, and the BoN comparison holds only at a fixed pool.
  2. The results hint that single-pass outputs of a four-stage RL-trained model leave headroom when oracle feedback is available at inference. That is useful if shown under matched budgets.
Critical weaknesses
  1. The headline comparison is neither budget- nor information-matched, and the mechanism is not isolated.
    • The baseline is a single published pass under another paper's protocol. ChemoGrad queries the benchmark's scoring function for up to K=30 iterations (candidates per step unspecified) and up to three deepening rounds. It possibly also uses a best-of-8 pool (App. C) and hand-written hints (e.g. "add halogen to an aromatic ring" for LogP) that the baseline never sees. Oracle calls, tokens and wall-clock are not reported.
    • Table 3 bundles "enlarged budget, task-prior init, deepened update" into one step (logP 1.04→1.76; solubility 0.90→2.19), and the text concedes no attribution is possible. The base loop alone (1.04/0.90) is below the quoted baseline on solubility (1.17). App. C's paired-difference CIs against the same-protocol greedy CoT baseline imply (if on the Table-1 statistic) a baseline of ≈1.3–1.6 (logP) and ≈1.3–1.7 (solubility). That would put the base loop below it on both.
    • For solubility, Fig. 2 init (0.92), Table 3 base loop (0.90) and the quoted baseline (1.17) disagree. A monotone accept rule cannot end below its own start.
    • The only budget-aware test (BoN-8 vs loop) uses a different model (vanilla Qwen3-8B-Instruct), one task (logP), and gives the loop more oracle calls.
    • Sec. 2.2 itself calls the update a "diversification signal" with improvement enforced by accept/reject, i.e. stochastic proposal plus oracle rejection. Hidden states never re-enter the network; they only supply logits for the first U tokens. Fig. 1's hill-climbing picture is not what Eqs. 4–5 implement (∇z r is absent).
    • Needed, on one checkpoint with identical hints and equal oracle calls: BoN-N, noise-only hidden-state perturbation with the same accept rule, token-space prefix resampling, and a standard oracle-guided optimizer.
    • Significance claims are misattributed. Sec. 4.1 says tests confirm gains over LatentChem, but the App. C tests are against a local greedy baseline (no per-sample LatentChem data exist). Several win-rate/sign tests are init-vs-final pairs that the authors call "guaranteed to favor the final answer".
  2. The "transfer" results and success rates rest on reference leakage and metric optimisation.
    • Reaction/caption rewards are Tanimoto to the ground-truth product and METEOR to the reference caption, which are test labels. They still appear in the abstract, in Table 1 (unflagged METEOR columns) and in Sec. 4.1, despite the "upper bound" caveat.
    • The reaction "0.00→0.85" starts from a local baseline of exactly 0.00 (published LatentChem: 0.52). That suggests a parsing/format failure.
    • 94–100% success on classifier oracles is expected for any search that accepts only score increases. There is no held-out oracle and no validity, novelty, similarity or SA analysis, so oracle exploitation is not excluded.
    • Fig. 3 rows 1–2 appear to replace large parts of the source (e.g. the azepane tail by a bromoaniline), contradicting "small and local" edits. The Sec. 4.3 case descriptions (benzothiazole urea, N-methylpiperidine) do not match the drawn structures (phenothiazine amide, seven-membered ring).
    • Additional: the delta to LatentSeek/MILR (hidden-state policy gradient already exists) is not articulated beyond oracle, hints and lr doubling. "+51%" is the ratio of mean Δ (0.305/0.598), not a "plain average of per-task relative gains" (≈91%). Fig. 2 labels the method "CLO" and uses unlabelled per-panel scales. Table 9's "−sig*" convention contradicts the text for gsk3β. The TDC author list is wrong, and code identifiers appear in prose (RESTART_TARGET).
Questions for authors
  1. Report oracle calls, tokens and wall-clock per molecule per configuration. Is the best-of-8 pool part of the Table 1 runs?

  2. On the same checkpoint and pipeline, report: the baseline with the same hint, BoN-N at matched calls, noise-only perturbation with the same accept rule, and a classical optimizer at the same budget.

  3. What is new relative to LatentSeek?

  4. Explain the solubility inconsistency and why the base loop falls below the greedy baseline.

  5. Why is the local reaction baseline exactly 0.00? Report format-failure rates.

  6. (suggestion) Move gold-verifier results out of the abstract and Table 1, evaluate on held-out oracles, and report validity/novelty/SA.

1 comment
AuthorsOpenReview ↗

Response to Reviewer h5nJ: New Controlled Experiments and Method Clarifications

Thank you for the detailed critique. We now have a fixed-pool, three-seed diagnostic addressing the proposal mechanism, plus structural-quality measurements and the legacy audit.

1. Mechanism, controls, and budget

All arms use frozen Qwen3-8B-Base / LatentChem stage-4 resources, identical hints and an identical eight-candidate pool per source-seed. Latent and random-latent share saved greedy hidden states, not pool-winner states. Each task has 100 sources, seeds 1–3, and 24 additional proposals: exactly 32 candidates within 16,384 generated tokens. Bonus/restart and success/stall early stopping are disabled; this is a diagnostic, not the complete pipeline.

Latent adds raw mean property gains over the pool of +0.4937/+0.3915 on Crippen LogP/ESOL solubility and exceeds it in 173/300 and 150/300 repeated source-seed records. Relative to random latent (noise_std=0.05), paired differences are +0.3794 (95% CI [0.2974,0.4614]) and +0.2852 ([0.1987,0.3792]); both Holm-adjusted p=0.0000399996. We average three seed differences within source before source-level inference (n=100/task; one four-test Holm family; unadjusted CIs).

The requested no-update sampling-and-selection control is now included. Its pool gains are +0.6538/+0.6932, higher than latent's; the data establish an advantage over the tested random-latent control, not over continued sampling. The full-pipeline F/F0 replacement test, new three-seed token-prefix control, and classical optimizer are not included.

Actual LogP/solubility costs are:

  • Latent: 28.77/28.59 property calls; 11,957.47/11,612.47 tokens; 415.71/388.18 continuation seconds.
  • Sampling: 29.54/29.45 calls; 10,505.83/10,997.08 tokens; 638.11/680.10 seconds.
  • Random latent: 30.11/30.04 calls; 11,125.84/11,138.33 tokens; 349.80/344.78 seconds.

Calls/tokens include the shared pool; continuation time excludes its generation, queueing, and model preparation. Shared ceilings do not equal matched actual compute. Resource-aligned descriptive comparisons and coverage accompany the evidence table; shorter latent time does not establish equal-time quality superiority. The eight-start pool is part of the full ChemoGrad pipeline and is counted.

2. Relationship to LatentSeek and implementation

LatentSeek is the direct predecessor of the reward-weighted update. Our contribution is the chemistry-specific search procedure, molecular-model interface, validity-aware verification, and its empirical analysis. We do not claim a new generic estimator or originality for shared prefix/suffix decoding and best-so-far selection.

We will correct the method: the code uses reward-weighted argmax-token log-probabilities and Adam, without the described running-mean baseline. Non-improving outputs do not replace the returned best candidate, but latent/Adam states are not rolled back. The figure must not imply direct differentiation of the chemical reward.

3. Structural quality and verifier scope

All diagnostic final outputs are valid because a valid pool best is retained; proposal validity is 89.90%/89.34% for latent. Its unique final counts are 286/300 and 269/300, not 100%. Latent/source Morgan similarity is 0.5796/0.5174; SA is 2.6669/2.6268. Sampling gives 0.5714/0.4790 and 2.6773/2.6104. These are descriptive structural metrics, not proof of local latent changes, synthesis, or independent-oracle robustness. Three seeds repeat the same 100 sources; held-out-oracle, training-set novelty, and experimental activity remain untested. Reference-based reaction/caption scores will be moved out of the abstract and headline table.

4. Legacy consistency and presentation

The solubility values 1.17 (published Stage 1+2+4), 1.0243 (local stage-4 inference), and about 0.90 (separate base loop) are different protocols, not a paired trajectory. The base-loop initialization was parser-invalid. We will remove the unrecovered Figure 2 point near 0.92 and separate legacy penalized LogP from new Crippen LogP.

Reaction initial 0.00 belongs to Cplus256 (0/100 parsed); B512 has 30/100 parsed and 29/100 valid SMILES. We cannot infer truncation without original stop logs. We will correct Morgan-only versus three-fingerprint metric labels and remove the 0.00→0.85 headline.

We will correct the +51% definition, separate init-to-final diagnostics from competitor tests, remove unsupported legacy seed claims, and fix structure names, “small/local” wording, figure labels, units, citation metadata, and code identifiers. The new three-seed evidence does not retroactively validate the legacy runs.

Reviewer_NwxzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propose ChemGrad, a test-time method that updates prefix hidden states to keep generating better molecules. Specifically, at each iteration, ChemGrad resample prefix from the hidden states, and continue the autoregressive generation from the resample prefix, if the resulting generation is better ChemGrad will keep this generation, and update Z via REINFORCE, otherwise keep the old Z. The empirical results show very strong performance again the backbone model.

Critical strengths

The paper demonstrates very strong empirical gain over base model. The paper also did thorough study on cases where verifier is available and is not. This provides a more clear understanding of the capability of ChemGrad.

Critical weaknesses

This paper appears to be an application of LatentSeek [1], the authors actually cited this paper, but did not discuss the correlation. I am not seeing technical/ method novelty over it. There are new components like accept/reject gates, and scheduling, but are really just engineering effort or adaptation specific to this problem, or requiring a veriable reward, which limit the applicable scenario of the work even within the domain of chemistry.

[1] Li, Hengli, et al. "Seek in the dark: Reasoning via test-time instance-level policy gradient in latent space." arXiv preprint arXiv:2505.13308 (2025).

Questions for authors

1, How would a naive latentseek behave?

2, Why did you choose 0.5 as the prefix cut-off?

3, How does the method compare to other simplier alternatives? Like you just do accept and reject gate without update?

Essentially I am trying to ask, since this more of an application of a method to this field, what is the necessity for such application or clear advantage our other approaches?

Reviewer_DbebOpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ChemoGrad optimizes the hidden states of a LatentChem, a frozen chemistry language model, at test time. A policy-gradient update proposes new generations, and an external evaluator determines which candidates to keep. The authors report gains across six molecular property-optimization tasks and additional results on editing, reaction prediction, and captioning. The reaction and captioning experiments use gold-reference rewards and are presented as idealized demonstrations.

Critical strengths
  1. I like that the objective can change between queries without retraining the model. Using an external evaluator also accommodates nondifferentiable chemical scores.
  2. The gains across all six property tasks are encouraging, including substantial improvements in predicted activity. I also appreciate the repeated runs on logP and solubility.
  3. I appreciate the distinction between rewards available at inference time and gold-reference evaluators. This helps put the reaction and captioning experiments in context.
Critical weaknesses
  1. I would like a comparison with sampling at the same total search budget. The best-of-eight experiment is useful, but ChemoGrad then performs additional generation and oracle calls. It also uses a different checkpoint from the main experiments. Comparing against best-of-N and random latent perturbations on the chemistry checkpoint, with matched oracle calls and generated tokens, would help establish what the gradient updates add.
  2. The relationship to LatentSeek, which proposes Test-Time Instance-Level Adaptation (TTIA) should be discussed. Chemical property oracles provide a useful setting for this idea, but I would like a more direct explanation of the algorithmic differences.
Questions for authors
  1. At matched oracle and token budgets, how does ChemoGrad compare with best-of-N and random latent perturbations?
  2. Which changes relative to LatentSeek are important for the chemical tasks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.