acceptodds
Under review as a conference paper at ICLR 2027

Latent JEPA: Abstract Future Prediction for Latent Reasoning in Chemistry

21 views

Abstract

Large language models offer a promising foundation for chemical reasoning, bringing together chemical knowledge and multistep problem solving. Chemical intuition can provide an initial sense of plausible outcomes before the details of a solution are fully worked out. Inspired by how such expectations complement explicit analysis, we study how continuous latent thoughts can be trained to anticipate informative aspects of future solutions without verbalizing every intermediate step. We introduce Latent JEPA, a framework that combines autoregressive learning with joint-embedding prediction of one or more future views. For chemical reasoning, we develop textual and molecular prediction objectives that connect latent thoughts to both subsequent reasoning and molecular outcomes. Experiments on ChemCoTBench show gains in molecular optimization and on several editing and reaction metrics. Representation analyses show that future prediction makes latent thoughts more informative about molecular outcomes and strengthens their correspondence with chemical structure. These findings support abstract future prediction as a learning principle for connecting continuous latent reasoning with scientific outcomes.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for chemistry, Joint-Embedding Predictive Architecture

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_d4TDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Latent-JEPA, a framework that augments continuous latent reasoning with prediction of representations of future reasoning and outcomes. Its primary chemical instantiation adds two auxiliary objectives to Coconut-Chem: the final latent thought predicts horizon-conditioned representations of the remaining reasoning and answer, obtained through a separate forward pass through the shared decoder, and a molecular-outcome representation supplied by a frozen SMI-TED encoder. Targets are centered and normalized, and the predictive branches are used only during training.

Experiments on ChemCoTBench report improvements across all six mean molecular-optimization scores, alongside improvements in several editing, structural-understanding, and reaction metrics. Additional experiments apply predictive supervision to LatentChem and Vanilla Coconut. Ablations investigate prediction views, target centering, and textual target-encoder updates. Held-out probing, within-task retrieval, and representational similarity analyses provide evidence that the resulting latent states contain more recoverable information about molecular outcomes and better reflect molecular relationships.

Critical strengths

The main Coconut-Chem comparison shares initialization, training data, curriculum, and training schedule between the baseline and predictive variant (Section 5.1; Appendix B). The reported mean property improvements increase on all six optimization objectives, including LogP from 0.7404 to 0.8318 and JNK3 from 0.1032 to 0.1296. Addition and deletion accuracy also improve, although substitution regresses (Table 1). Experiments with LatentChem and Vanilla Coconut provide further evidence that predictive supervision can benefit more than one reasoning configuration (Tables 4 and 9). Since target encoders and prediction heads are only needed during training, these improvements do not require modifying the generation procedure. This makes the approach a practically relevant extension to existing latent reasoners.

The representation analyses provide substantive evidence with appropriate controls. The main probes are fitted on 1,224 training examples and evaluated on 321 test examples excluded from both reasoner training and probe fitting. Final-step molecular-target prediction cosine increases from 0.186 to 0.397, with improvements on 277 of the 321 examples (Figure 2; Figure 10; Appendix K). Within-task retrieval and shuffled-target comparisons address the possibility that the results reflect task identity alone. Morgan-fingerprint similarity supplies an additional structural measure that was not directly used as a training target (Figure 3). The target-centering diagnostics and ablations further explain an important implementation choice. Together, these analyses substantiate the narrower claim that predictive supervision makes molecular-outcome information more accessible in latent states.

Critical weaknesses

Tables 1–2 and the performance ablations report point estimates without uncertainty on the task differences or reported variation across independently trained models. The three responses per problem are samples from the same checkpoint (Appendix L.2). Likewise, the uncertainty bands in the representation analyses hold the models and probes fixed (Appendix K.5). These measurements therefore do not establish the robustness of task gains to retraining.

This matters because several improvements are small (for example, major-product FTS increases from 0.6408 to 0.6492 and ring-system accuracy from 80.00% to 81.11%) and the entire editing evaluation contains only 100 problems. In addition, Appendix B states that loss coefficients and predictor widths were compared using the benchmark evaluation protocol, but does not identify a separate validation split or the criterion used to select the reported configuration. An explicit selection protocol and uncertainty estimates for the matched task comparisons are needed to assess how reliably the method improves performance. This is the main factor limiting my recommendation.

The distinctive contribution of combining textual and molecular future views is only partially isolated. Auxiliary supervision using representations of future text has substantial precedent in LLM-JEPA, Semformer, and Future Summary Prediction, which the paper appropriately cites. I therefore see the main methodological contribution as the particular integration of heterogeneous textual and molecular targets with recurrent latent reasoning. Establishing the benefit of this integration is consequential to the paper’s significance.

The view ablation in Table 8 is informative but only partially controlled: text-only training uses (\lambda_{{reason}}=0.05), whereas dual-view training uses (0.025). Thus, text-only versus dual-view changes both the presence of molecular supervision and the strength of textual supervision. The molecular-only versus dual-view comparison does hold (\lambda_{{chem}}) fixed and provides meaningful evidence for adding textual supervision; the corresponding controlled comparison for adding molecular supervision remains missing.

Furthermore, Appendix B specifies a textual coefficient of (0.05) for task-organization and recurrent-geometry analyses, compared with (0.025) for the main task results. The exact checkpoint correspondence for Figures 2–3 should be made explicit. These differences limit how directly the strongest representation findings can explain the principal task results. A coefficient-controlled comparison and diagnostics on the main performance configuration would make the dual-view contribution substantially more convincing.

Questions for authors

Which data and selection criteria were used to choose the loss coefficients, predictor widths, target-encoder configuration, and final checkpoints? Was there a validation set separate from the reported ChemCoTBench test set, and were any configuration choices informed by the reported test scores?

Can you provide paired, problem-level uncertainty estimates for the principal Coconut-Chem versus dual-view task differences, together with results across independently trained runs for this main comparison? Please distinguish uncertainty across evaluation problems, stochastic completions, and training runs. This would materially affect my assessment of the central empirical claim.

Does the dual-view model retain its advantage over text-only training when both use (\lambda_{\mathrm{reason}}=0.025), with the molecular coefficient fixed at (0.01) whenever that view is active? Please also identify the exact configurations and checkpoints used in Figures 2–3 and clarify whether the main task configuration exhibits the same representation trends.

5 comments
AuthorsOpenReview ↗

Response to Reviewer d4TD (1/2)

We thank the reviewer for the careful reading of the task comparisons and representation evidence, and for the specific questions on coefficient attribution and checkpoint correspondence. These questions motivated the additional comparisons below. We address W1–W4 and Q1–Q3 below.

Methodological contribution. LATENT-JEPA introduces a learning approach that uses JEPA-style future-representation prediction to shape recurrent continuous latent reasoning, opening a route for training latent thoughts to anticipate abstract representations of future solutions. Continuous latent reasoning provides a compact space for internal thoughts, and we explore how to better shape these thoughts so that they are more directly related to expectations about future solutions, in addition to supporting subsequent token generation. This offers a complementary learning path: latent thoughts are trained to anticipate abstract representations of what comes next before the solution is explicitly generated. We formalize this idea in Definition 1, which separates the latent reasoner from the future views, target encoders, and predictors used to supervise its thoughts. For chemical reasoning, we provide a dual-view instantiation in which the same latent thought predicts both the remaining textual solution and its molecular outcome in their respective representation spaces. This is one suitable instantiation, rather than a limit on the framework: the formulation allows other future views, scientific encoders, prediction horizons, and reasoning architectures to be explored. We also report empirical studies of this learning approach, examining its effects on chemical task performance, molecular-outcome information recoverable from latent thoughts, and the correspondence between latent representations and molecular structure. Together, the perspective, formulation, chemical instantiation, and empirical exploration provide a starting point for studying how future expectations can shape compact latent reasoning.

W1 & Q2: Uncertainty and training-run variation

Tables 1–2 in our response to Reviewer kPxb report the two-run means and sample SDs, completion-stream SDs, and paired problem-bootstrap intervals. Latent-JEPA achieves better mean performance on 14 of 17 metrics. The intervals use 10,000 within-task problem resamples, retaining all responses and both checkpoint pairs; the optimization-score transformation is recomputed in each resample. Intervals are wider for tasks with fewer test examples.

W2 & Q1: Configuration selection and predictor capacity

Selection protocol. ChemCoTBench has no separate development split. JEPA-specific coefficient and width sensitivities were assessed under its benchmark evaluation protocol; these comparisons informed the reference configuration and are reported as sensitivity analyses, not independent validation. The reference coefficients keep the weighted textual and molecular prediction losses on similar scales and below the autoregressive objective. Baseline and predictive checkpoints were exported after the same fixed number of training epochs, not selected by benchmark score.

Predictor widths. The textual and molecular target dimensions are 4096 and 768, respectively. We set the molecular predictor width to 1536 (twice its target dimension) and choose the textual width of 912 to match parameter counts (7.482M and 7.485M, respectively; see Table 10).

Table 10. Predictor capacity and parameter-count matching.

PredictorHidden widthParameters
Molecular15367,481,600
Textual (reference)9127,485,216
Textual (narrow)5124,207,616
Textual (wide)153612,598,272

The reference textual width of 912 matches the molecular predictor to within 0.048% in parameter count. In a sensitivity analysis using seed 12345, textual widths of 512, 912, and 1536 achieve better mean performance than Coconut-Chem on 11, 13, and 9 of the 17 metrics, respectively.

Target-encoder lifecycle. The default textual target is encoded online by the current decoder under stop-gradient, keeping decoder-native targets synchronized with the adapting generator. In the submitted lifecycle ablation, online encoding with training-mode dropout has the best average rank (1.68; EMA without dropout: 1.97; online without dropout: 2.35).

AuthorsOpenReview ↗

Response to Reviewer d4TD (2/2)

W3, W4 & Q3: View ablation, checkpoint correspondence, and main-configuration representations

Coefficient-controlled comparison. With the molecular coefficient fixed at 0.01, Dual achieves better mean performance than Text on 11 of 17 metrics at a textual coefficient of 0.025 and 14 of 17 at 0.05 (see Table 3 in our response to Reviewer kPxb).

Checkpoint correspondence. We will explicitly label the configurations in the revised paper. Figures 2–3 use a Latent-JEPA checkpoint trained with (\lambda_{\mathrm{reason}}=0.05) and (\lambda_{\mathrm{chem}}=0.01), whereas the main task results use a separately trained checkpoint with (\lambda_{\mathrm{reason}}=0.025) and the same molecular coefficient. Both use seed 12345, the same chemistry-aware initializer, four-phase curriculum, and six latent thoughts (Table 11).

Table 11. Final-step representation measurements. Probe evaluation uses benchmark-test examples excluded from both reasoner training and probe fitting. Figure 2 analyzes latent hidden states; Figure 3 analyzes recurrent feedback embeddings.

Final-step metricCoconut-ChemDual .025 (main task checkpoint)Dual .05 (original figures)
Probe prediction cosine.1856.4332.3974
Within-task Hit@532.85%78.45%73.04%
SMI-TED RSA.1103.2726.4003
Morgan-fingerprint RSA.0293.2444.3285

Both checkpoints show higher values than Coconut-Chem on all four representation measures. Morgan-fingerprint RSA provides an additional structural measure not used as a predictive training target. The probe intervals are conditional on the fixed models and probes and do not measure variation across retraining; training-run variation is reported separately in Tables 1–2.

Thank you. my concerns are addressed.

Reviewer_KndQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Latent-JEPA, a framework that augments continuous latent reasoning with auxiliary objectives for predicting representations of future reasoning and molecular outcomes. Built primarily on Coconut-Chem, it trains the final latent state to predict both language-model embeddings of the remaining solution and molecular embeddings from a frozen pretrained encoder, without adding prediction branches at inference.

Latent-JEP was evaluated on ChemCoTBench. It improves mean molecular optimization scores across all six objectives, improves or matches the Coconut-Chem baseline on molecular understanding and reaction prediction, and gives mixed results on editing.

Critical strengths
  • Good results: Adding Latent-JEPA to Coconut-Chem improves mean property improvement on all six optimization objectives.
  • The rationale is clear. Raw SMI-TED targets share a dominant direction, so an uncentered cosine loss can be lowered by predicting this shared direction rather than the individual outcome.
Critical weaknesses
  • The paper does not report variability across independent training seeds or confidence intervals for the main task metrics. Some task subsets are small: addition and deletion each contain only 20 problems, and their reported improvements correspond to four and six additional correct responses, respectively, across 60 completions.
  • The paper does not test whether gains require correctly paired outcome targets through training with shuffled targets.
  • Representation analyses do not yet explain the task improvements: Since the molecular objective explicitly supervises prediction of SMI-TED targets, better recovery of those targets is partly expected. The retrieval analyses in Figures 7 and 8 use examples held out from probe fitting but seen during reasoner training. The separate test-set probes and Morgan-fingerprint analyses provide stronger evidence of generalization, but exclude optimization tasks, where mean gains are most consistent.
Questions for authors
  • How were the loss coefficients, predictor width, and default configurations selected, and was a validation split used? Could you report several training seeds for Coconut-Chem and Latent-JEPA, with paired bootstrap intervals over problems?
  • Does the improvement persist with fewer or no latent updates, or when similar supervision is applied to a matched explicit-CoT model? How does task performance, not only probe performance, change with latent reasoning depth?
  • For optimization, how do outputs break down into invalid outputs, unchanged molecules, valid but non-improving edits, and successful edits, compared with Coconut-Chem? What explains greater mean QED improvement together with lower QED success, and do the conclusions hold without clipping?
  • How well does the trained molecular prediction head predict held-out targets, measured by cosine similarity and separation from within-task shuffled targets? Within each task, is this prediction quality associated with answer correctness or property improvement, particularly for molecular optimization? This would help connect the representation analyses to the reported task gains.
  • What are the per-task sizes, provenance, and quality-control procedures for the reasoning demonstrations? How much molecular or scaffold overlap exists between training and test data, and do the gains differ between familiar and structurally novel inputs?
6 comments
AuthorsOpenReview ↗

Response to Reviewer KndQ (1/3)

We thank the reviewer for the detailed experimental suggestions and for taking the time to specify the controls that would make the evidence more convincing. These suggestions directly shaped the additional analyses below. We address W1–W3 and Q1–Q5 below.

W1 & Q1: Multi-seed variability, configuration selection, and validation

Training and evaluation variability. We independently trained Coconut-Chem and Latent-JEPA with seeds 12345 and 23456, averaging three completion streams per checkpoint. The two-run means and SDs, completion-stream SDs, and paired 95% bootstrap intervals are reported in Tables 1–2 in our response to Reviewer kPxb. We compute pointwise 95% paired bootstrap intervals over 10,000 within-task problem resamples, retaining all responses and both checkpoint pairs per problem and recomputing the optimization-score transformation. Intervals are wider for tasks with fewer test examples.

Configuration protocol. ChemCoTBench has no separate development split. We assessed JEPA-specific coefficient and width sensitivities under its benchmark evaluation protocol; these comparisons informed the reference configuration and are not independent validation. We set the reference coefficients to keep the weighted textual and molecular prediction losses on similar scales and below the autoregressive objective. Baseline and predictive checkpoints were exported after the same fixed number of training epochs, not selected by benchmark score. Coefficient-matched results at both tested textual coefficients appear in Table 3 in our response to Reviewer kPxb.

Predictor capacity and target encoder. Both predictors map the same 4096-dimensional latent thought into their respective target spaces. Their leading parameter count scales as h(4096+dtarget)h(4096+d_{\mathrm{target}}), so a hidden unit in the 4096-dimensional textual predictor carries more parameters than one in the 768-dimensional molecular predictor. Starting from a molecular hidden width of 1536, twice its target dimension, parameter-count matching gives a textual width of approximately 912. The resulting predictors contain 7.482M and 7.485M parameters. For the decoder-native textual targets, online stop-gradient encoding keeps the target representation synchronized with the adapting decoder. The submitted lifecycle ablation gives average ranks of 1.68 for online encoding with dropout, 1.97 for EMA encoding without dropout, and 2.35 for online encoding without dropout.

W2: Shuffled-target control

We shuffled molecular targets within each task while retaining textual and autoregressive supervision.

Table 4. Molecular pairing control, mean ± sample SD across training seeds 12345 and 23456. Each checkpoint's three completion-stream scores are averaged first. Textual coefficient is 0.025; molecular coefficient is 0.01 when active. Rates and their SDs are percentages.

MetricCoconut-ChemText .025Dual .025Dual + molecular shuffle
LogP ↑0.7656 ± 0.03570.8151 ± 0.06850.7961 ± 0.05040.8170 ± 0.0566
Solubility ↑0.9293 ± 0.00480.9795 ± 0.01940.9738 ± 0.01120.9214 ± 0.0071
QED ↑0.2171 ± 0.00150.2206 ± 0.02100.2212 ± 0.01190.2220 ± 0.0107
DRD2 ↑0.2704 ± 0.00970.2727 ± 0.01480.2748 ± 0.01420.2667 ± 0.0051
JNK3 ↑0.1095 ± 0.00900.1108 ± 0.00610.1246 ± 0.00720.1188 ± 0.0025
GSK3-β ↑0.2266 ± 0.00260.2246 ± 0.01230.2391 ± 0.01760.2324 ± 0.0007
Functional-group error ↓0.0733 ± 0.00940.0700 ± 0.00940.0733 ± 0.00940.0667 ± 0.0141
Ring-count error ↓0.1417 ± 0.01180.1250 ± 0.03540.1167 ± 0.02360.1250 ± 0.0354
Murcko scaffold ↑0.7780 ± 0.00040.7917 ± 0.00450.7935 ± 0.01840.7692 ± 0.0206
Ring-system accuracy (%) ↑80.56 ± 0.7979.44 ± 1.5778.89 ± 3.1480.56 ± 0.79
Major-product FTS ↑0.6387 ± 0.00290.6309 ± 0.00570.6454 ± 0.00550.6301 ± 0.0117
Major-product validity (%) ↑86.67 ± 0.0086.83 ± 1.1888.17 ± 0.2486.00 ± 0.47
Byproduct FTS ↑0.1676 ± 0.02230.1634 ± 0.00150.1758 ± 0.00670.1603 ± 0.0116
Byproduct validity (%) ↑46.17 ± 1.1846.33 ± 0.4745.67 ± 0.4746.00 ± 0.94
Add (%) ↑72.50 ± 1.1879.17 ± 5.8974.17 ± 5.8981.67 ± 0.00
Delete (%) ↑78.33 ± 2.3674.17 ± 5.8981.67 ± 7.0777.50 ± 1.18
Substitute (%) ↑60.83 ± 6.6861.11 ± 3.1462.22 ± 1.5756.67 ± 5.50
AuthorsOpenReview ↗

Response to Reviewer KndQ (2/3)

Shuffling preserves the molecular target distribution and chemical validity but breaks the correspondence between each input and its future molecular outcome. Among the 14 metrics associated with molecular future supervision, correctly paired supervision achieves higher means on 10. The gains are particularly clear for substitution (+5.55 percentage points), Murcko scaffold similarity (+0.0243), and three of the four reaction metrics. The remaining three text-only metrics receive no molecular prediction loss and are excluded from this comparison. These results support the importance of correctly pairing each input with its molecular future target.

W3 & Q4: Representation analyses and task gains

We thank the reviewer for pointing out that molecular optimization, where our mean performance gains are most consistent, is not included in the representation analyses. We would like to clarify that, although optimization training demonstrations provide molecular endpoints for predictive supervision, benchmark test problems are open-ended and admit multiple valid improved molecules rather than a unique ground-truth outcome. Thus, no unique reference representation is available to evaluate latent-thought prediction consistency or its association with property improvement.

For the remaining tasks, we evaluate the frozen, trained molecular predictor on held-out test examples without fitting a new readout. Its predictions achieve a mean cosine similarity of 0.7046 with matched reference targets, versus 0.1914 with other targets from the same task (mean separation 0.5133; paired 95% CI [0.4942, 0.5314]).

We further examine whether prediction quality tracks answer quality. For each problem, we pair its pre-answer prediction cosine with the mean evaluator score of its three generated answers, then compute within-task Spearman correlations across problems. Table 5 reports these correlations and 95% CIs from 5,000 paired problem-bootstrap resamples per task.

Table 5. Within-task Spearman association [95% CI]. These are single-checkpoint associations.

TaskSpearman ρ [95% CI]
Addition.133 [−.275, .519]
Deletion.218 [−.176, .579]
Substitution.326 [.078, .537]
Murcko scaffold.607 [.400, .752]
Major product.390 [.209, .539]
Byproduct.489 [.278, .674]

Q2: Latent reasoning depth and explicit-CoT comparison

Latent reasoning depth. We use six latent thoughts throughout our main experiments. A systematic comparison across training depths would require separately training models under matched settings for each depth; we leave this study for future work.

No recurrent latent computation. We trained an explicit-CoT control with the same initializer, training budget, four-phase curriculum, and textual and molecular future views. Unlike the latent reasoner, it generates intermediate reasoning as explicit tokens rather than feeding a bounded continuous state back into subsequent computation.

Table 6. Explicit-CoT comparison, mean ± sample SD across training seeds 12345 and 23456. Each checkpoint's three completion-stream scores are averaged first. Rates and their SDs are percentages; bold marks a better mean than explicit CoT, accounting for metric direction.

MetricExplicit CoTExplicit CoT + JEPA
LogP ↑0.6811 ± 0.04120.7045 ± 0.0459
Solubility ↑0.9232 ± 0.08580.8268 ± 0.0386
QED ↑0.2011 ± 0.00750.2019 ± 0.0046
DRD2 ↑0.2028 ± 0.00710.2179 ± 0.0247
JNK3 ↑0.0957 ± 0.02260.0836 ± 0.0013
GSK3-β ↑0.1765 ± 0.01170.1725 ± 0.0065
Functional-group error ↓0.0633 ± 0.00470.0567 ± 0.0000
Ring-count error ↓0.1333 ± 0.02360.1583 ± 0.0118
Murcko scaffold ↑0.7554 ± 0.02940.7703 ± 0.0269
Ring-system accuracy (%) ↑77.78 ± 2.3675.56 ± 0.79
Major-product FTS ↑0.6014 ± 0.00750.6030 ± 0.0010
Major-product validity (%) ↑82.17 ± 1.1882.17 ± 1.18
Byproduct FTS ↑0.1721 ± 0.02190.1858 ± 0.0069
Byproduct validity (%) ↑45.33 ± 2.8346.17 ± 0.24
Add (%) ↑74.17 ± 1.1870.83 ± 3.54
Delete (%) ↑83.33 ± 0.0080.83 ± 1.18
Substitute (%) ↑60.28 ± 1.9659.72 ± 1.18

JEPA achieves better mean performance on 8 of 17 metrics in this explicit-CoT setting, compared with 14 of 17 for the latent reasoner. This comparison is consistent with future-view prediction being particularly useful when the supervised state participates in recurrent latent computation. Table 6 addresses the no-recurrent-computation part of the question through a separately trained explicit-CoT control.

AuthorsOpenReview ↗

Response to Reviewer KndQ (3/3)

Q3: Optimization behavior and clipping

We follow the benchmark evaluator's 5th/95th-percentile winsorization protocol and additionally report output decomposition and unclipped means.

Table 7. Output decomposition and unclipped improvement. Each cell shows Coconut-Chem → Latent-JEPA for the same two runs. Category rates use all responses; invalid/unscorable handling follows the evaluator.

ObjectiveInvalid/unscorable (%)Unchanged (%)Changed, no gain (%)Successful (%)Unclipped mean improvement
LogP5.83 → 4.170.50 → 0.5011.00 → 13.8382.67 → 81.500.6943 → 0.6960
Solubility7.00 → 5.830.83 → 0.507.33 → 6.6784.83 → 87.000.9424 → 0.9741
QED5.83 → 5.830.17 → 0.336.50 → 7.3387.50 → 86.500.2169 → 0.2189
DRD25.17 → 4.000.50 → 0.5016.67 → 16.1777.67 → 79.330.2657 → 0.2724
JNK37.17 → 5.331.00 → 0.6733.00 → 31.0058.83 → 63.000.1085 → 0.1265
GSK3-β4.50 → 4.000.33 → 0.3314.33 → 13.6780.83 → 82.000.2257 → 0.2391

All six unclipped means remain higher, although the LogP difference is small. For QED, the success rate falls slightly (87.50% to 86.50%), but improvement among successful outputs rises (0.2543 to 0.2614), producing a higher overall mean.

Q5: Data provenance and structural overlap

We use the released ChemCoTBench demonstrations without relabeling. The benchmark sources molecules from PubChem, ChEMBL, ZINC, and Deep-Mol-Opt, and reactions from USPTO, Pistachio, and Reaxys; its structured CoTs were distilled with DeepSeek-R1, checked with GPT-4o, and reviewed by 13 chemistry PhD annotators.

Table 8. Per-task training counts and test-source overlap. Exact identity and scaffold columns report seen/unseen test-source counts. Scaffold counts exclude acyclic sources with empty Murcko scaffolds.

TaskTraining nExact seen/unseenScaffold seen/unseen
LogP1,39495/597/3
Solubility1,41297/398/2
QED37769/3182/18
DRD21,49885/1593/7
JNK3180100/0100/0
GSK3-β724100/0100/0
Functional-group count1,51957/4374/26
Ring count1,60013/715/5
Murcko scaffold1,60032/835/5
Ring-system scaffold1,60050/1056/4
Major product2,78623/7759/30
Byproduct89023/7759/30
Add1,4990/205/15
Delete1,4990/206/14
Substitute1,4990/6020/39

Exact identity uses canonical isomeric SMILES without atom maps; scaffold identity uses nonempty Bemis–Murcko scaffolds. We apply no tautomer, protonation, or salt normalization. For reactions, the largest-heavy-atom reactant defines the primary source.

We compare Coconut-Chem with Dual 0.025 within each seen/unseen group, averaging over problems, completion streams, and runs; optimization winsorization is recomputed within each subgroup.

Table 9. Directional counts across available subgroup metrics. Entries are higher/lower/tied for Latent-JEPA relative to Coconut-Chem.

Familiarity splitSeen: higher/lower/tiedUnseen: higher/lower/tied
Exact source identity11/2/110/5/0
Bemis–Murcko scaffold11/3/39/3/3

Gains extend beyond exact source overlap: all editing inputs are exact-unseen, and all four reaction metrics have higher means on exact-unseen and scaffold-unseen sources. Optimization results are mixed, with no unseen JNK3/GSK3-β examples and fewer than 20 examples in several other unseen groups. We therefore do not claim uniform OOD generalization.

Reviewer_kPxbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the anticipatory aspect of continuous latent reasoning: whether latent thoughts can form expectations about useful abstractions of a future solution before it is explicitly generated. The authors propose LATENT-JEPA, which brings the JEPA principle into latent reasoning: at each supervised step, the current latent thought predicts representations of future views in their respective encoder spaces, and the predictive branches serve only as training supervision without altering the reasoner's recurrent computation or decoding procedure. For chemical reasoning, they instantiate the principle on Coconut-Chem with two complementary future views: a textual view (the remaining reasoning and answer in the language model's representation space, with target centering, L2​ normalization, stop-gradient, and online/EMA encoder comparisons) and a molecular view (a frozen SMI-TED encoder representing the reference chemical outcome). A shared latent thought predicts both views in their own spaces via separate predictors, avoiding the need for a common target representation.

Critical strengths
  1. Clear and extensible formalization: the paper moves JEPA from vision/video to "latent thoughts predicting future views" and uses Definition 1 to decouple the reasoning architecture from the choice of target representations (predictors map into each target encoder's space, multiple views and horizons are supported, and components may be reused downstream after training). This level of abstraction goes beyond adding an auxiliary loss and leaves room for swapping encoders or reasoners.
Critical weaknesses
  1. Effect sizes and statistical robustness of the main results are under-reported. Many gains in Tables 1 and 2 are small, yet the text summarizes them as "most metrics improve." Evaluation averages three sampled responses without reporting variance or confidence intervals for the same checkpoint, and no multi-seed training is reported. Since Coconut-Chem and LATENT-JEPA share initialization and training budget, readers cannot tell whether gains exceed decoding noise. Multi-seed means with variances, paired significance tests, and an explicit list of regressed metrics are needed.

  2. Baseline and protocol comparability is confounded. In Tables 1 and 2, GPT-4o, o1-mini, Gemini 2.5 Pro, etc. use the original ChemCoTBench protocol (Pass@1 for editing), whereas the authors' models average three samples; although Appendix B notes that "published baselines retain their original protocols," juxtaposing them in the main tables can mislead. In addition, Vanilla Coconut and Coconut-Chem differ in source interface, initialization, and training history (as the authors acknowledge in Appendix F), weakening the claim that predictive supervision transfers to settings without a source molecular encoder; in the single-vs-dual-view comparison, the textual coefficient (0.05 vs 0.025) and the number of views change together, preventing clean attribution (acknowledged in Table 8).

Questions for authors

1.Statistical robustness: what are the means and variances of the Table 1/Table 2 gains across multiple training seeds? Can paired bootstrap or significance tests be provided for key metrics, along with the sample standard deviations of the three sampled responses?

2.Attribution and confounding: can the authors provide a coefficient-matched comparison for the single-vs-dual-view setting? How much do the initialization/training-history differences between Vanilla Coconut and Coconut-Chem affect the conclusions, and is a same-initialization control available?

2 comments
AuthorsOpenReview ↗

Response to Reviewer kPxb (1/2)

We thank the reviewer for the time and care put into this review, and for recognizing the extensible formulation. We address W1–W2 and Q1–Q2 below.

W1 & Q1: Statistical robustness and training-run variation Across two independent runs (seeds 12345 and 23456), Latent-JEPA achieves better mean performance on 14 of 17 metrics, including all six optimization objectives. Ring-system accuracy and byproduct validity regress, while functional-group error is tied.

Table 1. Mean ± sample SD across two independent training runs; paired problem-bootstrap 95% CI for improvement. Rates are percentages; differences are percentage points. Error differences are sign-reversed so positive values favor Latent-JEPA.

MetricCoconut-ChemLatent-JEPAImprovement [95% CI]
LogP ↑0.7656 ± 0.03570.7961 ± 0.0504+0.0305 [-0.0569, +0.1274]
Solubility ↑0.9293 ± 0.00480.9738 ± 0.0112+0.0445 [-0.0393, +0.1247]
QED ↑0.2171 ± 0.00150.2212 ± 0.0119+0.0041 [-0.0112, +0.0192]
DRD2 ↑0.2704 ± 0.00970.2748 ± 0.0142+0.0044 [-0.0238, +0.0359]
JNK3 ↑0.1095 ± 0.00900.1246 ± 0.0072+0.0150 [-0.0014, +0.0341]
GSK3-β ↑0.2266 ± 0.00260.2391 ± 0.0176+0.0124 [-0.0037, +0.0302]
Functional-group error ↓0.0733 ± 0.00940.0733 ± 0.0094+0.0000 [-0.0217, +0.0200]
Ring-count error ↓0.1417 ± 0.01180.1167 ± 0.0236+0.0250 [+0.0000, +0.0750]
Murcko scaffold ↑0.7780 ± 0.00040.7935 ± 0.0184+0.0155 [-0.0137, +0.0468]
Ring-system accuracy (%) ↑80.56 ± 0.7978.89 ± 3.14-1.67 [-6.39, +2.78]
Major-product FTS ↑0.6387 ± 0.00290.6454 ± 0.0055+0.0066 [-0.0274, +0.0402]
Major-product validity (%) ↑86.67 ± 0.0088.17 ± 0.24+1.50 [-2.33, +5.33]
Byproduct FTS ↑0.1676 ± 0.02230.1758 ± 0.0067+0.0082 [-0.0115, +0.0291]
Byproduct validity (%) ↑46.17 ± 1.1845.67 ± 0.47-0.50 [-3.50, +2.50]
Add (%) ↑72.50 ± 1.1874.17 ± 5.89+1.67 [-5.83, +9.17]
Delete (%) ↑78.33 ± 2.3681.67 ± 7.07+3.33 [-9.17, +17.50]
Substitute (%) ↑60.83 ± 6.6862.22 ± 1.57+1.39 [-2.50, +5.28]

We compute pointwise 95% paired bootstrap intervals over 10,000 within-task problem resamples, retaining all responses and both checkpoint pairs per problem and recomputing the optimization-score transformation. Intervals are wider for tasks with fewer test examples.

Table 2. Sample SD across three completion streams per fixed checkpoint (percentage points for rates).

MetricCoconut-Chem 12345Latent-JEPA 12345Coconut-Chem 23456Latent-JEPA 23456
LogP ↑0.06050.02340.09920.0227
Solubility ↑0.07650.08490.07460.0791
QED ↑0.01080.01360.00590.0193
DRD2 ↑0.01660.04080.03520.0249
JNK3 ↑0.01810.01800.01050.0120
GSK3-β ↑0.02150.00720.01460.0250
Functional-group error ↓0.01530.00580.01000.0100
Ring-count error ↓0.00000.02890.02890.0000
Murcko scaffold ↑0.02040.00620.01330.0205
Ring-system accuracy (%) ↑3.332.554.194.41
Major-product FTS ↑0.00690.01770.02680.0187
Major-product validity (%) ↑1.154.162.522.65
Byproduct FTS ↑0.04040.02840.03350.0023
Byproduct validity (%) ↑2.083.214.361.00
Add (%) ↑7.645.775.775.00
Delete (%) ↑2.895.775.005.77
Substitute (%) ↑1.924.812.551.67
AuthorsOpenReview ↗

Response to Reviewer kPxb (2/2)

W2 & Q2: Protocol comparability and coefficient attribution

Protocol. Compared with the single-score reporting of published baselines, our evaluation provides a more comprehensive assessment through three independently sampled completions per problem and two independent training runs. We will clarify in the table caption.

Coefficient attribution. At a fixed textual coefficient, adding the molecular view gives a higher mean on 11 of 17 metrics at 0.025 and 14 of 17 at 0.05 (Table 3).

Vanilla Coconut. We would like to clarify that the relevant transfer comparison is Vanilla Coconut versus its JEPA variants, not Vanilla Coconut versus Coconut-Chem. Vanilla Coconut and its JEPA variants share the same text-only initializer, training data, curriculum, training budget, decoding settings, and evaluator, providing a same-initialization control. Source molecules are provided to the reasoner as text; SMI-TED is used only during training to encode future molecular targets and is not required at inference. The results show that JEPA-style molecular future supervision also benefits a latent reasoner operating without a molecular source encoder (Table 9 of the paper).

Table 3. Mean ± training-run SD. The molecular coefficient is 0.01 in both Dual settings. Bold marks a higher Dual mean than Text at the same textual coefficient, accounting for metric direction.

MetricText .025Dual .025Text .05Dual .05
LogP ↑0.8151 ± 0.06850.7961 ± 0.05040.7714 ± 0.01760.7981 ± 0.0184
Solubility ↑0.9795 ± 0.01940.9738 ± 0.01120.9278 ± 0.01410.9531 ± 0.0322
QED ↑0.2206 ± 0.02100.2212 ± 0.01190.2250 ± 0.00300.2273 ± 0.0030
DRD2 ↑0.2727 ± 0.01480.2748 ± 0.01420.2793 ± 0.01540.2952 ± 0.0113
JNK3 ↑0.1108 ± 0.00610.1246 ± 0.00720.1112 ± 0.00170.1127 ± 0.0118
GSK3-β ↑0.2246 ± 0.01230.2391 ± 0.01760.2316 ± 0.01410.2223 ± 0.0162
Functional-group error ↓0.0700 ± 0.00940.0733 ± 0.00940.0800 ± 0.00940.0633 ± 0.0094
Ring-count error ↓0.1250 ± 0.03540.1167 ± 0.02360.1333 ± 0.02360.1250 ± 0.0354
Murcko scaffold ↑0.7917 ± 0.00450.7935 ± 0.01840.7820 ± 0.00520.7797 ± 0.0299
Ring-system accuracy (%) ↑79.44 ± 1.5778.89 ± 3.1480.83 ± 1.1881.11 ± 0.00
Major-product FTS ↑0.6309 ± 0.00570.6454 ± 0.00550.6040 ± 0.00060.6288 ± 0.0118
Major-product validity (%) ↑86.83 ± 1.1888.17 ± 0.2483.50 ± 0.2485.50 ± 0.71
Byproduct FTS ↑0.1634 ± 0.00150.1758 ± 0.00670.1549 ± 0.01470.1778 ± 0.0114
Byproduct validity (%) ↑46.33 ± 0.4745.67 ± 0.4745.50 ± 2.1248.83 ± 0.24
Add (%) ↑79.17 ± 5.8974.17 ± 5.8977.50 ± 5.8978.33 ± 0.00
Delete (%) ↑74.17 ± 5.8981.67 ± 7.0785.00 ± 7.0780.83 ± 5.89
Substitute (%) ↑61.11 ± 3.1462.22 ± 1.5758.33 ± 0.7964.44 ± 1.57

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.