- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Latent-JEPA, a framework that augments continuous latent reasoning with prediction of representations of future reasoning and outcomes. Its primary chemical instantiation adds two auxiliary objectives to Coconut-Chem: the final latent thought predicts horizon-conditioned representations of the remaining reasoning and answer, obtained through a separate forward pass through the shared decoder, and a molecular-outcome representation supplied by a frozen SMI-TED encoder. Targets are centered and normalized, and the predictive branches are used only during training.
Experiments on ChemCoTBench report improvements across all six mean molecular-optimization scores, alongside improvements in several editing, structural-understanding, and reaction metrics. Additional experiments apply predictive supervision to LatentChem and Vanilla Coconut. Ablations investigate prediction views, target centering, and textual target-encoder updates. Held-out probing, within-task retrieval, and representational similarity analyses provide evidence that the resulting latent states contain more recoverable information about molecular outcomes and better reflect molecular relationships.
Critical strengths
The main Coconut-Chem comparison shares initialization, training data, curriculum, and training schedule between the baseline and predictive variant (Section 5.1; Appendix B). The reported mean property improvements increase on all six optimization objectives, including LogP from 0.7404 to 0.8318 and JNK3 from 0.1032 to 0.1296. Addition and deletion accuracy also improve, although substitution regresses (Table 1). Experiments with LatentChem and Vanilla Coconut provide further evidence that predictive supervision can benefit more than one reasoning configuration (Tables 4 and 9). Since target encoders and prediction heads are only needed during training, these improvements do not require modifying the generation procedure. This makes the approach a practically relevant extension to existing latent reasoners.
The representation analyses provide substantive evidence with appropriate controls. The main probes are fitted on 1,224 training examples and evaluated on 321 test examples excluded from both reasoner training and probe fitting. Final-step molecular-target prediction cosine increases from 0.186 to 0.397, with improvements on 277 of the 321 examples (Figure 2; Figure 10; Appendix K). Within-task retrieval and shuffled-target comparisons address the possibility that the results reflect task identity alone. Morgan-fingerprint similarity supplies an additional structural measure that was not directly used as a training target (Figure 3). The target-centering diagnostics and ablations further explain an important implementation choice. Together, these analyses substantiate the narrower claim that predictive supervision makes molecular-outcome information more accessible in latent states.
Critical weaknesses
Tables 1–2 and the performance ablations report point estimates without uncertainty on the task differences or reported variation across independently trained models. The three responses per problem are samples from the same checkpoint (Appendix L.2). Likewise, the uncertainty bands in the representation analyses hold the models and probes fixed (Appendix K.5). These measurements therefore do not establish the robustness of task gains to retraining.
This matters because several improvements are small (for example, major-product FTS increases from 0.6408 to 0.6492 and ring-system accuracy from 80.00% to 81.11%) and the entire editing evaluation contains only 100 problems. In addition, Appendix B states that loss coefficients and predictor widths were compared using the benchmark evaluation protocol, but does not identify a separate validation split or the criterion used to select the reported configuration. An explicit selection protocol and uncertainty estimates for the matched task comparisons are needed to assess how reliably the method improves performance. This is the main factor limiting my recommendation.
The distinctive contribution of combining textual and molecular future views is only partially isolated. Auxiliary supervision using representations of future text has substantial precedent in LLM-JEPA, Semformer, and Future Summary Prediction, which the paper appropriately cites. I therefore see the main methodological contribution as the particular integration of heterogeneous textual and molecular targets with recurrent latent reasoning. Establishing the benefit of this integration is consequential to the paper’s significance.
The view ablation in Table 8 is informative but only partially controlled: text-only training uses (\lambda_{{reason}}=0.05), whereas dual-view training uses (0.025). Thus, text-only versus dual-view changes both the presence of molecular supervision and the strength of textual supervision. The molecular-only versus dual-view comparison does hold (\lambda_{{chem}}) fixed and provides meaningful evidence for adding textual supervision; the corresponding controlled comparison for adding molecular supervision remains missing.
Furthermore, Appendix B specifies a textual coefficient of (0.05) for task-organization and recurrent-geometry analyses, compared with (0.025) for the main task results. The exact checkpoint correspondence for Figures 2–3 should be made explicit. These differences limit how directly the strongest representation findings can explain the principal task results. A coefficient-controlled comparison and diagnostics on the main performance configuration would make the dual-view contribution substantially more convincing.
Questions for authors
Which data and selection criteria were used to choose the loss coefficients, predictor widths, target-encoder configuration, and final checkpoints? Was there a validation set separate from the reported ChemCoTBench test set, and were any configuration choices informed by the reported test scores?
Can you provide paired, problem-level uncertainty estimates for the principal Coconut-Chem versus dual-view task differences, together with results across independently trained runs for this main comparison? Please distinguish uncertainty across evaluation problems, stochastic completions, and training runs. This would materially affect my assessment of the central empirical claim.
Does the dual-view model retain its advantage over text-only training when both use (\lambda_{\mathrm{reason}}=0.025), with the molecular coefficient fixed at (0.01) whenever that view is active? Please also identify the exact configurations and checkpoints used in Figures 2–3 and clarify whether the main task configuration exhibits the same representation trends.
5 comments
Response to Reviewer d4TD (2/2)
W3, W4 & Q3: View ablation, checkpoint correspondence, and main-configuration representations
Coefficient-controlled comparison. With the molecular coefficient fixed at 0.01, Dual achieves better mean performance than Text on 11 of 17 metrics at a textual coefficient of 0.025 and 14 of 17 at 0.05 (see Table 3 in our response to Reviewer kPxb).
Checkpoint correspondence. We will explicitly label the configurations in the revised paper. Figures 2–3 use a Latent-JEPA checkpoint trained with (\lambda_{\mathrm{reason}}=0.05) and (\lambda_{\mathrm{chem}}=0.01), whereas the main task results use a separately trained checkpoint with (\lambda_{\mathrm{reason}}=0.025) and the same molecular coefficient. Both use seed 12345, the same chemistry-aware initializer, four-phase curriculum, and six latent thoughts (Table 11).
Table 11. Final-step representation measurements. Probe evaluation uses benchmark-test examples excluded from both reasoner training and probe fitting. Figure 2 analyzes latent hidden states; Figure 3 analyzes recurrent feedback embeddings.
| Final-step metric | Coconut-Chem | Dual .025 (main task checkpoint) | Dual .05 (original figures) |
|---|---|---|---|
| Probe prediction cosine | .1856 | .4332 | .3974 |
| Within-task Hit@5 | 32.85% | 78.45% | 73.04% |
| SMI-TED RSA | .1103 | .2726 | .4003 |
| Morgan-fingerprint RSA | .0293 | .2444 | .3285 |
Both checkpoints show higher values than Coconut-Chem on all four representation measures. Morgan-fingerprint RSA provides an additional structural measure not used as a predictive training target. The probe intervals are conditional on the fixed models and probes and do not measure variation across retraining; training-run variation is reported separately in Tables 1–2.
Thank you. my concerns are addressed.
Response to Reviewer d4TD (1/2)
We thank the reviewer for the careful reading of the task comparisons and representation evidence, and for the specific questions on coefficient attribution and checkpoint correspondence. These questions motivated the additional comparisons below. We address W1–W4 and Q1–Q3 below.
Methodological contribution. LATENT-JEPA introduces a learning approach that uses JEPA-style future-representation prediction to shape recurrent continuous latent reasoning, opening a route for training latent thoughts to anticipate abstract representations of future solutions. Continuous latent reasoning provides a compact space for internal thoughts, and we explore how to better shape these thoughts so that they are more directly related to expectations about future solutions, in addition to supporting subsequent token generation. This offers a complementary learning path: latent thoughts are trained to anticipate abstract representations of what comes next before the solution is explicitly generated. We formalize this idea in Definition 1, which separates the latent reasoner from the future views, target encoders, and predictors used to supervise its thoughts. For chemical reasoning, we provide a dual-view instantiation in which the same latent thought predicts both the remaining textual solution and its molecular outcome in their respective representation spaces. This is one suitable instantiation, rather than a limit on the framework: the formulation allows other future views, scientific encoders, prediction horizons, and reasoning architectures to be explored. We also report empirical studies of this learning approach, examining its effects on chemical task performance, molecular-outcome information recoverable from latent thoughts, and the correspondence between latent representations and molecular structure. Together, the perspective, formulation, chemical instantiation, and empirical exploration provide a starting point for studying how future expectations can shape compact latent reasoning.
W1 & Q2: Uncertainty and training-run variation
Tables 1–2 in our response to Reviewer kPxb report the two-run means and sample SDs, completion-stream SDs, and paired problem-bootstrap intervals. Latent-JEPA achieves better mean performance on 14 of 17 metrics. The intervals use 10,000 within-task problem resamples, retaining all responses and both checkpoint pairs; the optimization-score transformation is recomputed in each resample. Intervals are wider for tasks with fewer test examples.
W2 & Q1: Configuration selection and predictor capacity
Selection protocol. ChemCoTBench has no separate development split. JEPA-specific coefficient and width sensitivities were assessed under its benchmark evaluation protocol; these comparisons informed the reference configuration and are reported as sensitivity analyses, not independent validation. The reference coefficients keep the weighted textual and molecular prediction losses on similar scales and below the autoregressive objective. Baseline and predictive checkpoints were exported after the same fixed number of training epochs, not selected by benchmark score.
Predictor widths. The textual and molecular target dimensions are 4096 and 768, respectively. We set the molecular predictor width to 1536 (twice its target dimension) and choose the textual width of 912 to match parameter counts (7.482M and 7.485M, respectively; see Table 10).
Table 10. Predictor capacity and parameter-count matching.
The reference textual width of 912 matches the molecular predictor to within 0.048% in parameter count. In a sensitivity analysis using seed 12345, textual widths of 512, 912, and 1536 achieve better mean performance than Coconut-Chem on 11, 13, and 9 of the 17 metrics, respectively.
Target-encoder lifecycle. The default textual target is encoded online by the current decoder under stop-gradient, keeping decoder-native targets synchronized with the adapting generator. In the submitted lifecycle ablation, online encoding with training-mode dropout has the best average rank (1.68; EMA without dropout: 1.97; online without dropout: 2.35).