acceptodds
Under review as a conference paper at ICLR 2027

Autoregressive latent diffusion for 3D molecule generation

Abstract

Three-dimensional (3D) molecule generation has been dominated by diffusion models, which achieve strong generation quality but typically require molecular size to be specified (or predicted) separately before generation. This can be limiting for fragment-based molecule generation, central to drug discovery, where the size of the generated structure is itself part of the design problem. Autoregressive models determine size during generation and naturally support partial-structure conditioning, but balancing unconditional and fragment-conditioned generation remains challenging. We introduce KRONOS, a latent autoregressive diffusion framework that generates molecules in the latent space of a Unified AutoEncoder (UAE), jointly modeling molecular graph topology and geometry, while retaining the flexibility of autoregressive generation. We further introduce a mixed training strategy inspired by the Fill-in-the-Middle (FIM) paradigm, enabling a single left-to-right autoregressive model to support both unconditional and fragment-conditioned generation. Experiments on QM9 and GEOM-Drugs demonstrate strong unconditional generation performance and competitive fragment-conditioned generation.

Primary area:
generative models
Keywords:
3D molecule generation, autoregressive generative models, latent diffusion, fragment-conditioned generation, drug discovery

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_kpMUOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces KRONOS, a framework for autoregressive 3D molecule generation in the continuous latent space of a Unified Autoencoder. A diffusion head models the next-token distribution, while a stop classifier determines generation length. Fragment-local encoding and a FIM-inspired mixed training strategy enable unconditional and fragment-conditioned generation within a single model. Experiments on QM9 and GEOM-Drugs evaluate generation quality, fragment completion, conformational stability, and sampling efficiency.

Critical strengths

1. Unified task support: A single model accommodates both unconditional generation and fragment-conditioned completion, with molecular size determined during generation.

2. Promising unconditional performance: KRONOS improves several generation-quality metrics over the evaluated autoregressive baselines on QM9 and GEOM-Drugs.

3. Multiple evaluation perspectives: The experiments include xTB refinement, sampling-efficiency measurements, and ablations on atom ordering, model size, and the training mixture.

Critical weaknesses

1. Limited methodological novelty: The core generative formulation is a relatively direct combination of existing approaches: UAE-3D already provides a unified latent representation of molecular topology and geometry; MAR established continuous-token autoregression with Diffusion Loss; and models such as QUETZAL and InertialAR already perform autoregressive 3D molecule generation without a separate molecular autoencoder. Variable-length generation and the use of Diffusion Loss therefore provide limited grounds for establishing a distinctive methodological contribution. The main additional designs are fragment-local encoding and mixed fragment-conditioned training. While these are relevant adaptations, the overall methodological advance appears incremental relative to the established representation and generative framework.

2. Insufficient evidence for the added value of latent autoregression: Although Tables 1–2 and Appendix A.8 demonstrate improvements over the evaluated baselines, the experiments do not sufficiently isolate the benefit of the latent AR formulation. In particular, there is no controlled comparison between autoregressive and diffusion priors operating on the same UAE representation. Comparisons with direct atom-level AR also leave the effects of the representation confounded with model capacity, training budget, and sampling implementation, making it difficult to assess whether the gains justify the additional autoencoder and training stage. Moreover, the pfp_f ablation reports only unconditional QM9 performance, without measuring the corresponding conditional-generation gains or establishing the claimed trade-off on GEOM.

3. Incomplete and insufficiently documented autoregressive comparisons: The claim of leading unconditional performance among autoregressive methods is difficult to assess without comparisons to relevant approaches such as Geo2Seq, InertialAR and Uni-3DAR. InertialAR and Uni-3DAR provide released checkpoints, while Geo2Seq provides an official reproduction pipeline. These comparisons should use a common evaluation implementation, with differences in data preprocessing, conformer selection, dataset splits, and sampling protocols stated explicitly. Tables 2 and 3 also use a retrained QUETZAL despite the availability of an official GEOM checkpoint. Retraining may be appropriate to match the data setup, but the paper does not explain this choice or document the retraining budget and checkpoint-selection criterion. Evaluating the released checkpoint as an additional reference, with its training-data differences clearly identified, would help assess the robustness of the comparison.

4. Insufficient validation of fragment-conditioned generation: Table 3 reports validity, stability, and uniqueness, but does not directly evaluate whether completions preserve the supplied fragment or successfully attach new structure to it. Algorithm 1 jointly decodes the seed and generated latents, while Appendix A.3 describes a decoder that integrates information across fragments. Fixing seed latents therefore does not by itself guarantee preservation of the seed’s atom identities, internal bonds, or coordinates. Appendix A.13 further acknowledges sensitivity to the fragment’s placement within a reference conformer, with out-of-distribution placements producing shorter and less reliable completions. The evaluation consequently leaves unclear both whether the conditioning constraints are satisfied and how well the method works with independently specified fragments. Seed retention, successful attachment, and sensitivity to centering and rigid transformations should be quantified, including the fraction of all completions that simultaneously preserve the seed, form a connected structure, and satisfy chemical-validity checks.

Questions for authors

Please see the Weaknesses section above.

Reviewer_7DPKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces KRONOS, a latent autoregressive model for unconditional molecule generation. KRONOS uses UAE to move all atomistic features to a latent space. Then KRONOS uses an Autoregressive Diffusion loss to generate molecules in 3D. KRONOS supports de novo molecule generation and seeding with a fragment, then generating. KRONOS achieves strong results on QM9 and GEOM-Drugs benchmarks.

Critical strengths

KRONOS demonstrates strong results on unconditional molecule generation benchmarks. KRONOS is often best or second best on 6 QM9 tasks, and is the best on 5/6 tasks on the GEOM-Drugs task. I also appreciate the authors denoting which methods were run from scratch vs retrained vs using existing generated molecules.

Critical weaknesses

See 'Questions' for more details.

I am concerned about the methodological novelty of the work. I understand combining UAE and autoregressive diffusion losses, but is there other methodological novelty? Some additional emphasis on what is novel versus what is used from other works could be useful. Right now it seems that UAE and AR-diffusion losses are already well studied, and KRONOS simply combines them together.

In addition, comparisons to other fragment-based methods are lacking. The authors mention previous diffusion and AR based methods, but there is no mention of other previous fragment based molecule generation methods. There are both AR (FRAME) and diffusion (DecompDiff, DrugFlow, MedSAGE) fragment based generation methods.

In addition, the results for fragment conditioned design do not appear to be strong. KRONOS underperforms NEAT consistently in Table 3. I am not sure if this should be a main selling point of the model.

Questions for authors

Some key references are missing. The authors present a novel fragment based AR-diffusion approach. However, there are already methods that support fragment based molecule generation, namely FRAME, MedSAGE, FLOWR, SAFE-GPT, MiCAM. What makes KRONOS unique compared to these methods? In addition, what about sequence based conformer generation methods? That could be additional methods to compare against for the conformational stability task.

The authors cite UAE-3D by Luo et al, and use a UAE-3D encoder and decoder as part of the method. The Luo et al paper has benchmarking results on QM9 and GEOM-Drugs, why do the authors not compare results against what's reported in the paper? And what about comparisons to other methods, such as FLOWR, SAFE-GPT? The authors report comparisons to AR and Diffusion/flow methods, but do not mention comparisons to other fragment-based methods.

Also, there is only 1 sentence in the introduction talking about the problem statement and relevance. Why is molecule generation important? All that is currently mentioned is that molecule generation is a 'central problem.' Why is unconditional molecule generation important? Can this be applied to conditional/target conditioned molecule generation?

Reviewer_arzJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces KRONOS, an autoregressive latent diffusion model for 3D molecular generation. The model generates one continuous latent representation per atom, using a diffusion process to model the distribution of the next latent token conditioned on previously generated tokens. The resulting latent sequence is then decoded by a Unified Autoencoder into a molecular structure, jointly recovering its topology and 3D geometry.

A central contribution is a mixed training strategy that enables both unconditional and fragment-conditioned generation within the same autoregressive model. Half of the training examples follow the default atom ordering, while the other half are decomposed into fragments. Atoms within each fragment follow a deterministic ordering, providing a structured compromise to the ordering sensitivity of autoregressive models, although it does not achieve permutation invariance.

The paper validates the approach on QM9 and GEOM-Drugs, evaluating unconditional and fragment-conditioned generation on several standard metrics. The model performs well on unconditional generation, but remains below the baseline on fragment-conditioned generation.

Despite this, the model proposes itself as a simple yet effective autoregressive model for 3D molecular generation.

Critical strengths
  • Satisfying results on unconditional generation, even against diffusion models
  • Fragment-based completion is well justified by some use-case situations. Although the approach remains dependent on a particular ordering scheme, the authors show how it is a good compromise that does not substantially compromise unconditional generation
Critical weaknesses

I have a couple of concerns that ultimately influenced my score, and that I would like to discuss

  • Methodology:

    • My main concern is KRONOS' reliance on BRICS-based fragmentation. While it is recognized by the authors and left as a future experiment, I strongly believe that an experiment showing KRONOS' behavior when conditioning on non-BRICS fragments would have been beneficial to the paper (even simple random connected subgraphs would have been enough).
  • Experiments:

    • My main concerns related to this paper are KRONOS' weak results on the fragment-conditioned task. Is there any reason why KRONOS should be considered as a competitive alternative to NEAT, for instance?
    • If I understand correctly, Table 1's experiment consists of generating 10K molecules, from which the reported metrics are extracted. However, MiDi's paper results on the same experiment seem better than the ones listed in your paper for the same model. Considering how you claim to have obtained MiDi's results using the official checkpoints, could you clarify the reason behind this discrepancy?

Two non-critical weaknesses I would like to see at least discussed are:

  • Recent advances in graph diffusion do not require specifying the number of nodes in advance [1, 2], and there are DDPMs capable of both inserting and deleting nodes during the diffusion process [3]. Comparing such methods with your solution, even if only in the introduction or related works, would have further strengthened your paper's motivations (it is true that some of these models are designed for 2D data, but they would be easily extensible to 3D molecules as well).
  • Your methodology's scope vaguely reminds me of [4]; a comparison with this method would have made for a very strong baseline.

[1] Ninniri et al. Classifier-free graph diffusion for molecular property targeting

[2] Ketata et al. Lift Your Molecules: Molecular Graph Generation in Latent Euclidean Space

[3] Ninniri et al. Graph Diffusion that can Insert and Delete

[4] Li et al. InertialAR: Autoregressive 3D Molecule Generation with Inertial Frames

Questions for authors

Questions/clarifications that would also affect my assessment:

  • Regarding the "Assessment of conformational stability" section, did you use the method described for the baseline as well?
  • You used 4400 NFEs for KRONOS, but almost half as many for NEAT? What happens at parity of NFEs?
  • Row 178: have you considered using OA-AR for your model? I believe that a brief discussion on what justified your choice would benefit the paper.
  • Do I understand correctly that, when you remove the edges between fragments, you do not reinforce them anywhere else, and that despite this, the reconstruction works well anyway?

Minor clarifications/corrections:

  • Equation 1: Is tt supposed to be the timestep of the denoising process associated with a generic, partially corrupted data point ztz_t? If that is the case, note how ztz_t may be easily confused with a generic node in the graph, considering how right above, on row 82, you introduce Z=(z1,...,zn)Z = (z_1, ..., z_n).
  • Row 194: "To construct fragment-conditioned training sequences, each molecule is partitioned into K fragments". This statement can be interpreted such that every single moleclue is partitioned in exactly K fragments (that is, K is a constant), which does not seem to be the case according to later statements.

Suggestions for future work

  • Some experiments on guidance-conditioned generation, which are broadly available in the literature, would have been nice to have (assuming good results)
Reviewer_L2XYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes KRONOS, a 3D molecular generative model that supports both de novo generation and fragment-based completion. The authors jointly encode the chemical and spatial information of each atom and then generate the atom representations one by one. Experiments show good geometric quality and sampling efficiency, offering a practically useful approach to 3D molecular generation.

Critical strengths
  1. Good generation quality and efficiency. The method improves the geometric metrics relative to the autoregressive baseline, and it can increase speed by reducing the number of sampling steps while largely maintaining the stability metrics.
  2. The design is coherent and oriented toward practical needs. By combining independent fragment encoding with completion training, a single model can handle both de novo generation and fragment extension.
Critical weaknesses
  1. The existing comparisons cannot determine whether the performance improvement comes from the molecular representation or from the generation scheme. The unified molecular representation comes from UAE-3D, and the practice of combining autoregression with per-atom diffusion already exists in QUETZAL. The authors combine the two, but do not compare different generation schemes under the same molecular representation, so it is impossible to tell which part the improvement mainly comes from.
  2. The validation of fragment completion is incomplete. The authors report the validity of complete molecules, but do not check whether the structure and coordinates of the input fragment are preserved; this directly concerns whether the method can meet the requirements of fragment extension.
Questions for authors

Fragment extension requires preserving the input structure. At the end of Algorithm 1, the input fragment and the newly generated part are decoded jointly; does this step fix the atom types, internal bonds, and coordinates of the original fragment? If there is no hard constraint, what are the fragment-preservation rate and the coordinate deviation in the current samples?

1 comment

I thank the authors for the detailed rebuttal and the additional experiments.

The fragment-preservation measurements fully resolve my second concern: the conditioning fragment is recovered exactly, with coordinate deviations at the reconstruction accuracy of the autoencoder.

On my first concern, the UDM-3D comparison is the right experiment, but its conclusion rests on a baseline the authors were unable to reproduce at its published level (76.3% against the reported 93.5%, while KRONOS reaches 92.6%). I appreciate the transparency here, and the independent reproduction attempt the authors cite does support their reading. Still, the attribution of the gain remains open, and the comparison between latent and direct atom-space autoregression is not settled by this experiment.

I therefore maintain my rating of 2 (weak rejection).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.