acceptodds
Under review as a conference paper at ICLR 2027

Prefix-Conditioned Boltzmann Generators: Towards General Molecular Sampling

Abstract

Reliable estimates of molecular properties require sampling the equilibrium conformational ensemble, which remains a bottleneck for molecular simulations. Boltzmann Generators (BGs) address this by utilizing generative models with tractable likelihoods to enable importance sampling towards the Boltzmann distribution. Recently, efforts have been made to scale BG training to increasingly large and diverse data. However, current BG approaches are often optimized for one particular chemistry, and only individual molecules. We introduce Prefix-Conditioned Boltzmann Generators (PBGs), an autoregressive Boltzmann generator that conditions directly on text and coordinate prefixes rather than a specialized atom-wise molecular encoder. Across a series of models trained on Many Peptides, we identify scaling relationships linking sampling quality to model size, training tokens, training compute, atom count, and coordinate discretization resolution. Guided by these relationships, we scale PBGs up to 640M parameters, achieving state-of-the-art performance on unseen peptide systems up to 157 atoms. We observe that models trained across diverse molecular classes and environmental conditions, achieve strong performance in our main benchmark at comparably lower training cost. This performance suggests shared information and potential transferability across molecular types. Beyond peptides, PBGs' prefix conditioning enables flexible generation across molecular systems and environments, including small-molecule sampling, solvation free energy estimation, and conditional microsolvation of systems with over 280 atoms. Together, these results establish PBGs as a promising route toward general molecular sampling.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Boltzmann Generator, Statistical Mechanics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4igzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper extends Discretized Boltzmann Generators (which is a new term coined by the authors) by replacing the atom-wise graph encoder with a text prefix.

Existing work is limited by how the conditioning is done, i.e., hand-designed features of a chemical graph are used as embeddings. This comes with three challenges: First, conditioning length grows with the number of atoms, second, a variable number of atoms is not permitted and third, extensions potentially require new hand-crafted features. The text prefix extends these graph conditionings by specifying the molecular graph, the solvent environment, the potential energy model, and temperature.

The authors train models between 8M and 85M parameters on Many Peptides, and then scale to 640M parameters, where they report the best rESS on 2AA/4AA/8AA peptides so far. They further train a joint model on peptides, MDQM9-nc and FreeSolv.

Additionally they apply speculative decoding, estimate solvation free energies. Four new MD datasets are also announced.

Critical strengths
  • The paper is written well and gives a nice and honest history of existing methods.
  • The scaling study is large and carefully documented, and the Many Peptides numbers of the 640M model are a big step forward.
  • The discretization analysis is nice! The finding that the resolution has an optimum (in contrast to Rehman et al.) and the derivation of the quantization energy error are nice additions.
  • The limitations section is honest about the scaling law and the small-molecule results, and the datasets will be useful if released.
Critical weaknesses
  • As far as I could see, GPU hours for the experiments were not reported
  • The actual method change, prefix instead of graph encoder, is never ablated. There is no run with the same transformer, data, Muon optimizer and token budget but Robin-style conditioning. Table 2 compares against a reproduced Robin whose parameter count and training tokens are not reported, so I cannot tell whether the 13× rESS gap between Robin PBG comes from the conditioning, the optimizer or just far more tokens.
  • The state-of-the-art comparison is not like-for-like. The hardest 8AA system was removed from the test set because the model fails on it (Appendix E), while the literature rows are presumably on all 30 systems (?), and the paper still bolds its own numbers.
  • There are no seeds or error bars anywhere.
  • There is no comparison against MD/REMD throughput, which is the paper's own motivation.
Questions for authors
  • What are Robin's parameter count and token budget in Table 2, and does your reproduction match the published numbers? Can you add the controlled ablation (same transformer, data, optimizer, tokens) with graph-encoder vs. prefix conditioning?
  • Would be great if the authors could report Table 2 on all 30 8AA systems including YFPHAGYT, and state whether the TarFlow and Prose rows are on 30 systems.
  • Why are the 160M and 320M models worse than the 85M model on 2AA/4AA despite more compute?
  • Are any rows of Table 2 repeated with different seeds?
  • What are the GPU-hours of the 640M run, and how many effective samples per second does MD or REMD produce on the same octapeptides and hardware?

In general, I like the paper and am willing to raise my score if the authors address my concerns.

3 comments
AuthorsOpenReview ↗

# Response to 4igz part 1

We thank reviewer 4igz for their encouraging and precise review. We are glad that the reviewer finds the paper "written well" with "a nice and honest history of existing methods", the scaling study "large and carefully documented", the 640M results "a big step forward", and the discretization analysis a nice addition. We addressed every question with new experiments. Below we answer them in turn. All new and changed text in the revised PDF is marked in green.

What are Robin's parameter count and token budget in Table 2, and does your reproduction match the published numbers? Can you add the controlled ablation (same transformer, data, optimizer, tokens) with graph-encoder vs. prefix conditioning?

ROBIN has 132M parameters and was trained to D/N≈1100D/N\approx1100 loss tokens per parameter (correspondence with the authors; Appendix D.1). Our evaluation ("ROBIN (T=1.0)" in Table 2, three sampling seeds) agrees with the published values (T=0.9) on rESS and on the torsion and TICA metrics. The 8AA energy W2 is higher, which we attribute mainly to the different logit temperature. We added the controlled ablation: at matched transformer, data, optimizer and tokens, the prefix is as good as the atom-wise encoder at 30M and better at 85M, and most of the gap to ROBIN comes from the optimizer settings and random translation.

For the ablation, we trained a new ladder of nine models at 30M and at 85M (Appendix D.6). All rungs share the PBG transformer, the data and 48N48N loss tokens. Rung L0 uses the ROBIN protocol and ROBIN's atom features, and each rung switches one difference to the PBG setting (Table 14). The last step changes only the conditioning.

Table R.1. Ladder from the ROBIN protocol to PBG at 85M, unclipped metrics, means over test systems and three sampling seeds (Table 15 of the revised PDF).

RungChange2AA rESS*2AA E-W2*4AA E-W2*8AA E-W2*
L0ROBIN protocol0.0015.5611.053.3
L1+ optimizer settings0.0014.066.7016.2
L6+ schedule, precision, packing, loss, quantization0.0013.755.9515.4
L7+ random translation0.0622.814.6112.3
L8+ HELM prompt (PBG)0.1201.253.6210.6

Three steps account for at least 90% of the L0-to-L8 change of each metric in log scale: L1, L7 and, at 85M, L8. All rungs see the same loss tokens, so the gap does not come from more training tokens.

Would be great if the authors could report Table 2 on all 30 8AA systems including YFPHAGYT, and state whether the TarFlow and Prose rows are on 30 systems.

The new Table 13 reports Table 2 on all 30 octapeptides. In Table 2, the TarFlow and Prose rows are on the same 29 octapeptides as our rows, after correspondence with the authors, so the comparison and the bold values are like-for-like (Appendices D.5 and E).

While answering this question, we found that the reference MD of YFPHAGYT contains a stereochemistry artifact of Many Peptides, which explains the poor performance (new Appendix D.5). Its published starting structure has L-Pro3, but a Phe2 side-chain atom lies within 1 Å of the Pro3 ring. The default OpenMM energy minimization removes this overlap and inverts the Pro3 Cα, so the reference trajectory has D-Pro3. The same artifact is in 276 training peptides, and all models learned it: 90.9% to 99.6% of PBG samples and 98.7% to 99.0% of ROBIN samples of YFPHAGYT have D-Pro3. Our filter compares samples with the sequence and rejected them. The public evaluation code of Tan et al. compares samples with one reference MD frame and cannot detect an artifact that the reference shares.

Table R.2. 8AA results on all 30 octapeptides with a filter based on the reference structure (Table 13 of the revised PDF). Literature rows are the published 30-system values.

ModelrESSE-W2T-W2TICA-W2
Prose0.01110.0382.4560.988
ROBIN (T=0.9)0.0134.2512.3250.943
PBG 640M0.0772.5971.8240.944
AuthorsOpenReview ↗

# Response to 4igz part 2

Why are the 160M and 320M models worse than the 85M model on 2AA/4AA despite more compute?

Because the Table 2 rows were trained to different budgets with different settings. At a matched budget, rESS increases with model size at every peptide length.

Table 2 reports the best available checkpoint of each size. The 85M model was trained to D/N=768D/N=768, the 160M model to D/N=352D/N=352 and the 320M model to D/N=86D/N=86 (Table 2), with different hardware and batch sizes (Table 26). To isolate model size, we trained every PBG size to D/N=48D/N=48 with several training seeds (new Appendix D.4, Table 11). At this matched budget, rESS increases with size at every peptide length and for every seed: on 2AA 0.27–0.29 (85M), 0.43 (160M), 0.47–0.48 (320M) and 0.66 (640M); on 4AA 0.09, 0.18, 0.22–0.24 and 0.40. The main text now points to this comparison.

Are any rows of Table 2 repeated with different seeds?

At the time of submission, no. Now every measured row of Table 2 is the mean of three sampling seeds, and new training replicates are in Appendix D.4.

  • Sampling seeds. Every measured row of Table 2 is evaluated three times on the same checkpoint, 10410^4 samples per system each. The table reports the mean and the largest deviation of a seed mean. Tables 4 and 15 also report means over three sampling seeds.
  • Training seeds (Table 11). We retrained every PBG size to D/N=48D/N=48 with three seeds for 30M–85M and two for 160M and 320M. Across training runs, rESS varies about 7 times as much as across sampling seeds, E-W2 and T-W2 about 2 times, and TICA-W2 about as much.

While preparing these replicates, we corrected the log-likelihood of reflected conformers in some rows of the submitted Table 2. Please see the global response for details.

What are the GPU-hours of the 640M run, and how many effective samples per second does MD or REMD produce on the same octapeptides and hardware?

The 640M model took 1,138 GPU-hours to train on 16 AMD MI250X GPUs. On the same H100 GPUs, PBG 640M converges at about 0.12 H100-hours per system, while MD needs 30 to 35 H100-hours to reach the same torsion and TICA errors and about one H100-hour for the energy error.

Table 26 now gives the GPU-hours of every Table 2 model, from 17 (49M) to 1,138 (640M). The 130 checkpoints of the scaling model took 1,259 GPU-hours on RTX PRO 6000 and H100 NVL GPUs (Appendix G).

For sampling, we do not report a ratio of effective sample sizes, because the importance-sampling ESS of a BG and the autocorrelation ESS of MD measure different quantities. Instead, we plot energy and torsion metrics against energy evaluations and H100-hours per system, for PBG 640M and for the MD that generated Many Peptides, both timed on the same H100 GPUs (new Figure 5, Appendix D.9). PBG 640M uses speculative decoding with the 15M drafter, which keeps the exact likelihood of the 640M model. We did not run REMD.

Concluding remarks

We thank the reviewer again for their time, the constructive review and the offer to raise the score. The conditioning ablation, the 30-system table, the seed replicates, the GPU-hours and the comparison with MD answer each question. The YFPHAGYT investigation also found an artifact in the Many Peptides data that affects previous transferable Boltzmann generators. We kindly ask the reviewer to reconsider their review in light of these new experiments and clarifications. We are happy to continue the discussion and provide any further clarifications.

Dear reviewer 4igz, We are very appreciative of your time and constructive feedback on our submission. We hope that our rebuttal responses have addressed your questions.

As the end of the rebuttal period is quickly approaching, we would be happy to answer any remaining or additional questions. If our responses and additional experiments address your concerns, we would be grateful if you would consider updating your assessment.

Reviewer_RNWAOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Prefix-Conditioned Boltzmann Generators (PBGs). A decoder-only Transformer uses a shared prefix specifying temperature, solvent, force field, molecular structure in HELM (with embedded SMILES for small molecules), and optionally known coordinates. It then autoregressively generates discretized three-dimensional atomic coordinates while retaining a tractable proposal likelihood for importance reweighting. The authors train about 130 models with 8M–85M non-embedding parameters on Many Peptides to study the effects of model size, delivered loss-token count, atom count, and coordinate resolution, then extend the series to 160M, 320M, and 640M non-embedding parameters. The 640M model performs strongly on rESS and on energy and torsion distribution metrics for peptides; speculative decoding reaches a maximum speedup of 2.14×. The paper also trains PBG-Joint across peptides, small molecules, and solvent environments, and explores solvation free-energy estimation and microsolvation around fixed solutes. The work establishes a flexible conditioning interface and reveals both scaling gains and a substantial atom-count bottleneck in peptide sampling. The current experiments do not yet establish beneficial cross-class transfer or reliable general molecular sampling.

Critical strengths

The paper provides a relatively systematic empirical study of discretized autoregressive Boltzmann generators at considerable scale. Beyond reporting the final model's performance, the authors vary model size, training loss-token count, and coordinate resolution, and analyze how sampling quality changes with atom count on a shared test set. In the appendix, the 640M PBG still substantially improves over ROBIN on rESS∗\mathrm{rESS}^{*} and E-W2∗\mathrm{E\text{-}W}_2^{*}, which use unclipped importance weights; the improvement in the main table is therefore not solely an artifact of weight clipping. The paper also reports that unclipped rESS∗\mathrm{rESS}^* remains low for octapeptides and that effective sampling efficiency declines steeply with system size. These negative results are useful to the field.

The analysis of quantization and reconstruction error gives the empirical trends a physical interpretation. Under the stated fine-grid and regularity assumptions, the authors derive a mean energy bias for midpoint reconstruction of E[ΔUmid]=βΔ2E[∥∇U∥2]/24+o(Δ2)\mathbb{E}[\Delta U_{\mathrm{mid}}]=\beta\Delta^2\mathbb{E}[\|\nabla U\|^2]/24+o(\Delta^2). If mean-squared force is approximately extensive, this bias grows roughly as nΔ2n\Delta^2. Quantization–reconstruction round trips of reference MD conformations support this prediction, whereas the error for generated samples grows faster with atom count. This distinction shows that coordinate quantization alone cannot explain the observed loss of sampling quality.

The shared prefix interface is practically appealing. HELM/SMILES, environment text, and coordinate tokens describe multiple molecular classes and conditions without a separate atom-wise encoder for each chemistry, and the interface permits water coordinates to be generated after a fixed solute prefix. The paper covers peptide and small-molecule sampling, implicit-solvent free-energy estimation, and explicit microsolvation, and reports hardware throughput for speculative decoding. These experiments demonstrate the flexibility of the approach, and the disclosure of datasets, architectures, and training configurations is helpful.

Critical weaknesses

The independent contribution of prefix conditioning has not been isolated. The paper presents prefix conditioning as a central methodological difference from ROBIN and reports better performance, yet its best-performing PBG also uses a larger model and a different training budget. There is no control that changes only the conditioning mechanism while matching model size, training loss tokens, coordinate resolution, and optimization settings. The results show that the scaled PBG family performs well, but they do not quantify how much of the gain comes from the prefix representation or whether removing the atom-wise encoder improves sampling at fixed resources.

The evidence for beneficial cross-class transfer is inconclusive. Comparing PBG-Joint with the 320M peptide-only model changes model size, dataset composition, and training compute. Relative to the 160M peptide-only model, PBG-Joint has lower rESS for dipeptides, tetrapeptides, and octapeptides, with no consistent improvement in the reported energy (E-W2\mathrm{E\text{-}W}_2) or backbone torsion (T-W2\mathrm{T\text{-}W}_2) metrics. Its proximity to the 320M model on some Wasserstein metrics could reflect cross-class transfer, but differences in compute allocation, regularization, or metric variability are also plausible. A stronger test would compare peptide-only training, a control mixture with additional peptide data, and joint training with small-molecule or cross-environment data, using the same 160M architecture and total compute. The authors should also report sampling proportions for each dataset and variation across random seeds.

Uncertainty in the scaling conclusions may be understated. The model constrains the parameter- and data-scaling exponents to be equal (α=β\alpha=\beta) and uses a generalized likelihood tilted by compute intensity. Checkpoints at different budgets along a shared training trajectory, as well as repeated measurements of the same test systems across models, may be correlated, while the stated likelihood does not explicitly model those correlations. Atom count also covaries with peptide length, chemical composition, and conformational complexity; a single log⁡(n/n0)\log(n/n_0) covariate may not separate size from system difficulty. Under these choices, the reported 95% interval for D∗/N=39.9D^*/N=39.9 is narrow (35.1–44.8), although extrapolation is checked with only the 160M, 320M, and 640M models, and the authors acknowledge the absence of validation on an independent dataset. These relationships are currently better viewed as empirical fits for Many Peptides under the present training setup. Sensitivity analyses that untie the exponents, remove compute tilting, add hierarchical system effects, and hold out one model size at a time would make the claims more convincing.

The scope of general molecular sampling remains limited. Results on short peptides are strong, but rESS∗\mathrm{rESS}^* is very low for complex small molecules, and sampling quality deteriorates with atom count, stereocenters, and rotatable bonds. In the explicit-solvent experiments, training data are generated with a periodic PME potential, whereas inference evaluates a cropped system under a nonperiodic reaction-field potential. This mismatch causes ESS to collapse. The resulting evaluation therefore demonstrates favorable water placement around fixed solutes, rather than reliable equilibrium sampling with explicit solvent. “Towards general molecular sampling” is a reasonable research direction, but the evidence does not yet support a stronger claim of generality.

Questions for authors

Independent contribution of prefix conditioning. With model size, loss-token count, coordinate resolution, and optimization settings matched, how would replacing ROBIN's atom-wise encoder with a text prefix affect rESS∗\mathrm{rESS}^*, E-W2∗\mathrm{E\text{-}W}_2^*, training speed, and extrapolation to longer systems? Could the authors add a resource-matched control that changes only the conditioning mechanism? If the primary advantage is interface flexibility rather than sampling quality on a fixed task, it would help to distinguish that advantage from gains due to scaling the model and data.

Beneficial cross-class transfer. Could the authors compare peptide-only and joint training with the same architecture and total compute, while controlling the amount and source of additional training tokens? Across data-mixture proportions and random seeds, how does joint training affect sampling quality separately for peptides, small molecules, and different environments?

Robustness of the scaling model. How much do the estimates of D∗/ND^*/N, the atom-count exponents, and CE-W2∗∝n5.56C^*_{\mathrm{E\text{-}W}_2}\propto n^{5.56} change when α=β\alpha=\beta is relaxed, compute-based likelihood tilting is removed, or hierarchical effects are introduced for test systems and shared training trajectories? Could the authors provide leave-one-model-size-out predictions or an independent test on MDQM9-nc to assess sensitivity to the modeling assumptions and the composition of Many Peptides?

3 comments
AuthorsOpenReview ↗

# Response to RNWA part 1

We thank reviewer RNWA for their thorough review and precise questions. We are glad that the reviewer finds our study "relatively systematic", values the physical interpretation of the quantization error, considers the shared prefix interface "practically appealing", and finds our negative results on sampling efficiency "useful to the field". The three requested controls are now in the paper. Below we address the questions raised during the review. All new and changed text in the revised PDF is marked in green.

Independent contribution of prefix conditioning. With model size, loss-token count, coordinate resolution, and optimization settings matched, how would replacing ROBIN's atom-wise encoder with a text prefix affect rESS, E-W2, training speed, and extrapolation to longer systems? Could the authors add a resource-matched control that changes only the conditioning mechanism?

At matched resources, the text prefix is as good as the atom-wise encoder at 30M and better at 85M, including on the octapeptides, our longest systems. It needs 16.5% more optimizer steps for the same loss tokens, which might explain part of the gain. Its main advantage is the flexibility of the interface, at no cost in sampling quality.

We added this control as the last step of a ladder from the ROBIN protocol to the PBG protocol (new Appendix D.6). Table 14 lists every difference between the two protocols. All nine rungs share the PBG network and see 48N48N loss tokens, at 30M and at 85M. The step from L7 to L8 changes only the conditioning, from atom features to the HELM prompt, at the same model size, loss tokens, 0.8 pm resolution and optimizer.

Table R.1. Conditioning control and ladder endpoints, unclipped metrics, means over test systems and three sampling seeds (Table 15 of the revised PDF).

SizeRung2AA rESS*2AA E-W2*4AA rESS*4AA E-W2*8AA E-W2*8AA Tg-SW2*
30ML0 ROBIN protocol0.00056.10.000185——
30ML7 atom features0.0102.320.0025.4321.80.910
30ML8 HELM prompt0.0142.050.0025.4220.60.897
85ML0 ROBIN protocol0.0015.560.00011.053.31.180
85ML7 atom features0.0622.810.0094.6112.30.708
85ML8 HELM prompt0.1201.250.0193.6210.60.653

Most of the gain over the ROBIN protocol comes from the optimizer settings (L1) and random translation (L7).

Beneficial cross-class transfer. Could the authors compare peptide-only and joint training with the same architecture and total compute [...]? Across data-mixture proportions and random seeds, how does joint training affect sampling quality [...]?

At matched architecture and compute, joint training neither improves nor harms peptide sampling. Our data therefore do not show beneficial transfer, and we revised our claims accordingly.

We trained a new 160M PBG on Many Peptides only, with the same architecture, optimizer, seed, batch rows and hardware as PBG-Joint 160M, and nearly equal loss tokens (Appendix D.7).

Table R.2. PBG-Joint 160M against PBG 160M (Many Peptides only), means over three sampling seeds (Table 16 of the revised PDF). Last column: test systems on which PBG-Joint is better, for rESS and E-W2.

SizerESS, joint / peptide-onlyE-W2, joint / peptide-onlyPBG-Joint better
2AA0.439 / 0.4420.278 / 0.2809/30, 18/30
4AA0.198 / 0.2110.580 / 0.5293/30, 9/30
8AA0.032 / 0.0344.096 / 4.04611/29, 16/29

PBG-Joint has slightly lower rESS at every length, and the torsion and TICA errors are mixed (Table 16). For scale, the two training seeds of the 160M model at D/N=48D/N=48 differ in rESS by up to 0.006 (Table 11). The revised abstract, introduction and Section 4.3 no longer claim transfer. We did not vary the data mixture.

AuthorsOpenReview ↗

# Response to RNWA part 2

Robustness of the scaling model. How much do the estimates of D*/N, the atom-count exponents, and α change when α=β is relaxed, compute-based likelihood tilting is removed, or hierarchical effects are introduced for test systems and shared training trajectories? Could the authors provide leave-one-model-size-out predictions or an independent test on MDQM9-nc [...]?

The atom-count exponents, the compute requirement at fixed energy error and the loss extrapolation are robust to every requested change. The compute-optimal ratio D∗/ND^\ast/N and α are not, and we now present D∗/N≈40D^\ast/N\approx40 only as a working allocation rule for Many Peptides.

Table R.3. Scaling model sensitivity on Many Peptides: posterior medians, with 95% CI for D∗/ND^\ast/N at N=85.5N=85.5M (Table 7 of the revised PDF).

Specificationα (β)D∗/ND^\ast/NCompute exponent at fixed E-W2*pEp_EHeld-out NLL RMSE
Default0.22740.5 [35.5, 45.5]5.612.170.0061
α ≠ β0.106 (0.454)44.8 [41.6, 48.1]5.392.170.0077
No compute tilting0.26829.6 [26.0, 33.2]4.302.090.0208
+ test-system effects0.21743.9 [38.9, 48.9]6.132.370.0061
+ training-lineage effects0.37126.0 [20.9, 31.4]4.782.380.0070
Priors on A, B instead of on D∗/ND^\ast/N0.337101 [86.6, 116]5.902.170.0036

We also refitted every specification five times, each time without one model size (Table 8). Across all fits, pEp_E stays between 2.06 and 2.39, and the held-out NLL RMSE has a median of 0.0074 nats. Excluding one specification that leaves the ratio unconstrained, D∗/ND^\ast/N ranges from 21.2 to 180. Even so, at ROBIN's size of 132M, every fit gives a D∗/ND^\ast/N far below ROBIN's D/N≈1100D/N\approx1100, which suggests that previous transferable Boltzmann generators were smaller than compute-optimal for their training budget.

Independent test on MDQM9-nc (new Appendix D.2). We evaluated 36 checkpoints from 8M to 160M and held out the 160M model. The training loss extrapolates to the held-out model (held-out NLL RMSE median 0.0096 nats), and D∗/ND^\ast/N is again far below ROBIN's (median 21.1). The energy error grows with atom count only when test-system effects are included (pE=1.73p_E=1.73). MDQM9-nc molecules have only 3 to 29 atoms, so this test has little power for the atom-count term.

The scope of general molecular sampling remains limited. [...] "Towards general molecular sampling" is a reasonable research direction, but the evidence does not yet support a stronger claim of generality.

We agree and narrowed our claims. We removed "Towards General Molecular Sampling" from the title, and the Limitations report the reduced sampling quality for small molecules and in vacuum.

Concluding remarks

We thank the reviewer for their time and careful review of our work. The resource-matched conditioning control, the matched transfer control and the sensitivity analysis have made our claims narrower and better supported. We kindly ask the reviewer to reconsider their review in light of these new experiments and clarifications. We are happy to continue the discussion and provide any further clarifications.

Dear reviewer RNWA, We are very appreciative of your time and constructive feedback on our submission. We hope that our rebuttal responses have addressed your questions.

As the end of the rebuttal period is quickly approaching, we would be happy to answer any remaining or additional questions. If our responses and additional experiments address your concerns, we would be grateful if you would consider updating your assessment.

Reviewer_SF6QOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

Recent Boltzmann Generators have scaled reliable equilibrium sampling to larger and more diverse molecular species, but they often specialise in limited molecular classes. The authors introduce Prefix-Conditioned Boltzmann Generators, a family of autoregressive BGs that enables conditioning on transferable molecular system generation through text and coordinate prefixes. PBGs are shown to generate conformational ensembles of peptides (canonical and non-canonical) and small molecules by conditioning the generation on amino acid sequences or SMILES strings. Moreover, the paper extends the use of BGs from conformational sampling to thermodynamic property estimation and microsolvation of fixed solutes (peptides or small molecules). PBGs also accelerate AR decoding using speculative decoding. This work also shows valuable insights into PBG empirical scaling characteristics. The authors will release the in-house simulated datasets Vacuum Peptides, Microsolvated Peptides, Many Cycles, and MD ensembles of FreeSolv small molecules.

Critical strengths

This work makes a strong contribution to general-purpose molecular conditioning. It demonstrates joint training across molecular classes, force fields, solvent environments, temperatures, and unit cells. To that end, the authors overcome a key limitation in current autoregressive BG conditioning schemes: conditioning information which limits chemical system flexibility. The first such model, Robin [2], used a noncausal transformer to embed the contextual atom embeddings which was then injected into the coordinate causal decoder. A big disadvantage of that was the use of one conditioning embedding per atom, making conditioning labels grow with atom size which is very inefficient compared to compacted atomistic sequence descriptors. Therefore, extension to new chemistries requires changes to featurisation and encoding. Therefore, PBG replaces the separate atomwise conditioning pathway with a unified textual interface: it uses the convenient encoding of the molecular system as a tokenised text prefix (followed by atomic coordinates during generation), which extends ARBG’s graph conditioning to serialized molecular-graph description, solvent name, force field name, and temperature. The molecular description mostly helps non-canonical peptides and small molecules (HELM for canonical peptides is very similar to Robin’s atomwise conditioning).

Another strong contribution of this work is the scaling analysis of sampling quality with respect to system size, model size, and discretisation resolution. It is by far the most detailed scaling study of BGs in the molecular data domain: they train 130 model variants and scale the model up to 640M parameters. Interestingly, they train a variant of the model, PBG-Joint, across molecular classes (small molecules and peptides) and despite using substantially fewer parameters and lower training compute, PBG-Joint matches or exceeds the peptide performance of the strongest specialist models. This suggests early signs of beneficial transfer across training contexts. Additionally, fairly small PBG models (down to 30M) are shown to perform overall on par with the previous SOTA much larger models like Prose [1] (285M parameters). This suggests high parameter efficiency.

Critical weaknesses

A major limitation of PBGs is its inability to overcome the poor scaling of sampling quality with molecular system size, a challenge also encountered by previous models [1, 2]. Although PBGS substantially improves upon Prose [1] and Robin [2], sampling quality in the 8-AA regime remains markedly worse than for 2-AA and 4-AA peptides, even for the largest 640M-parameter model. This limitation is further highlighted by the scaling analysis in Figure 2, which reveals a steep deterioration in sampling quality with increasing atom count: the fitted scaling model predicts that the training compute required to maintain a fixed energetic sampling accuracy grows as n^5.56, where n denotes the number of atoms. Moreover, the generalizability of this empirical scaling law beyond the relatively short peptides investigated remains unclear and the authors do not provide any scaling measurement of sampling quality with the data size or diversity (e.g. by training variants of PBGs on subsets of Many Peptides).

The authors also inaccurately present the limitations of previous models: they identify ARBG’s [2] requirement to specify the complete atom set and ordering prior to sampling as an important limitation of Robin, arguing that it prevents the generation of a variable number of solvent molecules. However, that seems to be a limitation of ARBG’s specific conditioning implementation rather than a fundamental restriction to the autoregressive framework. Furthermore, the proposed PBG framework does not overcome this limitation in the sense of determining the number of solvent molecules autonomously: for explicitly solvated systems, the number of atoms is still specified externally at inference. The contribution can be more accurately described as simplifying the conditioning of variable-size molecular systems.

The authors extend the use of BGs from conformational sampling to early thermodynamic property estimation (i.e. solvation free-energy estimation) estimating the free energy difference between the Boltzmann distributions of molecules simulated in vacuum and in implicit water respectively. The results for the peptide domain are, however, unconvincing due to using as reference labels an extremely simple sequence-based heuristic (GRAVY) which sums up pre-determined amino acid contributions purely based on the residue identity. This beats the purpose of using a complex and expensive to train BG to predict such a basic property. Authors need to use experimentally determined solvation free energies.

Regarding the microsolvation experiment, the authors are unable to demonstrate accurate Boltzmann sampling in the explicit solvent setting due to a mismatch between the training and evaluation potential, which they do acknowledge that it causes the effective sample sizes to collapse.

[1] Tan, C. B., Hassan, M., Klein, L., Syed, S., Beaini, D., Bronstein, M. M., Tong, A., & Neklyudov, K. (2025). Amortized Sampling with Transferable Normalizing Flows. Advances in Neural Information Processing Systems (NeurIPS 2025), 38, 104589–104624.

[2] Rehman, D., Tan, C. B., Bengio, Y., Bose, J., & Tong, A. (2026). Autoregressive Boltzmann Generators. Proceedings of the 43rd International Conference on Machine Learning (ICML 2026), PMLR 306, 104219–104250.

Questions for authors

To what extent do the observed improvements arise specifically from prefix conditioning rather than larger models, broader training distributions, and established inference techniques?

To what extent do you think the poor sampling quality for larger atom counts (8-AA peptides) is a data size or diversity bottleneck and is it inherent to the Many Peptides dataset?

Why do you only use net-neutral peptides in the microsolvation task?

5 comments
AuthorsOpenReview ↗

# Response to SF6Q part 1

We thank reviewer SF6Q for their detailed review. We are glad that the reviewer finds prefix conditioning "a strong contribution to general-purpose molecular conditioning", the scaling analysis "by far the most detailed scaling study of BGs in the molecular data domain", and the small PBG models highly parameter-efficient. Below we address the questions and weaknesses raised during the review. All new and changed text in the revised PDF is marked in green.

To what extent do the observed improvements arise specifically from prefix conditioning rather than larger models, broader training distributions, and established inference techniques?

Most of the improvement over ROBIN comes from improved model scaling and from changes to the optimizer and augmentation within the training protocol. Prefix conditioning adds a further gain at 85M and has little effect at 30M. Broader training data does not improve peptide sampling at matched compute (Appendix D.7, Table 16), and speculative decoding changes only the sampling speed, because it keeps the exact likelihood of the base model.

We separated these factors with a new ladder of nine models (Appendix D.6). Rung L0 trains the PBG network with the ROBIN protocol and ROBIN's atom features. Each further rung switches one difference to the PBG setting (Table 14), and rung L8 is a PBG. All rungs see the same 48N48N loss tokens.

Table R.1. Ladder from the ROBIN protocol to PBG at 85M, unclipped metrics, means over test systems and three sampling seeds (Table 15 of the revised PDF).

RungChange2AA rESS*2AA E-W2*4AA E-W2*8AA E-W2*
L0ROBIN protocol0.0015.5611.053.3
L1+ optimizer settings0.0014.066.7016.2
L6+ schedule, precision, packing, loss, quantization0.0013.755.9515.4
L7+ random translation0.0622.814.6112.3
L8+ HELM prompt (PBG)0.1201.253.6210.6
  • Training protocol. The optimizer settings (L1) and random translation (L7) account for most of the change.
  • Prefix conditioning. Replacing the atom features with the HELM prompt (L7 to L8) improves all three metrics at 85M and changes little at 30M.

To what extent do you think the poor sampling quality for larger atom counts (8-AA peptides) is a data size or diversity bottleneck and is it inherent to the Many Peptides dataset?

Generally, we attribute the poor sampling quality at larger molecular sizes to an increase in system dimensionality, supported by the robust identification of the atom-count scaling parameters. Within the available training data, the number of distinct training systems is a bottleneck and the simulated time per system is not.

We trained PBG 30M, 49M and 85M at D/N=48D/N=48 on 1/d1/d of the training frames, in two arms with the same number of frames and the same compute at each dd (new Appendix D.3, Figure 14). The depth arm keeps the first 200/d200/d ns of every trajectory and all 21,700 systems. The diversity arm keeps 1/d1/d of the systems with their full 200 ns trajectories. Shortening the trajectories down to 6.25 ns changed sampling quality little. Keeping fewer than 1/8 of the systems (2,713) had an observable effect, and with 681 systems or fewer, rESS* on unseen tetra- and octapeptides fell to its minimum of 10−410^{-4} and E-W2* rose by multiple orders of magnitude. Of course, model capacity also matters: at D/N=48D/N=48, 8AA rESS rises from 0.011 (85M) to 0.078 (640M) (Appendix D.4, Table 11).

AuthorsOpenReview ↗

# Response to SF6Q part 2

[...] the fitted scaling model predicts that the training compute required to maintain a fixed energetic sampling accuracy grows as n^5.56 [...] the generalizability of this empirical scaling law beyond the relatively short peptides investigated remains unclear

We agree that the steep scaling with atom count is the main limitation. The new analyses show that it is robust on Many Peptides, but not yet beyond peptides. Across all alternative specifications of the scaling model, each also refitted without one model size, the energy-error exponent pEp_E stays between 2.06 and 2.39, and the compute exponent at fixed energy error between 3.52 and 6.87 (Appendix D.1, Tables 7 and 8). On MDQM9-nc, whose molecules have only 3 to 29 atoms, the energy error grows with atom count only when test-system effects are included (Appendix D.2).

Why do you only use net-neutral peptides in the microsolvation task?

Because clusters of charged peptides would contain a counterion in some frames and not in others. The explicit-water MD of a charged peptide includes one Na+ or Cl- counterion per unit of net charge, and the training clusters keep only the 128 nearest waters. For 37 of the 44 charged test peptides, the counterion is inside this cluster in 10% to 90% of the frames (new Table 24). We therefore trained and evaluated the microsolvation model on net-neutral peptides only. The implicit-solvent free-energy estimates use all 90 test peptides, 44 of them charged (new Table 23).

they identify ARBG's [2] requirement to specify the complete atom set and ordering prior to sampling as an important limitation of Robin [...] the proposed PBG framework does not overcome this limitation in the sense of determining the number of solvent molecules autonomously

We agree with both points. The restriction comes from ROBIN's per-atom conditioning, not from the autoregressive factorization. PBG also does not choose the number of waters, which is set at inference (Section 3). What changes is that no per-atom features or solvent atom order must be built before sampling. We agree that the contribution is best described, in the reviewer's words, as simplifying the conditioning of variable-size molecular systems.

The results for the peptide domain are, however, unconvincing due to using as reference labels an extremely simple sequence-based heuristic (GRAVY) [...] Authors need to use experimentally determined solvation free energies.

We agree that GRAVY cannot validate free energies. We therefore validate against experiment on FreeSolv (Figure 6): BAR estimates give r=0.51r=0.51, and forward Zwanzig estimates without any MD samples of the molecule give r=0.66r=0.66. For the short sequences of Many Peptides, no experimental solvation free energies are publicly available, so GRAVY serves only as a sanity check that the estimates rank hydrophobic and hydrophilic sequences correctly (r=0.41r=0.41). We moved the solvation free energy results for peptides to Appendix D.11, which now states that no experimental values are available for them.

Concluding remarks

We thank the reviewer again for their time and careful review of our work. The suggested data ablation and the matched control for cross-class transfer have made the paper more precise about what drives sampling quality. We kindly ask the reviewer to reconsider their review, in particular the soundness score, in light of these new experiments and clarifications. We are happy to continue the discussion and provide any further clarifications.

Dear reviewer SF6Q, We are very appreciative of your time and constructive feedback on our submission. We hope that our rebuttal responses have addressed your questions.

As the end of the rebuttal period is quickly approaching, we would be happy to answer any remaining or additional questions. If our responses and additional experiments address your concerns, we would be grateful if you would consider updating your assessment.

Comments on the paper

2 comments
Area_Chair_fkMaOpenReview ↗

Authors: please respond to your reviewers

To the paper authors: Please acknowledge and respond to the thoughtful replies by reviewers. They have raised some going points about strengths and weaknesses of your paper, as well as questions that merit replies.

AuthorsOpenReview ↗

# Global response

We thank all reviewers for their careful reviews. The reviewers found our scaling study "by far the most detailed scaling study of BGs in the molecular data domain" (SF6Q) and "large and carefully documented" (4igz), valued the physical interpretation of the quantization error (RNWA, 4igz), and found the shared prefix interface "practically appealing" (RNWA). All three reviewers asked how much of the gain comes from prefix conditioning itself. Two asked whether joint training transfers across molecular classes, and two asked how robust the scaling model is. Below we summarize the new experiments and the changes to the revised PDF, in which all new and changed text is marked in green.

1. Prefix conditioning at matched resources (SF6Q, RNWA, 4igz). At matched model size, loss tokens, coordinate resolution and optimizer, the HELM prompt is as good as ROBIN's atom-wise encoder at 30M and better at 85M. Most of the remaining performance improvement over ROBIN comes from other changes in the training protocol, in particular the optimizer and the augmentation. A new ladder of nine models at two sizes changes the ROBIN protocol into the PBG protocol one step at a time (Appendix D.6, Tables 14 and 15).

2. Cross-class transfer (SF6Q, RNWA). At matched compute, joint training neither improves nor harms peptide sampling. A new 160M control trained on Many Peptides only has the same architecture, optimizer, seed, batch rows and hardware as PBG-Joint, and nearly equal loss tokens (Appendix D.7, Table 16). We accordingly removed the transfer claim from the abstract and the introduction, and removed "Towards General Molecular Sampling" from the title.

3. Robustness of the scaling model (SF6Q, RNWA). The atom-count scaling of the sampling error and the steep compute requirement at fixed error are robust. The compute-optimal ratio D∗/ND^\ast/N is less certain, and we now present it as a working allocation rule. We refitted the model under alternative specifications, including untied exponents, no compute tilting, and test-system and training-lineage effects, and refitted each of them without one model size (Appendix D.1, Tables 7 and 8). The energy-error exponent stays between 2.06 and 2.39. On an independent dataset, MDQM9-nc, the training loss also extrapolates to a held-out larger model and D∗/ND^\ast/N is again far below that of ROBIN (ca. 1000 D/N) (Appendix D.2), indicating that previous autoregressive BGs were overtrained/underparameterized.

4. System diversity against simulation depth (SF6Q). Within our budget, sampling quality is limited by the number of training systems, not by the simulated time per system (Appendix D.3, Figure 14). Shortening every trajectory to 6.25 ns changes little. Keeping 681 or fewer of the 21,700 systems collapses sampling on unseen tetra- and octapeptides. Please note that we never trained for more than one epoch.

5. Seeds and model size (RNWA, 4igz). Every measured row of Table 2 is now the mean of three sampling seeds. New training replicates at a matched budget show that rESS increases with model size at every peptide length and for every seed (Appendix D.4, Table 11).

6. Cost compared to MD (4igz). On the same H100 GPUs, PBG 640M converges at about 0.12 H100-hours per system. MD needs 30 to 35 H100-hours to reach the same torsion and TICA errors, and about one H100-hour for the energy error (new Figure 5, Appendix D.9). Table 26 now gives the training GPU-hours of every Table 2 model.

7. Identification of a long-standing stereochemistry artifact in the Many Peptides benchmark (4igz). The octapeptide YFPHAGYT has a D-proline in its reference MD that is not in its published starting structure, which is why we chose to exclude it. The default energy minimization inverts prolines that follow a phenylalanine, which also affects 276 training peptides. All models, including ROBIN, reproduce the artifact (Appendix D.5). Table 13 repeats Table 2 on all 30 octapeptides. We thank the reviewers for pointing us to this underlying issue in the dataset and believe it to be of interest to the community.

8. Technical Correction. While preparing the sampling seeds, we found that for some PBG rows of the submitted Table 2, the proposal log-likelihood of conformers reflected by the enantiomer correction was not that of the generated conformer, but of the reflected sample. All rows now use the generated conformer's log-likelihood, as the paper describes (Appendix G). This lowers the rESS of the affected rows slightly. The Wasserstein metrics change only within seed variation, and PBG 640M remains the best model by rESS at every peptide length.

We thank the reviewers for their constructive comments. We believe that the new experiments on conditioning, transfer, scaling robustness, and system diversity against simulation depth have made the paper much stronger.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.