- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors use an approach that separates geometric perception from probabilistic generation named SEAL-BG to address the computation and scaling bottleneck of ArBG. SEAL-BG based on an SE(3)-equivariant causal Transformer encodes the Cartesian coordinates of previously generated atoms, while normalized local-coordinate conditionals model bond lengths, bond angles, and periodic torsions. The authors demonstrate that SEAL-BG achieves lower energy distance and competitive torsion metrics on smaller peptide benchmarks, and improves energy agreement on the larger Trp-cage and Homeodomain systems, compared with ArBG, with much less training compute.
Critical strengths
Critical strengths include:
-
the novel design of separating perception from generation: SEAL-BG uses Cartesian coordinates for global geometric information but generates atoms in chemically meaningful local coordinates. Global Cartesian geometry informs the model about the entire portion generated so far, while local coordinates tell the model how to place the next atom in a chemically sensible way. So SEAL-BG holds advantages of long-range conformation and realistic local molecular structure.
-
large training efficiency gains shown across tested systems. SEAL-BG shows 15.2× fewer training floating-point operations (FLOPs) and 18.1× fewer A100 GPU-hours than ArBG, while keeping competitive or even improved energy agreement.
[Strengths] Other strengths are as follows:
-
the authors show that SEAL-BG can work on larger proteins, in addition to testing their model on small alanine peptides. The authors tested it on Homeodomain, which contains 947 atoms. This is much larger than the small peptides commonly used to test similar models.
-
SEAL-BG treats globally rotated versions of the same protein conformation the same, rather than different conformations.
-
Good presentation flow. The paper shows normalization and geometric covariance, detailed experimental settings, model hyperparameters, and simulation protocols to enable reproducibility.
Critical weaknesses
A critical weakness is that the paper’s corrected performance depends on clipped importance sampling without reporting standard weight-quality diagnostics, such as effective sample size before and after clipping, the fraction of total weight removed, or sensitivity to the clipping threshold. Also, how clipping shifts bias-variance tradeoff is not reported. It is therefore unclear whether the strong corrected metrics reflect broad overlap with the Boltzmann target or a small number of dominant samples.
[Weaknesses] Other weaknesses are: Limited ablation study: the authors only performed the full SEAL-BG and SEAL0BG without log-dispersion regularization on AL3, 4, and 6 and Chignolin using raw and clipped SNIS energy and torsion Wasserstein distances. Unclear statistical uncertainty: the authors did not report variation results across trained models or confidence intervals. So we do not know if small differences from ArBG are statistically reliable. No limitations addressed in the discussion/ conclusion. Although several limitations or caveats are mentioned throughout the paper, there is no dedicated section discussing these limitations and appropriately narrowing its claims.
Questions for authors
[Questions]
- Can you perform controlled, one-component-at-a-time ablations to better identify the source of SEAL-BG’s gains, such as equivariance ablation by replace the SE(3)-equivariant encoder with a standard causal Transformer and coordinate generation ablation by replacing local-coordinate density head with a Cartesian-coordinate head?
- Importance-weight reliability. What are the effective sample sizes, maximum normalized weights, and fractions of total weight removed before and after clipping for each system? Do the main conclusions hold without clipping or under different clipping thresholds? How does removing the top 0.2% of weights affect the bias and variance of the reported corrected metrics across independent proposal batches?
- Statistical reliabilty: report uncertainty metrics to show how reliable the differences are.
3 comments
W3 + Q3. Statistical reliability and uncertainty
We thank the reviewer for raising this point. We added SEAL-BG results over 3 seeds with error bars and retained the published uncertainty estimates for all baselines.
| Method | AL3 E-W2 | AL3 T-W2 | AL4 E-W2 | AL4 T-W2 | AL6 E-W2 | AL6 T-W2 | Chignolin E-W2 | Chignolin T-W2 |
|---|---|---|---|---|---|---|---|---|
| ECNF++ | 2.206 ± 0.813 | 0.962 ± 0.253 | 5.638 ± 0.483 | 1.002 ± 0.061 | 10.668 ± 0.285 | 1.902 ± 0.055 | — | — |
| RegFlow | 0.853 ± 0.105 | 1.577 ± 0.140 | 3.277 ± 0.546 | 2.342 ± 0.102 | — | — | — | — |
| SBG (SMC) | 0.598 ± 0.084 | 0.503 ± 0.029 | 1.007 ± 0.382 | 1.039 ± 0.069 | 1.189 ± 0.357 | 1.444 ± 0.140 | 10.819 ± 7.206 | 3.778 ± 0.440 |
| FALCON-A | 1.385 ± 0.182 | 0.343 ± 0.004 | 2.929 ± 0.068 | 1.094 ± 0.034 | 1.211 ± 0.105 | 1.163 ± 0.112 | — | — |
| FALCON | 0.544 ± 0.013 | 0.452 ± 0.011 | 0.686 ± 0.047 | 0.858 ± 0.077 | 0.892 ± 0.311 | 1.256 ± 0.132 | — | — |
| GIVT | 1.354 ± 0.058 | 0.343 ± 0.008 | 1.033 ± 0.449 | 1.113 ± 0.100 | 1.206 ± 0.056 | 1.527 ± 0.048 | 45.646 ± 20.989 | 3.031 ± 0.098 |
| MoL-PixelCNN++ | 0.506 ± 0.082 | 1.024 ± 0.686 | 1.643 ± 0.504 | 1.415 ± 0.110 | 1.429 ± 0.186 | 1.264 ± 0.205 | 140.717 ± 49.113 | 3.391 ± 0.093 |
| GMM-PixelCNN++ | 0.249 ± 0.025 | 0.364 ± 0.016 | 1.434 ± 0.783 | 0.806 ± 0.056 | 1.164 ± 0.037 | 1.285 ± 0.058 | 23.339 ± 6.485 | 3.007 ± 0.086 |
| ArBG | 0.202 ± 0.010 | 0.312 ± 0.003 | 0.449 ± 0.030 | 0.592 ± 0.010 | 0.328 ± 0.122 | 1.094 ± 0.052 | 1.723 ± 0.075 | 2.632 ± 0.044 |
| SEAL-BG | 0.178 ± 0.031 | 0.336 ± 0.010 | 0.397 ± 0.102 | 0.586 ± 0.028 | 0.345 ± 0.051 | 1.197 ± 0.042 | 1.853 ± 0.175 | 2.715 ± 0.038 |
Across the four systems, SEAL-BG stays close to ArBG in both E-W2 and T-W2
while consuming 39.807, 50.666, 76.437, and 346.910 PFLOPs, versus
463.856, 605.015, 1,053.223, and 11,891.405 PFLOPs for ArBG. This
balance between comparable accuracy and lower training cost makes SEAL-BG
better suited to scaling up system size and cheaper free-energy calculations.
W4. Missing limitations
We thank the reviewer for raising these points. We will discusses the limitations concerning energy-component agreement and the fixed generation order in conclusion.
Finally, we are grateful for the reviewer's constructive assessment, which led us to strengthen the empirical evidence and sharpen our claims. We hope the revisions resolve the concerns raised. In addition, following a suggestion from Reviewer yEYC, we trained a transferable SEAL-BG model, which performs strongly on unseen peptide systems. Those interested can find further details in our response to Reviewer yEYC. We would appreciate the reviewer's consideration of an increased score.
We thank the reviewer for the positive assessment and constructive feedback. We address the concerns below.
W1 + Q2. Clipped importance sampling and weight reliability
We thank the reviewer for isolating this issue. We primarily follow the default
0.2%clipping setting in ArBG to maintain consistency with their evaluation protocol. Also, in ArBG, ESS is treated as a metric with the caution that ESS is not a standalone measure of Boltzmann sampling quality, since ESS can favor mode-seeking proposals and, in high dimensions, may be dominated by a few overlap samples or numerical outliers. Likewise, log-weight variance, maximum normalized weight, and removed weight mass are useful metrics of weight concentration, but they can also be dominated by a few overlap samples or numerical outliers. Despite this, we added a weight-reliability panel covering full SNIS and both clipping settings. The results are below.In relative terms, SEAL-BG gives better weight concentration. Across the clipped settings SEAL-BG has higher ESS, lower log-weight variance, lower maximum weight, and lower removed mass, although the absolute values remain low for both methods, illustrating the limitation that these metrics can be dominated by a small number of samples. It also has lower E-W2 at all thresholds and lower T-W2 under clipping. The main comparison is therefore stable across thresholds.
W2 + Q1. Controlled component ablations
We thank the reviewer for this suggestion. Similar to our responses to Reviewers yEYC and Kjyv, we added two controlled ablations. The first replaces SEAL-BG's equivariant encoder with ArBG's plain causal Cartesian Transformer while retaining the local-coordinate head. The second retains the equivariant encoder and replaces the local-coordinate density with a Cartesian density head.
Both replacements increase E-W2 and T-W2 on every system. The Cartesian density head produces the larger E-W2 degradation. These results show that only the combination of the equivariant encoder and local-coordinate density attains the E-W2 and T-W2 of SEAL-BG.