- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper speeds up sampling of protein side-chain conformations. A learned autoregressive model, conditioned on the surrounding 3D structure, proposes changes to a block of side-chain angles (12 in the headline setting) drawn from a fixed, state-independent pool. Each move is accepted or rejected with a Metropolis–Hastings test on the full-system energy. These block moves are interleaved with ordinary Langevin MD. The model is trained on small proteins and applied unchanged to larger ones. Experiments on five proteins compare free-energy accuracy and sampling efficiency (ESS per GPU-second) against a rotamer-library sampler and a retrained learned sampler.
Critical strengths
As described (Table 23), the block moves are a valid Metropolis–Hastings step, so they preserve the right equilibrium distribution. The ordinary MD steps between block moves are not Metropolized in the headline runs (Table 24), but the authors measure that bias (Table 44) and repeat the comparison with Metropolized dynamics. The learned-baseline comparison is fair-minded. The authors retrain a Robin-style autoregressive proposal (their own reimplementation) on the same data and training budget, and still beat it by about 1.6–1.9× in ESS/GPU-s, including with Metropolized dynamics on three systems (Table 45).
Critical weaknesses
- The global-versus-local acceptance comparison behind the scaling claim can't be checked, and the paper's figures disagree on it. The abstract and Sec. 5.1 rest it on Fig. 2a. There, acceptance of a "global proposal" falls from 2×10⁻³ (N=20) to 3×10⁻⁹ (N=76), while BLOCS stays flat near 0.42. I could not find a description of that global proposal, or of which systems make up the ladder, beyond a one-line mention of a "global-reweighting stress test" (App. D.5). The paper's own Fig. 7e, over the same range of N, shows BLOCS dropping to about 0.16–0.17 at N=56–76. That is consistent with the real-protein acceptances in Table 36 (about 0.16 for Protein G and ubiquitin). Fig. 2a's flat 0.418 matches no protein-scale table. The direction (local beats global as size grows) is the expected one, and the complete-pool scan (Table 51) supports it. The problem is the specific headline curve, and no trained global sampler is compared on the scaling axis.
- Several efficiency numbers are hard to trace, and the classical baseline isn't tuned symmetrically. The BLOCS/rotamer speed-up is 3.0× in Table 16 (13-observable common pool), but 2.0× in Tables 12/30, where the pool is not stated. Rotamer MH is run at BLOCS's chosen block size and MD interval (m=12, r=100), tuned only in temperature (Table 11), and checked at one other setting (Protein G, m=4, Table 50). It is never swept over m or r, while BLOCS's settings were. BLOCS's free-energy errors (0.033–0.055 k_BT) are also comparable to the reference uncertainty (±0.034–0.057, Table 37), so the ranking against baselines is clearer than the absolute accuracy.
Overall: If the global comparator is defined, Figs. 2a and 7e are reconciled, and rotamer MH is run at its own best settings, I would raise to a weak accept.
Questions for authors
- What exactly is the global proposal in Fig. 2a, and which systems make up that ladder? Why do Figs. 2a and 7e disagree at the same N?
- How does rotamer MH do at its own best block size and MD interval, beyond temperature and the single Protein G m=4 point in Table 50?
- Table 17 names the pools for Tables 1/16/31/37/51, but not for Tables 12/30 or Figs. 2a, 3b, 3c, 4c, 7e. Please add those.
- Fig. 3a colours Trp-cage and Villin as training-range systems, while Table 14 lists them as held-out. Which is correct?
5 comments
Response to Reviewer ya2J : Condition 2, Rotamer MH at its own best settings; 3.0× vs. 2.0×
Symmetric tuning. We swept Rotamer MH over (30 cells) on Protein G, ubiquitin and lysozyme, re-tuning temperature at every cell (5 seeds per cell), and swept BLOCS over . Each uses the THERMO-COMMON construction rule at that , so the cell is the Table 1 run. Rotamer MH prefers small blocks and shorter MD intervals, and BLOCS prefers . Best cells (ESS/GPU-s):
| System | Rotamer best | Rotamer | BLOCS best | BLOCS | Best-vs-best | Canonical (12,100) |
|---|---|---|---|---|---|---|
| Protein G | (4, 50) | 0.641 | (8, 100) | 1.538 | 2.40× | 3.01× |
| Ubiquitin | (4, 50) | 0.552 | (8, 100) | 1.271 | 2.30× | 2.86× |
| Lysozyme | (2, 50) | 0.468 | (8, 100) | 0.989 | 2.11× | 2.69× |
| Protein G, complete eligible pool | (2, 50) | 0.551 | (4, 100) | 1.129 | 2.05× | 2.16× |
| Protein G, Metropolized dynamics | (4, 50) | 0.303 | (8, 100) | 0.672 | 2.22× | 2.79× |
Symmetric tuning lowers the gain (e.g. Protein G from 3.01× to 2.40×) and leaves it above 2× in every setting, including exact Metropolized composition. Tables 59 and 60 give the full grids, and the abstract and Sec. 5.4 now report both the canonical and the best-vs-best numbers.
3.0× (Table 16) vs. 2.0× (Tables 12/30). The two numbers come from different block populations, now named in every caption. THERMO-COMMON-M12 (Tables 1, 16) uses one common pool per system, with ESS as the median over its 13 locked observables. EXT-UNION-M12 (Tables 12, 20, 29, 30) uses, per system, the union of all frozen observable-centred candidate lists, run in the external-wrapper path required by the public checkpoints. With the ESS per target evaluations and the evaluations per GPU-s, the decomposition (Eq. 8) gives:
| Population | Rotamer / | BLOCS / | Statistical × execution | Ratio |
|---|---|---|---|---|
| THERMO-COMMON-M12 | 281 / 167.3 | 1216 / 116.0 | 4.33 × 0.693 | 1.410/0.470 = 3.00× |
| EXT-UNION-M12 | 312 / 167.3 | 912 / 114.6 | 2.92 × 0.685 | 1.045/0.522 = 2.00× |
The execution factor is the same in both populations (0.693 vs 0.685), and the difference lies in the statistical factor. Every block in an observable-centred list contains a slow residue, so Rotamer library jumps reach the slow rotamers directly ( 281 → 312). The same lists over-represent buried, tightly packed blocks, where BLOCS acceptance is lower ( 1216 → 912). On the common population, Rotamer MH, the matched AR and BLOCS reach 0.470, 0.805 and 1.410 ESS/GPU-s, now also given in the Table 12 caption.
Response to Reviewer ya2J : free-energy precision, pools, Q5
Errors vs. reference uncertainty. We ran two experiments. First, we extended the references by doubling the umbrella windows and running each window 4× longer:
| System | Observable | (orig. ± σ) | σ extended | Shift | BLOCS |
|---|---|---|---|---|---|
| Trp-cage | ARG16 | 0.87 ± 0.034 | 0.016 | 0.003 | 1.9 |
| Villin HP35 | PHE11 | −1.26 ± 0.041 | 0.019 | 0.004 | 1.9 |
| Protein G | TYR33 | −2.08 ± 0.047 | 0.022 | 0.002 | 1.9 |
| Ubiquitin | ILE44 | 0.73 ± 0.051 | 0.024 | 0.004 | 1.8 |
| Lysozyme | TRP62 | −1.51 ± 0.057 | 0.027 | 0.004 | 1.9 |
σ is halved and every reference value moves by less than , so all errors in Tables 1 and 16 are unchanged at the reported precision; over all 13 observables, re-scoring changes no system median by more than and changes no ranking. Second, we measured error against compute on Protein G with Metropolized dynamics (mean over 3 observables × 5 seeds):
| GPU-h | 1 | 2 | 4.0 | 6.7 | 7 | 12.3 | 24 |
|---|---|---|---|---|---|---|---|
| Rotamer MH | 0.61 | 0.44 | 0.31 | 0.236 | 0.225 | 0.098 | 0.071 |
| Matched AR | 0.29 | 0.21 | 0.160 | 0.100 | 0.094 | 0.074 | 0.058 |
| BLOCS | 0.21 | 0.142 | 0.099 | 0.056 | 0.052 | 0.041 | 0.031 |
BLOCS has the lowest error at every budget; its gap to Rotamer MH is at 1 GPU-h and at 24 GPU-h. BLOCS falls below ( for TYR33) after about 10 GPU-h, so its remaining error is resolved above reference noise. The 4.0, 6.7 and 12.3 GPU-h columns are the median times to in Table 45, and the 7 GPU-h column reproduces its error column; median first-passage time and checkpoint mean are different statistics.
Pools. Table 17 now defines all populations, and every caption names its own. Tables 12 and 30 use EXT-UNION-M12. Fig. 2a uses FIXED6-M12 (the submitted panel is CTRL-EMBED, App. D.5). Fig. 3b uses CORE-MULTIDOF (buried core residues, relative SASA < 10%, ). Fig. 3c uses THERMO-COMMON-M$m$ with each method at its best . Fig. 4c is now an illustrative event model, labelled as such. Fig. 7e uses LEGACY-PRODUCT-GLOBAL.
Q5 (Fig. 3a vs. Table 14). Table 14 is correct: Trp-cage and Villin HP35 are held out from the model behind Fig. 3a. The submitted bars come from the G1 checkpoint: the Trp-cage, Protein G, ubiquitin and lysozyme bars are the OBS-SPEC-CONTROL acceptances of Tables 36/38 (0.270, 0.159, 0.159, 0.141), and the BBA bar is the reference row of Table 43 (0.110). The "train" legend and shading were wrong for these values; they followed the training set of a legacy run (Chignolin, Trp-cage, Villin HP35) that produced only the submitted Fig. 7b and 7e. An earlier pilot produced the submitted Fig. 7c,d, which marked Protein G and ubiquitin as training proteins; those two panels are removed. No main-text result uses either run. In the revision, Fig. 3a uses the common FIXED6-M12 population with every protein marked held out and only the training size range shaded (Chignolin 0.296, Trp-cage 0.270, Villin 0.238, Protein G 0.154, ubiquitin 0.146, lysozyme 0.118), and the submitted held-out-structure bars (BBA, WW, Protein G, Protein L, ubiquitin, lysozyme) are kept, correctly labelled, as Fig. 7d.
Response to Reviewer ya2J: Condition 1, global comparator and Fig. 2a vs. Fig. 7e
Thank you for the precise conditions. We uploaded a revised PDF (changes in blue) and address each condition below with full results.
What the submitted Fig. 2a showed. The submitted "BLOCS" curve came from a controlled embedded-block ladder (CTRL-EMBED), not from real protein pools. It used Trp-cage, Villin HP35, Protein G and ubiquitin (N=20,36,56,76; the tick "35" was a labelling error). In each system BLOCS moved one fixed two-torsion block, the alanine-dipeptide block of App. F.4, grafted as a solvent-exposed pendant more than 9 Å from every protein atom. Its context, and hence its acceptance, is independent of N by construction, and 0.418 is the isolated-block value (Table 28). The global comparator was a per-system neural-spline flow (8 coupling layers, 4.1M parameters) over all eligible side-chain torsions, fitted by forward KL on 50 ns of that system's MD, with one full-system MH step per draw. Its Rao–Blackwellised acceptance α^=n−1∑imin(1,eSi) falls from 2.0×10−3 at N=20 to 3.1×10−9 at N=76. The panel now appears in App. D.5 under the CTRL-EMBED label.
New real-protein ladder (main-text Fig. 2a,b). We ran six held-out proteins (FIXED6-M12) with three proposals, each with one full-system MH step and r=100. (a) In the all-block joint proposal, the frozen BLOCS network proposes all eligible side-chain torsions at once, so proposal scope is the only variable that differs from (c). (b) The trained global proposal is a transferable autoregressive model over all torsions, retrained on our corpus with the same 39.4 GPU-h budget. (c) BLOCS uses m=12. For (a) and (b), acceptance is the RB estimate from 2×105 attempts per seed (5 seeds), with realised accepts from seed 1 in the last column:
Changing only the proposal scope moves acceptance at N=129 from 1.7×10−14 to 0.118. The trained global proposal improves on the joint proposal by about 10× and collapses in the same way, and at N≥56 both are at or below MD alone on Protein G (0.0044 ESS/GPU-s). BLOCS acceptance falls by 23% from Protein G to lysozyme, consistent with the 17.4% size penalty at fixed m in Table 51 and with the ≈0.16 values the reviewer cited.
Reconciling Fig. 7e. Fig. 7e comes from a separate legacy run whose training set contained Chignolin, Trp-cage and Villin HP35, and it uses a different proposal from Fig. 2a. Its local curve is single-χ1 acceptance: 0.50–0.54 on those three training proteins and 0.191 / 0.157 on held-out Protein G / ubiquitin. Its global curve is a weight-overlap estimate for a product-form global proposal, which is zero at N=56,76. The drop between N=36 and 56 coincides with that run's train/held-out boundary, so Fig. 7e does not isolate size dependence; with every protein held out, size dependence is measured by the new Fig. 2a and Table 51. Fig. 7 and App. D.5 now label the legacy run, its training set and its estimator.