acceptodds
Under review as a conference paper at ICLR 2027

Sampling CDR-H3 Ensembles with Antibody Language Models and Diffusion

Abstract

Antibodies derive their binding specificity primarily from the third complementarity-determining region of the heavy chain (CDR-H3), a hypervariable loop characterized by extreme sequence diversity and conformational flexibility. Despite its functional importance, accurate structural prediction of CDR-H3 remains an open problem. Here we present H3Fold, a fast generative framework for CDR-H3 structure prediction that couples antibody-specific language models with a denoising diffusion model. Sequence context is provided by two pre-trained encoders: H3BERTa, trained on over 18M CDR-H3 sequences, and AbRoBERTa, trained on over 57M full-length variable regions. To address the scarcity of high-resolution antibody structures, we train the diffusion trunk under a progressive curriculum that transitions from general protein backbone loops to CDR-H3 loops. Systematic ablations isolate the contribution of each architectural choice to structural quality. On benchmark splits filtered for strict sequence identity to the training set, H3Fold generates geometrically plausible, low-RMSD conformations at a fraction of the inference cost of existing predictors. Since the diffusion trunk samples structural ensembles, it can integrate directly into iterative antibody design pipelines where speed and conformational diversity are decisive.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Diffusion Models, Protein Structure Prediction, Antibodies, Protein Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_y1XwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents H3Fold, a conditional diffusion framework for generating antibody CDR-H3 Cα conformations. It combines loop-specific representations from H3BERTa with VH sequence representations from Ab-RoBERTa. A four-stage curriculum transfers geometric knowledge from general protein loops to immune loops and finally CDR-H3 structures. The method also introduces a stem-grounded canonical coordinate frame for loop placement. The evaluation includes architectural and training ablations, sequence-clustered internal splits, and an external temporal holdout. The results indicate promising inference efficiency, while revealing a substantial gap between internal and external performance and between oracle and practical candidate selection.

Critical strengths
  1. A well-motivated approach to efficient loop modeling. The combination of loop-local and VH-level sequence conditioning, progressive training, and a stem-grounded coordinate frame is technically reasonable. Focusing computation on CDR-H3 is relevant to iterative antibody engineering, where repeated full-structure prediction can be costly. The framework therefore has plausible practical value even without achieving the best single-structure accuracy.

  2. Substantial empirical investigation and transparent failure analysis. The ablations provide useful evidence about the importance of pretraining, immune-loop data, and sequence conditioning. The external holdout and the investigation of the internal–external performance gap are particularly valuable. Reporting unresolved failures, including the difficulty of cysteine-containing loops and the unexplained residual performance gap, makes the evaluation more informative than an internal benchmark alone.

Critical weaknesses
  1. The central accuracy–efficiency advantage is insufficiently established against directly relevant alternatives. The benchmark primarily compares H3Fold with full-antibody structure predictors, whereas H3Fold generates only a CDR-H3 Cα trace. Although Appendix H uses the same GPU, the methods perform different amounts of work and produce different outputs. A dedicated loop predictor such as ABlooper, discussed in the related work, would provide a more direct comparison with conditioning requirements explicitly accounted for. This matters because practical candidate selection remains less accurate than several evaluated predictors: on the common 69-case holdout, Table 4 reports median Kabsch RMSDs of 2.95 Å for a random H3Fold sample and 2.82 Å for centroid-based selection, compared with 2.19 Å for ESMFold. The 2.31 Å oracle result shows the potential benefit of selecting a better candidate from the generated ensemble, but this selection requires access to the reference structure. A controlled accuracy–runtime comparison is needed to establish the proposed method’s practical advantage.

  2. The evaluation provides limited evidence for the additional value of the generated ensembles. The internal validation and test sets retain one reference structure per unique CDR-H3 sequence, and the main metrics primarily assess agreement with that reference. Best-of-five RMSD measures the accuracy of the best sampled candidate relative to a reference structure, but it does not establish recovery of distinct plausible conformations for a given input. The aggregate pseudo-dihedral analysis offers some geometric evidence, yet sequence-conditioned coverage, diversity versus accuracy, and structural validity in the surrounding framework remain insufficiently characterized. These evaluations are consequential because ensemble generation is a central motivation of the paper and could justify the method despite its weaker practical single-candidate accuracy.

Questions for authors
  1. Direct comparison and practical advantage: Can the authors provide a comparison with a dedicated loop-modeling method such as ABlooper, explicitly accounting for differences in required inputs? An accuracy–runtime comparison under matched time or sampling budgets would help determine whether H3Fold offers a meaningful advantage for the intended workflow.

  2. Evidence for ensemble utility: What evidence shows that the generated samples cover distinct plausible conformations for the same input, beyond improving RMSD to one reference? Where suitable experimental conformations are available, a coverage analysis would be useful. Otherwise, can the authors provide a joint assessment of diversity, reference accuracy, and framework compatibility as the sampling budget increases?

  3. Inference-time clarification: Section 3.3 reports approximately 3.5 seconds for five candidates, whereas Table 11 reports 0.70 seconds per Fv and states that five samples were generated per loop. Does 0.70 seconds cover one candidate or all five? Please clarify the batching strategy, whether runtime is normalized per candidate or per input Fv, and which preprocessing steps are included. For placing generated loops into an antibody framework, which stem/backbone coordinates are assumed available, how are they obtained in the intended inference workflow, and is this cost included in the reported runtime?

  4. Consistency of random and oracle statistics: In Table 10, some subgroups have a higher oracle median than random-sample median; for example, the holdout Q1 subgroup reports 3.382 Å versus 3.138 Å. Is the random sample drawn from the same five-candidate pool used for oracle selection? If so, these values appear inconsistent with the best-of-five definition. If different draws or aggregation procedures are used, please explain the protocol and its implications for the gap analysis.

1 comment

W1 — Accuracy-efficiency comparison

Three loop-local baselines are added with conditioning requirements made explicit: H3Fold, ABlooper, Ibex and ABodyBuilder3 all require a framework or stem coordinates; IgFold, ESMFold, ColabFold, Chai-1 and tFold predict the full Vᴴ from sequence. The benchmark table is split into these two blocks.

Ibex 1.187 Å (best-of-8: 0.846, n=68), ABodyBuilder3 1.893 Å (n=68), ABlooper 2.671 Å (oracle 2.125, n=45). All three exceed our random sample; Ibex and ABodyBuilder3 exceed our oracle. Leakage is disclosed as described under Reviewer 2, W2. The SOTA accuracy claim is withdrawn. The paper is reframed around cheap, diverse, stem-grounded Cα ensemble generation at 0.85 s per candidate.

Paired Wilcoxon over the 69 shared holdout loops: our random sample vs ColabFold Δ=+0.747 Å (p=6.3×10⁻⁶); vs ESMFold Δ=+0.756 Å (p=1.8×10⁻⁴); vs tFold Δ=+0.659 Å (p=4.3×10⁻⁵). Oracle vs oracle: ColabFold still ahead (1.803 vs 2.307 Å, p=6.6×10⁻⁴).

W2 — Ensemble utility

A coverage analysis is not feasible with the available data, as described under Reviewer 1, Q3. No calibrated-ensemble claim is made.

The gap between oracle and reference-free selection as budget increases is characterised. Oracle falls from 2.265 to 1.894 Å from N=5 to N=25 on the holdout. Centroid moves from 2.917 to 2.609 Å. Three rankers were tested: centroid closes 49 % of the rand-to-oracle margin on validation but only 34 % on the holdout; a geometry-violation score closes 26 % on validation and costs 22 % on the holdout; mean-Cα ensemble selection closes 28 %/27 %. These results characterise the current limit on reference-free selection rather than making a positive ensemble claim.

Q1 — Direct comparison with loop-local methods

ABlooper, ABodyBuilder3 and Ibex are added with explicit conditioning requirements and leakage disclosure (see W1 above). The honest summary: at equal or greater compute cost, Ibex reaches 1.19 Å single-sample against our 2.95 Å random. H3Fold's advantage is in sample speed per candidate and stem-grounded placement, not in accuracy.

Q2 — Evidence for ensemble utility

No coverage evidence exists in our data for the reason described under W2. The joint assessment is delivered as (1) the N=25 accuracy–runtime curve under oracle and centroid selection and (2) the placement analysis giving terminus error, span error and framework clash rate in both crystal and predicted Fv contexts.

Q3 — Inference-time scope

The 0.70 s figure was per candidate; five candidates cost 3.5 s. Re-measured unbatched on A40: 0.851 s at N=1, 4.168 s/Fv at N=5. The H3BERTa and Ab-RoBERTa passes cost 19 ms (0.5 % of the N=5 total) and are included. DDIM sampling alone is 4.122 s. Table 11 now carries explicit per-candidate and per-Fv columns with PLM-included flags.

Stem coordinates are supplied from the framework (IMGT 104/118 Cα). Their cost is not in Table 11. The placement analysis quantifies what using a predicted framework costs: IgFold 26.1 % of termini beyond 1 Å; ABodyBuilder3 13.2 %; crystal framework 7.7 %.

Q4 — Oracle-random inversion

Unfortunately, the oracle column in Table 10 was DirectSel throughout. DirectSel is the sample with the lowest Direct RMSD (no superposition), not argmin Kabsch, and it can exceed a random draw. Under the true argmin-Kabsch oracle, holdout Q1 gives rand 3.138 Å, oracle 3.048 Å. The inversion disappears. Two further inversions found in the same audit are corrected (see Reviewer 2, W1). The evaluation pipeline now enforces oracle ≤ centroid ≤ rand per loop before writing results.

The L12–15 "model-level failure" open problem in §4.3.1 and the Conclusion rested on a 2.5 % DirectSel gain. The true argmin-Kabsch gain is 26 % (2.193 to 1.624 Å), comparable to the test set. The open-problem claim is withdrawn.

Reviewer_VyKmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

H3Fold samples C-alpha structures of antibody CDR-H3 loops using a diffusion transformer and two antibody language models. Training progresses from general protein loops to antibody loops, using a coordinate frame defined by the loop anchors. In the common external benchmark, centroid-selected predictions have median Kabsch C-alpha RMSD 2.82 Angstrom, versus 2.20 Angstrom for ColabFold.

Critical strengths
  • Restricting prediction to CDR-H3 is useful for repeated loop modelling. The reported runtimes suggest a substantial saving against the tested full-structure predictors, although the sample-count convention needs clarification.
  • The explicit anchor-frame construction and separate Direct, Roll and Kabsch metrics help distinguish placement from loop-shape error.
  • The broad ablation grid examines encoders, training data and geometric losses. The paper openly reports the external accuracy gap and investigates several possible causes.
Critical weaknesses
  • Selection reporting needs clarification: Table 3's 1.45 Angstrom test result matches Table 5's oracle value, but its selection rule is unstated. Table 10's holdout-Q1 oracle median exceeds its random median (3.382 versus 3.138 Angstrom), which requires explanation if both use identical candidate pools and scoring.
  • The benchmark omits relevant cited methods, particularly the fast loop predictor ABlooper, as well as ABodyBuilder3 and Ibex.
  • Appendix F.3 explicitly uses C-alpha torsion proxies, but does not validate its canonical alpha/beta regions for those coordinates. The 75.2% "non-canonical" fraction therefore needs qualified biological interpretation.
Questions for authors
  • Can you provide per-loop predictions and selection scores to reconcile Tables 3, 5 and 10, including whether random/oracle candidate pools are identical?
  • Was Ab-RoBERTa conditioning active in Stage 1, and were stage-removal ablations matched for optimization budget?
  • What identity definition and cross-split pairwise checks accompany the 60% clustering, given Table 10's 39 test loops above 62% identity?
  • Do the 0.70 s/Fv in Table 11 and 3.5 s for five samples in Section 3.3 differ in batching or timing scope?
1 comment

W1 — Selection rule and oracle inversion

Unfortunately, the oracle column in both tables was mislabelled. We reported DirectSel, the RMSD of the sample with the lowest Cα RMSD without superposition. DirectSel is not a lower bound and can exceed a random draw. This explains the holdout Q1 inversion (rand 3.138 Å, DirectSel 3.382 Å). Under the true argmin-Kabsch oracle, holdout Q1 gives rand 3.138 Å, oracle 3.048 Å. The inversion disappears. All results tables are corrected as described under Reviewer 1, W1.

In addition, we audited all subgroups and found two further inversions: holdout crystal-contact 5–10 Å (rand 2.377, DirectSel 2.395, oracle 2.119 Å) and test crystal-contact >15 Å (rand 1.988, DirectSel 2.047, oracle 1.572 Å). Both resolve under the true oracle. The evaluation pipeline now asserts oracle ≤ centroid ≤ rand per loop on every split before writing results.

W2 — Missing baselines

All three are added, scored through the same IMGT-anchor Cα extractor as our own predictions. Single-sample Kabsch on the holdout: Ibex 1.187 Å (best-of-8: 0.846 Å, n=68), ABodyBuilder3 1.893 Å (n=68), ABlooper 2.671 Å (oracle 2.125 Å, n=45). All three exceed our random sample; Ibex and ABodyBuilder3 exceed our oracle.

Training-set overlap is disclosed in the paper. ABodyBuilder3 trained on SAbDab through January 2024 and likely saw 32/68 scored structures. Ibex trained on a 2025 public dump and likely saw 66/68. These rows are not leakage-free and cannot be read as out-of-distribution comparisons. ABlooper processed 46/69 structures; its median covers a different and likely easier subset, disclosed in the table footnote. The benchmark table is split into loop-local methods (H3Fold, ABlooper, Ibex, ABodyBuilder3) and whole-Vᴴ predictors (IgFold, ESMFold, ColabFold, Chai-1, tFold).

W3 — Pseudo-Ramachandran validity

The α/β boxes are not valid for four-Cα virtual torsions. The section will replace with an empirical Cα-specific reference as described under Reviewer 1, W2.

Q1 — Per-loop reconciliation

The pools are identical: one persisted five-sample file per split, with rand fixed as sample index 0. The inconsistency arose from using DirectSel in some tables and true oracle in others, not from different candidate draws. Per-loop predictions and per-rule selection scores for all three tables are included in the reproducibility materials alongside the checkpoint SHA256 and DDIM configuration.

Q2 — Ab-RoBERTa gating and ablation budget

Ab-RoBERTa was gated off at Stage 1. get_phase2_scale(1) = 0, so the cross-attention sublayer is skipped entirely and no Vᴴ context is consumed. The backbone is loaded in no-grad mode but contributes nothing to the forward pass. §3.2 was correct; references to "frozen Ab-RoBERTa" in §3.5 and Table 1 were wrong and are corrected.

The stage ablations were not run with matched optimization budgets and cannot be corrected without a full retrain. The paper now treats the +2.46 Å pretraining effect as an upper bound: it conflates removing the curriculum with training for fewer steps.

Q3 — Identity definition and cross-split checks

Two identity measures were used without being clearly distinguished. CD-HIT clustering identity governs the train/validation/test split. Best-match identity, used for the holdout filter and quartile stratification, is computed over CDR-H3 sequences as prefix matches normalised by max(len_H3, len_train). The Methods section now defines both.

No exact-sequence leakage: train ∩ {validation, test, hard, holdout} is empty. Holdout maximum identity to Stage-3 training is 0.667; all 71 structures fall below 70 %. Test maximum is 0.95, with 21/162 above 70 %. The test split is not a <70 % split; the paper now states this rather than allowing the holdout's stringency to transfer implicitly. Quartile edges are 0.467 / 0.545 / 0.625.

Q4 — Timing

The 0.70 s and 3.5 s figures described the same measurement at different normalisations. Table 11's caption mislabelled the per-candidate figure as per-Fv. Corrected as described under Reviewer 1, W1.

Reviewer_4uHUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

H3Fold generates CDR-H3 C-alpha traces with a diffusion transformer conditioned on H3BERTa and VH-context Ab-RoBERTa representations. A four-stage curriculum transfers from general protein loops to antibody loops, and an anchor-defined canonical frame fixes coordinate orientation. The paper presents extensive ablations, sequence-filtered internal tests, and a temporal external comparison against antibody and general structure predictors.

Critical strengths
  1. Fast, localized loop resampling is a useful objective for iterative antibody modeling. The model is compact relative to full-structure predictors, and the curriculum and conditioning ablations show that both general-loop training and antibody context contribute to the reported accuracy. This is a practical contribution even without beating every baseline on point accuracy.

  2. The authors report the external-holdout performance degradation rather than hiding it, and analyze length, cysteine content, crystal contacts, and sequence similarity. The paper also distinguishes random, centroid-selected, and oracle samples in the external comparison, and includes pretraining exclusions against the CDR-H3 test sequences. These are valuable evaluation practices.

Critical weaknesses
  1. The operational accuracy and speed tradeoff is not consistently communicated. Table 3 presents an internal-test Kabsch RMSD of approximately 1.45 Å without identifying it there as oracle selection; Table 5 identifies the corresponding 1.448 Å result as best-of-5 oracle and gives 1.937 Å for a random sample. On the common external set in Table 4, random and centroid-selected predictions are 2.95 and 2.82 Å, while oracle selection reaches 2.31 Å. The useful question is whether a selectable, geometrically placed loop achieves a favorable cost/quality tradeoff; knowing that one of five samples is closer to the reference does not answer it. Timing also needs reconciliation: Section 3.3 says five candidates take about 3.5 seconds, whereas Table 11 reports 0.70 seconds per Fv with a caption saying diffusion models ran five samples. This ambiguity matters directly to the main practical claim.

  2. The physical interpretation of the pseudo-Ramachandran analysis is not justified as written. Appendix F.3 computes two torsions from overlapping four-C-alpha windows, then applies angle regions defined as conventional alpha/beta phi/psi regions. These virtual torsions are not the backbone phi and psi angles formed from N, C-alpha, and C atoms. Consequently, the reported fraction outside those regions cannot establish that CDR-H3 intrinsically violates ordinary Ramachandran preferences, or that the loss hurts because correct CDR-H3 geometry is noncanonical in that sense. The empirical observation that this particular loss hurts accuracy can remain valid, but its mechanistic interpretation requires a valid C-alpha-specific reference or reconstructed-backbone validation. More generally, best-of-five recovery of one crystallographic structure does not establish a calibrated conformational ensemble.

Questions for authors
  1. Please label oracle versus reference-free selection consistently in Tables 2–3 and report a common random/centroid-selected/oracle protocol across internal and external sets. How do predictions perform after placement into a supplied or predicted framework, including closure and framework-clash rates?

  2. Is 0.70 seconds the cost of one candidate or all five, and does the reported pipeline include the PLM encoders and the framework/anchor information needed for placement? Please reconcile Section 3.3 and Appendix H and give a directly comparable cost/accuracy operating point.

  3. Can the geometry interpretation be corrected using C-alpha virtual-angle reference distributions or conventional backbone angles after reconstruction? If the claim concerns conformational ensembles rather than proposal diversity, what held-out multi-conformation or coverage test supports it?

1 comment

W1 — Oracle labelling and timing

Unfortunately, the oracle column was systematically mislabelled in all tables and corresponded to DirectSel. This is the RMSD of the sample with the lowest Cα RMSD without superposition. All results tables are corrected and now report four rules from one persisted five-sample pool: rand (sample index 0), centroid (closest to ensemble mean), DirectSel (retained for reference), and oracle (argmin Kabsch). Corrected medians: test set rand 1.937 / centroid 1.579 / DirectSel 1.448 / oracle 1.290 Å; holdout-71 rand 2.873 / centroid 2.669 / DirectSel 2.448 / oracle 2.265 Å. Bootstrap 95 % CIs are reported on every median.

The best reference-free selector, centroid, gives 2.669 Å on the holdout against ColabFold's 2.20 Å. The SOTA claim is withdrawn from the abstract. The paper is reframed around stem-grounded ensemble generation rather than single-structure accuracy.

On timing: the 0.70 s figure was per candidate, not per Fv. Five candidates cost 0.70 × 5 = 3.5 s, consistent with §3.3. Table 11's caption mislabelled the per-candidate cost as per-Fv. Re-measured unbatched on A40: 0.851 s at N=1, 4.168 s/Fv at N=5. Table 11 now carries explicit samples-per-Fv and PLM-included columns. Corrected headline: H3Fold is approximately 45× faster than LocalColabFold at N=5.

An N=25 accuracy–runtime curve under reference-free centroid selection is added. On the holdout, centroid moves from 2.917 Å at N=5 to 2.609 Å at N=25; oracle falls from 2.265 to 1.894 Å. Centroid does not reach ColabFold's 2.20 Å at any tested budget.

W2 — Pseudo-Ramachandran interpretation

The φ/ψ boxes are defined for N–Cα–C backbone angles and are not valid for four-Cα virtual torsions. The mechanistic interpretation is withdrawn. The section will be rewritten around an empirical Cα-specific reference: highest-density bins of the PDB70 virtual-torsion distribution estimated over 661,154 torsions from 27,232 loops at 5° resolution. Fraction outside the region covering 50 % of PDB70 mass: PDB70 49.8 %, ground-truth CDR-H3 75.2 %, noRama 76.8 %, withRama 81.4 %. CDR-H3 genuinely occupies low-density Cα-torsion territory under this valid reference, and the model trained without the loss tracks the ground-truth distribution more closely. KL and sliced-Wasserstein divergences did not depend on a region definition and are unchanged; they now lead the section.

On calibrated ensembles: no such claim is made. See Reviewer 3, W2.

Q1 — Consistent labelling and placement analysis

Tables 2, 3 and 9 now carry rand / centroid / DirectSel / oracle computed from one persisted five-sample pool per split, matching Tables 4–5.

A placement analysis is added. With a crystal framework (n=65): median terminus error 0.49/0.56 Å, median loop-span error 0.15 Å, 7.7 % of loops beyond 1 Å terminus error, framework clash in 36.9 % of random samples. With a predicted framework the closure degrades: ABodyBuilder3 13.2 % of termini beyond 1 Å; IgFold 26.1 %.

Q2 — Timing scope

Re-measured unbatched on A40, same checkpoint: 0.851 s at N=1, 4.168 s/Fv at N=5. The PLM passes (H3BERTa + Ab-RoBERTa) cost 19 ms, 0.5 % of the N=5 total. DDIM sampling alone is 4.122 s. Anchor extraction is not in the timer and is negligible against DDIM. Table 11 now carries explicit per-candidate and per-Fv columns with PLM-included flags.

The directly comparable operating point is the N=25 curve plotting centroid-selected accuracy against measured wall-clock, with every baseline overlaid as a single point. Corrected headline: H3Fold is roughly 45× faster than LocalColabFold, not 270×.

Q3 — Geometry reference and ensemble coverage

The Cα-torsion reference is replaced as described under W2.

A coverage test is not feasible with the available data. The undeduplicated test index has 419 rows over 162 unique CDR-H3 sequences, but the median pairwise Kabsch RMSD among extra copies is 0.22 Å over 1,277 pairs. These are near-duplicate re-extractions of the same crystallographic conformation, not independently deposited alternatives. The paper makes no calibrated-ensemble claim and states that establishing one requires a purpose-built multi-conformer set.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.