- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SoupFold, a representation-level ensembling framework designed to improve biomolecular complex structure prediction by integrating complementary information from multiple pretrained co-folding models. The authors observe that the pair representations generated by independently trained structure prediction models exhibit substantial alignment while retaining distinct structural information, particularly when the models produce different predictions. Motivated by this observation, SoupFold uses lightweight multilayer perceptrons to map pair representations from several peer models into the representation space of a selected anchor model. The mapped representations are then combined through weighted averaging, and the resulting representation is passed to the anchor model's diffusion module for structure generation. Importantly, the framework does not require retraining the underlying co-folding models. The authors evaluate the method using AlphaFold3, Protenix, ESMFold2, and OpenDDE across antibody–antigen, protein–protein, protein–ligand, molecular-glue, GPCR, and oligomeric complex prediction benchmarks. SoupFold demonstrates substantial improvements over several standalone models, including an increase in antibody–antigen acceptable DockQ success from 47.65% to 61.76% for Protenix and from 59.41% to 68.24% for OpenDDE. Additional experiments indicate that representation-level ensembling can recover structural interfaces missed by individual models and can outperform conventional sample-level ensembling under the evaluated settings.
Critical strengths
The primary strength of SoupFold is its simple yet effective approach to exploiting complementary information across independently trained co-folding models. Rather than combining final predicted structures or retraining large foundation models, the proposed framework operates directly on intermediate pair representations, providing a flexible strategy for improving existing structure predictors. The use of lightweight representation transfer networks is attractive because it decouples the ensembling mechanism from the expensive process of training new co-folding models. The empirical evaluation is extensive, spanning multiple anchor models, conventional structure prediction tasks, and more challenging higher-order complexes.
Critical weaknesses
The most significant limitation concerns the computational cost and the mechanism underlying the reported performance improvements. Although SoupFold trains only lightweight transfer networks, inference requires executing the trunks of multiple large co-folding models, which may substantially increase runtime, GPU memory consumption, and deployment complexity relative to a standalone predictor. The paper includes a sampling-budget comparison, but it does not fully establish the computational efficiency of the method under matched wall-clock time or total GPU cost across all tasks. Another concern is that the transfer networks are trained to minimize mean-squared error against the anchor representation, which encourages reconstruction of the anchor's features rather than explicitly preserving complementary information. Therefore, the experiments do not fully distinguish genuine complementary-feature integration from representation averaging, denoising, or implicit regularization.
Questions for authors
-
Mechanism of representation complementarity: Since the transfer networks are optimized to reconstruct anchor-model representations, how can the authors demonstrate that SoupFold preserves genuinely complementary structural information rather than primarily performing representation denoising or averaging? Could controlled experiments with independent checkpoints of the same model, shuffled peer representations, or self-distillation help isolate this mechanism?
-
Computational cost and efficiency: What are the actual wall-clock inference times, peak GPU memory requirements, and total computational costs of SoupFold compared with individual co-folding models? How do the results change when competing methods are allocated identical computational budgets?
-
Generalization of transfer networks: The representation mappings are trained on protein-only complexes. How well do these mappings generalize to ligand-containing complexes, modified residues, and other heterogeneous molecular systems? Could the authors analyze transfer accuracy separately for protein–protein, protein–ligand, and ligand–ligand representation entries?
-
Negative transfer and model selection: SoupFold occasionally degrades performance, such as on the OpenDDE molecular-glue benchmark. Can the authors characterize when peer representations introduce conflicting or misleading signals? Could an adaptive confidence-aware weighting or gating mechanism prevent such degradation?
-
Comparison with stronger ensembling baselines: The sample-level ensembling baseline selects predictions using confidence scores from different models, which may not be calibrated consistently. Could the authors compare against confidence-calibrated pooling, learned ranking, or compute-matched oracle analyses to better establish the benefit of representation-level integration?
-
Evaluation selection bias and statistical significance: FoldBench results are reported only for complexes where all four models generate predictions. How many benchmark examples are excluded, and does SoupFold remain effective under a broader evaluation protocol? Could the authors report paired bootstrap confidence intervals or statistical significance tests for the reported improvements?
-
Scalability and model dependence: How does performance vary with the number and identity of peer models? Is the improvement primarily driven by a particularly strong peer, or does combining multiple weaker peers offer consistent additional benefits? Could the framework be extended to use AlphaFold3 as an anchor model and to evaluate nucleic acid-containing complexes?
5 comments
W2/Q1. Since transfer networks reconstruct the anchor, how does SoupFold preserve complementary information? Can authors consider controls with independent checkpoints of the same model, shuffled peer representations, or self distillation isolate this mechanism?
(a) Representation complementarity.
Complementary information can survive transfer because the MLP does not reconstruct the anchor exactly. We train a shared small MLP to minimize average MSE across complexes separated from the benchmarks to learn an overall peer-to-anchor alignment. Figure 3(b,left) of the manuscript shows that representations remain less aligned after transfer when the models predict different structures.
Figure 9 of the manuscript provides further evidence that this retained information is useful. Standalone models recover different subsets of native contacts for the same target, while SoupFold combines their representations to recover their union and induces high-quality predictions.
(b) Controlled experiments.
To address your comments, we compare SoupFold with self-distillation and shuffled peer representations. For self-distillation, we average representations from different random seeds of the same model. For shuffled peers, we permute peer token order before transfer. We cannot test independently trained checkpoints of the same model because they are unavailable. Table R9 shows that neither control reproduces SoupFold's gains.
Table R9. FoldBench Ab-Ag accept rates for the control experiments.
| Anchor | Standalone | Self-distillation | Shuffle | SoupFold |
|---|---|---|---|---|
| Protenix | 47.65 | 48.24 | 2.35 | 61.76 |
| ESMFold2 | 52.35 | 51.76 | 1.18 | 62.94 |
| OpenDDE | 59.41 | 59.41 | 1.76 | 68.24 |
These controls suggest that SoupFold's gains are not simply artifacts of averaging or perturbing. We have incorporated these into Section 3.4 and Appendix D.11 (Table 19) of the updated manuscript.
Q3. The transfer networks are trained on protein only complexes. How do they generalize to ligands, modified residues, or other heterogeneous molecular systems?
We would like to clarify that our transfer networks are trained on both protein complexes and protein-ligand complexes, but not on nucleic acids. We have revised the “protein-only” description in Appendix A of the updated manuscript to clarify this. The protein-ligand benchmarks in the paper, FoldBench PL, Runs N' Poses, molecular glue, and GPCR, contain ligands and modified residues.
(Q7) Could the framework be extended to evaluate nucleic acid-containing complexes?
To address your question about heterogeneous molecular systems, we evaluate SoupFold on the FoldBench protein-DNA with OpenDDE as the anchor. We present the results in Table R10. SoupFold improves the performance, showing that our transfer networks can benefit protein-DNA prediction.
Table R10. FoldBench protein-DNA accept rates using the highest-confidence prediction among five samples from a single seed.
| Model | DockQ ≥ 0.23 | DockQ ≥ 0.49 | DockQ ≥ 0.80 |
|---|---|---|---|
| AlphaFold3 | 75.95 | 63.61 | 45.25 |
| Protenix | 73.42 | 63.92 | 48.10 |
| ESMFold2 | 72.78 | 62.34 | 43.35 |
| OpenDDE | 77.22 | 62.97 | 39.87 |
| SoupFold (anchor: OpenDDE) | 79.43 | 66.14 | 42.40 |
We have incorporated this into Section 3.4 and Appendix D.10 (Table 18) of the updated manuscript, including discussion of training on nucleic-acid complexes as a future direction.
Q4. Can the authors characterize when peer representations introduce conflicting or misleading signals? Could adaptive weighting or gating prevent such degradation?
SoupFold does not guarantee success when the anchor succeeds but all peers fail. Although unsuccessful predictions tend to have smaller representation magnitudes and less influence when ensembled (Figure 3(a, right) in Section 2.2 of the manuscript), this does not fully prevent degradation, as shown in Figure 5(b) of the manuscript.
We explore peer selection and magnitude weighting in Table 4 of the manuscript. With ESMFold2 as the anchor, selecting one peer and weighting by magnitude improves antibody-antigen success at DockQ from to , relative to averaging all three peers equally. However, the same strategy reduces protein-protein success at DockQ from to . As discussed in Section 5 of the manuscript, more principled peer weighting can be a future direction for further improvements.
Q5. Can authors consider calibrated pooling, learned ranking, or compute-matched oracle-based comparisonn?
(a) Calibrated confidence and learned ranking.
To address your comment, we first compare SoupFold with confidence-calibrated pooling using isotonic regression [1] and learned ranking using VoroIF-GNN [2]. For calibration [1], we use the held-out dataset SAbDab-22H2, which consists of 99 antibody–antigen complexes and is disjoint from FoldBench.
We present the results in Table R11. SoupFold outperforms these baselines despite their larger sample pool.
Table R11. FoldBench Ab-Ag accept rates for pooled selection and SoupFold.
| Selection method | Samples | Ranking selection |
|---|---|---|
| Raw confidence | 100 | 60.59 |
| Isotonic calibration | 100 | 62.6 |
| Learned ranking (VoroIF-GNN) | 100 | 62.1 |
| SoupFold (OpenDDE anchor) | 25 | 68.24 |
We have incorporated this comparison into Section 3.4 (Table 7) of the updated manuscript.
(b) Compute-matched oracle-based comparison.
Table R12 compares pools with matched budgets under oracle selection (highest ground-truth DockQ).
Table R12. FoldBench Ab-Ag oracle accept rates and Foldseek-multimer cluster counts (TM-score threshold ).
| Prediction pool | Trunk passes | Diffusion samples | Clusters | Oracle selection |
|---|---|---|---|---|
| Standalone (AF3, ESMFold2, Protenix-v1, OpenDDE) | 20 | 100 | 10 | 85.88 |
| SoupFold (AF3, ESMFold2, Protenix-v1, OpenDDE as anchors) | 20 | 100 | 6 | 85.88 |
| Standalone + SoupFold | 20 | 200 | 13 | 89.41 |
Oracle accept rates are identical. Pooled standalone produces more diverse samples from distinct representations, potentially including correct but low-confidence predictions. SoupFold samples from ensembled representations, reducing diversity but more reliably inducing correct predictions (Table R11).
It is noteworthy that combining both pools ( samples) raises oracle accept to . This suggests that varying peer combinations, including standalone as the no-peer case, may broaden coverage. We have incorporated this analysis into Appendix D.9 (Table 17) of the updated manuscript.
Q6-1. How many systems were excluded, and does the conclusion hold under a broader evaluation protocol?
The original FoldBench evaluation excluded two Ab-Ag, five P-P, and P-L targets. To address your concern about broader evaluation, we rerun the excluded P-P and P-L targets after resolving technical failures. We modify a kernel for out-of-memory issues and modify codes to handle ligand parsing errors.
Two Ab-Ag, two P-P, and four P-L targets remain unscorable because the scorer fails on all predictions. Table R13 reports the additional targets for which both anchor-alone and SoupFold predictions can be scored. SoupFold does not degrade performance on these additional targets.
Table R13. Accept counts on additional FoldBench targets, shown as Standalone / SoupFold.
| Model | P-P (n = 3) | P-L (n = 24) |
|---|---|---|
| ESMFold2 | 2 / 2 | 13 / 13 |
| Protenix | 2 / 2 | 14 / 15 |
| OpenDDE | 2 / 2 | 14 / 14 |
Q6-2. Could the authors report paired bootstrap confidence intervals or statistical significance tests for the reported improvements?
To address your comments, we report paired bootstrap confidence intervals for the improvement over each anchor in Table R14.
Table R14. FoldBench accept-rate gains (percentage points), shown as mean gain [paired bootstrap CI].
| Anchor | Ab-Ag | P-P | P-L |
|---|---|---|---|
| Protenix | +12.35 [+6.47, +18.82] | +4.01 [+1.09, +6.93] | +3.23 [+0.19, +6.28] |
| ESMFold2 | +7.06 [0.00, +14.12] | +2.19 [−0.73, +5.11] | +3.42 [+0.57, +6.27] |
| OpenDDE | +8.82 [+4.71, +13.53] | +3.28 [+1.09, +5.84] | +8.37 [+5.13, +11.79] |
We have incorporated statistical analysis into Appendix D.1 (Tables 9) of the updated manuscript.
Dear Reviewer WRC7, we express our deep appreciation for your time and insightful comments. In our updated manuscript, we highlight the changes in blue.
In what follows, we address your comments one by one.
W1/Q2. While Figure 10 addresses sampling budgets, what are the wall clock time, peak memory, and total cost, and how do the results change under identical compute budgets?
To address your question, Table R6 reports per-system inference time and peak memory. The additional wall-clock time mainly stems from peer trunks while using the diffusion module only for the anchor model. SoupFold runs model trunks sequentially, bounding peak memory by the largest trunk.
Table R6. Trunk time (s) / diffusion time (s) / peak memory (GiB) for five samples on one AMD Instinct MI300X GPU.
We advance the comparison in Figure 10 of the manuscript to consider GPU-time budgets (Table R7). SoupFold retains its advantage over standalone models under comparable compute budgets.
Table R7. FoldBench Ab-Ag accept rate / GPU-h / number of seeds.
We have modified Section 3.3 and Figure 10 of the updated manuscript to consider GPU-time budgets. We have also incorporated these into Appendix D.7 (Table 15).
We revisit Table 1 of the manuscript using a single SoupFold seed versus standalone models with comparable budgets (Table R8), including P-P and P-L tasks, which are less sensitive to sampling budgets. SoupFold improves over each anchor with comparable GPU time.
Table R8. FoldBench accept rate / GPU-h / number of seeds.