acceptodds
Under review as a conference paper at ICLR 2027

Recycle only when you have to

Abstract

AlphaFold 3-like models (e.g., Boltz, Protenix, etc.) have pushed the frontier of protein binder design. This architecture is used both for the generation of binder candidates (e.g., by converting these models into backbone or full atom generative models) as well as for the screening of these candidates for valid binders. In this paper, we focus on the large-scale screening problem. We start from the observation that AlphaFold 3's confidence metrics (ipSAE, ipTM) as well as structural consistency metrics (scrmsd) can be predicted directly from the trunk even at early recycling. Because our proposed predictor is cheap, differentiable, and available at every cycle, it can be used in two distinct ways: as a decision signal to wisely spend the recycling budget for screening and as an objective for hallucination-based design. We show that both applications can lead to increased performance at significantly reduced computational cost.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Antibody Design, Alphafold3, Structure Prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yhU3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Instead of running an expensive protein-folding model to completion, predict the final folding quality from its intermediate representations. Use those predictions to make antibody design and screening faster. They made 2 types of critics, condifence and structure cyclecritic: a lightweight surrogate of the original folding model's final quality assessment. 2 applications using these critics are shown: one for scree ning and one for hallucination based design

Critical strengths
  1. Empirical motivation is quite clear. and the overall structure is simple, and efficient
  2. Paper treats recycling as an adaptive computation problem, which was not done in past.
  3. Results are promising, with boosts clear
Critical weaknesses
  1. methodological novelty: main novelty is adaptive early stopping, rather than a fundamentally new confidence estimation method. also, the architecture uses deepset, and lacks novelty in the architecture itself.
  2. Hallucination improvements are partly explained by increased sampling For Mosaic, CycleCritic allows approximately 1.6× more designs under the same computational budget. Naturally, generating more candidates increases the chance of finding high-scoring designs and also acknowledges in Appendix H
  3. Figure 3 does not establish a universal property of AF3-like models.Figure 3 shows that Protenix-v2 confidence remains relatively stable even after substantial coordinate perturbations.The authors interpret this as evidence that confidence can be predicted from trunk representations.However, there is another interpretation:The original confidence head may be insufficiently sensitive to structural errors.If CycleCritic learns to imitate these confidence scores, it inherits their limitations.This is particularly concerning for antibody–antigen complexes, where confidently predicted but incorrect interfaces are a known problem.Also, the paper reports that ESMFold2 behaves differently: its confidence predictions are more dependent on coordinates.
  4. Early stopping sacrifices some promising candidates. Some of these may be interesting difficult-to-predict binders.The method is particularly vulnerable if its early predictions are systematically pessimistic about unusual but valid binding modes.
  5. It is unclear whether the approach generalizes to:Other models, such as OpenDDE or Boltz, completely novel target familie, other complex types, such as protein–peptide interactions. Or, cases where the original folding model confidently predicts incorrect interfaces.
  6. If 1-2 recycles is enough(early trunk representations) why didn't the authors include this setting for comparison?
Questions for authors

I think this paper has promising results so far, but needs some more clear experiments and supporting evidence

  1. Methodological novelty

The main methodological contribution appears to be adaptive early stopping using a learned confidence predictor, rather than a fundamentally new confidence estimation method. Given existing approaches that extract confidence information directly from trunk representations (e.g., distogram-based proxies), could the authors clarify what is fundamentally new about CycleCritic beyond applying a lightweight predictor for early stopping?

  1. Comparison with simpler fixed-recycling baselines

The screening experiments compare CycleCritic against Protenix-v2 with 4 and 10 recycles. However, Figure 2 suggests that early recycling already contains substantial information about final confidence. Could the authors include additional baselines using only 1 or 2 fixed recycles? A simple two-stage approach—performing 1–2 recycles for initial screening and fully refolding only promising candidates—would also be informative. These comparisons would help establish whether the improvements come from learned adaptive computation rather than simply reducing the recycling budget.

  1. Hallucination improvements versus increased sampling

For Mosaic, CycleCritic enables approximately 1.6× more designs under the same computational budget. Consequently, improvements in top-50 design quality may largely result from increased sampling rather than improved per-design quality. Although the authors partially address this in Appendix H (Figure H.1), could they further quantify the improvements attributable to the CycleCritic objective itself? In particular, an ablation isolating CycleCritic from the replacement of PAE-based losses with distogram-based losses would help clarify its contribution.

  1. Reliability of confidence prediction and coordinate sensitivity

Figure 3 shows that Protenix-v2 confidence scores remain highly correlated with their original values even after substantial coordinate perturbations. While the authors interpret this as evidence that confidence is largely determined by trunk representations, an alternative explanation is that the confidence head is insufficiently sensitive to structural errors. Moreover, ESMFold2 exhibits different, more coordinate-dependent behavior. Could the authors investigate whether CycleCritic inherits these limitations, particularly for confidently predicted but incorrect antibody–antigen interfaces? Evaluation against experimentally determined structures using metrics such as DockQ would help clarify this issue. If time is available, can you expand this to OpenDDE as well?

  1. False negatives introduced by early stopping

The proposed gating strategy retains 97.3% of highly reproducible designs (≥5/9 successful refolds), but only 89.2% of candidates that pass at least one reference refold. This raises concerns that early stopping may preferentially discard challenging candidates, including unusual but potentially valid binding modes. Could the authors analyze the characteristics of these false negatives? In particular, are they enriched among difficult targets, uncommon epitopes, or structurally diverse binding modes? Understanding this would help establish when adaptive early stopping might be detrimental.

  1. Generalization across models, targets, and complex types

The experiments primarily focus on Protenix-v2 and ESMFold2, with downstream evaluation on seven antigens. How well does CycleCritic generalize to other AF3-like architectures, such as Boltz or OpenDDE, and to previously unseen antigen families? Furthermore, could the approach extend to other interaction types, such as protein–peptide complexes? It would be useful to understand whether the critics require model-specific retraining and how their prediction accuracy changes under these distribution shifts.

5 comments

We thank the reviewer for their valuable feedback.

To address the reviewers’ comments, we have conducted extensive new experiments and added the following to the revised manuscript:

  • Domain Generalization: Demonstrated that CycleCritic transfers zero-shot to de novo minibinders (Appendix J).
  • Model Generalization: Showed the early-trunk prediction principle applies across Mol-1, OpenDDE, and Boltz-1/2 (Appendix I).
  • Output Distribution: Verified that early stopping preserves the target distribution and does not penalize rare epitope modes (Appendix M).
  • Baselines & Stats: Added fixed 1- and 2-recycle baselines (Appendix N), saved-recycle distributions (Appendix K), and statistical significance testing (Table L.1).
  • Hallucination Metrics: Added full-population quantiles for Mosaic and ESMFold2 (Appendices F and G).

In the following, we will reply to the specific comments by the reviewer:

Weaknesses

[W1] methodological novelty: main novelty is adaptive early stopping, rather than a fundamentally new confidence estimation method. also, the architecture uses deepset, and lacks novelty in the architecture itself.

We indeed do not claim major architectural advances nor do we propose a new confidence metric. Our contribution is to provide an algorithmic mechanism to evaluate the need for incremental recycles. This allows dynamic allocation of inference compute. One may loosely think about this as a form of reasoning in structure prediction models although the analogy is imperfect.

[W2] Hallucination improvements are partly explained by increased sampling. For Mosaic, CycleCritic allows approximately 1.6× more designs under the same computational budget. Naturally, generating more candidates increases the chance of finding high-scoring designs and also acknowledges in Appendix H

We agree that the improvements for mosaic are mostly due to the computational efficiency gains provided by CycleCritic (see Appendix G, Tables G.2 and G.4, in the revised manuscript). The presented experiments with Mosaic show that dropping the entire diffusion and confidence head and replacing them with CycleCritic and distogram-based proxies does not lead to a drop in performance while leading to a speed up of 60%. While speedups typically incur a tradeoff, Appendix G (Tables G.2 and G.4) shows that the performance is, on average, slightly better.

[W3] Figure 3 does not establish a universal property of AF3-like models. Figure 3 shows that Protenix-v2 confidence remains relatively stable even after substantial coordinate perturbations.The authors interpret this as evidence that confidence can be predicted from trunk representations. However, there is another interpretation: The original confidence head may be insufficiently sensitive to structural errors. If CycleCritic learns to imitate these confidence scores, it inherits their limitations. This is particularly concerning for antibody–antigen complexes, where confidently predicted but incorrect interfaces are a known problem. Also, the paper reports that ESMFold2 behaves differently: its confidence predictions are more dependent on coordinates.

[W5] It is unclear whether the approach generalizes to: Other models, such as OpenDDE or Boltz, completely novel target familie, other complex types, such as protein–peptide interactions.

We agree with this valuable suggestion.

In the revised manuscript, we have added experiments that extend the CycleCritic to OpenDDE, Mol1, Boltz-1, and Boltz-2. Please see Appendices I and J in the updated manuscript. Briefly summarised, we find that for all models, single latent dimensions of the pair representation, correlate surprisingly well with the model’s own final ipSAE (Figure I.1). Boltz-2 is the weakest (ρ = 0.61), while ESMFold2, OpenDDE, Mol-1 and Protenix-v2 all reach ρ > 0.89. Although the designed VHHs and the natural SAbDab complexes show a slight shift in distribution, a single pooled dimension, an extremely simple potential CycleCritic, predicts both domains well. Tracking the Spearman correlation over recycling iterations (Figure I.2) shows that this signal emerges early during recycling. In Appendix I.1 (Table I.1) of the revised manuscript, we actually deploy the Analytic CycleCritic (see Section 4.1) to Mol-1 and find that with the thresholds fitted on Protenix-v2 it operates nearly as well.

Furthermore, we have added experiments demonstrating that CycleCritic directly extends to minibinder design in Appendix J (Table J.1 and Figures J.1–J.3). Interestingly, this works without any additional minibinder-specific training.

Questions

[Q1] The main methodological contribution appears to be adaptive early stopping using a learned confidence predictor, rather than a fundamentally new confidence estimation method. Given existing approaches that extract confidence information directly from trunk representations (e.g., distogram-based proxies), could the authors clarify what is fundamentally new about CycleCritic beyond applying a lightweight predictor for early stopping?

Distogram-based proxies are mainly used in the literature to avoid the diffusion rollout. The CycleCritic goes beyond this by also dynamically allocating the recycling budget. Compute costs are heavily dominated by the trunk. As such, CycleCritic can cut the main computational cost well beyond the diffusion rollout while the latter provides the ceiling for distogram proxy methods.

Applying diffusion only for promising candidates further ensures high fidelity confidence and structure prediction, resulting in being faster and higher precision than simply using distogram-based proxies, like previous work.

For hallucination losses, we show that CycleCritic outperforms distogram-based proxies, see Appendix F.4.2. Using the iptm-CC for hallucination yields over 50% higher increase in performance than the iptm proxy (+.0343 vs +.0229). Adding the signal of the ipsae-CC leads to a 90% increase(+.0435), showing the versatility of the approach.

[W6] If 1-2 recycles is enough(early trunk representations) why didn't the authors include this setting for comparison?

[Q2] The screening experiments compare CycleCritic against Protenix-v2 with 4 and 10 recycles. However, Figure 2 suggests that early recycling already contains substantial information about final confidence. Could the authors include additional baselines using only 1 or 2 fixed recycles?

We have added comparison to these baselines in the revised manuscript, see Appendix N and Figures N.1 & N.2. Briefly summarised, for 1&2 recycles the compute cost is dominated by diffusion and the performance drops significantly. This is why we used Protenix-mini, which optimised throughput while losing as little performance as possible. Protenix-mini is significantly faster than ptx1 and still has better agreement with ptx10.

Increasing the aggressiveness of the CycleCritic on the other hand allows the adaptive compute allocation to keep up with protenix-mini in throughput while being significantly closer to protenix10.

A simple two-stage approach—performing 1–2 recycles for initial screening and fully refolding only promising candidates—would also be informative. These comparisons would help establish whether the improvements come from learned adaptive computation rather than simply reducing the recycling budget.

As Figure N.1 shows, doing 1-2 cycles and diffusion on all samples for preselection is quite costly, so the proposed approach would definitely be slower than our deployed ptx10&CC. If we used a lightweight predictor to filter out promising candidates, then this would be an edge case of our approach. This approach then would not take advantage of the better signal at later stages and definitely perform worse. Note that in our experiments, based on preliminary experiments (compare e.g. Figure E.6), we only enabled the sCC-gates after the first recycle.

[Q3] For Mosaic, CycleCritic enables approximately 1.6× more designs under the same computational budget. Consequently, improvements in top-50 design quality may largely result from increased sampling rather than improved per-design quality. Although the authors partially address this in Appendix H (Figure H.1), could they further quantify the improvements attributable to the CycleCritic objective itself? In particular, an ablation isolating CycleCritic from the replacement of PAE-based losses with distogram-based losses would help clarify its contribution.

We have added population statistics in Appendix G (Tables G.2 and G.4) of the revised manuscript. In summary, when comparing the population statistics across the 50th, 25th, 10th, and 1st percentiles, we see that for Mosaic, CC is higher in 59% of cases (33/56) with a net average of +0.006. Here we are able to produce significantly more samples, while the generated designs are not worse, even slightly better. We have done a more extensive ablation study for ESMFold2 in Section F.4, but did not study the parameters for Mosaic in detail. While the experiments with Mosaic show that the CycleCritic and distogram based proxies can be drop in replacements, the parameters used are defaults and could most likely be optimized on both sides. Given the time-frame of the rebuttal period we are not able to provide these experiments in a fair fashion to the baseline, but would be happy to provide them for the camera ready version. Given that for ESMFOLD, the CycleCritic showed to be beneficial for hallucination-based design, we expect the same for Mosaic.

[Q4] Figure 3 shows that Protenix-v2 confidence scores remain highly correlated with their original values even after substantial coordinate perturbations. While the authors interpret this as evidence that confidence is largely determined by trunk representations, an alternative explanation is that the confidence head is insufficiently sensitive to structural errors. Moreover, ESMFold2 exhibits different, more coordinate-dependent behavior. Could the authors investigate whether CycleCritic inherits these limitations, particularly for confidently predicted but incorrect antibody–antigen interfaces? Evaluation against experimentally determined structures using metrics such as DockQ would help clarify this issue. If time is available, can you expand this to OpenDDE as well?

We agree that the limited sensitivity of Protenix-v2 confidence scores to large coordinate perturbations admits the alternative interpretation proposed by the reviewer. However, this interpretation is not mutually exclusive with our central observation that confidence is strongly determined by trunk representations. One can view the coordinate output of a co-folding model as a stochastic geometric realisation of the constraints encoded in the pair representation. The CC learns the expected confidence of the final structure from a given trunk state. Our ESMFold2 results demonstrate that confidence can be reliably predicted from trunk representations alone, despite the differing coordinate-dependence of its confidence head.

We agree that CycleCritic may inherit systematic biases in the original confidence scores, including overconfidence in incorrect antibody–antigen interfaces. CycleCritic is not intended to replace an experimentally calibrated structure-quality estimator. This is a separate modelling problem and beyond the scope of this work. However, for the screening application, this limitation is mitigated by the design of our method as the CycleCritic only rejects candidates early. Retained candidates still undergo full structure prediction and must satisfy the original confidence and structural self-consistency filters. Consequently, CycleCritic cannot independently accept an incorrect interface that the full model would reject. Its principal additional failure mode can only be premature rejection of otherwise acceptable candidates. Table E.2 shows that CycleCritic-gated screening achieves 0.96 precision compared with 0.95 for full Protenix-v2, at comparable recall (0.51 versus 0.52), while reducing computational cost.

Regarding DockQ evaluation with experimentally-determined structures, Figure A.1 in the manuscript provides evidence that intermediate recycling outputs contain information predictive of final DockQ for both Protenix-v2 and ESMFold2. Does the reviewer have a different experiment in mind?

In Appendix I (Figures I.1–I.2; see also Figure 3), we verified that early-cycle trunk representations in OpenDDE abag strongly predict final ipSAE (ρ=0.910\rho = 0.910). While we are unable to run the coordinate-perturbation sensitivity experiment of Figure A.2 on OpenDDE in the rebuttal timeframe, we will be glad to include it in the camera-ready manuscript.

Reviewer_D9hkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CycleCritic, a small network trained to predict confidence scores for early stopping in recycling in AlphaFold based models. CycleCritic can predict confidence scores for things like ipSAE, ipTM, pTM, pLDDT (confidence) or scRMSE (structure). The paper conducts experiments using Protenix v2, ESMFold2 and Mosaic, and explores performance/compute tradeoffs of adding CycleCritic. Overall the paper shows that CycleCritic can lead to similar performance at lower cost in terms of compute.

Critical strengths

CycleCritic is simple and the motivation for incorporating CycleCritic seems straightforward and effective, and discussion of the model architecture is very clear and thoroughly written.

Critical weaknesses

Overall, CycleCritic seems relatively limited in terms of scope and use case, and evaluation is overzealous and inconsistent between graph + analysis, better statistical principles needed (Figure 5a, performance is within error bars, but paper states “significantly outperforms all baselines”). Matching results of the baselines is still reasonable, but cannot simply state that it’s better. Given the type of analyses performed by authors in other fields who investigate early stopping, the authors have not provided sufficient analysis of the distribution of recycling steps saved by CycleCritic, along with proofs that the output distribution of AlphaFold-like models with CycleCritic is equivalent to the distribution with normal recycling under whatever sampling condition is used.

Questions for authors
  1. Figure 3 only includes results for Protenix, and the corresponding results for ESMFold2 (Figure A.3) show the opposite results. Later in the paper, it’s mentioned that CycleCritic also works for ESMFold2, which I think means that the results from Figure 3 are not generalizable and shouldn’t be included in the paper as part of a motivation for why CycleCritic works.
  2. Figures 6-7 show that there is improved performance by CycleCritic. It’s not clear why adding CycleCritic should ever perform better than the baseline, and there doesn’t seem to be any analysis on this. From my understanding, CycleCritic basically kills unpromising results, whereas the baseline would take all of them to completion, so to me it seems that CycleCritic can at most match the performance of the baseline — not exceed.
  3. ptx (4 cycle) seems to have comparable performance to ptx (10 cycle), are there results for CC & ptx (4 cycle) (referencing Figure 5)
5 comments

We thank the reviewer for their valuable feedback.

To address the reviewers’ comments, we have conducted extensive new experiments and added the following to the revised manuscript:

  • Domain Generalization: Demonstrated that CycleCritic transfers zero-shot to de novo minibinders (Appendix J).
  • Model Generalization: Showed the early-trunk prediction principle applies across Mol-1, OpenDDE, and Boltz-1/2 (Appendix I).
  • Output Distribution: Verified that early stopping preserves the target distribution and does not penalize rare epitope modes (Appendix M).
  • Baselines & Stats: Added fixed 1- and 2-recycle baselines (Appendix N), saved-recycle distributions (Appendix K), and statistical significance testing (Table L.1).
  • Hallucination Metrics: Added full-population quantiles for Mosaic and ESMFold2 (Appendices F and G).

In the following, we will reply to the specific comments by the reviewer:

Weaknesses

[W1] CycleCritic seems relatively limited in terms of scope and use case

AI Antibody design is an emerging field that has seen rapid progress over the last year. Increasingly, designs are approaching promising starting points for drug candidates. This is illustrated by billions of dollars of investment in AI antibody design companies such as Chai, Boltz, Xaira. As we show in the manuscript, our proposed method can substantially speed up state-of-the-art pipelines and therefore constitutes an important contribution to this emerging field.

[W2] Evaluation is overzealous and inconsistent between graph + analysis, better statistical principles needed (Figure 5a, performance is within error bars, but paper states “significantly outperforms all baselines”). Matching results of the baselines is still reasonable, but cannot simply state that it’s better

We acknowledge that our original phrasing was inexact. Note that Figure 5a shows confidence intervals across pooled designs, which does not lend to reading off significance values by eye. In Table L.1 of our revised manuscript, we have added paired cluster-bootstrap tests, stratified by target. The tests show that CC gating raises throughput significantly against both ptx@10 (+32% overall yield, p < 5×10⁻⁴) and ptx@4 (+14%, p < 5×10⁻⁴). It also improves classification against ptx@4 (F1 +7.4 pp, recall +11.0 pp, both p < 5×10⁻⁴).

In order to address the reviewers concern, we have rephrased this caption of Figure 5a as follows: We significantly outperform all baselines in yield per GPU hour and all but ptx10 in precision, recall and F1 score (see Table L.1).

[W3] Given the type of analyses performed by authors in other fields who investigate early stopping, the authors have not provided sufficient analysis of the distribution of recycling steps saved by CycleCritic, along with proofs that the output distribution of AlphaFold-like models with CycleCritic is equivalent to the distribution with normal recycling under whatever sampling condition is used.

Recycling steps saved: First, we characterize the distribution of saved recycling steps in Figure E.5 and Appendix K (Table K.1, Figure K.1): across the campaign, 85.6% of trajectories terminate before diffusion (29% at cycle ≤1\le 1), reducing mean trunk depth from 10 to 6.31 passes (36.9% fewer trunk evaluations).

Output distribution equivalence: Thank you for this suggestion. This is indeed a valid concern. We have added an extensive analysis of the output distribution to the revised manuscript, please see Appendix M.

Briefly summarised, we analyse the Wasserstein distance between CycleCritic-gated designs and the ungated design population. Specifically, we calculate these distances for five confidence metrics (abag_ipsae, sc_rmsd, iptm, pdockq2, interface_pae_norm) as well as 25 biochemical metrics, such as loop length, Kyte-Doolittle hydropathy, hydrophobicity, charge density and others. For comparison, we calculate analogous distances for the relevant baselines used throughout the paper (Protenix-v2 with four recycles, Protenix-score, and Protenix-mini).

Out of the analysed methods, CycleCritic is the only one that never significantly deviates from the distribution given by passers with Protenix-v2 (Table M.1). Further CycleCritic is closest in distribution to the target for 29 of the 30 metrics (see Table M.3).

In the revised manuscript, we also added detailed experiments demonstrating that the CycleCritic generalizes to minibinders without any additional retraining (Appendix J; Table J.1 and Figures J.1–J.3).

Questions

[Q1] Figure 3 only includes results for Protenix, and the corresponding results for ESMFold2 (Figure A.3) show the opposite results. Later in the paper, it’s mentioned that CycleCritic also works for ESMFold2, which I think means that the results from Figure 3 are not generalizable and shouldn’t be included in the paper as part of a motivation for why CycleCritic works.

Thank you for pointing this out. We agree that Figure 2 is the main motivation for CycleCritic. Figure 2b (and Figure I.2) shows that single latent dimensions of the pair representation correlate closely with final directional ipSAE even early during recycling across Protenix-v2, ESMFold2, OpenDDE, Mol-1, Boltz-1, and Boltz-2. For all models, single latent dimensions of the pair representation, mean-pooled over interface pairs, correlate surprisingly well with the model’s own final ipSAE. Boltz-2 is the weakest (ρ = 0.61), while ESMFold2, OpenDDE, Mol-1 and Protenix-v2 all reach ρ > 0.89. Figure I.1 shows a detailed scatter plot for the last cycle.

We have also updated Figure 2b to show these new results.

Figure 3 does indeed not hold for ESMFold2. The structure cannot be arbitrarily deformed without affecting confidence in an artificial manner. Please note that this is likely because training for ESMFold deviated from the AF3 confidence training protocol. Specifically, the authors dropped out the pair representation in 20% of the confidence training steps with the explicit goal of forcing this behaviour. Nevertheless, the confidence can still be predicted from the pair provided that the structure is not deformed, as demonstrated by Figure 2.

We fully agree with the reviewer that this point is not necessary for motivating the CycleCritic. To streamline the presentation, we have therefore relegated it to Appendix A.1 (Figure A.2) and replaced it with the scatterplot between single pooled latent dimensions in the pair rep and ag→ab ipsae.

[Q2] Figures 6-7 show that there is improved performance by CycleCritic. It’s not clear why adding CycleCritic should ever perform better than the baseline, and there doesn’t seem to be any analysis on this. From my understanding, CycleCritic basically kills unpromising results, whereas the baseline would take all of them to completion, so to me it seems that CycleCritic can at most match the performance of the baseline — not exceed.

This is an important point. Please note that Figure 6+7 are about hallucination, i.e., using the folding model as a generative model. In particular, these figures do not concern screening for which the full model accuracy indeed represents an upper bound on performance.

Hallucination works by minimising confidence-based losses via gradient descent through a folding model. This is challenging because the computational graph of the recycles, diffusion and confidence head needs to be unrolled. Fitting the full computational graph of, for example, 20 recycles in memory becomes not feasible. Standard hallucination approaches therefore either use low numbers of recycles or detach all but the final recycle. This reduces the fidelity of the gradient signal. The CycleCritic provides a cheap and differentiable proxy for a larger number of recycles. As such, it can drastically reduce the computational graph at comparable prediction fidelity.

In the ablation studies on ESMFold2 in section F.4, we see that the gradient signal provided by the CycleCritic is more helpful than other proxies based on the distogram. This is likely because we have trained it on a noised dataset, making it a robust predictor for hallucination based design.

[Q3] ptx (4 cycle) seems to have comparable performance to ptx (10 cycle), are there results for CC & ptx (4 cycle) (referencing Figure 5)

In Appendix N (Figures N.1–N.2) of the revised manuscript, we have investigated CC & ptx4 for the smaller dataset with 14k designs as well; We show that the CycleCritic also works as expected for 4 recycles as well.

For the scope of this paper, we prioritized staying as close to ptx10 as possible. Since distribution of passers with ptx4 is already different to ptx10, as we have seen in Appendix M (Tables M.1–M.3), CC & ptx4 will most likely not replicate the original passers well.

Thank you again for your considered review. With the discussion period drawing to a close, we would be glad to address any remaining concerns.

Reviewer_Btp5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This work introduces CycleCritic for efficient screening by leveraging differentiable signals of structure prediction trunks early in their recycling protocol. CycleCritic is trained given pair representations to predict the desired confidence scores used to filter de novo designs. The model can be used to replace expensive structure prediction and shows modest improvement in shifting the distribution towards more confident samples.

Critical strengths
  • Unique confidence distillation approach that targets the use of the folding models for their filtering applications
  • Applicable in multiple ways different hallucination and filtering techniques -This method does not have to be retrained for each downstream model but just the folding model and data subset for biological species of interest.
Critical weaknesses
  • The overall writing is quite choppy making it really hard to follow. The method too would have to be retrained for each system and in general ProtenixV2 while the stronger than prior mdoels for AB-Ag structure prediction performs general quite bad compared to general protein-protein interactions.
  • Unclear how the accuracy of the CycleCritic student model could become more accurate than the teacher -Success metrics are quite week and there is no accuracy analysis on how well CycleCritic matches in a 1-1 setting.
  • Many of the benchmark targets are included in Sabdab is there any generalization or is this work just a more efficient distilled confidence head?
Questions for authors
  • If CycleCritic is trained to reproduce known confidence scores how can its accuracy be better than the teacher model?
  • Does this method work for larget corpus like general minibinders in comparison to BoltzGen and BindCraft for miniprotein design. -What due the per sample nromalized results look like? The model being faster is a clear win but its important to understand if we only cared for accuracy what would figure 6 and others look like.
  • what is the persample accurcy fro mteh critic to the folding models at different recycing amounts?
2 comments

We thank the reviewer for their valuable feedback.

To address the reviewers’ comments, we have conducted extensive new experiments and added the following to the revised manuscript:

  • Domain Generalization: Demonstrated that CycleCritic transfers zero-shot to de novo minibinders (Appendix J).
  • Model Generalization: Showed the early-trunk prediction principle applies across Mol-1, OpenDDE, and Boltz-1/2 (Appendix I).
  • Output Distribution: Verified that early stopping preserves the target distribution and does not penalize rare epitope modes (Appendix M).
  • Baselines & Stats: Added fixed 1- and 2-recycle baselines (Appendix N), saved-recycle distributions (Appendix K), and statistical significance testing (Table L.1).
  • Hallucination Metrics: Added full-population quantiles for Mosaic and ESMFold2 (Appendices F and G).

In the following, we will reply to the specific comments by the reviewer:

Weaknesses

[W1] The method too would have to be retrained for each system and in general ProtenixV2 while the stronger than prior mdoels for AB-Ag structure prediction performs general quite bad compared to general protein-protein interactions.

In Appendix J of our revised manuscript (Table J.1 and Figures J.1–J.3), we show that the CycleCritic generalizes without any additional training to minibinders for Protenix. Furthermore, we explicitly propose an analytic version of the CycleCritic in Section 4.1, which, as we show in Appendix I.1 (Table I.1) of the revised manuscript, even generalizes to Mol-1. This version does not require any training and shows strong performance. Our revised manuscript also indicates that the CycleCritic is applicable to ESMFOLD2, Mol-1, OpenDDE, Boltz1, Boltz2.

[W2] Unclear how the accuracy of the CycleCritic student model could become more accurate than the teacher [Q1] If CycleCritic is trained to reproduce known confidence scores how can its accuracy be better than the teacher model?

CycleCritic is only an early kill-only filter; the original Protenix-v2 @ 10 teacher still scores all surviving designs (Figure 1). Filtering out borderline candidates early generally trades slightly lower recall for higher precision (Figure N.2). Moreover, our newly added paired cluster-bootstrap test (Table L.1) shows that CycleCritic's precision (77.94% vs. 76.07% for ptx@10) falls squarely within ptx@10's own seed-rotation range (75.78–78.49%).

In addition, screening more candidates at fixed compute increases overall true-positive yield by 32% (Table L.1).

[W3] Many of the benchmark targets are included in Sabdab is there any generalization or is this work just a more efficient distilled confidence head? [Q2] Does this method work for larget corpus like general minibinders in comparison to BoltzGen and BindCraft for miniprotein design.

We include in the revised manuscript experiments on minibinders, a modality completely disjoint from SAbDab and thus our training set. These experiments show that CycleCritic generalizes to this domain, see Appendix J (Table J.1 and Figures J.1–J.3) for a summary.

Please note that two out of the seven antigens, CD45 and PDGFRb, have no homologs in SabDab, even down to 50% sequence similarity with 20% coverage. For both CD45 and PDGFRb, the CycleCritic was especially useful with strong recall and significantly increased yield per GPU-h (Figure E.4).

Questions

[Q3] What due the per sample nromalized results look like? The model being faster is a clear win but its important to understand if we only cared for accuracy what would figure 6 and others look like.

We added full-population quantiles in Appendices F and G (Tables F.2, G.2, and G.4) for the data in Figures 6 and 7. Across the 50th, 25th, 10th, and 1st percentiles, CycleCritic is higher in 59% of cases (33/56, net average +0.006) for Mosaic and 71% of cases (40/56, net average +0.019) for ESMFold2 (improving on every target except IL7RA).

[Q4] what is the persample accurcy from the critic to the folding models at different recycing amounts?

We analyze this in Appendix K (Table K.1 and Figure K.1) of the revised manuscript. Briefly summarized, for our experiments per cycle, the kill precision was over 99.8 % for all cycles. Please note that the designs are over 98% negatives, which explains the high precision, but is precisely the de novo setting. After all recycling steps are done 97.3 % of 5/9 passers survive the CycleCritic’s gating while over 85% of the negatives were terminated early.

Thank you again for your considered review. With the discussion period drawing to a close, we would be glad to address any remaining concerns.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Reviewer Feedback and Author Rebuttal

We thank all three reviewers for their constructive feedback which we believe have strengthened the manuscript. The reviews recognised the practical utility, simplicity, and promising empirical results of CycleCritic, while identifying important questions concerning generalisation, methodological novelty, statistical rigour, and screening fidelity. In response, we have improved the manuscript with extensive additional experiments, baselines, and analyses. We regret that 2/3 reviewers did not engage further with us in the discussion period.

Below, we summarise how the revised manuscript addresses the reviewers' principal concerns.

Reviewer Btp5

  • Generalisation: We demonstrate zero-shot transfer of the learned CycleCritic to de novo minibinders, achieving a 2.3× increase in true-positive throughput without retraining (Appendix J). We additionally establish that the underlying early-trunk confidence signal generalises across six folding models (Appendix I).
  • Teacher–student fidelity: We clarify that CycleCritic is a kill-only filter, not a replacement for the full folding model. Surviving candidates undergo standard Protenix-v2 evaluation. The method achieves 32% higher true-positive yield per GPU-hour, while maintaining precision within the full model's seed-to-seed variability (Table L.1).
  • Per-design quality: We added full-population quality statistics for both hallucination pipelines (Appendices F–G), alongside per-cycle gating analyses (Appendix K).

Reviewer D9hk

  • Statistical rigour and adaptive computation: We added paired cluster-bootstrap significance tests, demonstrating significant improvements in throughput over ptx@10 (+32%) and ptx@4 (+14%). We additionally characterise the distribution of early exits, showing that CycleCritic terminates 85.6% of designs before diffusion and reduces average trunk evaluations by 36.9% (Appendices E, K, L).
  • Formal distributional argument: In our follow-up with the reviewer, we derived a total variation bound on the accepted-design distribution. Since CycleCritic only rejects candidates and all survivors undergo identical full-model evaluation, the TV distance between gated and ungated passer distributions is bounded by the fraction of full-model passers rejected by the gate. In our paired evaluation, this corresponds to approximately 1.7% (2,874 versus 2,925 accepted designs). This complements our additional extensive empirical analyses across 30 structural, confidence, and biochemical metrics, along with preserved epitope-mode coverage. CycleCritic's selected designs show no statistically significant differences from the full-refold reference under these tests, and preserve comparable epitope-mode coverage (Appendix M).
  • Iso-step evaluation: We clarified that both hallucination pipelines already use identical per-design optimisation schedules between baseline and CycleCritic (110 steps for Mosaic; 149 for ESMFold2). Alongside equal-compute comparisons, we provide matched-population-size analyses that disentangle the gains attributable to increased sampling throughput from changes in per-design quality (Appendices F–H).
  • Coordinate sensitivity: We clarified that confidence predictability from trunk representations does not require the confidence head to be invariant to arbitrary coordinate perturbations. The different behavior of Protenix-v2 and ESMFold2 is therefore compatible with CycleCritic's underlying principle, which we independently validate across additional architectures. We also identify differences in confidence-head training that plausibly explain the observed behavior and agree that this distinction merits explicit discussion in the main text.

Reviewer yhU3

  • Novelty and baselines: We clarify that our contribution is adaptive allocation of recycling compute, rather than a new confidence metric or neural architecture. We added fixed 1- and 2-recycle baselines and further adaptive gating comparisons (Appendix N).
  • Hallucination improvements: We distinguish gains from increased sampling throughput in Mosaic from improvements in per-design quality. For ESMFold2, controlled ablations demonstrate that learned CycleCritic guidance outperforms training-free distogram proxies (Appendices F–H).
  • False negatives and binding-mode diversity: We show that 97.3% of highly reproducible designs survive early gating. Additional analyses show that CycleCritic maintains comparable epitope-mode coverage to full ptx@10, including on difficult targets (Appendices K, M).
  • Generalisation and reliability: New experiments establish transfer to minibinders and additional folding architectures. We also clarify that CycleCritic inherits potential limitations of its teacher's confidence estimates, rather than claiming improved experimental structure accuracy (Appendices I, J).

Remaining limitations: We acknowledge reviewer yhU3's request for a fully isolated Mosaic loss ablation remains open. Nevertheless, we believe that the extensive new experiments directly address the principal concerns raised across the three reviews.

Overall contribution: The revised manuscript now provides substantially stronger evidence that CycleCritic enables more efficient allocation of folding compute while preserving high screening fidelity, with applicability across multiple co-folding architectures and binder-design modalities. We hope the AC will take these substantial additions into account when assessing the paper's methodological and practical contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.