acceptodds
Under review as a conference paper at ICLR 2027

Revisiting Structure-Based Filtering for AI-Designed Antibodies: Opportunities, Limits, and What Matters

Abstract

AI antibody design can generate far more candidates than can be experimentally tested, making post-generation filtering a central bottleneck. We study whether structure-prediction outputs can help prioritize AI-designed antibodies for experimental validation. We evaluate this problem on 4,184 experimentally characterized antibody–antigen pairs, combining newly generated wet-lab measurements with curated affinity data across diverse antigens and assay settings. We introduce two complementary filtering signals: Regional Interaction Contrast (RIC), which captures support for the predicted antibody–antigen interface, and Non-CDR Antibody Confidence (NAC), which measures antibody confidence outside the CDRs. Across AlphaFold 3, Protenix-v2, and ESMFold2, we find a striking difference in what information matters for filtering. Interface evidence is strongest when comparing candidates designed for different antigens, whereas antibody-level reliability is substantially more informative for ranking candidates within the same antigen. In particular, non-CDR antibody confidence consistently outperforms whole-antibody confidence across predictors, revealing an unexpected source of filtering signal beyond the predicted interface. Finally, we examine the information encoded in the predictor outputs themselves. Predicted structures contain substantial but incomplete binding-related signal, revealing both the opportunities and limits of structure-based filtering: better metrics can exploit current predictors more effectively, but their performance ultimately remains constrained by what those predictors can distinguish

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI antibody design, antibody-antigen binding, in silico screening, protein complex structure prediction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FSGUOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a new structure-based antibody filtering score. Structural confidence between antibody and antigen has been considered as "silver" standard or proxy of binding, despite its performance is concerning (e.g., IPTM, IPSAE). This paper aims to overcome the challenge by proposing new confidence metrics.

Critical strengths
  1. Thorough experiments with diverse baselines.
  2. The research question is of high significance. We all want to know what the best metrics are for the structure-based antibody filtering score.
Critical weaknesses

The conclusion is somewhat counterintuitive and not reliable. What would be the point of comparing the interface score across different antigens? Researchers use the interface score to shortlist good antibody designs given the same antigen. So this paper's take-home message is that antibody-level (outside of CDR) reliability is more informative than interface (CDR) for affinity ranking. This is not an "unexpected source filtering" signal. This is just reflecting how the experimental SPR data is constructed. My speculation is that the designs tested in the literature are not focused on CDRs (maybe framework optimization or incremental affinity maturation outside of CDRs). This paper contains several overstated claims (e.g., "unexpected source of filtering", overselling by comparing across antigens).

Expanding the experiments to wide variety of affinity data or performing your own wet lab test would strongly support the claim.

Questions for authors

None

1 comment

Dear Reviewer FSGU,

As the author–reviewer discussion period is nearing its end, we would be grateful if you could take a look at our rebuttal and revised manuscript when your schedule permits.

We would welcome any further questions or feedback and would be happy to continue the discussion. We understand that this is a busy period and sincerely appreciate your time and thoughtful consideration.

Best regards,

The Authors

Reviewer_bDDwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates structure-based post-generation filtering of AI-designed antibodies, where the objective is to identify candidates likely to bind their intended antigens while minimizing the number of expensive wet-lab experiments. The authors assemble a benchmark of 4,184 experimentally characterized antibody–antigen pairs from three sources: display-based enrichment experiments, the IgDesign SPR benchmark, and affinity measurements curated from publications and patents. They evaluate three structure-prediction models—AlphaFold 3, Protenix-v2, and ESMFold2—using multiple prediction seeds and samples. The paper introduces two filtering metrics:

  • Regional Interaction Contrast (RIC): Measures whether the most strongly predicted CDR–antigen contact regions receive consistently stronger structural support than alternative regions, using contact probabilities and predicted aligned error across prediction runs.
  • Non-CDR Antibody Confidence (NAC): Measures the average structural confidence, expressed through pLDDT, of antibody residues outside the complementarity-determining regions. The authors compare these metrics against existing structural confidence measures, including ipTM, pDockQ, pDockQ2, ipSAE, AntiConf, and other predictor-derived confidence scores. The experiments consider two selection scenarios: portfolio-level filtering across different antigens using a shared threshold, and campaign-specific filtering in which candidates are ranked within individual antigens. The principal findings are that RIC achieves the strongest performance in global filtering, while NAC consistently performs well in campaign-specific filtering. NAC also outperforms whole-antibody pLDDT in 23 of 24 predictor–metric comparisons reported for within-antigen evaluation. Finally, the paper investigates how much binding-related information is encoded in structure-prediction outputs. The authors find that predicted coordinates and confidence features contain substantial, but incomplete, information for distinguishing binders from non-binders.
Critical strengths
  1. Important and practically motivated problem. The paper addresses a genuine bottleneck in computational antibody discovery. The emphasis on selecting experimental candidates under limited screening budgets is well motivated and makes the results relevant beyond conventional structure-prediction benchmarking.
  2. Broad experimental evaluation. The benchmark includes 4,184 experimentally characterized antibody–antigen pairs, spanning 179 antigen constructs and multiple experimental modalities. Evaluating the methods across AlphaFold 3, Protenix-v2, and ESMFold2 provides useful evidence that the main observations are not specific to one structure predictor.
  3. Interesting and somewhat counterintuitive empirical finding. The strong performance of NAC in within-antigen filtering is arguably the paper's most important contribution. In particular, the observation that removing CDR residues improves filtering performance in 23 of 24 comparisons challenges the intuitive expectation that confidence in binding-related regions should be the most informative signal.
  4. Careful distinction between evaluation scenarios. The authors distinguish global filtering across antigens from candidate ranking within a particular antigen. This distinction is meaningful in practical discovery pipelines. Appendix A.1 further compares these settings on a matched candidate population, helping establish that the observed performance reversal is not simply caused by using different datasets.
  5. Comprehensive baseline comparisons and useful supplementary analyses. The evaluation includes several established interface-confidence measures and reports multiple operating points, including recall at fixed false-positive rates and fixed experimental budgets. The additional analyses of regional pLDDT, ensemble size, and threshold transfer help characterize the methods' behavior and limitations.
Critical weaknesses
  1. Limited generalization to prospective antibody-design campaigns. Although the paper is motivated by AI-designed antibodies, much of the benchmark consists of display-selected antibodies or candidates curated from existing literature and patents. These populations may differ substantially from the outputs of contemporary generative antibody-design models in sequence diversity, structural quality, and failure modes. Consequently, the experiments establish that structure-prediction outputs contain useful binding-related information, but they provide less direct evidence that the proposed metrics improve the hit rate of a prospective AI-design campaign. A validation on candidates generated by a modern antibody-design method, ideally with independent experimental measurements, would substantially strengthen the contribution.
  2. Optimistic performance estimates from threshold selection on evaluation labels. The main tables report utility after selecting thresholds that maximize performance on the evaluated candidates. This provides a useful retrospective comparison of score separability but does not directly estimate performance in a new campaign where binding labels are unavailable. Importantly, the authors recognize this limitation and conduct a leave-one-antigen-out threshold-transfer analysis in Appendix A.5. The results are concerning: for AlphaFold 3, NAC achieves an oracle utility of 0.1791 but a transferred utility of 0.0000, with the transferred cutoff selecting candidates for only 1 of 13 held-out antigens. Similar problems appear for the other predictors. This considerably limits the operational interpretation of the utility results. Fixed-budget top-QQ selection partly addresses the issue, but the paper would benefit from making this distinction more prominent and prioritizing prospective evaluation metrics.
  3. Potential confounding in the NAC signal. The superior performance of non-CDR pLDDT is interesting, but its underlying mechanism remains insufficiently understood. High non-CDR confidence may reflect properties other than binding affinity, including sequence familiarity, framework stability, antibody family, sequence similarity to training data, or other properties correlated with the experimental labels. The authors demonstrate that NAC is predictive, but the evidence does not yet establish whether this signal is directly associated with binding or primarily reflects correlated sequence and structural properties. More targeted controls would help clarify the interpretation and generalizability of the finding.
  4. Heterogeneous experimental labels and possible selection biases. The three benchmark components use different definitions of binding, including enrichment-based classifications and direct affinity measurements. In particular, display enrichment may depend on expression, growth, and experimental selection effects rather than binding alone. The class distributions also differ substantially: the literature/patent benchmark contains 1,510 positives and only 103 negatives, whereas the display-based dataset is balanced by construction. These differences complicate interpretation of pooled global filtering performance and may introduce benchmark-specific signals. Although the paper acknowledges these limitations, additional source-stratified analyses and uncertainty estimates would strengthen the conclusions.
  5. Limited evidence of the practical benefit relative to computational cost. The experimental protocol uses up to 50 predicted complexes per antibody–antigen pair and predictor. This is a substantial computational investment for a post-generation filtering stage intended to reduce discovery costs. Appendix A.4 provides a useful ensemble-size sensitivity analysis. However, a more explicit cost–performance comparison would help determine whether the gains from RIC justify its additional computational complexity relative to simpler metrics, particularly NAC and conventional pLDDT summaries.
  6. The methodological novelty is moderate. RIC introduces a relatively elaborate combination of contact ranking, PAE agreement, and aggregation across prediction runs. NAC is a simple regional average of an existing confidence metric. The contribution is therefore primarily empirical rather than algorithmic. This is not inherently problematic, particularly given the importance of the observed findings, but it increases the importance of rigorous evaluation, mechanistic understanding, and convincing evidence of practical impact.
Questions for authors

Q1. Generalization to AI-generated antibodies. Given that a substantial portion of the benchmark consists of display-selected or literature-derived antibodies, how confident are the authors that the observed superiority of NAC and RIC will generalize to candidates produced by modern generative antibody-design models? Could the authors provide a separate analysis restricted to genuinely AI-designed candidates, or evaluate the methods on a prospective design campaign? Q2. Understanding the NAC signal. What explains the strong predictive performance of non-CDR pLDDT? Could it be driven by framework stability, sequence familiarity, or antibody-family-specific properties rather than binding-related information? It would be valuable to investigate whether NAC remains predictive after controlling for sequence similarity, framework identity, and other potentially confounding factors. Q3. Threshold calibration and practical deployment. The leave-one-antigen-out results in Appendix A.5 show that NAC thresholds transfer poorly to unseen antigens, despite strong within-antigen ranking performance. How do the authors envision deploying NAC in a new campaign without labeled candidates? Would a fixed top-QQ strategy be the recommended approach, and can its effectiveness be validated prospectively? Q4. Statistical significance and robustness of the reported improvements. Can the authors provide confidence intervals or paired statistical tests for the improvements of NAC over whole-antibody pLDDT and other strong baselines? Since the within-antigen evaluation contains only 13 antigens, it would also be useful to understand whether the conclusions are robust to excluding individual antigens or assay groups. Q5. Complementarity of RIC and NAC. The paper presents RIC and NAC as complementary signals, but evaluates them primarily as individual filtering metrics. Have the authors investigated whether combining them improves performance? For example, could a simple weighted combination or a lightweight classifier trained on both scores outperform either metric alone, particularly under cross-antigen validation? Q6. Computational efficiency. How does the filtering performance change when the number of structure-prediction runs is reduced to a computationally practical level? Beyond the ensemble-size analysis already presented, could the authors quantify the additional computational cost per successfully recovered binder for RIC relative to simpler baselines? Q7. Influence of assay heterogeneity and benchmark construction. How sensitive are the conclusions to differences in binding-label definitions and positive/negative sampling strategies across the three benchmark sources? In particular, does the NAC advantage persist when evaluation is restricted to experimentally confirmed binding outcomes rather than display-enrichment proxies?

Reviewer_3UADOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies structure-based filtering for AI-designed antibodies, aiming to improve candidate selection for experimental validation. The authors propose two metrics: Regional Interaction Contrast (RIC), which measures support for predicted CDR-antigen interactions, and Non-CDR Antibody Confidence (NAC), which aggregates pLDDT outside the CDRs. Evaluating 4,184 antibody-antigen pairs with AlphaFold 3, Protenix-v2, and ESMFold2, the authors find that interface-related confidence performs better across antigens, while non-CDR confidence is more effective for ranking candidates against the same antigen.

Critical strengths

Effective in silico filtering is increasingly important as AI antibody design pipelines can generate thousands of candidates. Improving upon commonly used metrics such as ipTM and ipSAE is a worthwhile research direction. The paper provides an extensive empirical evaluation across multiple structure predictors and experimental datasets, with useful additional analyses on binder prevalence, ensemble size, and threshold transfer. The finding that non-CDR confidence can outperform interface-related metrics in predicting binding is interesting and worth investigating further.

Critical weaknesses

The proposed metrics are relatively simple modifications or combinations of existing structure-prediction confidence signals. NAC essentially averages non-CDR pLDDT, while RIC combines existing contact and PAE signals. While simplicity is not necessarily a weakness, I would expect more substantial improvements over existing metrics to justify the contribution. At a fixed 20% testing budget (Table 3), NAC improves recall over whole-antibody pLDDT by only 2.43, 0.03, and 2.01 percentage points for AlphaFold 3, Protenix-v2, and ESMFold2, respectively. Similarly, despite favorable portfolio-wide results, RIC achieves only 10.81% and 14.41% recall with AlphaFold 3 and Protenix-v2 in the matched-population analysis (Appendix A.1), both below the approximately 20% random-selection expectation. Appendix A.5 also shows that filtering thresholds transfer poorly to unseen antigens. Overall, I am not convinced that the proposed metrics offer a sufficiently meaningful practical improvement over existing approaches.

I find the evaluation somewhat disconnected from the intended application in modern AI antibody design pipelines. The benchmark consists of display-selected antibodies, IgDesign candidates, and literature/patent data. Only IgDesign directly involves computational design, and it focuses on CDR redesign using native complex backbones rather than de novo design. Display enrichment is also an indirect binding readout, while the literature/patent dataset is heavily skewed toward positives (1,510 versus 103 negatives). Although the authors include useful robustness analyses, these datasets may not reflect the distributions and challenges of actual AI-generated candidate libraries. I would have liked to see an evaluation that more closely simulates a realistic design campaign, comparing filtering strategies on computationally generated candidates under a fixed experimental budget. This would provide more direct evidence of the practical value of the proposed methods. I think the observation that non-CDR confidence predicts binding better than CDR or interface confidence is interesting, but the paper does not offer much insight into why this happens. For example, is NAC capturing intrinsic antibody foldability, differences between antibody frameworks or sequence families, or antigen-dependent information introduced during co-folding? Comparing NAC from complex predictions against independently folded antibody monomers, or controlling for sequence families, could help clarify this. The paper also does not establish whether RIC and NAC provide complementary information that could improve existing filtering pipelines. I think a deeper investigation into these observations would make the work more informative beyond proposing additional filtering metrics.

Questions for authors

Can the authors evaluate RIC and NAC on candidates generated by modern de novo antibody design methods under a fixed experimental budget, and compare them against standard ipTM/ipSAE filtering?

Does NAC remain predictive when computed from independently folded antibody monomers or after controlling for antibody frameworks and sequence families?

Can the authors provide paired uncertainty estimates for NAC's improvements over whole-antibody pLDDT and investigate whether combining RIC and NAC with existing metrics improves filtering?

1 comment

Dear Reviewer 3UAD,

As the author–reviewer discussion period is nearing its end, we would be grateful if you could take a look at our rebuttal and revised manuscript when your schedule permits.

We would welcome any further questions or feedback and would be happy to continue the discussion. We understand that this is a busy period and sincerely appreciate your time and thoughtful consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.