- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AbGaze, an antibody design framework that incorporates orientation-aware atom–surface representations into a dyMEAN-based architecture. The model encodes distances, approach directions, and surface-normal incidence in antibody-residue local frames, aggregates these features through attention, and uses geometric supervision to support sequence–structure design. Evaluation covers all-CDR design, CDR-H3 design, complex structure prediction, and affinity optimization.
Critical strengths
-
Clear geometric motivation. Explicitly representing how antibody atoms approach an antigen surface is a reasonable approach to capturing interface geometry beyond scalar distances.
-
Shared representation across tasks. The same representation supports multiple antibody modeling tasks, and the reported results suggest benefits for both sequence recovery and interface modeling.
-
Consistent benchmarking protocol. The authors re-run baselines under a shared evaluation protocol and remove training clusters overlapping with the held-out benchmark.
Critical weaknesses
-
Affinity optimization lacks independent validation. The same frozen ΔΔG regressor provides the optimization signal and evaluates the resulting candidates. Thus, the reported improvements primarily demonstrate successful optimization of this proxy and may reflect exploitation of the scorer. An independent evaluator or experimental measurements are needed to substantiate improved binding affinity.
-
Design evaluation uses oracle selection. Selecting the highest-AAR candidate among five samples requires access to the native sequence, which is unavailable in practical design. Although applied consistently across methods, this protocol measures an oracle sampling outcome rather than deployable performance. Mean performance across candidates and top-1 performance under a ground-truth-free selection rule should be reported.
-
Generalization evidence is limited. All four tasks are evaluated on the same 60-complex RAbD benchmark. CDR-H3-based clustering controls one source of similarity, but does not establish generalization to unseen antigen families. Additional benchmarks or antigen-disjoint evaluation would strengthen the conclusions.
-
Ablations do not fully isolate the proposed contributions. Replacing the entire orientation-aware module with distance-based message passing changes both geometric features and aggregation. Separate ablations of approach direction, surface-normal information, and attention are needed to identify the source of improvement.
-
Some gains are modest and require uncertainty estimates. For complex structure prediction, DockQ improves from 0.424 to 0.435 over the strongest baseline. Paired statistical analysis, confidence intervals, and per-target results would help establish the robustness of this improvement.
-
Sequence recovery is insufficient to establish design utility. Higher AAR demonstrates improved native-sequence reconstruction, but does not necessarily imply better binding or developability. The lack of experimental validation limits the practical conclusions.
Questions for authors
Do the affinity improvements persist when evaluated by an independent scorer that was not used during optimization, and when mutation counts are matched across methods?
How do the design results change when reporting mean performance across all five candidates or selecting candidates using predicted confidence?
Can the direction features, surface-normal features, and attention mechanism be ablated separately?
Are the interface improvements statistically significant, and do they persist on antigen-disjoint or additional held-out benchmarks?
How much computational overhead does the surface representation introduce relative to dyMEAN and AbFlow?
6 comments
Thank you for the detailed response and additional experiments. The component-wise ablations directly address my concern about attributing improvements to the individual geometric features and attention mechanism. The deterministic single-prediction results also substantially alleviate the concern that the sequence-recovery gains depend on oracle selection. I acknowledge these improvements to the evidence.
I also accept the clarification that optimization and final affinity evaluation use different predictors, and revise my original criticism accordingly. However, since the optimization predictor is supervised using outputs from the final evaluator, the two remain coupled. The reported affinity gains therefore demonstrate optimization of an evaluator-derived objective, while independent evidence of improved binding remains limited.
My main remaining concern is the strength of the interface-modeling evidence. All three reported bootstrap confidence intervals for DockQ differences include zero. This does not establish the absence of a benefit, but it leaves the modest average improvements uncertain. The statement that win rates exceed 50% also needs qualification, as the comparison against HERN reports 46.5%.
The additional SAbDab evaluation and Epitope-F1 results are useful. Nevertheless, the former does not establish antigen-family-disjoint generalization, and the latter measures reference contact recovery rather than independently validating functional binding. Experimental validation is not essential for every methodological paper, but stronger independent computational evidence would help substantiate the practical claims here.
Overall, I view AbGaze as a well-motivated geometric representation improvement, and the rebuttal addresses several concerns constructively. However, uncertainty around the interface gains and the evaluator-coupled affinity assessment still limit the evidence for the claimed advances. I therefore maintain my Weak Reject recommendation.
We thank the reviewer for the constructive follow-up and for recognizing the value of our component-wise ablations and deterministic evaluation. While we agree that statistical uncertainty in structure prediction and evaluator coupling in affinity optimization require careful framing, we want to clarify why the interface-level evidence carries substantial weight for our core contribution.
1. Core Evidence for Antibody Design and Interface Fidelity
AbGaze is designed for antigen-conditioned antibody sequence–structure co-design, rather than standalone structure prediction or affinity optimization. We included the latter two solely to probe whether our geometric representation generalizes beyond its primary setting.
Within the core design task, AbGaze consistently improves sequence recovery across CDRs under the unified benchmark. Crucially, these gains do not rely on oracle selection or iterative sampling: under single deterministic inference, AbGaze achieves 64.0% AAR and 0.407 DockQ. Component-wise ablations confirm that directional encoding, surface normals, and adaptive atom–surface attention each directly drive this performance.
More importantly, Epitope-F1 is far more than a secondary metric—it is the foundational benchmark of spatial recognition in structure-based design. On all-CDR design, AbGaze achieves 0.449 Epitope-F1, outperforming AbFlow (0.420) and dyMEAN (0.408). While Epitope-F1 evaluates reference contact recovery rather than raw binding energy, capturing the true paratope–epitope interface is the ultimate litmus test for whether a model truly understands antigen recognition. Achieving superior Epitope-F1 proves that AbGaze’s orientation-aware geometric interaction mechanism genuinely steers the model toward the correct biological hotspots on the antigen surface, rather than relying on sequence memorization or arbitrary packing. This spatial fidelity is the strict prerequisite for any meaningful functional design.
2. Supplementary Structure-Prediction Results
We acknowledge that the DockQ confidence intervals include zero, meaning the gains do not reach statistical significance. We also thank the reviewer for catching the win-rate error; the original comparison against HERN was indeed 46.5%.
To further test structural refinement, we evaluated a task-specifically fine-tuned variant against HERN:
| Comparison | Mean ΔDockQ | 95% CI | Win Rate | Median ΔDockQ |
|---|---|---|---|---|
| Fine-tuned AbGaze vs HERN | +0.0176 | [−0.0108, +0.0558] | 53.3% | +0.0040 |
Although this fine-tuned variant yields a higher mean ΔDockQ than the unified model (+0.011), its confidence interval still crosses zero. We present this strictly as supplementary evidence rather than claiming statistical significance. Uncertainty in this auxiliary task, however, does not undermine the clear spatial and sequence gains established in our primary design benchmarks.
3. Scope of Affinity and Generalization Claims
We agree that the optimization predictor and final evaluator remain coupled through shared supervision; thus, the reported affinity gains reflect optimization under the evaluator-derived objective rather than independent biochemical validation. Similarly, we recognize that our SAbDab evaluation does not guarantee strict antigen-family-disjoint generalization. We will explicitly state these boundaries in the revised manuscript.
Ultimately, these limitations in secondary tasks should be weighed separately from our primary contribution: an orientation-aware geometric interaction mechanism whose effectiveness is solidly backed by rigorous sequence–structure design benchmarks, deterministic inference, state-of-the-art interface recovery (Epitope-F1), and targeted ablations.
We thank the reviewer for recognizing the geometric motivation, unified multi-task representation, and comprehensive evaluation of AbGaze. We provide the following clarifications and additional experimental results.
W1 & Q1. Affinity Optimization Protocol
We clarify an important distinction in our affinity optimization protocol: optimization and final evaluation use different ΔΔG predictors.
Following dyMEAN and AbFlow, each method employs its own predictor to guide optimization. For AbGaze, this is a lightweight MLP trained on the generator's representations, whereas final evaluation uses the same pretrained MEAN GNN-based ΔΔG evaluator across all methods.
The optimization predictors are supervised using scores from the GNN evaluator, so the two stages are not statistically independent. Nevertheless, they involve distinct architectures and parameters rather than optimizing and evaluating with the same predictor.
We have revised Section 4.5 and Appendix C to explicitly distinguish these two stages, correcting the ambiguous description in the original manuscript.
W2 & Q2. Native-Independent Evaluation
The best-of-five AAR selection provides a standardized sequence-recovery benchmark across methods. We additionally evaluate AbGaze without native-sequence-based selection in Appendix F (Table 11).
With a single deterministic prediction, AbGaze achieves 64.0% AAR and 0.407 DockQ, outperforming the reported results of dyMEAN (60.1%, 0.396) and AbFlow (59.7%, 0.379).
Moreover, Appendix E (Table 10) demonstrates the diversity of our generative decoding: five samples yield an average of 4.92 distinct CDR-H3 sequences, with a mean pairwise Hamming distance of 0.458.
These results demonstrate strong single-draw performance alongside diverse candidate generation.
W3. Generalization Beyond RAbD
Our evaluation extends beyond RAbD. Appendix D (Table 9) additionally evaluates AbGaze on the SAbDab test split used by IgGM.
AbGaze outperforms IgGM on five of six CDRs, including H2 (0.694 vs. 0.644), H3 (0.397 vs. 0.360), and L3 (0.677 vs. 0.635).
These results provide additional evidence across test sets, complementing the four-task evaluation on RAbD. Strict antigen-family-disjoint generalization remains a separate evaluation setting.
W4 & Q3. Component-wise Ablations
We have added Appendix G (Table 12), independently ablating directional encoding, surface-normal information, and adaptive attention.
Removing these components decreases AAR by 0.014, 0.018, and 0.021, respectively, and DockQ by 0.020, 0.011, and 0.023. Removing directional encoding and attention also increases RMSD by 0.897 and 0.894, respectively.
Together with the module-level ablation in Table 5 (AAR −4.1 percentage points; DockQ −0.040), these results demonstrate the contributions of individual geometric components to sequence recovery and interface modeling.
W5 & Q4. Statistical Reliability
We conducted paired bootstrap analysis with 10,000 resamples to quantify the uncertainty of DockQ improvements.
Both comparisons exhibit positive mean and median improvements, with win rates above 50%.
W6. Practical Significance Beyond AAR
Our evaluation extends beyond sequence recovery to structural and interface metrics, including RMSD, lDDT, TM-score, and DockQ, which measures the accuracy of predicted antibody–antigen binding configurations.
We further introduce Epitope-F1, following the evaluation approach of ChiMERa-Bench. This metric measures agreement between the antigen residues contacted by the designed antibody and the reference epitope, combining contact precision and recall.
On all-CDR design, AbGaze achieves 0.449 Epitope-F1, outperforming AbFlow (0.420) and dyMEAN (0.408), corresponding to relative improvements of 6.9% and 10.0%, respectively.
Together with DockQ, these results demonstrate improvements in predicted interface geometry and epitope contact recovery, beyond native sequence reconstruction.
Q5. Computational Overhead
We additionally compare the computational costs of AbGaze with existing methods. AbFlow requires 4.56 s for training and 1.4 s per inference step (14s in total), while dyMEAN requires 2.21 s and 0.47 s, respectively.
Under the corresponding evaluation settings, AbGaze requires 6.34 s for training and 1.8 s per inference step (18 s in total).
These measurements characterize the computational cost of incorporating our orientation-aware atom–surface representation and adaptive attention mechanism.