- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PPFrame, a multimodal diffusion framework for protein–protein complex design, aiming to address the insufficient sequence–structure consistency in existing binder design methods and the difficulty of simultaneously satisfying binding interface geometry and physicochemical complementarity.
Specifically, PPFrame combines discrete amino acid sequence diffusion with continuous SE(3) structure diffusion to enable joint sequence–structure generation conditioned on the target protein. The authors designed IPAFormer, which alternates Invariant Point Attention and Transformer Self-Attention to integrate 3D geometric information with sequence dependencies, and introduces residue charge and hydrophobicity features as well as a self-conditioning inference mechanism to improve generation quality.
In addition, the authors constructed PPIBench from the PDB, containing 602,364 oriented complex samples, for model pretraining, and further fine-tuned the model on Mini-binder and antibody design tasks. Experimental results show that PPFrame achieves overall better performance than the selected baseline methods across the three tasks in terms of computational success rate, predicted binding energy, and sequence–structure consistency. The paper also uses ablation experiments to analyze the roles of components such as physicochemical features, self-conditioning inference, joint generation, and auxiliary losses.
Critical strengths
The first strength is well-motivated and coherent methodological design.The paper addresses an important challenge in protein-protein complex design by jointly modeling protein sequences and structures. The integration of discrete sequence diffusion, SE(3) structural diffusion, and physicochemical features provides a reasonable framework for improving sequence-structure consistency and binding interface complementarity.
The second strength is consistent empirical improvements across multiple protein design tasks. The proposed method achieves competitive performance across general protein complex design, mini-binder design, and antibody design, outperforming the evaluated baselines in average success rates and predicted binding energies. Notably, PPFrame demonstrates strong sequence-structure consistency, achieving 99.39% co-designability in antibody design. These results provide empirical evidence for the effectiveness of the proposed framework across different protein design scenarios.
Critical weaknesses
The first weakness is lack of direct comparison with the most closely related work, PPDiff. PPFrame and PPDiff address highly similar research problems and share substantial similarities in their experimental settings. In particular, their mini-binder and antibody benchmarks have the same data sources and identical dataset split statistics. However, PPFrame does not include PPDiff as a baseline or provide a direct comparison under a unified evaluation protocol. Since the reported success rates and candidate generation settings are not fully consistent across the two papers, the results reported separately cannot establish the relative performance of the two methods. This makes it difficult to assess whether PPFrame provides a meaningful improvement over existing joint sequence-structure diffusion approaches. A direct comparison using consistent data splits, evaluation metrics, and candidate generation budgets would substantially strengthen the paper's claims regarding its methodological contributions.
The second weakness is insufficient ablation studies to validate the key methodological contributions.The paper presents several ablation studies examining physicochemical features, self-conditioning, auxiliary losses, and joint sequence-structure generation. However, the two central methodological components emphasized by the authors, namely SE(3) structural modeling and the IPAFormer architecture, are not independently evaluated through controlled comparisons. More targeted ablation studies are needed to substantiate the claimed methodological advances.
The third weakness is insufficient code documentation and reproducibility support. The anonymous code repository does not appear to provide sufficient documentation and environment specifications to facilitate reproduction of the reported results. In particular, the lack of clear setup instructions, dependency specifications, and a well-documented workflow for data preparation, training, and evaluation makes it difficult to determine whether the proposed method and experimental results can be independently reproduced.
Questions for authors
Question1:Comparison with PPDiff. PPFrame and PPDiff address highly similar research problems and share substantial similarities in their experimental settings. In particular, their mini-binder and antibody benchmarks have the same data sources and identical reported dataset split statistics. However, PPFrame does not include PPDiff as a baseline or provide a direct comparison under a unified evaluation protocol. Since the reported success rates and candidate generation settings differ between the two papers, the results reported separately do not allow a fair assessment of their relative performance. Could the authors clarify whether the exact same sample-level training, validation, and test splits are used in both works? More importantly, could they provide a direct comparison with PPDiff using consistent evaluation metrics, candidate generation budgets, and selection strategies? Such a comparison would help establish whether PPFrame offers meaningful improvements over existing joint sequence-structure diffusion methods.
Question2: Ablation Studies on the Core Methodological Components. The paper emphasizes SE(3) structural modeling and the IPAFormer architecture as key methodological contributions. However, the current ablation studies do not independently evaluate their contributions. Specifically, I would like to ask the following questions: What is the performance gain from SE(3) frame-based diffusion compared with structure diffusion based solely on Cα coordinates? Can the authors quantify the contribution of this change under otherwise identical experimental settings? What is the independent contribution of IPAFormer compared with existing denoising architectures? Can the authors provide controlled comparisons to demonstrate the benefits of the proposed architecture? What are the individual contributions of IPA, Transformer self-attention, and ESM-2 embeddings? Can the authors conduct component-wise ablation studies to clarify how each component contributes to the overall performance improvements?
3 comments
Weakness3: The third weakness is insufficient code documentation and reproducibility support. In particular, the lack of clear setup instructions, dependency specifications, and a well-documented workflow for data preparation, training, and evaluation makes it difficult to determine whether the proposed method and experimental results can be independently reproduced.
Ans: We thank the reviewer for highlighting this issue and agree that clear documentation is essential for reproducibility. We have added a detailed README with environment setup instructions and step-by-step guidance for training and evaluation. These additions clarify how to run the code and support independent reproduction of the reported experiments.
Q2.1: What is the performance gain from SE(3) frame-based diffusion compared with structure diffusion based solely on Cα coordinates? Can the authors quantify the contribution of this change under otherwise identical experimental settings?
Ans: According to the reviewer’s suggestion, we constructed a Cα-only variant of PPFrame by replacing the IPA layers with equivariant graph convolutional layers and adapting the two auxiliary losses to the Cα representation. We trained and evaluated this variant on PPIBench.
Compared with the Cα-only variant, PPFrame improves sequence design success rate by 21.75 percentage points. The largest gains occur in structural designability and co-designability, which increase by 54.35 and 56.53 percentage points, respectively. These results support the benefits of PPFrame’s full-backbone modeling framework, particularly for structural fidelity and sequence-structure consistency. We have updated the results in Appendix Section C.5 and Table26 in our manuscript.
| ipTM | pLDDT | SR | Diversity | Novelty | Delta G | Designability | Co-designability | |
|---|---|---|---|---|---|---|---|---|
| Cα-only variant | 0.539 | 83.14 | 26.08% | 50.39% | 71.86% | -38.27 | 10.87% | 8.69% |
| PPFrame | 0.592 | 87.18 | 47.83% | 33.22% | 57.49% | -67.80 | 65.22% | 65.22% |
Q2.2: What are the individual contributions of IPA, Transformer self-attention, and ESM-2 embeddings?
Ans: We thank the reviewer for requesting a clearer breakdown. Our ablations assess the contributions of these components within PPFrame as follows:
IPA layers. As reported in our response to Weakness 2, removing IPA reduces ipTM from 0.592 to 0.326, pLDDT from 87.18 to 72.18, and sequence design success rate from 47.83% to 0%. These results support the importance of geometric information for generating sequences with high interface confidence.
Transformer self-attention. We have run the experiments in Section 5.4 (Fig. 2(d)). Removing sequence self-attention reduces structural designability from 65.22% to 41.30%. This demonstrates that sequence-context modeling contributes to structural design quality, supporting the complementary roles of sequence attention and IPA.
ESM-2 initialization. We additionally trained and evaluated a variant without ESM-2 initialization on PPIBench. As shown below, incorporating ESM-2 initialization improves sequence design success rate, alongside improvements in binding energy, structural designability and co-designability. These results suggest that pretrained sequence knowledge contributes primarily to sequence design quality, with additional benefits for structural quality and consistency. We have updated the results in Appendix Section C.5 and Table25 in our manuscript.
| ipTM | pLDDT | SR | Diversity | Novelty | Delta G | Designability | Co-designability | |
|---|---|---|---|---|---|---|---|---|
| Without ESM2 embedding | 0.568 | 82.16 | 34.78% | 46.15% | 57.21% | -60.79 | 63.04% | 60.86% |
| PPFrame | 0.592 | 87.18 | 47.83% | 33.22% | 57.49% | -67.80 | 65.22% | 65.22% |
We welcome any further questions or comments and would be happy to provide additional clarification.
I appreciate the clarifications regarding PPDiff. However, the differences described mostly amount to recombining existing components—structural representations, architectural modules, training objectives, and inference strategies. While the overall framework performs well, the evidence presented is not sufficient to establish a clear and independent methodological contribution. For this reason, I will maintain my rating of 2 (Weak Rejection).
Thanks for reviewing our paper and giving detailed feedback. Our responses are listed below:
Weakness1 and Q1: The first weakness is lack of direct comparison with the most closely related work, PPDiff. A direct comparison using consistent data splits, evaluation metrics, and candidate generation budgets would substantially strengthen the paper's claims regarding its methodological contributions. Could the authors clarify whether the exact same sample-level training, validation, and test splits are used in both works? More importantly, could they provide a direct comparison with PPDiff using consistent evaluation metrics, candidate generation budgets, and selection strategies?
Ans: We thank the reviewer for highlighting this connection and suggesting a direct comparison. We would like to first emphasize that PPFrame introduces substantive changes beyond replacing EGCL with IPA: a full-backbone representation (four atoms) based on residue frames, explicit physicochemical feature guidance, a more complex diffusion process, auxiliary structural supervision, and self-conditioned inference. Our ablations support the contributions of physicochemical features, auxiliary losses, and self-conditioned inference, providing evidence that the improvements extend beyond the choice of geometric attention layer.
Moreover, PPBench provides Cα-only coordinates, consistent with PPDiff’s Cα-only representation. In contrast, as described in Section 3.1 Problem Definition, PPFrame models backbone geometry involving N, Cα, C, and O atoms and therefore requires these coordinates for training and evaluation. Therefore, we curated PPIBench to support PPFrame training, leading to a different training/validation/test split between these two datasets.
To directly address the reviewer’s request, we trained and evaluated PPDiff on PPIBench. As shown below, PPFrame achieves a higher sequence design success rate (47.83% versus 28.26%) and more favorable binding energy (−67.80 versus −28.16). The largest differences concern structural fidelity: PPFrame achieves 65.22% on both designability and co-designability, compared with 0% for PPDiff under this evaluation. We also observed frequent steric clashes in PPDiff’s generated structures, consistent with its lower structural scores. The results demonstrate PPFrame’s advantage in producing designs that jointly satisfy sequence-quality and structural-consistency criteria. We have updated the results in Section5.9 and Table6, as well as the discussion with PPDiff in Appendix Section D in our manuscript.
Weakness2: The second weakness is insufficient ablation studies to validate the key methodological contributions. However, the two central methodological components emphasized by the authors, namely SE(3) structural modeling and the IPAFormer architecture, are not independently evaluated through controlled comparisons. More targeted ablation studies are needed to substantiate the claimed methodological advances.
Ans: We thank the reviewer for emphasizing the need for targeted ablations. To examine the contributions of the SE(3) structural modeling in IPAFormer, we trained a variant that removes the IPA layers, and evaluated it on the general protein-protein complex design task.
Removing IPA layers degrades sequence design quality. The resulting designs exhibit lower interface and structural confidence, with ipTM decreasing from 0.592 to 0.326 and pLDDT from 87.18 to 72.18. The sequence design success rate decreases from 47.83% to 0%, supporting the importance of geometric information for sequence generation. We have updated the results in Appendix Section C.5 and Table25 in our manuscript.