- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PLDesign proposes a model that projects the all-atom protein complex structure into a latent space. The latent space can be used for unconditional protein generation as well as target-conditioned binder design. A search algorithm applied in the latent space helps to better steer the generative process during inference time.
Critical strengths
-
Training an autoencoder to compress all-atom complex structure into latent space and further training flow-based generative model over the latent space is a novel and interesting exploration in the field of protein generative modeling.
-
Broad evaluation task coverage, spanning from reconstruction (to check latent quality) and de novo monomer/binder generation. Strong reconstruction performance.
-
Novel idea of search within the latent space.
Critical weaknesses
-
Metrics used for each task are not comprehensive, missing some important metrics. See details in Questions.
-
Relatively weak performance compared with baseline in generative tasks.
Questions for authors
Questions regarding model evaluation:
-
The table 1 shows strong dimer reconstruction performance of the PLDesign, which is very good. Besides the geometrical metrics such as RMSD, lDDT, and DockQ, can the authors also evaluate the side-chain quality? Things like bond length, dihedral angle distribution, or a combined metric such as MolProbity would make the reconstruction evaluation more comprehensive for side-chain quality.
-
In table 2, the authors benchmark PLDesign with other baselines on unconditional monomer design. For monomer generation with ProteinMPNN+ESMFold eval pipeline, related works usually report "designability" along with "diversity" measured using Foldseek. Why did the authors choose to report DPT instead of regular diversity? Also, given the PLDesign is an all-atom generative model, can the authors also evaluate the co-designability of the model to see its native sequence generation capability?
-
The in-silico success rate of binder design filtered by AF3 scores seems to be very low for PLDesign, compared with other baselines. Proteina-Complexa is not even included in the baselines in Table 23, let alone other hallucination-based methods such as BindCraft. Even though the AF3 scores do not correlate perfectly with wet-lab success, they are the best surrogate tools we can use to filter for actual binders. I can hardly say PLDesign, at the current stage, is a good model for binder design. Maybe the authors can try to further improve it by scaling up the training data or model size?
-
How does the PLDesign model's performance (reconstruction, de novo generation) scale with the size of the complex?
Questions about modeling:
-
From the architecture design, it seems to me that multimers with more than 2 chains can be encoded into latent space. However, the authors only trained on complexes with no more than 2 chains. Is there any specific reason that the authors did not try training and evaluating on samples with more than 2 chains? They are important test cases for multimer reconstruction and multi-chain target binder design tasks.
-
The search score
lat_self_lddtfunctionally trims diffusion trajectories with "outlier" lddt value in latent space, which makes it de-facto similar to low-temperature sampling. How does it compare to simply using lower temperature during the sampling process?