- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The authors propose SPINET, a protein inverse folding model that leverages molecular dynamics trajectories. SPINET employs cellular sheaves to propagate spatial information between residues within each frame, followed by an RNN to aggregate temporal information for each residue across frames. On the mdCATH 320 K dataset, the authors report that SPINET achieves a higher sequence recovery rate and structural confidence score than the compared baselines.
Critical strengths
This work extends protein inverse folding from conditioning on a single backbone structure or a few discrete conformations to leveraging entire molecular dynamics trajectories, addressing a relatively underexplored problem setting. The overall model design is also clear and well-structured.
Critical weaknesses
-
Comparisons with baselines are limited to 320 K. The main paper shows that SPINET outperforms PiFold, MapDiff, and DynamicMPNN only on the mdCATH test set at 320 K. No results for these baselines are provided at 348 K, 379 K, 413 K, or 450 K. Therefore, the results at 320 K do not establish that SPINET would consistently outperform these methods at other temperatures. Table 6 in Appendix D further raises concerns. The authors explicitly state that the main model suffered from overfitting in the temperature experiments. As a result, they used an earlier variant of SPINET for all five temperatures, removing the MLP after the RNN and the dropout before the RNN, reducing the number of channels from 16 to 8, and training for only 8 epochs at each temperature. This table does not include any baseline results. Moreover, neither the validation curves demonstrating the reported overfitting nor the test results of the final model at different temperatures are provided. I believe a fair comparison would require fixing the architecture and hyperparameters of the final SPINET model and training and evaluating it at each temperature under a consistent experimental protocol.
-
The baselines are not trained on the same type of structural input. PiFold, GVP, ScFold, and MapDiff use relaxed structures from the RCSB corresponding to the mdCATH domains. These models are neither trained nor evaluated using the molecular dynamics coordinates employed by SPINET. Although the data splits and sequence labels are aligned, the structural information available to the models is not comparable. The Traj–MLP baseline in Table 1 provides particularly relevant evidence. This model simply averages node features over time and has only approximately 23K parameters, yet it already achieves a top-1 sequence recovery rate of 40.6%. This exceeds PiFold (34.3%), ScFold (39.6%), GVP (27.0%), and the retrained DynamicMPNN (37.6%), while being comparable to the fine-tuned DynamicMPNN (40.7%). The fact that trajectory-derived geometric features without graph-based modeling already outperform most baselines suggests that a substantial portion of the observed improvements may come from the additional input information rather than the sheaf-based architecture itself. I suggest sampling multiple frames from each trajectory and using these frames to train and evaluate the static inverse folding models. Both single-frame performance and multi-frame predictions obtained by averaging logits should be reported. This would help disentangle the benefits of using trajectory information from those introduced by the sheaf-based spatial modeling and RNN-based temporal aggregation.
-
The conformational pair selection for DynamicMPNN also deviates from its original experimental setting. In the original paper, DynamicMPNN selects conformations with the largest RMSD or samples structurally diverse conformations using weights based on 1 − TM-score. In contrast, this paper first retains the 10 frames with the highest pairwise TM-scores and then selects the pair with the highest TM-score at test time, effectively choosing the two most structurally similar frames. Such a selection strategy provides very limited conformational diversity and may underestimate the performance of DynamicMPNN.
Questions for authors
-
Could the authors provide a comparison between the SPINET presented in the main paper and the baselines across different temperatures?
-
Could the authors explain why the sequence recovery rate at 450 K is substantially higher than that at 320 K in Table 6 of Appendix D?
-
Could the authors include a multi-state inference baseline for ProteinMPNN, where predictions are made on the same frames used by SPINET and the logits are averaged across frames before computing sequence recovery? Additionally, ADFLIP (Yi et al., ICML 2025) is also capable of multi-state protein design, and I suggest including its results for a more comprehensive comparison.
Summary of author-reviewer discussions
Dear Reviewers and AC,
As the discussion period draws to a close, we thank you for your time and thoughtful feedback. Your comments prompted a closer examination of our results, several clarifications, and a few additional experiments that helped strengthen the manuscript.
We appreciate the strengths recognized across the reviews. Reviewer Yxcm highlighted the sequence-recovery gains, the topology-separated evaluation, and the target-versus-decoy structural analysis. Reviewer fWGq recognized trajectory-conditioned inverse folding as a meaningful extension of the problem and noted that the architecture is motivated by changing residue interactions. Reviewer Dovy highlighted the relatively underexplored setting of inverse folding from MD trajectories and the clear overall model design.
SPINET predicts sequences from molecular dynamics trajectories, opening a path toward modifying an existing trajectory and proposing mutant sequences compatible with the desired conformational behavior. We believe this makes an important contribution to inverse protein design by incorporating protein motion into sequence generation, a capability central to designing switches, molecular motors, and enzymes whose function depends on conformational changes. Below, we summarize the clarifications, observations, and additional experiments prompted by the reviewers’ comments:
(1) Shuffling frames caused only a small decrease in sequence recovery. We explored shuffling frames within the MD trajectory windows and found that sequence recovery decreased only marginally. We believe this is because the variation in residue geometry and contacts over time carries more information than the precise frame order. Our learned restriction maps let neighboring residues exchange information about this variation, helping the model relate changes at one residue to changes at its neighbors. This interpretation is also supported by our Temporal GNN ablation, where replacing sheaf message passing with standard graph message passing substantially reduced recovery.
(2) BioEmu provides evidence of ensemble compatibility without target templates. Independent MD simulations of the designed sequences were infeasible during the discussion period. We therefore generated BioEmu ensembles from designed and wild-type sequences for 44 common mdCATH test domains. SPINET had the lowest mean contact error and tied DynamicMPNN on long-range contact error. These results support the ability of SPINET designs to generate ensembles with wild-type-like residue interaction patterns.
(3) We examined AF3 recovery without target templates. We ran AlphaFold 3 without templates or multiple sequence alignments and found low recovery and confidence even for the wild-type sequences. The wild-type control achieved a median TM-score of 0.215 and pLDDT of 55.27. This helped clarify why template-free structural recovery via AF3 provides a limited basis for judging the designed sequences.
(4) We clarified conformer selection and expanded our DynamicMPNN comparisons. Our original description contained a typo. We accidentally stated that we maximized TM-score to select conformations for training DynamicMPNN. We actually selected structurally dissimilar conformations by maximizing 1 − TM-score and have corrected this description in the revised manuscript. We also evaluated larger conformer sets, which improved DynamicMPNN’s top-1 recovery to 55.4%, compared with SPINET’s 56.7%. This increased computational cost, with training taking approximately 11 minutes per epoch for DynamicMPNN versus 3 min 34 s for SPINET on an NVIDIA H200. We additionally evaluated the pretrained checkpoint released by the DynamicMPNN authors. It achieved 31.6% recovery on our mdCATH test set, compared with 37.6% after retraining the architecture on our mdCATH training set. These comparisons establish stronger baselines and help us assess how the training dataset and number of input conformations affect recovery and runtime.
(5) SPINET retains a sequence-recovery advantage against stronger baselines. We trained PiFold, ScFold, and ProteinMPNN on multiple MD frames and averaged their logits across frames at inference. We also added multi-state ADFLIP. These stronger comparisons improved baseline recovery, but SPINET still achieved the highest top-1 recovery at 320 K, reaching 56.7%, compared with 53.3% for ADFLIP, 50.2% for PiFold, 48.5% for ScFold, and 43.7% for ProteinMPNN. We clarified the training subsampling intervals and runtimes in the manuscript.
(6) We extended the baseline comparisons to 450 K. At the highest temperature available in mdCATH, SPINET achieved 70.0% top-1 recovery, compared with 62.4% for PiFold, 58.4% for ProteinMPNN, and 34.6% for two-conformer DynamicMPNN. We believe the greater motion at higher temperatures may expose more residue-specific backbone geometries and dihedral preferences, making sequence recovery easier.