- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proses ProTAL, a framework for detecting temporally inconsistent local transitions in protein dynamics trajectories. The authors argue that individual protein structures can be physically plausible but their ordering may be inconsistent. The model is trained on MD trajectories with artificial perturbations and they show experinemts on benchmarks like ATLAS.
Critical strengths
The authors take a different perspective which is interesting of identifying potential errors in protein dynamics trajectories, and i think developing better ways to evaluate these dynamics is perhaps an important and underexplored direction. But, I think the authors should shift their focus on local inconsistencies to more global trajectory behavior (lmetstable state occupancy, conform pathways, etc.)
Critical weaknesses
The main weakness is lack of biological motivation. MD trajectories do not have one canonical ordering (they are stochastic...), it is unclear why detecting local inconsistencies is useful when MD analysis is usually about overall metastable state occupancy and mode hopping transition pathways. The paper does not make a convincing case for why flagging individual transitions would provide useful biological insight or help assess these broader properties of the dynamics. Also the model is trained and evaluated on these artificial trajectory corruptions and there is not clear generalizability evidence for meaningful errors in generated protein trajectories.
Questions for authors
Major concerns with biological motivation. I am not convinced by the motivation of this paper. Protein dynamics is inherently stochastic, there is no single correct temporal ordering of conformations. What matters biologically is whether trajectories capture the correct energy landscape: metastable states, their populations and transition pathways / kinetics, rather than the precise plausibility of local fluctuation transitions. It is much less clear why these local inconsistencies in the frames would be useful and the paper does not motivate this need biologically. For example, a traj could have a few unusual transistions while still accurately reproducing the relevant conformation populations. Or conversely, the trj could appear locally smooth while misreprsenting the rates of transitions or occupancy of states. Moreover the authors do not convincingly establish that the anomalies they detect actually correspond to errors in biological meaningful quantities. With this, it is hard to motivate acceptance for the paper regardless of the method as this problem does not seem properly justified.
Limited generalization beyond synthetic anomalies. My second major concern is that the experimental setup is mainly focused on recognizing artificial perturbations. The authors train the modal by swapping/replacing/splicing frames and then evaluate on the same corruptions. This concern is reinforced by the results on unseen time warping where the recall drops significantly (~3%) compared to the main synthetic benchmark. It seems as if the model is just learning to predict the artifical perturbations. There needs to be much stronger experimental evidence along with better biological motivation.