- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces the joint protein domain parsing and fold classification task, supported by TEDBench-Joint, a large-scale benchmark derived from TED (The Encyclopedia of Domains). Based on the introduced benchmark, the authors propose DoTR, an end-to-end SE(3)-invariant model that predicts an unordered set of residue-level domain masks and fold labels from protein 3D structures. DoTR outperforms existing domain-parsing methods and two-stage pipelines, demonstrating the value of learning both tasks jointly.
Critical strengths
-
The authors seek to combine domain parsing and protein fold classification into a joint prediction task and construct a corresponding benchmark. The manuscript clearly describes the data-processing pipeline in detail.
-
To accomplish the goal of joint protein structural prediction, the authors complement the benchmark with a proposed model, DoTR, and report preliminary in-distribution evaluation results that may serve as a reference for future research.
Critical weaknesses
-
From a research motivation perspective, I remain unconvinced that combining domain parsing and protein fold classification is biologically necessary. The two tasks assess predictive capabilities at different levels. Demonstrating that they can be performed jointly does not, by itself, establish a need to do so. As currently presented, the benchmark appears to serve primarily as an evaluation platform for future methodological development, rather than addressing a clearly articulated biological research need.
-
From a method technical perspective, DoTR’s prediction pipeline does not appear to offer a particularly distinctive algorithmic contribution. Moreover, the experimental evaluation is limited to in-distribution testing on the authors’ own dataset and includes only a small number of baseline models. As a result, the evidence is not yet sufficiently compelling to support the paper’s claims.
-
The Related Work section should provide a systematic review of prior research to help readers understand how the present study is situated within the literature and what it contributes, rather than focusing primarily on claims of novelty and contribution.
-
The Limitations section is an important part of a complete paper, as it summarizes the study’s limitations and outlines directions for future work. While I understand that space in the main text is limited, I would not recommend placing this section at the end of the appendix.
Questions for authors
I encourage the authors to use the rebuttal to clarify the biological rationale for a joint benchmark and to articulate the core technical innovations of the proposed DoTR model.
6 comments
2. Methodological contribution and generalization beyond TEDBench-Joint (Weakness 2)
Methodological contribution. We agree that DoTR builds on several established components, and we do not claim its novelty for these components individually. Our main contributions include a joint formulation and evaluation of the problem, accompanied by the first large-scale benchmark as well as the strong baseline models, which outperform existing methods on both parsing and unified metrics. We also note that our class-agnostic version of DoTR still differs from prior approaches by replacing the pairwise residue objective of Chainsaw [4] and Merizo [5], with an instance-based objective, where each query predicts one domain mask and its existence probability, trained with bipartite matching.
Generalization beyond TEDBench-Joint. We evaluated DoTR on the benchmark introduced in the Chainsaw paper [4], which contains CATH-annotated experimental PDB structures and training, validation, and test splits of 36,000, 1,000, and 1,363 chains, respectively. This provides an external evaluation against curated annotations on experimental structures. We report the main results below (Tables H1 and H2). Full results will be incorporated into the revision.
Results. We first evaluated DoTR-B-FT, its class-agnostic variant DoTR-B-FT-CA, and Chainsaw checkpoints trained only on TEDBench-Joint. All three performed below the strongest released parsers on this external dataset, particularly on the Boundary metric. We attribute the performance drops to a large difference in the ratio of non-domain residues (NDRs) between predicted and experimentally curated structures.
We then slightly fine-tuned these checkpoints on the Chainsaw training split for at most five epochs, selecting checkpoints and hyperparameters using the validation split. We compared these models with the released Chainsaw and Merizo checkpoints and with UniDoc, reproducing the reported baseline results [4]. After fine-tuning, DoTR-B-FT-CA achieved the highest point estimates across all four parsing metrics. The unified task transfers as well (Table H2). The end-to-end DoTR-B-FT reaches 70.8 mAP on experimental structures, and is 4.7 mAP and 7.5 PQ above the strongest external two-stage pipeline (Chainsaw-FT + MiAE-B).
Limited empirical evaluation. We expanded our empirical evaluation with results on the external experimentally curated dataset and additional comparisons of different versions of our models (see general response). Together with large inference time improvements with respect to the TED pipeline [3] (~33x in our setup; see Figure 3), these results show potential for annotating metagenomic-scale datasets [6].
Revision. We will incorporate these results and discussions into the revision. We will detail the exact changes in a future response.
Table H1: Domain parsing on an external experimentally curated dataset. DoTR-B-FT, DoTR-B-FT-CA, and Chainsaw-FT are AFDB-trained and fine-tuned on the Chainsaw training set. ± denotes 1.96× the bootstrap standard error, from 1000 resamples of the test chains drawn with replacement. Point estimates are the values measured on the test set. Chainsaw* and Merizo* are released checkpoints. Merizo was trained on CATH chains whose superfamilies overlap with this test set, so its result may be inflated by training-set leakage. Bold: best (ties both bolded).
| Model | Size | IoU | FracParsed | Boundary | NDO |
|---|---|---|---|---|---|
| DoTR-B-FT | 107.3M | 88.3 ± 1.1 | 80.9 ± 2.0 | 81.1 ± 1.2 | 89.3 ± 1.0 |
| DoTR-B-FT-CA | 107.0M | 89.8 ± 1.0 | 82.3 ± 2.0 | 82.7 ± 1.2 | 91.4 ± 0.8 |
| Chainsaw-FT | 0.6M | 87.4 ± 1.1 | 78.5 ± 2.0 | 79.3 ± 1.2 | 89.4 ± 0.9 |
| Chainsaw* | 0.6M | 87.8 ± 1.0 | 78.4 ± 2.1 | 78.8 ± 1.2 | 89.1 ± 0.9 |
| UniDoc | — | 84.7 ± 1.2 | 72.4 ± 2.3 | 80.6 ± 1.3 | 87.6 ± 0.9 |
| Merizo* | 51.8M | 85.6 ± 1.1 | 72.8 ± 2.3 | 74.8 ± 1.3 | 85.5 ± 1.0 |
Table H2: Unified metrics on an external experimentally curated dataset. mAP, AP₅₀, AP₇₅, and PQ on the 1,281 test chains whose domains all carry a CATH T-level label. MiAE-B is the classifier fine-tuned on the Chainsaw training split. GT + MiAE-B is the classifier applied to ground-truth segments. ± denotes 1.96× the bootstrap standard error, from 1000 resamples of the test chains drawn with replacement. Bold: best among the models (the upper bound excluded).
| Model | Size | mAP | AP₅₀ | AP₇₅ | PQ |
|---|---|---|---|---|---|
| DoTR-B-FT | 107.3M | 70.8 ± 2.9 | 77.9 ± 2.9 | 72.3 ± 3.1 | 68.8 ± 3.2 |
| DoTR-B-FT-CA + MiAE-B | 209.7M | 71.3 ± 2.8 | 78.0 ± 2.8 | 72.8 ± 2.9 | 65.0 ± 3.1 |
| Chainsaw-FT + MiAE-B | 103.2M | 66.1 ± 2.9 | 77.0 ± 2.8 | 66.3 ± 3.2 | 61.3 ± 3.0 |
| GT + MiAE-B | — | 88.0 ± 2.3 | 88.0 ± 2.3 | 88.0 ± 2.3 | 80.8 ± 3.0 |
3. Related Work (Weakness 3)
We would like to point out that, due to the space constraints, we provide a detailed review of related work in Appendix A, and we present our contribution within this context. We clarify that this organization was introduced to save space in the main paper.
Revision. We will move some of the content to the main paper if space permits in the revision. We will detail the exact changes in a future response.
4. Limitations (Weakness 4)
We agree with the reviewer that the limitations and directions for future work are important parts of the paper.
Revision. We will move some of the content to the main paper if space permits in the revision. We will detail the exact changes in a future response.
References
[1] Andreeva, Antonina, et al. "SCOP database in 2004: refinements integrate structure and sequence family data." Nucleic acids research 32.suppl_1 (2004): D226-D229.
[2] Redfern, Oliver C., et al. "CATHEDRAL: a fast and effective algorithm to predict folds and domain boundaries from multidomain protein structures." PLoS computational biology 3.11 (2007): e232.
[3] Lau, Andy M., et al. "Exploring structural diversity across the protein universe with The Encyclopedia of Domains." Science 386.6721 (2024): eadq4946.
[4] Wells, Jude, et al. "Chainsaw: protein domain segmentation with fully convolutional neural networks." Bioinformatics 40.5 (2024): btae296.
[5] Lau, Andy M., Shaun M. Kandathil, and David T. Jones. "Merizo: a rapid and accurate protein domain segmentation method using invariant point attention." Nature Communications 14.1 (2023): 8445.
[6] Yeo, Jingi, et al. "Metagenomic-scale analysis of the predicted protein structure universe." bioRxiv (2025): 2025-04.
1. Biological rationale for joint modeling of domain parsing and fold classification (Weakness 1)
Our biological objective is to identify which recurring structural units constitute a protein and where they occur. SCOP explicitly defines domains through their occurrence independently or in different multidomain contexts, and describes revising boundaries when such recurrence becomes apparent [1]. CATHEDRAL directly exploits fold recurrence to recognize domains and assign their boundaries [2]. Thus, localization and fold recognition are complementary aspects of an established annotation problem.
Both outputs also support biological discovery: TED combines domain boundaries and classifications to characterize domain architectures and uncover previously unobserved interactions between superfamilies [3]. In other words, their CATH annotations highly depend on the domain parsing results.
Therefore, we believe a joint modeling of parsing and classification highly benefits the development and evaluation of methods, leading to improved classification performance. Our experiments support this (see our general response "DoTR without class supervision: Refined interpretation of joint learning").
Revision. We will incorporate this discussion into the revision. We will detail the exact changes in a future response.