acceptodds
Under review as a conference paper at ICLR 2027

Unifying Protein Domain Parsing and Classification at Scale

Abstract

Protein domains are modular units of protein structure that play central roles in protein folding, function, and evolution. Annotating a protein structure requires answering two coupled questions: which residues form each domain, and what fold does each domain adopt? Existing methods largely address these questions separately, with domain parsing and fold classification performed by sequential stages. Such pipelines cannot optimize the two tasks jointly and may propagate parsing errors into subsequent classification. We introduce joint protein domain parsing and fold classification and construct TEDBench-Joint, a large-scale benchmark derived from The Encyclopedia of Domains (TED). We further propose DoTR, an end-to-end SE(3)-invariant model that encodes residue-level 3D structure and uses a transformer decoder to directly predict an unordered set of domains, each represented by a residue mask and a fold label. Experiments on TEDBench-Joint show that DoTR outperforms dedicated domain-parsing methods on fold-agnostic parsing metrics and two-stage parsing-and-classification pipelines on fold-aware metrics. These results demonstrate that domain parsing and fold classification can be effectively learned within a single model, providing a unified approach to protein domain annotation at scale.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
protein domain parsing, protein domain classification, benchmarking, fold classification

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ymgVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces the joint protein domain parsing and fold classification task, supported by TEDBench-Joint, a large-scale benchmark derived from TED (The Encyclopedia of Domains). Based on the introduced benchmark, the authors propose DoTR, an end-to-end SE(3)-invariant model that predicts an unordered set of residue-level domain masks and fold labels from protein 3D structures. DoTR outperforms existing domain-parsing methods and two-stage pipelines, demonstrating the value of learning both tasks jointly.

Critical strengths
  1. The authors seek to combine domain parsing and protein fold classification into a joint prediction task and construct a corresponding benchmark. The manuscript clearly describes the data-processing pipeline in detail.

  2. To accomplish the goal of joint protein structural prediction, the authors complement the benchmark with a proposed model, DoTR, and report preliminary in-distribution evaluation results that may serve as a reference for future research.

Critical weaknesses
  1. From a research motivation perspective, I remain unconvinced that combining domain parsing and protein fold classification is biologically necessary. The two tasks assess predictive capabilities at different levels. Demonstrating that they can be performed jointly does not, by itself, establish a need to do so. As currently presented, the benchmark appears to serve primarily as an evaluation platform for future methodological development, rather than addressing a clearly articulated biological research need.

  2. From a method technical perspective, DoTR’s prediction pipeline does not appear to offer a particularly distinctive algorithmic contribution. Moreover, the experimental evaluation is limited to in-distribution testing on the authors’ own dataset and includes only a small number of baseline models. As a result, the evidence is not yet sufficiently compelling to support the paper’s claims.

  3. The Related Work section should provide a systematic review of prior research to help readers understand how the present study is situated within the literature and what it contributes, rather than focusing primarily on claims of novelty and contribution.

  4. The Limitations section is an important part of a complete paper, as it summarizes the study’s limitations and outlines directions for future work. While I understand that space in the main text is limited, I would not recommend placing this section at the end of the appendix.

Questions for authors

I encourage the authors to use the rebuttal to clarify the biological rationale for a joint benchmark and to articulate the core technical innovations of the proposed DoTR model.

6 comments

1. Biological rationale for joint modeling of domain parsing and fold classification (Weakness 1)

Our biological objective is to identify which recurring structural units constitute a protein and where they occur. SCOP explicitly defines domains through their occurrence independently or in different multidomain contexts, and describes revising boundaries when such recurrence becomes apparent [1]. CATHEDRAL directly exploits fold recurrence to recognize domains and assign their boundaries [2]. Thus, localization and fold recognition are complementary aspects of an established annotation problem.

Both outputs also support biological discovery: TED combines domain boundaries and classifications to characterize domain architectures and uncover previously unobserved interactions between superfamilies [3]. In other words, their CATH annotations highly depend on the domain parsing results.

Therefore, we believe a joint modeling of parsing and classification highly benefits the development and evaluation of methods, leading to improved classification performance. Our experiments support this (see our general response "DoTR without class supervision: Refined interpretation of joint learning").

Revision. We will incorporate this discussion into the revision. We will detail the exact changes in a future response.

2. Methodological contribution and generalization beyond TEDBench-Joint (Weakness 2)

Methodological contribution. We agree that DoTR builds on several established components, and we do not claim its novelty for these components individually. Our main contributions include a joint formulation and evaluation of the problem, accompanied by the first large-scale benchmark as well as the strong baseline models, which outperform existing methods on both parsing and unified metrics. We also note that our class-agnostic version of DoTR still differs from prior approaches by replacing the pairwise residue objective of Chainsaw [4] and Merizo [5], with an instance-based objective, where each query predicts one domain mask and its existence probability, trained with bipartite matching.

Generalization beyond TEDBench-Joint. We evaluated DoTR on the benchmark introduced in the Chainsaw paper [4], which contains CATH-annotated experimental PDB structures and training, validation, and test splits of 36,000, 1,000, and 1,363 chains, respectively. This provides an external evaluation against curated annotations on experimental structures. We report the main results below (Tables H1 and H2). Full results will be incorporated into the revision.

Results. We first evaluated DoTR-B-FT, its class-agnostic variant DoTR-B-FT-CA, and Chainsaw checkpoints trained only on TEDBench-Joint. All three performed below the strongest released parsers on this external dataset, particularly on the Boundary metric. We attribute the performance drops to a large difference in the ratio of non-domain residues (NDRs) between predicted and experimentally curated structures.

We then slightly fine-tuned these checkpoints on the Chainsaw training split for at most five epochs, selecting checkpoints and hyperparameters using the validation split. We compared these models with the released Chainsaw and Merizo checkpoints and with UniDoc, reproducing the reported baseline results [4]. After fine-tuning, DoTR-B-FT-CA achieved the highest point estimates across all four parsing metrics. The unified task transfers as well (Table H2). The end-to-end DoTR-B-FT reaches 70.8 mAP on experimental structures, and is 4.7 mAP and 7.5 PQ above the strongest external two-stage pipeline (Chainsaw-FT + MiAE-B).

Limited empirical evaluation. We expanded our empirical evaluation with results on the external experimentally curated dataset and additional comparisons of different versions of our models (see general response). Together with large inference time improvements with respect to the TED pipeline [3] (~33x in our setup; see Figure 3), these results show potential for annotating metagenomic-scale datasets [6].

Revision. We will incorporate these results and discussions into the revision. We will detail the exact changes in a future response.

Table H1: Domain parsing on an external experimentally curated dataset. DoTR-B-FT, DoTR-B-FT-CA, and Chainsaw-FT are AFDB-trained and fine-tuned on the Chainsaw training set. ± denotes 1.96× the bootstrap standard error, from 1000 resamples of the test chains drawn with replacement. Point estimates are the values measured on the test set. Chainsaw* and Merizo* are released checkpoints. Merizo was trained on CATH chains whose superfamilies overlap with this test set, so its result may be inflated by training-set leakage. Bold: best (ties both bolded).

ModelSizeIoUFracParsedBoundaryNDO
DoTR-B-FT107.3M88.3 ± 1.180.9 ± 2.081.1 ± 1.289.3 ± 1.0
DoTR-B-FT-CA107.0M89.8 ± 1.082.3 ± 2.082.7 ± 1.291.4 ± 0.8
Chainsaw-FT0.6M87.4 ± 1.178.5 ± 2.079.3 ± 1.289.4 ± 0.9
Chainsaw*0.6M87.8 ± 1.078.4 ± 2.178.8 ± 1.289.1 ± 0.9
UniDoc—84.7 ± 1.272.4 ± 2.380.6 ± 1.387.6 ± 0.9
Merizo*51.8M85.6 ± 1.172.8 ± 2.374.8 ± 1.385.5 ± 1.0

Table H2: Unified metrics on an external experimentally curated dataset. mAP, AP₅₀, AP₇₅, and PQ on the 1,281 test chains whose domains all carry a CATH T-level label. MiAE-B is the classifier fine-tuned on the Chainsaw training split. GT + MiAE-B is the classifier applied to ground-truth segments. ± denotes 1.96× the bootstrap standard error, from 1000 resamples of the test chains drawn with replacement. Bold: best among the models (the upper bound excluded).

ModelSizemAPAP₅₀AP₇₅PQ
DoTR-B-FT107.3M70.8 ± 2.977.9 ± 2.972.3 ± 3.168.8 ± 3.2
DoTR-B-FT-CA + MiAE-B209.7M71.3 ± 2.878.0 ± 2.872.8 ± 2.965.0 ± 3.1
Chainsaw-FT + MiAE-B103.2M66.1 ± 2.977.0 ± 2.866.3 ± 3.261.3 ± 3.0
GT + MiAE-B—88.0 ± 2.388.0 ± 2.388.0 ± 2.380.8 ± 3.0

3. Related Work (Weakness 3)

We would like to point out that, due to the space constraints, we provide a detailed review of related work in Appendix A, and we present our contribution within this context. We clarify that this organization was introduced to save space in the main paper.

Revision. We will move some of the content to the main paper if space permits in the revision. We will detail the exact changes in a future response.

4. Limitations (Weakness 4)

We agree with the reviewer that the limitations and directions for future work are important parts of the paper.

Revision. We will move some of the content to the main paper if space permits in the revision. We will detail the exact changes in a future response.

References

[1] Andreeva, Antonina, et al. "SCOP database in 2004: refinements integrate structure and sequence family data." Nucleic acids research 32.suppl_1 (2004): D226-D229.

[2] Redfern, Oliver C., et al. "CATHEDRAL: a fast and effective algorithm to predict folds and domain boundaries from multidomain protein structures." PLoS computational biology 3.11 (2007): e232.

[3] Lau, Andy M., et al. "Exploring structural diversity across the protein universe with The Encyclopedia of Domains." Science 386.6721 (2024): eadq4946.

[4] Wells, Jude, et al. "Chainsaw: protein domain segmentation with fully convolutional neural networks." Bioinformatics 40.5 (2024): btae296.

[5] Lau, Andy M., Shaun M. Kandathil, and David T. Jones. "Merizo: a rapid and accurate protein domain segmentation method using invariant point attention." Nature Communications 14.1 (2023): 8445.

[6] Yeo, Jingi, et al. "Metagenomic-scale analysis of the predicted protein structure universe." bioRxiv (2025): 2025-04.

Reviewer_FTDDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TEDBench-Joint, a large-scale benchmark for jointly evaluating protein domain parsing and fold classification, alongside DoTR (Domain Transformer), an end-to-end model designed to solve both tasks simultaneously. Unlike conventional approaches that first identify domain boundaries and subsequently assign structural classes, DoTR formulates domain annotation as a structured set prediction problem. It employs an SE(3)-invariant geometric encoder to extract residue-level structural representations and a two-way Transformer decoder with learnable queries to predict an unordered set of domain masks and corresponding CATH fold labels. Hungarian bipartite matching enables permutation-invariant training, and the model can represent discontinuous domains through arbitrary residue masks.

Critical strengths

The most compelling contribution is the introduction of a unified benchmark and task formulation for protein domain annotation, addressing an important gap between existing domain parsing and fold classification research. TEDBench-Joint provides substantial scale, structural-cluster-based train/test separation, multi-label stratification, and carefully designed sampling procedures that preserve the representation of multi-domain proteins. These choices make the benchmark a potentially valuable resource beyond the proposed model itself.

Critical weaknesses

Despite the strong empirical results, the benchmark's dependence on automatically generated annotations raises concerns about annotation reliability and evaluation circularity. TED domain boundaries originate from a consensus of existing parsing methods, and the derived labels depend on existing structural classification procedures. Consequently, DoTR is primarily evaluated on its ability to reproduce these annotations rather than independently validated biological domain definitions. This issue is particularly important because approximately 18.7% of residues receive uncertain or void assignments, while the training procedure treats them as non-domain residues. The filtering procedure further restricts the benchmark to high-confidence, globular structures with sufficiently many secondary-structure elements, potentially limiting generalization to small domains, disordered regions, and lower-confidence predicted structures. Methodologically, the architecture largely adapts established Transformer-based set prediction and segmentation techniques, so its algorithmic novelty is more limited than its task formulation and benchmark contribution.

Questions for authors
  1. Ground-truth annotation reliability: Since TEDBench-Joint relies on automatically generated TED annotations, could the authors evaluate DoTR on an independent, experimentally curated domain annotation dataset? How sensitive are the results to disagreement among the domain parsing methods used to construct TED?

  2. Effect of void residues: Approximately 18.7% of residues are marked as void, and these residues are treated as non-domain residues during training. Could the authors investigate explicitly masking uncertain residues during training or using uncertainty-aware supervision?

  3. Isolating the benefit of joint optimization: Could the authors compare DoTR with a matched-capacity architecture that shares the same geometric encoder but trains parsing and fold classification as separate tasks? This would help distinguish the benefits of joint learning from those of architectural differences.

  4. Generalization beyond curated structures: How does DoTR perform on low-pLDDT structures, small domains, intrinsically disordered regions, and proteins excluded by the benchmark filtering criteria?

  5. Unseen structural classes: The current formulation is restricted to known CATH classes. Could the authors evaluate generalization to previously unseen fold classes or investigate an open-set formulation that combines domain discovery with fold recognition?

  6. Discontinuous domains and complex proteins: The stratified analysis identifies a substantial performance gap for discontinuous domains. What causes this failure mode, and could explicit long-range structural interactions, alternative query mechanisms, or domain-connectivity constraints improve performance?

  7. Accuracy versus computational cost: Could the authors provide detailed training-resource requirements and evaluate how inference speed and accuracy scale with protein length and domain count, particularly for large multi-domain proteins beyond the typical training distribution?

6 comments

1. External dataset of experimental structures (Question 1)

To address the concern of annotation reliability, we evaluated DoTR on the benchmark introduced in the Chainsaw paper [2], which contains CATH-annotated experimental PDB structures and training, validation, and test splits of 36,000, 1,000, and 1,363 chains, respectively. This provides an external evaluation against curated annotations on experimental structures. We report the main results below (Tables C1 and C2).

Direct transfer. We first evaluated DoTR-B-FT, its class-agnostic variant DoTR-B-FT-CA, and Chainsaw checkpoints trained only on TEDBench-Joint. All three performed below the strongest released parsers on this external dataset, particularly on the Boundary metric. These results reveal a limitation in direct transfer from TEDBench-Joint to curated experimental structures. We attribute the performance drops to a large difference in the ratio of non-domain residues (NDRs) between predicted and experimentally curated structures. Unlike the global fold classification of TEDBench [1], which showed positive direct transfer, our task is dense and therefore more sensitive to distribution shifts in boundary definitions.

Fine-tuning. We then slightly fine-tuned these checkpoints on the Chainsaw training split for at most five epochs, selecting checkpoints and hyperparameters on the validation split. Each model retained its own training objective. For DoTR, we tuned the learning rate, weight decay, layer-wise learning-rate decay, and training sampler. For Chainsaw, we tuned the applicable settings: learning rate, weight decay, and sampler.

For DoTR-B-FT, we additionally removed the class-score term from the matching cost and disabled the class-confidence threshold at inference to mitigate the training and checkpoint selection instabilities. Unlike TEDBench-Joint, the Chainsaw dataset splits are not balanced for T-level classes. Cross-entropy class supervision is preserved.

Results. We compared these models with the released Chainsaw and Merizo checkpoints and with UniDoc, reproducing the reported baseline results [2]. After fine-tuning, DoTR-B-FT-CA achieved the highest point estimates across all four parsing metrics. Relative to the released Chainsaw checkpoint, it improved IoU by 2.0 points, FracParsed by 3.9 points, Boundary by 3.9 points, and NDO by 2.3 points.

The unified task transfers as well. Table C2 reports the unified metrics on the 1,281 test chains whose domains all carry a CATH T-level label, using the MiAE-B classifier fine-tuned on the Chainsaw training split for every two-stage pipeline. We compare against the overall strongest external two-stage pipeline on these metrics: Chainsaw-FT + MiAE-B. The end-to-end DoTR-B-FT reaches 70.8 mAP on experimental structures, and is 4.7 mAP and 7.5 PQ above the strongest external two-stage pipeline.

Revision. We will incorporate these results and discussions into the revision. We will detail the exact changes in a future response.

Table C1: Domain parsing on an external experimentally curated dataset. DoTR-B-FT, DoTR-B-FT-CA, and Chainsaw-FT are AFDB-trained and fine-tuned on the Chainsaw training set. ± denotes 1.96× the bootstrap standard error, from 1000 resamples of the test chains drawn with replacement. Point estimates are the values measured on the test set. Chainsaw* and Merizo* are released checkpoints. Merizo was trained on CATH chains whose superfamilies overlap with this test set, so its result may be inflated by training-set leakage. Bold: best (ties both bolded).

ModelSizeIoUFracParsedBoundaryNDO
DoTR-B-FT107.3M88.3 ± 1.180.9 ± 2.081.1 ± 1.289.3 ± 1.0
DoTR-B-FT-CA107.0M89.8 ± 1.082.3 ± 2.082.7 ± 1.291.4 ± 0.8
Chainsaw-FT0.6M87.4 ± 1.178.5 ± 2.079.3 ± 1.289.4 ± 0.9
Chainsaw*0.6M87.8 ± 1.078.4 ± 2.178.8 ± 1.289.1 ± 0.9
UniDoc—84.7 ± 1.272.4 ± 2.380.6 ± 1.387.6 ± 0.9
Merizo*51.8M85.6 ± 1.172.8 ± 2.374.8 ± 1.385.5 ± 1.0

Table C2: Unified metrics on an external experimentally curated dataset. mAP, AP₅₀, AP₇₅, and PQ on the 1,281 test chains whose domains all carry a CATH T-level label. MiAE-B is the classifier fine-tuned on the Chainsaw training split. GT + MiAE-B is the classifier applied to ground-truth segments. ± denotes 1.96× the bootstrap standard error, from 1000 resamples of the test chains drawn with replacement. Bold: best among the models (upper bound excluded).

ModelSizemAPAP₅₀AP₇₅PQ
DoTR-B-FT107.3M70.8 ± 2.977.9 ± 2.972.3 ± 3.168.8 ± 3.2
DoTR-B-FT-CA + MiAE-B209.7M71.3 ± 2.878.0 ± 2.872.8 ± 2.965.0 ± 3.1
Chainsaw-FT + MiAE-B103.2M66.1 ± 2.977.0 ± 2.866.3 ± 3.261.3 ± 3.0
GT + MiAE-B—88.0 ± 2.388.0 ± 2.388.0 ± 2.380.8 ± 3.0

2. Effect of void residues (Question 2)

We agree that uncertain annotations should be distinguished from confirmed non-domain residues (NDRs). We already provide this comparison for the inference evaluation settings (Section E.1 in the Appendix). To assess the effect of this choice during training, we retrained both the strengthened DoTR-B-FT and DoTR-B-FT-CA with void residues excluded from the residue-level matching costs and training losses. We evaluated both training conventions under both scoring protocols: excluding void residues from evaluation or scoring them as NDRs. Tables D1 and D2 include DoTR-B-FT results, and the full results will be included in the revision.

Results. When void residues are excluded from scoring, masked training improves all reported point estimates, including +3.4 Boundary points and +2.3 mAP points (Table D1). When void residues are instead scored as NDRs, masked training reduces IoU, FracParsed, NDO, mAP, AP₇₅, and PQ, while Boundary improves by 1.0 point and AP₅₀ remains nearly unchanged (Table D2). Importantly, the relative ordering of the end-to-end and two-stage approaches remains unchanged, supporting the robustness of our results.

Revision. We will incorporate these results and discussions into the revision. We will detail the exact changes in a future response.

Notes on Tables D1 and D2. Standard trains void residues as NDRs, as in the submission, and masked excludes them from the training losses. The models use the strengthened configuration. IoU, FracParsed, Boundary and NDO are parsing metrics; mAP, AP₅₀, AP₇₅ and PQ are unified metrics. ± denotes 1.96× the bootstrap standard error, from 1000 resamples of the 96,864 test chains drawn with replacement. Point estimates are the values measured on the test set. Bold: best (ties both bolded).

Table D1: Standard vs. masked void training. Void residues excluded from scoring.

ModelTrainingIoUFracParsedBoundaryNDOmAPAP₅₀AP₇₅PQ
DoTR-B-FTstandard95.0 ± 0.193.8 ± 0.186.3 ± 0.194.6 ± 0.175.0 ± 0.878.8 ± 0.877.5 ± 0.878.0 ± 0.8
DoTR-B-FTmasked95.8 ± 0.195.5 ± 0.189.7 ± 0.196.4 ± 0.177.3 ± 0.881.2 ± 0.879.7 ± 0.878.4 ± 0.8

Table D2: Standard vs. masked void training. Void residues scored as NDRs rather than ignored.

ModelTrainingIoUFracParsedBoundaryNDOmAPAP₅₀AP₇₅PQ
DoTR-B-FTstandard92.9 ± 0.189.2 ± 0.282.7 ± 0.184.7 ± 0.271.9 ± 0.877.0 ± 0.874.9 ± 0.873.5 ± 0.8
DoTR-B-FTmasked91.6 ± 0.185.8 ± 0.283.7 ± 0.182.7 ± 0.268.7 ± 0.877.1 ± 0.871.1 ± 0.971.8 ± 0.8

3. Isolating the benefits of joint optimization (Question 3)

Please find the answer to the joint optimization question in our general response "DoTR without class supervision: Refined interpretation of joint learning."

4. Methodological novelty (Weaknesses)

Regarding methodological novelty, our contribution is the formulation and implementation of joint domain parsing and fold classification as end-to-end set prediction, together with TEDBench-Joint and its evaluation framework. We believe such a unified framework would benefit the community of protein structure and function annotations.

5. Generalization to low-confidence structures and discontinuous domains (Questions 4 and 6)

Results. Table E1 below stratifies the test accuracy of the end-to-end DoTR-B-FT by mean pLDDT, domain size, and domain continuity. Structure quality mainly affects the unified metrics, as mAP rises from 72.4 in the lowest pLDDT group to 81.2 in the highest and PQ from 74.8 to 81.0. The parsing metrics move less, with IoU nearly unchanged and Boundary improving from 84.0 to 88.6. Small domains show no disadvantage, as the three size groups perform comparably on both metrics.

Intrinsically disordered regions carry no domain labels in TED and are therefore outside the scope of the benchmark. Incorporating proteins removed by the filtering criteria is possible, but it requires making substantial choices for data curation and interpretation, which we deem out of scope for this work. A small subset of such proteins is used in an answer to the reviewer's Question 5 below, but it concerns a held-out set setting. We note that domain annotation for low-confidence predicted structures is a shared limitation for all structured based methods, including the TED pipeline [4].

For discontinuous domains, Table E1 shows the gap reported in the paper, with mAP dropping by 18.5 points. A likely contributing factor is their scarce presence in the data, as only about 13% of ground-truth domains are discontinuous, which limits the training signal for this prediction mode. The suggested directions of explicit long-range interactions, alternative query mechanisms, and connectivity constraints could all be explored, but they require substantial design choices and retraining of the model, which is not feasible within the rebuttal period.

Revision. We will incorporate these results and discussions into the revision. We will detail the exact changes in a future response.

Table E1: Stratification of the end-to-end DoTR-B-FT by mean pLDDT (left), domain size in residues per ground-truth domain (middle), and domain continuity (right). Results are on the test split under the submission's configuration, with void residues excluded from scoring. IoU is length-weighted per chain for the pLDDT axis and averaged per domain for the domain size and continuity axes. mAP restricts scoring to the ground-truth domains of the stratum. Boundary is averaged per chain and PQ is computed over the chains of the stratum, so both are reported only for the pLDDT axis. n counts chains for the pLDDT axis and ground-truth domains otherwise.

Mean pLDDTnIoUBoundarymAPPQ│Domain sizenIoUmAP│ContinuitynIoUmAP
[80, 90)56,52494.784.072.474.8│<10048,60596.469.4│continuous151,19797.074.9
[90, 95)28,46994.986.376.277.4│100–20095,42796.371.9│discontinuous23,41191.956.4
[95, 100]11,87195.688.681.281.0│200+30,57696.069.2│
Reviewer_bYgwOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors present an extension of the TEDBench benchmark for classifying protein domains, TEBench-Joint, which focuses on the more relevant problem of both identifying residues within domains and then also classifying their domain identity, particularly within multi-domain proteins. TEDBench-Joint is similarly a large-scale benchmark constructed from The Encyclopedia of Domains (TED) and contains ~1M proteins with 1.75M domain annotations. The authors also develop the Domain Transformer (DoTR) method for unified parsing and classification of protein domains, as opposed to two-step workflows consisting of an unsupervised parsing/segmentation method followed by matching parsed domains to labels. The authors show that DoTR models show moderate scaling properties, generally outperform existing unsupervised parsing methods, and also have significantly faster run times.

Critical strengths
  1. Work is timely and well-motivated: improved structure prediction methods combined with large-scale sequencing studies (e.g. metagenomics) have resulted in extremely large protein structure databases. The ability to identify and extract functional domains within these databases currently lacks a standardized benchmark for assessing new approaches.
  2. Natural extension of TEDBench to the harder task of domain segmentation within proteins rather than just identifying a single dominant domain within a protein. TEDBench-Joint is likely to be a useful benchmark for the community
  3. DoTR method is empirically shown to perform well in their setting. Performance stratifications by various protein properties help understand where the model performs better or worse.
Critical weaknesses
  1. As authors state in limitations, while the benchmark could feasibly be modified to benchmark approaches to identify novel domains, the proposed DoTR approach is incapable of this. While a faster, more accurate pipeline for supervised segmentation and labeling of protein domains is impactful, the ability to identify novel domains is what will really drive discovery in increasingly large protein structure databases.
  2. The TEDBench-Joint benchmark is one of the two main contributions of the paper, and a key part of benchmarks is the evaluation criteria. Evaluation criteria are never actually described in the main text.
  3. Similarly to above, a key aspect of benchmark impact is their usability. Some description of how the benchmark supports use (e.g. how datasets are parsed and stored, associated evaluation framework, etc) would be helpful. Particularly for this case, an anonymized GitHub repo would also help with assessment.
Questions for authors

Questions affecting assessment

  1. The benchmark methods state that domains are retained if they have a CATH label at the Topology or Homologous Superfamily level (lines 130-131), but later state that 1,210 T-groups are aggregated to 1,055 labels (lines 144-145). Does the final label set contain both T-level and H-level CATH labels or only T-level? If only T-level, what's the rationale for not using the finer-grained labels?
  2. It seems like the fact that DoTR training is supervised by explicit domain labels is both a strength and a weakness. A weakness because it means the model is unable to perform parsing of arbitrary domains similar to Chainsaw and Merizo, but a strength because the domain label supervision may actually help with the parsing step for known domains. A couple questions related to this:
    • Have the authors considered training a version of DoTR without class supervision? I believe this would start to look more similar to Merizo's approach, but with the various architectural and dataset improvements the authors have made. This would help us understand how much of the parsing improvement comes from domain-level supervision vs dataset and architecture improvements.
    • Have the authors considered evaluating the current DoTR models on a heldout set of CATH T- or H-level labels? Here pairing could be based on maximizing IoU instead of predicted class label. This would help us understand how well a DoTR model would perform at identifying novel domains

Summary of review

I believe the work's two main contributions (TEDBench-Joint and DoTR) would both be of interest to the ICLR community. TEDBench-Joint covers a gap in TEDBench (that I'm honestly somewhat surprised wasn't raised as an issue previously): that substructural classification is not just a protein-based label but also a segmentation task. DoTR seems to bring both improvements to domain segmentation methods (which have seen relatively little attention from the community) and also general runtime improvements that will be increasingly important as protein structure databases continue to grow. I think the main gap in the work is any treatment or consideration of how DoTR fits into the more important use case of domain segmentation in absence of supervised labels. I think a weak accept reflects these contributions and main gap.

4 comments

1. Identification and segmentation of domains from unseen CATH labels (Weakness 1 and Question 2.2)

Our paper focuses on fixed-set segmentation and classification, which we find increasingly relevant with recent results indicating that unexplored large-scale databases contain many novel domain combinations, and only a few novel folds [1]. However, we agree that identifying domains of unseen folds is still an important application, and it presents an avenue for future work as we mentioned in the limitations. We therefore evaluated our strongest DoTR models on domains whose CATH T-level labels never occur in training. Predictions are paired with ground-truth domains by IoU only, as the reviewer suggests.

Setup. The held-out set contains 63 chains, each with one domain from one of 26 T-groups absent from training. These include 57 chains from Foldseek clusters outside TEDBench-Joint, which were excluded due to the chain-level pLDDT below 80. The remaining 6 are test-split domains whose T-group does not occur in the training set, although it belongs to a merged A.x label that does. As in the test split, we keep one chain per cluster and label multiset. A held-out domain counts as detected if a predicted domain is paired with it at IoU ≥ 0.5. We also report the IoU of the held-out domain, the fraction of held-out domains with IoU ≥ 0.8 (FracParsed), and the chain-level Boundary and NDO scores (they do not have domain-level definitions). In 51 of the 63 chains, the held-out domain is the only domain.

Decoding for unseen folds. By default, DoTR-B-FT keeps a predicted domain only if a single known class is more probable than the no-object class. For an unseen fold, the probability is spread over several known classes, so a well-segmented domain can be discarded. For discovery, we instead keep a predicted domain if the total probability of all known classes exceeds the no-object probability, and we set the class-confidence threshold to zero. This rule asks whether a domain is present rather than which known class it belongs to. It adds no parameters and requires no retraining.

Results. DoTR-B-FT-CA detects 62 of the 63 held-out domains, with an IoU of 90.6 when void residues are excluded from scoring (Table B1). DoTR-B-FT instead detects 52 of them. The same ordering holds when void residues are scored as NDRs (Table B2). DoTR can therefore identify and segment domains of folds it was never trained on, and the class-agnostic variant is the better choice for discovery. In the closed-set setting, the class-aware decision to reject segments that match no known class may prevent spurious segmentations, while here it can restrict the discovery of new domains. DoTR-B-FT was not designed for this task, so these results do not imply that it cannot be adapted to it while retaining its closed-set benefits.

Revisions and implications. We will incorporate these results and discussions into the revision. We will detail the exact changes in a future response.

Table B1: Parsing of held-out folds. Void residues excluded from scoring. 63 chains with one domain from each of 26 T-groups absent from training. Detected, IoU, and FracParsed refer to the held-out domain. Boundary and NDO are computed per chain.

ModelDetectedIoUFracParsedBoundaryNDO
DoTR-B-FT82.575.169.870.377.9
DoTR-B-FT-CA98.490.688.983.791.2

Table B2: Parsing of held-out folds. Void residues scored as NDRs rather than ignored.

ModelDetectedIoUFracParsedBoundaryNDO
DoTR-B-FT81.072.565.171.373.2
DoTR-B-FT-CA98.486.177.880.281.9

2. Details of the evaluation setting and benchmark's data processing (Weakness 2 and Weakness 3)

We agree that the benchmark’s evaluation criteria and data processing details are important, and we kindly note that evaluation details are already provided in Appendix C.

Revision. We will incorporate detailed data preparation instructions to explain how TEDBench-Joint is constructed from publicly available source files. We will detail the exact changes in a future response. To facilitate utility, we will release data pre-processing scripts, processed datasets, and all model checkpoints upon publication.

3. Clarification and justification for using CATH T-level labels (Question 1)

Clarification. The final classification targets are exclusively CATH Topology (T-level) labels, as specified in Section 4.1. We agree that the wording in lines 130–131 is ambiguous. TED assigns H-level annotations when supported and T-level annotations when classification is confident only at that level. Because CATH is hierarchical, each H-level annotation also determines a corresponding T-level label. We therefore retain domains annotated at either level and use their T-level labels as classification targets.

The subsequent aggregation from 1,210 T-groups to 1,055 prediction labels is a separate processing step. Following TEDBench [2], T-groups supported by fewer than 10 Foldseek clusters are merged within their CATH architecture C.A into an aggregated label C.A.x, which represents the other topologies within that architecture. This yields 1,033 individual T-groups and 22 aggregated labels, 1,055 classes in total.

Justification. We follow prior work [2] in focusing on T-level classification. Extending the benchmark to H-level classification would require revisiting the data splits and the information available to the model.

Revision. We will incorporate these clarifications into the revision. We will detail the exact changes in a future response.

4. The effect of class supervision on DoTR (Question 2.1)

Please find the answer to this question in our general response "DoTR without class supervision: Refined interpretation of joint learning."

References

[1] Yeo, Jingi, et al. "Metagenomic-scale analysis of the predicted protein structure universe." bioRxiv (2025): 2025-04.

[2] Chen, Dexiong, et al. "Protein Fold Classification at Scale: Benchmarking and Pretraining." Proceedings of the 43rd International Conference on Machine Learning (ICML) 306 (2026): 16125-16143.

Thanks for the detailed response. The new analyses and explanations have been helpful for me as follows:

  • The AFESM paper (citation [1]) found 12 novel domain folds in 820 million proteins, so I agree with the author's assessment here that while identifying novel folds is important, it is likely of fairly narrow applicability compared to parsing and classification of previously observed categorizations. Thanks for this reference.
  • Performance on held-out CATH T-level labels seems broadly comparable to performance on classes seen at training time, which supports the applicability of DoTR to the unseen labels case. While not necessary for the rebuttal period, a comparison to other methods in this particular setup (same set of held-out CATH T-level labels) would be helpful for contextualizing this performance to previously used methods like Chainsaw and Merizo
  • The results from the class agnostic version of DoTR are helpful and make it clear that class supervision is not the driver of parsing performance, rather the improvements are more likely to come from model architecture and training data.
  • Clarification of label selection is helpful.
  • Adding descriptions of evaluation metrics to the main text is helpful for clarity. While descriptions were previously available in the Appendix, I note that by ICLR guidelines, reviewers are not actually required to read the appendices and thus the main text should convey all necessary details to understand the results.

The author's responses to my questions have resolved my concerns, so I've raised my score. Good work!

Dear Reviewer bYgw,

Thank you again for your time and a thoughtful review! We believe your feedback has strengthened our paper, and we appreciate your recognition of the potential value of our work.

We are glad that our response has been helpful for your assessment. Your additional suggestions for after the rebuttal period will be taken into account, especially the comparison to existing parsing methods in the held-out setting (point 2 from the response).

Comments on the paper

3 comments

We thank the reviewers for their constructive feedback and for recognizing TEDBench-Joint’s potential value to the community (bYgw, FTDD), its detailed data-processing pipeline (ymgV), and its careful dataset design (FTDD). We also appreciate the positive assessments of DoTR’s empirical performance (bYgw, FTDD), inference efficiency (bYgw), and informative stratified analyses (bYgw), and its utility as a reference for future research (ymgV).

Below, we address the concerns shared by reviewers. We will upload a revised version of the paper with changes marked in green. We will detail the exact changes in a future response.

1. DoTR without class supervision: Refined interpretation of joint learning

We agree that the submission does not sufficiently isolate the contribution of class supervision. We have therefore expanded the comparison.

Experimental setup. We trained DoTR-B-FT-CA, which uses the same architecture as DoTR-B-FT but replaces the multiclass classification head with a single domain-presence class, following [1]. We evaluate this model as a standalone parser and as the first stage of a two-stage pipeline using MiAE-B + seq for classification. We additionally evaluate DoTR-B-FT followed by the same classifier to distinguish the effect of parser training from that of joint versus separate classification.

We report these comparisons under two configurations:

  • Submission configuration (Tables A1 and A2).
  • Strengthened configuration (Tables A3 and A4), which applies auxiliary deep supervision following Mask2Former [2] and mask-wise merging inference [3] identically to both DoTR variants, without adding trainable parameters.

For each configuration, we evaluate both void-handling protocols: excluding void residues from scoring or treating them as non-domain residues. The quantitative comparisons with tables and references are provided in the subsequent general response.

Results. Removing class supervision leaves parsing performance broadly similar, with small improvements in several metrics. Thus, the results do not support attributing DoTR’s parsing gains to class supervision. Instead, they show that strong parsing performance is retained by the class-agnostic model. Further ablations would be needed to isolate the individual contributions of the architecture, set-prediction objective, and training data.

For the unified task, the end-to-end model achieves higher point estimates across the AP metrics in all four settings. For example, in the strengthened configuration with void residues excluded, its mAP is 75.0, compared with 73.5 for the class-agnostic parser followed by MiAE-B + seq (Table A3). PQ is close across methods, and its ordering depends on the configuration and void-handling protocol.

This difference between AP and PQ is consistent with a benefit in proposal scoring and ranking: AP depends on confidence-ranked predictions, whereas PQ evaluates the final partition without using confidence rankings. We also verified that the two-stage AP deficit is not an artifact of how the confidence scores from two stages are fused by showing that, in the strengthened configuration, sweeping both the combination of confidence terms and their relative weighting changes mAP by less than 0.05 points.

This distinguishes two findings more clearly:

  1. End-to-end DoTR outperforms two-stage pipelines built from existing parsers.
  2. A two-stage pipeline using our own class-agnostic parser is also competitive, with small parsing gains in some settings and lower AP than the end-to-end model.

These findings also refine our interpretation of joint learning: class supervision is not necessary for strong parsing, but joint prediction provides the best classification in terms of AP. The end-to-end model also uses approximately half as many parameters as our two-stage alternatives and produces parsing and classification outputs in one forward pass. Under our reported timing setup, it is approximately 33× faster than the TED pipeline [4], making it possible to annotate metagenomics-scale datasets [5].

Revisions and implications. We will incorporate these results and discussions into the revision. We will detail the exact changes in a future response.

2. Quantitative comparisons

All tables compare end-to-end DoTR with our two-stage alternatives on TEDBench-Joint. IoU, FracParsed, Boundary, and NDO are parsing metrics; mAP, AP₅₀, AP₇₅, and PQ are unified metrics. Values following ± are 1.96 × the bootstrap standard error, estimated from 1,000 bootstrap resamples of the 96,864 test chains. Point estimates are computed on the original test set.

DoTR-B-FT-CA is trained with a single domain-presence class. MiAE-B denotes MiAE-B + seq throughout the tables. Two-stage methods inherit their parsing metrics from their first-stage parser. Bold indicates the best point estimate, including ties at the displayed precision.

Revision. We will incorporate these results into the revision. We will detail the exact changes in a future response.

Table A1: Submission models' configuration. Void residues excluded from scoring.

ModelSizeIoUFracParsedBoundaryNDOmAPAP₅₀AP₇₅PQ
DoTR-B-FT107.3M94.9 ± 0.194.2 ± 0.185.2 ± 0.195.1 ± 0.173.0 ± 0.876.9 ± 0.875.5 ± 0.877.1 ± 0.8
DoTR-B-FT-CA + MiAE-B209.7M95.0 ± 0.194.3 ± 0.185.6 ± 0.195.3 ± 0.172.5 ± 0.876.4 ± 0.874.9 ± 0.877.9 ± 0.8
DoTR-B-FT + MiAE-B209.9M94.9 ± 0.194.2 ± 0.185.2 ± 0.195.1 ± 0.169.6 ± 0.873.3 ± 0.871.9 ± 0.877.9 ± 0.8

Table A2: Submission models' configuration. Void residues scored as non-domain residues (NDRs) rather than ignored.

ModelSizeIoUFracParsedBoundaryNDOmAPAP₅₀AP₇₅PQ
DoTR-B-FT107.3M91.7 ± 0.186.9 ± 0.279.8 ± 0.284.4 ± 0.270.0 ± 0.875.1 ± 0.873.1 ± 0.872.1 ± 0.8
DoTR-B-FT-CA + MiAE-B209.7M91.8 ± 0.186.7 ± 0.279.9 ± 0.283.4 ± 0.269.3 ± 0.874.6 ± 0.872.3 ± 0.971.9 ± 0.8
DoTR-B-FT + MiAE-B209.9M91.7 ± 0.186.9 ± 0.279.8 ± 0.284.4 ± 0.266.7 ± 0.871.6 ± 0.869.6 ± 0.873.0 ± 0.8

Table A3: Strengthened configuration (auxiliary deep supervision and mask-wise merging, applied identically to both models). Void residues excluded from scoring.

ModelSizeIoUFracParsedBoundaryNDOmAPAP₅₀AP₇₅PQ
DoTR-B-FT107.3M95.0 ± 0.193.8 ± 0.186.3 ± 0.194.6 ± 0.175.0 ± 0.878.8 ± 0.877.5 ± 0.878.0 ± 0.8
DoTR-B-FT-CA + MiAE-B209.7M95.2 ± 0.194.1 ± 0.186.8 ± 0.195.1 ± 0.173.5 ± 0.877.1 ± 0.875.8 ± 0.878.6 ± 0.8
DoTR-B-FT + MiAE-B209.9M95.0 ± 0.193.8 ± 0.186.3 ± 0.194.6 ± 0.170.8 ± 0.874.5 ± 0.873.1 ± 0.878.7 ± 0.7

Table A4: Strengthened configuration. Void residues scored as NDRs rather than ignored.

ModelSizeIoUFracParsedBoundaryNDOmAPAP₅₀AP₇₅PQ
DoTR-B-FT107.3M92.9 ± 0.189.2 ± 0.282.7 ± 0.184.7 ± 0.271.9 ± 0.877.0 ± 0.874.9 ± 0.873.5 ± 0.8
DoTR-B-FT-CA + MiAE-B209.7M93.2 ± 0.189.6 ± 0.282.7 ± 0.184.4 ± 0.270.5 ± 0.875.3 ± 0.873.4 ± 0.873.3 ± 0.8
DoTR-B-FT + MiAE-B209.9M92.9 ± 0.189.2 ± 0.282.7 ± 0.184.7 ± 0.267.7 ± 0.872.6 ± 0.870.5 ± 0.874.2 ± 0.7

References

[1] Xu, Xin, et al. "Masqclip for open-vocabulary universal image segmentation." 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023.

[2] Cheng, Bowen, et al. "Masked-attention mask transformer for universal image segmentation." 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.

[3] Li, Zhiqi, et al. "Panoptic segformer: Delving deeper into panoptic segmentation with transformers." 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.

[4] Lau, Andy M., et al. "Exploring structural diversity across the protein universe with The Encyclopedia of Domains." Science 386.6721 (2024): eadq4946.

[5] Yeo, Jingi, et al. "Metagenomic-scale analysis of the predicted protein structure universe." bioRxiv (2025): 2025-04.

We thank all reviewers again for their valuable feedback, which we believe has strengthened our paper. We have incorporated their suggestions into our revised manuscript, and we have uploaded this revision.

Summary of the revision

We have highlighted changes to our manuscript in green. Below, we summarize a non-comprehensive list of the most important changes.

  • DoTR without class supervision: Reviewers have pointed out that our experiments in the original submission have not sufficiently isolated the contribution of class supervision. We have therefore expanded the comparison and provided new experimental results, as explained in the initial general response. To this end:

    • We have revised the abstract, introduction, contributions, Section 5, and conclusion to state that DoTR outperforms two-stage pipelines built from existing parsers on both TEDBench-Joint and an external dataset of experimental structures.
    • We have added the class-agnostic DoTR-B-FT-CA, evaluated as a standalone parser and as the first stage of a two-stage pipeline, to Section 5 (5.1, 5.2, and 5.3).
    • We have updated the discussion in Sections 5.2 and 5.3 to point out that the two-stage pipeline using our own class-agnostic parser is competitive with the end-to-end model, with small parsing gains in some settings and lower AP.
    • In Appendix E.4, we have reported the detailed comparison under both void-handling protocols and under a strengthened configuration, described in Appendices F.1 and D.3.
  • External dataset of experimental structures: Reviewers FTDD and ymgV have pointed out that we have only evaluated our models on the TEDBench-Joint dataset. We have therefore evaluated our models on an external experimental dataset of PDB structures annotated in CATH v4.3. To this end:

    • We have added the results and discussion in Section 5.4 (Table 3).
    • We have included full results with additional training data settings in Appendix E.5 and the experimental setting in Appendix F.4.
    • We have updated the abstract, introduction, contributions, the opening of Section 5, and the conclusion to include the transfer to an external experimental dataset.
  • Generalization to unseen folds: Reviewers bYgw and FTDD have asked whether DoTR can identify, segment, and recognize domains of fold classes unseen during training. To this end:

    • We have added an evaluation on held-out folds with class-agnostic matching and an open-set readout of the embedding space in Appendix E.7, with the experimental setting in Appendix F.5.
    • We have added a pointer to these results in Section 5.5.
  • Evaluation and benchmark construction details: Reviewer bYgw has asked for the evaluation metrics and protocols in the main text, as well as for details on how to use and reproduce the benchmark. To this end:

    • We have added a paragraph on the parsing and unified evaluation metrics to Section 3, while retaining the full definitions in Appendix C, now renamed to "Evaluation Details."
    • We have added Appendix B.7, which describes how TEDBench-Joint is constructed from publicly available source files, listed in Table 8.
  • CATH label levels: Reviewer bYgw has pointed out that the description of the CATH label levels used by the benchmark was ambiguous. To this end:

    • We have simplified the description in Section 3.
    • We have updated the descriptions in Appendices B.1 and B.2.
  • Void residues and annotation uncertainty: Reviewer FTDD has asked how masking void residues during training affects performance. To this end:

    • We have clarified in Appendix B.6 that void residues are those on which TED's parsing methods disagree.
    • We have retrained DoTR-B-FT and DoTR-B-FT-CA with void residues masked from the matching costs and training losses, and reported both training conventions under both scoring protocols in Appendix E.8 (Tables 21 and 22). Section 5.1 refers to these results.
  • Additional stratified analyses: Reviewer FTDD has asked how DoTR performs across structure quality, domain size, domain continuity, chain length, and the number of domains per chain. To this end:

    • We have added these stratified results to Appendix E.6 (Tables 17 and 18).
  • Related work and biological rationale: Reviewer ymgV has asked for a more systematic review of related work in the main text and for the biological rationale for joint modeling of domain parsing and fold classification. To this end:

    • We have moved the review of related work, which was in Appendix A in the original submission, to Section 2.
    • We have updated Appendix A, which now also addresses the biological rationale.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.