- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes TEMPLAR, which automatically induces and refines structured radiology reporting templates through anatomy-guided concept clustering, knowledge graphs, external evidence retrieval, and feedback-based refinement. The framework is evaluated on four report datasets for template coverage, reconstruction fidelity, and diagnostic fidelity.
Critical strengths
Automated template construction addresses a practical need by reducing manual design effort and improving report standardization.
Critical weaknesses
- Adaptive report structuring is clinically meaningful, but its value is limited. For corpora of current size, iterative template induction using a strong GPT or Claude model, followed by minor manual refinement, is a straightforward alternative worth comparing against.
- State-of-the-art models, such as GPT and Gemini, perform unexpectedly poorly, trailing the proposed method by more than ten points. This raises concerns about comparison fairness. Please clarify how these baselines were implemented.
- The datasets are too small to support the “large-scale” description, with even the largest containing fewer than 5,000 reports.
- The experiments use only subsets of the original datasets. For example, MERLIN originally contains over 20,000 reports, but only 4,080 are used here.
Questions for authors
- What effect would be achieved if GPT or Claude were to be optimized carefully?
- Why were only subsets of the original datasets used, and how were these subsets selected?
General Reply and Summary of Revisions
We thank EERH for the questions on clinical value, Gn6G for the guidance on fair comparison and evaluation reliability, and tuVo for examining the methodological contribution and baseline design. These comments guided the following experiments and clarifications. Manuscript revisions are highlighted in yellow.
Clinical motivation and public-template baselines (EERH). We clarified how automated induction supports template development and adaptation across clinical settings, and added public PET/CT and CT reporting templates as baselines (Secs. 1 and 4.1; Tables 1–2).
Controlled comparisons (Gn6G, tuVo). We reevaluated every method's template with the same Clinical Agent for structuring, reconstruction, and diagnosis. We also clarified that the LLM baselines process the complete training split using shared report batches and instructions (Sec. 4.1; Tables 1–2; Appendix C.2).
Clinically grounded evaluation (EERH, Gn6G). We added GREEN to assess clinically relevant reconstruction errors (Tables 1–2; Appendix B.3.3) and expanded physician assessment of template design and clinical utility, with raters blinded to method identity (Sec. 4.2; Figs. 3 and 6; Appendices B.3.2 and D.3).
Evaluation reliability (Gn6G). We documented the revised evaluator configuration and added repeated evaluations and case-paired statistical analyses to assess run-to-run variation and statistical uncertainty under the common protocol (Tables 1–2 and 8–11; Appendices A and D.2).
Agent roles and template evolution (Gn6G). We clarified the task-agent roles, their fixed execution order, and the separate Evaluation Agent. Template evolution focuses on coarse-to-fine correction of template structure and slot semantics, including one round of downstream feedback refinement. Longitudinal adaptation to changing clinical practice is a future direction (Abstract; Secs. 1, 3.2, and 5; Appendix B.2).
Strong models and task-specific design (tuVo). We discuss why general model capability does not guarantee the best clinical template quality, and how TEMPLAR addresses the organization and refinement of clinical concepts explicitly. GPT/Claude backbones can also be incorporated into this procedure (discussion in our response to tuVo, W1/Q1; method in Secs. 3.1–3.2).
Clinical privacy and local deployment (tuVo). We discuss institutional restrictions on external processing of clinical reports and the local deployment option offered by the open-weight counterpart of our Qwen template-construction backbone (discussion in our response to tuVo, W1/Q1; experimental backbone configuration in Appendix A).
Cohort selection and sensitivity analysis (Gn6G, tuVo). We clarified cohort selection and train/test separation, and added a separate aggregate comparison excluding the machine-generated BDMAP reports (Secs. 4.1–4.2; Table 7; Appendices C.1 and D.1).
The reviewer-specific responses provide the supporting results and manuscript locations for each update.