- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MediPPD addresses joint physical measurement and localized dermal interpretation of PPD skin-test images. It introduces a 2,216-patient benchmark and separates a reference-calibrated diameter estimator from multiview semantic grounding and clinical-context fusion. A bounded semantic refinement improves the reported segmentation while leaving diameter estimation unchanged. On a fixed patient-disjoint test partition, the paper reports 0.8631 Dice, 2.3299 mm diameter MAE, 0.7837 strong-finding AUPRC, and 0.2471 localized [email protected]. Validation includes architectural ablations, five training seeds, context audits, perturbation tests, and calibration analyses.
Critical strengths
- The benchmark pairs physical measurement, spatial masks, case-level findings, and localized evidence for the same patients. This supports a useful distinction between correct recognition and correct localization, while the patient-disjoint split and explicit separation of development and test roles improve the evaluation design.
- The asymmetric interaction between geometric and semantic branches is coherent and produces substantial reported improvements across multiple outputs. The authors also evaluate five seeds (Table 14), reference degradation, and clinical-context controls. This is broader validation than a single headline segmentation or classification result.
Critical weaknesses
- The metrology ablations are internally inconsistent. Table 3 reports 3.5172 mm MAE for reference-calibrated geometry without correction; Appendix D.1 states that the area-equivalent anchor has 4.4622 mm MAE; Table 15 reports 2.4600 mm when the correction bound is zero, matching the uncorrected major-axis result discussed in D.1. Under Equations 3-4, a zero bound returns the area-equivalent anchor for the given geometry prediction. If these experiments use different geometry checkpoints, retraining protocols, or evaluation subsets, that distinction needs to be stated. Without such clarification, the reported values cannot be reconciled and the correction's contribution cannot be reliably assessed.
- The benchmark's clinical validity is not adequately documented. Appendix A defines clinician-based masks and diameters but does not explain the acquisition sites, annotator qualifications, palpation-to-image boundary procedure, adjudication, or inter-reader agreement. Table 7 also labels the region as 'Erythema/induration,' although these are clinically distinct. Without a clear reference-standard protocol and population description, it is difficult to know what the images and targets represent or how far the results generalize.
Questions for authors
- Please reconcile the three uncorrected metrology MAEs in Table 3, Appendix D.1, and Table 15. Which geometric anchor, geometry checkpoint or retraining protocol, and evaluation subset are used in each, and does the default estimator implement Equations 3-4 exactly?
- How were palpation-defined induration boundaries transferred to image masks, how many clinicians annotated or adjudicated them, and what is their agreement? Please also clarify whether Table 7 combines redness with induration or only uses an imprecise label for the same target.
- What ethics approval or exemption, consent or waiver, and de-identification/data-release permissions cover the patient images and linked clinical records? This determines whether the proposed benchmark can be responsibly released.
3 comments
W2 / Q2. Clinical reference standard, boundary transfer, and reader agreement
We thank the reviewer for questioning the clinical reference standard and annotation reliability. We clarify that the segmentation target is palpable induration rather than visible erythema. Four senior tuberculosis specialists independently annotated the images, followed by final adjudication by a fifth senior specialist.
The benchmark contains 2,216 images from 2,216 patients, with patient-disjoint training, validation, and test splits of 1,551, 221, and 444 cases, respectively.
T5. Clinical outcome distribution in the full cohort.
| Clinical outcome | Patients | Proportion |
|---|---|---|
| Strong positive | 896 | 40.43% |
| Moderate positive | 461 | 20.80% |
| Positive | 179 | 8.08% |
| Negative | 680 | 30.69% |
| Total | 2,216 | 100.00% |
The clinical boundary is determined through palpation and transferred to the original image coordinates for pixel-level annotation. Clinical transverse diameter is recorded independently rather than derived from mask area. We will explicitly document this distinction in Appendix A.2.
T6. Image-level annotation distribution in the full cohort.
| Annotation category | Images | Proportion |
|---|---|---|
| Reference cap | 2,216 | 100.00% |
| Induration-positive | 1,533 | 69.18% |
| Induration-negative | 683 | 30.82% |
| Blistering | 79 | 3.56% |
| Necrosis | 57 | 2.57% |
| Double-ring reaction | 72 | 3.25% |
To quantify independent reader agreement, we conducted a blinded reproducibility audit on the 444 held-out cases. Four primary readers independently annotated each case without access to other annotations or adjudicated masks. Agreement was measured before the fifth specialist performed final adjudication.
T7. Independent inter-reader agreement.
| Annotation task | Agreement metric | Result | 95% CI |
|---|---|---|---|
| Induration masks | Mean pairwise Dice | 0.872 | 0.864–0.880 |
| Blistering | Fleiss' κ | 0.781 | 0.742–0.820 |
| Necrosis | Fleiss' κ | 0.746 | 0.701–0.791 |
| Double-ring | Fleiss' κ | 0.812 | 0.776–0.848 |
| Spatial evidence | Mean pairwise Dice | 0.846 | 0.836–0.856 |
To further evaluate physical boundary consistency beyond region-level overlap, we measured the displacement between independently annotated boundaries after calibration using the 30-mm reference object.
T8. Physical boundary agreement.
| Boundary metric | Result | 95% CI |
|---|---|---|
| Mean pairwise Dice | 0.872 | 0.864–0.880 |
| ASSD (mm) ↓ | 0.74 | 0.69–0.79 |
| HD95 (mm) ↓ | 2.18 | 2.03–2.33 |
The agreement results support reproducibility at both the regional and physical-boundary levels.
Q3. Research ethics and data release
All required institutional authorizations and permissions for the study and public release of the benchmark have been obtained. The authorization covers the release of patient images, annotations, and associated permitted clinical records. The dataset and code are available through the following anonymous repositories:
- Code: https://anonymous.4open.science/r/MediPPD-anony
- Dataset: https://anonymous-hf.com/a/x6u3exlxm2g4/
These resources provide the data and implementation materials needed for independent reproduction and fair comparison.
We hope these results and clarifications address your concerns and invite reconsideration.
Response
Thank you for the detailed rebuttal and additional experiments. The authors have made substantial efforts to address the concerns, particularly regarding the metrology inconsistencies, annotation reliability, and architectural design. However, my main concern about clinical generalizability remains insufficiently addressed. The dataset is still collected from a single center, and the digital color perturbation experiments do not adequately establish robustness under real-world variations in illumination and camera ISP processing. Moreover, the relatively broad Bland–Altman limits and limited accuracy within a 2 mm tolerance raise concerns about the reliability of the proposed system for clinical use, particularly near diagnostic thresholds. Overall, while the rebuttal improves the technical completeness of the work, it does not sufficiently resolve these fundamental limitations. I therefore maintain my original rating.
We thank the reviewer for recognizing the benchmark and asymmetric routing. We reconcile the metrology results and will clarify the clinical reference standard and release documentation.
W1 / Q1. Reconciliation of geometric anchors, checkpoints, and zero-bound identity
We agree that identical geometry predictions, reference calibration, and evaluation cases should yield identical results. In our experiments, however, these values were obtained from different execution rounds conducted at different stages of the study. Each result was valid for its respective experimental run; the ambiguity arose from presenting these results together without clearly distinguishing their run-specific provenance, making them appear inconsistent. We have therefore revised the affected entries and clarified their corresponding experimental settings to avoid this ambiguity.
Specifically, the outdated area-equivalent MAE of 4.4622 mm is corrected to 3.5172 mm, while the 2.4600 mm result is retained as the uncorrected major-axis comparator, not the zero-bound output of the default estimator.
T1. Reconciliation of geometric readouts.
The 2.4600 mm result corresponds to a different geometric definition. It should not be compared as the zero-bound result of the default area-equivalent estimator.
We further evaluated different residual bounds using the same geometric anchor to verify whether zero correction reproduces the uncorrected baseline and whether bounded refinement improves measurement accuracy.
T2. Residual-bound sensitivity.
With zero correction, the estimator reproduces the uncorrected area-equivalent baseline. The ±3 mm configuration achieves the best performance, reducing MAE from 3.5172 to 2.3299 mm (33.75%) and RMSE from 4.8980 to 3.7901 mm.
We retain the area-equivalent anchor because it preserves the complete calibrated induration region, while the residual estimator incorporates directional morphometric descriptors and visual features.
T3. Morphometric-descriptor contribution.
The full morphometric representation achieves the lowest MAE, supporting the complementary contribution of calibrated geometric and visual information. We claim that it provides an effective basis for the complete refinement framework. And the major-axis anchor does not account for the full two-dimensional extent of the induration, making it an incomplete representation of irregular lesion geometry.
Overall, we have corrected the reporting inconsistency and verified the zero-bound identity. The revised results consistently support the intended measurement design: calibrated area-equivalent geometry followed by bounded, task-specific residual refinement.