- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how NBI images available during training can improve WLI-only lesion classification in partially paired collections. PaCT uses verified WLI–NBI pairs as shared identity anchors. A frozen NBI teacher constructs a graph from paired and additional unpaired NBI images, producing relations among the paired anchors. A frozen WLI reference model computes each unpaired WLI query’s affinities to the WLI anchors and uses the verified identities to read the NBI relations, generating a fixed query-specific distillation target. The WLI student learns from class supervision and relational distillation, while inference requires only WLI.
The paper provides a well-defined mechanism for extending the supervisory role of verified pairs to unpaired WLI images. The main unresolved issues concern the reliability of the fixed targets, the additional contribution of NBI-derived relations relative to graph-based regularization, and the correspondence between the reported experiments and the public implementation.
Critical strengths
S1. Using NBI during training to improve a classifier that requires only routine WLI at inference has clear application value.
S2. Section 3.4 connects the two modalities through verified anchor identities. This produces a well-defined target distribution over shared identities and allows unpaired WLI queries to access NBI relations without requiring direct alignment or isomorphism between the two feature spaces.
S3. Table 2 reports weighted F1 improvements of 2.64 and 2.13 percentage points over the best-performing baseline on PICCOLO and PolypSet, respectively. Table 3 examines the contribution of additional NBI graph nodes while retaining the same teacher, and Table 4 tests within-class reassignment of pair identities. These controls provide useful evidence for the roles of additional NBI observations and verified correspondence.
Critical weaknesses
W1. The additional benefit of NBI-derived relational information is not fully isolated. The paper acknowledges existing group-level transfer methods such as PaGKD, and the main comparisons support average performance gains. However, M1 replaces the relational kernel with the identity matrix, removing both NBI relations and non-trivial graph propagation. This control does not distinguish the contribution of NBI-specific relations from the benefits of graph-based regularization. A matched WLI-only graph using the same anchors, graph operators, initialization, and training budget would more directly establish the value of the cross-modal information.
W2. Target reliability depends on the frozen WLI reference affinities and the coverage of the paired anchors. Verified pairing establishes correspondence between anchor identities, but it does not ensure that a WLI query’s neighborhood identifies useful NBI relations. If these affinities are misleading or relevant anchors are absent, the resulting target can remain inaccurate throughout student training because it is cached and fixed. Table 4 supports the usefulness of correct identities, but the paper does not directly examine target quality or transfer benefits under unreliable query neighborhoods or limited anchor coverage.
W3. The statistical and data-protocol reporting limits interpretation of the comparisons. Table 2 reports standard deviations across three seed-level means, each averaged over five folds; these do not directly quantify uncertainty in paired performance differences at the lesion or patient/video-group level. Tables 3 and 4 also provide no uncertainty intervals for the ablation differences. Figure 3 shows non-monotonic point estimates on PICCOLO, whose cause cannot be determined from the reported results. In addition, Appendix A.2 describes grouping principles but does not report the per-fold counts promised in Section 4.1. More explicit pair-verification procedures and anonymized group/split manifests would support independent checking of the evaluation protocol.
W4. The public implementation does not directly support some reported experimental settings. The inspected PPR implementation rejects a restart probability of ρ = 1, although this value is used for the self-distillation ablation. The public interface also couples graph-neighborhood and bandwidth-neighborhood settings, whereas the manuscript describes a sweep that changes the graph neighborhood while retaining the bandwidth setting. The authors should identify the implementation and configurations that generated these results. These discrepancies leave the reproducibility and interpretation of the affected ablations unresolved; their implications for the main results require clarification.
Questions for authors
Q1. How much of the improvement remains against a matched WLI-only graph control using the same anchors, graph operators, initialization, and training budget?
Q2. How reliable are the transferred targets when WLI query neighborhoods are misleading or paired-anchor coverage is limited? A focused analysis would help establish the conditions for useful transfer.
Q3. Which implementation and configurations generated the ρ = 1 ablation and the graph-neighborhood sweep with fixed bandwidth settings? Please clarify their correspondence to the public release.
8 comments
W2. Table 4 supports the usefulness of correct identities, but the paper does not directly examine target quality or transfer benefits under unreliable query neighborhoods or limited anchor coverage.
Q2. How reliable are the transferred targets when WLI query neighborhoods are misleading or paired-anchor coverage is limited? A focused analysis would help establish the conditions for useful transfer.
Response:
We thank the reviewer for highlighting the importance of target reliability and anchor coverage. We address these concerns as follows.
1. Reliability on misleading WLI queries.
We analyzed the training-time targets offline under seed 2026. We counted each unpaired WLI image once per fold if it entered at least one residual distillation batch. This included 3,793 of 3,859 candidate image–fold records on PICCOLO and all 1,573 on PolypSet. The 66 omitted PICCOLO records had valid cached targets and zero exposures. All three retention levels use the same query set. We average the diagnostic metrics equally over five folds.
For each query, we sum the anchor weights in Q and P by class and compare the highest-weight class to the ground truth. In Table 1, Q error is the initial query error rate; Δ True is the change in true-class weight among initially incorrect queries. The correction rate is measured among initially incorrect queries, and the new error rate is measured among initially correct queries. P wrong >50% is the fraction of all queries for which one incorrect class receives strictly more than 50% of P.
Table 1: Target reliability on WLI queries with 100% anchors.
| Dataset | Q error (%) | Δ True (pp) | Correction rate (%) | New error rate (%) | P wrong >50% (%) |
|---|---|---|---|---|---|
| PICCOLO | 6.01 | +1.657 | 1.61 | 0.30 | 5.94 |
| PolypSet | 17.38 | +1.276 | 0.17 | 0.10 | 17.32 |
Propagation increases the average true-class weight for initially misranked queries, although most retain an incorrect top-ranked class. On PolypSet, one 50/50 tie was counted as incorrect by argmax and fell outside the strict majority criterion.
2. Target reliability under reduced anchor retention.
We used nested, class-stratified banks at 25%, 50%, and 100% nominal retention. At reduced retention levels, coverage of every lesion or sequence-proxy group represented in the full bank was not guaranteed. Within each fold and seed, the frozen NBI teacher, WLI reference, and student initialization were shared. The NBI graph used the selected paired NBI anchors and the original unpaired NBI pool. All original paired WLI images remained on the paired CE route, and the original unpaired WLI queries remained on the CE + KL route. The graph and targets were recomputed for each bank.
Table 2 reports training-target diagnostics under seed 2026, averaged over five folds. Pairs is the mean number of retained anchor pairs; Δ True is computed on each setting's own initially misranked queries.
Table 2: Target diagnostics under different anchor retention rates on PICCOLO and PolypSet.
| Dataset | Retention | Pairs | Q error (%) | Δ True (pp) |
|---|---|---|---|---|
| PICCOLO | 25% | 183.6 | 6.71 | +0.906 |
| PICCOLO | 50% | 366.8 | 6.25 | +1.012 |
| PICCOLO | 100% | 732.8 | 6.01 | +1.657 |
| PolypSet | 25% | 30.0 | 18.28 | +0.116 |
| PolypSet | 50% | 59.8 | 17.83 | +0.432 |
| PolypSet | 100% | 119.0 | 17.38 | +1.276 |
Table 3 reports held-out AUC at different anchor retention rates from the separate student ablation, averaged over three seeds and five folds, using the full test WLI set in each fold.
Table 3: Held-out AUC on PICCOLO and PolypSet.
| Dataset | 25% | 50% | 100% |
|---|---|---|---|
| PICCOLO | 75.67 | 77.16 | 78.23 |
| PolypSet | 88.98 | 90.09 | 92.03 |
As shown in Table 2, with fewer anchors, initial errors were more frequent and the average true-class weight gain on each setting's initially misranked queries was smaller. Table 3 shows that held-out AUC was higher with more retained anchors on both datasets. These results characterize the effect of reducing within-group anchors while preserving group coverage.
W3. The statistical and data-protocol reporting limits interpretation of the comparisons. Table 2 reports standard deviations across three seed-level means, each averaged over five folds; these do not directly quantify uncertainty in paired performance differences at the lesion or patient/video-group level. Tables 3 and 4 also provide no uncertainty intervals for the ablation differences. Figure 3 shows non-monotonic point estimates on PICCOLO, whose cause cannot be determined from the reported results. In addition, Appendix A.2 describes grouping principles but does not report the per-fold counts promised in Section 4.1. More explicit pair-verification procedures and anonymized group/split manifests would support independent checking of the evaluation protocol.
Response:
We thank the reviewer for the questions about statistical uncertainty and the data protocol. We clarify these points as follows.
1. Interpretation of statistical results.
The standard deviations in Table 2 summarize variability across three training seeds, after averaging each seed's results over five folds. They do not quantify uncertainty in paired between-method differences over held-out groups. The differences in Tables 3 and 4 are reported as point estimates.
2. Clarification of the results in Figure 3.
The sweep varies the fraction of unpaired NBI images included as graph nodes, while teacher pretraining, the paired anchors, and the WLI training pool remain fixed. On PICCOLO, using 25% and 100% of the unpaired NBI pool as graph nodes yields similar accuracy (78.05% vs. 78.13%), while AUC increases from 77.53% to 78.23%. The reported sweep does not establish the cause of the non-monotonic intermediate results.
3. Per-fold counts and data protocol.
Table 1 below provides the per-fold counts for Appendix A.2. Parentheses give class counts in the order Hyperplasia / Adenoma / Adenocarcinoma for PICCOLO and Hyperplasia / Adenoma for PolypSet. Training pairs are counted as pairs. The validation and test columns include all WLI images assigned to those sets.
Table 1 Per-fold data counts on PICCOLO and PolypSet.
| Dataset | Fold | Train pairs | Train unpaired WLI | Train unpaired NBI | Val WLI | Test WLI |
|---|---|---|---|---|---|---|
| PICCOLO | 0 | 737 (189/422/126) | 754 (118/610/26) | 178 (34/123/21) | 193 (26/144/23) | 436 (71/278/87) |
| PICCOLO | 1 | 765 (188/468/109) | 721 (107/515/99) | 147 (18/108/21) | 193 (34/136/23) | 441 (75/335/31) |
| PICCOLO | 2 | 716 (183/450/83) | 792 (98/595/99) | 187 (29/153/5) | 196 (34/131/31) | 416 (89/278/49) |
| PICCOLO | 3 | 727 (195/438/94) | 799 (100/602/97) | 177 (24/137/16) | 189 (34/132/23) | 405 (75/282/48) |
| PICCOLO | 4 | 719 (157/454/108) | 793 (125/584/84) | 190 (34/135/21) | 186 (28/135/23) | 422 (94/281/47) |
| PolypSet | 0 | 120 (48/72) | 311 (135/176) | 350 (138/212) | 58 (26/32) | 126 (54/72) |
| PolypSet | 1 | 120 (48/72) | 307 (133/174) | 343 (139/204) | 60 (25/35) | 128 (57/71) |
| PolypSet | 2 | 117 (45/72) | 322 (141/181) | 343 (159/184) | 54 (22/32) | 122 (55/67) |
| PolypSet | 3 | 123 (51/72) | 319 (143/176) | 313 (149/164) | 58 (26/32) | 115 (43/72) |
| PolypSet | 4 | 115 (45/70) | 314 (135/179) | 337 (158/179) | 62 (29/33) | 124 (54/70) |
The updated anonymous release includes the actual fold manifests and training-pair ledgers under splits/grouped_v1/. The manifests record split membership, group aliases, modality, label, and pair ID. The ledgers identify the WLI and NBI endpoints of every training pair.
Response:
We thank the reviewer for suggesting the matched WLI-only graph control. We address this concern as follows. We added the requested WLI-only graph control. Both graph variants use kNN construction followed by PPR. The WLI control builds its kernel from the paired WLI anchors and forms the target as PjW=QjUW,0KW.
The WLI graph and M2 use the same verified paired-anchor identities, WLI student initialization, optimization budget, fold split, random seeds, and checkpoint-selection rule. Both graphs contain paired anchors only. Results use the same three random seeds and five-fold evaluation as the paper's ablation experiments. M1 and M2 correspond to the variants reported in Table 3 of the paper, while the WLI graph is the newly added control. M1 serves as the identity-kernel reference.
Table: WLI self-graph control on PICCOLO and PolypSet.
The WLI graph improves over M1 on both datasets. Relative to the WLI graph, M2 improves ACC, weighted F1, and AUC by 3.85, 3.82, and 2.13 percentage points on PICCOLO, and by 1.78, 1.94, and 2.01 percentage points on PolypSet. Both graphs contain paired anchors only. The results support a benefit from NBI-derived relations beyond the tested WLI graph propagation.