- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether frozen vision representations contain rare, fixed subsets of native channels that remain highly predictive for segmentation. It proposes Dense-STG followed by Patience-Annealed Backward Feature Elimination (PA-BFE). After selection, all other channels are removed and a fresh decoder is trained from scratch. On ACDC with frozen DINOv3 features, a 4/384-channel support achieves 0.772 test Dice versus 0.811 for the full representation, while random four-channel supports perform substantially worse. The paper further studies repeated discovery, disjoint supports, deletion audits, decoder seeds, cross-dataset transfer, and multiple datasets, encoders, and decoder architectures.
Critical strengths
- Clean evaluation protocol. The discovery model is discarded, unselected channels are physically removed, and a fresh decoder is trained from scratch. This substantially reduces concerns about selector–decoder co-adaptation and directly tests the predictive value of the retained channels.
- Strong empirical validation. The selected supports clearly outperform same-width random supports. Repeated searches, disjoint alternatives, deletion audits, seed replications, cross-setting experiments, and ACDC-to-M&Ms transfer provide broad evidence that the phenomenon is real and non-unique.
Critical weaknesses
-
The slice-level medical Dice convention may make Full-relative retention look more favorable. The medical metric averages hard Dice over 2D slices and foreground classes, assigns a score of one to both-empty prediction/target pairs, and retains background-only slices. Shared both-empty terms can mechanically pull the Selected/Full ratio toward one. The 128×128 experiment addresses label resolution but retains the same aggregation convention, while the patient bootstrap quantifies sampling uncertainty without changing the image-weighted estimand. Reporting class-wise per-volume Dice on the original-resolution masks, together with results restricted to present structures, would clarify how much of the headline 95.2% retention persists under a volumetric evaluation.
-
The exact four-channel endpoint is not fully robust. The main search uses one decoder seed and repeatedly queries the same development split. With seeds 23 and 41, the fixed four-channel support has development gaps of −0.032 and −0.041, exceeding the stated 0.02 tolerance. Thus, the broader sparse-support finding is convincing, but the precise four-channel sufficiency and local-irreducibility claims appear seed-sensitive.
Questions for authors
1.Can the authors report class-wise patient- or volume-level Dice on the original-resolution ACDC and M&Ms masks, with absent-class handling clearly specified?
2.What support size is obtained when deletion acceptance is based on multiple decoder seeds? Do all three-channel children of the four-channel support remain below the threshold across seeds?
3.Can the authors repeat discovery across multiple patient-level train/development splits and report endpoint size, performance, and coordinate overlap?
4.Under a matched search budget, how does PA-BFE compare with Dense-STG alone or standard greedy backward elimination?
2 comments
Thank you for authors' response and clarifications which address several of my concerns. I would keep its original rate.
Response to Reviewer nFaP
We sincerely thank the reviewer for recognizing the fresh-decoder evaluation protocol and the evidence from repeated searches, disjoint supports, and cross-dataset transfer. We address the questions about medical Dice aggregation, the stability of the selected support, and the contribution of the search procedure below.
Summary of Key Concerns and Main Responses
W1. Please see Q1.
W2. Please see Q2–Q3.
Q1. Class-wise volumetric Dice on ACDC and M&Ms
Thank you for highlighting the effect of both-empty slices on the original Dice metric. The reported 95.2% retention on ACDC uses slice-averaged Dice, and the finer-resolution evaluation in Supplement A.4.1 retains that aggregation rule. Following your suggestion, we have added class-wise volumetric Dice on the original-resolution ACDC and M&Ms masks (Supplement A.10.1, Table S10).
For each cardiac phase, we compute Dice over the complete volume for each structure present in the reference mask. A slice on which both masks are empty therefore contributes no separate score. We average phase scores within each patient and give each patient equal weight.
Table R1. Original-resolution volumetric Dice. Each Dice cell gives the full-decoder score followed by the selected four-channel score. Confidence intervals for mean retention use 10,000 paired patient-bootstrap resamples.
The selected supports retain over 97% of full-decoder volumetric Dice for each of the three structures on both datasets. High retention therefore persists when both-empty slices receive no separate credit (Sec. 4.5).
Q2–Q3. Stability across decoder seeds and patient splits
Thank you for distinguishing the stability of the exact support from the recurrence of compact, predictive supports. We have tested both sources of variation (Supplement A.10.2, Table S11).
1. Multi-seed acceptance (Q2). Using mean development Dice across three decoder seeds, the search selected four channels. All four three-channel subsets of the original support fell below the acceptance threshold at every seed (Table S11, panel A).
2. Patient-level splits (Q3). Repeating discovery on two alternative splits yielded four- and eight-channel supports. They shared three and four channels with the original support, respectively, and both retained over 97% of matched full-decoder Dice on the same held-out patients. High retention therefore persisted across splits even as the width and membership of the support varied (Table S11, panel B).
Q4. Search strategy under a common budget
Thank you for suggesting this comparison. We evaluated Dense-STG alone and compared deletion strategies from its confirmed 17-channel ACDC support (Supplement A.8, Table S7, panel A). Dense-STG already provided a predictive starting point. Within the same 17-check budget, PA-BFE reduced it to four channels, while exhaustive greedy still retained 16, with similar test Dice. The benefit of PA-BFE here is how quickly it reaches a narrow support while preserving predictive performance.
We thank the reviewer again for the constructive feedback and welcome further discussion.