- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces WINZE, a class-agnostic and training-free approach for volumetric instance segmentation in natural-color tomography. According to the supplied submission description, WINZE applies a frozen SAM 2 model section-by-section in three orthogonal orientations and uses rule-based cross-view assembly to convert the resulting 2D masks into volumetric instances. The reported evaluation comprises 1,239 manually annotated instances and seven segment-everything baselines. The abstract claims that WINZE ranks first in instance [email protected] on four of six scored scopes and reaches 1.3–3.1× the strongest comparator on three sets of scored planes. The problem is practically relevant: generic 2D foundation models do not directly provide stable volumetric identities, and dense 3D annotation is expensive.
Critical strengths
- Practically consequential problem formulation
Leveraging a frozen 2D foundation model for volumetric instance segmentation without task-specific training is attractive where dense 3D annotation is costly. The proposed use of three physical views directly targets the known limitation that independent slice-wise processing discards volumetric structure. Prior literature explicitly distinguishes slice-wise approaches from volume-wise approaches and notes that independent 2D segmentation does not use 3D structure.
- Potentially useful annotation and benchmarking effort
A manually annotated set of 1,239 instances, if independently sampled and carefully quality-controlled, could be a meaningful resource in a data-scarce application. The comparison with seven segment-everything approaches is broader than a single-baseline demonstration and may reveal a real practical advantage.
Critical weaknesses
- The stated evaluation does not clearly validate the central 3D claim
The decisive concern is the apparent mismatch between volumetric instance assembly and scores reported on planes/scopes using instance [email protected]. Slice-level agreement can remain high even when a method: merges two touching objects between annotated planes; fragments one object across depth; switches identity after an occlusion or narrow neck; invents or removes handles, cavities, branches, or disconnected components; produces mutually inconsistent labels across orthogonal views. These are not corner cases; they are the defining errors of volumetric instance assembly. By comparison, prior slice-to-volume work motivates propagation specifically from cross-slice consistency and evaluates the association component against a no-association slice baseline.
- Novelty and causal contribution are insufficiently isolated
Applying a frozen segmenter to orthogonal slices and reconciling masks is intuitively reasonable, but the contribution may reduce to a collection of task-specific rules unless the paper demonstrates that the assembly procedure embodies a general and non-obvious principle. The relevant literature already includes orthogonal annotation, orthogonal ensembles, multi-view co-training, multi-view fusion, masked multi-view volumetric segmentation, and learned slice-to-volume propagation; representative bibliographic entries include Orthogonal Annotation (CVPR 2023) [1], Orthogonal Ensemble Networks (MICCAI 2021) [2], S2VNet (CVPR 2024) [3], and SwinMM (MICCAI 2023) [4].
[1] Orthogonal Annotation Benefits Barely-Supervised Medical Image Segmentation. CVPR 2023.
[2] Orthogonal Ensemble Networks for Biomedical Image Segmentation. MICCAI 2021.
[3] Clustering Propagation for Universal Medical Image Segmentation (S2VNet). CVPR 2024.
[4] SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation. MICCAI 2023.
Questions for authors
-
What genuinely volumetric ground truth and metrics are available? Please report, at minimum, full-volume instance precision/recall/F1 at multiple 3D IoU thresholds, panoptic quality or AP, split and merge counts, fragmentation per ground-truth object, identity consistency across depth, and connected-component/topology errors. If only sparse planes are annotated, please explain exactly which 3D claims can be inferred and provide a validation study comparing sparse-plane ranking with densely annotated subvolumes.
-
How were the 1,239 instances sampled and annotated? Please report the number of independent volumes, specimens, materials, scanners, acquisition settings, annotators, adjudication protocol, inter-annotator agreement, object-size distribution, and whether multiple planes from one object are counted as independent observations. What is the statistical unit used for uncertainty estimates?
-
Were any annotations used to design or tune WINZE? For every threshold and rule, identify its value, search range, tuning specimens, and whether those specimens overlap with reported evaluation volumes. Please provide leave-one-specimen-out or held-out sensitivity results.
-
What is the exact assembly algorithm? Please provide pseudocode defining mask generation, within-view linking, cross-view matching, graph construction, transitive closure, voxel assignment, unmatched masks, conflicting labels, merge/split resolution, nested objects, holes, disconnected components, and the order in which rules are applied.
-
Can the authors isolate the source of improvement? Please provide one-/two-/three-view results; leave-one-orientation-out results; assembly-rule deletions; no-assembly and simple-fusion baselines; SAM versus SAM 2 under the same pipeline; and backbone-only versus filtering-only versus assembly-only contributions.
-
How does the method behave outside the reported acquisition regime? Please stratify results by voxel anisotropy, resolution, object density, object diameter, elongation, branching, contact/overlap, color contrast, partial-volume artifacts, and view-dependent appearance.
-
What are runtime, peak memory, and storage costs per volume? Three-orientation sectioning can multiply SAM 2 inference and mask-storage costs. Please report hardware, number of sections and masks, wall-clock time by stage, scaling with volume dimensions/object count, determinism, and compute-matched baseline results.
2 comments
Response to Reviewer LFHf (2/2)
Q3: Use of annotations for tuning and held-out sensitivity
Ground-truth annotations were not used to design WINZE or to set its thresholds. The thresholds were selected through qualitative inspection of the outputs on the development cube rather than a numerical search. Some thresholds were also set on a cube from the same volume as DS00136_cube0004 that does not overlap it (Appendix B.1). There is thus no search range. For each threshold and rule, the revised Appendix C identifies its value and any overlap with the reported evaluation volumes. As suggested, we performed a held-out sensitivity analysis on the evaluation cubes by varying each main threshold by ±20%. F1 changed little for most thresholds (Table 25).
Q4: Exact assembly algorithm
We provide pseudocode for the assembly procedure as Algorithm 1 in Appendix C, covering the items the reviewer lists. WINZE has no splitting stage and no constraints that guarantee the preservation of nested structures, holes or connectivity. These are thus not included in the pseudocode.
Q5: View, rule and baseline comparisons
We provided the suggested results to isolate the source of improvements (Tables 23, 24 and 29). F1 dropped substantially when removing whole-mask grouping, which uses both XZ and YZ, or track filtering (Table 23). These stages newly added in WINZE therefore explain the improvements (Section 5.3). For SAM versus SAM 2, using SAM (ViT-B) reduced F1 in almost all comparisons (Table 29).
Q6: Behavior outside the reported acquisition regime
Following the reviewer’s suggestion, we simulated changes in acquisition conditions by subsampling along Z, lowering in-plane resolution, reducing contrast, adding noise and blurring the existing data (Table 30). Performance decreased when the in-plane resolution was lowered, which shows that in-plane resolution is important when applying WINZE (Appendix G.9). We addressed the requested stratifications except for view-dependent appearance, which cannot be generated by modifying the data (Appendix G.7). The stratified analyses showed that WINZE was more likely to miss small objects and objects with low color contrast against the background. These are, however, possibly due to limitations of SAM 2, not those of newly developed parts. In contrast, touching objects were detected at least as well as isolated objects (Tables 11, 15 and 28).
As natural-color tomography is a recent technique, additional high-quality ground-truth datasets beyond those reported here are currently unavailable.
Q7: Runtime, memory and storage
We measured the requested quantities and report them in Tables 19 and 27. As a summary, hardware and wall-clock time by stage were already reported in Appendix F and Table 19 of the submitted manuscript. We additionally measured the numbers of sections and masks, peak memory, storage, scaling with volume dimensions, and determinism (Table 19). For scaling with object count, we report detection and assembly times as a function of mask count, which serves as a proxy for object count (Table 19).
Compared with the seven existing methods discussed in the manuscript, WINZE required a longer runtime, approximately 14 times their mean runtime (Table 27). Its peak GPU memory use (9.6 GiB) was within the range of the existing GPU-based methods (2.4–14.0 GiB). Storage per volume, including intermediate files, was approximately 1.3 GB for WINZE, compared with 0.08–351 MB for the existing methods (Table 27). These results show that WINZE can improve the results with reasonable resource requirements.
Response to Reviewer LFHf (1/2)
We thank the reviewer for highlighting the distinction between agreement on individual planes and maintaining consistent object identities in 3D. Following the reviewer’s suggestions, we conducted additional experiments mainly on the 3D evaluation. They showed that the plane-level evaluation is consistent with the 3D evaluation, and clarified the kinds of 3D errors that WINZE makes. We respond to each point below.
W1: Validating the 3D assembly beyond plane-level F1
Our evaluation does not rely solely on agreement within the XY planes. The scored XZ and YZ planes can reveal splits and merges across depth where those errors intersect the annotated planes. The densely annotated 3D ROIs provide a direct 3D evaluation (Section 4.3). Following the reviewer’s comment, we compared F1 results in the planes and in the 3D ROIs for all pairs of the eight methods using three volumes (revised Table 2). In ~93% of the pairs, the same method showed higher scores. The two evaluations thus gave consistent results on the volumes we evaluated (Appendix G.5).
We also counted the split and merged objects in 3D, which plane scores cannot fully capture. We compared WINZE with three alternatives: no whole-mask grouping, IoU linking, and independent IDs per XY section. WINZE still produces fragmented instances, but its fragmentation per ground-truth object was equal to or lower than in all three (Table 26). The no-association slice baseline had no true positives in the 3D ROIs, indicating that association across sections is necessary (Tables 23 and 26). In the revised manuscript, we present these results and clarify the role and scope of each evaluation (Section 4 and Appendix G.5).
W2: Contribution relative to prior orthogonal and multi-view methods
The novelty of WINZE lies not in using orthogonal slices but in the principle of determining 3D labels without altering the shape of the 2D masks of SAM 2. Because of this principle, WINZE is expected to apply to a broad range of volume data that SAM 2 can segment (Section 6). The ablation study isolated the contribution of this mechanism, as removing whole-mask grouping or track filtering lowered F1 substantially (Table 23). Removing the stages taken from u-Segment3D changed F1 little (Section 5.3). WINZE is therefore not merely a collection of task-specific rules.
The four cited methods assume training with annotations and output class-specific regions. WINZE differs from them in that it produces instance labels without additional training. We state this in the revised manuscript (Appendix A).
Q1: Volumetric ground truth and 3D metrics
Following the reviewer’s suggestion, we report instance precision, recall and F1 at 3D IoU thresholds of 0.3 and 0.5, split and merge counts, panoptic quality, fragmentation per ground-truth object, and connected-component counts in three densely annotated 64³ ROIs (Tables 22, 23 and 26). No full-volume ground truth is available because ground-truth data for natural-color tomography remain scarce. We thus used sparsely annotated planes instead. On the evaluated cubes, the order of F1 scores among the compared methods was broadly consistent between the annotated planes and the densely annotated subvolumes (Table 26). The plane-level evaluation therefore reflects the relative performance of methods in 3D. We state this in the revised manuscript (Appendix G.5).
Q2: Sampling, annotation protocol and statistical unit
Following the reviewer’s suggestion, we expanded the information on the annotation protocol and specimens. The submitted manuscript already included the numbers of volumes and specimens, the imaging methods and voxel dimensions, and the selection of cubes, planes and ROIs. The annotation procedure, which used one annotator without measuring inter-annotator agreement, was also described in it (Section 2.2 and Appendix B). In the revised manuscript, we additionally reported the size distribution of the ground-truth objects (Table 31). We also stated that there was no adjudication, that multiple sections of an object are not counted as independent observations, and that the specimen is the statistical unit (Section 4.3 and Appendix B.1). We did not report interval estimates because only two specimens for evaluation were available, which is too few for a meaningful estimate (Section 4.3).