- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this manuscript, the authors present a neuron segmentation method named SNAP that jointly predicts supervoxels and their mutual affinities. The proposed SNAP framework employs two U-Nets to predict supervoxel assignments and voxel features, and utilizes a Transformer to model the contextual relationships between supervoxels. Neuron instances are then obtained through affinity prediction and agglomerative clustering, thereby simplifying the traditional post-processing workflow. Experimental results on AC3/AC4 and FIB-25 demonstrate strong segmentation performance, particularly in reducing split errors.
Critical strengths
-
The proposed SNAP approach achieved considerable performance improvements on datasets with two different imaging configurations, particularly in terms of reduced split errors. This indicates that the proposed method is not limited to a single experimental scenario.
-
Compared to traditional watershed-based pipelines, the proposed SNAP framework simplifies the post-processing workflow.
Critical weaknesses
-
Although the authors emphasize joint learning as a key contribution, Table 3 mainly compares different affinity prediction formulations. The current experiments do not clearly demonstrate the benefit of joint training over separate training, or how much improvement comes from replacing the conventional watershed step while keeping the other components unchanged.
-
The manuscript emphasizes scalability and simplified post-processing, but runtime evaluation is limited to a relatively small validation volume. The proposed SNAP method is also slower than the affinity-based baseline, and the lack of full-volume runtime and memory analysis leaves its practical scalability insufficiently validated.
Questions for authors
-
Joint end-to-end training is presented as a key contribution, but its benefit is not clearly demonstrated. A comparison with separate training or stopped gradients through the assignment map P would help clarify its contribution.
-
This manuscript attributes the reduction in split errors to avoiding watershed, but this claim lacks direct experimental evidence. An ablation study separating the contributions of learned supervoxels and affinity prediction would strengthen this claim.
-
The scalability claim lacks full-volume efficiency analysis. Could the authors report the runtime and peak memory usage on the full FIB-25 volume and clarify how the global assignment tensor and affinity graph are handled?
3 comments
Response - Part 2/2
Q3. Full-volume scalability, runtime, and memory requirements
We thank the reviewer for highlighting the importance of evaluating computational efficiency beyond the relatively small validation volume. To address this concern, we extended our runtime experiments to FIB-25 ROI 1, containing approximately 4.35 billion voxels, substantially larger than the validation volume.
The measured runtimes are:
| Method | Model + Finalization | Post-processing | Agglomeration | Total |
|---|---|---|---|---|
| Watershed baseline | 2287 s | 186 s | 295 s | 2768 s |
| SNAP (ours) | 3090 s | — | 192 s | 3282 s |
SNAP introduces an approximately 18.6% runtime overhead compared with the affinity baseline, lower than the approximately 28.6% overhead observed on the validation volume. These results demonstrate that SNAP maintains a moderate computational overhead at a substantially larger scale, with a decreasing relative overhead as the volume size increases, highlighting its potential for efficient large-scale neuron segmentation.
Memory-efficient assembly of overlapping supervoxel predictions.
We also clarify that SNAP does not require storing a dense full-volume assignment tensor in memory. The two U-Nets and the affinity Transformer operate on fixed-size patches. Supervoxel assignments and affinities are predicted jointly within each patch, and overlapping predictions are aggregated using a streaming procedure. With 50% patch overlap, voxel-wise assignments from up to eight patches are combined using Hann-window weighting. Instead of materializing all 27 float32 assignment channels across the complete volume, we maintain a sliding frontier of half-tile blocks in host memory. Once all overlapping contributions have been accumulated, the corresponding predictions are reduced to uint8 supervoxel assignments and written to a memory-mapped file. Supervoxel affinities are similarly accumulated in memory-mapped arrays. For the two FIB-25 test regions containing approximately 4.35 and 5.09 billion voxels, the sliding frontier requires 9.7 and 11.1 GiB of host RAM, respectively. The corresponding supervoxel graphs contain approximately 8.9 and 10.3 million nodes. GPU memory consumption during network inference depends on the patch and batch size rather than the complete volume dimensions. Host memory required for prediction aggregation scales with the cross-sectional area, while the final assignments are stored on disk.
Scalability on the full test volume.
For context, the complete FIB-25 volume contains approximately 346 billion voxels. The theoretical estimated number of supervoxels is approximately 700 million. Blocks that are not totally contained in the neuropil mask are ignored in practice. We measured runtime and peak memory during the full inference using the same computing resources as the other runtime experiments. The results are a peak memory of 113 GiB of host RAM during graph agglomerative clustering procedure, a GPU peak memory of 23.6 GiB with a batch size of 16 during model inference, and a total end-to-end inference runtime of 12.2 hours.
Importantly, the streaming implementation is an optimization motivated partly by the I/O constraints of our GPFS-backed computing environment, rather than an intrinsic requirement of SNAP. On systems with faster storage, overlapping predictions could alternatively be written directly to disk and aggregated in a subsequent pass. Likewise, the agglomeration stage operates on the supervoxel graph rather than on dense voxel-wise predictions. Large-scale graph clustering can be partitioned and distributed, as discussed in the manuscript and demonstrated in prior work on distributed clustering [1]. We therefore do not consider full-volume assignment storage an inherent limitation of the proposed method, although practical scalability naturally depends on available computational resources and the implementation of graph agglomeration. We have revised the manuscript to clarify the inference procedure and remove the potentially misleading implication that the entire assignment tensor is materialized in memory. Together, the additional large-region runtime measurements and the memory-efficient inference implementation provide stronger evidence of SNAP's applicability to large-scale neuron reconstruction.
References:
[1] Lu et al. “Large-scale image segmentation based on distributed clustering algorithms” arXiv (2021).
I appreciate the authors' detailed response. My first and third concerns have been adequately addressed. Although my second concern remains partially unresolved, as the contributions of learned supervoxels and affinity prediction have not been fully isolated, I believe most of my concerns have been addressed. In light of these improvements, I will raise my rating.
Response - Part 1/2
We thank the reviewer for recognizing SNAP’s strong segmentation performance across different imaging configurations, and its simplified post-processing workflow.
Q1. Contribution of joint end-to-end optimization
We thank the reviewer for this suggestion and agree that this effect should be isolated. We trained SNAP with the assignment tensor P detached in both supervoxel poolings (ES=D−1PE and CS=D−1PC, with D computed from the detached P). With this change, the affinity loss no longer backpropagates into the Supervoxel U-Net. All other settings (architecture, training schedule, hyperparameters, threshold selection) are unchanged. Results on the FIB-25 validation volume:
Letting the affinity objective influence the supervoxel assignments reduces VOI by 0.038. Most of this comes from fewer merge errors (0.532→0.501), while split errors are nearly unchanged. This matches the expected mechanism. Agglomeration can only merge whole supervoxels, so a supervoxel that straddles two neurons cannot be corrected afterwards and produces a merge error. The affinity gradient discourages such supervoxels.
This gain is smaller than that of the other components. In Table 3, all configurations except the detached one are trained end-to-end. Replacing DEL affinities on U-Net embeddings with DEL affinities on Transformer-contextualized embeddings reduces VOI from 1.734 to 1.248. Replacing the DEL formulation with the learned MLP affinity head reduces it further to 1.061. We have therefore revised the introduction to present end-to-end training as a complementary design choice rather than the central contribution. We have also added the detached-gradient configuration to Table 3.
Q2. Separating the contributions of learned supervoxels and affinity prediction
We thank the reviewer for raising this important point. We agree that a controlled comparison with a conventional watershed-based affinity pipeline is necessary to better understand the origin of SNAP's improvements. To address this concern, we conducted additional experiments using a watershed-based affinity baseline with matched U-Net backbones, training conditions, and overlapping inference strategy.
Both methods were evaluated on the validation volume and included in the ablation study as well as on two large FIB-25 regions, with the complete segmentation pipeline applied directly to each crop:
SNAP reduces VOI by 55.2% on ROI 1 and 59.9% on ROI 2 compared with the watershed-based baseline, which represents a drastic reduction regarding merge and split errors. These results provide direct experimental evidence that the proposed learned-supervoxel segmentation pipeline substantially outperforms a conventional watershed-based affinity pipeline under matched backbone, training, and inference conditions.
We have included these additional results in the revised manuscript.