acceptodds
Under review as a conference paper at ICLR 2027

SNAP: Neuron Segmentation from Supervoxels and Pairwise Affinities

Abstract

The ability to accurately reconstruct the architecture of neurons is a prerequisite for understanding the function of natural neural circuits. Existing neuron segmentation methods often rely on complex post-processing pipelines that require careful hyperparameter tuning. We introduce SNAP, a neuron segmentation method that jointly predicts supervoxels and their mutual affinities, reducing inference to a simple affinity-driven agglomeration procedure. On the established AC3/AC4 and FIB-25 benchmarks, our method achieves state-of-the-art results with inference time comparable to conventional affinity-based approaches. Experiments on these datasets demonstrate the robustness of our method across imaging modalities and dataset scales. Beyond segmentation performance, we revisit commonly used evaluation protocols and clarify inconsistencies in dataset splits and evaluation settings that complicate comparison between existing methods. Our results show that state-of-the-art neuron segmentation can be achieved without complex post-processing while providing a clearer and more reproducible basis for future comparisons.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Neuron Segmentation, Biomedical Image Segmentation, Electron Microscopy Image

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_JqRTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this manuscript, the authors present a neuron segmentation method named SNAP that jointly predicts supervoxels and their mutual affinities. The proposed SNAP framework employs two U-Nets to predict supervoxel assignments and voxel features, and utilizes a Transformer to model the contextual relationships between supervoxels. Neuron instances are then obtained through affinity prediction and agglomerative clustering, thereby simplifying the traditional post-processing workflow. Experimental results on AC3/AC4 and FIB-25 demonstrate strong segmentation performance, particularly in reducing split errors.

Critical strengths
  • The proposed SNAP approach achieved considerable performance improvements on datasets with two different imaging configurations, particularly in terms of reduced split errors. This indicates that the proposed method is not limited to a single experimental scenario.

  • Compared to traditional watershed-based pipelines, the proposed SNAP framework simplifies the post-processing workflow.

Critical weaknesses
  • Although the authors emphasize joint learning as a key contribution, Table 3 mainly compares different affinity prediction formulations. The current experiments do not clearly demonstrate the benefit of joint training over separate training, or how much improvement comes from replacing the conventional watershed step while keeping the other components unchanged.

  • The manuscript emphasizes scalability and simplified post-processing, but runtime evaluation is limited to a relatively small validation volume. The proposed SNAP method is also slower than the affinity-based baseline, and the lack of full-volume runtime and memory analysis leaves its practical scalability insufficiently validated.

Questions for authors
  1. Joint end-to-end training is presented as a key contribution, but its benefit is not clearly demonstrated. A comparison with separate training or stopped gradients through the assignment map P would help clarify its contribution.

  2. This manuscript attributes the reduction in split errors to avoiding watershed, but this claim lacks direct experimental evidence. An ablation study separating the contributions of learned supervoxels and affinity prediction would strengthen this claim.

  3. The scalability claim lacks full-volume efficiency analysis. Could the authors report the runtime and peak memory usage on the full FIB-25 volume and clarify how the global assignment tensor and affinity graph are handled?

3 comments
AuthorsOpenReview ↗

Response - Part 1/2

We thank the reviewer for recognizing SNAP’s strong segmentation performance across different imaging configurations, and its simplified post-processing workflow.

Q1. Contribution of joint end-to-end optimization

We thank the reviewer for this suggestion and agree that this effect should be isolated. We trained SNAP with the assignment tensor PP detached in both supervoxel poolings (ES=D−1PEE^S = D^{-1}PE and CS=D−1PCC^S = D^{-1}PC, with DD computed from the detached PP). With this change, the affinity loss no longer backpropagates into the Supervoxel U-Net. All other settings (architecture, training schedule, hyperparameters, threshold selection) are unchanged. Results on the FIB-25 validation volume:

ConfigurationVOI\mathrm{VOI}VOIsplit\mathrm{VOI}_{\mathrm{split}}VOImerge\mathrm{VOI}_{\mathrm{merge}}
SNAP, detached1.0990.5670.532
SNAP (end-to-end)1.0610.5610.501

Letting the affinity objective influence the supervoxel assignments reduces VOI by 0.038. Most of this comes from fewer merge errors (0.532→0.5010.532 \to 0.501), while split errors are nearly unchanged. This matches the expected mechanism. Agglomeration can only merge whole supervoxels, so a supervoxel that straddles two neurons cannot be corrected afterwards and produces a merge error. The affinity gradient discourages such supervoxels.

This gain is smaller than that of the other components. In Table 3, all configurations except the detached one are trained end-to-end. Replacing DEL affinities on U-Net embeddings with DEL affinities on Transformer-contextualized embeddings reduces VOI from 1.734 to 1.248. Replacing the DEL formulation with the learned MLP affinity head reduces it further to 1.061. We have therefore revised the introduction to present end-to-end training as a complementary design choice rather than the central contribution. We have also added the detached-gradient configuration to Table 3.

Q2. Separating the contributions of learned supervoxels and affinity prediction

We thank the reviewer for raising this important point. We agree that a controlled comparison with a conventional watershed-based affinity pipeline is necessary to better understand the origin of SNAP's improvements. To address this concern, we conducted additional experiments using a watershed-based affinity baseline with matched U-Net backbones, training conditions, and overlapping inference strategy.

Both methods were evaluated on the validation volume and included in the ablation study as well as on two large FIB-25 regions, with the complete segmentation pipeline applied directly to each crop:

MethodROI 1 VOIROI 1 SplitROI 1 MergeROI 2 VOIROI 2 SplitROI 2 Merge
Watershed baseline1.0990.9630.1351.1810.9830.198
SNAP (ours)0.4920.4390.0540.4730.4060.067

SNAP reduces VOI by 55.2% on ROI 1 and 59.9% on ROI 2 compared with the watershed-based baseline, which represents a drastic reduction regarding merge and split errors. These results provide direct experimental evidence that the proposed learned-supervoxel segmentation pipeline substantially outperforms a conventional watershed-based affinity pipeline under matched backbone, training, and inference conditions.

We have included these additional results in the revised manuscript.

AuthorsOpenReview ↗

Response - Part 2/2

Q3. Full-volume scalability, runtime, and memory requirements

We thank the reviewer for highlighting the importance of evaluating computational efficiency beyond the relatively small validation volume. To address this concern, we extended our runtime experiments to FIB-25 ROI 1, containing approximately 4.35 billion voxels, substantially larger than the 5203520^3 validation volume.

The measured runtimes are:

MethodModel + FinalizationPost-processingAgglomerationTotal
Watershed baseline2287 s186 s295 s2768 s
SNAP (ours)3090 s—192 s3282 s

SNAP introduces an approximately 18.6% runtime overhead compared with the affinity baseline, lower than the approximately 28.6% overhead observed on the validation volume. These results demonstrate that SNAP maintains a moderate computational overhead at a substantially larger scale, with a decreasing relative overhead as the volume size increases, highlighting its potential for efficient large-scale neuron segmentation.

Memory-efficient assembly of overlapping supervoxel predictions.

We also clarify that SNAP does not require storing a dense full-volume assignment tensor in memory. The two U-Nets and the affinity Transformer operate on fixed-size 96396^3 patches. Supervoxel assignments and affinities are predicted jointly within each patch, and overlapping predictions are aggregated using a streaming procedure. With 50% patch overlap, voxel-wise assignments from up to eight patches are combined using Hann-window weighting. Instead of materializing all 27 float32 assignment channels across the complete volume, we maintain a sliding frontier of half-tile blocks in host memory. Once all overlapping contributions have been accumulated, the corresponding predictions are reduced to uint8 supervoxel assignments and written to a memory-mapped file. Supervoxel affinities are similarly accumulated in memory-mapped arrays. For the two FIB-25 test regions containing approximately 4.35 and 5.09 billion voxels, the sliding frontier requires 9.7 and 11.1 GiB of host RAM, respectively. The corresponding supervoxel graphs contain approximately 8.9 and 10.3 million nodes. GPU memory consumption during network inference depends on the patch and batch size rather than the complete volume dimensions. Host memory required for prediction aggregation scales with the cross-sectional area, while the final assignments are stored on disk.

Scalability on the full test volume.

For context, the complete FIB-25 volume contains approximately 346 billion voxels. The theoretical estimated number of supervoxels is approximately 700 million. Blocks that are not totally contained in the neuropil mask are ignored in practice. We measured runtime and peak memory during the full inference using the same computing resources as the other runtime experiments. The results are a peak memory of 113 GiB of host RAM during graph agglomerative clustering procedure, a GPU peak memory of 23.6 GiB with a batch size of 16 during model inference, and a total end-to-end inference runtime of 12.2 hours.

Importantly, the streaming implementation is an optimization motivated partly by the I/O constraints of our GPFS-backed computing environment, rather than an intrinsic requirement of SNAP. On systems with faster storage, overlapping predictions could alternatively be written directly to disk and aggregated in a subsequent pass. Likewise, the agglomeration stage operates on the supervoxel graph rather than on dense voxel-wise predictions. Large-scale graph clustering can be partitioned and distributed, as discussed in the manuscript and demonstrated in prior work on distributed clustering [1]. We therefore do not consider full-volume assignment storage an inherent limitation of the proposed method, although practical scalability naturally depends on available computational resources and the implementation of graph agglomeration. We have revised the manuscript to clarify the inference procedure and remove the potentially misleading implication that the entire assignment tensor is materialized in memory. Together, the additional large-region runtime measurements and the memory-efficient inference implementation provide stronger evidence of SNAP's applicability to large-scale neuron reconstruction.

References:

[1] Lu et al. “Large-scale image segmentation based on distributed clustering algorithms” arXiv (2021).

I appreciate the authors' detailed response. My first and third concerns have been adequately addressed. Although my second concern remains partially unresolved, as the contributions of learned supervoxels and affinity prediction have not been fully isolated, I believe most of my concerns have been addressed. In light of these improvements, I will raise my rating.

Reviewer_EGiiOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SNAP, an EM neuron segmentation framework that jointly learns supervoxel assignments and pairwise affinities. A 3D U-Net predicts soft voxel-to-supervoxel assignments, while another U-Net extracts dense embeddings that are pooled into supervoxel representations. A lightweight Transformer and MLP then predict affinities between neighboring supervoxels, followed by UPGMA-based agglomeration. The method avoids conventional watershed-based oversegmentation and achieves strong results on AC3/AC4 and FIB-25, with relatively moderate additional inference cost.

Critical strengths
  1. The joint supervoxel-affinity formulation is technically meaningful and delivers strong segmentation performance. Instead of generating supervoxels from thresholded voxel affinities, SNAP directly learns the supervoxel partition and predicts region-level affinities. The soft pooling operation allows affinity supervision to propagate into supervoxel learning, which is a reasonable way to connect the two components. The experimental gains are substantial: on AC3/AC4, SNAP reduces VOI to 0.615 compared with 0.797 for FGNet, and on the full FIB-25 evaluation region, it achieves 0.700 compared with 1.056 for FFN. These are meaningful improvements for neuron reconstruction.

  2. The evaluation covers both anisotropic and isotropic EM data and includes large-volume segmentation. The paper evaluates on AC3/AC4 and FIB-25, discusses their different evaluation protocols, and performs full-volume inference on FIB-25. I also appreciate the runtime breakdown in Appendix H. The 135.89 s versus 104.35 s comparison with a conventional affinity pipeline suggests that the accuracy improvement does not require an extremely expensive post-processing procedure.

Critical weaknesses
  1. The ablations do not sufficiently isolate the central contribution of joint optimization. Table 3 compares the DEL affinity formulation, Transformer features, and MLP affinity prediction. However, it does not directly test whether gradients from the affinity loss into the supervoxel assignment network are responsible for the improvement. A comparison between jointly trained supervoxels, frozen or detached supervoxels, and a conventional affinity-watershed-agglomeration pipeline using matched backbones would be important. Right now, it is difficult to determine how much of the gain comes from learned supervoxels versus the stronger affinity prediction architecture.

  2. The practical trade-off between split errors, merge errors, and full-volume inference is not fully characterized. SNAP substantially reduces split errors, but its merge error is not always the best. For example, on FIB-25, SNAP obtains VOI_merge = 0.069, compared with 0.053 for FFN. In connectomics, incorrect mergers can be particularly expensive to proofread. Moreover, the reported runtime is measured on the 520³ validation volume rather than providing a complete throughput and peak-memory breakdown for the much larger test volume. The method also involves two U-Nets and overlapping 27-way supervoxel predictions, so the actual large-scale memory cost deserves further clarification.

Post-rebuttal assessment (corrected): I retain my original rating of 3. The detached-P result (VOI 1.099 versus 1.061), matched watershed comparison, and full-volume measurements improve the evaluation. However, the full-pipeline comparison does not isolate the contribution of learned supervoxels from the stronger affinity architecture, as raised by Reviewer JqRT. The matched AC3/AC4 baseline requested by Reviewer RQd4 is also still missing. Joint gradient flow provides a modest additional gain, and proofreading effort remains unmeasured. These results support weak acceptance rather than the earlier increase to clear acceptance.

Questions for authors
  1. Could the authors provide an ablation that disables gradient propagation from the affinity head into the supervoxel network, while keeping the architecture and training budget unchanged? This would directly establish whether end-to-end joint learning is necessary for the reported improvement.

  2. For full-volume FIB-25 inference, how are the overlapping supervoxel assignments and global affinity graph stored and assembled? Please clarify peak memory usage, throughput, and whether the reduced split errors translate into fewer proofreading operations despite the remaining merge errors.

4 comments
AuthorsOpenReview ↗

Response - Part 1/2

We thank the reviewer for recognizing the technical relevance of our joint supervoxel–affinity formulation, its strong segmentation performance, and our effort to evaluate across imaging modalities and scales.

Q1. Ablation of gradient propagation through supervoxel assignments and watershed baseline

We agree that directly evaluating the contribution of gradient propagation from the affinity objective to the supervoxel assignment network is important.

Following the reviewer's suggestion, we conducted an additional experiment in which the assignment tensor PP is detached before being used by the affinity prediction module. The results on the FIB-25 validation volume are:

ConfigurationVOI\mathrm{VOI}VOIsplit\mathrm{VOI}_{\mathrm{split}}VOImerge\mathrm{VOI}_{\mathrm{merge}}
SNAP, detached1.0990.5670.532
SNAP (end-to-end)1.0610.5610.501

We observe that attaching PP further improves the segmentation performance on both VOI metrics, which confirm that end-to-end optimization contributes positively to segmentation accuracy.

However, we do not claim that joint optimization is strictly necessary to obtain strong segmentation results. Rather, it constitutes one of several complementary contributions of SNAP. In particular, the ablation experiments indicate that learning the supervoxel partition directly and incorporating long-range contextual information through the Transformer play substantial roles in the overall performance improvement.

To further isolate the contribution of learned supervoxels, we also evaluated a conventional watershed-based affinity segmentation pipeline using matched U-Net backbones, training conditions, and overlapping inference strategy. The results on two large FIB-25 test regions are:

MethodROI 1 VOIROI 1 SplitROI 1 MergeROI 2 VOIROI 2 SplitROI 2 Merge
Watershed baseline1.0990.9630.1351.1810.9830.198
SNAP (ours)0.4920.4390.0540.4730.4060.067

SNAP reduces VOI by 55.2% on ROI 1 and 59.9% on ROI 2 compared with the watershed baseline, with substantial reductions in both split and merge errors. These results demonstrate that SNAP's improvements are not solely attributable to the different backbone or training configuration, but also arise from its learned supervoxel representation and region-level affinity prediction.

We have incorporated these additional experiments into the revised manuscript and clarified the respective contributions of the different components.

Q2. Split merge trade-off, large-volume inference, memory usage

Split/merge trade-off and proofreading.

We acknowledge that SNAP does not achieve the lowest merge error in all cases. However, on the full FIB-25 region, SNAP reduces VOIsplit\mathrm{VOI}_{\mathrm{split}} by 37.1% compared with FFN, with only a modest increase in VOImerge\mathrm{VOI}_{\mathrm{merge}} (0.053 to 0.069). Importantly, the reported results use an agglomeration threshold selected on the validation set to minimize total VOI, rather than VOImerge\mathrm{VOI}_{\mathrm{merge}} specifically. To further characterize this trade-off, we have added a split–merge trade-off curve in Appendix of the manuscript (see Figure 5) showing that a more conservative threshold allows SNAP to achieve both lower split and lower merge errors than FFN. Interestingly, we observe that most thresholds obtain state-of-the-art results, showing a greater robustness to this parameter than other methods. Users can therefore adjust the operating point according to their proofreading requirements. Furthermore, modern graph-based proofreading tools challenge the traditional assumption that merge errors are substantially more expensive to correct than split errors [1]. SNAP's hierarchical agglomeration framework is compatible with such approaches, which leverage supervoxel graphs and minimum-cut operations to correct erroneous mergers [2]. Together, these results demonstrate that SNAP provides a favorable and adjustable split–merge trade-off.

We have clarified this point in the manuscript.

AuthorsOpenReview ↗

Response - Part 2/2

Q2. Split merge trade-off, large-volume inference, memory usage (continued)

Large-volume inference and computational overhead.

To better characterize scalability beyond the 5203520^3 validation volume, we extended our runtime evaluation to FIB-25 ROI 1, containing approximately 4.35 billion voxels. Using the same computational resources and inference configuration, we obtained the following results:

MethodModel + FinalizationPost-processingAgglomerationTotal
Watershed baseline2287 s186 s295 s2768 s
SNAP (ours)3090 s—192 s3282 s

SNAP introduces an approximately 18.6% runtime overhead relative to the conventional affinity-based pipeline, compared with approximately 28.6% on the smaller validation volume. These results demonstrate that SNAP maintains a moderate computational overhead at a substantially larger scale, with a decreasing relative overhead as the volume size increases, highlighting its potential for efficient large-scale neuron segmentation.

Memory-efficient assembly of overlapping supervoxel predictions.

We clarify that SNAP does not require storing the complete dense 27-channel supervoxel assignment tensor in memory. Both U-Nets and the affinity Transformer operate on fixed-size 96396^3 patches. Predictions are generated locally and assembled through a streaming aggregation procedure. With 50% patch overlap, each voxel receives contributions from up to eight overlapping patches, which are combined using Hann-window weighting. Rather than materializing 27 float32 assignment channels over the entire volume, we maintain a sliding frontier of half-tile blocks in host memory. Once all overlapping contributions to a block have been accumulated, the resulting assignments are finalized, converted into a compact uint8 representation, and written to a memory-mapped file. Supervoxel affinities are similarly accumulated using memory-mapped arrays. Consequently, the memory required for overlapping prediction aggregation scales with the cross-sectional area of the volume rather than its total number of voxels. For the two FIB-25 test regions containing approximately 4.35 and 5.09 billion voxels, the sliding frontier requires 9.7 and 11.1 GiB of host RAM, respectively. The corresponding supervoxel graphs contain approximately 8.9 and 10.3 million nodes. GPU memory consumption during neural network inference is determined by the patch size and batch size, independently of the overall volume dimensions.

Full-volume scalability.

We additionally measured the computational requirements of SNAP on the complete FIB-25 test volume, containing approximately 346 billion voxels. The full-volume inference required:

  • 12.2 hours of total end-to-end processing time.

  • 23.6 GiB of peak GPU memory during model inference, using a batch size of 16.

  • 113 GiB of peak host RAM, reached during graph agglomeration.

The complete volume corresponds to approximately 700 million potential supervoxels before excluding blocks outside the neuropil mask. These measurements demonstrate that SNAP can process volumes containing hundreds of billions of voxels without materializing dense full-volume assignment tensors in memory. The principal host-memory bottleneck occurs during graph agglomeration rather than neural network inference or overlapping prediction aggregation.

Finally, we emphasize that the streaming implementation is partly motivated by the I/O characteristics of our computing environment. On systems with faster storage, overlapping predictions could alternatively be written directly to disk and aggregated in a subsequent pass. Furthermore, as discussed in the manuscript, the graph agglomeration stage can benefit from partitioned or distributed clustering strategies. In particular, [3] demonstrate the feasibility of distributed clustering for large-scale image segmentation, providing a practical foundation for further scaling this stage of our pipeline.

We have revised the manuscript to clarify these implementation details.

References:

[1] Sheridan et al. “Local shape descriptors for neuron segmentation.” Nature Methods (2023).

[2] Dorkenwald et al. “FlyWire: online community for whole-brain connectomics” Nature Methods (2022).

[3] Lu et al. “Large-scale image segmentation based on distributed clustering algorithms” arXiv (2021).

Reviewer_EGiiOpenReview ↗

Follow-up

Correction to my earlier score update: I retain my original rating of 3. The detached-P ablation (VOI 1.099 versus 1.061) and Appendix I's resource measurements address specific concerns. However, JqRT's attribution question is only partly resolved: the watershed comparison changes both the partitioning and affinity architecture, so it establishes the full pipeline's gain rather than the effect of learned supervoxels alone. RQd4's request for a matched AC3/AC4 baseline also remains unanswered. The 0.038 VOI gain from joint gradient flow is modest, and reduced proofreading effort is not measured. The additional evidence strengthens the evaluation but does not justify clear acceptance.

AuthorsOpenReview ↗

Response to the follow-up

We thank the Reviewer for taking the time to respond to our rebuttal.

Baseline comparison

The gain from our method compared to the watershed-based baseline is coupled by design: we pass the learned supervoxels through a Transformer to predict affinities between them. Establishing a baseline that predicts affinities with the exact same architecture is not possible because the pooling operation depends on the learned partition. One could, for example, apply watershed to affinities predicted by a standard U-Net and then pass the fragments to a Transformer, but this is a substantially different pipeline from ours. Furthermore, making it work at large scale, or even devising an overlapping prediction strategy, is a complex task that would require a whole study. Otherwise, pipelines that predict pixel-wise affinities and involve a Transformer architecture exist, and we already compare our method with them [1, 2].

AC3/AC4 baseline comparison

Here are the results of the comparison between our watershed-based baseline and the Superhuman method [3] on AC3/AC4:

MethodVOIVOI splitVOI mergeARAND
Superhuman$^{*}$ [3]1.4081.1450.2630.122
Watershed baseline1.3561.2140.1420.115

As shown in the table, the results are very close to those of the Superhuman method. We therefore decided not to include this comparison in the main paper, as it would provide limited additional insight.

Nevertheless, if the Reviewer considers this comparison important for the main paper, we would be happy to include it.

Proofreading effort

We agree with the Reviewer that a quantitative comparison of proofreading effort would be valuable and would further strengthen our evaluation. However, conducting such a comparison is not straightforward, as proofreading techniques and their associated metrics depend on the underlying segmentation methods and the correction tools available.

Furthermore, existing methods do not consistently report proofreading-effort measurements across all the datasets considered in our evaluation. For instance, Local Shape Descriptors [4] introduced the min-cut metric (MCM), which estimates proofreading effort by counting the number of merge and split operations required to correct segmentation errors. However, this metric was only evaluated on a subset of the Zebrafinch dataset and was not reported for FIB-25. More importantly, the authors explicitly acknowledge that MCM could not be computed for Flood-Filling Networks (FFN), as it requires a supervoxel graph that was not available for the provided FFN segmentation.

Consequently, establishing a fair and rigorous comparison of proofreading effort across all evaluated methods and datasets would require developing and implementing consistent proofreading evaluation protocols, effectively constituting a separate study beyond the scope of the present work.

Nevertheless, we emphasize that our method consistently achieves lower VOI scores than all evaluated affinity-based methods. Furthermore, since our approach relies on hierarchical agglomerative clustering, it remains compatible with the same proofreading strategies employed for affinity-based segmentation. Interestingly, [4] also observed that VOI and MCM generally agree in their ranking of segmentation methods, further supporting the relevance of VOI as an indirect indicator of proofreading effort. Taken together, these observations provide reasonable grounds to expect that our method would require less proofreading effort than the evaluated affinity-based approaches, although a direct quantitative assessment would be necessary to confirm this expectation.

References

[1] Luo et al. “Electron Microscopy Images as Set of Fragments for Mitochondrial Segmentation.” AAAI (2024).

[2] Li et al. “FGNet: Leveraging Feature-Guided Attention to Refine SAM2 for 3D EM Neuron Segmentation.” AAAI (2026).

[3] Lee et al. “Superhuman Accuracy on the SNEMI3D Connectomics Challenge.” arXiv (2017).

[4] Sheridan et al. “Local shape descriptors for neuron segmentation.” Nature Methods (2023).

Reviewer_RQd4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes SNAP, a neuron segmentation method for 3D electron microscopy volumes. SNAP jointly learns soft voxel-to-supervoxel assignments and affinities between neighboring supervoxels, using a Transformer to provide long-range context. This simplifies the core post-processing to thresholded agglomeration over supervoxels. On AC3/AC4 and FIB-25, it reports substantial improvements over prior methods, mainly through fewer split errors. On the FIB-25 validation volume, the reported runtime overhead is approximately 29% compared with an affinity-based pipeline. The paper also discusses inconsistencies in existing evaluation protocols and proposes a cleaner split for FIB-25.

Critical strengths
  • Strong reported results with a clear motivation. SNAP reports large gains on two benchmarks with different imaging characteristics, including a VOI of 0.615 versus 0.797 for the best prior method reported on AC3/AC4. The gains come mainly from fewer split errors, consistent with the motivation of avoiding the boundary-map step that can disconnect thin structures. The motivation, method design, and observed results fit together.
  • Attention to evaluation practice. The paper documents inconsistencies in dataset splits and tuning protocols across prior work. On FIB-25, the authors use no more training data than FFN and tune hyperparameters on a separate validation volume rather than on test annotations. This is a more careful evaluation setting than some prior protocols and is useful for the community.
Critical weaknesses
  • No controlled baseline comparison. The baseline accuracy numbers are taken from prior work, so differences in architecture, training, and tuning make it difficult to attribute the gains to the proposed formulation itself. The authors already implemented an affinity baseline using the same U-Net architecture and overlapping inference strategy (Appendix H), but report only its runtime. Reporting its accuracy under matched training and tuning conditions would provide a particularly informative comparison for establishing how much of the improvement comes from the proposed method.
  • The benefit of joint optimization is not directly tested. A central contribution is jointly learning supervoxels and their affinities, yet no experiment isolates the benefit of allowing the affinity objective to influence the supervoxel assignments. Table 3 provides some evidence about the affinity formulation and the Transformer, but it does not isolate joint optimization or specify the dataset used. The experiments demonstrate strong overall performance without establishing the specific benefit of joint optimization.
Questions for authors
  • What is the segmentation accuracy of the same-backbone affinity baseline from Appendix H under matched training and tuning conditions—VOI on both datasets and Arand on AC3/AC4?
  • Could the authors provide an ablation isolating the benefit of joint optimization, for example by stopping the affinity gradient from flowing through the soft assignments?
  • The paper states that a validation set is used for checkpoint selection, but the AC3/AC4 validation split is not specified. Could the authors clarify this split and confirm that it is disjoint from the test stacks? Please also specify the dataset and configuration used in Table 3.

I lean toward acceptance based on the reported gains and the coherent method design. A controlled baseline comparison and an ablation of joint optimization would help resolve my main uncertainties.

1 comment

We thank the reviewer for the recognition of our strong results and our attention to evaluation protocols.

Q1. Accuracy of the matched-backbone affinity baseline

We agree that a controlled comparison with a conventional affinity-based segmentation pipeline is important for better understanding the contributions of our approach. To address this concern, we conducted additional experiments using a watershed-based affinity baseline with matched U-Net backbones, training conditions, and overlapping inference strategy. On two large FIB-25 test regions:

MethodROI 1 VOIROI 1 SplitROI 1 MergeROI 2 VOIROI 2 SplitROI 2 Merge
Watershed baseline1.0990.9630.1351.1810.9830.198
SNAP (ours)0.4920.4390.0540.4730.4060.067

SNAP reduces VOI by 55.2% on ROI 1 and 59.9% on ROI 2 compared with the watershed-based baseline, which represents a drastic reduction regarding merge and split errors.

Regarding AC3/AC4, conventional affinity-based approaches have already been extensively optimized on this benchmark. In particular, our affinity backbone is closely related to that of Superhuman, whose published results are included in our comparison.

We have added the results of this controlled comparison in the revised manuscript (see Table 3 and Table 5).

Q2. Contribution of joint optimization

We thank the reviewer for this suggestion and have added the proposed gradient-detachment ablation to Table 3.

Specifically, we compare the standard end-to-end formulation with a variant in which the supervoxel assignment tensor PP is detached before affinity prediction, preventing the affinity objective from updating the supervoxel assignment network.

The results on the FIB-25 validation volume are:

ConfigurationVOIVOIsplit\mathrm{VOI}_{\mathrm{split}}VOImerge\mathrm{VOI}_{\mathrm{merge}}
SNAP, detached1.0990.5670.532
SNAP (end-to-end)1.0610.5610.501

We observe that attaching PP further improves the segmentation performance, mainly on avoiding merge errors. This observation is consistent with our original idea that supervoxels straddling two neurons are discouraged by the affinity gradient.

Q3. AC3/AC4 validation split and Table 3 configuration

We thank the reviewer for pointing out these missing details. For AC3/AC4, we follow the training and validation protocol used in previous work: the first 80 stacks of AC4 are used for training, while the last 20 stacks of AC4 are reserved for validation and checkpoint selection. The validation data are therefore disjoint from the test stacks. We have updated the dataset description to explicitly specify this partition.

We have also clarified that the ablation experiments reported in Table 3 are performed on the FIB-25 validation volume. These clarifications have been incorporated into the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of our response and revisions

We sincerely thank all three reviewers for their time and constructive feedback. We appreciate their recognition of SNAP’s strong neuron reconstruction performance (RQd4, EGii, JqRT), the technical relevance of its formulation (RQd4, EGii), the simplification of the post-processing pipeline (RQd4, JqRT), and our attention to evaluation protocols (RQd4, EGii, JqRT).

We have addressed the main concerns through additional experiments and clarifications, particularly regarding the contribution of joint optimization, controlled comparisons with conventional affinity-based segmentation, and computational scalability.

Specifically, we have revised the manuscript to include:

  1. Joint optimization ablation: We added an experiment comparing end-to-end training with a variant in which gradients from the affinity objective are prevented from propagating through the supervoxel assignments. The results confirm that joint optimization provides an additional improvement in segmentation accuracy. We additionally clarified the contribution of this component which was interpreted as the central contribution by all reviewers, whereas it is a part of our framework.
  2. Controlled affinity baseline: We conducted additional experiments using an affinity-based segmentation pipeline with matched U-Net backbones, training conditions, and inference strategy, allowing a more controlled comparison with SNAP.
  3. Large-volume efficiency: We extended the runtime comparison to a substantially larger FIB-25 region containing approximately 4.35 billion voxels, additionally measured runtime and peak memory usage on the full FIB-25 test volume, and clarified how overlapping predictions and supervoxel affinities are processed without materializing dense full-volume assignment tensors in memory.
  4. Evaluation protocol: We clarified the AC3/AC4 training and validation split and specified the dataset used for the ablation experiments.

We provide detailed responses to each reviewer's questions below. The corresponding manuscript modifications are highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.