- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes sparse cubical complexes to accelerate persistent homology (PH) computations in topology-aware image segmentation. Following Betti Matching, the proposed sparseBM makes loss computation up to about 100 times faster. It is evaluated on practical 3D segmentation datasets.
Critical strengths
The method makes PH-based losses fast enough to be practical for 3D segmentation. Loss computation becomes up to about 100 times faster, and the training overhead on 3D data is small, so a PH-based loss can be used with realistic patch sizes, where the original Betti Matching is infeasible.
Critical weaknesses
- The relation to Betti Matching and the method details are not clearly presented
The method follows "the principles of Betti Matching", but the method section does not explain the procedure clearly (Betti Matching is described only briefly in the related work). The constructions used in the experiments must also be pieced together (S, C=min(L, P), the convention that filtration values are one minus the predicted probability, etc). Moreover, discarding cells above a threshold is a standard technique (e.g., Ripser and Cubical Ripser), so presenting the contribution as "sparse cubical complexes that omit high-valued regions" somewhat overstates its novelty, although retaining the original filtration values on a common comparison-based subcomplex is a sensible refinement.
- The evaluation is not very fair
Since sparseBM optimizes a Betti Matching objective, using the Betti Matching error as the main topological metric favors the proposed method, and the claim that it "substantially reduce topological errors" rests mainly on this metric. No meaningful changes in clDice, VOI, or NSD are observed. Also, the model selection is aligned with the same metric. For every method, the loss weight is chosen by the lowest validation Betti Matching error.
TopoMortar (Valverde et al., BMVC 2025, https://bmvc2025.bmva.org/proceedings/596/) pointed out pitfalls in evaluating topology losses. Data augmentation, which is under-reported in the literature, strongly affects topology accuracy and can make a Dice+CE baseline surpass most topology losses, and with few seeds any loss can be made to appear best. The submission states that it follows nnU-Net principles but without specifying the augmentations, and its results and paired t-tests rely on three seeds.
Questions for authors
- Could the authors write out the sparseBM loss explicitly, including its combination with Dice+CE and the selected weight for each dataset?
- How does sparseBM compare with the baselines on topological metrics it does not directly optimize?
- Can the main results be reported with more seeds?
11 comments
BM as primary metric
R SQck: The paper uses BM error as the primary metric, and it is related to the proposed training objective. Metrics that are not directly optimized should also be used.
TLDR. We now evaluated all methods on two additional metrics that measure pure topology and are not directly related to our method. Furthermore, we added three downstream evaluations on metrics connected to the datasets' use cases. Our method's improvements persist in all these additional measurements. Many other metrics (VOI, NSD) are unsuitable for measuring pure topology (Berger et al., IPMI 2025), and we provide simple explanations and examples below.
We believe that BM error is the most principled metric for capturing pure topological accuracy, accounting for spatial correspondence of topological features. To further assess topological accuracy, we added two additional metrics that are not directly optimized by our method: the global difference in the number of topological features (Betti-number error; Hu et al., NeurIPS 2019) and the DIU (discrepancy of intersection and union; Lux et al., ICLR 2025). We added these metrics to the main results table and show significant improvements across both. Due to space limitations in the main table, we moved NSD and VOI to the appendix. We are open to including the results the reviewers deem most appropriate in the final main tables and look forward to their feedback on this.
We agree that alternative metrics (VOI, NSD, clDice) are informative for certain downstream tasks and that they often correlate with topological correctness. However, none of them measure topology itself. Each is a geometric or information-theoretic surrogate that also responds to region size, boundary position, or overlap, and each can miss topological errors (Berger et al., IPMI 2025, "Pitfalls of topology-aware image segmentation"). The mentioned pure topology metrics (BM error, Betti number error, DIU) are not affected by region sizes or boundary displacements. We provide minimal 2D examples that showcase the differences at the end of this response.
Furthermore, we present three examples to demonstrate that the measured topological improvements can yield gains on the downstream task. For a more detailed description of these results, we refer readers to the new Appendix A.5.
Improvement in ejection fraction error and closed ring rate on ACDC. In every short-axis slice, the myocardium forms a closed ring around the left-ventricular cavity. If the ring is broken, the cavity and thus the ejection fraction (EF), the key clinical measure of cardiac function, cannot be computed correctly. sparseBM reduces the share of broken rings by 43% compared to the best baseline (2.4% vs. 4.2%) and lowers the EF error from at least 3.31 to 2.75 percentage points, both significantly better than all baselines.
Improvement in lesion detection on BraTS-METS. For brain metastases, the number of lesions is clinically relevant, e.g., for treatment planning. Under the official BraTS lesion-wise evaluation, sparseBM improves the lesion detection F1 score from at most 0.744 to 0.778, significantly better than all baselines, and achieves the best lesion-wise Dice and HD95.
Improvements on tree length detected (TLD) and branch detection (BD) in airway tree segmentation. As presented in Appendix A4.2, we applied our method as a post-processing step on airway tree segmentations. Across all base segmentation networks, the post-processing improved TLD and BD. Although we did not use any additional optimizations (e.g., TTA, ensemble methods, etc.), our method achieved a top ranking in this year's ATM challenge, underscoring its real-world effectiveness.
Despite these gains, we note that our method focuses on enhancing general topological accuracy. The question of whether this corresponds to improved downstream performance for domain-specific tasks is not strictly in the scope of this paper. Notably, Betti matching, and thus sparseBM, allows applying the loss to specific dimensions only, e.g., to connected components for lesion detection or to loops for closed myocardial rings, which can be relevant for downstream tasks. However, since we aim to show the general working mechanism, we do not adjust the loss to favor specific downstream evaluations/tasks.
Examples for conflating factors of NSD, clDice, and VOI metric
Minimal 2D examples showcasing the conflation of region sizes and boundary location in NSD, VOI, and clDice (foreground 4-connected, background 8-connected, NSD tolerance τ = 1 px):
(a) break (b) hole (c) shifted (d) dilated (e) dent
......... ......... ......... ......... .........
......... .#######. ......... .+++++++. .###o###.
..##o##.. .###o###. ..ooooo.. .+#####+. .#######.
..##o##.. .#######. ..ooooo.. .+#####+. .#######.
..##o##.. .#######. ..#####.. .+#####+. .#######.
......... .#######. ..+++++.. .+++++++. .#######.
......... ......... ..+++++.. ......... .........
......... ......... .........
# prediction and ground truth, o ground truth only, + prediction only, . background
| Betti err. | BM err. | DIU | VOI ↓ | NSD ↑ | clDice ↑ | |
|---|---|---|---|---|---|---|
| (a) break | 1 | 1 | 1 | 0.56 | 1.00 | 0.86 |
| (b) hole | 1 | 1 | 1 | 0.20 | 0.98 | 1.00 |
| (c) shifted | 0 | 0 | 0 | 1.44 | 0.58 | 0.00 |
| (d) dilated | 0 | 0 | 0 | 1.22 | 0.88 | 1.00 |
| (e) dent | 0 | 0 | 0 | 0.20 | 1.00 | 1.00 |
The three topological metrics are 1 exactly for the two topological errors (a, b) and 0 otherwise. Each surrogate scores a topologically correct prediction worse than a topologically wrong one.
More detailed description of procedure and parameters
We thank the reviewer for the insightful comments and the helpful ideas on how to improve the paper further. We provide detailed answers below. Each raised topic is structured into three parts: a brief summary of the reviewer's feedback, a TL;DR reply, and a more detailed reply with evidence and references to the updated manuscript. We look forward to receiving further feedback on the explanations provided and the updated manuscript.
R SQck: The relation to Betti Matching and the method details are not clearly presented. Write out the sparseBM loss explicitly, including its combination with Dice+CE and the selected weight for each dataset.
TLDR. We added a corresponding description to the method section in the manuscript, in which we explain the loss computation procedure in detail. In addition to Figure 7 in the supplement, we added Table 5, listing the weights used for each dataset. The weights for all methods are as follows:
For all methods, we conducted hyperparameter searches to determine the optimal loss weight. Contrary to sparseBM (Figure 7), we did not find a rule to "predict" the best weights for the comparison methods. Especially fixed weights from the respective papers turned out to be suboptimal on many datasets.