- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a two-stage framework for sparsely supervised segmentation and regional quality estimation in volume electron microscopy. Stage I trains an initial segmenter, constructs a fixed three-dimensional consensus mask from its predictions, and converts that mask into a signed distance field (SDF). A separately initialized final segmenter receives sparse semantic supervision together with SDF regression and boundary-region consistency losses. Stage II predicts class-conditional local Dice from raw image regions and SDFs computed from the final segmentation. Experiments on BetaSeg and UroCell report improved segmentation and regional quality estimation.
Critical strengths
-
The problem and representation are well motivated. Sparse labels constrain boundaries poorly, and a continuous distance representation is a reasonable way to expose spatial structure.
-
The ablation study is well conducted. Table 3 separates the dense prediction head, validation protocol, and geometric objectives. Tables 10 and 13 examine teacher construction and compare SDFs with binary masks and class metadata.
-
The final performance of the model is good on benchmarks.
Critical weaknesses
-
The headline segmentation gain does not clearly reflect the contribution of the proposed geometric objectives. The reported 4.13 percentage-point improvement over ε-Seg on BetaSeg includes changes to both the output head and the checkpoint-selection protocol. Table 3 shows that the controlled baseline already achieves 90.07% Macro Dice, compared with 90.63% for the full method. Thus, most of the reported improvement comes from the dense head and representative-slice validation, whereas the methodological emphasis is on the two geometric losses.
-
The evidence for robustness and generalization is limited. Each dataset contains only one held-out evaluation volume, and the reported configurations use a fixed random seed. This makes it difficult to assess variability across training runs and generalization across volumes.
-
The benefit of SDF supervision over simpler alternatives remains unclear. What does SDF supervision add over directly training the final segmenter with the same consensus pseudo masks? A comparison using the same teacher predictions, consensus masks, backbonewould help isolate the value of the geometric representation.
-
The choice of a separate quality estimator needs further justification. Why is regional quality estimation trained independently after the final segmenter is frozen, rather than incorporated into a jointly trained or shared-backbone model? I recognize that the current segmenter and quality estimator operate on different input dimensionalities and that the quality targets depend on the segmentation predictions. Nevertheless, discussing these trade-offs would help explain the design choice and clarify whether modularity, optimization stability, or empirical performance motivates the separate training procedure.
Questions for authors
The training workflow should be described more explicitly. Although the framework is organized into two conceptual stages, it involves three separate model-training procedures: training the initial segmenter, training a separately initialized final segmenter, and training the regional quality estimator. Between the first two procedures, the initial predictions are processed offline to construct the consensus masks and teacher SDFs. Now, it is rather harder to get the whole training pipeline picture.
6 comments
Response to Reviewer nNfP (Part 2/4) — W3
Reviewer comment — W3
The benefit of SDF supervision over simpler alternatives remains unclear. What does SDF supervision add over directly training the final segmenter with the same consensus pseudo masks? A comparison using the same teacher predictions, consensus masks, backbone would help isolate the value of the geometric representation.
Response
We thank the reviewer for this suggestion. To compare SDF geometric supervision with direct supervision from consensus masks, we added comparisons under three configurations on both datasets. Within each configuration, the segmentation network and validation protocol are fixed. Mask and Full SDF use the same teacher predictions and consensus masks for direct Mask supervision and SDF geometric supervision, respectively.
| Dataset | Dense head | Rep.-val | No additional teacher supervision | Mask | Full SDF |
|---|---|---|---|---|---|
| BetaSeg | × | × | 86.50 | 87.33 | 87.69 |
| BetaSeg | ✓ | × | 88.88 | 89.82 | 90.05 |
| BetaSeg | ✓ | ✓ | 90.07 | 90.46 | 90.63 |
| UroCell | × | × | 56.11 | 54.95 | 60.47 |
| UroCell | ✓ | × | 69.69 | 66.89 | 70.87 |
| UroCell | ✓ | ✓ | 70.76 | 66.75 | 71.44 |
Results are Macro Dice (%) on the held-out test set, including the background class. × denotes the official center head or default validation protocol; ✓ denotes the dense head or representative-slice Dice validation (Rep.-val). Bold indicates the best result within each configuration.
Full SDF outperforms direct Mask supervision and the baseline without additional teacher supervision in all six comparisons shown. Relative to Mask, the gains are 0.17–0.36 percentage points on BetaSeg and 3.98–5.52 percentage points on UroCell.
Whereas binary masks encode class membership, SDFs additionally encode continuous distances inside and outside the target and the location of its boundary. These results support the additional value of converting the same consensus prediction into geometric supervision. We report these comparisons in Table 4 of the revised manuscript.
Response to Reviewer nNfP (Part 3/4) — W4
Reviewer comment — W4
The choice of a separate quality estimator needs further justification. Why is regional quality estimation trained independently after the final segmenter is frozen, rather than incorporated into a jointly trained or shared-backbone model? I recognize that the current segmenter and quality estimator operate on different input dimensionalities and that the quality targets depend on the segmentation predictions. Nevertheless, discussing the trade-offs would help explain the design choice and clarify whether modularity, optimization stability, or empirical performance motivates the separate training procedure.
Response
We thank the reviewer for their question about independent training. Stage II is designed as a regional quality-assessment module applied after segmentation. It computes local SDFs from the final prediction masks and combines them with raw images as estimator inputs. The module does not depend on internal segmentation features or a specific backbone, allowing independent training and adjustment without modifying the segmentation network.
Independent training also keeps the quality-supervision task fixed. Regional Dice depends on the segmentation prediction, so freezing the final segmenter keeps both inputs and quality targets fixed during estimator training. The two-dimensional segmenter and three-dimensional regional quality estimator can also use architectures suited to their respective tasks.
To assess this module under different annotation conditions, we compare the original quality-supervision setting with extremely sparse supervision that reuses existing Stage I manually annotated points. In the extremely sparse quality-supervision setting, regional Dice is computed only at these points for estimator training and validation-based model selection. Both settings are evaluated against test-region Dice computed using the original evaluation protocol.
| Dataset | Quality supervision | Input | Macro MAE ↓ | Macro ρ ↑ | AUPRC <0.90 ↑ | AUPRC <0.80 ↑ |
|---|---|---|---|---|---|---|
| BetaSeg | Original | Raw | 0.0308 | 0.5917 | 0.8894 | 0.3604 |
| BetaSeg | Original | Raw + local SDF | 0.0243 | 0.8000 | 0.9617 | 0.6814 |
| BetaSeg | Extremely sparse | Raw | 0.0551 | 0.3638 | 0.9097 | 0.1296 |
| BetaSeg | Extremely sparse | Raw + local SDF | 0.0567 | 0.6707 | 0.9458 | 0.2127 |
| UroCell | Original | Raw | 0.2276 | 0.3125 | 0.8939 | 0.8342 |
| UroCell | Original | Raw + local SDF | 0.1477 | 0.7421 | 0.9745 | 0.9487 |
| UroCell | Extremely sparse | Raw | 0.3262 | 0.1475 | 0.8864 | 0.8223 |
| UroCell | Extremely sparse | Raw + local SDF | 0.2639 | 0.4085 | 0.9011 | 0.8828 |
The original-setting results are from the paper; the extremely sparse setting is a new experiment. Both settings use the same 597 test regions on BetaSeg and 600 on UroCell. Bold indicates the better result within each dataset and supervision setting.
Local SDFs improve quality ranking and low-quality-region detection on both datasets under both the original and extremely sparse supervision settings. These results show that the independent quality estimator can reuse existing Stage I sparse annotations for training and use prediction-derived geometry for regional quality assessment.
The trade-off is an additional quality-estimation network and training step. We choose independent training primarily for modularity, fixed quality targets, and task-specific architectures. Section 3.2 explains the separate training design, and Appendix C.2 reports the supervision comparison (Table 19).
Response to Reviewer nNfP (Part 1/4) — W1/W2
Reviewer comment — W1
Reviewer comment — W2
Response
We thank the reviewer for their comments on the contribution of geometric supervision and experimental stability. In earlier experiments with different random seeds, we observed small performance variations. The overall improvement over the official ϵ-Seg implementation reflects the combined effects of the dense semantic prediction head, validation protocol, and geometric supervision. To isolate the contribution of geometric supervision, we added matched comparisons under three configurations on BetaSeg and UroCell. Within each configuration, we keep the prediction head and validation protocol fixed and compare no additional teacher supervision with Full SDF.
Results are Macro Dice (%) on the held-out test set, including the background class. × denotes the official center head or default validation protocol; ✓ denotes the dense head or representative-slice Dice validation (Rep.-val). Gains are in percentage points.
We also report progressive ablations on both datasets, showing the changes after successively adding geometric supervision, the dense semantic prediction head, and representative-slice validation.
Full SDF improves performance in all six matched comparisons. With the official center head and default model selection, the final segmenter gains 1.19 and 4.36 percentage points on BetaSeg and UroCell, respectively. With the dense head and representative-slice validation, the gains are 0.56 and 0.68 percentage points. These results support an additional contribution from geometric supervision across the evaluated configurations. In the revised manuscript, Section 4.3 distinguishes the overall improvement from the incremental contribution of geometric supervision. Table 4 reports the matched comparisons, and Appendix B.2 reports the progressive ablation (Table 13).
BetaSeg follows the training/test split used in prior work, whereas UroCell uses our fixed split. Each dataset has one held-out test volume, and all compared methods use the same split to ensure a fair comparison.