- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work proposes a novel computation allocation paradigm: instead of speeding up each individual computation, it reduces the number of locations requiring costly computation, with CRC providing statistical guarantees for where computations can be skipped.
Critical strengths
It enjoys clear theory: admissible aggregation operators paired with CRC guarantee that the expected missed signal quality is bounded by α. It is highly generalizable, with tailored aggregation operators for segmentation and regression, and experiments spanning multiple scientific domains. It delivers substantial acceleration: FLOPs can be reduced by over 80×, with notable wall-clock speedups on both CPUs and GPUs.
Critical weaknesses
The paper emphasizes certified selection risk guarantees and uncertainty-aware sparse computation, yet what it actually guarantees is the amount of ground-truth signal contained within the discarded regions. Furthermore, the method has limited novelty, and it lacks fair comparisons against existing adaptive inference approaches.
Questions for authors
Beyond the aforementioned weaknesses, I note that the paper is missing citations and comparisons with several closely related approaches:
- Conformal Risk Training: End-to-End Optimization of Conformal Risk Control
- Enhancing image-conditional coverage in segmentation: adaptive thresholding via differentiable miscoverage loss
- Automatically adaptive conformal risk control
6 comments
Detailed comment (1/3)
W1 — What the guarantee covers. We thank the reviewer for pointing out this ambiguity. The reviewer’s interpretation is correct: CRC bounds the expected fraction of ground-truth signal contained in regions excluded from refinement by . Here, “signal” is defined through the cell aggregation operator , allowing flexibility in what is measured.
The formulation is general across admissible definitions of signal: any nonnegative aggregation satisfying nullity and occupancy-preservation can be used, with the same CRC framework controlling the expected fraction of the resulting total weight excluded from refinement. Our wording may have obscured this flexibility, and we will make it explicit. This is the selection risk referred to in the paper; it does not guarantee final prediction accuracy. We will clarify this definition in the abstract and introduction and revise the phrase “uncertainty-aware sparse computation” to describe the calibrated missed-signal guarantee more precisely.
W2 — Novelty.
We thank the reviewer for highlighting these closely related approaches. We agree that cascades [1] and using conformal methods to guide computational allocation are established ideas. We have expanded the related-work discussion to make these connections explicit. Conformal early exit [3, 4] calibrates when intermediate predictions can be accepted, while C3PO [2] optimizes query-level model cascades under probabilistic cost constraints. Spatial methods such as ROI cascades and PointRend already decide where to compute.
Our contribution addresses spatial allocation within dense prediction: a coarse prediction selects regions for expensive fine-resolution computation, and CRC controls the expected fraction of ground-truth signal excluded from refinement. Our admissible label aggregation operators provide a common bounded, monotone loss formulation for segmentation occupancy and regression mass. We combine conformalized candidate selection with sparse execution and evaluate the framework across scientific domains, modalities, and sparsity regimes. Thus, our contribution is the formulation and execution framework for certified spatial refinement, rather than the general idea of a conformal cascade.
The computational benefits also extend to training. On HECKTOR, training the full pipeline—including both the coarse model and refiner—requires 15$\times$ fewer FLOPs than training the Full baseline, with DSC decreasing from 0.736 to 0.723. Our additional ROI and top-k experiments further demonstrate that the selection-risk formulation applies beyond the original refinement configuration. We will revise the introduction to make these contributions explicit.
References
[1] Viola, Paul, and Michael Jones. "Rapid object detection using a boosted cascade of simple features." Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001. Vol. 1. IEEE, 2001.
[2] Valkanas et al. 3po: Optimized large language model cas- cades with probabilistic cost constraints for reasoning.” Advances in Neural Information Processing Systems 38 (2026): 84481-84522.
[3] Jazbec, Metod, et al. "Fast yet safe: Early-exiting with risk control." Advances in Neural Information Processing Systems 37 (2024): 129825-129854.
[4] Schuster, Tal, et al. "Confident adaptive language modeling." Advances in Neural Information Processing Systems 35 (2022): 17456-17472.
Rebuttal (2/3)
W3 — Additional comparisons.
We thank the reviewer for providing us with the opportunity to improve our work by providing additional baseline comparisons.
(i) Adaptive selection. We adapt the selection principles of DynamicViT [5] and PointRend [6]. The original implementations do not directly support our 3D volume segmentation pipelines, so we adapt these selection principles within a shared pipeline, holding the coarse model, refinement network, and halo fixed. Top-k budgets match CRC's mean selected fraction on calibration data, isolating the allocation strategy from changes in architecture or training.
- Score top-k (DynamicViT-inspired): Score top-k retains a fixed fraction of locations per sample, ranked by the coarse foreground / signal score. We use this as a proxy for DynamicViT, which learns token importance scores.
- Uncertainty top-k (PointRend-inspired): prioritize uncertain locations, using the smallest for segmentation and largest for regression. This adapts PointRend's uncertainty-prioritized refinement principle across tasks.
- CRC top-k: calibrate the keep fraction of the score-ranked top-k family directly against the missed-signal loss. Its selections are nested, so CRC applies. This arm targets risk rather than matching the original selector's budget.
Missed-signal risk on held-out data at (lower is better; the CRC top-k column gives the cells it needs):
| Task | CRC threshold: cells refined | CRC threshold | Score top-k (DynamicViT-inspired) | Uncertainty top-k (PointRend-inspired) | CRC top-k |
|---|---|---|---|---|---|
| MSD U-Net | 2.1% | 0.073 | 0.112 | 0.53 | 0.076 (2.5%) |
| MSD Swin-UNETR | 1.7% | 0.072 | 0.137 | 0.60 | 0.077 (2.1%) |
| MSD VISTA3D | 2.2% | 0.071 | 0.133 | 0.68 | 0.077 (2.1%) |
| HECKTOR | 0.52% | 0.086 | 0.061 | 0.062 | 0.086 (0.31%) |
| Galaxy | 2.2% | 0.040 | 0.038 | 0.78 | 0.094 (1.45%) |
| CASCADE | 8.5% | 0.046 | 0.26 | 0.25 | 0.065 (22.7%) |
MSD and HECKTOR use three seeds and 200 random half-splits of the original calibration scans, with calibration and evaluation separated within each split. These overlapping resplits assess sensitivity to calibration data; they are not 200 independent test datasets. Galaxy and CASCADE use held-out test sets. Budget matching is performed on calibration data.
Uncertainty-prioritized selection exceeds the target risk in five of six settings (0.25–0.78), except for HECKTOR. Uncertainty and signal coverage are different objectives. Score top-k improves on threshold CRC in HECKTOR and galaxy, but exceeds on MSD and CASCADE.
Our method combined with top-k achieves mean held-out risk below in every setting. On HECKTOR it uses 40% fewer cells than threshold CRC. The guarantee remains in expectation under exchangeability. Full results are in App. E.
Q1 — Related work. We thank the reviewer for highlighting these works. We will cite and discuss all three in Related Work and Sec. 8:
- Conformal Risk Training (Yeh et al., NeurIPS 2025) differentiates through CRC during training and extends control to Optimized Certainty-Equivalent risks, including tail risks and downstream decisions. Our omitted-signal loss offers an objective for reducing selected volume at a fixed risk target.
- COAT (Luo et al., ICLR 2026) learns image-specific thresholds through a differentiable miscoverage objective to improve image-conditional coverage. It naturally fits with our coarse model to improve input-adaptive spatial selection.
- AA-CRC (Blot et al., AISTATS 2025) develops approximate conditional risk control with data-adapted conditioning functions and can improve the selection threshold from the coarse model.
These methods complement our use of calibrated selection to determine where an expensive refiner executes. Integrating them requires preserving their training and calibration assumptions, and provide avenues for significant and exciting future work.
References
[5] Rao, Yongming, et al. "Dynamicvit: Efficient vision transformers with dynamic token sparsification." Advances in Neural Information Processing Systems 34 (2021): 13937-13949.
[6] Kirillov, Alexander, et al. "Pointrend: Image segmentation as rendering." 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2020.
Rebuttal (Summary)
We thank the reviewer for their thoughtful review and for recognizing the theory, generality, and speedups. We appreciate the opportunity to strengthen our work with clarifications and provide additional baselines. We address the concerns as follows:
We provide the detailed responses and results below for the reviewer's convenience.