- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RC‑Mamba targets multi‑source remote‑sensing classification. Built upon the MSFMamba backbone, it establishes a complete “risk measurement‑learning‑handling” pipeline via G‑BRT group‑conditional Bayesian risk thresholding, TCMR threshold‑coupled margin regularization, and RASR selective refinement. It achieves balanced and controllable prediction risk and reliability across different groups while boosting classification accuracy.
Critical strengths
This work builds a full end‑to‑end pipeline covering group‑wise risk measurement, model training constraints, and residual‑risk handling at inference time. Existing conditional conformal and conformal risk‑control methods mostly perform post‑hoc calibration after training and can only adjust thresholds for a fixed pre‑trained model. In contrast, RC‑Mamba derives group‑aware risk standards from G‑BRT, converts group‑level risk violations into differentiable loss signals through TCMR and feeds them back into the Mamba backbone. Consequently, group‑wise reliability constraints are directly embedded into representation learning. Finally, RASR applies selective test‑time augmentation exclusively to high‑risk samples, avoiding the heavy computational overhead incurred by full‑dataset TTA.
Critical weaknesses
Limited theoretical guarantees of G‑BRT inherited from CPBQ Bayesian‑quadrature G‑BRT constructs risk upper bounds using Dirichlet sampling following CPBQ. CPBQ only delivers conservative upper bounds under the Bayesian posterior perspective rather than strict frequentist finite‑sample guarantees offered by conventional conformal prediction. Many statements in the manuscript risk conflating these two paradigms and give the impression that “strict group‑wise risk control” is attained. Nevertheless, G‑BRT provides no rigorous frequentist group‑risk guarantees and merely yields Bayesian conservative estimates. The boundary of its theoretical guarantees is insufficiently clarified.
Inadequate reporting of statistical significance Though 3 random seeds are adopted in experiments, only mean values are presented in tables without standard deviations. It cannot be verified whether the OA improvements of 0.08‑0.84 points brought by RC‑Mamba over MSFMamba are statistically significant. Minor gains on some datasets may plausibly stem from random fluctuation.
Insufficient ablation studies Current ablations only examine progressive module stacking. Several critical ablations are missing: ① Ablation on group tile size: how tile dimensions affect overall accuracy (OA) and group‑wise coverage; ② Sweep ablation for TCMR hyper‑parameters (,$\gamma$); ③ Ablation for the EMA smoothing coefficient () to verify the necessity of smoothing; ④ Analysis of how reference cache‑set size influences G‑BRT threshold estimation.
Incomplete baselines for reliability evaluation Only post‑hoc baselines including Split‑CP, CRC and RCPS are compared. Training‑aware conformal‑learning approaches (e.g., Learning Optimal Conformal Classifiers) are absent, as are major baselines from conditional conformal prediction (Conditional CP).
Questions for authors
- Please clearly elaborate the distinction between the Bayesian posterior upper bound of G‑BRT and frequentist finite‑sample group‑risk guarantees. Are there theoretical scenarios in which the predefined group‑risk target () can be violated even under recommended hyper‑parameter settings of (,)? Please explain. Should relevant statements across the manuscript be revised to prevent over‑claiming G‑BRT’s theoretical guarantees?
- Please supplement standard deviations across multiple random seeds for all metrics to justify the statistical significance of gains in both classification accuracy and reliability. Explicitly mark cases where performance improvements are statistically insignificant for certain datasets.
- Ablation regarding group tile size is missing. Tile size is a manually configured critical grouping hyper‑parameter; please analyze its impacts on OA and group‑level coverage. Hyper‑parameter sweep ablations for TCMR are required, since these hyper‑parameters directly govern regularization strength and training dynamics; please report sweep results and justify your hyper‑parameter choices. An ablation for EMA coefficient should be conducted to validate whether EMA smoothing is indispensable, by comparing threshold estimation quality and final reliability metrics with and without EMA. Please also provide an ablation for the G‑BRT reference cache‑set size and analyze how sample volume in the cache affects threshold estimation and downstream model performance.
- Why are training‑aware conformal methods and conditional conformal prediction not included in reliability comparisons? Please add experimental results for these baselines and demonstrate the advantages and necessity of the proposed joint‑training pipeline (G‑BRT + TCMR) against existing training‑aware / conditional conformal alternatives.