- Rating
- 2
- Confidence
- 1(−2)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies conformal prediction sets for high-resolution image reconstruction under covariate shift. The method uses nested spatial averages for calibration and computes exact upper and lower bounds of regional averages through a linear-time tree algorithm. The paper also analyzes the relationship between spatial resolution and image-space radius and discusses extensions to more general types of shifts. Experiments on several medical image datasets and a masked autoencoder show that the method can achieve about 0.90 intra-group coverage; compared to pixel-level constraints, using region-level constraints can significantly reduce the prediction set radius.
Critical strengths
- The method is designed for precise range computation under nested spatial constraints, and it is simple and computationally efficient.
- It theoretically establishes a connection between spatial resolution, image regularity, and achievable predictive radius, which is useful for understanding when coarse-grained constraints can be applied
Critical weaknesses
- The method is relatively low in novelty. Group calibration and finite-dimensional covariate-shift calibration are quite similar to existing work, so the paper's main innovation seems to focus on the construction of spatial ranges. The paper needs to more clearly state the substantive differences compared to existing methods.
- The experiments do not sufficiently isolate the benefits brought by "exact range computation" itself. In the experiments, the widths of the exact range and separate intervals are very close, and the larger radius decrease mainly comes from replacing pixel-level constraints with 2×2 region constraints. Additionally, the comparison with related conditional conformal methods is not yet very convincing.
Questions for authors
- Under matched coverage and computational budgets, how does this method perform compared to the strongest existing finite-sample conditional/covariate-shift calibration methods?
- Can the benefits of exact range computation be evaluated separately, without mixing them with the gains from using coarser spatial constraints?
- Can experiments under truly continuous covariate shifts be added, instead of mainly focusing on group proportion shifts?
3 comments
Response to Reviewer rWXn (2/2)
Continuing our response, we address continuous within-group shifts (Appendix S22, pp. S103–S104) and differences from existing calibration methods (Section 3, p. 4).
Q3. Continuous covariate shifts
Thank you very much for this insightful question. In Appendix S22 (p. S103), we have added a continuous image population with known within-group density ratios and unchanged conditional fine-image distributions. Independently observed coarse averages are uniform on , with observed . For and , the target ratio is , where for nonzero and . Fine images have continuous random variation with exact coarse averages and one conditional distribution given .
In each of 100 independent replications, we use 400 calibration images and 200 queries per tilt, sharing five fixed predictors, the maximum regional score, nominal level 0.90, and randomization. For the group-specific conditional span, we include all ten functions , and weighted calibration uses the same groups with query mass at infinity. At , group calibration covers 0.5882 (radius 0.04744), randomized conditional calibration covers 0.8980 (Monte Carlo standard error 0.0035, radius 0.05219), and oracle and maximum weighted calibration both cover 0.9136 (radius 0.05400). Randomized conditional coverage is 0.8959 and 0.9001 in the two groups (please see Table 41, p. S104, and Table 42, p. S104).
The ten-feature construction protects each group under each declared tilt, averaging over calibration, target draws, and randomization, whereas a six-feature construction protects pooled tilts and source-group averages. Declared tilts and their fixed convex mixtures specify each guarantee. Across all 100,000 queries per panel, we obtain finite nonnegative conditional thresholds. Computing conditional thresholds takes 1.916 seconds (group-specific) and 3.033 seconds (pooled) per 1,000 queries, versus less than 0.001 seconds for each sorting-based rule. Here, exact ranges equal separate score intervals, which isolates calibration under shift.
We also retain the fixed-cohort brightness study (Appendix S14) and independent level-selection comparison (Appendix S16). On official medical splits, the smallest group coverage is 0.848–0.876 in four cohorts. Estimated-ratio adjustments improve these minima, though four cohorts remain below 0.90. In Proposition 3, we retain unmeasured density-ratio and conditional-image errors in the target-coverage bound.
W1. Differences from existing methods
The paper needs to more clearly state the substantive differences compared to existing methods.
We appreciate your bringing this to our attention, and we apologize for the lack of clarity in our previous manuscript. We differ from existing calibration methods in post-calibration geometry: exact feasible ranges, the minimax accepted center, and resolution-dependent radius bounds. In Algorithm 1 (p. 4), we use Mondrian calibration of Vovk et al. (2005), and Theorem 1 (p. 4) restates group-score coverage. We follow Gibbs et al. (2025) for our finite-dimensional target and conditional LP without claiming calibration novelty (Appendix S6.1, p. S28, and Appendix S14, p. S90).
- Exact ranges. Separate score intervals omit observed coarse block averages. On a nonempty accepted set, Proposition 2 (p. 6) gives exact one-level bounds and a radius equal to half the widest range, which we extend to nested partition trees in two linear-cost passes. These are feasible extrema over one common set, and we do not claim meaningful empirical width reduction (Theorem S1, p. S5).
- Accepted reconstruction. We minimize worst-case regional error over accepted centers. For four subregions per coarse block, the optimal certificate is at most times the unrestricted radius (Proposition S1, p. S3). On nonempty tree-constrained sets, the projection comparison holds with sharp factor 2, though it does not imply smaller realized errors (Proposition S37, p. S88).
- Resolution. For fixed , the regularity class satisfies at each dyadic level, where replaces pixels by block averages and . For , we bound the approximation remainder by using summaries, and the calibrated threshold stays in the radius bound. For unrestricted images, if the accepted set defined by linear measurements contains an image in with , its normalized radius is at least (Theorem 3, p. 6).
Thank you again for your careful reading and constructive comments.
Thank you for the constructive responses and discussion. I should clarify that I have very limited familiarity with the literature in this area, and I am concerned that this assignment may not be well aligned with my expertise. Acquiring sufficient background to provide a well-informed assessment requires substantial time, and I therefore acknowledge the limitations of my review. After revisiting the paper and considering the other reviewers’ comments, I remain inclined toward a weak reject. My concerns primarily relate to the writing, figure presentation, and overall organization, as well as the nearly 100 pages of supplementary material, whose necessity and relevance to the main claims are not sufficiently clear to me. These issues detract from the clarity and quality of the presentation. Nevertheless, given my limited expertise in this area, I will continue to consider the substantive arguments raised by other reviewers and remain open to increasing my score.
Response to Reviewer rWXn (1/2)
We thank Reviewer rWXn very much for the thoughtful comments. This first part addresses the calibration comparison and the separate effects of exact ranges and center choice in Appendix S16 (p. S98) and Appendix S13 (p. S88).
Q1. Coverage and computation
Following your suggestion, we have added independent level selection near 0.90 empirical coverage and report actual computation costs. For each method, score construction, and tilt θ∈−4,4, we select a level from 0.860,0.865,…,0.940 on 100 tuning replications and evaluate 100 new replications. Each uses 400 calibration observations and 200 queries per tilt, sharing calibration draws, queries, and jitter across methods. Achieved mean coverage spans 0.89715 to 0.90450, and conditional linear-program (LP) radii are 0.9–1.7% larger than group-calibration radii (Appendix S16 and Table 35, p. S98).
Times cover threshold calculation and set geometry for 400 queries. In the common-nominal-level study, our 300-second single-thread cap per method and replication was an equal budget ceiling, not equal elapsed time, and did not bind: the longest run took 4.41 seconds and none failed. At near-matched empirical coverages, group calibration is about 62–74 times faster than the conditional LP with smaller radii. Because their coverage guarantees protect different shift classes, this comparison does not establish universal superiority (Appendix S14, p. S90, and Appendix S16 and Table 35, p. S98).
Q2. Isolating exact ranges
We appreciate this comment and apologize for the earlier lack of clarity. Yes, with the predictor, observation, functional class, and calibrated set fixed, exact leaf ranges reduce the mean maximum regional half-width over separate score intervals by only 0.056% and 0.053%, while the larger 20–44% reduction changes the calibrated class. The fixed-set calculation still gives exact feasible ranges and supports the optimal accepted certificate. Please see Table 32 and its following discussion (p. S90).
We decompose these effects on the same five-U-Net predictions, λ=0.0025, and 32×32 observations over 100 controlled-shift and 100 official-split replications with 400 queries each.