acceptodds
Under review as a conference paper at ICLR 2027

Conformal Selective Generation: Regional Delivery with Proxy Risk Control for Image Inpainting

Abstract

Image inpainting produces plausible completions, yet deployment requires deciding which generated regions to show. We introduce Conformal Selective Generation (CSG), a framework for regional selective release of a completion designated before scoring. CSG separates the deliverable from auxiliary samples used to estimate uncertainty. It applies a fixed regional score and calibrates a RELEASE/ABSTAIN threshold through conformal risk control across sources. A source loss with a fixed denominator measures released proxy errors and preserves dependence among regions and repeated instances from the same source. Under a frozen protocol and source exchangeability, calibration controls marginal expected proxy risk for any score fixed during development that induces nested release sets. Our primary score measures DINOv2 disagreement between the designated output and the auxiliary centroid. We also evaluate DINO, CLIP, LPIPS, and scores adapted to regional release under the same protocol. On a benchmark of natural images, the primary instantiation achieves 65.3% mean region coverage at mean proxy risk 0.098 across 200 paired source re-splits. Alternative scores yield different risk–coverage tradeoffs. Experiments across four restorers, auxiliary bank sizes, human judgments, perturbations, and mask geometries characterize utility and applicability limits. CSG provides a protocol for partial delivery under an explicit proxy risk budget, with the regional score as a separate modeling choice.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
image inpainting, selective generation, conformal risk control, uncertainty estimation, regional abstention

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fXtBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Conformal Selective Generation (CSG) for image inpainting. Instead of showing all generated content, CSG decides which regions to release and which to withhold. It fixes one completion for delivery and uses additional generated samples to estimate regional uncertainty. The main score measures DINOv2 feature disagreement between the designated completion and the auxiliary samples. Conformal risk control is then used to select a release threshold. The method aims to control the expected proxy error risk at the source level, where proxy errors are defined using CLIP distance from a reference image.

The authors evaluate CSG on Places365 using four image inpainting methods. They compare different scoring methods and conduct experiments on human judgments, perturbations, and mask geometries. The main method achieves 65.3% region coverage at a mean proxy risk of 0.098. The experiments also reveal limitations under common-mode errors and challenging region geometries.

Critical strengths

1. Clear Problem Formulation and Risk Control

The paper addresses a meaningful problem: deciding which generated regions can be safely shown to users. The proposed framework provides a clear formulation of regional release decisions and uses conformal risk control to provide a statistical guarantee under stated assumptions.

2. Comprehensive Experimental Evaluation

The experiments are extensive and well designed. The authors evaluate multiple image restoration models, scoring methods, and settings, including human evaluations and distribution shifts. They also report important failure cases, such as common-mode errors and challenging region geometries. These analyses help clarify both the strengths and limitations of the proposed method.

Critical weaknesses

1. Limited Novelty

The framework mainly combines regional selection with existing conformal risk control. Although the problem is meaningful, the paper does not clearly show what new methodological or practical advantages it offers over related conformal masking methods, such as Kutiel et al. and Adame et al. This makes the novelty of the contribution less convincing.

2. Practical Reliability of the Risk Guarantee

The theoretical guarantee only applies to CLIP-defined proxy errors, rather than actual errors among released regions. The results are also sensitive to the choice of proxy error threshold and show clear limitations under common-mode errors. These issues raise concerns about how well the proposed framework can ensure reliable image inpainting in practice.

Questions for authors

1. Novelty and Contribution

What are the main advantages of CSG over existing conformal masking methods, such as Kutiel et al. and Adame et al.? Which experimental results best demonstrate these advantages?

2. Proxy Error Definition

Why is CLIP distance chosen to define proxy errors? What evidence supports its use as a measure of actual reconstruction errors?

3. Sensitivity to τ\tau

The results are sensitive to the choice of τ\tau. Why is the 75th percentile selected for the main experiments? How should this threshold be chosen in real applications?

4. Common-mode Errors

The experiments show that common-mode errors can significantly affect risk control. How could CSG handle such errors when new reference-labeled calibration data is unavailable?

5. Candidate Region Construction

What motivates the use of SLIC and the specific parameters (128 regions and ρ=0.05\rho=0.05)? How do these choices relate to the intended applications?

6. Designated Completion

How exactly is the designated completion selected for each instance? Could the authors clarify the seed/index selection procedure to support reproducibility?

7. Writing and Clarity (Suggestion)

The experiments are extensive, but the presentation could be improved. Some important design choices and definitions, such as candidate region construction, the reference-aware oracle, and appearance + structure fusion, should be explained more clearly in the main text.

Reviewer_CgksOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Conformal Selective Generation (CSG), a method for deciding which inpainted regions to show. One completion is fixed before scoring. K auxiliary samples then give each superpixel region a DINOv2 centroid-disagreement score. Conformal risk control (CRC) over source groups selects a RELEASE/ABSTAIN threshold that bounds the expected error burden measured by a CLIP proxy. On 1,000 Places365 instances, the primary score reaches 65.3% region coverage at proxy risk 0.098.

Critical strengths

1.It reports split variability, failure modes and limitations with unusual openness.

Critical weaknesses
  1. The novelty could be stated more sharply. As the authors acknowledge, Proposition 1 applies CRC directly, and grouping by source is standard cluster-level exchangeability. Kutiel et al. and Adame et al. already calibrate spatial masks. It would help to explain what discrete regions and a predesignated output make possible that continuous masks do not, and to back this with an empirical comparison.

  2. The proxy needs more support. The guarantee applies to a CLIP proxy whose threshold is a development quantile, yet held-out AUPRC against human labels is only 0.113. Releasing every region already gives a human risk of 0.066, below α, so the human study cannot yet test risk control. CLIP scores outperform DINO scores (Table 28a), which suggests the proxy may partly reward similarity to its own representation. A second, independent proxy or a larger human calibration set would make this much stronger.

  3. The primary score's advantage is not consistent.

    • DINO pair and DINO spread match or beat the centroid score.
    • The gain over LPIPS disappears on DPS.
    • The matched-quota human comparisons are not significant (Table 33).

    Explaining why the centroid score was chosen as primary, for example its response to changes in the designated output (Table 4), would help readers.

  4. Some representative baselines are missing. Natural comparisons include:

    • the official versions of Kutiel et al. and Adame et al.;
    • the image-to-image RCPS framework [1];
    • pixel-wise diffusion uncertainty such as BayesDiff [2].

    The restorer set would also be more convincing with widely used inpainters such as LaMa [3] and RePaint [4], as well as more recent diffusion inpainting pipelines. Inference-time guidance methods such as FreeInpaint [5] could also serve as an informative stress test. Because they steer latents toward prompt-alignment and visual-quality objectives, they may reduce diversity among the K auxiliary samples. That would weaken the signal available to disagreement-based scores.

  5. Scope and presentation.

    • The experiments use 256×256 images and masks below 15%.
    • Tiny and long thin regions remain error-prone.
    • Under common-mode shifts, 30 of 36 conditions exceed the budget.
    • Several passages read like revision logs, and the ethics statement has no formal approval record. Tidying these would improve the manuscript considerably.

[1] Angelopoulos et al. Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging. ICML 2022. [2] Kou et al. BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference. ICLR 2024. [3] Suvorov et al. Resolution-robust Large Mask Inpainting with Fourier Convolutions. WACV 2022. [4] Lugmayr et al. RePaint: Inpainting using Denoising Diffusion Probabilistic Models. CVPR 2022. [5] Gong et al. FreeInpaint: Tuning-free Prompt Alignment and Visual Rationality Enhancement in Image Inpainting. AAAI 2026.

Questions for authors

See weaknesses.

Reviewer_xYrTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Conformal Selective Generation (CSG), a framework for selectively releasing regions of an image inpainting output. It separates the completion designated for release from auxiliary completions used for scoring, aggregates regional proxy losses at the source level, and applies Conformal Risk Control to select a release threshold. The resulting guarantee controls marginal expected proxy risk under source exchangeability.

Critical strengths

Flexible protocol with clearly defined output roles and calibration units. CSG separates the designated completion from auxiliary samples and aggregates losses by source, accommodating dependence among regions and instances from the same source. Different scoring functions can be evaluated under the same loss definition and calibration rule.

Critical weaknesses

Insufficient evidence of improved quality in the released inpainting results. The evaluation mainly focuses on proxy risk and coverage, which do not directly establish that users receive better inpainting results. For example, EAMamba has a proxy risk of 0.097 but a conditional error of 60.3% at only 17.0% coverage. Although the human evaluation provides useful evidence, clearer comparisons of released-region quality are needed to demonstrate the practical benefit.

Insufficient experimental analysis of computational overhead. The default setting generates one designated completion and eight auxiliary completions per input, with additional preparation required for calibration. Although separate timing measurements are reported, an end-to-end comparison against standard single-completion inpainting is missing. Reporting total runtime and the quality–cost trade-off across auxiliary sample counts would help establish whether the benefits justify the additional computation.

Questions for authors
  1. [W1] Can the authors more directly demonstrate improvements in released-region quality over alternative selection methods at matched coverage, while accounting for information withheld through abstention?
  2. [W2] What is the end-to-end overhead compared with single-completion inpainting, and how does reducing the number of auxiliary samples affect released-region quality, coverage, and runtime?
  3. Rather than predesignating one completion for release, could the method select a higher-quality completion from the nine generated candidates and then apply selective release? Would this improve quality or coverage under the same generation budget, and what changes to scoring and calibration would be required to preserve the risk guarantee?
  4. The main quantitative evaluation focuses on Places365. Could the authors provide results on a dataset with different image content, such as faces or objects, to assess whether the quality and coverage benefits of selective release generalize beyond scene images?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.