acceptodds
Desk rejected at ICLR 2027

Energy-Weighted Conformal Classification: Efficiency Failure Modes and Cost-Bounded Adaptation

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Energy weighting adapts conformal prediction sets using information beyond class probabilities, yet coverage alone leaves set size uncontrolled. We show that exact joint density can be dominated by label-independent nuisance features, while nearly constant weights can overwhelm narrow temperature-scaled score gaps; both effects survive recalibration. To protect efficiency, we introduce pooled cost projection (PCP), which measures score displacement in labels per input, clips it, and symmetrically recalibrates for each query. Under exchangeability, PCP preserves marginal coverage and bounds expected additions to the same separately calibrated base by a chosen budget B. A half-label budget repairs a nonempty construction from a raw-target lower bound of 47.583 labels to at most 2.5 + ε. On an energy-fine-tuned WRN and two ImageNetV2 target architectures, PCP reduces mean set size by 5.36–8.68% relative to adaptive linear shrinkage at the same formal budget. Label-count protection thus provides a principled interface between informative scores and efficient calibrated sets.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
conformal prediction, energy weighting, prediction set efficiency, uncertainty quantification, cost-bounded adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_P23dOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how energy weighting affects the size of conformal classification sets after recalibration. Two constructions show efficiency failures while maintaining coverage. Independent Gaussian nuisance features can alter energy weights without changing the label posterior. Almost constant weights can greatly enlarge APS sets when the scoring temperature is sufficiently small.

The proposed method, pooled cost projection (PCP), pools calibration inputs with one query and counts candidate labels under base and target scores. It clips their count difference, uses the base score to break ties, and recalibrates the transformed keys for that query. The transformation uses no labels and treats all pooled inputs symmetrically.

Under conditional exchangeability, with fitting and selection choices fixed using external data, PCP preserves marginal coverage and bounds expected added labels relative to the separately calibrated base. The expectation averages calibration and query randomness. Adaptive linear shrinkage receives the same formal guarantee.

Experiments examine constructed failures, an energy fine-tuned CIFAR-100 model, and transfers between classifiers on ImageNetV2. The authors report smaller mean sets than protected linear controls in the main settings, with additional analyses of budgets, faults, and computation.

Critical strengths
  1. Theorem 3 gives a useful finite-sample bound on expected added labels relative to the same separately calibrated base. The proof controls movement of both candidate scores and the recalibrated threshold, and handles ties explicitly. Including recalibration in the certificate is an important part of the result. The label-free, symmetric transformation also gives a clear coverage argument under conditional exchangeability, with fitting and selection completed on external data.

  2. The two constructions explain distinct failure mechanisms: nuisance features can dominate density without changing the label posterior, and small weight changes can overwhelm narrow APS score gaps. The appendices make the temperature and calibration restrictions explicit. The high-precision checks show that the second construction persists when very small score gaps are retained. The finite-calibration example gives a concrete application of the addition bound. The linear control with the same formal addition budget, fixed-direction ablations, and separate counts of additions and removals help interpret the empirical gains.

Critical weaknesses
  1. The evidence for efficiency gains remains specific to the chosen base and settings. In the exploratory LAC-base experiments in Table 12, PCP does not improve mean set size over the base (11.678 versus 11.485 for WRN, and 1.626 versus 1.596 for JEM). At 99% on ResNet, the comparison with linear shrinkage has mixed signs across the three splits (Appendix J). These observations do not establish a significant deterioration, but they leave limited evidence for gains from a competitive base or across operating points. The repeated splits and additional partitions reuse images and checkpoints, so they mainly measure partition sensitivity rather than independent replications across models or data pools. The paper acknowledges these limits and treats the reported standard deviations descriptively. The remaining concern is the breadth of the empirical support, not the expected-addition guarantee.

  2. Practical cost and experimental reproducibility need more evidence. The stated direct implementation costs O(NKlog⁡(NK))O(NK\log(NK)) time and O(NK)O(NK) memory per query. With 2,000 calibration inputs and 1,000 classes, the pool contains 2,001,000 candidates for each score. Calibration outputs can be cached, but the supplied timing panel covers 100-class constructed examples and does not establish per-query latency or peak memory for the exact procedure on ImageNetV2. I could not find an implementation or replay archive on the submission page, while the reproducibility statement promises a future release. This leaves the reported score-order checks and mask replays difficult to verify independently. An anonymous implementation with frozen configurations and replay files, together with realistic timing and memory measurements, would make the experimental evidence easier to assess.

Questions for authors
  1. Can auxiliary data identify when PCP improves on an already competitive base, given the LAC-base results and mixed 99% ResNet comparisons? Would independent checkpoints or image pools support the observed gains beyond partition sensitivity?

  2. Please report exact per-query conformal-update latency and peak memory for ImageNetV2, using cached calibration scores and separating this cost from model inference.

  3. Please provide an anonymous archive with the implementation, frozen configurations, score-order checks, and files needed to reproduce the reported mask replays.

1 comment
Reviewer_P23dOpenReview ↗

Thanks for your answer

Thanks for sharing the ImageNetV2 timing and memory measurements, and for making it easy to reproduce the results. This really helps with my concerns about how it was implemented and whether it can be repeated. The fact that you did multiple training runs on your own backs up the idea of SAPS, and having a fallback option like LAC is really useful. I do not raise my rating though, because how well it works depends on the specific setup, especially when comparing to 99% on ImageNetV2. Can you please add the new results and the link to the revised paper?

Reviewer_jn18OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies efficiency failures in energy-weighted conformal classification and proposes pooled cost projection (PCP). PCP maps base and target scores into pooled label-count coordinates, clips their displacement, and performs query-symmetric recalibration. Under exchangeability, the method preserves marginal coverage and bounds expected label additions relative to the same separately calibrated base set. This is an expectation guarantee over calibration and query randomness, rather than a per-example cost bound.

Critical strengths

1.The paper clearly distinguishes conformal validity from prediction-set efficiency and provides concrete counterexamples showing that recalibration alone does not prevent severe efficiency failures.

2.The finite-sample analysis is technically careful, accounting for threshold movement, ties, and the symmetry needed for conformal validity. The expected-addition guarantee relative to the same calibrated base is a meaningful contribution.

3.The experiments include several direction, score-family, and mechanism controls. The appendix also transparently reports unfavorable operating points and distinguishes formal guarantees from realized costs.

Critical weaknesses

The practical value of the protection mechanism is not sufficiently established. The counterexamples demonstrate that severe efficiency failures are possible, but the real-data results provide limited guidance on when protection is worth its efficiency and computational costs. The existing deletion-window analysis is informative, but it does not yet establish an observable criterion for predicting when PCP will provide a useful cost–efficiency trade-off.

The empirical advantage of the particular clipping rule remains insufficiently characterized. Comparing PCP and adaptive linear shrinkage at the same formal budget BB is valid because both receive the same theoretical protection. However, their realized additions can differ substantially, so the results do not establish whether PCP remains preferable at comparable actual addition costs. Moreover, within the broader bounded-adjustment class covered by Theorem 3, the main comparison remains clipping versus global linear shrinkage. A simple bounded nonlinear alternative would help determine whether the observed advantage is specific to clipping or reflects greater flexibility than global scaling.

Questions for authors

1.If configurations are selected using only independent auxiliary data before final calibration and testing, does PCP remain preferable to adaptive linear shrinkage at comparable realized addition costs? A set-size–addition trade-off curve would help clarify this.

2.Under the same base, target, data roles, and theoretical protection, would another bounded nonlinear adjustment achieve similar improvements?

Reviewer_yvU7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Pooled Cost Projection (PCP), a method to make energy-weighted conformal classification more efficient by controlling prediction set sizes. The authors use a label-count displacement clipping and symmetric reallocation mechanism governed by a chosen budget, which ensures that marginal coverage is preserved and bounds the expected number of added labels with a statistical guarantee, thereby saving computational resources while providing provable control over the impact of nuisance features. Experiments on an energy-tuned WRN and two ImageNetV2 architectures demonstrate the effectiveness of the proposed method, achieving a 5.36–8.68% reduction in mean set size.

Critical strengths
  1. The paper is well written and easy to follow.
  2. The experimental settings are clearly explained.
  3. The effectiveness of the proposed method is validated both theoretically and empirically.
Critical weaknesses
  1. The empirical evaluation is somewhat limited in scope, focusing exclusively on image classification tasks.
  2. Additional baselines, such as [1] and [2], should be considered.
  3. Conditional coverage should be analyzed more thoroughly.
  4. The code is currently unavailable, which hinders reproducibility.

[1] Length Optimization in Conformal Prediction
[2] An Information Theoretic Perspective on Conformal Prediction

Questions for authors
  1. More references should be included in the introduction.
  2. Please better explain the necessity of energy weighting as motivation. Without energy weighting, would the theoretical and empirical results remain at the same level?
  3. Can the theorems and experiments be extended to additional data structures, such as graphs or text, and to additional tasks, such as LLM generation and regression?
  4. What are the differences between this work and existing work? References [1] and [2] also discuss the relationship between set size and temperature scaling.
  5. How is the case of violated exchangeability handled? It would be better to provide theoretical and empirical results for this case.

[1] The Entropic Overhead of Conformal Correction
[2] Does confidence calibration improve conformal prediction?

Comments on the paper

1 comment
Reviewer_jn18OpenReview ↗

Response to Author

Thank author for the detailed response and additional experiments. The comparison addresses all my question and make me understand the paper more comprehensively. I would like to raise my rate to 3 weak accept.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.