acceptodds
Under review as a conference paper at ICLR 2027

Conformal Prediction Sets Quantify Information Gain: A Theoretical Perspective

Abstract

Conformal prediction is a popular tool for uncertainty quantification that outputs prediction sets with finite-sample coverage guarantees. While prediction set size is commonly used as a heuristic measure of uncertainty, the information-theoretic basis for this interpretation remains poorly understood. In this work, we provide such a foundation using a decision-theoretic generalization of entropy tailored to set-valued prediction. In particular, we introduce a family of generalized information measures based on the size and coverage of conformal prediction sets. Notably, Shannon mutual information admits an exact integral representation in terms of these measures. We then show that, in standard classification settings, the reduction in conformal set size from additional information (i) is sandwiched between calibration-dependent members of this family and (ii) obeys a data processing inequality, both up to finite-sample calibration and model error terms. Together, our results formally relate conformal prediction to classical information-theoretic quantities and justify using set-size reduction as an information gain metric. Empirically, we validate our theory across 11 classification settings and show that set-size reduction and Shannon mutual information can rank features differently in a greedy feature selection experiment.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
conformal prediction, information theory, uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_BLFeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work expands upon the information theoretic perspectives of the set size in conformal prediction through the notions of generalized entropy and information. More specifically, the authors use a decision-theoretic generalization of the entropy that replaces the log probability of the target variable with a loss function for that variable and repurpose it to conformal prediction by using a family of set-valued prediction losses, indexed by λ\lambda. The generalized / conformal entropy is then the expected loss when one selects the optimal set predictor and the generalized / conformal mutual information follows by taking the difference of two conformal entropies. Interestingly, one can also recover the Shannon mutual information by integrating the conformal mutual information over λ\lambda. The authors then employ this theory in two ways. Firstly, they connect set-size reduction in conformal prediction to the classical concept of information gain, showing that the set-size reduction can be sandwiched by conformal mutual information terms, calibration and model error terms. Secondly, they show a data processing inequality property for conformal set-size reduction where, when there is some post processing of the features, the set size reduction with the new features is upper bounded by the set-size reduction of the original features and some additional terms. Finally, the theory is extended from simple probability conformal scores to APS scores. Experimentally, the authors evaluate the validity of their theoretical results on both real and synthetic experiments as well as probe the set-size reduction and Shannon mutual information in a feature selection experiment.

Critical strengths

The main critical strength of the paper is the more direct connection of decision-theoretic quantities to conformal prediction. This can potentially open interesting directions for future works and expansion of domains where conformal prediction can be useful. Both conformal entropy and conformal mutual information are quite intuitive and the fact that a simple integration recovers mutual information is quite interesting.

Critical weaknesses

In my opinion, a critical weakness of this work is the quite limited number of practical experiments; essentially there is only one main experiment on feature selection and the rest are just empirical verification of the theory. This makes it hard to connect the overall theory presented in the paper to something that is practically actionable. For example, the authors showed in Figure 2 an interesting case where Shannon MI cannot distinguish features that reduce the set-size but conformal MI, depending on the value of λ\lambda can. I would have expected more of such applications of the theory in experiments to demonstrate why this new direction is more promising and relevant to pursue than other ones.

Questions for authors

Overall, while I appreciate and find very interesting the decision-theoretic extensions of the information theoretic views of conformal prediction, I am borderline negative for this work, mostly due to the quite limited number of practical experiments that do not make the theory very actionable. If those are sufficiently addressed I would be happy to raise my score. As for the questions

  • The conformal information divergence as the authors point out is a pseudo-divergence and thus it can be zero even if the model distribution qq is not the same as the data distribution pp. Can the authors expand on when this can and cannot be problematic in practice?
  • For the data processing inequality for conformal set-size reduction, is there a reason that the authors consider the quite specific Markov chain Y→(X,Z)→(X~,Z)Y \rightarrow (X, Z) \rightarrow (\tilde{X}, Z) where only XX (and not also ZZ) is modified? Why do you even need to introduce ZZ in the first place? By looking at the proof, you only work with X(i)X^{(i)} and X~(i)\tilde{X}^{(i)} anyway, which hide ZZ in their construction.
  • Can the authors discuss and motivate their theory with more practical examples where it can have the edge over existing methods?
4 comments

We thank the reviewer for their detailed comments and for noting that our work can open interesting research directions related to conformal prediction. Below, we address the request for additional practical experiments and answer the remaining questions.

Question 1. The conformal information divergence is a pseudo-divergence because it only depends on whether qq places each label on the same side of the probability threshold λ\lambda as pp. Our set-size results operate at a single calibrated threshold, so this poses no issue. However, it may be problematic in applications that require multiple coverage levels. In such cases, one can instead use a mixture of DλD_\lambda, which is a (strict) divergence whenever the mixing distribution has full support over (0,1](0,1].

Question 2. Our main object of interest is the set-size reduction Δα\Delta_\alpha, which measures the information gained beyond base features ZZ. The DPI compares this gain when acquiring XX versus X~\tilde X under the same (fixed) base features. As the reviewer points out, ZZ only enters the proof through X(i)X^{(i)} and X~(i)\tilde{X}^{(i)}, since ∣Cα(Z)∣|\mathcal C_\alpha(Z)| cancels in the difference Δα−Δ~α\Delta_\alpha - \tilde\Delta_\alpha. Nonetheless, we allow possible ZZ features for conceptual purposes and to state the result in full generality.

We also note that the Markov chain Y→(X,Z)→(X~,Z)Y \to (X, Z) \to (\tilde X, Z) is the standard condition for the conditional DPI, since equivalently, Y⊥X~∣(X,Z)Y \perp \tilde X \mid (X, Z). This allows X~\tilde X to be any noisy function of XX and ZZ. We will clarify this after Theorem 3.

Question 3 / Weakness. We agree that additional experiments would help demonstrate how our perspective can guide the design of new methods. As an example, we revisit conformal training through the lens of conformal entropy. Then, following the comment on Figure 2, we use our feature selection experiment to explicitly show that the ranking of features by set-size reduction frequently changes with the coverage level on real data.

Conformal training. Our conformal loss admits a corresponding conformal cross-entropy,

Hλ(p,q)=Ep[ℓλ(Y,Γλ∗(X;q))]=Hλ(Y∣X)+Dλ(p∥q).H_\lambda(p, q) = \mathbb E_p [\ell_\lambda(Y, \Gamma_\lambda^*(X; q))] = H_\lambda(Y \mid X) + D_\lambda(p \parallel q).

As in Theorem 1, cross-entropy training uniformly weights Hλ(p,q)H_\lambda(p,q) over λ∈(0,1]\lambda \in (0,1]. We instead propose to minimize Eλ∼stopgrad(Wθ)[Hλ(p,qθ)]\mathbb E_{\lambda \sim \mathrm{stopgrad}(W_\theta)}[H_\lambda(p, q_\theta)], where WθW_\theta is an estimate of the distribution of the calibrated threshold using held-out data. We note that our objective, which we arrive at from first principles, has a similar form to [1], with some minor differences. Moreover, the objective is smooth in the predicted probabilities, so unlike existing conformal training methods, it does not require soft set assignments or differentiable sorting.

We compare against a cross-entropy baseline, ConfTr [1], and the MB Fano and DPI training objectives in [2], both of which are derived from information-theoretic bounds for conformal prediction. Following prior conformal training work, we use the MNIST, Fashion-MNIST, and EMNIST datasets. At 90% coverage, our objective consistently matches or improves on cross-entropy in set size using probability and APS scores without degrading accuracy. The relative performance of other methods varies across datasets and conformal scores (see the results in the following comment).

Feature selection. To demonstrate that the behavior observed in Figure 2 also arises on real data, we extend our feature selection experiment to coverage levels ranging from 50% to 99%. For each dataset, a substantial fraction of feature pairs reverse their relative ranking in set-size reduction as coverage varies. Shannon mutual information, which induces a single coverage-independent ranking, cannot capture this behavior.

Broader remarks. While our conformal training objective has a slight overall edge over existing baselines, we emphasize that a central contribution of our work is the information-theoretic perspective itself. This allowed us to arrive at an objective similar to existing conformal training methods from first-principles reasoning. Moreover, set size is commonly used to measure uncertainty and information gain, and our work provides a principled explanation for this practice too, which we believe is valuable in its own right.

In terms of algorithm design, our feature selection results demonstrate that Shannon mutual information misses coverage-dependent behavior. This suggests that information-theoretic objectives (e.g., those in active learning and feature acquisition) admit coverage-targeted counterparts. Our conformal training objective is one concrete example, and we believe these connections offer several promising directions for future work. We will include these results and the accompanying discussion in the appendix.

Conformal training results. We report the mean ± 2 standard errors over 10 training seeds, each averaged over 10 calibration/test splits. The best model is bolded, and models that significantly improve on the CE baseline are underlined (one-sided paired t-test across seeds, p<0.05p < 0.05).

ModelProb. sizeAPS sizeAccuracy (%)Coverage (Prob. / APS)
MNIST
CE baseline0.950±0.0000.950 \pm 0.0001.183±0.0011.183 \pm 0.00192.47±0.0492.47 \pm 0.040.902/0.9010.902 / 0.901
ConfTr1.127±0.0051.127 \pm 0.0051.141±0.005‾\underline{1.141 \pm 0.005}86.22±0.1186.22 \pm 0.110.902/0.9020.902 / 0.902
MB Fano0.954±0.0100.954 \pm 0.0101.179±0.0111.179 \pm 0.01192.40±0.5392.40 \pm 0.530.902/0.9010.902 / 0.901
DPI0.960±0.0010.960 \pm 0.0011.307±0.0021.307 \pm 0.00291.91±0.0691.91 \pm 0.060.901/0.9000.901 / 0.900
Conformal entropy (Ours)0.948±0.000‾\underline{\mathbf{0.948 \pm 0.000}}1.140±0.001‾\underline{\mathbf{1.140 \pm 0.001}}92.62±0.03‾\underline{\mathbf{92.62 \pm 0.03}}0.902/0.9020.902 / 0.902
Fashion-MNIST
CE baseline1.032±0.0031.032 \pm 0.0031.175±0.0021.175 \pm 0.00288.68±0.1488.68 \pm 0.140.900/0.9010.900 / 0.901
ConfTr1.042±0.0021.042 \pm 0.0021.537±0.0191.537 \pm 0.01988.18±0.0988.18 \pm 0.090.900/0.9020.900 / 0.902
MB Fano1.013±0.002‾\underline{1.013 \pm 0.002}1.308±0.0051.308 \pm 0.00589.33±0.10‾\underline{89.33 \pm 0.10}0.900/0.9020.900 / 0.902
DPI1.009±0.003‾\underline{\mathbf{1.009 \pm 0.003}}1.212±0.0031.212 \pm 0.00389.50±0.07‾\underline{\mathbf{89.50 \pm 0.07}}0.900/0.9030.900 / 0.903
Conformal entropy (Ours)1.027±0.002‾\underline{1.027 \pm 0.002}1.052±0.001‾\underline{\mathbf{1.052 \pm 0.001}}88.81±0.0588.81 \pm 0.050.899/0.9000.899 / 0.900
EMNIST
CE baseline1.193±0.0031.193 \pm 0.0031.643±0.0041.643 \pm 0.00483.57±0.0583.57 \pm 0.050.902/0.9020.902 / 0.902
ConfTr1.336±0.0231.336 \pm 0.0231.557±0.028‾\underline{1.557 \pm 0.028}78.63±0.7578.63 \pm 0.750.900/0.9010.900 / 0.901
MB Fano1.200±0.0041.200 \pm 0.0041.828±0.0071.828 \pm 0.00783.53±0.1883.53 \pm 0.180.902/0.9020.902 / 0.902
DPI1.191±0.003\mathbf{1.191 \pm 0.003}1.613±0.004‾\underline{1.613 \pm 0.004}83.70±0.07\mathbf{83.70 \pm 0.07}0.901/0.9020.901 / 0.902
Conformal entropy (Ours)1.202±0.0031.202 \pm 0.0031.313±0.003‾\underline{\mathbf{1.313 \pm 0.003}}83.36±0.0783.36 \pm 0.070.902/0.9020.902 / 0.902

 

Feature selection reversal results. We report the fraction of feature pairs whose ranking by set-size reduction reverses between coverage levels from 50% to 99%, and the Spearman correlation between the feature rankings at 99% and 50% coverage. Reversals are counted when significant across 50 calibration/test splits.

DatasetReversed feature pairsRank corr. @ 99% vs. 50%
Wine5858% (32/55)(32 / 55)0.430.43
HAR2424% (301/1275)(301 / 1275)0.690.69
Letter1313% (16/120)(16 / 120)0.900.90
Fashion-MNIST3131% (622/2016)(622 / 2016)0.650.65
CIFAR-102121% (425/2016)(425 / 2016)0.810.81
CIFAR-1002121% (416/2016)(416 / 2016)0.720.72

 

[1] Learning Optimal Conformal Classifiers. Stutz et al. ICLR, 2022.

[2] An Information Theoretic Perspective on Conformal Prediction. Correia et al. NeurIPS, 2024.

I would like to thank the authors for their response and new results. These address my concerns and I am happy to increase my score.

We thank the reviewer for their questions and positive feedback, and for taking the time to consider our response and additional results. We will incorporate the new analysis in the revised manuscript where appropriate.

Reviewer_WdGHOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper provides a theoretical framework for quantifying information gain via conformal prediction sets. The authors introduce decision-theoretic information measures to prove that conformal set-size reduction is bounded by calibration-dependent terms and obeys a data processing inequality, linking it to Shannon mutual information. They validate this theory across 11 classification settings, demonstrating its use as a principled information gain metric.

Critical strengths
  1. The effectiveness of the proposed method is validated both theoretically and empirically.
  2. It is interesting that the method maintains accuracy while reducing set size.
  3. Eleven datasets spanning synthetic, image, and tabular domains are considered.
Critical weaknesses
  1. The scope of the theorems and experiments is limited: only TPS and APS are considered, and only classification tasks are addressed.
  2. Conditional coverage is not fully discussed in both theoretical analysis and experiments.
  3. The reduction in set size is not substantial (cf. Table 1).
  4. Currently, the code is not publicly available.
Questions for authors
  1. Can the theorems and experiments be extended to more (non-)conformity scores, such as RAPS [1] and SAPS [2], and to more tasks, such as LLM generation and regression?
  2. Conditional coverage is not discussed. Could the authors provide further theoretical and empirical analysis?
  3. What are the differences between this work and existing work? [3] also provides an upper bound on the entropy of set size, and [4] discusses how to reduce set size while maintaining conditional coverage.
  4. In Table 1 for the Letter dataset, \delta_\alpha(Prob.) is better than \delta_\alpha(APS) for APS. Although this is explained in the paper, could the authors offer a practical strategy for choosing between them?

[1] Uncertainty Sets for Image Classifiers Using Conformal Prediction
[2] Conformal Prediction for Deep Classifier via Label Ranking
[3] The Entropic Overhead of Conformal Correction
[4] Length Optimization in Conformal Prediction

3 comments

We thank the reviewer for their constructive review and for highlighting the strengths of our empirical analysis. We respond to the questions below.

Weakness 1 / Question 1. Our APS analysis applies more generally to any score whose prediction sets add labels in decreasing order of predicted probability, with optional randomization at the boundary label. Sets of this form are Bayes-optimal under the conformal loss for an appropriate choice of the input-dependent threshold Λ\Lambda. This family includes RAPS and SAPS, so analogous versions of Theorems 4 and 5 hold for these scores under mild assumptions. We will add this discussion to the paper.

To support this claim empirically, we repeat our sandwich validation experiment with RAPS and SAPS on the synthetic datasets using 10,000 calibration and test points. We tune the RAPS and SAPS hyperparameters on held-out data. Across 100 random splits, our bounds hold exactly for both scores, and normalized widths are similar to those for APS.

ScoreS1 violationsS1 widthS2 violationsS2 width
APS0/1000/1000.1010±0.00080.1010 \pm 0.00080/1000/1000.0918±0.00080.0918 \pm 0.0008
RAPS0/1000/1000.1338±0.00250.1338 \pm 0.00250/1000/1000.0815±0.00090.0815 \pm 0.0009
SAPS0/1000/1000.1032±0.00100.1032 \pm 0.00100/1000/1000.1102±0.00130.1102 \pm 0.0013

Regarding other tasks, LLM applications with a finite set of candidate answers, such as multiple-choice question answering, reduce directly to our classification setting. Extensions to open-ended generation and regression are more challenging and are an interesting direction for future work.

Weakness 2 / Question 2. Since our work studies the expected set-size reduction, our theoretical results only involve marginal coverage to control the finite-sample calibration terms. A conditional version of our analysis would require control of the conditional miscoverage, which standard conformal guarantees do not provide. We also note that we do not modify how the prediction sets are constructed, so any conditional coverage properties are inherited from the underlying score.

Empirically, we examine whether selecting features by Δα\Delta_\alpha affects class-conditional coverage in our feature selection experiment. We use the average class coverage gap metric from [5], which is the average deviation of per-class coverage from the nominal level. Selecting features by Δα\Delta_\alpha does not meaningfully change class-conditional coverage relative to the other criteria.

DatasetCriterionProb. CovGap (↓\downarrow)APS CovGap (↓\downarrow)
S3Δα\Delta_\alpha (Prob.)4.64±0.034.64 \pm 0.031.64±0.021.64 \pm 0.02
Δα\Delta_\alpha (APS)4.20±0.03\mathbf{4.20 \pm 0.03}1.58±0.02\mathbf{1.58 \pm 0.02}
Shannon MI4.80±0.034.80 \pm 0.031.79±0.021.79 \pm 0.02
Accuracy8.38±0.068.38 \pm 0.062.51±0.022.51 \pm 0.02
Random5.40±0.335.40 \pm 0.332.06±0.072.06 \pm 0.07
CIFAR-100Δα\Delta_\alpha (Prob.)5.86±0.11\mathbf{5.86 \pm 0.11}5.39±0.06\mathbf{5.39 \pm 0.06}
Δα\Delta_\alpha (APS)5.86±0.08\mathbf{5.86 \pm 0.08}5.44±0.055.44 \pm 0.05
Shannon MI6.12±0.086.12 \pm 0.085.60±0.075.60 \pm 0.07
Accuracy5.90±0.105.90 \pm 0.105.48±0.065.48 \pm 0.06
Random5.95±0.075.95 \pm 0.075.51±0.065.51 \pm 0.06
LetterΔα\Delta_\alpha (Prob.)5.22±0.105.22 \pm 0.104.75±0.064.75 \pm 0.06
Δα\Delta_\alpha (APS)5.23±0.115.23 \pm 0.114.63±0.07\mathbf{4.63 \pm 0.07}
Shannon MI5.16±0.10\mathbf{5.16 \pm 0.10}4.64±0.084.64 \pm 0.08
Accuracy5.17±0.095.17 \pm 0.094.65±0.084.65 \pm 0.08
Random5.49±0.125.49 \pm 0.124.77±0.134.77 \pm 0.13

 

[5] Class-Conditional Conformal Prediction with Many Classes. Ding et al. NeurIPS, 2023.

Weakness 3. The reviewer is correct that the set-size improvements in Table 1 are modest on some datasets. Nonetheless, selecting features by Δα\Delta_\alpha consistently yields the smallest sets, and since all metrics in Table 1 are averaged over the entire selection trajectory, they dilute the larger gains that appear when only a few features have been selected (see Figure 12).

Weakness 4. The code for our experiments is included in the supplementary material, and we will add a link to an open-source repository in the paper upon publication.

Question 3. We thank the reviewer for bringing up these relevant works and discuss them below.

We are unable to find a paper with the exact title of [3], and believe the reviewer is referring to “Conformal Correction for Efficiency May Be at Odds with Entropy” (Xu et al., 2025). Their paper upper bounds the expected APS set size in terms of the prediction entropy and concludes that the two can have competing effects. We instead study the reduction in set size and show that it is bracketed on both sides by conformal mutual information at the calibrated thresholds. Combined with Theorem 1 / Lemma 1, this offers a complementary explanation for why set size and Shannon quantities can disagree. Our analysis also extends beyond APS, covering the probability score (main paper) and RAPS/SAPS (rebuttal).

In [4], the authors focus on a separate problem, i.e., constructing prediction sets that minimize expected set size subject to conditional coverage constraints. Both our work and [4] build on the Lagrangian of this optimization problem, which corresponds to our conformal loss with either fixed λ\lambda or input-dependent Λ\Lambda. However, they use the dual to derive an algorithm that learns optimal prediction sets, whereas we relate conformal set size to information-theoretic quantities using generalized information measures.

Question 4. In practice, we recommend using the Δα\Delta_\alpha criterion that matches the score used to construct the final prediction sets. As shown in Table 5, this choice typically yields the smallest sets. The few exceptions, such as Letter under APS, have small differences that are within the variability across random splits, so we attribute such behavior to estimation noise rather than a systematic effect.

Dear Reviewer WdGH,

As the discussion period is coming to a close, we wanted to kindly follow up and ask whether our responses have addressed your concerns. We would be happy to provide further clarification if needed!

Reviewer_tQkyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper develops an information-theoretic justification for using conformal prediction set-size reduction as a measure of information gain. It defines a family of decision-theoretic confomal entropies and mutual informations based on set size and miscoverage, proves that Shannon mutual information is the integral of this family, and establishes approximate sandwich bounds and a data processing inequality for set-size reduction, with calibration and model-error terms.

Critical strengths

The paper addresses a meaningful gap between common practice and theory. The paper notes that prior theoretical work provided one-sided entropy bounds rather than establishing whether set-size reduction behaves as a principled information measure.

The theoretical framework offers a useful, nontrivial connection to classical information theory. The paper’s family of set-valued information measures has an exact integral relationship with Shannon mutual information.

The paper derives an approximate data processing inequality and explicitly includes calibration and model-imperfection slack, which makes the claims more informative than an idealized oracle-only result.

Critical weaknesses

The empirical validation on real data is suggestive, but does not directly verify the model-error component of the theory. For real-world datasets, the true conditional distributions, the relevant information and divergence terms are inaccessible; the experiments use model plug-ins and set the divergence term to zero.

Some of the finite-sample slack may materially weaken the practical value of the guarantees. The paper reports that at small calibration sizes and large class counts, the reciprocal-threshold factor can make the bounds vacuous.

Questions for authors

See the weakness section.

1 comment

We thank the reviewer for their thoughtful feedback and for recognizing the nontrivial and useful connection to classical information theory. We briefly comment on the weaknesses noted in the review below.

Weakness 1. The reviewer is correct that the model error terms cannot be computed without access to the oracle probabilities. However, they can be bounded using the expected KL divergence. For example, using the probability score,

D(i)≤2⋅E[KL(p(⋅∣X(i))∥q(⋅∣X(i)))]/λ^α(i).D^{(i)} \leq \sqrt{2 \cdot \mathbb{E} \bigl[ \mathrm{KL} \bigl(p(\cdot \mid X^{(i)}) \parallel q(\cdot \mid X^{(i)}) \bigr) \bigr]} \Big/ \hat\lambda_\alpha^{(i)}.

Thus, standard cross-entropy training controls the model error terms, and for applications where the model is well-trained, we would expect them to be small. We will add this discussion to the appendix.

Weakness 2. Although the finite-sample calibration slack can be large when λ\lambda is small, it decreases at an O(n−1/2)O(n^{-1/2}) rate under mild assumptions. Empirically, we find this term is negligible once the calibration set is moderately large (see Figure 3).

Comments on the paper

1 comment

We thank the reviewers for their constructive feedback and recognition of our contributions, including the nontrivial and direct connection to classical information theory (Reviewer tQky & BLFe), the effectiveness and breadth of the empirical evaluation (Reviewer WdGH), and the potential to open interesting directions for future work (Reviewer BLFe). Below, we summarize the main concerns raised and our responses during the discussion period.

Reviewer tQky noted potential limitations in the applicability of the sandwich bounds. We have shown that the divergence terms can be bounded by the expected KL divergence, which is minimized during training. We also clarified that, in our experiments, the finite-sample slack terms become negligible once the calibration set is moderately large.

Reviewer WdGH raised concerns about the scope of our theoretical results beyond TPS and APS scores, as well as the lack of discussion on conditional coverage. In response, we clarified that our APS-based results extend to a broader family of conformal scores (including RAPS and SAPS) and validated this empirically. We also explained that our analysis does not modify the conformal set construction and thus preserves any conditional coverage properties of the underlying score. To support this, we provided additional results showing that the Δα\Delta_\alpha criterion does not meaningfully affect class-conditional coverage in the feature selection experiment.

Reviewer BLFe's primary concern was the limited number of practical experiments demonstrating the utility of our framework. During the rebuttal phase, we added an experiment showing how our conformal entropy framework motivates the design of new training objectives, such as conformal cross-entropy, from first principles. This objective is closely related to existing approaches to conformal training and achieves slightly better overall performance on standard benchmarks. These results demonstrate one concrete application of our information-theoretic perspective on conformal prediction and suggest promising directions for future work. The reviewer acknowledged these additions and updated their assessment following the discussion.

We believe these additional theoretical clarifications and empirical results have addressed the main concerns raised and further strengthened our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.