acceptodds
Under review as a conference paper at ICLR 2027

Conformal Prediction and Conditional Coverage for Tabular Foundation Models

Abstract

Tabular foundation models (TFMs) provide predictive distributions for regression, but their prediction regions can exhibit undercoverage or overcoverage even when point predictions are accurate. We introduce C-USIM (Conditionally-Uniformized Score Integration Method), a lightweight application of highest predictive density split conformal prediction that accommodates multimodal predictions. Given calibration and test outputs, it requires no additional training or model inference. It provides finite-sample marginal validity under our assumptions. We bound conditional-marginal coverage gaps using distribution-estimation error and score discreteness, and examine coverage heterogeneity through percentile rank-score plots. Experiments with TabPFN and TabICL show improved marginal coverage accuracy and lower average conditional and group coverage errors. Under a fixed data budget, allocating more observations to calibration can reduce marginal coverage error despite less accurate point predictions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
conformal prediction, tabular foundation models, conditional coverage, highest predictive density

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_TQjrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper applies the existing HPD-split method to tabular foundation models such as TabPFN and TabICL. The main method is almost identical to HPD-split, and the theoretical results largely follow standard probability arguments. The experiments mainly compare against uncalibrated prediction sets, making the coverage improvements expected. Also, all six synthetic settings use unimodal distributions, despite the claimed advantage of handling multimodal distributions. Overall, the paper has limited methodological novelty, and the experiments do not clearly demonstrate advantages over existing conformal prediction methods.

Critical strengths

None

Critical weaknesses
  1. Limited methodological novelty. C-USIM essentially applies the existing HPD-split method to the predictive distributions of TabPFN and TabICL. The theoretical results also largely follow standard probability arguments. It is therefore unclear what the paper adds beyond combining existing methods.

  2. Insufficient experimental evidence. The main experiments compare C-USIM with uncalibrated prediction sets, so the coverage improvements are largely expected. There are no comparisons with strong calibrated baselines such as CQR or CHR. Moreover, all six synthetic settings use unimodal conditional distributions, despite multimodal prediction sets being a key claimed advantage. As a result, the experiments do not clearly demonstrate benefits over existing methods.

Questions for authors
  1. What is the main methodological contribution beyond applying HPD-split to tabular foundation models?
  2. What evidence shows that C-USIM offers advantages over existing calibrated methods? (The current experiments mainly compare against uncalibrated prediction sets, where coverage improvements are expected.)
6 comments
AuthorsOpenReview ↗

Response to Reviewer TQjr (Part 1)

Thank you for highlighting the need to clarify our contribution beyond HPD-split and to provide comparisons with calibrated baselines and multimodal examples. We address these concerns below.

1. Contribution beyond HPD-split

C-USIM builds on the established HPD-split density-rank score and split-conformal calibration principle. We agree that this relationship should be stated more explicitly.

To our knowledge, C-USIM is the first study to apply HPD-split directly to predictive densities obtained from pretrained tabular foundation models for regression, enabling conformal prediction sets that can be disconnected. This claim specifically concerns density-level calibration of TFM predictive distributions; prior work has already applied conformal prediction to TFMs using point predictions or predicted quantiles.

Our contribution combines the following aspects:

  • Direct use of TFM distributional outputs. We instantiate HPD-split using the bin-probability and quantile outputs of pretrained TFMs. Once calibration and test predictions are available, the procedure requires only post-processing, without additional model training, inference, or fitting a separate conditional density or score-correction model. The resulting prediction sets can retain separated high-density regions in multimodal predictive distributions.

  • Analysis and diagnostics for conditional coverage. We analyze conditional-marginal coverage gaps through distribution-estimation error and score discreteness, which are relevant to the discretized predictive distributions supplied by TFMs. Our percentile rank-score representation illustrates how a shared calibration threshold affects coverage across inputs and reveals remaining coverage heterogeneity. The finite-sample guarantee is marginal validity under the stated assumptions; the conditional-coverage analysis characterizes deviations rather than establishing exact conditional coverage.

  • Evaluation under a fixed label budget. We examine the trade-off between allocating labels to the TFM context and reserving them for calibration. We also include a same-context, same-predictive-density plug-in HPD comparison to isolate the effect of calibration. The additional CQR, CHR, and multimodal experiments reported below assess coverage and prediction-set length using matched predictive outputs and data splits.

We will revise the manuscript during the discussion period to clarify the relationship to HPD-split, acknowledge the relevant prior TFM conformal-prediction work, and state the scope of these contributions explicitly.

2. Comparison with calibrated baselines

2. Comparison with calibrated baselines

We agree that comparisons with uncalibrated prediction sets alone are insufficient to establish advantages over existing conformal methods. We therefore evaluated CQR, CHR, and C-USIM on the six original synthetic settings and three additional bimodal settings using both TabPFNv3 and TabICL regressor v2.

For each TFM, all three methods share identical predictive outputs and data splits: 512 context, 1,024 calibration, and 256 test observations, with nominal coverage of 95% and ten seeds per setting. The same observations and seeds are also used across the two TFMs. Each method uses the same finite-histogram representation within a TFM, and the comparison implementations are held fixed across TFMs. Conditional coverage is estimated using the same 1,000 responses sampled from the true conditional distribution at each test input for all three methods. Coverage and CCAD are computed across test inputs before averaging over seeds.

Each table entry reports coverage (%) / CCAD (percentage points) / mean total set length, averaged over seeds. CCAD measures the average absolute deviation of the estimated conditional coverage from 95%. Total set length includes every interval component; no small components are removed.

AuthorsOpenReview ↗

Response to Reviewer TQjr (Part 2)

TFMSettingCQRCHRC-USIM
TabPFN1D-195.16 / 3.29 / 0.69295.47 / 2.20 / 1.19595.28 / 2.16 / 0.717
TabPFN1D-294.63 / 2.21 / 1.20594.71 / 2.47 / 1.37294.74 / 2.19 / 1.290
TabPFN1D-395.16 / 1.73 / 0.65695.45 / 2.02 / 0.77894.91 / 1.71 / 0.580
TabPFNMD-195.61 / 2.56 / 1.91395.57 / 2.43 / 2.60095.33 / 2.20 / 2.100
TabPFNMD-294.98 / 1.83 / 3.10295.01 / 1.90 / 3.37994.81 / 1.64 / 3.313
TabPFNMD-395.11 / 2.13 / 2.54994.84 / 2.40 / 2.71695.07 / 2.05 / 2.706
TabICL1D-195.29 / 3.64 / 0.75195.31 / 3.39 / 0.74695.31 / 3.38 / 0.747
TabICL1D-294.80 / 2.46 / 1.25794.93 / 2.63 / 1.28794.93 / 2.63 / 1.287
TabICL1D-395.32 / 2.62 / 0.75795.10 / 3.17 / 0.74795.13 / 3.17 / 0.725
TabICLMD-195.44 / 2.54 / 1.88095.43 / 2.64 / 1.90195.43 / 2.64 / 1.901
TabICLMD-294.91 / 1.59 / 3.08795.01 / 1.57 / 3.10995.01 / 1.57 / 3.109
TabICLMD-395.11 / 2.27 / 2.60195.17 / 2.27 / 2.60995.17 / 2.27 / 2.609

Across these six original settings and both TFMs, C-USIM achieves overall performance comparable to CQR and CHR, with mean coverage close to the 95% target. The additional bimodal experiments below demonstrate the benefit of allowing disconnected prediction sets.

3. Additional experiments with multimodal conditional distributions

We agree that the original six unimodal settings did not directly demonstrate the benefit of disconnected prediction regions. We therefore added three bimodal settings covering different separation, scale, and mixing-weight structures.

In all three examples, the covariates are independent, with Xj∼Unif(0,1)X_j\sim\mathrm{Unif}(0,1) for j=1,…,dj=1,\ldots,d, and m(x)=sin⁡(2πx1)m(x)=\sin(2\pi x_1). Given X=xX=x, draw S∼Bernoulli(w(x))S\sim\mathrm{Bernoulli}(w(x)) independently of Z∼N(0,1)Z\sim N(0,1), and define

Y=m(X)+(c0+σ0Z)1(S=0)+(c1+σ1Z)1(S=1).\displaystyle Y=m(X)+(c_0+\sigma_0 Z)\mathbf{1}(S=0)+(c_1+\sigma_1 Z)\mathbf{1}(S=1).

This satisfies E[Y∣X=x]=m(x)+c0(1−w(x))+c1w(x)\mathbb{E}[Y\mid X=x]=m(x)+c_0(1-w(x))+c_1w(x) for every x∈[0,1]dx\in[0,1]^d.

  • BM-1: Symmetric, well-separated mixture. d=1d=1, w(x)=0.5w(x)=0.5, c0=−1.2c_0=-1.2, c1=1.2c_1=1.2, and σ0=σ1=0.15\sigma_0=\sigma_1=0.15.
  • BM-2: Unequal mixture weights and variances. d=1d=1, w(x)=0.3w(x)=0.3, c0=−0.36c_0=-0.36, c1=0.84c_1=0.84, σ0=0.08\sigma_0=0.08, and σ1=0.20\sigma_1=0.20.
  • BM-3: Covariate-dependent mixture weights. d=2d=2, w(x)=0.1+0.8x2w(x)=0.1+0.8x_2, c0=−0.6c_0=-0.6, c1=0.6c_1=0.6, and σ0=σ1=0.15\sigma_0=\sigma_1=0.15.

Under the same paired evaluation protocol, the results are:

TFMSettingCQRCHRC-USIM
TabPFNBM-195.28 / 1.52 / 2.92395.23 / 1.64 / 2.96494.85 / 1.50 / 1.275
TabPFNBM-295.18 / 1.67 / 1.63695.22 / 1.57 / 1.83595.01 / 1.59 / 1.137
TabPFNBM-394.79 / 1.65 / 1.68194.98 / 1.71 / 1.79194.85 / 1.59 / 1.245
TabICLBM-195.12 / 2.11 / 2.94395.15 / 2.17 / 2.94594.91 / 2.05 / 1.476
TabICLBM-295.26 / 2.51 / 1.64895.19 / 2.89 / 1.65595.18 / 2.54 / 1.340
TabICLBM-394.98 / 2.54 / 1.72795.03 / 2.68 / 1.73194.92 / 2.47 / 1.462

C-USIM produces shorter prediction sets than both CQR and CHR in all three bimodal settings with both TFMs, while maintaining mean coverage near 95%. The relative reductions in mean total set length are:

TFMSettingReduction vs. CQR (%)Reduction vs. CHR (%)
TabPFNBM-156.4057.00
TabPFNBM-230.5438.05
TabPFNBM-325.9830.52
TabICLBM-149.8449.87
TabICLBM-218.7019.09
TabICLBM-315.3815.59

These reductions are calculated from the unrounded, seed-averaged lengths. All connected components are included when calculating coverage and total length.

In summary, C-USIM achieves overall performance comparable to CQR and CHR across the six original synthetic settings, while producing shorter prediction sets in all three additional bimodal settings with both TabPFN and TabICL. Across these six TFM–bimodal combinations, C-USIM reduces mean total set length by 15.4-56.2% relative to CQR and 15.6-56.8% relative to CHR, while all three methods maintain mean coverage close to the 95% target. These results highlight a methodological advantage of C-USIM in multimodal regression: density-level calibration allows it to retain separated high-density regions while excluding the intervening low-density gaps that a single prediction interval spanning those regions must include. This flexibility enables more efficient prediction sets without additional model fitting or TFM inference.

We will revise the manuscript during the discussion period to clarify the contribution and relationship to prior work, and incorporate these additional baseline and multimodal results.

Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts, particularly the new comparisons with CQR and CHR and the multimodal experiments. However, my following major concerns remain unresolved, and I therefore maintain my original score of 1.

  1. The methodological novelty remains limited. C-USIM essentially applies the existing HPD-split/C-HDR construction to predictive distributions from tabular foundation models. The rebuttal does not identify a substantive methodological contribution beyond replacing the conditional density estimator with a pretrained TFM.

  2. The empirical advantages remain insufficiently established. While the additional experiments address some of my concerns, the improvements in multimodal settings primarily reflect the well-known advantage of disconnected HPD regions over contiguous intervals. Comparisons with existing methods that also produce disconnected prediction sets, such as HPD-split/C-HDR and CTI, remain missing.

  3. The theoretical and conditional-coverage contributions remain limited. Theorem 1 largely follows standard KS-distance and estimation-error arguments, without demonstrating that its bound is informative in practice. Theorem 2 recovers the classical Beta order-statistic result. Moreover, the reported conditional-coverage improvements are not clearly disentangled from ordinary marginal calibration effects.

Overall, although the rebuttal improves the empirical completeness of the paper, it does not resolve my fundamental concern that the work primarily repackages an established conformal prediction method without sufficient new methodological, theoretical, or empirical contributions to justify acceptance.

Reviewer_okawOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes C-USIM, a lightweight conformal prediction method that calibrates prediction regions produced by tabular foundation models (TabPFN and TabICL), aiming to achieve reliable marginal coverage while improving conditional coverage without additional model training. The authors provide theoretical bounds on conditional–marginal coverage gaps and demonstrate through synthetic and real-world experiments that C-USIM improves marginal coverage accuracy and generally reduces conditional or group coverage errors, although prediction regions become longer.

Critical strengths

This paper tries to solve a fundamental problem in statistics by utilizing pretrained model without extra training.

It provides a very good connection between Tabular Foundation Model, Highest Predictive Density and Comformal Prediction.

The proposed method is simple and appears effective according to the experimental results.

Critical weaknesses

When the new dataset shows a clear distribution shift from TFM's pretraining distribution, Conformal Prediction may still guarantee marginal coverage under same-distribution calibration and test conditions, but how much will conditional coverage and prediction set efficiency deteriorate?

Although the paper emphasizes conditional coverage, C-USIM employs a single global conformal threshold and does not explicitly correct input-dependent miscalibration. The observed reductions in conditional and group coverage errors may therefore be largely attributable to improvements in marginal coverage, especially given that the uncalibrated HPD baseline systematically undercovers and C-USIM produces longer prediction regions. The current experiments do not adequately disentangle these effects.

The methodological novelty seems limited as far as I see, as C-USIM largely adapts existing HPD-split conformal prediction.

Questions for authors

I don't fully get the word "conditional coverage" in your title since C-USIM does not perform explicit input-conditional calibration.

4 comments
AuthorsOpenReview ↗

Response to Reviewer okaw (Part 1)

Thank you for highlighting the need to clarify the meaning of conditional coverage in our paper, the interpretation of the empirical improvements, and the limitations under distribution shift.

1. Global calibration and the meaning of “conditional coverage”

We agree that C-USIM uses a global calibration threshold and does not perform explicit input-conditional calibration. However, its density-rank score depends on each input through the TFM predictive distribution. C-USIM therefore combines an input-dependent score construction with a calibration threshold shared across inputs.

The HPD construction provides a theoretical motivation for this approach. Under an oracle conditional density and continuity of the induced density-score distribution, the density-rank score is conditionally uniform and hence pivotal: its distribution is identical across inputs. This explains why separate input-specific calibration is unnecessary in that ideal setting, although finite calibration samples still introduce threshold uncertainty. With estimated TFM distributions, estimation error and score discreteness can prevent pivotality. Our theoretical analysis bounds the conditional–marginal coverage gap under the stated assumptions; it does not establish that calibration universally reduces conditional coverage error relative to plug-in HPD.

As already stated in the Introduction and Section 2.3, the finite-sample conformal guarantee is marginal validity under the stated assumptions, and conditional coverage errors can remain. The phrase “conditional coverage” in the title reflects the gap analysis and ideal-case result in Section 5, together with our empirical evaluation of conditional coverage errors. We will make this existing distinction more explicit, particularly by distinguishing the performance objectives listed in Section 4.2 from the finite-sample coverage guarantee.

2. Conditional coverage improvements versus marginal coverage correction

Regarding the empirical results, we agree that correcting marginal undercoverage can contribute to the observed reduction in CCAD. CCAD nevertheless directly evaluates average conditional coverage accuracy. For a fixed fitted predictor and calibration realization, let C^1−α(x)\widehat C_{1-\alpha}(x) denote the prediction set at input xx, and define

c(x):=P(Y∈C^1−α(x)∣X=x).\displaystyle c(x):=\mathbb P\left(Y\in\widehat C_{1-\alpha}(x)\mid X=x\right).

CCAD measures the average absolute deviation from the conditional coverage target:

CCAD:=EX[∣c(X)−(1−α)∣].\displaystyle \mathrm{CCAD} :=\mathbb E_X\left[\left|c(X)-(1-\alpha)\right|\right].

This differs from marginal coverage error,

∣EX[c(X)]−(1−α)∣,\displaystyle \left|\mathbb E_X[c(X)]-(1-\alpha)\right|,

which takes the absolute deviation only after averaging coverage across inputs. Because CCAD takes absolute deviations before averaging, undercoverage at some inputs cannot cancel overcoverage at others.

Accordingly, a reduction in CCAD represents an improvement in average conditional coverage accuracy, including when achieved through a global threshold adjustment. Correcting marginal undercoverage can be a mechanism underlying this improvement. However, CCAD alone does not disentangle this mechanism from changes in coverage heterogeneity across inputs. Nor does a lower CCAD establish improvement at every input or a better coverage–size trade-off. We therefore interpret these results as reductions in average conditional coverage error and consider them alongside prediction-set length. The same-predictive-density comparisons in Appendix B.8 show lower mean CCAD together with longer mean prediction sets, illustrating this trade-off. The manuscript also reports a case where marginal coverage improves while CCAD increases (TabICL, 1D-1; Appendix B.6.1), illustrating that the two improvements are distinct.

AuthorsOpenReview ↗

Response to Reviewer okaw (Part 2)

3. Mismatch between pretraining and downstream distributions

Pretraining–downstream mismatch may affect conditional coverage and prediction-set efficiency, even when marginal validity is preserved. Our theoretical analysis relates the conditional–marginal coverage gap to predictive-distribution estimation error and score discreteness under the stated assumptions.

The effects of such mismatch depend on how well the pretraining prior represents the downstream task. For example, a response function poorly represented by the prior may lead the TFM to overestimate predictive uncertainty and produce an overly diffuse predictive distribution. Such behavior would reflect a limitation of the underlying TFM’s predictive distribution, rather than an issue specific to C-USIM. Although conformal calibration adjusts the overall coverage level, conditional coverage and prediction-set efficiency still depend on predictive-distribution accuracy. Assessing this mismatch is also constrained by differences across TFMs in the availability of pretraining data-generation code and sampling configurations, which can make it difficult to characterize the actual prior and quantify its mismatch with downstream tasks. Open implementations enable more targeted investigations, but an explicit definition of mismatch and controlled evaluation are still needed.

We will discuss this limitation in the manuscript’s Limitations and Future Work section. If test cases or response functions that are poorly represented by a TFM’s pretraining prior can be identified or constructed, they could be used in future experiments to evaluate conditional coverage and prediction-set efficiency under such mismatch.

4. Contribution beyond HPD-split

Our main contributions are conditional–marginal coverage-gap analysis, which bounds the gap using estimation error and score discreteness under the stated assumptions, and percentile rank–score diagnostics, which relate a shared calibration threshold to coverage at individual inputs. Through context–calibration allocation experiments under a fixed label budget, we show that allocating more observations to calibration can improve marginal coverage accuracy despite less accurate point predictions. C-USIM provides a lightweight implementation of the established HPD-split framework for TFM outputs, supporting this analysis and evaluation without fitting an additional conditional density or score-correction model.

Please see our response to Reviewer TQjr (Part 1, Section 1) for the detailed contribution statement. For comparisons with CQR and CHR using identical TFM predictions and data splits, including conditional coverage error and total prediction-set length, please see Part 1, Section 2 and Part 2.

I thank the authors for their careful response to my comments, which have substantially addressed my concerns. I will raise my score.

AuthorsOpenReview ↗

Thank you for your consideration

Thank you for carefully considering our response and for your decision to raise your score. We sincerely appreciate your time and constructive feedback.

Update: Perhaps we misunderstood, but since your original rating was 3, we took your comment to mean that you intended to raise it. The rating still appears as 3 on OpenReview. Would you mind checking whether the rating update has gone through?

Reviewer_xd8pOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes C-USIM, a conformal prediction method for improving the coverage of regression prediction regions produced by Tabular Foundation Models (TFMs). The method constructs density-rank nonconformity scores from model-predicted distributions following similar procedures as HPD. The theoretical analysis provides a bound on the conditional–marginal coverage gap in terms of distribution estimation error and score discretization, and introduces percentile rank–score plots to analyze conditional coverage. Experiments on synthetic and real-world datasets demonstrate improved marginal coverage and generally reduced conditional or group coverage errors compared with uncalibrated HPD regions, although prediction sets tend to be longer. The paper also investigates training–calibration data allocation, showing that larger calibration sets can improve coverage accuracy at the expense of point prediction accuracy.

Critical strengths

The proposed method leverages the predictive distributions of existing tabular foundation models to provide coverage guarantees. Empirical experiments on synthetic and real-world datasets demonstrate that the method achieves the target coverage.

Critical weaknesses

Limited theoretical contributions.

The upper bound in Theorem 1 can be quite loose. In particular, even when the predicted conditional distributions are perfectly accurate and the true conditional–marginal coverage gap is zero, the bound can remain large due to the discretization error. Moreover, the bound depends on unknown distribution estimation errors, and the paper does not demonstrate whether these errors are sufficiently small in practice. Therefore, it is unclear whether the theoretical result provides a meaningful guarantee.

Theorem 2 appears to be a direct consequence of the well-known Beta distribution of calibration-conditional coverage in split conformal prediction (Vovk, 2012; Angelopoulos and Bates, 2023). Under the strong assumption that the score distribution is independent of the covariates, the covariate-conditional coverage reduces to the standard calibration-conditional coverage. The theoretical novelty of this result appears limited.

  • Vovk V. Conditional validity of inductive conformal predictors[C]//Asian conference on machine learning. PMLR, 2012: 475-490.
  • Angelopoulos A N, Bates S. Conformal prediction: A gentle introduction[J]. Foundations and Trends in Machine Learning, 2023, 16(4): 494-591.
Questions for authors
  1. Under what conditions does the bound in Theorem 1 become informative, and how tight is this bound for the tabular foundation models considered in the experiments?

  2. How does C-USIM compare with established conformal prediction methods, such as CQR and CHR, in terms of conditional coverage error and prediction-set length?

1 comment
AuthorsOpenReview ↗

Response to Reviewer xd8p

Thank you for highlighting the need to clarify the significance and limitations of our theoretical results.

1. Significance of Theorem 1

Under the stated assumptions, Theorem 1 bounds the conditional–marginal coverage gap through bin-probability estimation error and score discretization. Building on existing HPD-split and pivotal-score theory (Izbicki et al., 2022; Laplante, 2026), our analysis makes these effects explicit for the finite-resolution predictive outputs of pretrained TFMs. To our knowledge, this is the first study to explicitly separate these two contributions in a conditional–marginal coverage-gap bound for HPD-split calibration of pretrained TFM predictive distributions.

A practical implication is that, when distribution-estimation errors are comparable, the maximum predicted bin probability mass provides an additional criterion for comparing models and predictive representations. The bound becomes informative when both estimation error and this discretization term are small. Here, comparable estimation error concerns the predictive distribution, rather than point-prediction accuracy alone.

We agree that the bound can be conservative, even when the actual conditional-marginal gap is zero. Its significance lies in providing sufficient conditions for a small gap and making explicit how distribution-estimation error and score discretization jointly affect the bound.

2. Clarifying the role of Theorem 2

Theorem 2 characterizes the distribution of conditional coverage under the idealized assumption of a common continuous score distribution across covariates. This recovers the standard Beta law across calibration samples in split conformal prediction (Vovk, 2012; Angelopoulos and Bates, 2023) and provides a reference for interpreting our percentile rank–score plots. These plots illustrate how departures from this ideal can lead to coverage heterogeneity across inputs. The visualization is also applicable to other methods that construct pivotal scores from predictive densities.

3. Comparison with CQR and CHR

We have added paired comparisons with CQR and CHR using identical predictive outputs and data splits within each TFM, covering both TabPFN and TabICL with ten seeds per setting. Across the six original synthetic settings, C-USIM achieves broadly comparable overall performance in conditional coverage error and prediction-set length, with trade-offs depending on the setting.

In the three additional bimodal settings with both TFMs, C-USIM reduces mean total prediction-set length by 15.4-56.2% relative to CQR and 15.6-56.8% relative to CHR, while maintaining mean coverage near the 95% target and comparable conditional coverage error. Its ability to retain separated high-density regions while excluding intervening low-density gaps enables shorter sets in these examples.

The complete numerical results and experimental protocol are provided in our response to Reviewer TQjr (Part 1, Section 2, and Part 2).

References

Reviewer_kEfLOpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NADS (Neural Active Density Splitting), a novel approach for split conformal prediction and density-based uncertainty quantification in tabular regression tasks. Traditional methods often rely on coarse binning or restrictive density assumptions, while existing conditional calibration methods struggle with multimodality or high computational overhead. NADS leverages a flexible density estimation framework via random splitting to construct valid prediction regions that adapt to heteroscedastic and multimodal distributions. The authors evaluate NADS on diverse regression benchmarks, demonstrating consistent improvements in marginal coverage and conditional coverage error under fixed data budgets.

Critical strengths
  1. Methodological Novelty: The formulation of random splitting for density estimation within conformal prediction offers a clean, mathematically sound way to handle complex conditional distributions without heavy parametric assumptions.
  2. Strong Empirical Performance: The experiments cover a comprehensive suite of datasets, showing clear improvements over standard plug-in and baseline conformal methods in terms of conditional coverage absolute deviation.
  3. Clarity and Presentation: The paper is exceptionally well-written, with intuitive visualizations that clearly explain the theoretical motivation and empirical behaviors.
Critical weaknesses
  1. Hyperparameter Sensitivity: The performance of random splitting heavily relies on specific splitting criteria and split counts. A deeper discussion or sensitivity analysis regarding how these hyperparameters affect computational cost and final set efficiency would strengthen the paper.
  2. Comparison with State-of-the-Art Structured/Discontinuous Baselines: While the paper compares against standard plug-in HPD and traditional split conformal methods, it lacks discussion or empirical comparison with recent advanced non-contiguous conformal prediction baselines (Conformal thresholded intervals for efficient regression, Volume Optimality in Conformal Prediction with Structured Prediction Sets). Including these would better position the exact utility of NADS.
Questions for authors
  1. How sensitive is the empirical coverage guarantee to the choice of the random splitting depth and the number of split iterations?
  2. What is the overhead of NADS in terms of inference and calibration time compared to standard split conformal regression (e.g., CQR)?
2 comments
AuthorsOpenReview ↗

Response to Reviewer kEfL (Part 1)

Thank you for your constructive feedback and suggestions. We have interpreted the references to “NADS” in your review as referring to C-USIM and respond accordingly below.

1. Sensitivity to splitting and density representation

In C-USIM, splitting refers to allocating observations between the TFM context and the calibration set. The predictive density is reconstructed directly from the pretrained TFM’s outputs, so the procedure does not involve a recursive splitting depth or repeated density-splitting iterations. The relevant choices are the context–calibration allocation and the resolution of the predictive density representation, which are examined in Section 6.3 (Figure 4) and Theorem 1, respectively. Specifically, a finer density representation can reduce the discretization term max⁡ip^i(x)\max_i \hat p_i^{(x)} in the bound of Theorem 1, while estimating the resulting higher-dimensional probability vector may increase the L1L^1 estimation error.

The finite-sample marginal coverage guarantee follows from conformal calibration under the stated exchangeability assumptions. It does not require the estimated predictive density to be accurate. Consequently, changing the density representation does not invalidate this guarantee, provided that the score construction is fixed independently of the calibration labels and the assumptions remain satisfied. The context observations determine the predictive distribution used to construct the scores, while the calibration observations determine the empirical cutoff. For a fixed predictor, increasing the calibration sample size can stabilize the estimated cutoff.

2. Comparison with non-contiguous conformal prediction methods

We thank the reviewer for these relevant references. Both methods permit disconnected prediction sets, and we will clarify their relationship to C-USIM.

Conformal Thresholded Intervals (Luo and Zhou, 2025) thresholds the widths of estimated equal-probability interquantile intervals using a shared calibration cutoff. C-USIM instead calibrates a density-rank score: the cumulative probability mass assigned to regions with density at least as high as that of the candidate response. Thus, both prioritize high-density regions, but CTI thresholds interval widths directly, whereas C-USIM uses an input-specific probability scale. This distinction affects how scores are compared across inputs, even when their within-input ordering agrees.

Gao et al. (2025) require specifying an interval-count parameter kk in advance and use dynamic programming to construct prediction sets within the resulting class of unions of at most kk intervals. C-USIM requires no such parameter: the number of connected components is determined by the estimated predictive density and the calibrated threshold and can vary across inputs.

Thus, C-USIM differs from CTI in the score it calibrates and from Gao et al.’s method in constructing density-level regions without a fixed limit on the number of intervals.

AuthorsOpenReview ↗

Response to Reviewer kEfL (Part 2)

3. Inference and calibration overhead relative to CQR

Once the TFM outputs for the calibration and test observations are available, C-USIM requires no additional model fitting or forward pass. Its additional computation consists of density-based score calculation, calibration-quantile selection, and prediction-set construction. With BB density bins, our implementation computes calibration scores in O(ncalB)O(n_{\mathrm{cal}}B) and constructs test prediction sets in O(ntestBlog⁡B)O(n_{\mathrm{test}}B\log B), in addition to selecting the calibration order statistic.

We measured this overhead on nine synthetic settings, including three bimodal settings, with ten seeds per setting. Each run used TabPFN v3 with eight estimators, 512 context observations, 1,024 calibration observations, 256 test observations, and 5,000 density bins, at nominal coverage 95%. All methods shared identical cached TFM predictions. Postprocessing ran on an Intel Xeon Gold 6530 CPU with four threads.

StageMean ± SD per run (seconds)
Shared TFM prediction stage, recorded previously1.569 ± 0.193
CQR calibration and interval construction0.092 ± 0.002
C-USIM calibration and set construction0.630 ± 0.063
CHR calibration and interval construction7.062 ± 0.184

The shared TFM timing includes the wrapper’s context setup (fit), joint calibration/test prediction, and copying the outputs to the CPU cache. It was recorded when the predictions were originally generated and was not remeasured in this comparison. The three postprocessing timings were measured anew and exclude model inference, metric evaluation, and disk I/O. Values are means and sample standard deviations across 90 setting–seed combinations.

Although C-USIM requires an additional 0.538 seconds per run relative to CQR, this overhead yields substantially shorter prediction sets at comparable marginal coverage near the same 95% target. Across the three bimodal settings, C-USIM reduced mean total set length by 56.2%, 30.3%, and 25.8%, respectively, while mean coverage remained 94.86-95.06%, compared with 94.76-95.26% for CQR. (See Response to Reviewer TQjr for detailed result.) By excluding low-density gaps between separated modes, C-USIM therefore provides considerably more informative prediction sets without additional TFM inference calls. These results demonstrate a favorable practical trade-off: in these multimodal settings, a modest absolute increase in postprocessing time delivers a substantial improvement in prediction-set efficiency while maintaining comparable coverage.

References

  • Luo, R., and Zhou, Z. Conformal Thresholded Intervals for Efficient Regression. Proceedings of the AAAI Conference on Artificial Intelligence, 39(18):19216–19223, 2025.
  • Gao, C., Shan, L., Srinivas, V., and Vijayaraghavan, A. Volume Optimality in Conformal Prediction with Structured Prediction Sets. Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:18495–18527, 2025.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions

We have uploaded a revised manuscript incorporating the reviewers’ suggestions and the clarifications and additional results discussed in our responses. The main changes are summarized below:

  • Clarified C-USIM’s relationship to HPD-split and the scope of our contributions (Sections 1 and 3).
  • Clarified the motivation and interpretation of the theoretical analysis, including the roles of estimation error and score discretization, and the ideal setting with a common score distribution across inputs (Section 5).
  • Added three bimodal settings and updated the synthetic results, figures, and percentile rank–score diagnostics to cover all nine settings (Section 6.1; Appendices B.2, B.3, and B.6.1).
  • Added comparisons with CQR, CHR, and CTI using matched TFM predictions and data splits (Appendix B.8.4, Table 10).
  • Extended the context–calibration allocation and same-predictive-density comparisons to the bimodal settings (Appendices B.7 and B.8.1).
  • Clarified the distinction between marginal validity and conditional-coverage objectives, and the interpretation of CCAD improvements (Section 4.2; Appendix B.4).
  • Added discussion of pretraining–downstream mismatch as a limitation and future research direction (Section 7).

Major additions and revisions are marked NEW and FIX, respectively.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.