- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper applies the existing HPD-split method to tabular foundation models such as TabPFN and TabICL. The main method is almost identical to HPD-split, and the theoretical results largely follow standard probability arguments. The experiments mainly compare against uncalibrated prediction sets, making the coverage improvements expected. Also, all six synthetic settings use unimodal distributions, despite the claimed advantage of handling multimodal distributions. Overall, the paper has limited methodological novelty, and the experiments do not clearly demonstrate advantages over existing conformal prediction methods.
Critical strengths
None
Critical weaknesses
-
Limited methodological novelty. C-USIM essentially applies the existing HPD-split method to the predictive distributions of TabPFN and TabICL. The theoretical results also largely follow standard probability arguments. It is therefore unclear what the paper adds beyond combining existing methods.
-
Insufficient experimental evidence. The main experiments compare C-USIM with uncalibrated prediction sets, so the coverage improvements are largely expected. There are no comparisons with strong calibrated baselines such as CQR or CHR. Moreover, all six synthetic settings use unimodal conditional distributions, despite multimodal prediction sets being a key claimed advantage. As a result, the experiments do not clearly demonstrate benefits over existing methods.
Questions for authors
- What is the main methodological contribution beyond applying HPD-split to tabular foundation models?
- What evidence shows that C-USIM offers advantages over existing calibrated methods? (The current experiments mainly compare against uncalibrated prediction sets, where coverage improvements are expected.)
6 comments
Response to Reviewer TQjr (Part 2)
| TFM | Setting | CQR | CHR | C-USIM |
|---|---|---|---|---|
| TabPFN | 1D-1 | 95.16 / 3.29 / 0.692 | 95.47 / 2.20 / 1.195 | 95.28 / 2.16 / 0.717 |
| TabPFN | 1D-2 | 94.63 / 2.21 / 1.205 | 94.71 / 2.47 / 1.372 | 94.74 / 2.19 / 1.290 |
| TabPFN | 1D-3 | 95.16 / 1.73 / 0.656 | 95.45 / 2.02 / 0.778 | 94.91 / 1.71 / 0.580 |
| TabPFN | MD-1 | 95.61 / 2.56 / 1.913 | 95.57 / 2.43 / 2.600 | 95.33 / 2.20 / 2.100 |
| TabPFN | MD-2 | 94.98 / 1.83 / 3.102 | 95.01 / 1.90 / 3.379 | 94.81 / 1.64 / 3.313 |
| TabPFN | MD-3 | 95.11 / 2.13 / 2.549 | 94.84 / 2.40 / 2.716 | 95.07 / 2.05 / 2.706 |
| TabICL | 1D-1 | 95.29 / 3.64 / 0.751 | 95.31 / 3.39 / 0.746 | 95.31 / 3.38 / 0.747 |
| TabICL | 1D-2 | 94.80 / 2.46 / 1.257 | 94.93 / 2.63 / 1.287 | 94.93 / 2.63 / 1.287 |
| TabICL | 1D-3 | 95.32 / 2.62 / 0.757 | 95.10 / 3.17 / 0.747 | 95.13 / 3.17 / 0.725 |
| TabICL | MD-1 | 95.44 / 2.54 / 1.880 | 95.43 / 2.64 / 1.901 | 95.43 / 2.64 / 1.901 |
| TabICL | MD-2 | 94.91 / 1.59 / 3.087 | 95.01 / 1.57 / 3.109 | 95.01 / 1.57 / 3.109 |
| TabICL | MD-3 | 95.11 / 2.27 / 2.601 | 95.17 / 2.27 / 2.609 | 95.17 / 2.27 / 2.609 |
Across these six original settings and both TFMs, C-USIM achieves overall performance comparable to CQR and CHR, with mean coverage close to the 95% target. The additional bimodal experiments below demonstrate the benefit of allowing disconnected prediction sets.
3. Additional experiments with multimodal conditional distributions
We agree that the original six unimodal settings did not directly demonstrate the benefit of disconnected prediction regions. We therefore added three bimodal settings covering different separation, scale, and mixing-weight structures.
In all three examples, the covariates are independent, with for , and . Given , draw independently of , and define
This satisfies for every .
- BM-1: Symmetric, well-separated mixture. , , , , and .
- BM-2: Unequal mixture weights and variances. , , , , , and .
- BM-3: Covariate-dependent mixture weights. , , , , and .
Under the same paired evaluation protocol, the results are:
| TFM | Setting | CQR | CHR | C-USIM |
|---|---|---|---|---|
| TabPFN | BM-1 | 95.28 / 1.52 / 2.923 | 95.23 / 1.64 / 2.964 | 94.85 / 1.50 / 1.275 |
| TabPFN | BM-2 | 95.18 / 1.67 / 1.636 | 95.22 / 1.57 / 1.835 | 95.01 / 1.59 / 1.137 |
| TabPFN | BM-3 | 94.79 / 1.65 / 1.681 | 94.98 / 1.71 / 1.791 | 94.85 / 1.59 / 1.245 |
| TabICL | BM-1 | 95.12 / 2.11 / 2.943 | 95.15 / 2.17 / 2.945 | 94.91 / 2.05 / 1.476 |
| TabICL | BM-2 | 95.26 / 2.51 / 1.648 | 95.19 / 2.89 / 1.655 | 95.18 / 2.54 / 1.340 |
| TabICL | BM-3 | 94.98 / 2.54 / 1.727 | 95.03 / 2.68 / 1.731 | 94.92 / 2.47 / 1.462 |
C-USIM produces shorter prediction sets than both CQR and CHR in all three bimodal settings with both TFMs, while maintaining mean coverage near 95%. The relative reductions in mean total set length are:
| TFM | Setting | Reduction vs. CQR (%) | Reduction vs. CHR (%) |
|---|---|---|---|
| TabPFN | BM-1 | 56.40 | 57.00 |
| TabPFN | BM-2 | 30.54 | 38.05 |
| TabPFN | BM-3 | 25.98 | 30.52 |
| TabICL | BM-1 | 49.84 | 49.87 |
| TabICL | BM-2 | 18.70 | 19.09 |
| TabICL | BM-3 | 15.38 | 15.59 |
These reductions are calculated from the unrounded, seed-averaged lengths. All connected components are included when calculating coverage and total length.
In summary, C-USIM achieves overall performance comparable to CQR and CHR across the six original synthetic settings, while producing shorter prediction sets in all three additional bimodal settings with both TabPFN and TabICL. Across these six TFM–bimodal combinations, C-USIM reduces mean total set length by 15.4-56.2% relative to CQR and 15.6-56.8% relative to CHR, while all three methods maintain mean coverage close to the 95% target. These results highlight a methodological advantage of C-USIM in multimodal regression: density-level calibration allows it to retain separated high-density regions while excluding the intervening low-density gaps that a single prediction interval spanning those regions must include. This flexibility enables more efficient prediction sets without additional model fitting or TFM inference.
We will revise the manuscript during the discussion period to clarify the contribution and relationship to prior work, and incorporate these additional baseline and multimodal results.
Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts, particularly the new comparisons with CQR and CHR and the multimodal experiments. However, my following major concerns remain unresolved, and I therefore maintain my original score of 1.
-
The methodological novelty remains limited. C-USIM essentially applies the existing HPD-split/C-HDR construction to predictive distributions from tabular foundation models. The rebuttal does not identify a substantive methodological contribution beyond replacing the conditional density estimator with a pretrained TFM.
-
The empirical advantages remain insufficiently established. While the additional experiments address some of my concerns, the improvements in multimodal settings primarily reflect the well-known advantage of disconnected HPD regions over contiguous intervals. Comparisons with existing methods that also produce disconnected prediction sets, such as HPD-split/C-HDR and CTI, remain missing.
-
The theoretical and conditional-coverage contributions remain limited. Theorem 1 largely follows standard KS-distance and estimation-error arguments, without demonstrating that its bound is informative in practice. Theorem 2 recovers the classical Beta order-statistic result. Moreover, the reported conditional-coverage improvements are not clearly disentangled from ordinary marginal calibration effects.
Overall, although the rebuttal improves the empirical completeness of the paper, it does not resolve my fundamental concern that the work primarily repackages an established conformal prediction method without sufficient new methodological, theoretical, or empirical contributions to justify acceptance.
Response to Reviewer TQjr (Part 1)
Thank you for highlighting the need to clarify our contribution beyond HPD-split and to provide comparisons with calibrated baselines and multimodal examples. We address these concerns below.
1. Contribution beyond HPD-split
C-USIM builds on the established HPD-split density-rank score and split-conformal calibration principle. We agree that this relationship should be stated more explicitly.
To our knowledge, C-USIM is the first study to apply HPD-split directly to predictive densities obtained from pretrained tabular foundation models for regression, enabling conformal prediction sets that can be disconnected. This claim specifically concerns density-level calibration of TFM predictive distributions; prior work has already applied conformal prediction to TFMs using point predictions or predicted quantiles.
Our contribution combines the following aspects:
Direct use of TFM distributional outputs. We instantiate HPD-split using the bin-probability and quantile outputs of pretrained TFMs. Once calibration and test predictions are available, the procedure requires only post-processing, without additional model training, inference, or fitting a separate conditional density or score-correction model. The resulting prediction sets can retain separated high-density regions in multimodal predictive distributions.
Analysis and diagnostics for conditional coverage. We analyze conditional-marginal coverage gaps through distribution-estimation error and score discreteness, which are relevant to the discretized predictive distributions supplied by TFMs. Our percentile rank-score representation illustrates how a shared calibration threshold affects coverage across inputs and reveals remaining coverage heterogeneity. The finite-sample guarantee is marginal validity under the stated assumptions; the conditional-coverage analysis characterizes deviations rather than establishing exact conditional coverage.
Evaluation under a fixed label budget. We examine the trade-off between allocating labels to the TFM context and reserving them for calibration. We also include a same-context, same-predictive-density plug-in HPD comparison to isolate the effect of calibration. The additional CQR, CHR, and multimodal experiments reported below assess coverage and prediction-set length using matched predictive outputs and data splits.
We will revise the manuscript during the discussion period to clarify the relationship to HPD-split, acknowledge the relevant prior TFM conformal-prediction work, and state the scope of these contributions explicitly.
2. Comparison with calibrated baselines
2. Comparison with calibrated baselines
We agree that comparisons with uncalibrated prediction sets alone are insufficient to establish advantages over existing conformal methods. We therefore evaluated CQR, CHR, and C-USIM on the six original synthetic settings and three additional bimodal settings using both TabPFNv3 and TabICL regressor v2.
For each TFM, all three methods share identical predictive outputs and data splits: 512 context, 1,024 calibration, and 256 test observations, with nominal coverage of 95% and ten seeds per setting. The same observations and seeds are also used across the two TFMs. Each method uses the same finite-histogram representation within a TFM, and the comparison implementations are held fixed across TFMs. Conditional coverage is estimated using the same 1,000 responses sampled from the true conditional distribution at each test input for all three methods. Coverage and CCAD are computed across test inputs before averaging over seeds.
Each table entry reports coverage (%) / CCAD (percentage points) / mean total set length, averaged over seeds. CCAD measures the average absolute deviation of the estimated conditional coverage from 95%. Total set length includes every interval component; no small components are removed.