- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TabCon and explores whether TFMs can construct confidence intervals for the conditional mean in regression tasks in an amortized manner. Building on point estimates from a frozen TabPFN, TabCon trains estimation error prediction heads using synthetic data and constructs confidence intervals from their quantiles. Experiments on synthetic and semi-synthetic data demonstrate good coverage and short interval lengths.
Critical strengths
The paper makes an interesting attempt to explore how well the existing pretraining paradigm of TFMs can achieve calibration in a statistical sense, and demonstrates good performance under the paper’s experimental settings.
Critical weaknesses
For a rigorous statistical inference problem, the current experiments have limited persuasiveness, particularly regarding the statistical clarity of the experimental settings and the boundaries of the model’s capabilities.
Questions for authors
-
What is the joint distribution of the covariates used in training and evaluation, and how is it generated?
-
Holding the other data-generating mechanisms fixed, how does performance change when changes, e.g., when some variables shift from symmetric to one-sided distributions?
-
Under what conditions does TabCon fail (undercoverage) substantially or completely?
-
In an i.i.d. setting with ( positive definite, ), how does TabCon compare with the exact OLS t-interval? Does it offer an IL advantage at comparable coverage?
-
Could the authors include stronger CI baselines with theoretical coverage guarantees under explicit assumptions, and evaluate them where these assumptions hold?
(If these concerns are adequately addressed, I would be willing to raise my score.)
9 comments
Response C2: Performance when changes (Q2)
This is an excellent suggestion. We therefore designed a new experiment in which only the covariate distribution changes, while the regression function, the noise distribution and all other generating parameters are kept fixed. Specifically, we consider three families of regression functions, corresponding to RFF, MLP and Gaussian-process draws, with independent Gaussian noise. To move from symmetric to one-sided covariates, we start from Gaussian covariates and change the marginal distribution of 0%, 50% or 100% of them to a half-normal distribution with the same mean and variance.
Table 1. Coverage (%) / mean interval length of TabCon at the 95% nominal level. From left to right, the proportion of covariates with a half-normal marginal distribution increases.
| Function family | 0% half-normal | 50% half-normal | 100% half-normal |
|---|---|---|---|
| RFF | 97.35 / 3.7951 | 97.19 / 3.6976 | 97.24 / 3.6276 |
| MLP | 98.21 / 3.1773 | 98.23 / 3.1085 | 98.01 / 3.0532 |
| GP | 98.32 / 3.4250 | 98.80 / 3.4340 | 99.08 / 3.4942 |
| Overall | 97.96 / 3.4658 | 98.07 / 3.4133 | 98.11 / 3.3917 |
Table 1 shows that the coverage of TabCon remains stable, and above the nominal level, as the covariates move from symmetric to one-sided distributions. These results provide additional evidence that TabCon is robust to this type of change in the covariate distribution.
Response C3: TabCon limitations and failures (Q3)
This is another excellent point which helped us identify the conditions under which TabCon may fail. We address it from two perspectives.
First, on real covariates, undercoverage is limited to a few datasets, mostly with discrete, mixed-type or strongly dependent covariates or a high noise level (see our response B2 to Reviewer AZfp).
Second, we simulated two covariate shifts of increasing severity between the context data and the query points. In the first one, the context data follow a Gaussian linear model and the query points are placed at increasing distance from them (Table 2). As the distance grows, TabCon produces wider intervals and its coverage decreases, whereas the exact OLS -interval remains close to the nominal level.
Table 2. Coverage (%) of TabCon and of the exact OLS -interval at the 95% nominal level in a Gaussian linear model with , and . From top to bottom, the query points are placed at increasing distance from the context data.
| Norm of the query point | TabCon | OLS -interval |
|---|---|---|
| 99.06 | 94.43 | |
| 97.55 | 94.38 | |
| 95.36 | 94.27 | |
| 91.56 | 94.38 |
In the second setting, the query points lie outside the support of the context data. We restrict the context to and set , which introduces a jump of size in the region . Since the jump does not affect the context data, is not identified outside their support, and no method can guarantee coverage there without structural assumptions on . Accordingly, TabCon covers 45% of the out-of-support query points for and none for and , while its coverage within the support remains above the nominal level.
In summary, TabCon may substantially undercover when the query points lie in low-density regions or outside the support of the context data.
Response C4: TabCon vs OLS intervals (Q4)
Interesting suggestion. We consider the Gaussian linear model , with , independent of , for and . We further consider two settings for the regression coefficients: a dense setting, in which all coefficients are nonzero, and a sparse setting, in which only two coefficients are nonzero. We compare TabCon with the exact OLS interval for the conditional mean, based on the Student- distribution with degrees of freedom.
At the 95% nominal level, TabCon attains higher coverage than OLS (98.88% versus 94.77%) with shorter intervals (3.32 versus 4.20). Since TabCon is conservative in this setting, we also follow your suggestion and compare the two methods at the same empirical coverage level. For each method, we select the nominal level that yields 95% coverage directly on the test data to provide a descriptive comparison at matched in-sample coverage. Results for the two settings are reported in Tables 3 and 4, respectively. We note that, in both settings, the OLS interval remains exact. At comparable coverage, TabCon yields intervals about 46% shorter than OLS on average.
Table 3. Coverage (%) and mean interval length (IL) of TabCon and of the exact OLS -interval in the dense Gaussian linear model at similar coverage, with nominal levels directly selected on test data, averaged over . The last column reports the reduction in IL of TabCon relative to OLS.
| TabCon CP | OLS CP | TabCon IL | OLS IL | IL reduction | |
|---|---|---|---|---|---|
| 0.25 | 95.07 | 95.08 | 2.670 | 2.185 | -22.2% |
| 0.50 | 95.10 | 95.07 | 3.131 | 3.735 | 16.2% |
| 0.75 | 95.08 | 95.12 | 3.874 | 6.732 | 42.5% |
| Overall | 95.08 | 95.09 | 3.225 | 4.217 | 23.5% |
Table 4. Coverage (%) and mean IL of TabCon and of the exact OLS -interval in the sparse Gaussian linear model at similar coverage. The rest is the same to Table 3.
| TabCon CP | OLS CP | TabCon IL | OLS IL | IL reduction | |
|---|---|---|---|---|---|
| 0.25 | 95.08 | 95.07 | 1.246 | 2.167 | 42.5% |
| 0.50 | 95.08 | 95.10 | 1.280 | 3.826 | 66.5% |
| 0.75 | 95.12 | 95.07 | 1.392 | 6.829 | 79.6% |
| Overall | 95.10 | 95.08 | 1.306 | 4.274 | 69.4% |
This result is perhaps surprising, given that the OLS confidence interval is the classical benchmark under the Gaussian linear model. Yet, TabCon, which is pretrained on a much broader class of data generating processes, achieves substantially shorter intervals while maintaining comparable coverage. A deeper look at the two tables suggests that this improvement may stem from two sources:
- The most substantial gains are observed in the sparse setting. This is intuitive, as standard OLS does not explicitly exploit the sparsity structure and may therefore be inefficient in this setting. The results suggest that TabCon can adapt to such structure, potentially incorporating variable selection when constructing confidence intervals.
- The improvement becomes more pronounced as the ratio increases. This is consistent with the James–Stein phenomenon [3], a classical result in multivariate estimation, which demonstrates that introducing shrinkage can improve estimation accuracy. In a similar spirit, TabCon can leverage regularities learned through pretraining across a broad collection of tasks, hence borrowing structural information across datasets rather than relying only on the information available in the current dataset. This cross-task information allows TabCon to achieve more efficient uncertainty quantification in high-dimensional settings.
In summary, even in the classical Gaussian linear model, TabCon performs well and can outperform the classical OLS benchmark, when the regression coefficients are sparse or the covariates are high-dimensional.
References
[3] James, W. and Stein, C. (1961). Estimation with Quadratic Loss. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 361–379.
We thank the reviewer for the comments and for the specific questions, which helped us to improve the statistical rigor of our experimental evaluation and better characterize the capabilities and limitations of TabCon. Since your questions detail the weakness, we focus on answering them point by point below. In particular, we describe the generation of the covariates (C1), study the sensitivity of TabCon to the covariate distribution (C2), characterize the conditions under which TabCon undercovers (C3), and compare TabCon with exact or theoretically guaranteed CIs in settings where their assumptions hold (C4 and C5).
We appreciate this question. The joint distribution of the covariates PX is defined at the task level, and each task is generated with its own covariate distribution. Our main generating mechanism, similarly to the prior of TabPFN [1,2], is a structural causal model (SCM) defined on a random directed acyclic graph (DAG), to which we add parametric covariate families and real covariates, as follows.
In all synthetic settings, the response is then generated as Yi=fi+εi, where the regression target fi is computed by the generator and is therefore known at every query point, and εi follows the noise mechanisms described in Appendix C. We will add a detailed description, including all parameter ranges, to our manuscript.
References
[1] N. Hollmann, S. Müller, K. Eggensperger and F. Hutter (2023). TabPFN: A transformer that solves small tabular classification problems in a second. International Conference on Learning Representations (ICLR).
[2] N. Hollmann, S. Müller, L. Purucker, A. Krishnakumar, M. Körfer, S. B. Hoo, R. T. Schirrmeister and F. Hutter (2025). Accurate predictions on small data with a tabular foundation model. Nature, 637, 319–326.