- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper rebuts a recent paper [1] on the consistency behaviour of univariate tabular foundation models (TFMs) when deployed on multivariate classification tasks by using autoregressive prediction. It argues that [1]’s approach of dropping a certain column from the context set leads to an incorrect analysis of the consistency properties, because dropping the column from the context set amounts to a change in the evidence, not a marginalization. The paper then performs an analysis similar to that of [1], but restores the information sets to correctly analyse the consistency behaviour of TFMs.
[1] Klötergens, Christian, et al. "Do Tabular Foundation Models Agree with Themselves?." arXiv preprint arXiv:2608.06004 (2026).
Critical strengths
- The paper correctly fixes the assumption from [1] and provides grounded theoretical reasoning in Section 2. It fixes major problems in [1] and gives a more grounded treatment of the analysis.
- The main contribution of the paper is supported by theoretical justification and adequate experiments.
Critical weaknesses
- The paper could be made stronger by explaining why this behaviour is observed, especially in connection with the architecture design of these tabular foundation models.
- In addition, the study seems limited to two different TFMs, and the TabPFN used is an older version rather than the latest one. Given the paper goal, Introducing more TFM models would strengthen the paper.
Minor but important: The paper is a bit dense, and the writing could still be improved.
Questions for authors
-
Would it make sense to extend the analysis to regression tasks? If so, what are the challenges and limitations? (Future work)
-
Related to the cons above, are the properties discussed in the paper tied to the choice of the TFMs’ architecture? It would be nice if the paper discussed further which architectural components of the TFMs might be responsible for the observed behaviours. (Clarification)
Minor: It might be nice to also connect the findings to the regression case and discuss related work, especially in the Neural Processes literature, such as [2,3].
[2] Bruinsma, Wessel P., et al. "Autoregressive conditional neural processes." arXiv preprint arXiv:2303.14468 (2023).
[3] Hassan, Conor, et al. "Efficient autoregressive inference for transformer probabilistic models." International Conference on Learning Representations. Vol. 2026.