- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper considers tabular data settings in which the relation between the target variable and the source variables are conditional on latent "regimes" - which they term as multiregime data. They extend the data prior of tabular foundation models with this type of data to see if it helps downstream generalization.
Critical strengths
-
Practicality: The considered multiregime settings appear practical downstream; for instance when learning tasks are to be done on datasets that are pooled from multiple different sources; e.g. in the paper: prediction on patient data from three different hospitals.
-
New and different ways to create multiregime priors: They propose 2 distinct modifications to the SCM based TFM prior. First creates a different target node/variable per regime, and uses regime-specific cut-off points to translate the scalar target variable to class labels. Second shares the same SCM structure but employs different cutoff points per regime. They also play with the assignment of regimes; either encoded in input/observed data or latent membership not directly derivable from input.
Critical weaknesses
-
Minor improvements: Biggest concern for this work is that the idea does not seem to pan out; improvements are quite marginals and even at places they are significant, the effect size is practically irrelevant. Various versions of TabICL readily perform well on multiregime settings simulated in-distribution. (Table 1 and Fig 2)
-
AI generated content: While it alright for the authors to benefit from AI in generating, say, drawings/figures for the paper, the text also appears to be written by AI, including the interpretation of the results. This leaves the reviewer with lack of confidence in terms of how the findings are communicated.
Questions for authors
-
Did you try inferring if real world datasets follow multi-regime? If so, how would one go about doing it? If they did in fact follow multi-regime, however it is defined, that would be a clearer motivation for the work.
-
If target y follows different functions of the input, as in multiregime, it is hard to imagine standard pretrained TFMs to directly be able to * infer * those multiple functions in-context (via in context learning). Thus, how could the authors explain TabICL variants to do readily so well in such datasets? Perhaps the created multi-function input is not challenging enough or constructed to probe where they would fall short.
It seems that the paper needs 1) more real-world motivation, and 2) better investigation of the shortcomings of in-context learning (ICL) to multi-regime input.