- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces OneTableBench, which uniformly evaluates the general-purpose capabilities of tabular models across data generation, missing value imputation, and prediction tasks. Through comparisons of cross-task consistency, capability transfer, and specialized models, it reveals the performance trade-offs and transfer limitations of general-purpose models across different tasks.
Critical strengths
1.The research questions addressed by the benchmark does have some value, and the authors have conducted a systematic set of experiments. 2.The paper is well written and the benchmark systematically organize the different kinds of datasets from unified perspectives .
Critical weaknesses
1.The construct validity is insufficient. Although the authors construct the benchmark from the statistical view, the model did not represent the general tabula datasets. 2.The author explicitly limits the scope of the study to these three tasks. This scope seems not sufficient to support the paper’s broader discussion of generalist tabular models.
Questions for authors
- Why is it necessary to unify the generation, imputation and prediction in one benchmark? What is the application for this unified benchmark?
- Compared with individual benchmarks like generation, what is the benefit of using the proposed OneTableBench?
- What does the emphasized conditional query bring benefits for users?
- The authors mention “This perspective requires four levels of evidence: measurement validity, cross-query rank alignment, objective-level transfer or interference, and deployment against fixed specialist portfolios.“, why are the four levels of evidence important? What can be improved for present benchmarks?
2 comments
Thank you for the detailed response. However, the motivation of “whether pretraining could also support the imputation and generation” does not sufficiently justify why these three tasks should be evaluated together to assess generality. Although the authors clarify the scope of their definition, my concern regarding the justification for task selection remains unresolved. Therefore, I will maintain my original score.
We thank the reviewer for raising questions about the value of evaluating the three tasks together.
Weakness:
1. Does the benchmark represent real-world tabular datasets?
We started from 492 OpenML datasets reported in 24 studies. We clustered the 377 eligible datasets by size, feature types, missingness, and dependence structure, then selected one per cluster before evaluating any method. Of the 50 selected datasets, 40 are real-world or derived from real-world data. Six additional datasets with known causal structures provide a separate test of dependence recovery (Appendix A). The findings are grounded in the single-table settings covered by this selection. Independent datasets would help test how well they generalize.
2. Can three tasks support claims about generalist models?
We use generality to mean that one fixed system can generate rows, fill missing features, and predict a designated target. This supports claims about those three uses of a single table. Broader claims would require tests on workloads such as forecasting or reasoning across related tables.
Questions:
1. Why evaluate the three tasks together, and who benefits?
PFNs showed that a pretrained model can predict on new tables using examples in context. This led us to ask whether pretraining could also support imputation and generation. TabFORGE and TabGift show progress on generation [1, 3]. TabMosaic handles generation, imputation, and class-conditional generation with one pretrained model [2].
The three tasks query the same table distribution for complete rows, missing features, or a target. Evaluating them together lets us test whether shared training helps across tasks. A team may need to fill missing entries, generate synthetic data to test a pipeline, and predict outcomes on new records. OneTableBench tests when one system can serve all three needs and when task specialists perform better.
TabMosaic and TabGift appeared after the submission deadline. TabFORGE has public code, and the authors' model repository currently has no downloadable pretrained checkpoint. Our experiments cover systems such as LimiX-2, TabICLv2, and TabPFN-3 through their available inference procedures.
2. What does OneTableBench add to a generation-only benchmark?
OneTableBench compares the same methods on shared datasets and held-out splits while preserving task-specific metrics and information access. For example, methods that retain more predictive utility in synthetic data tend to rank higher on direct prediction. Some generation fidelity metrics show the opposite relationship with classification performance. This helps users see which strengths tend to co-occur across tasks.
3. How do conditional queries benefit users?
A query specifies which values users have and which they need. If a user wants to fill in missing floor area before a home is sold, the imputation test should hide the sale price. For historical records with known prices, the test can make the price available. Users can choose results that match their workflow, and developers can test both settings through one interface.
4. Why do the four levels of evidence matter, and how can benchmarks improve?
References
[1] Jiang, Xiangjian, et al. "Tabular Foundation Model for Generative Modelling." arXiv preprint arXiv:2605.09424 (2026).
[2] Kim, Jayoung, Sanghyun Hong, and Noseong Park. "TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks." arXiv preprint arXiv:2610.15699 (2026).
[3] Li, Yu-Sheng, Jun-Peng Jiang, and Han-Jia Ye. "TabGift: A Context-Efficient Foundation Model for Tabular Data Generation." arXiv preprint arXiv:2610.16990 (2026).