- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes TabMosaic, a unified in-context diffusion model for tabular generation, imputation, and class-conditional generation on unseen tables. The method formulates these tasks as modeling the joint conditional distribution of arbitrary target cells given the observed cells and context, and jointly denoises mixed numerical and categorical targets without imposing an autoregressive cell ordering. The paper further provides theoretical results connecting the training objective to the conditional negative log-likelihood and the posterior predictive distribution. Across 48 real-world tables, TabMosaic achieves strong performance across all three generative tasks without table-specific training or fine-tuning.
Critical strengths
-
The paper presents a meaningful and well-motivated unified formulation of tabular generative tasks. It casts unconditional generation, imputation, and class-conditional generation as instances of modeling the joint conditional distribution over arbitrary target cells. This formulation provides a conceptually clear way to handle multiple generative tasks within a single framework.
-
TabMosaic can be applied to unseen tables without table-specific training or fine-tuning. The same pretrained model and inference procedure are used across tables and tasks by changing only the observed and target cell sets. This substantially reduces the adaptation cost associated with conventional per-table generative models.
-
The theoretical analysis is closely connected to the proposed method. The paper shows that the diffusion objective upper-bounds the conditional negative log-likelihood and targets the posterior predictive distribution under the pretraining prior. These results are further examined experimentally using synthetic priors with tractable posterior predictive distributions.
Critical weaknesses
-
Generalization to high-dimensional tables is not sufficiently evaluated. The experiments exclude the three TabArena datasets with more than 1,000 features, leaving evaluated datasets with at most 212 features. Because the Stage 2 pretraining distribution is also largely restricted to approximately 200 features, the evaluation does not establish whether TabMosaic generalizes beyond its favorable pretraining range.
-
The treatment of incomplete baseline runs may affect the aggregate rankings. Depending on the task, missing baseline results are replaced with reference-baseline scores or scores below the worst completed method. Although computational failures are practically relevant, this procedure combines execution coverage and generative quality within a single Elo score and may disproportionately penalize methods that fail on a subset of datasets.
-
The aggregate Elo results obscure important metric-level trade-offs. TabMosaic ranks first overall and achieves strong downstream utility, but it does not consistently outperform the strongest baselines in α-precision, β-recall, authenticity, or column-level distributional distances. A more detailed analysis of the datasets and metrics on which TabMosaic underperforms would provide a more balanced interpretation of the results.
-
The staged-pretraining ablation changes several factors simultaneously. Stage 2 introduces longer tables, incomplete contexts, and the multi-corruption estimator at the same time. Consequently, the comparison between Stage 1 and Stage 2 does not isolate which of these components is responsible for the observed improvements.
-
The empirical support for “diverse tabular generative tasks” is limited to three task settings. The experiments directly evaluate unconditional generation, imputation, and class-conditional generation, but do not examine other applications of the arbitrary-target formulation, such as general attribute-conditioned generation, constraint-based generation, or robustness under distribution shift. The scope of the conclusions should therefore remain aligned with the three evaluated tasks.
Questions for authors
-
How does TabMosaic perform on tables with substantially more features than those included in its pretraining distribution?
-
Do the generation and class-conditional generation rankings remain consistent when restricted to datasets completed by all compared methods?
-
On which datasets and evaluation metrics does TabMosaic underperform the strongest task-specific baselines, and what factors explain these cases?
-
Could the authors separately ablate the effects of longer-table pretraining, incomplete-context training, and the multi-corruption estimator introduced in Stage 2?
-
How well does the unified conditional-query formulation extend to other settings, such as arbitrary attribute-conditioned generation or distribution shift?
2 comments
Thank you for the comprehensive clarifications, additional ablation study and evaluations. I appreciate the authors' thorough efforts in revising the manuscript.
We thank Reviewer LZH6 for the detailed review and the constructive suggestions.
W1 / Q1. Tables with more than 1,000 features. We thank the reviewer for raising this point. We examined the generalization of TabMosaic with respect to the number of features in our preliminary experiments, before running the main experiments. TabMosaic generalizes well from the feature counts of its pretraining prior to moderately wider tables. However, tables with more than 1,000 features lie far outside the pretraining distribution, and generalizing to them remains challenging. We view this as a limitation of the pretraining prior rather than of our formulation, and we have stated it in the conclusion of the revised paper. As future work, we plan to unify tasks beyond generation in a single pretrained model that also extends to high-dimensional tables, together with a memory-efficient training scheme.
W2 / Q2. Treatment of incomplete baseline runs. Please see General response 3. In short, among the top-5 methods of each task on the datasets that all of them complete, TabMosaic still ranks first on all three tasks without any score replacement.
W3 / Q3. Metric-level trade-offs. Please see General response 4. In short, TabMosaic and TabNAT trade wins on generation metric. TabMosaic has the highest utility, whereas the per-table models are better on coverage, and TabNAT also on most column-level distances, since they are trained on each dataset and fit its support more tightly.
W4 / Q4. Separate ablation of the Stage 2 components. General response 1 separates context missingness and multi-corruption. The effect of longer-table pretraining is given by the comparison between Stage 1 and the base model of General response 1, which are trained on short and long tables, respectively. The longer tables improve generation utility from 0.903 to 0.917 (better on 40 of 48 tables), and the gain concentrates on large tables (17 of the 18 tables with more than 8,192 training rows). We have added this ablation to Appendix M.1
W5 / Q5. Beyond the three evaluated tasks. We additionally evaluate attribute-conditioned generation, where the condition is an arbitrary set of feature columns, on the 30 tables with at most 8,192 training rows. For each table, we hold out 20% of the training rows as queries and use the rest as the context, or as the training data of per-table models. For each query row, we give the values of a random subset C of the features, i.e., |C| = ⌈ F/4 ⌉ or ⌈ F/2 ⌉, where F is the number of features of the dataset, and generate all remaining columns including the label. We compare TabMosaic with the best performing baselines, i.e., TabPFN-3, TabICLv2, TabNAT, and ForestDiffusion, and with a reference that samples each remaining column independently from its marginal. We use the same four metrics and Elo as in our generation benchmark.
Table E. Attribute-conditioned generation on the 30 tables with at most 8,192 training rows. We report Elo with 95% intervals.
TabMosaic ranks first at both condition sizes, on par with TabPFN-3 and clearly ahead of the per-table generators. We thank the reviewer for this suggestion, which extends our evaluation to attribute-conditioned generation, and we have added these results to Appendix L.