acceptodds
Under review as a conference paper at ICLR 2027

TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks

10 views

Abstract

Generative modeling for tabular data has advanced to the point where models can synthesize realistic records, recover missing values, and generate samples conditioned on class labels or observed attributes. Despite recent advances, existing approaches either require table-specific training or restrict in-context learning to single-cell prediction. In this work, we formulate these tabular generative tasks as a single problem: modeling the joint conditional distribution of an arbitrary set of target cells given the observed cells of an unseen table. To this end, we turn to diffusion, whose denoising objective is defined jointly over all target cells rather than vone cell at a time. Under the pretraining distribution, this objective is minimized by the posterior predictive distribution over the target set. Building on this, we propose TabMosaic, a unified in-context diffusion model for generating mixed-type target cells in unseen, potentially incomplete tables without table-specific training. We further show that its pretraining objective bounds the conditional negative log-likelihood and the divergence from the posterior predictive for arbitrary target sets, extending the Bayesian account of in-context learning from a single-value prediction to joint multi-cell generation. Across 48 real-world tables, TabMosaic ranks first on all evaluated generative tasks against 20 baseline approaches.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Tabular data, Foundation model, In-context learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LZH6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes TabMosaic, a unified in-context diffusion model for tabular generation, imputation, and class-conditional generation on unseen tables. The method formulates these tasks as modeling the joint conditional distribution of arbitrary target cells given the observed cells and context, and jointly denoises mixed numerical and categorical targets without imposing an autoregressive cell ordering. The paper further provides theoretical results connecting the training objective to the conditional negative log-likelihood and the posterior predictive distribution. Across 48 real-world tables, TabMosaic achieves strong performance across all three generative tasks without table-specific training or fine-tuning.

Critical strengths
  • The paper presents a meaningful and well-motivated unified formulation of tabular generative tasks. It casts unconditional generation, imputation, and class-conditional generation as instances of modeling the joint conditional distribution over arbitrary target cells. This formulation provides a conceptually clear way to handle multiple generative tasks within a single framework.

  • TabMosaic can be applied to unseen tables without table-specific training or fine-tuning. The same pretrained model and inference procedure are used across tables and tasks by changing only the observed and target cell sets. This substantially reduces the adaptation cost associated with conventional per-table generative models.

  • The theoretical analysis is closely connected to the proposed method. The paper shows that the diffusion objective upper-bounds the conditional negative log-likelihood and targets the posterior predictive distribution under the pretraining prior. These results are further examined experimentally using synthetic priors with tractable posterior predictive distributions.

Critical weaknesses
  • Generalization to high-dimensional tables is not sufficiently evaluated. The experiments exclude the three TabArena datasets with more than 1,000 features, leaving evaluated datasets with at most 212 features. Because the Stage 2 pretraining distribution is also largely restricted to approximately 200 features, the evaluation does not establish whether TabMosaic generalizes beyond its favorable pretraining range.

  • The treatment of incomplete baseline runs may affect the aggregate rankings. Depending on the task, missing baseline results are replaced with reference-baseline scores or scores below the worst completed method. Although computational failures are practically relevant, this procedure combines execution coverage and generative quality within a single Elo score and may disproportionately penalize methods that fail on a subset of datasets.

  • The aggregate Elo results obscure important metric-level trade-offs. TabMosaic ranks first overall and achieves strong downstream utility, but it does not consistently outperform the strongest baselines in α-precision, β-recall, authenticity, or column-level distributional distances. A more detailed analysis of the datasets and metrics on which TabMosaic underperforms would provide a more balanced interpretation of the results.

  • The staged-pretraining ablation changes several factors simultaneously. Stage 2 introduces longer tables, incomplete contexts, and the multi-corruption estimator at the same time. Consequently, the comparison between Stage 1 and Stage 2 does not isolate which of these components is responsible for the observed improvements.

  • The empirical support for “diverse tabular generative tasks” is limited to three task settings. The experiments directly evaluate unconditional generation, imputation, and class-conditional generation, but do not examine other applications of the arbitrary-target formulation, such as general attribute-conditioned generation, constraint-based generation, or robustness under distribution shift. The scope of the conclusions should therefore remain aligned with the three evaluated tasks.

Questions for authors
  • How does TabMosaic perform on tables with substantially more features than those included in its pretraining distribution?

  • Do the generation and class-conditional generation rankings remain consistent when restricted to datasets completed by all compared methods?

  • On which datasets and evaluation metrics does TabMosaic underperform the strongest task-specific baselines, and what factors explain these cases?

  • Could the authors separately ablate the effects of longer-table pretraining, incomplete-context training, and the multi-corruption estimator introduced in Stage 2?

  • How well does the unified conditional-query formulation extend to other settings, such as arbitrary attribute-conditioned generation or distribution shift?

2 comments

We thank Reviewer LZH6 for the detailed review and the constructive suggestions.

W1 / Q1. Tables with more than 1,000 features. We thank the reviewer for raising this point. We examined the generalization of TabMosaic with respect to the number of features in our preliminary experiments, before running the main experiments. TabMosaic generalizes well from the feature counts of its pretraining prior to moderately wider tables. However, tables with more than 1,000 features lie far outside the pretraining distribution, and generalizing to them remains challenging. We view this as a limitation of the pretraining prior rather than of our formulation, and we have stated it in the conclusion of the revised paper. As future work, we plan to unify tasks beyond generation in a single pretrained model that also extends to high-dimensional tables, together with a memory-efficient training scheme.

W2 / Q2. Treatment of incomplete baseline runs. Please see General response 3. In short, among the top-5 methods of each task on the datasets that all of them complete, TabMosaic still ranks first on all three tasks without any score replacement.

W3 / Q3. Metric-level trade-offs. Please see General response 4. In short, TabMosaic and TabNAT trade wins on generation metric. TabMosaic has the highest utility, whereas the per-table models are better on coverage, and TabNAT also on most column-level distances, since they are trained on each dataset and fit its support more tightly.

W4 / Q4. Separate ablation of the Stage 2 components. General response 1 separates context missingness and multi-corruption. The effect of longer-table pretraining is given by the comparison between Stage 1 and the base model of General response 1, which are trained on short and long tables, respectively. The longer tables improve generation utility from 0.903 to 0.917 (better on 40 of 48 tables), and the gain concentrates on large tables (17 of the 18 tables with more than 8,192 training rows). We have added this ablation to Appendix M.1

W5 / Q5. Beyond the three evaluated tasks. We additionally evaluate attribute-conditioned generation, where the condition is an arbitrary set of feature columns, on the 30 tables with at most 8,192 training rows. For each table, we hold out 20% of the training rows as queries and use the rest as the context, or as the training data of per-table models. For each query row, we give the values of a random subset C of the features, i.e., |C| = ⌈\lceil F/4 ⌉\rceil or ⌈\lceil F/2 ⌉\rceil, where F is the number of features of the dataset, and generate all remaining columns including the label. We compare TabMosaic with the best performing baselines, i.e., TabPFN-3, TabICLv2, TabNAT, and ForestDiffusion, and with a reference that samples each remaining column independently from its marginal. We use the same four metrics and Elo as in our generation benchmark.

Table E. Attribute-conditioned generation on the 30 tables with at most 8,192 training rows. We report Elo with 95% intervals.

Model|C| = ⌈\lceil F/4 ⌉\rceil|C| = ⌈\lceil F/2 ⌉\rceil
TabMosaic (ours)1093 [1054, 1132]1089 [1049, 1131]
TabPFN-31090 [1053, 1140]1080 [1044, 1114]
TabICLv21017 [988, 1047]1051 [1020, 1091]
TabNAT978 [945, 1014]972 [939, 1002]
ForestDiffusion972 [939, 998]962 [934, 995]
Independent marginals849 [785, 908]844 [779, 904]

TabMosaic ranks first at both condition sizes, on par with TabPFN-3 and clearly ahead of the per-table generators. We thank the reviewer for this suggestion, which extends our evaluation to attribute-conditioned generation, and we have added these results to Appendix L.

Thank you for the comprehensive clarifications, additional ablation study and evaluations. I appreciate the authors' thorough efforts in revising the manuscript.

Reviewer_LzCSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed a scheme called TabMosaic, which is a unified in-context diffusion model to formulate the tabular generative task as a single problem. It is specifically designed for generating mixed-type target cells in unseen, potentially incomplete table without table-specific training. The author shows that TabMosaic reaches the first on tested generative tasks across 48 real-world table.

Critical strengths
  1. The topic of this paper is meaningful which brings to the real-life problem. And the author provides a good wild range evaluation with lead to a better results compare with all the baseline models.

  2. The author provides a unified framework for data generation, missing-value imputation, and conditional generation. The same model can handle new tables without additional training procedures, making it practical for applications involving different dataset environment.

Critical weaknesses
  1. The paper claims of ranking need a little bit more explanations while the confidence intervals overlap with strong baselines. Meanwhile, the table 5 also reports some better results from baseline like TabNAT, It is better to have more explain on this part.

  2. The paper build experiments but it didn’t clearly show how much improvement comes from joint diffusion itself. Stage 2 changes several factors at the same time like context length, This makes it difficult to identify the contribution of each component.

Questions for authors
  1. Can the author provide an ablation study to show how much improvement comes from joint diffusion itself, using the same model backbone and other parameters?

  2. It is better if the author can report the generation and class-conditional generation results on the same datasets completed by all compared methods. Does TabMosaic still rank first without replacing missing results with reference scores?

  3. Why are the Elo values in Fig. 2 different from those in the main text and appendix tables? Typo? Could the authors clarify which values are correct? Thanks.

2 comments

We thank Reviewer LzCS for the careful reading and the concrete questions.

W1. Confidence intervals overlap with strong baselines. We follow the Elo procedure of TabArena to rank the methods, and our claim is based on the Elo point estimates, by which TabMosaic ranks first on all three tasks. However, as the reviewer points out, the 95% intervals of TabMosaic overlap with those of the strongest baselines on generation and class-conditional generation, i.e., TabNAT and TabPFN-3, respectively. We have stated this in Section 5.2 of the revised paper. For the per-metric results, please see General response 4. In short, TabMosaic and TabNAT trade wins on generation metric. TabMosaic has the highest utility, whereas the per-table models are better on coverage, and TabNAT also on most column-level distances, since they are trained on each dataset and fit its support more tightly.

W2 / Q1. Contribution of joint diffusion. Please see General response 2, which replaces only the loss with the cross-entropy objective on the same backbone. In short, the advantage of our joint objective grows as more cells are generated jointly and as the targets include continuous features.

W3. Stage 2 changes several factors. Please see General response 1, which separates the longer tables, context missingness, and multi-corruption by comparing the Stage-1 checkpoint and four Stage 2 variants. In short, context missingness trades a slight loss on generation and class-conditional generation for a large gain on imputation, and multi-corruption yields a consistent gain in all six comparisons.

Q2. Rankings on the tables completed by all methods. Please see General response 3. In short, among the top-5 methods of each task on the datasets that all of them complete, TabMosaic still ranks first on all three tasks without any score replacement.

Q3. Inconsistent Elo values. Thank you for catching this, and we apologize for the confusion. The submitted version reported an earlier snapshot of our results. We have updated Fig. 2, the main text, and the appendix to the final results, in which TabMosaic still ranks first on all three tasks, on par with TabNAT on generation and with TabPFN-3 on class-conditional generation. Compared with Fig. 2 of the submitted version, the only change in ranks is that ForestDiffusion and TabPFN-3 swap the 5th and 6th places in generation.

AuthorsOpenReview ↗

Gentle reminder

Dear Reviewer LzCS,

Thank you again for your time and thoughtful review. As the discussion period is coming to a close, we would like to kindly check whether our response has addressed your concerns. If you have any further questions or comments, we would be happy to address them.

Best regards,

Authors

Reviewer_XSyUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TabMosaic, an in-context diffusion model that can perform tabular generation, imputation, and class-conditional generation. By combining mixed-type diffusion with a pretrained tabular transformer, the model learns to generate arbitrary subsets of cells conditioned on observed cells and a context table. This allows the model to generalize in-context without dataset-specific training. The authors provide theoretical connections to Bayesian posterior prediction and demonstrate strong performance across 48 real-world datasets.

Critical strengths
  1. Modeling arbitrary target subsets through conditional diffusion provides a simple and flexible framework for multiple tabular generative tasks. The ability to handle unseen tables and arbitrary missing patterns without fine-tuning is particularly appealing.
  2. The experiments cover a very broad range of datasets, tasks, and baselines. TabMosaic consistently achieves competitive or superior performance, especially for imputation. The comparison against exact posterior predictive distributions on synthetic priors also strengthens the empirical evidence.
Critical weaknesses
  1. Limited novelty. The method largely combines established techniques from mixed-type tabular diffusion and in-context tabular learning. While the integration is useful, the individual components are not particularly novel.
  2. The ablation study is insufficient to isolate the contributions of the proposed training techniques. Stage 2 simultaneously introduces longer contexts, missingness augmentation, additional training, and multi-corruption sampling. It remains unclear which components are responsible for the performance improvements, particularly whether multi-corruption sampling provides meaningful benefits over standard single-corruption training under a matched computational budget.
Questions for authors

See weaknesses.

2 comments

We thank Reviewer XSyU for the positive assessment and the helpful suggestions.

W1. Limited novelty. Our contribution lies in how we combine diffusion and in-context learning, i.e., i) we formulate generation, imputation, and conditional generation as a single query for the joint conditional of an arbitrary target set in an unseen table, ii) we show that a joint diffusion objective is minimized by the posterior predictive for every target set and bounds the conditional NLL (Theorem 1), and iii) we realize this as a single in-context model that needs no dataset-specific training. General response 2 shows that the joint objective matters, compared with the single-column cross-entropy objective used by existing tabular foundation models with the same backbone.

W2. Ablation of Stage 2. Please see General response 1, which separates context missingness and multi-corruption. In short, context missingness trades a slight loss on generation and class-conditional generation for a large gain on imputation, and multi-corruption yields a consistent gain in all six comparisons.

Thank you for the detailed clarification of the novelty and the ablation of the method components. They help me better interpret the paper's contribution, especially the empirical evidence. I'm convinced and will maintain my positive assessment.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General responses 3 and 4

General response 3. Rankings on the datasets completed by all methods

We rank the methods with the Elo procedure of TabArena. When a method fails on a table, we assign it the score of a reference baseline on that table, or, in class-conditional generation, which has no reference baseline, a score just below the worst method.

To separate execution coverage from generative quality, we compare the top-5 methods of each task in Fig. 2 on the datasets that all of them complete, without any score replacement. We focus on the strongest methods because several lower-ranked baselines fail on large tables, and requiring them to complete every table would mostly remove the large tables from the comparison.

Table C. Elo among the top-5 methods of each task in Fig. 2, on the datasets that all of them complete.

RankGeneration (44 tables)Imputation (47 tables)Class-cond. (31 tables)
1TabMosaic 1036 [1006, 1063]TabMosaic 1293 [1248, 1361]TabMosaic 1034 [1001, 1067]
2TabNAT 1024 [997, 1051]TabICLv2 1040 [1007, 1079]TabPFN-3 1026 [999, 1053]
3ForestDiff. 998 [967, 1028]ReMasker (tuned) 960 [927, 996]ForestDiff. 1003 [971, 1034]
4TabDiff 978 [948, 1011]DiffPuter 919 [879, 953]TabNAT 982 [948, 1024]
5TabSyn 963 [937, 989]ReMasker (default) 899 [857, 937]TabEBM 955 [911, 999]
6–HyperImpute 886 [849, 915]–

TabMosaic ranks first on all three tasks. On imputation, the margin is large (1293 vs. 1040 for TabICLv2). On generation and class-conditional generation, TabMosaic is comparable to TabNAT (1036 vs. 1024) and TabPFN-3 (1034 vs. 1026), respectively, whose intervals overlap with ours. Therefore, our conclusions do not depend on the score replacement.

General response 4. Metric-level trade-offs

Table D reports the number of datasets on which TabMosaic and the strongest baseline, TabNAT, win against each other for each generation metric.

Table D. TabMosaic vs. TabNAT on generation (48 tables)

MetricTabMosaic avg.TabNAT avg.Δ avg.TabMosaic winsTabNAT wins
Utility0.9140.899+0.0152622
α-precision0.7900.809−0.0192226
β-recall0.4080.424−0.0162523
Authenticity0.5690.558+0.0112423

The two methods trade wins on every metric. Since the four metrics measure different aspects of synthetic data and a single metric can be misleading due to its corner cases, we rank the generators by the four metrics jointly.

Across the full results of the three tasks (Tables 7–9 of the revised paper), the trade-off is consistent. i) On generation and class-conditional generation, TabMosaic has the highest utility (0.914 and 0.926), whereas the per-table models are better on coverage, and TabNAT also on most column-level distances, since they are trained on each dataset and fit its support more tightly. ii) On imputation, TabMosaic is the best on every metric, i.e., error, RMSE, categorical error, and downstream utility.

AuthorsOpenReview ↗

General responses 1 and 2

General response 1. Stage 2 ablation

Tabular foundation models with staged training, e.g., TabICL, enlarge the tables from one stage to the next. Following this practice, we ablate the two components that our Stage 2 adds, i.e., context missingness and multi-corruption. The base model is therefore trained without context missingness and with a single corruption. All Stage 2 variants are trained for 10k steps from the same Stage-1 checkpoint and compared, together with the Stage-1 checkpoint, by Elo among the five, on the same tables as in the main text.

Table A. Stage 2 ablation: Elo among the five models

ModelGeneration (48 tables)Imputation (48 tables)Class-cond. (36 tables)
Stage 1943 [909, 981]838 [790, 890]910 [866, 949]
Stage 2 Base1024 [1001, 1046]820 [767, 863]1034 [1001, 1065]
+ context missingness998 [975, 1022]1220 [1165, 1279]992 [959, 1018]
+ multi-corruption1026 [1001, 1052]860 [814, 899]1043 [1015, 1069]
+ context missingness & multi-corruption (ours)1008 [982, 1037]1265 [1195, 1331]1024 [992, 1060]

We find that: i) The longer tables of Stage 2 improve generation and class-conditional generation over Stage 1 (+81 and +124), but not imputation (838 vs. 820). ii) Context missingness introduces a trade-off. It slightly lowers the Elo on generation and class-conditional generation, but it substantially improves imputation (+400 and +405). Across the three tasks, the models trained with context missingness are therefore the most balanced, with the highest average Elo (1099 for ours and 1070 without multi-corruption, against 959 for the base model and 897 for Stage 1). iii) Multi-corruption yields a consistent gain in all six comparisons (+2 to +45), both with and without context missingness.

We have added this ablation to Appendix M.1.

General response 2. Joint diffusion objective vs. cross-entropy objective

To isolate our proposed joint diffusion objective, we compare it with the cross-entropy (CE) objective of existing tabular foundation models, e.g., TabPFN and TabICL. In the small-scale setting of Fig. 3, we train two models that differ only in the loss, i.e., i) our proposed model and ii) a CE variant trained to predict a single target cell given the others. We evaluate both on three priors with an exact posterior predictive distribution (PPD). The Dirichlet (categorical only) and GP (continuous only) priors are the same as in the paper. The latent-class prior mixes both types, and its cells depend on each other through a per-row latent class, so the target conditional cannot be factorized in any column order. Table B reports the energy distance to the exact PPD (lower is better), where |S| is the number of target cells in a query row and the MC floor is the distance attained by samples from the exact PPD.

Table B. Energy distance to the exact PPD (×10−3\times 10^{-3}, lower is better)

Prior|S|MC floorOursCE
Dirichlet (categorical)10.643.420.88
21.513.372.35
32.323.322.98
4 (all)2.943.213.27
GP (continuous)10.232.855.92
23.145.6677.34
3 (all)4.805.519.87
Latent-class (mixed)10.704.312.96
21.304.904.07
31.705.284.76
42.684.595.42
5 (all)3.694.957.32

We find that: i) On the Dirichlet prior, CE is better when fewer cells are generated, i.e., |S| ≤\le 3. However, when all cells are generated, our proposed model is on par with CE. ii) On the GP prior, our proposed model outperforms CE in all settings. The gap is largest at |S| = 2 (5.66 vs. 77.34), where the two target cells are tightly coupled through x3=f(x1,x2)x_3 = f(x_1, x_2). iii) On the latent-class prior, as fewer features are given as the condition, i.e., |S| = 4 and 5, our proposed model approaches the MC floor, whereas CE does not.

In other words, the advantage of the joint diffusion objective grows as more cells are generated jointly and as the targets include continuous features.

We have added this comparison to Appendix J.

AuthorsOpenReview ↗

General responses

We thank all reviewers for their thoughtful and constructive feedback.

In response to the reviews, we have conducted the following additional experiments:

  1. A Stage 2 ablation that separates context missingness and multi-corruption (General Response 1).
  2. A controlled comparison of our joint diffusion objective with the cross-entropy objective on the same backbone (General Response 2).
  3. Attribute-conditioned generation, where the condition is an arbitrary set of feature columns (Response to LZH6).

We have also provided the following additional analyses of our main results:

  1. Rankings among the top-5 methods on the tables that all of them complete, without score replacement (General Response 3).
  2. Per-metric win counts against the strongest baseline and the metric-level trade-offs (General Response 4).

We have uploaded a revised paper, in which all changes are marked in blue. In the revision, we corrected the inconsistent Elo values between Fig. 2, the main text, and the appendix, and added the Stage 2 ablation (Appendix M.1), the comparison with the cross-entropy objective (Appendix J), and attribute-conditioned generation (Appendix L).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.