acceptodds
Withdrawn from ICLR 2027

TabGift: A Context-Efficient Foundation Model for Tabular Data Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Tabular data generation aims to synthesize realistic samples from limited observations while faithfully capturing the underlying data distribution. Tabular foundation models (TFMs), pretrained on diverse tabular tasks and adapted through in-context learning, have demonstrated strong predictive capabilities, suggesting the potential of extending this paradigm to tabular data generation. However, the strong predictive performance of TFMs often relies on sufficiently large context sets, whereas tabular generation is typically needed precisely when target data are scarce. Extending the foundation-model paradigm to generation therefore requires reliable distribution inference from only a limited context. To this end, we propose TabGift, a context-efficient tabular foundation model enabling both efficient distribution modeling and effective generation from only a few context samples. TabGift maps heterogeneous tables into a shared latent space with a pretrained tabular encoder, where an in-context diffusion model learns to denoise latent samples conditioned on a small set of clean examples. This design transfers generative priors across datasets and enables distribution inference from limited context, while a lightweight dataset-specific MLP decoder maps generated representations back to the original data space. Experiments on 7 classification and 11 regression datasets show that TabGift consistently outperforms representative baselines in synthetic data quality, with particularly strong gains under limited-context settings, while achieving substantially faster adaptation and generation.

Primary area:
generative models
Keywords:
Tabular Date Generation, Tabular Foundation Model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qFLPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

TabGift generates tabular data from a few samples. It encodes rows with a frozen SwiftPFN encoder, runs an in-context diffusion model in this shared latent space with the real rows as context, and maps the generated latents back with a small MLP decoder trained per dataset. The denoiser is pretrained on synthetic tables and then on OpenML datasets. On 18 TALENT datasets with 16 to 512 training rows, it gives the best downstream utility at small sizes and takes about 10 seconds on Adult

Critical strengths

S1: The paper brings the tabular foundation model idea to data generation in the setting where generation is most needed, with only 16 to 64 rows. The design, a frozen encoder with a shared in-context denoiser and a small decoder, is simple and lets one denoiser handle tables with different schemas.

S2: The method is fast. Total time is about 10 seconds on Adult and hardly depends on the number of rows on Table 2. It is faster than TabEBM, TabDDPM, CTGAN, TVAE and NFlow, and only the small decoder has to be trained per dataset.The authors also report that the advantage shrinks at larger sizes and that privacy is weaker, instead of hiding these results.

Critical weaknesses

W1: The evaluation setup weakens the main comparison. (a) The reported pretraining and evaluation dataset lists overlap. The second pretraining stage uses real datasets from OpenML-CC18 and CTR23 (Appendix A.1, Tables 3 and 4), and several of the 18 test datasets also appear there, namely adult and pc3 (Table 4 and Table 5) and superconductivity and kin8nm (Table 3 and Table 6). Even with different splits, the denoiser may have learned these joint distributions, which would inflate exactly the small sample results the paper focuses on. (b) The most relevant baselines are missing. The related work cites other foundation model or in-context generators (TabPFGen, TabFORGE, DiffICL and CTSyn) and strong diffusion models (TabSyn and TabDiff), but only TabEBM is compared. LLM based few-shot generators designed for small data, such as GReaT or CLLM, are also missing.

W2: Several claims go beyond the evidence. First, the context efficiency in the title is not tested. No experiment changes the context size at a fixed number of rows, compares with a long context in-context generator, or measures compute against context length, and the first contribution, "showing that long-context dependence fundamentally conflicts with the data-scarce settings", is stated but not shown. There are also no quantitative ablations of the second pretraining stage, the encoder, the decoder or the number of sampling steps, and the reconstruction error of the encoder and decoder alone is not reported, although the encoder was trained for prediction and is fed all zero labels and a dummy test row here. Second, the abstract says TabGift "consistently outperforms representative baselines", but in Table 1 it is below TabEBM, BN and SMOTE in regression at 256 and 512 rows, for example 0.8359 against 0.9772, 0.9722 and 0.9192 at 512, and below SMOTE, BN and TabEBM in classification at 512. At small sizes the margins are within one standard deviation, for example 1.0729 ± 0.1026 against 1.0542 ± 0.0939 for TabEBM at 16 rows, and no significance test is given. Finally, the conclusion says the method addresses "poor privacy protection", but Section 5.3.2 and Figure 4 show it among the weaker methods on privacy, with delta presence around 0.8. With 16 rows, an MLP decoder trained on those same rows may come close to copying training records.

W3: There are several inconsistencies and presentation issues. (a) Section 5.3.1 first says downstream models use synthetic data generated "on top of the training data", which reads as augmentation, and then defines the metric as the performance of models trained on synthetic data, which reads as training on synthetic data only. It is unclear which protocol is used and how many synthetic rows are generated. (b) Section 4.1 says 72 real datasets are used, but Tables 3 and 4 list about 100. Section 5.1 says 7 classification datasets remain after removing those with more than 50 features, but Table 5 lists 9, and 8 of them meet this rule. Appendix A.2 points to Tables 4 and 3 for the TALENT statistics, which are in Tables 5 and 6. (c) Several references have no authors, such as those cited as dpg (2025), cor (2025) and lim (2026), and some references are off topic for tabular generation, such as a fairness survey and a report on Amazon's recruiting tool. Figure 3 shows raw pretraining loss curves, and that space would be better used for ablations. Table 1 marks the top three results only with red, orange and blue text, and Figure 4 uses overlapping colored ellipses with small labels.

Questions for authors

Q1 (see W1): Please confirm whether adult, pc3, superconductivity, kin8nm or any other test dataset was used in the second pretraining stage, and report Table 1 and Figure 4 again without these datasets, or with a model pretrained without them. Can the authors also add, under the same protocol, at least one of TabPFGen, DiffICL or CTSyn, one LLM based few-shot generator such as CLLM or GReaT, and TabSyn?

Q2 (see W2 ): Please add an experiment that isolates context efficiency, for example generation quality and runtime as the context size changes at a fixed number of rows, compared with an in-context generator that does not compress into a latent space. Please also add ablations of the second pretraining stage and of the decoder, report the reconstruction error of the encoder and decoder, and add significance tests for Table 1, for example a Wilcoxon test across datasets.

Q3 (see W2 and W3): Is the downstream metric computed by training on synthetic data only or on real plus synthetic data, and how many synthetic rows are used? How is SMOTE used for regression? With 16 rows, what share of generated rows are very close to a training row, compared with SMOTE?

Q4 (see W3): I suggest fixing the incomplete references and the table references, marking the top results in Table 1 with bold or underline instead of color only, and redrawing Figure 4 so that the methods do not overlap and can be distinguished without relying on color.

Reviewer_Ek2SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TABGIFT, a context-efficient foundation model for tabular data generation under limited-data settings. TABGIFT maps heterogeneous tables into a shared latent space using a pretrained SwiftPFN encoder and employs an in-context diffusion model to generate latent representations conditioned on a small set of observed samples. A lightweight dataset-specific MLP decoder maps the generated representations back to the original data space, while the encoder and diffusion model remain frozen during adaptation. Experiments on 18 classification and regression datasets show that TABGIFT achieves strong synthetic-data quality under limited-context settings while reducing target-specific adaptation and generation costs relative to the evaluated baselines.

Critical strengths
  • The paper addresses an important problem in tabular data generation. Generating useful synthetic data from a small number of observations is practically relevant, and the paper clearly identifies the tension between the long-context dependence of existing tabular foundation models and the limited-data settings in which synthetic data are most needed.

  • The paper is concise and generally well written. The motivation, architecture, and experimental setup are presented in a clear and logically organized manner, making the main ideas relatively easy to follow.

  • The related-work discussion is comprehensive and well organized. The paper covers conventional tabular generators, recent tabular foundation models, and emerging foundation-model-based generation approaches, providing useful context for positioning TABGIFT.

Critical weaknesses
  • The ablation analysis does not sufficiently validate the main design choices of TABGIFT. The experiments vary the context size but do not isolate the contributions of the SwiftPFN encoder, the proposed sample-wise attention mask, the two-stage pretraining strategy, or the dataset-specific MLP decoder. The paper mentions that synthetic-only pretraining, a universal decoder, and joint encoder–decoder training performed worse, but does not report quantitative comparisons.

  • The empirical comparison omits a closely related recent method. Although the paper discusses and cites ReTabSyn (ICML 2026), it is not included as an experimental baseline. Both methods address tabular generation under limited-data settings and evaluate the downstream utility of synthetic data, despite using different technical approaches. A direct comparison would provide a stronger assessment of TABGIFT’s effectiveness in the target setting.

  • The main empirical results rely heavily on aggregated relative performance. Averaging across datasets and downstream predictors makes it difficult to determine whether TABGIFT consistently improves performance across individual datasets or whether the overall gains are driven by a subset of favorable cases. Dataset-level win rates, paired statistical tests, and a more detailed analysis of failure cases would provide stronger support for the claims of consistent improvement.

  • The fidelity and privacy analysis is limited to the smallest context size. The reported analysis at context size 16 indicates that TABGIFT performs reasonably well in statistical fidelity but is somewhat weaker in privacy preservation. Because context efficiency is central to the paper, it is important to understand whether this fidelity–privacy trade-off changes as the amount of target data increases.

  • The efficiency analysis is based on a limited runtime setting. The comparison primarily considers generating 1,024 samples from the Adult dataset. This does not establish how runtime and memory scale with context length, feature dimensionality, or the number of generated samples, even though computational efficiency is a central claim of the paper.

Questions for authors
  • Could the authors provide component-wise ablations for the SwiftPFN encoder, sample-wise attention mask, two-stage pretraining strategy, and dataset-specific decoder?

  • Could the authors include a comparison with ReTabSyn (ICML 2026), or clarify why it is not a suitable baseline for the limited-data experiments?

  • Are TABGIFT’s improvements statistically significant and consistent across individual datasets, rather than primarily reflected in the aggregated mean results?

  • How do statistical fidelity and privacy risk change across the evaluated context sizes?

  • How do TABGIFT’s runtime and memory requirements scale with context length, feature dimensionality, and the number of generated samples?

Reviewer_bRBHOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TabGIFT, a latent diffusion model targeting small-dataset generation. TabGIFT uses SwiftPFN, a pretrained TFM, as a frozen encoder that maps each row of a table into a shared latent space. An in-context diffusion model takes the latent representations of the target table as context and generates new latents, which a dataset-specific MLP decoder maps back to the original data space. The diffusion model is trained in two stages: i) on synthetic data generated from the SCM prior of TabICL, and ii) on 72 real-world datasets. The authors compare TabGIFT with eight baselines on 18 real-world datasets and claim that it achieves superior synthetic data quality, especially in the low-data regime, while maintaining fast adaptation and generation speed.

Critical strengths
  1. The motivation is reasonable, i.e., data generation is most needed when only a small amount of data is available.
  2. The generation cost is low: adaptation to a new dataset takes about 10 seconds.
Critical weaknesses
  1. Several real-world datasets used in the second training stage (e.g., Adult, PC3, and kin8nm) are also used for evaluation. The authors should exclude the evaluation datasets from training or report the results without the overlapping datasets.
  2. Since this paper targets small datasets, where baselines that require dataset-specific training, e.g., CTGAN, TVAE, TabDDPM, tend to memorize the limited training set and fail to perform properly, the comparison with in-context models is more important than that with per-dataset trained baselines. Tabular foundation models such as TabPFN and TabICL also support unsupervised generation (example code is provided in their official GitHub repositories). However, the comparison with these TFMs is missing.
  3. Evidence is needed that the advantage in the small-data regime comes from the actual generation quality. i) In Fig. 4, TabGIFT shows a lower DCR than SMOTE, which interpolates between real data samples, i.e., TabGIFT provides weaker privacy protection than SMOTE. ii) In Table 1, as the dataset size increases, the performance of TabGIFT degrades, similar to the other dataset specific baselines, and TabGIFT's degradation is the largest. These observations suggest that the dataset-specific MLP decoder, trained on small training sets (e.g., 16 or 32 rows), may memorize the training data. An evaluation of whether the generated samples are nearly identical to the training samples, e.g., in terms of diversity, is needed.
Questions for authors
  1. Since TabEBM is designed for class-conditional generation, could the authors clarify how it was applied to the regression datasets?
  2. Could the authors provide more details on how the dataset-specific MLP decoder is trained, e.g., the number of epochs and the learning rate?
  3. Could the authors provide the details of the inference setting of TabGIFT, e.g., the number of denoising steps, σ_max, and σ_min?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.