- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
TabGift generates tabular data from a few samples. It encodes rows with a frozen SwiftPFN encoder, runs an in-context diffusion model in this shared latent space with the real rows as context, and maps the generated latents back with a small MLP decoder trained per dataset. The denoiser is pretrained on synthetic tables and then on OpenML datasets. On 18 TALENT datasets with 16 to 512 training rows, it gives the best downstream utility at small sizes and takes about 10 seconds on Adult
Critical strengths
S1: The paper brings the tabular foundation model idea to data generation in the setting where generation is most needed, with only 16 to 64 rows. The design, a frozen encoder with a shared in-context denoiser and a small decoder, is simple and lets one denoiser handle tables with different schemas.
S2: The method is fast. Total time is about 10 seconds on Adult and hardly depends on the number of rows on Table 2. It is faster than TabEBM, TabDDPM, CTGAN, TVAE and NFlow, and only the small decoder has to be trained per dataset.The authors also report that the advantage shrinks at larger sizes and that privacy is weaker, instead of hiding these results.
Critical weaknesses
W1: The evaluation setup weakens the main comparison. (a) The reported pretraining and evaluation dataset lists overlap. The second pretraining stage uses real datasets from OpenML-CC18 and CTR23 (Appendix A.1, Tables 3 and 4), and several of the 18 test datasets also appear there, namely adult and pc3 (Table 4 and Table 5) and superconductivity and kin8nm (Table 3 and Table 6). Even with different splits, the denoiser may have learned these joint distributions, which would inflate exactly the small sample results the paper focuses on. (b) The most relevant baselines are missing. The related work cites other foundation model or in-context generators (TabPFGen, TabFORGE, DiffICL and CTSyn) and strong diffusion models (TabSyn and TabDiff), but only TabEBM is compared. LLM based few-shot generators designed for small data, such as GReaT or CLLM, are also missing.
W2: Several claims go beyond the evidence. First, the context efficiency in the title is not tested. No experiment changes the context size at a fixed number of rows, compares with a long context in-context generator, or measures compute against context length, and the first contribution, "showing that long-context dependence fundamentally conflicts with the data-scarce settings", is stated but not shown. There are also no quantitative ablations of the second pretraining stage, the encoder, the decoder or the number of sampling steps, and the reconstruction error of the encoder and decoder alone is not reported, although the encoder was trained for prediction and is fed all zero labels and a dummy test row here. Second, the abstract says TabGift "consistently outperforms representative baselines", but in Table 1 it is below TabEBM, BN and SMOTE in regression at 256 and 512 rows, for example 0.8359 against 0.9772, 0.9722 and 0.9192 at 512, and below SMOTE, BN and TabEBM in classification at 512. At small sizes the margins are within one standard deviation, for example 1.0729 ± 0.1026 against 1.0542 ± 0.0939 for TabEBM at 16 rows, and no significance test is given. Finally, the conclusion says the method addresses "poor privacy protection", but Section 5.3.2 and Figure 4 show it among the weaker methods on privacy, with delta presence around 0.8. With 16 rows, an MLP decoder trained on those same rows may come close to copying training records.
W3: There are several inconsistencies and presentation issues. (a) Section 5.3.1 first says downstream models use synthetic data generated "on top of the training data", which reads as augmentation, and then defines the metric as the performance of models trained on synthetic data, which reads as training on synthetic data only. It is unclear which protocol is used and how many synthetic rows are generated. (b) Section 4.1 says 72 real datasets are used, but Tables 3 and 4 list about 100. Section 5.1 says 7 classification datasets remain after removing those with more than 50 features, but Table 5 lists 9, and 8 of them meet this rule. Appendix A.2 points to Tables 4 and 3 for the TALENT statistics, which are in Tables 5 and 6. (c) Several references have no authors, such as those cited as dpg (2025), cor (2025) and lim (2026), and some references are off topic for tabular generation, such as a fairness survey and a report on Amazon's recruiting tool. Figure 3 shows raw pretraining loss curves, and that space would be better used for ablations. Table 1 marks the top three results only with red, orange and blue text, and Figure 4 uses overlapping colored ellipses with small labels.
Questions for authors
Q1 (see W1): Please confirm whether adult, pc3, superconductivity, kin8nm or any other test dataset was used in the second pretraining stage, and report Table 1 and Figure 4 again without these datasets, or with a model pretrained without them. Can the authors also add, under the same protocol, at least one of TabPFGen, DiffICL or CTSyn, one LLM based few-shot generator such as CLLM or GReaT, and TabSyn?
Q2 (see W2 ): Please add an experiment that isolates context efficiency, for example generation quality and runtime as the context size changes at a fixed number of rows, compared with an in-context generator that does not compress into a latent space. Please also add ablations of the second pretraining stage and of the decoder, report the reconstruction error of the encoder and decoder, and add significance tests for Table 1, for example a Wilcoxon test across datasets.
Q3 (see W2 and W3): Is the downstream metric computed by training on synthetic data only or on real plus synthetic data, and how many synthetic rows are used? How is SMOTE used for regression? With 16 rows, what share of generated rows are very close to a training row, compared with SMOTE?
Q4 (see W3): I suggest fixing the incomplete references and the table references, marking the top results in Table 1 with bold or underline instead of color only, and redrawing Figure 4 so that the methods do not overlap and can be distinguished without relying on color.