acceptodds
Under review as a conference paper at ICLR 2027

TabTailor: Tail and Ordinal-Aware Mixed-Type Diffusion Model for Tabular Data Synthesis

Abstract

Generative models have significantly enhanced the quality of synthetic tabular data and have been applied to diverse real-world domains. However, current tabular generative models lack the ability to preserve tails of numerical features and they consider ordinal features as unordered. Therefore, preserving heavy-tailed numerical distribution and the ordering of ordinal attributes remains challenging. We propose TabTailor, a mixed-type hybrid diffusion framework that aims to preserve the tail and ordinal semantics of the tabular data. For numerical variables, TabTailor uses an adaptive semiparametric marginal transform to model each feature’s tails before the fixed cosine VP diffusion process. The categorical variables employ masking budgets and learnable per feature-and-state-dependent cor- ruption schedules. TabTailor also uses a cumulative semantic embedding and a strictly increasing rank coordinate to represent ordinal attributes as an embedding, called CMOE (Cumulative Monotonic Ordinal Embedding). A denoising back-bone is implemented using a bidirectional transformer for modelling numerical, nominal and ordinal side information jointly, and a state-consistent mixed-type stochastic-restart sampler is used to mitigate decoding error accumulation. The denoiser and the learnable corruption schedule are trained jointly under one diffusion objective. TabTailor performs better than state-of-the-art baselines in seven benchmark datasets and ten metrics while maintaining overall preservation of the numerical tails and ordinal structure.

Primary area:
generative models
Keywords:
Tabular data synthesis, Tail preservation, Stochastic restart, Rare-states, Continuous-time diffusion, Learnable schedule

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_xdxzOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TABTAILOR, a mixed-type tabular diffusion model designed to better preserve heavy-tailed numerical features and ordered categorical features. It combines a semiparametric marginal transform (ATI) for numerical tails, a cumulative monotonic ordinal embedding (CMOE) with state-dependent masking schedules for categories, and a stochastic-restart sampler. The paper compares TABTAILOR against prior tabular generators, primarily TabSyn and TabDiff, across seven standard tabular datasets (such as Adult, Default, and Shoppers).

Critical strengths

(1) The paper targets two genuinely important blind spots in tabular synthesis—heavy-tailed numerical behavior and ordinal structure. Tackling tail fidelity and ordinal semantics addresses real, under-explored failure modes in tabular synthesis and is practically important in domains with extremes and ordered categories (finance, healthcare, e-commerce).

(2) The approach could complement or improve upon recent diffusion or flow tabular generators that often compress tails and ignore ordinal structure.

Critical weaknesses

(1) My main concern is that the evidence does not yet establish the contribution. The paper defines Tail-AD, Tail-JSD, and ordinal rank Wasserstein, but I could not find quantitative results for these measures. The distribution plots are encouraging, although they mainly show agreement with training data. I would have expected held-out tail and ordinal results to be central to the evaluation.

(2) The ordinal example is limited. Figure 3 uses education and education-num from Adult, which encode the same underlying attribute. TabDiff already appears close to the real distribution here, so this example does not clearly demonstrate the additional benefit of the proposed ordinal modeling.

(3) The status of the broader experiments are difficult for me to follow. Sec 4.3–Sec 4.4 and the conclusion describe benchmark results as awaiting completion, while Appendix I contains TABTAILOR results. The comparison values for TabSyn and TabDiff are taken from earlier work. These inconsistencies need to be resolved.

(4) Several components are introduced together, but the ablation section does not report their individual effects. In particular, I do not understand how much of the tail improvement comes from the fitted marginal transform and how much requires the changes to diffusion and sampling. A completed application study, such as the proposed Ann-Thyroid experiment, would also help show why these improvements matter beyond the distribution plots.

Questions for authors
  1. Could you reconcile the pending experiments in Sections 4.3–5 with Appendix I, and provide the missing tail/ordinal metrics, raw-versus-calibrated comparisons, and Ann-Thyroid results?

  2. How much of the improvement comes from ATI’s fitted tails? Could you compare against empirical-only ATI and sampling without restarts?

  3. Beyond Adult’s two encodings of education, does CMOE improve on treating ordinal features as nominal categories, and how does it compare with TabDiff?

Reviewer_an3SOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes TabTailor, a diffusion-based framework that uses specific encodings to better accommodate heavy-tailed numerical features or ordinal categorical features. The model applies existing techniques on masked diffusion to learn the categorical part of the joint distribution. Compared to existing work, the authors also utilize a transformer-based denoiser. In combination, the results show gains mostly in the quality of the marginal distributions.

Critical strengths
  • Extending models to include more prior information on the nature of features, such as heavy-tailed distributions and or ordered structure, can be useful for tabular data. I expect generative models to generally perform better the more useful inductive biases are introduced to them.
Critical weaknesses
  • The claims about existing models being bad at tails of distributions or ordered categorical features is interesting but no evidence for that claim is presented.
  • The writing quality and clarity is lacking.
  • There is no AI usage statement.
  • There is no results table in the main text and no uncertainty around metrics in Tables 7, 8, 9. Also, taking historical numbers from previous papers is very unusual for tabular data generation.
  • The comparison to baselines is unfair: TabTailor uses a much more powerful, transformer-based backend than the baselines (e.g., TabDDPM uses only an MLP, TabDiff uses a two-layer transformer but only for encoding). Additionally, the denoiser utilizes Fourier features, which are known to allow for more detailed modeling of numerical features [1]. Hence, results are very likely primarily driven by differences in architectures. There is no evidence on whether the proposed ATI, tail-based loss, etc. add anything above using a more complex denoiser.
  • Evaluation on seven datasets is not enough for tabular data generation, as tabular data is very heterogeneous.
  • Reported C2ST score (which is the only metric that evaluates the joint distribution) is based on a logistic regression (which is not reported in this paper but I had to look this up in the TabDiff paper). This is known to be insufficient to evaluate the details of the generated tabular data, as the logistic classifier is easy to satisfy (see [2] and [3]).
  • Related work and baselines do not include TabRep (addresses ordinal features, [4]) nor TabCascade (addresses details of numerical feature distributions, including heavy tails, [5]).
  • The authors mention adding the Ann-Thyroid dataset in addition to the TabDiff datasets. However, this dataset does not appear anywhere in the paper.
  • The ablations mentioned in section 4.4 are non-existent.

[1] On Embeddings for Numerical Features in Tabular Deep Learning. https://arxiv.org/abs/2203.05556

[2] A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Science. http://arxiv.org/abs/2403.12636

[3] A Sobering Look at Tabular Data Generation via Probabilistic Circuits. https://arxiv.org/abs/2603.23016

[4] TabRep: Training Tabular Diffusion Models with a Simple and Effective Continuous Representation. https://arxiv.org/abs/2504.04798

[5] Cascaded Flow Matching for Heterogeneous Tabular Data with Mixed-Type Features. https://arxiv.org/abs/2601.22816

Questions for authors
  • Significant parts of the paper require a substantial rework.
  • Could ordinal features not simply be treated as numerical in existing models?
  • How does TabTailor compare to TabCascade (allows for easier tail generation) and TabRep (allows for ordinal feature encoding)?
  • How do the distributions or CDFs of transformed numerical features look like after you applied ATI?
  • How do the C2ST scores hold up when using a more powerful gradient-boosting model as the classifier?
  • How much of the performance gain is attributable to the architecture versus the encoding?
  • Your are modeling a single dynamical system but sample two independent times tct_c and tNt_N during training. Should these not be connected through some global time tt? Otherwise you may train on cases where tc≫tNt_c \gg t_{N}, something you would never see at inference time?
Reviewer_qTVROpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes TabTailor, a mixed-type diffusion framework for tabular data synthesis, aiming to better preserve numerical tail distributions and ordinal structures. For numerical features, the method introduces an Adaptive Tail-Invariant (ATI) transformation and a tail-weighted loss to improve the modeling of rare and extreme values. For categorical features, it employs learnable state-dependent masking schedules and a Cumulative Monotonic Ordinal Embedding (CMOE) to capture category frequencies and ordinal relationships. The framework further integrates a bidirectional Transformer for joint denoising of mixed-type features and a stochastic-restart sampling strategy to mitigate error accumulation during generation. The paper evaluates TabTailor on several benchmark datasets and reports improvements in data fidelity, particularly in preserving numerical tails and ordinal structures.

Critical strengths
  • The paper is well-motivated and addresses an important problem in tabular data synthesis. Specifically, preserving the tail distributions of numerical features and the ordering relationships of ordinal attributes remains challenging for existing tabular generative models. These characteristics are important for maintaining the statistical fidelity and semantic structure of synthetic tabular data, particularly in real-world applications involving rare values and mixed-type features. The paper clearly identifies these limitations and proposes a targeted framework to address them, making the research problem meaningful and worth exploring.
Critical weaknesses
  • The paper contains many variables, symbols, abbreviations, and operators that are insufficiently explained, making the proposed method difficult to understand and evaluate. For example, in Section 3.2, CDF is not expanded when first introduced, and the empirical CDF (\widehat F_j^{\mathrm{emp}}) in Eq. (1) is not explicitly defined. In Eq. (2), the definition of the candidate (h) and the computation of (\mathrm{Stab}(h)) and (\mathrm{Wid}(h)) need clearer explanations. The choice of quantile levels (Q={0.99,0.995}) in Eq. (3) also lacks justification. Moreover, the notation (\widehat F_j(X_j^-)) and the inverse function (\Phi^{-1}) in Eq. (4), as well as the semicolon operator and variables in Eq. (7), should be explicitly explained. Although some details are provided in the appendix, the main text would benefit from clearer and more rigorous mathematical definitions to help readers assess the technical validity and novelty of the proposed method.
Questions for authors
  • The paper contains numerous unexplained variables, abbreviations, and operators, which seriously hinders the assessment of the technical feasibility and novelty of the proposed method. These issues are not limited to the examples listed in my weaknesses section. I strongly advise the authors to thoroughly revise Section 3 and provide rigorous, explicit mathematical definitions and explanations for all formulas, variables, and operators.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.