- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies whether synthetic tabular data preserves the combinations of attributes present in the original data. It introduces an evaluation based on Formal Concept Analysis and compares 13 learned generators with two structure-free baselines across 63 binary datasets. The authors find that learned generators tend to produce fewer closed attribute sets and smaller implication bases, without a measurable advantage over the baselines on these counts.
Critical strengths
The motivation is convincing for me. It is easy to see how a synthetic table could match column marginals or give a decent average classifier score while still dropping specific attribute combinations that matter for subgroup analysis or database queries.
The empirical comparison in Figure 2 is also quite striking. Seeing 13 widely used generators (across GANs, diffusion, and tabular foundation models) fail to beat trivial coin-toss and row-sum baselines on concept counts is a interesting result.
Critical weaknesses
(1)I have some concern about the dataset. Section 3.2 states that every dataset is clarified and reduced before fitting the generators, which removes all duplicate rows and columns. For statistical generative models (GANs, VAEs), duplicate rows are how empirical frequencies are represented. Removing them before training alters the data distribution and reduces datasets to tiny matrices (mostly 10–60 rows) where every pattern appears only once by construction. This makes it hard to tell whether the structural loss discussed in Section 5 is an inherent flaw of the generators' training objectives on rare patterns, or an artifact of training distribution-based models on deduplicated formal contexts.
(2) Lacking application scenarios: The datasets are very small in tabular synthesis applications, and a substantial fraction come from related wiki-edit networks. While Sections 1 and 2 frame the work around tabular synthesis applications (such as subgroup prediction and database testing), Appendix A reveals that 48 of the 63 contexts are network adjacency or incidence matrices from KONECT. It is unclear how well findings on tiny graph adjacency matrices reflect typical tabular data settings where columns represent distinct semantic features. I understand why exact computation requires this setting. However, it leaves open whether the observed failures mainly reflect a difficult small-data regime or a broader limitation of current generators.
Questions for authors
-
Can you show even one concrete example from your datasets where this loss of closed sets actually breaks a query or hurts subgroup prediction, even though standard marginal/utility metrics look fine?
-
Due to the over-representation of WIKI-edit contexts, can you show the robustness of your result across diverse subsets. Does the main conclusion remain similar when the WIKI-edit datasets are excluded? It remains unclear whether the observed failure is a general limitation of tabular generators or mostly an artifact of fitting models on tiny, deduplicated graph matrices.