acceptodds
Under review as a conference paper at ICLR 2027

On Privacy in Data-Space Tabular Diffusion Models: Influential Factors, Attacker Knowledge, and Metrics

Abstract

Tabular data plays an important role in many fields and industries, including those with elevated privacy considerations and risks. As such, there is a rising interest in generating high-quality synthetic proxies for real tabular data as a means of reducing privacy risk and proprietary data exposure. With data-space tabular diffusion models (TDMs) demonstrating leading performance in synthesizing such data, understanding and measuring the privacy risks associated with these models is imperative. Leveraging state-of-the-art membership inference attacks for such TDMs in both black- and white-box settings, this work quantifies the impact of training setup, synthesis choices, and attacker knowledge on privacy leakage. Moreover, the results demonstrate that adversaries need not have perfect knowledge of the training setup, identical data distributions, or massive compute resources to construct successful attacks. Finally, the substantial pitfalls associated with heuristic privacy metrics, such as distance-to-closest record, are identified.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy leakage, tabular diffusion models, membership inference attacks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Pn86OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies membership inference leakage in tabular diffusion models (ClavaDDPM and Tabdiff) in white-box and black-box settings. It uses two attacks from MIDST challenge - Tartan Federer and Ensembe approaches. They vary factors such as training set size and steps, diffusion timestamps, batch size and ratio of synthetic to training data. For clavaDDPM, they also weaken attacker assumptions through shadow models, mismatched shadow configs, and shifted or noisy shadow-model data. Experiments are conducted on Berka and Diabetes datasets, reporting TPR at FPR of 0.1 and DCR, along with other heuristic privacy metrics. The paper's main contribution is an empirical study, as it takes SOTA MIAs and uses them to measure how much models leak under distinct settings and what factors are important. The privacy risk follows the usual overfitting factors like more training, bigger models and less training data mean more leakage. An interesting finding is that generating more synthetic rows relative to training set sharply raises attack success, while DCR could stay flat.

Critical strengths

This performs a broad and controlled privacy study that uses SOTA attacks rather than distance heuristics. It covers two tabular diffusion models under white-box and black box settings, and a variety of training and release choices. The high-level effects are clear in the plot results, and it gives practical and concrete evidence that heuristic metrics can miss leakage. The attacker resources experiments are useful, as it weakens the common argument that idealized attack settings overstate practical risk.

Critical weaknesses

Some concerns are the following:

  • quantitative conclusions lack statistical support, and some factors are confounded. Multiple runs and deviations over those would be helpful to gain confidence in empirical results, as some conclusions are drawn based on small differences. A quantitative correlation between DCR and MIA success would be beneficial.
  • Scope of general claims: the ensemble attack, experiments on attacker compute, shadow model and data mismatch are run only on ClavaDDPM. TabDiff seems to behave different on batch size results. The conclusion that large compute, perfect knowledge and IID data are not prerequisites for successful attacks seems to rest only on one model.
  • The paper mentions Yao et al ("The DCR delusion: Measuring the privacy risk of synthetic data"), which appears to have already ran the MIDST attacks on ClavaDDPM and found no correlation between DCR and MIA success. It would help clarify what are the differences between these results.

Overall, I think this is a useful and timely empirical study. However, the main contributions lack novelty or empirical surprises, that are sufficient to justify high-quality submission.

Questions for authors
  • Could the authors report numebr of seeds, standard deviations, data splits, especially for the shadow-count and data-mismatch results? It would clarify whether the smaller effects are real.
  • It would be useful to see a quantitative association between DCR and MIA success across settings, such as a rank correlation
  • Could the authors report TPR at lower FPRs or full ROC curves for the main levers? I would like to know whether the conclusions hold at the operating points most relevant to worst-case privacy
6 comments

Dear Reviewer Pn86,

We thank you for your feedback and recognizing the breadth of our empirical evaluation, the use of state-of-the-art attacks, and the practical significance of our findings on privacy metrics and attacker resource requirements. Below, we address your questions.

On the number of seeds. We report the number of independent runs and maximum standard errors (SE) of TPR@10% FPR for each experimental setting below.

Model / DatasetExperimentsRunsMax SE
ClavaDDPM / BerkaNon-Ensemble experiments50.01
ClavaDDPM / DiabetesMost experiments, including shadow-count and data-mismatch50.05
ClavaDDPM / EnsembleEnsemble attack experiments30.02
TabDiff / Berka & DiabetesTraining dataset size50.02

Experimental setup: Across independent seeds, data splits are randomized and constructed from scratch, target and shadow models are retrained, and challenge points are independently sampled. Attack classifiers are also independently trained.

For the distribution-mismatch experiments, IID and non-IID adversary datasets are independently generated for each seed, following the same data-splitting and model-training procedures described above.

For the Ensemble attack, we use 3 independent seeds due to its computational cost, varying data splits, target and shadow models, challenge points, and attack training.

Additionally, during the rebuttal period, we repeated the TabDiff training dataset size experiments 5 times for the white-box attack on both Berka and Diabetes. All datasets were independently constructed, and models were retrained from scratch. The resulting mean TPR@10% FPR differed from the originally reported results by at most 0.01, with standard errors below 0.02.

The conclusion that large compute, perfect knowledge and IID data are not prerequisites for successful attacks seems to rest only on one model. Our intention in reporting the attacker-compute, shadow-count, and data-mismatch experiments on ClavaDDPM was to challenge the assumption that favorable attacker conditions, such as substantial compute, perfect knowledge, and distribution-matched data, are necessary for successful MIAs. These experiments demonstrate that substantial membership leakage remains possible even when these assumptions are relaxed.

For example, in the ClavaDDPM/Berka noisy adversary data experiment, the adversary's data differs substantially from the target's training distribution. We quantify this mismatch through differences in the correlation matrices of numerical features, mutual-information matrices of categorical features, and statistical tests comparing feature marginals.

Corr. ΔMI ΔDifferent Marginals
2.0001.449100%

Despite this distribution mismatch, both white-box and black-box attacks remain effective. (TPR at 10% FPR, reported as mean ± standard error across 5 runs.)

AttackIID TPR (%)Noisy TPR (%)Drop
TF-WB44.30 ± 0.3039.16 ± 0.30−5.14
TF-BB25.50 ± 0.6524.78 ± 0.65−0.72

Even when all feature marginals differ significantly, TF-WB achieves 39.16% TPR and TF-BB achieves 24.78% TPR, compared with 10% at chance level. Similarly, our shadow-count and model-configuration mismatch experiments demonstrate that successful attacks remain possible with limited attacker compute and imperfect knowledge of the target model.

We acknowledge that the attacker-resource and data-mismatch results are specific to ClavaDDPM and do not establish architecture-independent behavior. However, they demonstrate that typical assumptions about the attacker access are not always necessary for successful membership inference.

We would also like to emphasize that batch size is the only training or synthesis factor for which we observe substantially different trends between ClavaDDPM and TabDiff.

The paper mentions Yao et al... It would help clarify what are the differences between these results.

We evaluate a different DCR formulation. Yao et al. evaluate a percentile-based privacy test comparing synthetic-to-training and holdout-to-training nearest-neighbor distances, as well as a continuous measure based on mean synthetic-to-training distance. In contrast, we use the holdout-based DCR introduced by Zhang et al. [1], which measures the proportion of synthetic records closer to training than holdout records, thereby accounting for differences in local data density. We show that even with this additional consideration, DCR does not reliably reflect membership leakage.

We systematically investigate whether this density aware DCR tracks changes in membership leakage across different training and synthesis configurations. While Yao et al. establish that DCR is unreliable as a standalone privacy measure, we further examine whether it reflects the direction and magnitude of changes in attack-measured risk. Our results show that changes in DCR do not consistently correspond to changes in MIA success.

We also extend the analysis to other empirical privacy metrics (Appendix E). Metrics such as NNDR, Hitting Rate, and ε-Identifiability Risk [2-6] are used in the literature to assess privacy risks in synthetic data, including comparisons between generative models. For example, Lee and Min (AAAI 2025) use NNDR and DCR to compare the privacy protection of CG-TGAN and TabDDPM. However, our experiments demonstrate that these metrics do not necessarily reflect membership leakage. Across model architectures, NNDR remains relatively stable (0.28–0.33), while MIA success increases from approximately 12% to 26% TPR@10% FPR. Similarly, increasing the synthetic dataset size from 1× to 1M leaves NNDR nearly unchanged (approximately 0.35), while MIA success increases from 24% to 35%. (Figure 9)

It would be useful to see a quantitative association between DCR and MIA success across settings, such as a rank correlation.

We thank the reviewer for this suggestion. We computed Spearman's rank correlation between MIA success (TPR@10% FPR) and the absolute deviation of DCR from the ideal baseline across the reported training dataset sizes. Correlations are calculated separately for 5 independent seeds for the TF attack for Clavaddpm and tabdiff and 3 independent seeds for Ensemble.

AttackClavaDDPM (Berka)ClavaDDPM (Diabetes)TabDiff (Berka)TabDiff (Diabetes)
TF (BB)0.8500.900-0.050-0.500
Ensemble (BB)1.0000.800----

While DCR shows strong positive correlations for ClavaDDPM, this relationship does not hold for TabDiff. Moreover, rank correlation does not capture the magnitude of membership leakage. These results further highlight the limitations of DCR as a general privacy measure.

Could the authors report TPR at lower FPRs or full ROC curves for the main levers?

We thank the reviewer for this suggestion. We evaluated our white-box attacks at 1% FPR across the main experimental settings and observed that the trends reported at 10% FPR generally persist. As an example, we report results across training dataset sizes for TabDDPM and TabDiff on Berka and Diabetes, averaged over 5 independent runs.

TPR@1% FPR (%) across training dataset sizes

DatasetModel5K10K20K50K100K
BerkaClavaDDPM44.839.130.45.93.1
BerkaTabDiff5.83.23.43.1--
DiabetesClavaDDPM74.350.71.0----
DiabetesTabDiff29.6714.336.33----

These results support our findings at 10% FPR, showing that increasing training dataset size generally reduces membership leakage even at stricter operating points. We also observed consistent trends for the other main experimental levers.

We report representative numerical results here. We would be happy to provide additional results if the reviewer finds them useful.

[1] Zhang, Hengrui, et al. “Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space.” International Conference on Learning Representations (ICLR), 2024

[2] Zhao, Zilong, et al. "CTAB-GAN: Effective Table Data Synthesizing." Asian Conference on Machine Learning (ACML). PMLR, 2021.

[3] Lee, Seungcheol, and Moohong Min. "CG-TGAN: Conditional Generative Adversarial Networks with Graph Neural Networks for Tabular Data Synthesizing." Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 17, 2025.

[4] Trudslev, Frederik Marinus, et al. "PrivEval: A Tool for Interactive Evaluation of Privacy Metrics in Synthetic Data Generation." Proceedings of the VLDB Endowment, vol. 18, no. 12, 2025, pp. 5271–5274.

[5] Lautrup, Anton D., et al. "SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Data." Data Mining and Knowledge Discovery, vol. 39, no. 1, Article 6, 2025.

[6] Yoon, Jinsung, Lydia N. Drumright, and Mihaela van der Schaar. “Anonymization Through Data Synthesis Using Generative Adversarial Networks (ADS-GAN).” IEEE Journal of Biomedical and Health Informatics, 24(8), 2378–2388, 2020.

Reviewer_i8XvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates membership inference attacks (MIAs) on synthetic tabular data generated by diffusion models. It evaluates existing attack techniques across a broad range of experimental configurations and examines how factors such as diffusion steps, batch size, training iterations, and training dataset size affect measured membership leakage. The study aims to improve our understanding of the privacy risks associated with diffusion-based tabular data generation.

Critical strengths
  1. Important and relevant problem. The paper investigates membership inference risks in diffusion-based tabular data generation. Given the strong utility of diffusion models in non-private tabular synthesis, understanding their potential membership leakage is a practically relevant research question.

  2. Extensive experimental evaluation. The paper evaluates MIA performance across a broad range of parameter settings. This coverage helps readers understand how measured membership leakage varies with training configurations and provides useful empirical observations about the sensitivity of attack performance to these choices.

Critical weaknesses
  1. Insufficient positioning relative to DP tabular synthesis. The motivation emphasizes the limitations and utility costs of differential privacy, but the discussion does not sufficiently distinguish DP tabular synthesis from DP training in other settings. Existing methods [1–5] can provide formal privacy guarantees and competitive utility without relying on diffusion models or DP-SGD. Their existence does not remove the value of studying non-private diffusion models, but it makes a more precise justification necessary. Comparisons with representative DP tabular synthesis methods would help establish the practical utility advantages, if any, that motivate accepting the membership risks of non-private diffusion models.

  2. Missing joint analysis of utility and membership leakage. The paper studies factors such as diffusion steps, batch size, training iterations, and training dataset size, which can affect both generation utility and MIA success. As the appendix also indicates, changes in these settings can substantially influence utility. Consequently, reporting attack performance without a joint utility analysis makes it difficult to assess whether lower attack success accompanies a practically useful model or simply poorer generation quality. Although the conclusion identifies the utility–MIA trade-off as future work, I consider it central to interpreting the current results. Such an analysis would also make the findings more actionable for selecting training configurations.

  3. Limited methodological novelty and explanatory depth. The study primarily combines existing generation models and MIA techniques. An empirical contribution can be valuable without introducing a new attack, but it should provide sufficiently strong insights beyond comparisons across configurations. The paper would benefit from a deeper investigation of which properties of tabular data contribute to membership leakage, such as mixed numerical and categorical attributes or dependencies among columns. Tabular-specific attack designs are one possible direction, while targeted analyses using existing attacks could also strengthen the contribution.

References:

Questions for authors
  1. Can the authors clarify the intended deployment setting and the utility advantages that motivate using non-private diffusion models over representative DP tabular synthesis methods? Are there existing results or comparisons that support this motivation?

  2. Can the authors provide joint utility–MIA results for the configurations already evaluated? In particular, do the observed differences in attack success persist when comparing configurations with similar generation utility?

  3. Which findings reveal membership leakage mechanisms specific to tabular diffusion models, rather than general effects of training configuration or model utility?

6 comments

Dear Reviewer i8Xv,

We would like to thank you for your feedback and for acknowledging the relevance and timeliness of studying the factors that contribute to privacy leakage in diffusion models. Below, we provide clarifications and new experiments to address your questions.

On insufficient positioning relative to DP tabular synthesis. We agree that our motivation should better distinguish DP-SGD training of diffusion models from DP tabular synthesis more broadly. To ground the utility comparison between DP and non-private tabular synthesis, we will cite Du and Li [6]. On Adult, they report a relative downstream F₁ gap of 0.101 for MST at ε = 1 and 0.014 for non-private TabDDPM, with MST achieving the smallest gap among the DP methods evaluated on Adult. We agree that [1–5] include methods that provide formal guarantees without diffusion or DP-SGD and that these methods can approach real-data performance on datasets dominated by low-order dependencies.

These approaches have specific limitations. AIM [1] optimizes a specified workload of marginal queries, and RAP++ [2] reports difficulty preserving gains from nonlinear classifiers. Reference [3] identifies strong statistical DP methods and reports poor convergence for TabDDPM trained with DP-SGD. MargNet [4] trains on privatized one-way and two-way marginals, which provide direct supervision for low-order dependencies. References [4, 5] discuss the challenges of capturing higher-order dependencies. For example, [5] reports an approximately 30-percentage-point accuracy gap for Tab-PE on Artificial Characters at ε = 1 relative to training on real data. The utility trade-off therefore depends on the dataset and downstream task. Our study addresses settings in which practitioners consider releasing synthetic tabular data from a non-private diffusion model to meet downstream utility requirements. We will revise the Introduction and Related Work to incorporate these.

On missing joint analysis of utility and membership leakage. Appendix F reports data fidelity and downstream task performance across the factors considered in our study. For ClavaDDPM on Berka, the table below pairs the membership inference results from Table 3 with the fidelity and downstream utility results at the same training checkpoints. White-box (WB) and black-box (BB) TPR are both measured at 10% FPR. ΔR² and ΔF₁ measure downstream regression and classification utility, respectively. β-coverage, α-precision, average KS, average TVD, correlation difference, and mutual-information difference measure fidelity. See Appendix F for metric definitions.

Training stepsWB TPR (%)BB TPR (%)β-coverage ↑α-precision ↑Avg. KS ↓Avg. TVD ↓Corr. diff. ↓MI diff. ↓ΔR² ↑ΔF₁ ↑
50k26.011.30.4510.9940.04030.009300.03240.00411−0.00629−0.0184
100k37.315.30.4580.9960.03960.008320.03350.00519−0.00100−0.0207
200k42.520.00.4790.9930.03900.005820.03770.00358+0.00240−0.0192
500k51.527.80.5050.9940.03820.007170.04320.00632+0.000055−0.0146

From 50k to 500k training steps, white-box TPR rises by 25.5 percentage points and black-box TPR rises by 16.5 points. Across the four checkpoints, α-precision stays between 0.993 and 0.996, β-coverage improves from 0.451 to 0.505, and downstream ΔF₁ and ΔR² each vary by less than 0.01, however the privacy leakage levels vary significantly.

On limited methodological novelty and explanatory depth. We would like to emphasize that our work fills a gap in systematic, attack-based guidance for understanding privacy leakage in tabular diffusion models (TDMs).

Factors studied in other literature. Almost all previous studies on the levers considered here are done for simple architectures and classification. So all of our work significantly and rigorously extends previous scattered studies. Each of the works also has shortcomings i.e. measurement deficiencies (NNDR used for example), or are only partial studies.

The computational requirements of successful attacks. Prior studies report different benefits from increasing the number of shadow models. In our ClavaDDPM experiments, a single shadow model achieves attack performance close to that obtained with multiple shadows. This shows that comparable attack performance is possible with fewer shadow-model training runs, even when training each generative model is computationally expensive.

Imperfect knowledge and data access tailored to tabular synthesis. We evaluate mismatched model configurations and concrete restrictions on attacker data, including disjoint accounts, different time windows, corrupted attributes, and data constructed from marginal statistics. These experiments demonstrate that successful attacks on ClavaDDPM remain possible under the specific departures from ideal attacker assumptions that we tested.

The privacy implications of synthetic-data release volume. We quantify how the ratio of synthetic records to training records affects attack success. This addresses a practical decision for TDMs about how much synthetic data to release from a trained model.

Empirical privacy metrics and membership leakage. In Appendix E, we extend the analysis beyond DCR to other metrics used to assess synthetic-data privacy, including NNDR [7, 8], Hitting Rate [9, 10], and ε-Identifiability Risk [10, 11]. Our results show that these metrics do not reliably track changes in membership leakage.

The value of combining attack signals. Our evaluation of the existing Ensemble attack shows that distance-based, canary-based, and reference-based features can jointly support successful attacks on ClavaDDPM, even when standalone DCR provides an unreliable assessment of membership leakage.

Which findings reveal membership leakage mechanisms?

To identify which features carry membership signal, we focus on ClavaDDPM and TabDiff for Berka, which has the same number of categorical and numerical features and report the numerical only and categorical only based attacks for vaious training dataset sizes.

Numerical-only attacks outperform categorical-only attacks at four of five sizes for ClavaDDPM, with the largest gaps at 5k and 10k. Categorical-only attacks lead at all three sizes for TabDiff. This shows that the relevance of features for the attack can be model dependent and no general claims can be made.

Berka / ClavaDDPM (white-box TF TPR at 10% FPR, %)

TrainFullNum.Cat.
5k61.061.737.1
10k52.052.135.5
20k44.441.230.2
50k17.013.214.0
100k15.714.313.1

Berka / TabDiff (white-box TF TPR at 10% FPR, %)

TrainFullNum.Cat.
5k15.89.214.0
10k14.89.016.2
20k12.87.612.2
Reviewer_T7JVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies privacy leakage in data-space tabular diffusion models (TDMs), with a focus on ClavaDDPM and TabDiff. The paper reviews existing black-box (BB) and white-box (WB) attacks and aims to quantifies the mechanisms that impact the privacy leakage of TDMs, namely the training and architecture setups, the generation process, the data access, the adversarial knowledge and the compute power. It also investigates whether distance-to-closest-record (DCR) works compared to MIAs.

Critical strengths
  • Comprehensive empirical evaluation across multiple settings. The paper provides a systematic empirical investigation of several factors influencing membership inference against tabular diffusion models. Two diffusion architectures, two datasets, BB and WB and multiple attacks are studied.
Critical weaknesses
  • Limited scientific novelty beyond existing findings. While the volume of experiments is substantial, it is difficult to assess what conclusions improve existing understanding. The authors appropriately cite prior work, and position their paper as a broadening of current empirical evidence, which is valuable. Nevertheless, it mainly reinforces known qualitative conclusions. For example, the failure of DCR has already been assessed for ClavaDDPM and TabDDPM for black-box and white box attacks in prior work [1]. [2] already studies training duration and dataset size. There are also some mischaracterisations of current existing work. The paper states in Section 4.1 that [3] suggests that "diffusion models are more robust to memorization". However, [3] argues that diffusion models may be subject to a dynamical regularization effect, which does not prevent overfitting over long training procedures and may be combatible with this work. Also, in Section 4.4, the effectiveness of a single shadow model is described as "contrasting" [4], while [4] explicitely shows that one shadow model can achieve similar performance other approaches involving multiple shadow models.
  • Insufficient analysis of the findings that would make this paper distinctive from prior work. The results strongly depend on the model and attack. For example, batch size has opposite effects for the two target models under TF and little effect under Ensemble. While this observation is interesting, it is not clear where does this phenomenon come from. It could be a consequence of a lower privacy leakage or a worse generation quality, as well as an artifact of the analysis pipeline. Also, the training budget convention needs clarification. Appendix A.1 specifies a number of training steps for ClavaDDPM, while Appendix A.2 specifies a number of epochs for TabDiff. If ClavaDDPM uses a fixed number of updates while TabDiff uses a fixed number of epochs, then it may explain why increasing batch size does not have the same effect for those two models.

References

[1] Zexi Yao, Natasa Kr co, Georgi Ganev, and Yves-Alexandre de Montjoye. The DCR delusion: Measuring the privacy risk of synthetic data, Arxiv:2505.01524, 2025.

[2] Zhengyu Fang, Zhimeng Jiang, Huiyuan Chen, Xiao Li, and Jing Li. Understanding and mitigating memorization in diffusion models for tabular data. In Proceedings of the 42nd International Conference on Machine Learning (ICML 2025), 2025.

[3] Tony Bonnaire, Raphael Urfin, Giulio Biroli, and Marc Mezard. Why diffusion models don’t memorize: The role of implicit dynamical regularization in training. In NeurIPS, 2025.

[4] Ahmad Salem, Yang Zhang, Martin Humbert, Michael Fritz, and Manuel Backes. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In Network and Distributed Systems Security Symposium (NDSS) 2019, California, USA, February 24–27 2019.

Questions for authors
  • Can the authors clarify what is fixed during training? Number of steps or number of epochs? Are TabDiff epoch count adjusted to match the number of steps of ClavaDDPM?
5 comments

Dear Reviewer T7JV, Thank you for taking the time to provide feedback on our paper and for acknowledging its comprehensive empirical evaluation.

Training budget and batch size. Within each model and dataset, we keep the number of training steps fixed except in experiments that explicitly vary it. We will clarify this in Appendix A.

Regarding generation quality and batch size. We would like to bring the reviewer’s attention to Appendix F, where we report detailed data fidelity and downstream utility metrics. The table below summarizes the results for TabDiff/Diabetes (Fig. 2 / Tab.14).

Metric2,0484,0968,192
(WB) (TPR@10%)74.5%46.1%18.6%
α-precision ↑0.890.930.83
MI Mat difference ↓4.52.03.9
Corr Mat difference ↓1.40.91.2
β-coverage ↑0.460.460.43
KS ↓0.020.020.02
TVD ↓0.010.0080.008

These results suggest that changes in generation quality alone cannot explain the observed decline in attack success. We would also like to emphasize that among the levers, batch size is the only one for which ClavaDDPM and TabDiff showed substantially different trends in membership inference success.

Regarding the number of shadow models Our use of “contrasting” was intended to refer to the expectation that additional shadow models improve attack performance by providing more training data, as discussed by [11]. We agree that Salem et al. [4] demonstrate that a single shadow model can achieve performance comparable to multiple shadow models for discriminative classifiers. We extend this finding to tabular diffusion models, which are substantially more computationally expensive to train and are evaluated using different membership inference attacks. We will correct the wording in the revised manuscript to clarify this distinction.

On limited scientific novelty. Our work fills a gap in the systematic understanding of privacy leakage in TDMs. Prior findings on individual factors leave open how the training choices, synthesis settings, and attacker constraints we evaluate here affect membership leakage in these state-of-the-art models.

In relation to [1], we evaluate a different DCR formulation. Yao et al. [1] evaluate a percentile-based one-shot privacy test comparing synthetic-to-training and holdout-to-training nearest-neighbor distances, as well as a continuous measure based on mean synthetic-to-training distance. In contrast, we use the holdout-based DCR introduced by [10], which measures the proportion of synthetic records closer to training than holdout records, thereby accounting for differences in local data density. We show that even with this additional consideration, DCR does not reliably reflect membership leakage. We also further examine whether it reflects the direction and magnitude of changes in MIA risk. Our results show that changes in DCR do not consistently correspond to changes in MIA success.

We extend the analysis to many nmore empirical privacy metrics (Appendix E). Metrics such as NNDR [5, 6], Hitting Rate [7, 8], and ε-Identifiability Risk [8,9] are used in the literature to assess privacy risks in synthetic data, including comparisons between generative models. For example, Lee and Min (AAAI 2025) use NNDR and DCR to compare the privacy protection of CG-TGAN and TabDDPM. However, our experiments demonstrate that these metrics do not necessarily reflect membership leakage. Across model architectures, NNDR remains relatively stable (0.28–0.33), while MIA success increases from approximately 12% to 26% TPR@10% FPR. Similarly, increasing the synthetic dataset size from 1× to 1M leaves NNDR nearly unchanged, while MIA success increases from 24% to 35%. (Figure 9)

These findings also clarify the distinction between our work and [2] and [3], which investigate memorization across different training configurations using NNDR. Since our results show that NNDR does not reliably reflect membership leakage, their findings on memorization do not necessarily translate to membership inference risk. We also acknowledge that Bonnaire et al. [3] identify a dynamical regularization effect that delays, rather than prevents, memorization, and we will clarify this characterization in the revised manuscript. Furthermore, [3] focuses on image diffusion models, whereas our work investigates membership leakage in tabular diffusion models.

In addition to these, our study provides the evidence needed to assess these choices through the following contributions:

The value of combining attack signals. Our evaluation of the existing Ensemble attack demonstrates that distance, canary, and reference-based features can jointly support successful attacks on ClavaDDPM, even when standalone DCR provides an unreliable assessment of privacy risk.

Factors studied in other literature. Almost all previous studies on the levers considered here are done for simple architectures and classification. So all of our work significantly and rigorously extends previous scattered studies. Each of the works also has shortcomings i.e. measurement deficiencies (NNDR used for example), or are only partial studies.

The computational requirements of successful attacks. Prior studies provide different conclusions about the benefit of additional shadow models. We show that a single shadow model can achieve attack performance close to that obtained with many in TDMs. This establishes that the computational cost of training these generative models does not necessarily translate into a requirement for substantial attacker resources.

Imperfect knowledge and data access tailored to tabular synthesis. We evaluate mismatched model configurations and concrete restrictions on attacker data, including disjoint accounts, different time windows, corrupted attributes, and data constructed from marginal statistics. These experiments demonstrate that successful attacks on TDMS remain possible under specific departures from ideal attacker assumptions.

The privacy implications of synthetic-data release volume. We quantify how the synthetic-to-training data ratio affects attack success. This addresses a practical decision specific to generative models, that has not been studies for TDMS before.

The privacy implications of diffusion steps. We examine how changing the number of diffusion steps affects membership leakage, providing evidence about a configuration choice specific to diffusion models that classification studies cannot address.

[5] Zhao, Zilong, et al. "CTAB-GAN: Effective Table Data Synthesizing." Asian Conference on Machine Learning (ACML). PMLR, 2021.

[6] Lee, Seungcheol, and Moohong Min. "CG-TGAN: Conditional Generative Adversarial Networks with Graph Neural Networks for Tabular Data Synthesizing." Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 17, 2025.

[7] Trudslev, Frederik Marinus, et al. "PrivEval: A Tool for Interactive Evaluation of Privacy Metrics in Synthetic Data Generation." Proceedings of the VLDB Endowment, vol. 18, no. 12, 2025, pp. 5271–5274.

[8] Lautrup, Anton D., et al. "SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Data." Data Mining and Knowledge Discovery, vol. 39, no. 1, Article 6, 2025.

[9] Yoon, Jinsung, Lydia N. Drumright, and Mihaela van der Schaar. "Anonymization Through Data Synthesis Using Generative Adversarial Networks (ADS-GAN)." IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 8, 2020, pp. 2378–2388.

[10] Zhang, Hengrui, et al. “Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space.” International Conference on Learning Representations (ICLR), 2024

[11] Hu, Hongsheng, et al. “Membership Inference Attacks on Machine Learning: A Survey.” ACM Computing Surveys, 54(11s), Article 235, 2022.

Comments on the paper

1 comment

While running additional TabDiff experiments on Berka, we found that the released implementation drew non-member challenge records from accounts in other folds. As a result, the attack could exploit differences between account groups, inflating measured membership inference success.

To align the evaluation with our experimental setup, we reran the attack using non-member challenge records drawn from the holdout set of each target model’s own fold. The TabDiff attack results for Berka that we shared during the rebuttal already use this corrected protocol.

We have uploaded the revised manuscript with the corrected results. The original qualitative attack trends still hold. The correction affects only the attack evaluation. The data-quality results are unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.