acceptodds
Under review as a conference paper at ICLR 2027

Synthetic Data Shield: Obfuscating Contexts for In-Context Tabular Prediction

Abstract

Tabular foundation models are becoming a popular approach to tabular classification based on in-context prediction. To classify a new entry, the model receives the labeled training set and the new row, then predicts its label in a single inference pass without task-specific training on the target dataset. A major privacy concern of this paradigm is that when a third party performs classification, the training data used as context must also be shared with that party. We propose **Synthetic Data Shield** (SDS), a method that does not require sending real training data to a third party. SDS creates a local synthetic training context and sends only this synthetic data, keeping the original training records hidden. The foundation model remains unchanged, while accuracy stays close to that of the original data. We evaluate our method through three privacy axes: membership inference, near-record exposure, and attribute inference. Our rigorous analysis shows that prediction utility, membership leakage, near-record exposure, and attribute inference relate to different statistical properties of the synthetic data generator, allowing us to control membership and near-record exposure separately while attribute inference tracks model accuracy in our measurements. Membership attacks stay close to chance, and synthetic records sit nearly as far from the training rows as from held-out rows of the same population. Attribute inference differs because statistical relations useful for prediction can also help predict a hidden attribute. We further show that adding noise can reduce this leakage but also causes a small loss in model accuracy. We test SDS on 71 tabular datasets and across tabular foundation models such as TabPFN-2.5, TabICL, LimiX, TabFM, and TabuLa-8B. We use the same synthetic contexts across these models (TabuLa-8B reads a 32-shot subset of each), with the mean accuracy loss staying within 2.5 percentage points for the tested transfer models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
tabular foundation models, in-context learning, privacy, synthetic data, reconstruction attacks, tabular data

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_HMJvOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Submission 41043 looks at privacy risks in In-Context Tabular Foundation Models (TFMs), such as TabPFN-2.5 and TabICLv2. Instead of updating parameters for each dataset, these models classify data in one step by using the training dataset as input.

Sending raw training data to host services can expose sensitive information. Differential Privacy methods often add too much noise, which lowers accuracy. Deep generative models may memorize data or perform poorly with few samples.

The authors present Synthetic Data Shield (SDS), a local preprocessing method that trains a class-conditional generator and sends only synthetic samples to the TFM service.

This work is useful because it takes a thorough privacy approach, tests on 71 datasets and 6 TFM models, and reports clearly.

Critical strengths

Analytical Rigour & De-Coupled Privacy Taxonomy: The paper provides a principled mathematical framework that disentangles privacy into four independent distributional axes11more_horiz. By formally linking prediction utility to Wasserstein-1 distance, membership inference to replace-one KL divergence, and attribute inference to Total Variation, the authors prove that membership leakage and spatial layout can be suppressed independently of model accuracy. In my view, a key contribution is that Theorem 3 shows attribute inference is a structural residual risk of population learning rather than record memorization.

Exhaustive Cross-Model & Multi-Suite Validation: The experimental design is exceptionally thorough, spanning 71 distinct datasets across 5 benchmark suites (Core 6, Headline 14-tier, Mixed-Type 15, Breadth 40, TabZilla-Hard 31). Furthermore, demonstrating zero-shot model transfer, where a single generated synthetic context is evaluated across 6 heterogeneous TFMs (including column-row attention networks and the 8B-parameter LLM TabuLa-8B) with consistent accuracy preservation provides convincing evidence of model-agnostic utility.

Comprehensive Multi-Threat Privacy Auditing & Positive Controls: Rather than relying solely on superficial heuristics like Distance-to-Closest-Record (DCR), the paper audits SDS against a comprehensive suite of state-of-the-art attacks, including NN-MIA, query-adaptive QAPD, shadow models, density-ratio DOMIAS, and no-box local likelihood ratio Gen-LRA attacks. The authors validate the diagnostic sensitivity of their audit suite by running the same attacks against a deliberately memorizing generator as a positive control (where AUC jumps to approximately 0.73, proving that SDS's near-chance results reflect genuine protection rather than weak auditing.

Critical weaknesses

Structural Failure on High-Order Deterministic / Combinatorial Decision Rules:

  • The underlying generative models (BIC-GMM + Copula, Chow–Liu trees) rely on smooth, low-order joint statistical structures14more_horiz. On datasets where class labels are governed by exact deterministic higher-order combinatorial rules or sharp arithmetic boundaries (e.g., Tic-Tac-Toe \Delta = -8.2%, Balance-Scale \Delta = -8.3%). I believe that in-context tabular foundation models excel at learning complex decision boundaries. When SDS smooths away higher-order combinatorial interactions, it undermines TFM performance precisely on the non-linear, rule-based tasks where TFMs offer the greatest advantage over simpler linear/tree (my view, of course).

Longitudinal Leakage Compounding on Small-Sample Datasets (N_{mem} < 100):

  • When a data owner issues repeated synthetic context releases over time from a small training cohort (N_{mem} < 100, such as Iris or Seeds), pooling multiple synthetic contexts allows an adversary to track the generator's sampling variance. That should be a major concern. That said, it should be noted that the authors introduce an adaptive two-sided band-pass DCR filter (p10−p75p_{10}-p_{75}) to mitigate small-N; this filter introduces distance-score asymmetry on larger datasets (N_{mem} > 100) that actually increases adversarial advantage above baseli. However, the system lacks a single, non-distorting protection mechanism for continuous re-scoring scenarios across arbitrary dataset sizes.
Questions for authors

Handling Higher-Order Deterministic Logic: It is known that Chow–Liu trees only capture tree-structured pairwise dependencies. Also, standard assumptions in Gaussian mixtures (GMM) or Copulas assume joint smoothness; how can SDS be improved to preserve sharp, higher-order combinatorial rules like parity, XOR, or multi-feature logic without listing every row?

Universal Distance Filtering: The two-sided band-pass DCR filter seems to reduce longitudinal leakage compounding on small datasets (N_{mem} < 100), but it distorts score symmetry on larger datasets. Is there a density-preserving filter that can stop longitudinal compounding without causing asymmetric score shifts on bigger tables?

Restoring PR-AUC on Imbalanced Data: What makes the synthetic context reorder predictions for the minority class? Is it possible to adjust per-class oversampling quotas dynamically?

2 comments
AuthorsOpenReview ↗

Response to Reviewer HMJv

Thank you for this detailed review, and in particular for identifying the four-axis decomposition and recognizing our structural reading of attribute inference in Theorem 3. We also appreciate your comments on our cross-model validation with the 8B LLM reader and the use of a positive control in the audit.

  1. Deterministic combinatorial rules: As you note, the current low-order generators are not reliable here. Consider a three-bit parity rule with balanced, independent bits: the label depends on all three even when pairwise correlations give no signal. This is a limitation of the representation, not of the in-context reader. In a new experiment on Tic-Tac-Toe (3 folds), we increase the maximum number of parents in a standalone Bayesian network from one to three. This narrows the gap from -6.8 to -4.7 points, but the nearest-neighbor membership AUC goes from 0.52 to 0.58. This is the standalone Bayesian-network generator, not the full SDS pipeline. The gap for the full SDS pipeline in the paper is -8.2 points. The variants we tried still miss Balance-Scale and monks-2.

    The revision reports the curve (Table 15) and recommends validating the task on owner-held data before release. In a new experiment we set tolerances in advance and check the synthetic context against the real one on a held-out 1/5 of the owner’s training rows. With the discrete generator SDS uses for these tasks, it rejects 17 of 18 runs on Tic-Tac-Toe, Balance-Scale and monks-2, and the one accepted run loses 6.5 points. It accepts 70% of releases on the 14 datasets, with an accuracy loss of 0.8 points on a different test fold. Only 2 of the 59 accepted releases lose more than 5 points. It does turn away 1 out of 5 useful releases. The revision gives the table of what it accepted and rejected. The check measures utility. Privacy still needs to be assessed separately. If the required rule is lost, then SDS in its current form is unsuitable for the task.

  2. Longitudinal compounding and the band-pass asymmetry: We have not established that a filter will work for all table sizes. For a fixed context that is scored repeatedly, we can use the same synthetic release each time. Sending the same context again gives the recipient no new synthetic rows to pool. Any leakage from the original release remains, and refitting on an updated cohort is a separate case.

    If fresh contexts are needed, the band-pass filter can help for tiny cohorts (on Iris, membership advantage goes from 0.161 to 0.006 at 20 releases), but can make things worse for larger tables (App. J.6). In a new sweep, parameter hardening avoids the distance-based selection. It stays within 0.004 of the generator without hardening on the 6 larger datasets, with no observed added asymmetry, but it does not stop the growth on Iris or Wine. It lowers accuracy by an additional 2.8 points over SDS without hardening on the 14-dataset suite (3 folds, 1 seed), on top of the gap from synthesis itself. The revision adds this sweep (Table 51), corrects the cost (Table 27), and distinguishes repeated use of one release, fresh releases and cohort updates (App. F.2). Density preservation and privacy under composition remain open.

  3. PR-AUC on imbalanced data: One possibility is that smoothing the class-conditional model changes the ranking of borderline cases in the minority class. In a new diagnostic experiment, we find that minority classes do have fewer mixture components selected (3.0 vs. 5.0 for majority classes). This supports the explanation but does not isolate the cause.

    In a new experiment (5 folds, 2 seeds), choosing the oversampling quota on the owner’s validation split improves the PR-AUC gap from -6.2 to -5.4 points, with no meaningful increase in AUC in the nearest-neighbor membership check. Calibration can be helped by using the temperature scaling of Guo et al. on the same split. With fixed quotas, expected calibration error goes from 0.085 to 0.050, compared with 0.041 in the real context. A positive temperature does not change the ranking or predicted classes on these binary tasks, so it does not fix PR-AUC, but it can help calibration.

    The validation split is drawn from 20% of the training fold and is separate from the test fold. We select the quota and temperature using the generator trained without those rows, but draw the released context from the generator refitted with them. Both choices are made on the owner’s side. In a new audit of this complete procedure, every mean AUC difference vs. fixed quotas is at most 0.004 across five membership attacks. We report these runs in App. K (Table 57), along with minority precision, recall and calibration next to balanced accuracy. The revision adds your caveat that clinical suitability remains to be established.

AuthorsOpenReview ↗

Follow-up to Reviewer HMJv

Thank you for your thoughtful review and for highlighting our privacy taxonomy, broad evaluation, and rigorous privacy audit; your suggestions have helped us improve the paper. As the discussion period ends soon, please let us know if you have any further questions, and we hope you will take our new results into account in your final assessment.

Reviewer_dESzOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SDS, which replaces the real training context sent to a cloud-hosted tabular foundation model with locally generated synthetic data. Experiments across 71 datasets evaluate predictive utility, membership inference, near-record exposure, and attribute inference.

Critical strengths
  1. Practical approach with low integration overhead. SDS addresses the exposure of historical training records when using externally hosted tabular foundation models. It generates a synthetic context locally without modifying or retraining the hosted predictor. The lightweight statistical generators are well suited to small tabular datasets, and the transfer experiments across multiple predictors support the approach’s applicability beyond a single model.
  2. Broad evaluation with meaningful privacy controls. The evaluation spans 71 datasets and examines predictive utility alongside membership inference, near-record exposure, and attribute inference. Multiple attack families, nonlinear attribute attackers, population controls, and repeated-release experiments provide a more informative assessment than nearest-neighbor distances alone. The authors also acknowledge important limitations, including residual attribute leakage and the inability of the geometric reconstruction metric to establish privacy.
Critical weaknesses
  1. Privacy of test records. SDS protects the training context but sends test/query records to the external service in the clear. In many practical settings, these records also contain sensitive information. How could SDS be extended to protect query records, and have the authors evaluated the resulting privacy–utility trade-off?
  2. Presentation of results. Many important results—including component ablations, stronger privacy attacks, and controls needed to interpret the reconstruction and attribute-inference findings—are presented mainly in the appendices. This makes it difficult to assess the evidence and limitations from the main paper alone. Please bring the most consequential findings and qualifications into the main text.
Questions for authors
  1. The paper evaluates several MIA families, including shadow-model, query-adaptive, and density-based attacks. How were these attacks selected and tuned? Could suitable black-box MIDST attacks, such as Wu et al. (2025) or MIA-EPT, provide additional evidence? Please clarify their applicability to SDS and any required adaptations or auxiliary information.
  2. Why does the evaluation not include additional lightweight tabular generators, such as Bayesian networks or sequential decision-tree-based synthesis (e.g., synthpop)? Given SDS’s emphasis on small datasets and low computational cost, these appear particularly relevant baselines. Comparing them under matched training-data and synthetic-context sizes would help clarify whether SDS offers a better privacy–utility trade-off than other inexpensive synthesis methods.
  3. Why does the evaluation omit recent tabular generators such as TabDDPM, TabSyn, and TabDiff, as well as autoregressive approaches such as GReaT? Comparisons under matched training-data and synthetic-context sizes would help establish whether SDS offers a better privacy–utility trade-off than these alternatives. If these methods are unsuitable for the small-data setting considered, could the authors substantiate this experimentally or explain the practical constraints preventing their inclusion?
2 comments
AuthorsOpenReview ↗

Response to Reviewer dESz

Thank you for noting the low integration overhead, the transfer across readers, the breadth of the audit, and our open reporting of residual leakage.

  1. Privacy of query records: We designed SDS so the owner can share a synthetic context, which the recipient uses with the unchanged reader (Hollmann et al., Qu et al.), while keeping the real training table local. Who sees the queries depends on where the reader runs inference. If the recipient runs it locally, they keep the queries. If they use a hosted service, the host sees the queries, and SDS does not protect against this.

    In the setting of sharing data between institutions (Rieke et al.), Hospital A would provide a synthetic context for Hospital B or a health fund to score its patients locally. Hospital A receives no queries, and Hospital B receives a synthetic context instead of Hospital A’s patient table. The revision makes this distinction explicit in Section 2.

  2. Presentation. We added Table 4 to the main text. It summarizes the component ablation, comparisons to matched generators, the strongest attacks and the memorizing control, the population control for attribute leakage, and failures under repeated releases.

    Each headline result states its dataset suite and its limitations. For example, the geometric reconstruction score can also be high for a permuted copy of the real rows, so we keep it as a diagnostic.

  3. Attack selection: Our audit spans nearest-neighbor distance, query-adaptive predictions, shadow models, density ratios and local likelihood changes. We agree that the default versions of the attacks may not be the strongest possible. We specify in App. F.24 (Table 34) what auxiliary data each attack uses, and how we adapt and tune it.

    In a new experiment for this rebuttal, we adapted two more attacks to SDS: MIA-EPT, which uses attribute prediction errors, and the black-box attack of Wu et al., which uses the denoising loss of a diffusion surrogate trained on the released table, as in the original. Access to a synthetic table alone does not make the competition protocols the same, so we document the adaptations and the shadow-training data.

    With full covariance, the higher of the two AUCs reached 0.62–0.66 on Waveform, Optdigits and Dermatology. We switched the mixture half of SDS to diagonal covariance on the five tables with at least 30 features, with no decrease in mean accuracy. On the Table 32 audit split (3 seeds), mean AUCs against SDS after the switch are 0.535 and 0.542. Both attacks stay below 0.6 on every dataset. Against the memorizing control, the attacks reach 0.674 and 0.667, vs. 0.729 for the paper’s average of the strongest attack on each dataset. We report both attacks per dataset (App. F.23, Table 33).

  4. Bayesian networks and synthpop: In a new experiment for this rebuttal at matched training and context sizes, the mean accuracy gap of a Bayesian network with two parents across the 14-dataset suite is -16.7 points, vs. -1.2 for SDS rerun on the same 3 folds (one seed, -1.4 in the paper).

    For our own synthpop-style sequential CART, the mean accuracy gap is also -1.2 points, but the mean nearest-neighbor membership AUC is 0.626, vs. 0.541 for SDS in the same run (0.54 in the paper). CART’s AUC is higher on all 14 datasets, so it is competitive on utility and leaks more under this attack. The Bayesian-network result does not isolate the effect of discretization. Table 67 gives both comparisons.

  5. TabDDPM, TabSyn, TabDiff, GReaT: In a separate new experiment at matched training and context sizes, we compare the mean accuracy across 14 datasets, using 3 folds with one seed (we rerun SDS in the same setting). The training schedules were reduced to 4%–50% of the repository defaults without tuning hyperparameters (Table 20). The mean accuracy gap is -1.0 points for SDS (vs. -1.2 above), -2.2 for TabSyn, -6.2 for TabDiff and -15.1 for TabDDPM. The training loss for TabDDPM was still dropping at 4,000 steps.

    The mean nearest-neighbor membership AUC is 0.543 for SDS (vs. 0.541 above and 0.54 in the paper), 0.612 for TabSyn and 0.520 for TabDiff. TabDiff has a lower AUC than SDS on 12 of 14 datasets, SDS has a smaller accuracy gap on 12 of 14, and neither dominates on both measures.

    GReaT writes each row token by token with a language model, so we attempted 5 of the 14 datasets with a one-hour limit per run. It finished Iris, Wine, Diabetes and Vehicle, but sampling on BreastCancer with 30 features did not finish in time. For the 4 that it finished, the mean accuracy gap is -29.4 points, vs. -1.4 for SDS, although it has a lower membership AUC (0.505 vs. 0.549). Under these budgets, SDS has the smallest mean accuracy gap and fits in less than one second. The deep generators may do better at their standard schedules. The revision gives training schedules, runtime, convergence and incomplete runs (App. F.11, Tables 19 and 20).

Thank you for clarifying the query privacy assumptions and expanding the privacy audit and baseline comparisons.

Reviewer_2D34OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

the authors here propose to replace labeled training context (for ICL) that is sent to a hosted tabular foundation model (like e.g. tabpfn, tabfm, etc.) with a synthetic context that is fitted locally. then, the server sees only synthetic rows, which may have data privacy benefits. empirically the authors show that under their scheme the accuracy stays within a few points on a relatively large collection of small standard ML datasets. they also show empirically MIA attacks stay near chance on those datasets.

Critical strengths

this is a large and extensive empirical look at the proposed method. they consider a wide variety of attack families and ablations on those. the appendix potentially contains a large amount of useful data points for practitioners, although i wonder if they author's couldve done a better job of selecting out the most representative results to solidify takeaways, instead of just dumping additional experiments. i had trouble going through them all.

Critical weaknesses

i think privacy as a motivation is really loose for this scheme. the paper seems to admit this at certain points? SDS defends against using a hosted service for ICL, and motivates this by saying some of these ICL models have gotten really big. but tabpfn v2 I think is only around 10M parameters? fits on my laptop at least. and a few of teh other models they consider are also small. so the threat the paper defends against is one the data owner can avoid by not using a hosted service, and the tradeoffs are not obviously in SDS's favor based on their experimental results

so what does SDS get you? from a privacy perspective, it does not satisfy DP (the authors admit this). this is unsatisfying as then we just have to hope that the attackers run the attacks from the paper. also, the authors then compare against DP methods. i appreciate that the authors were upfront about SDS not satisfying DP, but then im not sure what im supposed to get out of the tables showing a DP synthetic data mechanism (PrivBayes, not necessarily a state-of-the-art one, and unclear which implementation) losing by 40+ points. DP is a worst-case guarantee, and is difficult to satisfy, but has a much much stronger privacy guarantee. there's an entire literature on DP synth data that the authors would need more carefully consider if they want to present results like this.

the author's approach then is an algorithm that stitches together standard synth data methods. a copula, a GMM, etc. and then uses the nice advantage attack from stadler et al. for assessing attack strength. the criteria of the algorithm are not theoretically motivated, so i'm not sure if this is really an algorithmic / methodological contribution, more than just advice for which algorithm to use in which setting.

also, MIA with a GMM that only has a few components will of course not memorize rows. attribute inference does seem to work against the overall framework, which would be concerning to me if i wanted to deploy this thing.

i find the theory to be decorative and distracting. theorem 1 makes a lipschitz assumption and then applies the traingle inequality, and i think thats it? its then not really used to guide the experiments as far as i can tell, and im not really sure how it improves my understanding of the method. im also not really sure what im suppose to get from proposition 1, it is oddly stated. perhaps the authors could help here?

Questions for authors
  1. what does section 5 establish that is not either assumed (as in thm 1) or vacuous on the data you chose (as in thm 2)?
  2. why should i be surprised that using a lossy few component GMM does not memorize? then, of course attribute inference and other attacks would work on such a model (i.e. recovery of rare attributes, etc.?) for certain datasets. i think under-parameterizing a model is likely not a sufficient privacy scheme, even if it is minimally lossy for a select collection of data.
  3. Don't DP-TabICL (Carey et al. 2024) and TabPATE (Wahdany et al. 2026) address exactly this problem? can you run some comparisons, they seem like the most relevant baselines (i.e. better than just PrivBayes without adjustment)?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 2D34, Part 1 of 2 (points 1 to 3)

Thank you for your thoughtful review and for noting the breadth of our empirical studies, covering a wide range of attack families and ablations, and the useful results in the appendix. We agree that the appendix could be curated better. In the new version, we promote the most important results to the main text (Table 4 and Figure 3).

  1. Motivation and model size: We think that this part was unclear, and the example we chose (hosting) has contributed to this. In the new version, we write this much more explicitly (Sections 1 and 2). A data owner wishes to allow a second party to use a predictor based on the owner’s confidential labeled data. In tabular in-context learning (ICL), the weights of the foundation model are held fixed, and the predictor for each task depends on a set of examples (the context, see Hollmann et al. and Qu et al.). To share this predictor, the owner needs to share the context or some representation of it. SDS provides a synthetic labeled table that the second party can use with the unchanged reader. We measure leakage from the synthetic release separately.

    As an example, Hospital A may wish to share a readmission predictor with Hospital B, or with a health fund. Hospital A shares a synthetic context, as well as the schema of the features and the preprocessing steps. Hospital B can then use the reader model to score its own patients. A’s patient table stays at A, and B’s queries stay at B. This motivation is well established in the healthcare literature (Rieke et al.). We have seen similar limits on sharing data in healthcare, banking and cellular networks.

    We agree that if the data owner wishes to score its own data, it can simply run TabPFN-2.5 locally on a laptop. If the recipient uses a hosted service, that service will see the queries, and we do not provide protection against that. The owner’s training rows stay with the owner.

    For this rebuttal, we also considered the alternative of training and shipping a tuned XGBoost model (14 datasets, 3 folds, one seed). We found it to be 1.0 point less accurate than using SDS and the reader model. XGBoost is less accurate on 10 of the 14 datasets. For 3 of the remaining 4, its lead is under one point, and for all 4, SDS achieves a lower attack AUC. If we run the same loss-threshold attack on the same records, we get 0.569 AUC for XGBoost vs. 0.536 for SDS, averaged across datasets (App. A).

  2. DP, and the DP comparison: We do not claim to provide DP guarantees, and synthetic data does not automatically provide anonymity (Stadler et al.). SDS reduces measured exposure. The -1.4-point headline gap uses full covariance (14 datasets, 5 folds, 3 seeds). DP provides formal worst-case guarantees, and the utility cost depends on the mechanism and its noise. Most PrivBayes and AIM results in the paper were on par with the constant-class predictor on these small tables.

    For this rebuttal, we ran AIM and MST, two marginal-based DP methods, on the 14-dataset suite, using the official code and full privacy accounting (3 seeds). Both are less accurate than SDS on all 14 datasets at every tested ε from 1 to 100, by 16 to 55 points on average. In this DP comparison, the mean of the strongest attack AUC per dataset is 0.51 to 0.54 for AIM and MST, vs. 0.552 for SDS. Neither dominates SDS on both axes (App. F.21).

    The revision names the implementation of PrivBayes we used, its settings and the conditions under which we compared it to SDS. We show the constant-class scores next to the DP results (Tables 29 and 30) and remove the broader claim that DP necessarily destroys utility.

  3. “We have to hope the attackers run the attacks from the paper”: You make a fair point. Our audit covers a range of attacks, and we check how well each attack detects a generator that deliberately memorizes the training rows. The attacks detect it on average, but not on every dataset. In the separate Table 32 audit, we average the AUC of the strongest of four attacks on each dataset, and find that it is 0.555 for SDS and 0.729 for the memorizing control. Other attacks may do better.

    For this rebuttal, we have adapted two black-box attacks from MIDST 2025, MIA-EPT and Wu et al. (where the latter trains a diffusion model as a surrogate on the released data, as in the original). On the Table 32 audit split (3 seeds), they achieve a mean AUC of 0.535 and 0.542 for SDS after the switch described below, respectively, vs. 0.674 and 0.667 for the memorizing generator. With full covariance, the higher of the two AUCs reached 0.62–0.66 on Waveform, Optdigits and Dermatology. We switched the mixture half to diagonal covariance on the five tables with at least 30 features. Both attacks then stayed below 0.6 on every dataset, with no decrease in mean accuracy (App. F.23, Table 33). The conclusion is reduced measured leakage under the attacks we ran, not immunity to membership inference.

AuthorsOpenReview ↗

Response to Reviewer 2D34, Part 2 of 2 (points 4 to 7)

  1. Novelty: We agree that the contribution should be judged by the problem and the evidence. We give a simple way to share a tabular in-context predictor whose labeled context is confidential, and test it across 71 datasets. There are six reader implementations in the main comparison (Section 6), and a separate experiment with TabICL v1. The mean accuracy gaps across transfer readers range from -1.1 to -2.5 points.

    The matched baselines show why the choice of generator matters. Synthpop-style CART matches our accuracy gap but has higher membership inference AUC on all 14 datasets (Table 67). TabSyn is worse on both measures under the tested budget (Table 19). The ablations support the ensemble and routing choices.

    TuneTables does replace the real context, but does so by releasing a prompt tuned to one reader. Our synthetic rows work across the readers we tested. Changing what gets transferred has a rich literature, e.g. knowledge distillation (Hinton et al.) and dataset distillation (Wang et al.). SDS has a different objective: sharing a synthetic context for an unchanged reader while measuring its utility and disclosure risk. The contribution is the formulation, method and evidence. We use established mixture estimators.

  2. Few-component GMM and attribute inference: We agree that low capacity on its own does not guarantee privacy. What we find is that a context built with a GMM with few components can still support a strong reader, with lower measured membership leakage. SDS has an accuracy gap of -1.4 points in the ablation on the 14-dataset suite (3 folds), vs. -2.6 for one Gaussian per class, and -5.1 for adding noise to the real rows, which still leaks more.

    We agree that SDS releases do support some attribute inference. In App. L.17, adding them gives Ridge and Extra Trees an excess R² of 0.087 and 0.021, respectively, over an attacker trained on an independent population sample. We can harden the release to reduce raw attribute R² from 0.77 to 0.52 on the real-data core, but it costs another 0.9 accuracy points there (Table 26) and 2.8 points on the 14-dataset suite (Table 27).

    In a new experiment, we find that the average excess R² for Extra Trees on the rarest tenth of records is 0.025 (standard error 0.024), with 0.19 on the worst dataset. We have not established that these rare records are safe. The revision gives the results per dataset (Table 74).

  3. DP-TabICL and TabPATE: In new experiments, we have reimplemented DP-TabICL and TabPATE on the 14-dataset suite (App. F.21, Table 31). All settings are fixed in advance, we score every dataset, and we use 3 seeds. The full privacy accounting includes the feature bounds, which we release with a DP quantile mechanism and charge to ε.

    The variants lose 35 to 63 points at ε from 1 to 10 (vs. 1.4 for SDS), and trail SDS on all 14 datasets at ε=1 and ε=4 (paired standard errors of 2.3 to 7.6 points). They are still worse at ε=100, by 9 to 35 points. Only the noise-free version of the perturbed-record release matches SDS, but has strongest-attack AUC of 0.875 vs. 0.729 for the memorizing control. In the App. F.21 comparison, averaging the strongest attack per dataset gives 0.50 to 0.53 AUC for the DP releases at ε≤10 vs. 0.552 for SDS, so none dominates on both axes.

  4. Theory: A key contribution of Section 5 is separating these risks and auditing them separately. We ask whether the synthetic distribution preserves predictions, whether replacing a record changes the release, and whether released correlations help predict an attribute. We add generator stability and population control experiments alongside the distance-based ones. We agree that Theorem 1 is simple given a Lipschitz assumption, and have moved it to App. C.3.

    For Theorem 2 we show that the replace-one KL of the full release law is the quantity to track for membership, but the bounds are mostly vacuous in the headline regime. In one new run, we use the audit of Steinke et al. and plant held-out records as canaries. The mean empirical ε lower bound on typical records is 0.04 for SDS, vs. 0.004 to 0.03 for the DP releases at ε=1 and 10, and 1.26 for the memorizing control (App. F.21). These are lower bounds under one attack. Similar values do not establish comparable privacy guarantees.

    For Theorem 3 we show that if the generator is distributionally faithful, and the loss is bounded, then it can transfer the ability to predict the attribute. This does not account for the observed R², and we do not prove that all of it is population level. Proposition 1 says that the distances among synthetic rows converge to those of the fitted generator, not of the training table. In Section 5 we explain the limits of each result, including why this does not show protection against record recovery.

thanks for the additional experiments. i appreciate your efforts, but my score is unchanged. as i see it, the core issues remain and are unsurmountable at this point: there is no formal guarantee and it seems like your stated motivation for the paper is shifting which worries me (if you're doing synthetic data release, then you need to compare directly to more DP synth data methods). again, the theory is basically decorative.

i am also maintaining my score because my fellow reviewers are, in my opinion, too positive here. one reviewer gives credit to the paper for “proving” that membership leakage can be suppressed independently of accuracy. the authors themselves admit that theorem 2 is ostly vacuous, and they don’t prove a population level result (the attacks are detecting memorization on average but not for every dataset).

also, the additional experiments are not super favorable? seq. CART is the same performance-wise as SDS, and TabDiff is just better (as far as i can tell? am i misinterpreting?)

AuthorsOpenReview ↗

Follow-up to Reviewer 2D34 on the new comparisons, motivation and theory.

Thank you for engaging with our response! Following your suggestions, we added DP-TabICL, TabPATE, AIM and MST with full privacy accounting, and a rare-record analysis, and moved Theorem 1 to the appendix. We believe that these strengthen the case for SDS.

Q: Am I misinterpreting the new comparisons?

Indeed, none of the added baselines match SDS on both accuracy and measured membership leakage (Table 4). Those approaching SDS’s accuracy have higher leakage; those achieving comparable or lower leakage lose at least 5 accuracy points.

Among these baselines, SDS offers the most favorable trade-off for our objective of preserving accuracy while limiting measured leakage. This also holds in our DP comparisons with full privacy accounting: AIM and MST (MST generalizes the winner of the NIST contest for DP synthetic data, and AIM is its successor) lose 17–22 accuracy points even at ε=100 and 24-56 points at ε from 1 to 10, while DP-TabICL and TabPATE lose 35–63 points at ε=1–10, against 1.4 for SDS. Their attack AUCs are lower (0.50–0.54 versus 0.552 for SDS), but their accuracy is substantially worse (Table 31). These empirical comparisons do not imply equivalent privacy guarantees.

Sequential CART has the same accuracy gap as SDS (-1.2) but a higher membership AUC on all 14 datasets (0.626 vs 0.541, Table 67).

TabDiff is 5.2 points less accurate than SDS (-6.2 vs -1.0) for an AUC that is only 0.023 lower (0.520 vs 0.543, Table 19). One can always remove leakage completely by releasing pure noise.

Motivation. The threat model is already stated in the abstract of the original submission: "when a third party performs classification, the training data used as context must also be shared with that party." Section 2 already had the same honest-but-curious external party. Only the example changed, not the problem or the threat model.

Theory. The theory serves to explain which property of the generator controls which risk. We never claimed that it is a formal privacy guarantee, and we did not use the word “proving”. The sentence you quote is about the attacks, not about our method. It states that the attacks detect memorization on average, that is, they can tell when the data comes from the deliberately memorizing control.

Summary. The paper builds on the recent development that tabular foundation models have emerged as a promising method for small tabular datasets (Hollmann et al., 2025; Qu et al., 2025), and that using them across institutions requires sharing the training data, which privacy rules restrict, e.g., in healthcare (Yoon et al., 2020). It is well established that generating synthetic data does not remove the privacy risk (Stadler et al., 2022). Therefore, we use a large number of different attacks to assess the privacy risk of the generated data. This is a method paper, and our strength lies in the empirical part. You identified this as a strength in your original review: “this is a large and extensive empirical look at the proposed method.” All new experiments follow the same trend: no method achieves similarly high accuracy while leaking less, and the methods that leak less have much lower accuracy.

We hope the new results address your concerns, and that you will consider them, together with the empirical strengths you noted in your original review, in your final score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.