- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The tabular generative models in use nowadays are able to fabricate or impute realistic scientific datasets effortlessly. These datasets can bypass traditional statistical fraud forensics i.e., Benford’s law and GRIM or SPRITE checks, creating an urgent need to embed verifiable provenance at generation time rather than relying on post-hoc detection. However, existing tabular watermarks (i) can be detected and scrubbed by adversaries, (ii) lack the rigorous false-positive guarantees needed for research misconduct investigations, (iii) and break under routine table formatting edits. To solve this, the authors introduce TABULAR PANCAKES, a training-free cryptographic watermark that replaces the generator's standard Gaussian latent prior with a lattice-based distribution that looks like ordinary Gaussian noise in all directions except a secret periodic subspace, alongside a direct data-space variant for non-invertible diffusion models whose reverse trajectories severely amplify inversion errors. Evaluations across four datasets and six generator families show that the watermark (i) is cryptographically undetectable without the secret key, (ii) provably requires substantially higher distortion to scrub without the key than with it, (iii) remains strictly invariant to row shuffling, column renaming, and unit rescaling, and (iv) provides certified statistical significance alongside row-level localization to pinpoint synthetic records inside partially fabricated tables.
Critical strengths
The motivation behind the study is well laid out and proves why a study in this direction is necessary. The method proposed by the authors is built upon cryptographic and statistical foundation tailored for forensic auditing. Basically, rather than relying on heuristic markers that classifiers can easily detect and scrub, the authors ground their scheme in lattice hardness which provides evidentiary standards that include certified false-positive control and robust invariance to routine table edits. Also, another vital technical contribution by the authors is the identification of the latent inversion barrier in diffusion-VAE pipelines which proves that reverse ODE trajectories amplify errors by up to 100x and wipes out fine phase signals. Based on these contributions, I believe the authors have shown the significance of thier study.
Critical weaknesses
The fundamental issues of watermarking still exists. (1) Retraining a model using only the marked synthetic data destroys the mark, forcing the authors to rely on lower-bounding the high utility and distortion costs of such removal rather than preventing it entirely. (2) Datasets with very few continuous or numeric columns heavily constrain performance, they have mentioned this in their limitations, but this shows that their method is only useful for continuous dataset. Furthermore, they didn't evaluate on deterministic datasets (i.e., datasets with columns which are generated based on other columns, such as a column which is linearly dependent on the combination of other columns). These kinds of datasets may be continuos in nature, but what happens when their approach is used for marking such?
Questions for authors
Please check weaknesses above.
4 comments
Author Response (2/2)
W2a: "Only useful for continuous datasets."
We agree that the method is best suited to tables with mostly numeric columns and state this in the revision. Categorical columns, however, are not unused: they carry the keyed synchronisation signal, and we add a new experiment characterising when this channel remains undetectable.
We investigate collision as a possible failure mode: rows that induce the same synchronisation pattern could be grouped and compared, and systematic agreement within groups could reveal the mark. We measure this with category-conditional two-sample tests. Here denotes the number of secret marking directions, i.e. the dimension of the hidden periodic subspace; the synchronisation pattern takes one value per direction, so the number of distinct patterns grows exponentially in k and repeated patterns become rare. We include as a theoretical worst-case baseline; here, the channel indeed leaks and the marked table contains 2.77M colliding row pairs. Notably, however, the leak decays with the collision count: at the tests are at chance (AUC 0.50-0.51; 5 colliding pairs), which is in agreement with the analytically predicted counts. With a simple mitigation technique---additional synchronisation levels and binning---using already suffices while using provides strict indistinguishability. The revision therefore prescribes with this mitigation for mixed-type tables, which coincides with the configuration recommended for localization and therefore, adds no new constraint in practice. Because the collision count is computable from the key parameters and the table schema, the safety condition can be verified before deployment.
W2b: "What happens on deterministic or linearly dependent columns?"
To answer this question, we constructed a suite of tables with exact column dependencies (linear combinations, sums, integer relations, and redundant columns with small jitter). The default codec is designed for unconstrained tables and is not the appropriate tool here; for dependency-constrained data the revision adopts a rank-restricted codec, which preserves the constraints by construction. An exact linear dependency among columns corresponds to a zero-variance direction of the data covariance. The rank codec performs PCA whitening truncated at a threshold and embeds the mark only in the retained subspace. The constraint null space has zero variance and is therefore never touched: the watermark cannot alter any direction in which the data does not vary, and dependent columns are reconstructed exactly from the retained components. Evaluation on the suite confirms the guarantee:
| Suite with exact dependencies | rank codec (=1e-3) |
|---|---|
| constraint violations | 0.000 (pre-formatting drift < 1e-10) |
| detection statistic S | 0.846 (vs. 0.867 unconstrained; cost 0.021) |
| redundant-column jitter | S = 0.861 (default codec: 0.645) |
Two refinements remain and are left for future work. First, the final formatting step (clipping to the observed data range) perturbs 3-4% of rows, leaving a secondary statistical trace; constraint-aware formatting would remove this step's effect. Second, columns supported on coarse integer grids weaken detection (); this is the same integer-quantisation regime as in W2a and motivates integer-lattice codecs, a direction we consider highly promising given the method's lattice foundation.
Summary.
We present three supplementary experiments. Our retraining study shows that removal by distillation is either paid for in fidelity or leaves a certifiable residue. The dependency suite introduces the rank codec, which preserves exact constraints by construction at negligible detection cost. The collision analysis equips the categorical channel with a safety condition, , that can be verified before deployment. Together, these results extend the paper's certified provenance guarantees (keyed undetectability, exact false-positive control, invariance to routine edits) to structured and mixed-type data. They also sharpen the comparison with latent-space schemes: under the deployed decode-and-invert round trip, no latent-space watermark certifies at any rows, whereas the data-space mark certifies from and is unaffected by shuffling, deletion, cell noise, and rescaling.
We hope these results provide a rigorous basis for data provenance in scientific forensics. Thank you for your insights; we hope these revisions helped address your concerns and welcome further discussion!
(2/2)
Nice! even though it came close to the end of the discussion window. Thanks for the clarification with new experiments, and I wasn't expecting the algorithmic extension. The new experiments directly address my concerns. Please do well to add them to the revision. I will update my soundness and overall rating to further support acceptance.
Thank you for actively participating and updating your score! We are delighted that our rebuttal addressed your concerns and will certainly incorporate these changes in the revision.
Author Response (1/2)
Thank you for your positive and insightful review! We are encouraged that you found the study well-motivated and our lattice-based construction a sound foundation for forensic auditing. In response to the two weaknesses, we conducted new experiments on (i) mark survival under student-generator retraining and (ii) marking of dependency-constrained and mixed-type tables. They quantify the cost of removal (W1) and extend the codec to preserve exact column dependencies (W2). We summarise results below and revise Sections 3.4, 5.4, 6 and the Appendix accordingly:
W1: "Retraining on the marked data destroys the mark."
To summarise, we find that retraining does not reliably destroy the mark, and when it does, this does not come for free. Students that preserve data quality inherit a certifiable mark; students that shed it measurably degrade the data. To test this, we trained five student generators on marked output (3 datasets x 3 train sizes 6k-60k x 3 keys x 3 student seeds; 27 certified runs per cell) and ran the deployed certificate on each student's own samples (adult shown; full grid in App. D).
*The Gaussian copula reproduces each column's marginal exactly by construction, so the column-shape metric cannot register its cost; the cost appears in cross-column structure, as a 0.05 drop in downstream prediction accuracy. Concretely:
We include and discuss these results in Sec. 5.4. The watermark is a component of the generator's output distribution: a low-amplitude periodic modulation of the density along the secret directions. A well-trained student can match that distribution and reproduce the modulation; therefore, the certificate fires on the student's own samples (SMOTE, ARF). If it cannot represent the modulation, its output distribution misses a component of the training distribution, and the gap surfaces as the fidelity costs in the table (CTGAN, TVAE, copula). Shedding the mark by retraining would require an imperfect fit of the training distribution and triggers a removal--fidelity trade-off. This trade-off is governed by the mark’s amplitude (γ, β, k), which is chosen by the auditor. While retraining cannot be wholly prevented, our results show that under Tabular Pancakes, this would result in either a certifiable residue or a measurable quality gap. We note that the false-positive guarantee holds unchanged in both because it rests on key rerandomization alone and does not rely on assumptions about the adversary.
Removal without retraining is also now measured in full. Gradient search with up to 10^6 restarts fails to locate the latent secret subspace (zero recoveries; ascent converges only from starts within ~0.2 rad of the true direction). Across seven budgeted removal attacks, even direction-informed attackers need cell-level RMSE of 0.05-0.30 sigma to defeat the certificate, and model-based re-imputation never succeeds within budget, for the same reason SMOTE and ARF inherit the mark above: the imputation model re-learns the marked conditionals. The complete results are in App. D.
[1] Sablayrolles, A., Douze, M., Schmid, C., & Jégou, H. (2020, November). Radioactive data: tracing through training. In International Conference on Machine Learning (pp. 8326-8335). PMLR.
(1/2; dependency-constrained and categorical data in the next reply)