- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies a zero-shot adversary who observes a table with a sensitive column removed and attempts to infer its values without receiving ground-truth target labels. The main contribution is an adaptive rule for determining how many pseudo-labeled examples to select from each predicted class.
Critical strengths
The paper studies whether access to an entire released table enables more inference than inspecting its rows independently. It distinguishes individual feature disclosure from access to aggregate target proportions.
The evaluation includes two datasets, four ACS states, several sensitive attributes, multiple LLMs, and both classification and geographic prediction.
Critical weaknesses
This paper uses zero-shot LLM predictions as pseudo-labels to fit a downstream classifier, which is an established pattern in NLP. Examples include self-training on zero-shot classifier outputs (e.g., Gera et al., 2022) and LLM-as-annotator distillation. The related work frames pseudo-labelling only through Noisy Student and FixMatch.
The attacker is assumed to has no labelled examples, but the full ACS and NHIS public files, with all these attributes included, are available. An attacker would simply train a supervised model on that data and do much better than either E0 or E1. The paper also does not discuss whether predicting someone’s attributes from general population patterns should count as a privacy harm to the people in the released table.
E1 learns patterns from unlabeled rows, but any unlabeled samples from the same population would provide the same patterns. To show that releasing the table adds risk, the authors could build the context from a different group of people and see whether the gain remains. They could also test whether the gain grows as the released table becomes larger.
All the results come from deterministic selection with seed 42 on test sets of 1,200 samples. The reported result differences are quite small and close to be noise.
Questions for authors
What are the key differences of this paper compared to the closely related methods?
How much of E1’s gain can be matched by simply rescaling the LLM scores, without training any model?
Could you provide per-class precision and recall for selected targets?
Does E1’s gain persist when the pseudo-context comes from a different, non-released cohort? How does it scale with cohort size?
Why does E1 help Qwen3-14B but mostly not GPT-4.1?
Can you explain the mismatch between Tables 9 and 19, and confirm which TabPFN version was used?
6 comments
Score rescaling and model differences (Q2, Q5)
Question 2.
How much of E1's gain can be matched by simply rescaling the LLM scores, without training any model?
Response: We test whether a class-dependent adjustment of the existing scores can replace tabular learning. Let be the original LLM score and its mean across samples for class . On all 66 Qwen strict/weak tasks, we apply two fixed rules:
- Ratio rescaling: . Dividing by the class mean reduces the relative advantage of classes that receive high scores across most samples.
- Positive-evidence rescaling: . Only support above the class mean is retained; scores at or below that mean contribute zero evidence. Degenerate columns are set to zero.
Each rule predicts for every sample; an all-zero evidence row retains E0. Neither rule constructs context or fits a model. E1 instead uses evidence to select pseudo-labels, learns feature dependencies, and fuses tabular predictions with the original scores. No target labels tune any rule. Mean Accuracy/Macro Recall changes from E0 (percentage points) are:
| Dataset / condition | Score / class mean | Positive evidence | E1 |
|---|---|---|---|
| ACS strict | −1.22 / +2.39 | +1.59 / +1.75 | +3.35 / +2.20 |
| ACS weak | −2.51 / +3.40 | +1.20 / +2.57 | +2.43 / +1.73 |
| NHIS strict | −8.47 / +0.04 | −3.13 / +0.09 | +0.22 / +0.71 |
| NHIS weak | −0.05 / +4.17 | +0.47 / +3.33 | +0.85 / +1.32 |
E1 achieves higher mean accuracy than both score-only transformations in all four groups, while also improving macro recall over E0 in each group. Thus the tabular learner adds recovery beyond these direct score adjustments. Both controls use across-sample class means, making this a comparison of score correction with learned transfer under the same information access. The existing permutation diagnostic in Table 21 further supports the role of feature–pseudo-label correspondence: retaining context rows and label counts while shuffling labels changes the ACS weak accuracy increment from +2.43 to −9.47 points.
Revised manuscript: Appendix, "Score-only controls" (transformations and matched results); Table 21, Appendix F.1 (existing permutation diagnostic).
Question 5.
Why does E1 help Qwen3-14B but mostly not GPT-4.1?
Response: GPT-4.1 provides stronger initial predictions and therefore leaves less recovery to the table-level step. In our 24 matched strict/weak tasks, 59.50% of the errors corrected by Qwen E1 were already correct under GPT E0. Nevertheless, E1 still recovers additional private attributes beyond GPT's own row-level predictions. This additional recovery is the relevant test of whether the table can provide useful information after a strong LLM has made its initial predictions.
Additional recovery with GPT-4.1. Across the seven ACS New York targets under strict disclosure, E1 raises GPT's mean accuracy from 47.32% to 48.24% (+0.92 percentage points). At the target level, INC improves from 73.17% to 79.08% (+5.92 points), and SCH from 38.58% to 40.83% (+2.25 points). On NHIS strict AGE, accuracy improves from 40.08% to 43.83% (+3.75 points). These are net improvements over GPT's own E0, after accounting for both corrected predictions and newly introduced errors. They use the same initial GPT outputs, no ground-truth target labels, and no additional LLM calls: the extra recovery comes from constructing pseudo-context and applying tabular inference.
Why some group averages decrease. E1 fits pseudo-labels as supervision. Their mistakes can therefore be learned as feature dependencies and propagated to samples whose E0 predictions were correct. Stronger E0 reduces correction opportunities but does not ensure that selection produces better training labels. For NHIS weak SCH, GPT context precision is 38.80%, versus 38.92% E0 accuracy, with only four of five classes represented: selection supplies noisy, incomplete supervision. The loss arises in tabular transfer, not simply fusion: GPT fused accuracy exceeds direct TabPFN accuracy in all four matched groups, while both remain below E0 in the negative groups (Appendix F.2, Tables 26-27). These comparisons are consistent with pseudo-label error propagation.
These results show that table-level inference can recover additional private attributes even after GPT-4.1's stronger initial predictions. Section 5.2 reports the matched comparisons, and Appendix F.2 analyzes the context quality and prediction changes associated with the different outcomes.
Revised manuscript: Section 5.2 (matched-model results); Appendix F.2, Tables 26-27 (context quality and task-level prediction changes).
Context source, dataset size, and stability (W3-W4, Q4)
Weakness 3.
E1 learns patterns from unlabeled rows, but any unlabeled samples from the same population would provide the same patterns. To show that releasing the table adds risk, the authors could build the context from a different group of people and see whether the gain remains. They could also test whether the gain grows as the released table becomes larger.
Response: We tested context from different people through all 168 cross-state transfers: four destination states, each using the other three states, seven targets, and strict/weak disclosure. Destination E0 scores and the 1,200 queried samples stay fixed. External-context mean accuracy/macro-recall increments are +3.04/+1.88 points under strict and +2.18/+1.39 under weak. On the identical samples outside the destination's original context, increments remain +3.50/+2.36 and +2.96/+1.56. The transfer therefore also works with pseudo-context from different people. The next control in Q4 addresses the more specific non-released-cohort question while holding population source fixed.
Effect of available table size. The size study fixes 200 queried samples and increases the available donor pool. Context is reselected at each size; donor count is not selected context size. Three fixed partitions across 66 tasks give 990 runs. Mean ACS accuracy increments over the same query E0 predictions are:
| Donor samples | Strict ΔAccuracy (pp) | Weak ΔAccuracy (pp) |
|---|---|---|
| 100 | +1.15 | +1.39 |
| 200 | +1.61 | +1.86 |
| 400 | +2.48 | +1.90 |
| 800 | +2.92 | +2.16 |
| 1,000 | +3.18 | +2.18 |
From 100 to 1,000 donors, ACS macro-recall increments also rise from +0.00 to +1.85 points under strict and from +0.08 to +1.48 under weak. Increasing donor availability strengthens average ACS recovery on a fixed evaluation set.
Revised manuscript: Section 6; Appendix, Context source, available sample size, and backend seeds (cross-state and size controls).
Question 4.
Does E1's gain persist when the pseudo-context comes from a different, non-released cohort? How does it scale with cohort size?
Response: A non-released cohort means a separate auxiliary group available to the attacker but not included in the dataset being attacked. To test this distinction, we constructed a same-population, simulated-release comparison from three fixed partitions of the cached participants. Each partition defines 200 fixed queries and 400 internal donors; a disjoint 400-sample pool supplies external context. Queries never enter either context. Across 396 runs, ACS internal/external accuracy increments are +2.48/+2.85 points under strict and +1.90/+1.72 under weak. Both donor sources support additional recovery. This controlled partition changes release membership while retaining the population source and donor count.
Revised manuscript: Section 6; Appendix, Context source, available sample size, and backend seeds (disjoint donor controls).
Weakness 4.
All the results come from deterministic selection with seed 42 on test sets of 1,200 samples. The reported result differences are quite small and close to be noise.
Response: The average ACS gains persist across all four states and across repeated TabPFN fits, with small aggregate variation across seeds. Each state has positive mean accuracy/macro-recall increments under both strict and weak disclosure. We additionally reran all 66 tasks with TabPFN seeds 17, 42, 82, and 2026, keeping E0 scores and pseudo-context fixed. All 66 seed-82 runs exactly reproduce the archived predictions. The four-seed mean increments ± sample standard deviations are:
| Dataset / condition | Delta accuracy (pp) | Delta macro recall (pp) |
|---|---|---|
| ACS strict | +3.21 ± 0.22 | +2.09 ± 0.18 |
| ACS weak | +2.41 ± 0.01 | +1.70 ± 0.03 |
| NHIS strict | +0.10 ± 0.09 | +0.62 ± 0.08 |
| NHIS weak | +0.85 ± 0.04 | +1.30 ± 0.03 |
The ACS accuracy and recall increments remain positive for every tested seed; NHIS macro recall also improves for every seed under both conditions. The ACS mean increments are substantially larger than their variation across backend seeds, supporting reproducible average improvement. Appendix F.3 additionally reports paired bootstrap intervals for all 66 fixed-prediction evaluations. The LLM generation seed is 42 and the tabular seed is 82; Appendix B.3 specifies their roles.
Revised manuscript: Appendix F.3 (existing paired intervals); Appendix B.3 (seed roles). Appendix, Context source, available sample size, and backend seeds (four-seed refits).
Prior work and privacy setting (W1-W2, Q1)
We would like to express our sincere gratitude for your careful review and valuable comments and suggestions. Our point-by-point responses are given below.
Weakness 1 / Question 1.
Response: We agree that zero-shot pseudo-labeling and teacher-to-student transfer are established ideas. We have added Gera et al. (2022) to Section 2.2 and do not claim that this general pipeline is new. The key differences are:
Our contribution is therefore the systematic tabular privacy study and this specific evidence-based transfer strategy, rather than pseudo-labeling itself.
Revised manuscript: Section 2.2; Section 1 (contribution positioning).
Weakness 2.
Response: We model a publisher who deliberately removes private columns before sharing a dataset. The attacker observes the retained fields but has no ground-truth examples of the withheld target. Complete ACS/NHIS files let us simulate this disclosure policy and evaluate recovery; their public availability is not a claim that the labels are inaccessible outside the experiment. When representative labeled data are available, supervised inference is the appropriate stronger-access reference.
Yes: population-based inference can compromise the intended confidentiality of a private attribute. Hiding a value does not protect it if the remaining features and general population knowledge still permit its recovery. Our study is a specific instance of the scenario you describe: attributes intentionally withheld by the publisher are inferred from disclosed features and pretrained knowledge. Such disclosure need not involve memorization of an individual's training data. We measure that attribute-recovery risk. This differs from attributing a formal, participation-specific privacy loss to publishing a particular individual; the external-cohort controls in W3/Q4 examine whether the statistical benefit requires those same released individuals.
Revised manuscript: Sections 3.1 and 6.