- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates privacy risks in tabular foundation models (TFMs) by adapting model-stealing techniques to models conditioned on private in-context datasets. The proposed approach allows an attacker with black-box query access to construct surrogate models without requiring auxiliary data from the victim's distribution. Specifically, the attacker generates random queries based on the public data schema and trains surrogate models to reproduce the victim's predictions. The authors then propose membership inference (MIA), attribute inference (AIA), and property inference (PIA) attacks that leverage the stolen surrogates to extract information about the victim's private in-context dataset. Experiments across several tabular classification and regression datasets demonstrate that model stealing poses a privacy risk to TFMs and can improve the effectiveness of privacy attacks compared to the considered non-stealing baselines.
Critical strengths
- The paper addresses an interesting and relevant privacy problem and is the first to investigate model-stealing attacks against tabular foundation models (TFMs). The idea of exploiting the victim's predictive capabilities to generate pseudo-labeled data for training surrogate models is clever, as it removes the restrictive assumption that the attacker has access to an auxiliary dataset drawn from the victim's data distribution.
Critical weaknesses
-
The paper lacks sufficient algorithmic detail, making several components of the proposed methodology difficult to understand and reproduce. None of the attacks is presented through a sufficiently precise algorithmic description, either in the main text or in the appendix. For instance, the model-stealing procedure is described as minimizing the KL divergence between the victim's and surrogate's predictions, with an additional discussion of optimizing the surrogate's context in the embedding space. However, the exact optimization procedure and its implementation are not clearly specified. Similar issues arise in the descriptions of the privacy attacks and their additional components, such as the neighborhood-based refinement procedure used for MIA. These algorithmic aspects should be formalized more carefully to ensure reproducibility.
-
The neighborhood-based refinement procedure used in MIA requires more than 32,000 additional queries to the victim model for each target record, while providing only a marginal improvement in attack performance. This raises concerns about the practicality and scalability of the attack, particularly when targeting a large number of records or interacting with a rate-limited API. Similarly, the model-stealing stage requires a large number of queries to achieve high fidelity.
-
The proposed attacks are largely motivated by intuition, without sufficient theoretical or empirical evidence supporting the assumptions underlying their effectiveness. For MIA, the surrogate models are trained to reproduce the predictions of a victim that may already contain the target record in its context. Consequently, information about the target record may already be encoded in the surrogate, even under the non-membership hypothesis, raising questions about the validity of the constructed IN and OUT distributions. For AIA, it is unclear why the victim should systematically exhibit higher relative confidence than the surrogate on the correct attribute completion, especially given that the surrogate is explicitly trained to approximate the victim's predictive distribution. A similar concern applies to PIA, where the attack assumes that predictions for which the victim is more confident than the surrogates are more informative about the private context. The authors do not sufficiently justify this assumption or the choice of retaining only the top 10% of ranked samples. Additional theoretical motivation or empirical analyses validating these assumptions would considerably strengthen the proposed methodology.
Questions for authors
- The MIA constructs in and out distributions based on the presence or absence of the target record in the surrogate's context. However, if the target is a member, the surrogates may have already learned information about it during model stealing. How could this affect the reliability of the attack?
- Could the authors clarify how the decision threshold is selected in the MIA? In particular, how can the attacker calibrate this threshold without access to known member and non-member records, as assumed by the threat model?
- Could the authors provide further intuition or empirical evidence explaining why the victim should exhibit higher relative confidence than the surrogates on the correct attribute completion?
- Could the authors provide a clearer explanation of the rationale behind the rank score and the selection of the top 10% of probes in the property inference attack?
- Could the authors further explain the results in Figure 4? Given that the surrogates are trained to approximate the victim's behavior, why does an architectural mismatch sometimes improve attack performance? For example, PFN surrogates consistently achieve slightly higher AIA accuracy, while MIA against PFN victims performs better with mismatched surrogates. PIA also occasionally benefits from architectural mismatch. What could explain these differences?
- Could the authors provide more details on the types and cardinalities of the attributes targeted by AIA, including the treatment of continuous attributes? This is important for interpreting reconstruction accuracy, since an accuracy of 30% could be below the random-guessing baseline for a binary attribute.
4 comments
Q2. How is the MIA threshold set without known members?
The attack needs no labeled records: the adversary scores the unlabeled candidate records it wants to test and flags the top of them as members. We use membership labels only to evaluate this rule: with , it reaches 28.3% TPR at 3.2% FPR, against 17.5% TPR at 3.4% FPR for the non-stealing baseline. TPR@5%FPR and AUC in the paper summarize the attack over all thresholds, the standard way to report membership inference (Carlini et al., 2022).
Q5. Why does a mismatched surrogate sometimes do better (Fig. 4)?
We reran Fig. 4 with three seeds, a larger membership pool and six backbones. With these changes, a mismatched surrogate no longer helps or hurts the attack in a consistent way. It beats the matched surrogate in 17 of the 30 pairs, which is close to chance. We hypothesize that this is because a surrogate only has to reproduce the victim's predictions on the transfer set, and several backbones can do that equally well. The amount of leakage depends mainly on the victim, since some backbones react much more strongly to a single context record than others. Table 13 (App. C.7) shows that, for a fixed victim, the membership signal varies by at most 0.17 logits across surrogates, whereas across victims it ranges from 0.64 to 2.57 logits. TPR@5%FPR behaves the same way. It varies by at most 14.7 points across surrogates but ranges from 10.7% to 77.4% across victims.
Q6. Which attributes does AIA target, and what is chance accuracy?
To address the reviewer’s question, we added a random-guessing baseline (Rand) to Table 2(a). The unknown attributes are drawn at random from all feature columns, and the label is always part of the known anchor. Across the 45 datasets, the attacked attributes are:
| Attribute type | Share | Candidates | Random guess |
|---|---|---|---|
| Binary | 10.5% | 2 categories | 1/2 |
| Categorical | 23.2% | 3 to 7,518 categories (median 5) | 1/k |
| Numeric | 66.3% | 16 bins over the declared range | 1/16 |
At , our attack recovers 67.7% of the unknown attributes, against 15.4% for Rand. On binary attributes alone, it reaches 89.5%, against 50% for random guessing.
If our answers have addressed the reviewer’s questions, we would greatly appreciate if they could update their score.
Follow-Up
As the rebuttal period is coming to an end, we would like to thank the Reviewer once again for their constructive feedback. To address their comments, we have additionally:
- Added pseudocode for model stealing and all three privacy attacks (App. B).
- Made per-record refinement optional, so that our attack requires only 7.4 victim queries per candidate, including stealing, i.e., 35x fewer than the non-stealing baseline, and still attains a higher TPR@5%FPR (32.7% vs. 23.2%) (App. C.2, Table 7).
- Validated the assumption underlying each attack: the surrogates reproduce only 15% of the victim's membership signal, the victim's confidence lead is larger at the true AIA completion in every run, and the probes selected by RANK lie closer to the private context (App. C.7, Table 12); the 10% threshold is not critical, as every kept fraction from 10% to 90% improves on the baseline (App. C.5).
- Extended the backbone-mismatch study to three seeds and six backbones, where a mismatched surrogate outperforms the matched one in 17 of 30 pairs, i.e., at chance level (App. C.7, Table 13).
- Added a random-guessing baseline for AIA, which our attack far exceeds (67.7% vs. 15.4%; 89.5% vs. 50% on binary attributes) (Table 2(a)).
- Clarified that MIA requires no labeled records: flagging the top 5% of candidates as members yields 28.3% TPR at 3.2% FPR, compared with 17.5% at 3.4% for the baseline.
We believe these additions improve the reproducibility of our attacks and provide empirical support for each attack's assumptions. We would appreciate your reconsideration of the rating, including an increase if the new experiments address your concerns.
I thank the authors for their detailed rebuttal and the substantial improvements to the manuscript.
The added pseudocode improves reproducibility. The additional empirical analyses provide enough evidence. The experiments on architectural mismatch and the random-guessing baseline for AIA also satisfactorily address my questions.
I still have a minor reservation regarding MIA calibration, particularly whether the constructed IN and OUT distributions accurately reflect the membership hypotheses. Nevertheless, the additional empirical evidence helps alleviate this concern.
Overall, I appreciate the authors' efforts and consider the rebuttal substantial and convincing on most points. The additional analyses and improvements have strengthened my confidence in the proposed approach. I am therefore inclined to increase my score.
We thank the reviewer for recognizing that our paper "is the first to investigate model-stealing attacks against tabular foundation models" and for finding our pseudo-labeling idea "clever", as it removes the need for an auxiliary dataset from the victim's distribution. We address their comments and questions below.
We added pseudocode for all components in App. B.
To address the reviewer’s comment, we made the unrefined attack our default. After stealing, it needs one victim query per target, and refinement is now an option (App. C.2). We also ran the attack at smaller stealing budgets B (Fig. 5) and counted all victim queries, sharing the stealing cost over the 10,256 candidates of a membership pool (Table 7):
Without refinement, our default attack uses 35 times fewer queries per candidate than the non-stealing baseline and still reaches a higher TPR@5%FPR. Stealing also works with fewer queries: fidelity is already .825 at B=210 and .934 at B=214 (Fig. 2(b)), and at these budgets all three privacy attacks still match or outperform their non-stealing baselines (Fig. 5). Our default budget fits in 128 requests to SAP's RPT API, below its limit of 1,000 requests per hour (SAP).
To address the reviewer’s comment, we tested the assumption behind each attack and added the results to App. C.7 (Table 12). All three hold: the surrogates absorb only about 15% of the membership signal, the victim's lead over the surrogates is larger at the true completion than at the wrong ones in every run, and the probes that RANK keeps lie closer to the private context. We give the details of each attack below.
We agree that the surrogates pick up some information about the private context during stealing, as is known for distillation (Jagielski et al., 2023). In theory, this weakens our attack, since the OUT surrogates then carry a slight membership signal themselves. In practice, a record in the victim’s context has a much stronger effect. The victim is 1.27 logits more confident on members than on non-members, the surrogates only 0.19, so they reproduce about 15% of the membership signal (App. C.7). Our attack therefore still reaches 32.7% TPR@5%FPR, against 23.2% for the non-stealing baseline.
The surrogates reproduce the victim’s overall function, but they are fitted on queries far from the real records (App. C.6), so they miss the effect of an individual context record. In every run, the gap between the victim’s and the surrogates’ confidence is larger at the true completion than at the wrong ones. As a result, our attack ranks the true completion first for 68.4% of the targets, against 14.7% for a random guess.
Most schema-sampled probes lie far from the private context, where the victim behaves like one more surrogate. RANK keeps the probes where the victim is more confident than most surrogates, and these lie closer to a context record in the victim’s representation (Table 12). The 10% is not critical: every kept fraction from 90% down to 10% improves on the non-stealing baseline (App. C.5, Table 8).