- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies membership inference attacks (MIA) in sequential Test-Time Adaptation (TTA). The authors point out that distribution-level adaptation can hide the membership signal of individual samples, making existing MIA methods less effective in this setting.
To address this problem, the authors propose Manifold Calibration Attack (MCA). The method first removes shared adaptation effects using local reference samples, identifies candidate-relevant steps along the adaptation trajectory, and then computes membership scores using Jacobian-based attribution.
Critical strengths
- Interesting problem setting
I think the privacy problem considered in this paper is interesting. Unlike standard inference, TTA continuously updates the model using test samples, so these samples may leave some information in the model states. In particular, the authors point out that distribution-level adaptation and sample-specific effects are mixed together during TTA. This makes it difficult to directly apply existing MIA methods. I think this is a reasonable motivation for designing an attack specifically for TTA.
- Experiments across different TTA settings
The experiments cover several datasets including CIFAR-10-C, CIFAR-100-C, and ImageNet-C, with different model architectures and TTA methods such as BN Adapt, TENT, SAR, and CoTTA. The authors also provide ablation studies and additional experiments on reference mismatch, adaptation hyperparameters, and differential privacy. These analyses are useful for understanding the proposed method. Some results on ImageNet-scale settings also show noticeable
Critical weaknesses
- Limited novelty compared to existing approaches
The main method combines local manifold calibration, temporal localization, and Jacobian-based attribution. I think this combination is reasonable for the TTA setting, but I am not fully convinced about its methodological novelty.
Previous trajectory-based MIA studies have already considered membership signals from model updates over time. Gradient-based attribution has also been studied in previous works. In this sense, the proposed method seems to be more of a combination and adaptation of existing techniques, rather than a fundamentally new attack principle.
The authors provide ablation results showing that each component contributes to the performance. However, I think it is still unclear whether the full MCA pipeline is necessary compared to simpler approaches. For example, it would be useful to compare against a simpler temporal or gradient-based attack with local calibration, under the same access to model states and reference samples.
- Relatively weak attack performance and practical applicability
According to Table 1, MCA achieves an average ROC-AUC of 0.561, compared to 0.544 for LiRA, which is the strongest baseline on average. Although MCA gives the best average result, the absolute AUC is still relatively close to random guessing (0.5).
Also, in several settings, MCA achieves AUC values close to 0.5, and sometimes performs worse than the existing baselines. For TPR@5%FPR, MCA improves the average result from 8.263% to 9.575%, but the overall detection performance is still limited.
I agree that some ImageNet-scale settings show more meaningful improvements. However, the performance varies considerably across different TTA methods. Considering the complexity of the proposed attack, I think the average improvement is rather small. It would be helpful to explain how consistently MCA improves over the strongest baselines, and how these results should be interpreted in terms of actual membership leakage.
- Another concern is the practical attack setting.
MCA assumes that the attacker can access intermediate model states throughout the adaptation process. It also requires reference calibration and Jacobian computation for candidate samples. While this assumption can be reasonable in some collaborative TTA scenarios, I am not sure how realistic it is in more general deployment environments. The computational cost of the attack is also not sufficiently discussed.
Questions for authors
N/A