Exposing Membership Leakage in Test-Time Adaptation via Manifold Calibration

Test-time adaptation (TTA) improves model robustness under distribution shifts by updating pretrained models on unlabeled test data during inference. However, test samples may leave detectable traces during model adaptation, raising a privacy concern that remains largely unexplored. Existing sequential membership inference attacks (MIAs) consider training or model-update trajectories, but do not account for the distribution-driven adaptation dynamics in TTA that can obscure candidate-specific membership traces. To address this gap, we propose the Manifold Calibration Attack (MCA), a label-free, trajectory-based attack designed for sequential TTA that separates candidate-specific update effects from shared adaptation dynamics. Specifically, MCA calibrates each candidate’s step-wise response against a local manifold neighborhood to remove shared adaptation motion, localizes candidate-relevant changes along the adaptation trajectory, and uses nuisance-aware Jacobian attribution to measure how strongly the selected updates align with the candidate’s functional subspace. Our analysis reveals that membership evidence is often stronger in early adaptation across different TTA methods. Extensive experiments across 13 settings spanning multiple datasets, model architectures, corruption streams, and representative TTA methods show that MCA achieves the best macro-average ROC-AUC and low-FPR detection performance among the evaluated attacks.