Failure-Aware Test-Time Adaptation

Test-time adaptation (TTA) mitigates distribution shifts by adapting pre-trained models to unlabeled target domains during deployment. While existing methods successfully boost target-domain accuracy via entropy minimization or self-training, they optimize for predictive performance while largely ignoring the reliability of the model's confidence estimates. Consequently, under severe domain shifts, these adaptation mechanisms break down the capacity for failure detection, causing models to become highly overconfident in their misclassifications. To overcome this limitation, we introduce Masked Augmentation for Reliable Self-adaptation (MARS), a plug-and-play framework for failure-aware TTA designed to simultaneously maximize adaptation accuracy and failure detection reliability. MARS leverages masked augmentations of input samples to serve two complementary objectives: (1) generating masking-ratio-conditioned soft labels to calibrate confidence rankings, and (2) measuring prediction consistency between original and masked samples to yield a robust uncertainty signal. Crucially, this dual-mechanism integrates seamlessly with existing TTA baselines without architectural modifications, supported by rigorous theoretical analysis. Across standard, continual, and mixed-domain setups on diverse benchmarks, MARS substantially improves failure detection metrics while matching or exceeding the target-domain accuracy of state-of-the-art methods. Our code will be made publicly available.