Active Test-Time Adaptation via Shift-Aware Querying and Class-Conditional Alignment

Representations trained on one distribution can encode both predictive information that remains stable across distributions and distribution-specific associations that become unreliable under distribution shift. Test-time adaptation should reduce reliance on these associations while preserving predictive information, but unlabeled target data provides limited supervision for this objective. We propose Active Contrastive Test-Time Adaptation (ACTTA), which combines selective label acquisition with class-conditional representation alignment. ACTTA queries diverse target samples whose representations deviate from class-conditional source statistics. A supervised contrastive objective encourages representation alignment to reduce distribution-specific associations, while cross-entropy maintains class discrimination. Under an idealized data-generating process, our analysis characterizes conditions under which class-conditional invariance excludes distribution-varying information while preserving label information. Experiments on four vision and language benchmarks spanning temporal, geographic, and synthetic shifts show that ACTTA achieves the highest performance among the evaluated methods. Further evaluations demonstrate improved generalization to unseen distributions and preservation of performance in the training distribution.