The Space Between Views: A Simplex Approach to Test-Time Adaptation

We hypothesize that the region spanned by augmented views contains useful structure for test-time adaptation (TTA). Can TTA exploit it? Existing methods typically treat augmented views as a finite set, aggregating, filtering, or regularizing their predictions. We propose SimplexTTA, which models the convex hull of augmented test features and minimizes the expected adaptation loss over the resulting simplex. To make this objective practical, we derive a second-order approximation around the simplex barycenter. This, in turn, yields a curvature-weighted correction that captures the interaction between feature spread and the local curvature of the loss. The resulting objective can directly replace the standard adaptation loss in existing gradient-based TTA methods with minimal modification. Across multiple image classification benchmarks, including corruption benchmarks, SimplexTTA consistently improves state-of-the-art TTA methods, providing empirical support for our hypothesis.