Exploiting Mode Connectivity for Out-of-Distribution Detection via Low-Dimensional Subspace Modeling

Detecting Out-of-Distribution (OoD) from In-Distribution (InD) samples is fundamental to trustworthy deep learning. To characterize InD-OoD disparities, existing detection methods largely focus on outputs from a single local optimum, or mode, which provides only a limited and potentially noisy view of the underlying InD-OoD data-model relationship. To move beyond this local perspective, we study OoD detection through the lens of mode connectivity, where two independently trained modes can be connected by a low-loss path on InD data, suggesting that this path potentially encodes richer InD and OoD information than any isolated mode alone. We first reveal that models sampled along a low-loss connecting path exhibit nearly consistent InD accuracy but markedly different OoD detection performance, implying that OoD discriminability is not uniformly preserved along the low-loss path. Based on this finding, we propose a low-dimensional subspace modeling approach over the connecting path. Our analysis shows that low-variance principal directions are closely associated with unstable OoD behavior, whereas dominant directions capture stable structure from the path. By removing the former and aggregating the latter, we construct a principal mode, which is compatible with existing detection methods, delivers stronger OoD detection than even mode ensembling, and maintains competitive InD recognition. Our study highlights low-dimensional parameter subspace as a new and effective perspective for understanding and improving OoD detection.