PROBE: Protected Observation-Subspace Energy for Fine-Grained Open-Set Recognition on Frozen Classifiers

Fine-grained open-set recognition must reject unseen categories that lie close to known classes while tolerating observation-induced variation in the features. In existing post-hoc scores, feature magnitude varies with prediction confidence and image conditions, and within-class covariance alone does not distinguish observation-induced variation from directions that separate known-class means. We introduce Protected Observation-Subspace Energy (PROBE), which combines class-specific isotropic energies on unit-normalized features with an additive covariance adjustment constructed from paired feature displacements under mild image corruptions, in which a between-class penalty discourages additional tolerance along directions separating known-class means. Among post-hoc and end-to-end baselines, PROBE achieves the highest mean Hard AUROC and Hard open-set classification rate (OSCR) on three Semantic Shift Benchmark (SSB) datasets and the highest mean AUROC and OSCR on three crop-level unmanned aerial vehicle (UAV) protocols constructed here. With the covariance spectrum held fixed, directions fitted with the between-class penalty yield higher mean AUROC than directions fitted without it on every dataset. Fitting uses only known-class samples of a frozen classifier and has a closed-form solution, and inference scores a single test image without corrupting it, retraining the classifier, or changing its label predictions.