Beyond Known-Class Prompts: Sparse Factor Objectness for Open-World Object Detection

Open-world object detection (OWOD) requires recognizing known objects while discovering unknown ones. Open-vocabulary detectors (OVDs) provide strong visual representations but rely on class-name prompts for detection. Without unknown-class prompts, unknown objects may be missed even when their region representations retain informative object cues. We propose a Sparse-AutoEncoder-based Open-Vocabulary Open-World framework for OWOD (SAE-OVOW) to recover class-agnostic objectness directly from a frozen OVD without unknown-class annotations. SAE-OVOW uses a sparse autoencoder to decompose known-aligned region embeddings into sparse latent factors. It then selects factors that are reliably activated, exhibit similar distributions across known-aligned and unmatched regions, and have diverse decoder directions. The selected activations are aggregated into sparse factor objectness and combined with known-class uncertainty for unknown detection. Experiments on M-OWODB and S-OWODB show that SAE-OVOW consistently improves U-Recall while maintaining competitive known-class performance. On S-OWODB Task 2, it improves U-Recall by 12.7 points and overall mAP by 5.3 points over the best competing results. Latent removal-and-recovery experiments verify the functional relevance of the selected factors, while Task-1 post-hoc audits on M-OWODB and S-OWODB identify coherent visual patterns in 119 of 200 factors.