ESP-YOLO: Stabilizing Incremental Object Detection via Eigenvalue Scaled Projection Regularization and Confidence-Aware Distillation

Incremental Object Detection (IOD) requires sequentially learning to localize and classify novel object categories while preserving the stability of historical representations to mitigate catastrophic forgetting. While single-stage frameworks like YOLO are ideal baselines for real-world IOD deployment due to their efficiency, they remain under-explored. This gap primarily stems from YOLO's end-to-end prediction mechanism, where backpropagated gradients from new tasks cause drastic perturbations to backbone parameters. Unlike two-stage models that inherently isolate such impacts via class-agnostic region proposal networks, YOLO suffers from significantly more severe intermediate feature drift and exacerbated class confusion. To overcome these architectural bottlenecks, we propose a framework featuring Eigenvalue Scaled Projection Regularization (ESPReg) and Confidence-Aware Distillation (CAD). Specifically, ESPReg utilizes the eigenvalues of feature covariance to quantify the importance of spatial directions, guiding weight updates to minimize interference with old tasks. We theoretically prove that ESPReg effectively maintains intermediate feature stability. Furthermore, CAD leverages high-confidence regions to guide the distillation process, isolating background interference to resolve severe class confusion. Extensive experiments across single-stage and multi-stage settings demonstrate that our approach surpasses state-of-the-art IOD methods by 1.4% and 7.3% on the PASCAL VOC and MS COCO benchmarks, respectively.