Exemplar-free Cross-Category Incremental Keypoint Detection: When New Categories Bring Heterogeneous Schemas

Object categories often have heterogeneous keypoint schemas differing in keypoint number, order, and semantics, yet keypoint detectors are typically trained for fixed categories with predefined schemas, making continual expansion to new categories challenging. Under the standard top-down setting, we study exemplar-free Cross-Category Incremental Keypoint Detection (CCIKD). At each incremental stage, the model learns only from new categories while preserving previous keypoint detection performance. This setting exposes a fundamental output-space challenge: assigning a separate identity to every category-keypoint causes unbounded output growth, while shared heads reuse identities through fixed local indices, causing Output-Identity Collision (OIC), where incompatible keypoints are forced to share parameters. During continual updates, such collisions cause semantic ambiguity, leading to greater forgetting of previously learned knowledge. CCIKD therefore requires preserving old knowledge while organizing an evolving output space under heterogeneous schemas. To address these challenges, we propose POISE, prototype-guided Persistent Output Identity Space Expansion, which builds visual–geometric prototypes and an append-only identity bank. POISE selectively reuses compatible identities, transfers related output parameters for initialization, and expands the bank for unmatched keypoints, while decoupling output-identity organization from preservation objectives. Experiments on IKDD and MP100 across diverse orders, protocols, and continual-learning strategies demonstrate improved performance and reduced forgetting, with increasingly larger gains on longer and more heterogeneous streams. In particular, with EWC on MP100-8stage, POISE boosts FinalPCK from 45.51 to 70.66.