SCLTrack: Structured Correspondence Learning for Category-Level Garment Pose Tracking

Tracking category-level garment pose from partial point clouds requires correspondence to remain reliable despite large non-rigid deformation, self-contact, occlusion, and continuously changing observations. A key difficulty is that dense inter-frame features alone must simultaneously encode regional structure, instantaneous geometry, and persistent identity, although these cues follow different spatial and temporal behaviors. We present SCLTrack, a Structured Correspondence Learning framework that explicitly organizes these complementary cues before canonical correspondence estimation. SCLTrack introduces a semantic branch to capture region-level structural context, a geometry branch to preserve local metric relations, and a consistency branch that retrieves temporal evidence from an ordered canonical memory. Their point-wise representations are integrated with the base inter-frame feature through adaptive gated residual fusion. The enhanced representation is further refined in NOCS space and decoded through a canonical volumetric representation and implicit warp field, which also updates the canonical memory for subsequent frames. Extensive experiments demonstrate that SCLTrack significantly enhances the accuracy of garment pose tracking by up to 14.5% over GaPT-DAR, while reducing correspondence, surface, and NOCS errors by 2.1%, 6.5%, and 8.9%, respectively.