CEFFYL: Modelling Deformable Object Shape for Improved Point Tracking

Point tracking algorithms have consistently struggled with long-term and complex motion of objects. We introduce CEFFYL, a shape modelling method that utilises Canonical shape Embeddings and Feature Filtering to Yield Long-term point tracks. Specifically, we model the amodal representation of each pixel as a set of points on the rest pose of the object category. Once this mapping is established, correspondence and point tracking is estimated from this shared canonical space. We additionally learn a richer descriptor feature for local correspondences within the canonical space. We train CEFFYL on synthetic data of horses, particularly difficult objects to track due to their deformable nature and complex leg motion at speed. CEFFYL outperforms state-of-the-art point tracker AllTracker by $8.9$ and $16.7$ points on Horse10 and long Showground videos, respectively. We also outperform the single-frame DualPM shape model, the most recent work of this type, increasing performance by $7.9$ and $8.5$ points on the same metrics.