Align, Saturate, Release: The Finite Useful Lifetime of Foundation Guidance in Sparse Feature Matching
Abstract
Sparse feature matching establishes point-level correspondences between images and underpins structure-from-motion, visual localization, and relative pose estimation, where a matcher must gain target-domain accuracy without inflating the pipeline that finally runs. Aligning the matcher's representation to a pretrained foundation encoder is a common remedy, but existing designs commit to it permanently: the foundation branch either stays in the deployed pipeline or supervises training throughout. Since any competent matcher already aggregates image context internally, the open question is not whether such supervision helps, but how long it keeps steering a representation that is absorbing it. We recast foundation guidance as a temporary optimization constraint with a finite useful lifetime and make that lifetime measurable. Foundation-to-Geometry Hand-off (FGH) schedules the constraint explicitly: during source preconditioning it aligns the assignment-producing hidden representation with a frozen foundation reference on the matcher's own sparse support, then releases the entire reference branch before target-domain geometric specialization. A shared-parent, two-seed 2x2 intervention varying only guidance availability reveals a history-conditioned diminishing marginal utility: guidance helps a representation with no earlier guidance history, whereas continuation after early alignment adds almost no observed gain. Diagnostics trace this to saturation rather than stage identity—the alignment residual collapses from near one to below 0.005 and the effective guidance-to-task gradient ratio decays from 2.4% to below 0.04%—while a permutation probe attributes the benefit to representation shaping rather than co-location alone. Instantiated with frozen DINOv2 on a SuperPoint–LightGlue matcher, FGH improves descriptively over its unguided baseline on homography and relative-pose benchmarks while deploying an architecturally identical matcher.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.