SafeRoute: Retention-Constrained Local Evidence Control for Mixed-Shot Recognition

Vision-language adaptation under heterogeneous support can improve average accuracy while overturning correct predictions for classes with no labeled examples. We formulate this mixed-shot setting as constrained inference over a fixed vocabulary and introduce SAFEROUTE, a score-level interface that separates the permission to change a prediction from the strength of adaptation. We prove an exact observable characterization of universal pointwise retention: without the query label, every unsupported frozen winner must be preserved, while supported frozen winners define the maximal open route on which top-1 changes are admissible. Inside this route, local evidence control (LEC) adjusts a support-selected interpolation scalar using endpoint margins, winner disagreement, residual direction, and local support contrast. LEC contains the matched scalar controller as a special case and is conditionally invariant to noncompeting vocabulary additions. Across seven datasets, three vision-language encoders, four support profiles, and 420 TaskRes episodes, SAFEROUTE-LEC improves class-macro accuracy over the budget-matched scalar by 1.278 percentage points under mixed support and by 1.211 to 1.402 points across all profiles. It also improves the same-loss constant by 1.048 points under mixed support, ranks first in every encoder-profile mean and every mixed-support dataset mean, retains all 173,043 protected decisions, and produces no change in 3,780 specified dummy-class tests. These results establish a modular retention interface and show that query-local evidence provides substantial utility beyond strong support-only calibration without modifying parent training.