SHIFT: Receiver-Conditioned Prediction-Space Transfer for Low-Data RGB-to-Thermal Adaptation

Large-scale data and pretraining give RGB models rich semantic and geometric knowledge, whereas thermal perception remains limited by scarce and expensive task annotations. We study low-data RGB-to-thermal adaptation with a pretrained RGB model, abundant unlabeled RGB-thermal pairs, and only a small thermal support set, while keeping deployment thermal-only. Existing domain adaptation and cross-modal distillation methods transfer features, teacher predictions, or pseudo-labels, but they do not identify which transferred changes are new to the current thermal model and executable by its permitted update mechanism. We introduce SHIFT, a receiver-conditioned prediction-space transfer framework that selects RGB-induced changes that are novel relative to the native thermal trajectory and executable by the designated thermal receiver. Using predictive Fisher geometry, SHIFT removes task-invariant and trajectory-explained components, projects the remaining change onto the receiver's locally executable response space, and freezes a low-dimensional family of executable prediction directions. Thermal labels then estimate only their signed combination. On MSRS semantic segmentation, M3FD object detection, and MS2 monocular and stereo depth estimation, full SHIFT improves over four adaptation strategies initialized from the native thermal baseline under a 10% label budget. Shared-resource comparisons and 5/20-shot evaluations further support the value of receiver-conditioned filtering and low-dimensional calibration.