TrackLUT: Semantic Color Tracking for Adaptive Video Color Transfer

Reference-based video color transfer remains challenging in dynamic scenes, where color transformations must adapt to evolving content and illumination while avoiding temporal flicker. Existing keyframe-based approaches provide limited adaptability, whereas independent frame-wise predictions often sacrifice temporal consistency. To address these limitations, we propose TrackLUT, a frame-adaptive framework that predicts a 3D lookup table (LUT) for each frame and exploits semantic correspondence for both spatial color transfer and temporal propagation. Specifically, semantic features establish reliable correspondences between target content and reference regions, while decoupled color features encode transferable appearance cues. This design enables selective color retrieval from semantically relevant regions, reducing erroneous mappings caused by spatial and appearance discrepancies. To maintain temporal coherence, we further align the historical color state with each incoming frame through semantic correspondence, allowing relevant appearance information to propagate as the scene evolves. The retrieved reference features and aligned temporal state are jointly decoded into a frame-specific LUT. Experiments demonstrate that TrackLUT adapts effectively to substantial content and illumination changes while preserving reference-style fidelity, structural consistency, and temporal stability, enabling efficient real-time video color transfer.