- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work addresses reference-based video color transfer in dynamic scenes. Existing keyframe-based methods struggle with changing illumination and scene content, and independent per-frame predictions tend to cause temporal flicker. The authors propose TrackLUT, which predicts a 3D LUT for each frame. Using semantic correspondence for color mapping and temporal state propagation, the model separates semantic and color features to sample colors from semantically consistent reference regions and avoid wrong mappings. Reference features and aligned historical states are decoded into frame-adaptive LUTs. The method is claimed to cope with content and lighting changes while preserving style, structure, temporal stability and real-time performance.
Critical strengths
- The technical idea of frame-adaptive 3D LUT prediction guided by semantic correspondence is well aligned with the task requirement. It attempts to combine the computational efficiency of LUT-based color mapping with adaptive capacity for dynamic lighting, which is a reasonable direction for reference-based video color transfer.
- The design decouples semantic matching and color appearance representation, which is an intuitive strategy to mitigate misalignment-induced wrong color transfer.
Critical weaknesses
- The evaluation is inadequate to support the main claims. While quantitative results are reported, visual evidence is insufficient to confirm that semantic propagation mitigates the inherent flicker and color drift of per-frame LUTs. There is no targeted ablation or comparison to justify the advantages of the frame-adaptive LUT over prior LUT-based approaches. Qualitative results are only shown for some baselines, and tests are limited to simple videos without complex sequences or supplementary demos. This makes it hard to reliably judge the practical effectiveness of TrackLUT.
- The presentation and motivation of the paper are suboptimal. The citation format contains errors, and the writing over-emphasizes technical details at the cost of intuitive motivation. The high-level rationale is not easy to follow, which hinders readers from quickly grasping the core insight and contributions.
Questions for authors
- From the visual results you have included, could you elaborate on how your temporal semantic propagation alleviates flicker and color drift of per-frame LUTs under complex dynamic scenes? What key observations support this claim?
- In your current experiments, what evidence or analysis can demonstrate the advantages of your frame-adaptive 3D LUT design compared with other LUT-based color transfer methods?
We sincerely thank the reviewer for the insightful and constructive comments, which have helped us improve the clarity, motivation, and experimental completeness of our work. We have revised the manuscript and conducted additional analyses to address the concerns regarding temporal stability, qualitative comparisons, and frame-wise adaptability. Our point-by-point responses are provided below.
W1:
Thank you for this valuable comment.
(1) Temporal stability. Table 3 provides targeted ablations of HAF and Temporal LUT Gate, showing reductions of 12.0% and 7.5% in wΔE-1 and wΔE-5, respectively. We further provide a qualitative comparison with EMA in Supplementary Fig. A.1, illustrating how semantic-guided history propagation handles newly appearing foreground content.
(2) Qualitative comparisons. All baselines are quantitatively evaluated in Table 1. Due to space constraints, we select representative methods with competitive performance for visualization, including NLUT (best Content Sim.), CAP-VSTNet (best Style Sim.), and ColorFM (competitive overall performance), covering different methodological paradigms. Figure 3 further compares their results under different illumination conditions.
(3) Real-world evaluation. Our test set contains 55 real-world video-reference pairs. We additionally provide temporal lightness response analysis on four representative videos in Supplementary Fig. B.1, demonstrating TrackLUT's responsiveness to changing video appearance.
W2:
We thank the reviewer for this constructive comment. We have revised the manuscript to improve the clarity of its motivation and overall presentation. First, we have corrected several citation formatting errors. Second, we have refined the Introduction to provide a more intuitive motivation by emphasizing the fundamental trade-off between frame-wise adaptability and temporal consistency.
Q1:
Thank you for this insightful comment.
TrackLUT mitigates temporal flickering and color drift through semantic-guided historical feature propagation. Specifically, HAF establishes patch-level semantic correspondences between consecutive frames using DINOv2 features. For each current-frame patch, it retrieves the corresponding historical color state and adaptively determines its contribution according to semantic similarity. Regions with reliable correspondences retain more historical color information, reducing inconsistent frame-wise color retrieval, while newly appearing or semantically mismatched regions rely more on current-frame features to avoid propagating irrelevant historical information.
Several visual observations support this design. First, in Supplementary Fig. A.1, when a foreground subject enters the scene, the HAF weight maps exhibit spatially selective responses, illustrating content-aware historical information propagation. Compared with EMA, TrackLUT better accommodates the newly appearing content while maintaining a coherent appearance. Second, Fig. 3 shows that under substantial indoor-to-outdoor illumination changes, fixed keyframe LUT methods exhibit under- or over-stylization, whereas TrackLUT adapts its color mapping while maintaining style coherence. Finally, Supplementary Fig. B.1 shows that the transferred videos preserve the major temporal lightness trends of the input sequences.
Q2:
Thank you for this insightful comment.
First, Table 1 shows that TrackLUT achieves the lowest distance to the ideal point (0.282), outperforming ColorFM (0.295).
Furthermore, Figs. 3 and 5 demonstrate that our method achieves stronger semantic matching capabilities compared with other methods.
Finally, Fig. 3 provides direct visual evidence under indoor-to-outdoor illumination changes. When the appearance of the current frame deviates from that of the keyframe, fixed keyframe LUTs tend to result in under- or over-stylization. In contrast, TrackLUT dynamically adjusts the predicted frame-specific LUTs to accommodate evolving content and illumination while maintaining overall style coherence. Supplementary Fig. A.1 further illustrates the advantages of adaptive LUT prediction over the EMA baseline.