acceptodds
Under review as a conference paper at ICLR 2027

TCFlow: Temporally Consistent Flow for Single-Reference Video Color Transfer

Abstract

We study single-reference video color transfer: given a content video and one reference image, the goal is to generate a photorealistic and temporally stable video whose color palette matches the reference. This task is fundamentally constrained by temporal transport inconsistency, where minor frame-level distribution shifts caused by motion, occlusion, or lighting induce divergent color mappings and perceptible flickering. Existing approaches expose a stability-expressiveness trade-off: clip-level global transforms are temporally stable but limited in modeling complex reference-specific correspondences, whereas frame-wise methods are more expressive but often temporally incoherent. To address this limitation, we propose TCFlow, a unified meta-flow framework for robust and temporally consistent video color transfer. TCFlow predicts a clip-shared flow that transports the aggregated video-level content distribution toward a single reference palette. This clip-shared formulation inherently decouples global color structure from frame-specific variations, effectively suppressing color flickering. To ensure high-fidelity transfer, we introduce a color aggregator that synthesizes global color statistics across the video clip, enabling expressive and accurate color mapping without optical flow, dense motion supervision, or test-time optimization. Experiments show that TCFlow substantially suppresses color flickering and reduces frame-wise mapping variation. Beyond standard benchmarks, we validate TCFlow on a dataset of real-world media art exhibitions, where it consistently outperforms state-of-the-art methods in stability and photorealism. Further source code is provided in the supplementary material. Additional results, including qualitative comparisons and more video results, are available at https://tc-flow-iclr.vercel.app/

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video color transfer

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_bSaQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TCFlow, a framework for single-reference video color transfer that aims to achieve accurate color matching while maintaining temporal consistency. The key idea is to replace frame-wise color transformations with a clip-shared rectified flow, which maps the aggregated video color distribution to a common latent space and then transfers it to the reference color distribution. The method further introduces a color aggregator to predict the shared flow parameters efficiently without test-time optimization. Experiments on DL3DV-10K and a real-world media art dataset demonstrate improvements in color fidelity, structural preservation, and temporal stability compared with existing methods. Ablation studies further investigate the effectiveness of the clip-shared flow and color aggregation module.

Critical strengths

None

Critical weaknesses

None

Questions for authors
  1. Novelty: How does TCFlow differ fundamentally from ModFlows? Would simply applying a shared flow to temporally pooled video colors achieve similar results?

  2. Temporal consistency: The proposed flickering metrics rely on frame-wise color histogram differences, which may be affected by motion and occlusion. Could the authors provide motion-compensated evaluations to better validate temporal consistency?

  3. Color Aggregator: Table 2 shows that the aggregator improves color fidelity but worsens temporal consistency and Lipschitz scores. How do the authors justify this trade-off?

  4. Theoretical formulation: How can a single invertible flow map different frame-level color distributions to the same latent distribution? Is the alignment only guaranteed for the aggregated video distribution?

2 comments
AuthorsOpenReview ↗

Response to Reviewer bSaQ (1)

[Q1: Novelty]

TCFlow differs from ModFlows in two key aspects. First, while ModFlows estimates frame-wise color transport mappings, TCFlow formulates transport over a temporally pooled clip-level color distribution and applies a shared mapping across all frames. Second, TCFlow introduces a learnable Color Aggregator that integrates temporal color information to estimate video-level flow parameters, going beyond simple temporal pooling.

The contribution of the learned aggregation is demonstrated by the TCFlow w/o Color Aggregator ablation in Table 2. This baseline uses mean aggregation over the clip-level color distribution without a learned Color Aggregator. Compared with this baseline, the full TCFlow reduces SWD from 0.0599 to 0.0462 and MS-SWD from 1.6606 to 1.2355, while also improving reference-weighted track-color flicker ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}} from 0.9585 to 0.8475. These results show that mean aggregation over clip-level color distributions alone does not fully capture the benefits of TCFlow; learned temporal aggregation further improves reference color alignment and temporal consistency.

We have revised the Introduction to better motivate the need for balancing reference-color fidelity and temporal consistency (see Lines 047-079), and clarified the contribution of learned temporal aggregation in the ablation study.


[Q2: Motion-Compensated Temporal Consistency]

We supplement the histogram-based temporal metrics with a motion-aligned evaluation to assess color stability along corresponding scene points while accounting for occlusions. Specifically, we use CoTracker3 [1] to extract point trajectories and visibility masks from the original input videos, ensuring that all methods are evaluated using identical correspondences. We first define ΔEtrack\Delta E_{\mathrm{track}} to measure temporal changes in color-transfer residuals along valid tracks in CIELAB space. We further incorporate reference color fidelity through a relative SWD weighting, defining a composite metric, ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}}, that jointly reflects track-based color flicker and reference color distribution alignment. The results are summarized below:

MethodDL3DV-10K ↓\downarrowMedia Art ↓\downarrowREDS ↓\downarrowMOSE-v2 ↓\downarrow
CCPL1.82650.83132.19470.3445
CAP-VST1.16170.81541.53570.2939
VideoColorGrading1.09701.29802.62160.5137
ModFlows1.59590.85761.34940.2730
SA-LUT1.93321.52653.42570.7103
ColorFM1.08350.92101.10930.2401
PETAL1.50781.06571.25770.3509
TCFlow (Ours)0.84750.65860.92620.1855

TCFlow achieves the lowest ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}} across all four benchmarks: DL3DV-10K (0.8475), Media Art (0.6586), REDS (0.9262), and MOSE-v2 (0.1855). These results demonstrate that TCFlow consistently outperforms competing methods in both motion-aligned temporal stability and reference color alignment. We have added the motion-aligned evaluation, metric formulations, and implementation details to the revised paper (see Appendix A.4 and Tables 1, 7, 8 and 9).


Reference

[1] CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. ICCV 2025.

AuthorsOpenReview ↗

Response to Reviewer bSaQ (2)

[Q3: Color Aggregator]

The Color Aggregator improves reference color fidelity, although it introduces trade-offs in certain numerical metrics, including depth, edge and Lipschitz scores. To further investigate temporal stability, we introduce a motion-aligned, reference-weighted metric ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}}, which evaluates color consistency along corresponding scene points while accounting for occlusions and reference color alignment. Unlike histogram-based temporal metrics, this composite metric combines motion-aligned color stability with reference color fidelity. Notably, the full TCFlow improves this metric from 0.9585 to 0.8475 compared with the variant without the Color Aggregator (see Table 2).

We further examine the perceptual quality of TCFlow through a user study with 23 participants (see Appendix A.8). All participants evaluated the same 19 media art video sequences, comparing ModFlows, VideoColorGrading, ColorFM, and TCFlow based on overall perceptual quality, considering both reference color fidelity and temporal stability.

TCFlow received 54.5% of all preferences, outperforming ModFlows (21.1%), ColorFM (18.3%), and VideoColorGrading (6.2%). The overall preference differences were statistically significant (Friedman test: χ2(3)=25.46,p=1.23×10−5\chi^2(3)=25.46, p=1.23\times10^{-5}). These results suggest that the trade-offs observed in individual numerical metrics do not necessarily translate into lower perceived video quality. The motion-aligned evaluation and user study provide complementary evidence of TCFlow's improved performance under the proposed reference-weighted metric and higher perceptual preference.


[Q4: Theoretical formulation]

TCFlow shares a mapping across frames within each clip, while targeting the pooled clip distribution for alignment. A single invertible map cannot push distinct frame distributions to exactly the same latent distribution; this is not a requirement of our training formulation.

Specifically, Eq. (3) defines the balanced clip distribution as PV=1N∑nPInP_V=\frac{1}{N}\sum_n P_{I_n}. Writing F=ΨϕVclipF=\Psi_{\phi_V}^{\mathrm{clip}}, the revised Section 3.3 explicitly states F♯PV=1N∑n=1NF♯PIn≈η.F_{♯}P_V = \frac{1}{N}\sum_{n=1}^{N} F_{♯}P_{I_n} \approx \eta.

The individual transformed frame distributions may differ; their mixture is the distribution targeted toward η\eta. This formulation matches the existing teacher objective in Eq. (8), which samples colors from the pooled clip distribution, and its subsequent distillation in Eq. (9).

For idealized exact population-level alignment, if F♯PV=ηF_{♯}P_V=\eta and H♯PR=ηH_{♯}P_R=\eta, where H=ΨϕRRH=\Psi_{\phi_R}^{R} is invertible, then (H−1∘F)♯PV=PR(H^{-1}\circ F)_{♯}P_V=P_R.

This implication concerns the pooled distribution and does not establish alignment of every frame individually. In practice, these are approximate learning targets rather than unconditional guarantees.

We have also clarified in Section 3.1 that Eq. (1) expresses an empirically evaluated frame-wise color-fidelity criterion. These revisions make the scope of the formulation explicit while retaining the existing training and inference procedures.

Reviewer_4o5UOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TCFlow for single-reference video color transfer, using a clip-shared rectified flow to reduce temporal inconsistencies introduced by frame-wise color mappings. A temporal color aggregator predicts flow parameters distilled from per-video teacher flows, enabling inference without test-time optimization. Experiments on standard and curated benchmarks demonstrate improved temporal stability while maintaining competitive color fidelity and content preservation.

Critical strengths
  • The paper identifies a clear source of temporal instability in frame-wise color transfer and addresses it with a simple, well-motivated clip-shared mapping. This provides a coherent approach to improving temporal consistency without optical flow or test-time optimization.

  • Quantitative results across multiple benchmarks, together with the supplementary videos, support improved temporal stability while retaining competitive reference color fidelity. This demonstrates the practical value of the approach beyond improvements in individual frame quality.

Critical weaknesses
  1. Insufficient positioning and baseline coverage. The paper should better distinguish its contribution from recent reference-based color and photorealistic style transfer methods, particularly ColorFM [1], PETAL [2], and SA-LUT [3]. ColorFM is especially relevant because it also combines flow-based color transport with an optimization-to-learning framework and demonstrates video applications. Comparisons under clearly specified video protocols would better establish TCFlow’s claimed fidelity–stability advantage.

  2. Incomplete contribution isolation. Figure 4 and Table 2 provide useful ablations, but a clip-shared adaptation of ModFlows is not explicitly evaluated. This comparison would help distinguish gains from simply sharing a mapping from those of the proposed video-level estimation and aggregation.

References:

[1] ColorFM: An Optimization-to-Learning Framework for Color Transfer via Flow Matching. ECCV 2026. Paper

[2] Enlightening Photographic Style Transfer with a Self-Supervised Photographic Embedding (PETAL). ECCV 2026. Project

[3] SA-LUT: Spatial Adaptive 4D Look-Up Table for Photorealistic Style Transfer. ICCV 2025. Paper

Questions for authors
  1. Could the authors evaluate sensitivity to the number and temporal spacing of sampled frames while keeping the total color-sampling budget fixed? This would clarify how much temporal coverage is needed to estimate a reliable clip-level mapping.

  2. Could the authors supplement the histogram-based temporal metrics with an occlusion-aware, motion-aligned consistency evaluation? This would help establish whether the reported gains reflect stable colors on corresponding scene regions.

  3. How does TCFlow handle long videos processed in separate clips? An evaluation of color continuity across clip boundaries, particularly under lighting or scene changes, would clarify the practical limits of the clip-shared formulation.

I would consider raising my score if the authors adequately address the key weaknesses, particularly the baseline coverage and the isolation of the proposed contributions.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 4o5U (1)

[W1: Insufficient Positioning and Baseline Coverage]

To better clarify the methodological differences from the recent reference-based color transfer methods ColorFM [1], PETAL [2], and SA-LUT [3]. ColorFM formulates color transfer as pixel-distribution transport via flow matching, combining semantic-guided hierarchical color coupling with an optimization-to-learning framework. In contrast, TCFlow focuses on video-level temporal consistency by predicting a shared color transport mapping across an entire clip, with a learned Color Aggregator that integrates temporal color information to estimate shared flow parameters. PETAL learns a continuous photographic embedding through self-supervision to guide an adaptive neural LUT, enabling expressive photographic style transfer without test-time optimization. SA-LUT introduces a spatially adaptive 4D LUT guided by content-style cross-attention, enabling region-dependent color transformations while preserving structural integrity. Unlike these LUT-based approaches, TCFlow explicitly models color distribution transport at the clip level and uses shared flow parameters to reduce frame-to-frame variations in color mapping.

We further evaluate these methods across four benchmarks: DL3DV-10K, Media Art, REDS, and MOSE-v2, using the evaluation metrics. We trained ColorFM on our dataset, while PETAL was evaluated using its official pretrained checkpoint, as training scripts were unavailable at the time of our experiments. For SA-LUT, we used its official pretrained checkpoint because we could not construct the LUT training dataset required to reproduce its training, similar to the limitation encountered with VideoColorGrading (see Lines 327–331).

The results show consistent trends across the four benchmarks (see Tables 1, 7, 8, and 9). TCFlow achieves the lowest SWD on all four benchmarks, demonstrating strong reference-color distribution alignment, and obtains the best or highly competitive MS-SWD results. It also achieves the best overall structural preservation, as reflected by the grayscale, depth, and edge metrics. Importantly, TCFlow achieves the lowest reference-weighted track-color flicker score across all four benchmarks. Although ColorFM exhibits lower Lipschitz values on some benchmarks, TCFlow provides a favorable balance between reference-color fidelity, structural preservation, and temporal consistency.


[W2: Incomplete contribution isolation]

We clarify that the clip-shared adaptation of ModFlows is already evaluated as TCFlow w/o Color Aggregator in Table 2. Specifically, this baseline directly optimizes a single shared flow on the aggregated clip-level color distribution, without using the learned Color Aggregator. It therefore serves as the clip-shared mapping baseline, while the comparison with full TCFlow isolates the additional contribution of the proposed learned video-level parameter estimation and aggregation.

As shown in Table 2, incorporating the learned Color Aggregator improves color distribution alignment, reducing SWD from 0.0599 to 0.0462 and MS-SWD from 1.6606 to 1.2355. It also improves Reference-Weighted Track-Color Flicker, reducing from 0.9585 to 0.8475. These gains come with trade-offs in the Lipschitz, depth, and edge metrics. This comparison demonstrates that the improvements in color fidelity and temporal consistency are not solely attributable to using a clip-shared mapping, but also benefit from the proposed learned video-level estimation and aggregation. We have revised the Impact of Color Aggregator paragraph (see Lines 481-498) to clarify the baseline construction, the role of the learned aggregator, and the resulting performance trade-offs.


References

[1] ColorFM: An Optimization-to-Learning Framework for Color Transfer via Flow Matching. ECCV 2026.

[2] Enlightening Photographic Style Transfer with a Self-Supervised Photographic Embedding. ECCV 2026.

[3] SA-LUT: Spatial Adaptive 4D Look-Up Table for Photorealistic Style Transfer. ICCV 2025.

AuthorsOpenReview ↗

Response to Reviewer 4o5U (2)

[Q1: Sensitivity to Frame Count and Spacing]

We evaluate the sensitivity to frame count and temporal spacing under a fixed color-sampling budget of 51,200 pixels per training step across 100 video-style pairs. Specifically, we compare 4, 8, and 16 sampled frames, allocating 12,800, 6,400, and 3,200 pixels per frame, respectively, and examine uniform-stride and random temporal sampling for the 4- and 8-frame settings (see Appendix A.7). The results are summarized below:

MethodsGrayscale (↓\downarrow)Depth (↓\downarrow)Edge (↓\downarrow)SWD (↓\downarrow)MS-SWD (↓\downarrow)Lipschitz (↓\downarrow)ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}} (↓\downarrow)
Frame 4 (random)0.22060.19280.22160.10275.166519.38540.9043
Frame 4 (stride)0.20460.19160.22010.10185.171719.55500.9031
Frame 8 (random)0.20420.19190.22140.10265.142819.74500.9025
Frame 8 (stride)0.20450.19130.21990.10205.175619.22030.9037
Frame 16 (Total)0.20420.19110.22040.10185.158419.41310.8982

The results show that increasing the number of sampled frames provides modest improvements, while uniform temporal coverage enables reliable clip-level mapping with fewer frames. In particular, Frame 8 (stride) achieves performance comparable to using all 16 frames, with similar depth and reference-weighted track-color flicker scores, while obtaining the best edge score and lowest Lipschitz constant. Even Frame 4 (stride) achieves the same SWD as Frame 16, indicating that a small number of uniformly distributed frames can capture representative clip-level color statistics.

The effect of temporal spacing is more pronounced when only four frames are sampled: random sampling yields a higher grayscale error than uniform-stride sampling. In contrast, the two sampling strategies produce comparable results with eight frames. These findings suggest that uniform temporal coverage is particularly beneficial under a limited frame budget, and that sampling 8 uniformly spaced frames provides a favorable balance between temporal coverage and sampling efficiency.


[Q2: Occlusion-aware, Motion-aligned Temporal Consistency]

We supplement the histogram-based temporal metrics with a tracking-based evaluation that measures color stability along corresponding scene points while accounting for occlusions. Specifically, we use CoTracker3 [1] to extract point trajectories and visibility masks from the original input videos, ensuring that all methods are evaluated on identical physical correspondences. We first define ΔEtrack\Delta E_{\mathrm{track}} to measure temporal changes in color-transfer residuals along valid tracks in CIELAB space, reducing the influence of scene motion, occlusions, and natural illumination changes.

To avoid favoring methods that achieve low flicker through weak or near-identity color transfers, we further incorporate reference color fidelity by weighting ΔEtrack\Delta E_{\mathrm{track}} with the relative SWD between the transferred output and the reference image. We denote this composite metric as ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}}, distinguishing it from the unweighted tracking-based flicker ΔEtrack\Delta E_{\mathrm{track}}. The detailed implementation and mathematical formulations of both metrics are provided in the revised temporal consistency evaluation section. The results are summarized below:

MethodDL3DV-10K ↓\downarrowMedia art ↓\downarrowREDS ↓\downarrowMOSE-v2 ↓\downarrow
CCPL1.82650.83132.19470.3445
CAP-VST1.16170.81541.53570.2939
VideoColorGrading1.09701.29802.62160.5137
ModFlows1.59590.85761.34940.2730
SA-LUT1.93321.52653.42570.7103
ColorFM1.08350.92101.10930.2401
PETAL1.50781.06571.25770.3509
TCFlow (Ours)0.84750.65860.92620.1855

The results show that TCFlow achieves the lowest ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}} across all four benchmarks: DL3DV-10K (0.8475), Media Art (0.6586), REDS (0.9262), and MOSE-v2 (0.1855). These results demonstrate that TCFlow achieves more stable color transformations along corresponding scene regions while maintaining better reference color alignment than the compared methods. We have added this evaluation, its mathematical formulations, and the revised metric notation (see Appendix A.4 and Tables 1, 7, 8 and 9).


Reference

[1] CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. ICCV 2025.

AuthorsOpenReview ↗

Response to Reviewer 4o5U (3)

[Q3: Long Video Processing and Inter-Clip Continuity]

Although TCFlow is designed for 16-frame clips, longer videos can be processed without retraining by aggregating the predicted Neural ODE parameters ϕ^k∈R8195\hat{\phi}_k \in \mathbb{R}^{8195} across clips.

We investigate four strategies:

  1. Independent Per-Clip Inference (per-clip): Processes each 16-frame clip independently, without cross-clip parameter sharing.
  2. Global Weight Averaging (global-average): Averages parameters across all clips to construct a shared video-level flow.
  3. Sliding-Window Weight Averaging (window-average): Averages parameters over a local window (w=3)(w=3).
  4. Overlapping-Frame Blending (frame-overlap): Processes 16-frame windows with an 8-frame overlap and linearly blends overlapping outputs.

We evaluate these strategies against independent per-clip inference on 10 long real-camera dynamic videos (e.g., roller coaster and cycling footage) comprising 8,446 frames and 523 clip boundaries. Table results below:

StrategyRGB L1 (B/I)Lab ΔE (B/I)Boundary ΔEMS-SWD ↓
per-clip21.36 / 19.30 (1.11x)12.80 / 11.13 (1.15x)3.42 (4.81x)1.6908
global-average19.20 / 19.12 (1.00x)11.18 / 11.12 (1.01x)0.64 (0.99x)1.9415
window-average20.38 / 19.31 (1.06x)12.04 / 11.15 (1.08x)2.35 (3.41x)1.7104
frame-overlap19.36 / 19.29 (1.00x)11.19 / 11.14 (1.00x)0.72 (1.00x)1.6829

Independent per-clip inference produces a pronounced boundary color shift (3.42 ΔE\Delta E), corresponding to 4.81 times the intra-clip variation. Sliding-window averaging reduces this shift by approximately 31% (from 3.42 to 2.35 ΔE\Delta E), while global averaging and overlapping-frame boundary color shifts to approximately intra-clip levels (0.99× and 1.00×, respectively). In terms of style fidelity, frame-overlap achieves the best MS-SWD (1.6829), outperforming both independent per-clip inference (1.6908) and global averaging (1.9415).

These results reveal a trade-off between temporal continuity and local color adaptation. Global averaging provides consistent transitions but may underfit substantial illumination variations, whereas overlapping-frame blending better preserves style fidelity and boundary continuity. For abrupt scene cuts, parameter sharing or output blending across unrelated shots may be suboptimal; applying these strategies independently within detected shots is a practical alternative.

We identify shot-boundary-aware parameter aggregation and more computationally efficient temporal adaptation as promising directions for future research, aiming to preserve local color fidelity while maintaining inter-clip continuity without redundant inference.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to Reviewer cyS5 (1)

[W1 & Q1: Clarifying Per-Frame Alignment and Eq. (1)]

To clarify the scope of the transport statement and Eq. (1), the intended distributional alignment target is the pooled distribution of each input clip, as defined in Eq. (3) and used in the teacher objective in Eq. (8). Exact alignment is an idealized transport condition; the learned flows approximate this target in practice.

We have revised Section 3.3 to distinguish sharing a mapping from matching individual frame distributions. The revised per-frame statement specifies $\Psi_{\phi_V}^{(1)}=\cdots=\Psi_{\phi_V}^{(N)} =\Psi_{\phi_V}^{\mathrm{clip}},$ while the distributional target is explicitly written as (ΨϕVclip)♯PV≈η.(\Psi_{\phi_V}^{\mathrm{clip}})_{♯}P_V\approx\eta.

The accompanying text now states that the transformed frame distributions may differ and that their mixture is targeted toward η\eta.

Eq. (1) describes the desired frame-wise reference-color fidelity, not a convergence guarantee implied by pooled-distribution training. To make this interpretation explicit, we have replaced →0\rightarrow 0 with 1N∑n=1NDcol(PI^n,PR)should be small\frac{1}{N}\sum_{n=1}^{N} \mathcal{D}_{\mathrm{col}}(P_{\widehat{I}_n}, P_R) \quad\text{should be small}.

Section 3.1 now identifies this quantity as an output-quality criterion evaluated empirically and explicitly distinguishes it from learning the shared transport on the pooled clip distribution. Thus, the revisions clarify the roles of the transport target and the evaluation criterion without changing the method.


[W2 & Q2: Perceptual Temporal Stability and motion-aligned Evaluation]

We agree that global histogram and fixed-region patch metrics do not directly capture temporal color stability along corresponding scene points. To address this limitation, we introduce a motion-aligned evaluation using CoTracker3 [1]. We extract point trajectories and visibility masks from the original input videos and evaluate all methods using identical valid correspondences. Specifically, ΔEtrack\Delta E_{\mathrm{track}} measures temporal changes in color-transfer residuals along tracked points, while ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}} further incorporates reference color fidelity through relative SWD weighting.

To isolate the effect of shared mapping, we compare frame-wise and clip-shared mappings using the same teachers and backbone. The results are summarized below:

MappingΔEtrack↓\Delta E_{\mathrm{track}} \downarrowΔEtrackref↓\Delta E_{\mathrm{track}}^{\mathrm{ref}} \downarrowDref↓D_{\mathrm{ref}} \downarrow
Frame-wise2.53340.78465.7242
Clip-shared1.01670.28265.1378

As shown in the table, clip-shared mapping reduces ΔEtrack\Delta E_{\mathrm{track}} from 2.5334 to 1.0167 and ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}} from 0.7846 to 0.2826, while also reducing the reference distance DrefD_{\mathrm{ref}} from 5.7242 to 5.1378.

These results demonstrate that applying a shared mapping across frames improves motion-aligned temporal consistency while maintaining better reference color alignment. We have added the motion-aligned evaluation, metric formulations, and clip-shared mapping comparison to the revised paper (see Appendices A.4 and A.9 and Tables 1, 7, 8, and 9).


Reference

[1] CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. ICCV 2025.

AuthorsOpenReview ↗

Response to Reviewer cyS5 (2)

[W3: Role of Color Aggregator and Perceptual Validation]

We agree that the learned aggregator improves SWD but worsens global and patch discrepancies. To isolate the effect of a shared mapping from that of the learned aggregator, we conduct a controlled comparison between frame-wise and clip-shared mappings using the same teacher models and backbone. We further introduce a motion-aligned evaluation based on CoTracker3, measuring temporal changes in color-transfer residuals along corresponding scene points. Clip-shared mapping reduces EtrackE_{\mathrm{track}} from 2.5334 to 1.0167 and the reference-weighted track-color flicker metric ΔEtrackref\Delta E_{\mathrm{track}}^{\mathrm{ref}} from 0.7846 to 0.2826, while also reducing the reference distance from 5.7242 to 5.1378 (see Appendix A.9).

Furthermore, a user study with 23 participants evaluating 19 video sequences shows that TCFlow receives 54.5% of all preferences, compared with 21.1% for ModFlows, 18.3% for ColorFM, and 6.2% for VideoColorGrading (see Appendix A.8). This provides complementary perceptual evidence of the overall quality of TCFlow, although it does not isolate the effect of the Color Aggregator. Together, these results distinguish the temporal consistency benefit of shared mapping from the reference-color fidelity improvements and metric trade-offs associated with learned temporal aggregation. We have revised the discussion to clarify these distinct contributions and avoid attributing the improvements of the full model solely to the learned Color Aggregator.


[W4 & Q3: Real-time]

We agree that the term real-time was not adequately supported by the reported throughput and have revised the corresponding description. TCFlow is designed for offline clip-level video color transfer rather than online streaming, since parameter prediction requires the entire input clip. (see Section 3.5 and Appendix A.3)

We have clarified the computational efficiency evaluation by reporting the overall inference cost, including reference encoding, clip-level parameter prediction, and flow application. (see Appendix A.3) Table 6 reports the total model parameters, GFLOPs per frame, latency per frame, and throughput (FPS). All methods are evaluated using 16-frame clips at a resolution of 1280 × 720. For TCFlow, the Neural ODE uses 8 integration steps in FP32 precision with a pixel batch size of 4,096. Latency is measured after 5 warm-up iterations and averaged over 15 measurement runs. GFLOPs are reported per frame by dividing the total clip-level computational cost by 16.

Under this evaluation, TCFlow achieves 230.89 GFLOPs/frame and 224.60 ms/frame (4.45 FPS), compared with 230.69 GFLOPs/frame and 410.94 ms/frame (2.43 FPS) for ModFlows. These results indicate that TCFlow reduces the measured per-frame latency by 45.35% compared with ModFlows while maintaining a comparable computational cost.

Thank you for the clarifications and additional experiments. The pooled-distribution interpretation resolves my theoretical concern, and the motion-aligned evaluation strengthens the evidence for shared mapping. The revised offline positioning also addresses the unsupported real-time claim. However, the new comparison establishes the benefit of sharing a mapping rather than the benefit of the learned temporal aggregator. A controlled comparison against mean aggregation under the motion-aligned metrics remains missing, and the user study does not isolate this component either. Thus, the temporal contribution of a central proposed module remains insufficiently established. Although the response addresses several concerns, I am maintaining my score of 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.