- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CoRe, a streaming video relighting method that applies lighting edits without rebuilding the attention cache. It routes several heads to keys and values from the current lighting only, while the others keep the history cache. The selection is learned under a fixed head budget with a loss matching post-edit outputs to constant-light references, then frozen. Experiments on the LongLive streaming generator cover fixed-light clips and 45.6 to 60 s streams with one edit. On text-conditioned streams, the residual of the old lighting after an edit falls from 0.18 to 0.07 against the same backbone trained only under constant lighting, at 23.8 instead of 20 FPS.
Critical strengths
- The insight is straightforward: Routing each head to video memory or current lighting avoids cache rebuilds. Instead of keeping or clearing the whole cache, the learned heads read only the current lighting, and the other heads keep the video history.
- The experimental results show promise. Stale lighting drops sharply while throughput increases. Compared with the same backbone trained only under constant lighting, the old-light residual (OldRes) falls from 0.18 to 0.07 on text-conditioned streams, with throughput rising from 20 to 23.8 FPS (Table 2). On HDR-conditioned streams it falls from 0.19 to 0.05 (Table 11).
Critical weaknesses
- The mechanism behind the gain is not isolated. Direct evidence that stale lighting concentrates in a subset of heads is one causal-patching figure (Figure 13) with undefined axis metrics and no error bars or sample size. The HDR-stream ablations (Table 9) lack a variant whose selected heads lose cache access but receive no lighting keys and values, so the source of the gain is unclear.
- Training and evaluation details needed for reproduction are missing. The paper reports no optimizer, learning rate, iteration count, batch size, loss weights, or length of the post-edit recovery window, and no number of clips or streams per table. These gaps matter because training has three stages: constant-light pretraining, head search, and training with frozen heads.
Questions for authors
None.