- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents LightShift, a controllable indoor relighting framework that repurposes a pretrained video diffusion model (Stable Video Diffusion) without requiring test-time geometric or intrinsic estimation. The authors formulate relighting as in-sequence visual conditioning, arranging the source image, a scribble-based lighting instruction, and the relit output as a three-frame sequence, supported by a two-stage synthetic data generation pipeline. The method is validated through benchmark relighting comparisons, user studies, and ablation experiments analyzing video pretraining and conditioning mechanisms.
Critical strengths
- Intrinsic-Free Relighting via Video Priors: Repurposing a pretrained video diffusion model eliminates test-time intrinsic or geometry estimation. Directly synthesizing the relit frame avoids cascading decomposition errors typical of inverse rendering pipelines.
- Validation of In-Sequence Conditioning: Controlled ablations confirm the necessity of full spatio-temporal video pretraining over spatial-only initialization. Furthermore, in-sequence prompt conditioning substantially outperforms an auxiliary ControlNet baseline.
Critical weaknesses
- Coarse Control Space Without Fine-Grained Attributes: Control is restricted to ternary brighten and darken scribbles ({0, 0.5, 1}), without support for light color, intensity. LightShift cannot perform precise color temperature or intensity adjustments, limiting practical utility.
- Oracle Benchmark and Undocumented Resolution Locking: Table 1 evaluates only on dense scribble masks derived from ground-truth target differences, acting as an oracle signal rather than realistic user strokes. The paper would be strengthened by a systematic quantitative/qualitative evaluation on sparse, freehand user scribbles or semantically segmented light fixture masks.
- Weak Baselines and Protocol Mismatches: The baseline comparisons do not conclusively demonstrate LightShift's superiority. IC-Light is not designed for localized scribble guidance and achieves competitive PSNR primarily by making conservative, minimal illumination edits. Furthermore, the evaluation omits contemporary multimodal image editing models (e.g., Flux, Qwen-Image, Ideogram) that can execute lighting instructions zero-shot, missing a crucial test against modern editing frameworks. Additional qualitative comparisons should be possible, as already shown in related work like LuxRemix and TokenLight.
- Omitted Limitations and Missing Literature: The paper lacks a dedicated limitations section discussing failure modes or the potential drift and texture degradation under the iterative relighting shown in Figure 1. In addition, the related work omits foundational frameworks that repurpose video diffusion models for image editing, notably UniReal, as well as 3D-aware generative relighting methods like GR3EN.
Questions for authors
Some complex shading changes, like casting shadows from a window onto a wall or bed as shown in Fig 3 and 6, are predominantly achieved by the provided visual prompts. In the real use case, how should users provide the visual prompts for such complex shading changes? Is there an auxiliary tool to help users provide the visual prompts for such complex shading changes?