- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work tackles the challenging task of image relighting where a 2D image is relit to a given input lighting condition. The authors propose modelling illumination variation in 2D image by jointly considering regional luminance and spatial information. The contributions are twofold. First, they propose an Illumination Intensity Ranking (IIR) loss which instead of pixel-wise global information operates on image patches. It combines luminance and positional consistency and encourages the model to reproduce both the intensity and spatial distribution of illumination. Secondly, they build upon Latent Bridge Matching and propose Illumination Conditioned Latent Bridge Matching (IC-LBM) that injects illumination tokens through cross-attention layers instead of directly concatenating conditioning information with noisy latents. It is evaluated on Multi-Illumination in the Wild (MIIW) dataset. It consists of indoor scenes captured under multiple illumination condition with specular and diffuse light probes capturing the lighting conditions. They demonstrate improvements in reconstruction quality, illumination controllability and relighting through quantitative visualisation and evaluation.
Critical strengths
- Well motivated problem formulation: Pixel wise reconstruction objectives do not model spatially varying illumination in images which the paper motivates well. The proposed patch-wise illumination strategy provides a reasonable approach to addressing this issue.
- Clear methodology: The figures are well designed and the text makes the proposed framework easy to follow. The formulation of the IIR loss is well constructed.
- Detailed ablation studies of the loss function: The contribution of luminance and positional consistency, the influence of patch count and the weighting strategy are well evaluated. These experiments provide useful evidence for understanding how different components of IIR contribute to reported improvements.
- Effective visual presentation: The visualisations of luminance heat maps and gradient directions help illustrate the limitations of existing approaches and the motivation behind the paper.
Critical weaknesses
-
Related work section: It provides a broad overview of existing image relighting approaches but does not discuss the limitations or how the papers fail to address the research gap which the proposed work does. A more critical discussion would help position the paper.
-
Lack of existing illumination aware works: Several existing works explore the use of patch based illumination aware supervision such as :
- Intrinsic Image Harmonization (Guo et al., CVPR 2021)
- Deep Image-Based Illumination Harmonization (Bao et al., CVPR 2022)
- Zero-Reference Deep Curve Estimation (Guo et al., CVPR 2020)
- Intrinsic Image Decomposition via Ordinal Shading (Careaga & Aksoy, 2023)
While these methods operate in different settings, they explore related objectives involving spatial illumination consistency. The paper should address how the proposed IIR loss differs from these works and what advantages the ranking based formulation provides over simpler patch wise luminance reconstruction loss.
Additionally they should also include A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis (Poirier-Ginter et al., 2024) which leverages a diffusion model to perform image relighting.
-
Generalizability of illumination conditioning strategy: Given that one of core contributions is the improved illumination conditioning strategy, the evaluation is restricted to MIIW dataset where illumination is conditioned using specular chrome-probe images. It is unclear whether the conditioning representation generalised to other lighting conditions and datasets. For instance, DiLIghtNet consists of illumination using environment map. Does the conditioning provide similar improvements.
-
Generalizability and effectiveness of IIR loss: Given the second core contribution to be the IIR loss, it’s unclear whether the improvements are specific to the proposed IC-LBM framework or extends to other related works. Would incorporating IIR into the original LBM framework yield similar improvements? Does the improvement extend to other relighting datasets? Existing ablations establish benefits of individual components but further clarification of advantages of ranking formulation over simpler regional luminance supervision would strengthen the central contribution.
-
Implementation details of baseline comparison: The paper lacks the details of how the existing approaches were setup and tested. Given they all have different illumination conditioning strategies, the paper lacks how the inputs to each model were setup.
Questions for authors
- Illumination-aware losses: How does the proposed IIR loss differ from existing illumination-aware objectives, such as those in Intrinsic Image Harmonization (CVPR 2021), Deep Image-Based Illumination Harmonization (CVPR 2022), and Zero-DCE (CVPR 2020)? In particular, what advantages does luminance ranking and positional consistency provide over a simpler patch-wise luminance reconstruction loss?
- Effectiveness of IIR independently of IC-LBM: Would incorporating the proposed IIR loss into the original LBM framework yield similar improvements? This would help clarify whether the benefits of IIR generalize beyond the proposed conditioning architecture.
- Generalizability of illumination conditioning: The proposed conditioning strategy is evaluated using chrome-probe images from MIIW. Would the same approach provide improvements with alternative lighting representations, such as environment maps or spherical harmonics? Is the improvement primarily attributable to the conditioning architecture or the specific lighting representation?
- Fairness of baseline comparisons: Could the authors clarify how target illumination conditions were provided to each baseline in Section 4.3? Were all methods trained or fine-tuned on the same MIIW split, and were any architectural modifications required to accommodate the illumination conditions?
- Generalizability across datasets: Both central contributions are evaluated exclusively on MIIW. Are there reasons to expect the proposed conditioning strategy and IIR loss to generalize to other relighting datasets with different illumination distributions? A discussion of the expected limitations would be helpful.
- Sensitivity to image resolution: Appendix A examines the effect of patch count at a fixed image resolution. How sensitive is IIR to changes in image resolution during training and inference? Does the optimal patch configuration depend on the spatial scale of illumination variations?
12 comments
Response to Q1, W4
Thank you for your review and question. Intrinsic Image Harmonization (CVPR 2021) isolates illumination changes through intrinsic decomposition. Deep Image-Based Illumination Harmonization (CVPR 2022) uses physical rendering to characterize shadow relationships. Zero-DCE (CVPR 2020) focuses on contrast information for low-light enhancement. Our IIR is designed for controllable image illumination editing and models illumination variation directly in 2D images, without additional geometric conditions or intrinsic decomposition. With the luminance ordering of image patches as labels, IIR cleverly couples the comparison of luminance levels with the comparison of bright-region locations.
A simple patch-wise luminance loss does not explicitly model illumination variation. For controllable illumination editing, the simple patch-wise luminance error tends to favor an average optimum over rather than a solution that matches the condition. We have added a detailed comparison between IIR and simple patch-wise luminance losses to Appendix A. With the same architecture and training configuration, IIR outperforms patch-wise luminance losses in PSNR, SSIM, RMSE, and all three illumination controllability metrics. In particular, Target Recall@1 improves by 20.77%, showing that IIR guides outputs toward the specified condition rather than an average optimum.
Response to Q2, W4
Thank you for your comment. To isolate the contribution of IIR, we add it directly to the original LBM framework. With retaining the condition injection mechanism, IIR improves PSNR by 2.36 dB and SSIM by 0.0993, reducing RMSE by 0.0399. The additional results have been included in the updated Table 1.
Response to W1, W2
Thank you for your careful review and constructive suggestions. We have revised the related work section, added relevant references, and summarized the limitations of existing methods to clarify the advantages of our approach.