- Rating
- 4(+2)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SimStage, a framework that repurposes a pretrained video diffusion model (Wan 2.2) to jointly synthesize a full 331-image OLAT reflectance basis from a single portrait within one generation window. To adapt the video backbone from temporal sequences to spherical illumination, the authors propose LiRoPE, which replaces a subset of temporal RoPE channels with homogeneous 3D translations encoding relative light-direction displacements, paired with an invertible log-sRGB mapping to pass linear HDR radiance through the frozen video VAE. Once generated, the OLAT basis supports arbitrary environment-map portrait relighting via linear composition, which the authors validate on the FaceOLAT dataset alongside ablations on window size and basis coherence.
Critical strengths
- Full-Basis OLAT Formulation with LiRoPE: Jointly generating all 331 OLAT responses within a single video diffusion window effectively eliminates the cross-light structural drift that blurs linearly composited relightings. Rebinding a subset of temporal RoPE channels to relative 3D light-direction translations is mathematically clean. It avoids spherical coordinate singularities and SO(3) gauge obstructions and is empirically essential for convergence.
- Thorough Analysis of Joint Coupling and Low-Rank Feasibility: The paper backs its design with strong analytical and empirical studies, including an integration error decomposition, SVD spectral analysis showing that both the 331-light reflectance field and learned LoRA updates have very low effective rank, and controlled ablations against split-window generation and FramePack context compression.
Critical weaknesses
- Weak Baselines and Missing Open-Source Competitors: Table 1 compares against only a single baseline (3DPR, which is bottlenecked by 3D GAN inversion) while omitting POLAR (CVPR 2026 Oral), a directly comparable 2D single-portrait OLAT generator with publicly released training/inference code and a 16-bit HDR OLAT dataset. Table 2 relies on zero-shot checkpoints of weak or mismatched baselines (2021 CNN RITW, object-level NeuralGaffer, 3D-GAN-based Lite2Relight/3DPR), while omitting much stronger open-source video-diffusion relighting models with released weights such as DiffusionRenderer (CVPR 2025), UniRelight (NeurIPS 2025), and Relit-LiVE (SIGGRAPH 2026).
- Incomplete Related Work: The 17-line Related Work section omits closely related OLAT and conditional video-diffusion portrait/human relighting literature, including Lux Post Facto (CVPR 2025) and BodyReLux (SIGGRAPH 2026), and only mentions DifFRelight (SIGGRAPH Asia 2024) in Appendix Table 4 rather than discussing it in the main text.
- Hard-Locked 2:1 Aspect Ratio and Incomplete RoPE Rebinding: Concatenating the 1:2 latlong environment map feature with the portrait latent via a spatial
Transpose(Fig. 2, undocumented in text) locks the model to a 2:1 vertical input (512 x 256), requiring 512x512 evaluations to be resampled from 256x512 outputs. Meanwhile, keeping 48 channels of standard frame-index Temporal RoPE prevents permutation invariance and degrades length extrapolation (Fig. 5). - Restricted to Static Single-View Portraits & High Inference Overhead: Consuming the video backbone's temporal axis for illumination restricts SimStage to static, fixed-view portraits. It cannot relight dynamic talking-head videos with temporal consistency. Moreover, because single-frame generation collapses ( in Fig. 5), relighting even one target environment requires denoising all 333 frames with a 14B dual-expert model, which also makes extending to dynamic humans very expensive. Additionally, strong directional shadows/highlights in the input portrait might be partially baked into the outputs (Fig. 9); the authors failed to explicitly discuss this.
Questions for authors
- Generalization Beyond Static Portraits: Since the video backbone is pretrained on general scenes, how well does SimStage generalize beyond cropped heads—for instance, to full-body human captures or general objects? Also, could this light-axis formulation be extended to dynamic portrait videos without losing temporal consistency?
- Architecture & Input Sensitivity: Could the authors also clarify how non-2:1 aspect ratios can be handled without cropping/resampling, why 48 temporal RoPE channels were left un-rebound, and how the input lighting, camera parameters p, and brightness scalar b are handled for in-the-wild inputs?
11 comments
Reply to Weakness 2|Incomplete related work
Reply to Weakness 2:
We have added Lux Post Facto, BodyReLux, and DifFRelight to the main-text Related Work section, and updated the corresponding BibTeX entries and bibliography. Please see the Related Work section and References in the revised PDF. The same revised PDF also includes an updated Figure 2 with enlarged text and adjusted Figure 8 to improve readability.
Reply to Weakness 3|Hard-locked 2:1 aspect ratio and incomplete RoPE rebinding
Reply to Weakness 3:
The input aspect ratio is not inherently constrained by that of the OLAT conditioning. A separate patchifier can process the OLAT conditioning, after which the resulting tokens can be flattened and concatenated with the input tokens. Training with this alternative design yields comparable results, and this implementation choice is independent of our main contributions. In this work, however, we used a fixed 2:1 vertical input aspect ratio for both training and inference; therefore, the current model does not support variable input aspect ratios.
Retaining the remaining 48 temporal RoPE channels is intentional, and the model would lose its pretrained knowledge, let alone preserve permutation invariance and length extrapolation, if the remaining 48 temporal RoPE channels were replaced: LiRoPE introduces relative light geometry while preserving part of the backbone’s pretrained positional structure. We do not claim strict permutation equivariance of the complete network, and Fig. 5 does not establish that these retained channels cause length-extrapolation degradation.
In ReRoPE (p. 4, Fig. 3), the authors clearly demonstrated how the Wan model behaves when all temporal RoPE channels are replaced.
Following ReRoPE’s band-selection rationale, we replace the lowest-frequency temporal channels while preserving higher-frequency temporal and spatial bands. This partial rebinding is compatible with our graph-walk training, where neighboring positions generally correspond to nearby light directions.
Fig. 5 changes both inference length and pairwise light-interaction statistics, without isolating the retained temporal bands. Our experiments support partial rebinding and matching training and inference windows; they do not establish whether replacing additional channels would improve extrapolation.
Reply to Weakness 1|Weak baselines and missing open-source competitors
Reply to Weakness 1:
We did not compare our results with POLAR because its official pretrained weights were not released.
We admire the work from the POLAR team and we were very willing to compare our OLAT results with POLAR, yet, as of November 9, its official GitHub repository stated "Note: Pretrained checkpoints are not bundled with this repository. Checkpoint distribution details will be added in a later update." We also reached out to some of the authors of POLAR by email for available preview weights, but they have not replied yet. We are so sorry that we were not able to add POLAR to our OLAT comparison before our submission.
However we managed to infer 3 cases of the in-the-wild OLAT and relight results comparsion to what POLAR provided in their Figure 6, displayed in the github repo we provided: rebuttal repository Since Simstage has more lights thank POLAR, it performs better than POLAR when light come from behind.
DiffusionRenderer, UniRelight, and Relit-LiVE are not designed specifically for portrait relighting, hence we did not include them in our baseline comparison at first. We have additionally evaluated DiffusionRenderer, UniRelight, and Relit-LiVE alongside the existing baselines under a unified square-domain LDR protocol. Please see our response to Reviewer DzWi, Question 3, for the evaluation protocol and exposure-alignment details. The results are:
SimStage has the best reported values across all four metrics in this additional evaluation.
Additional rebuttal materials are available in our rebuttal repository.