acceptodds
Under review as a conference paper at ICLR 2027

SimStage: Harnessing Video Diffusion for Consistent Virtual Light Stage Synthesis

Abstract

One-light-at-a-time (OLAT) prediction provides explicit and reusable control for portrait relighting, but its promise depends on recovering an entire lighting basis with both Basis Fidelity and Basis Coherence: each response must preserve accurate HDR appearance, while all responses must remain structurally aligned so that details survive composition. We introduce SimStage, a Full-Basis OLAT Generator that directly predicts all lighting responses within a single generation window. Our key insight is Light-Axis Rebinding: rather than treating OLATs as independent predictions, we redesign the joint-generation paradigm of video diffusion so that its sequence axis represents illumination instead of time. This is non-trivial—native temporal position modeling fails to reliably learn even substantially shorter OLAT windows. We therefore introduce LiRoPE, which replaces temporal relative positioning with calibrated light-direction geometry and enables full-basis joint generation. Together with an HDR radiance encoding, SimStage produces an integration-ready OLAT basis from a single portrait. Experiments on FaceOLAT improve composited relighting PSNR by  dB over 3DPR, while preserving substantially sharper details and accurately controlling shading, cast shadows, and specular highlights.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Portrait Relighting, Video Diffusion, Reflectance Field, Light Stage, Diffusion Transformer, Rotary Position Embedding, HDR Imaging

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_KmkROpenReview ↗
Rating
4(+2)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SimStage, a framework that repurposes a pretrained video diffusion model (Wan 2.2) to jointly synthesize a full 331-image OLAT reflectance basis from a single portrait within one generation window. To adapt the video backbone from temporal sequences to spherical illumination, the authors propose LiRoPE, which replaces a subset of temporal RoPE channels with homogeneous 3D translations encoding relative light-direction displacements, paired with an invertible log-sRGB mapping to pass linear HDR radiance through the frozen video VAE. Once generated, the OLAT basis supports arbitrary environment-map portrait relighting via linear composition, which the authors validate on the FaceOLAT dataset alongside ablations on window size and basis coherence.

Critical strengths
  1. Full-Basis OLAT Formulation with LiRoPE: Jointly generating all 331 OLAT responses within a single video diffusion window effectively eliminates the cross-light structural drift that blurs linearly composited relightings. Rebinding a subset of temporal RoPE channels to relative 3D light-direction translations is mathematically clean. It avoids spherical coordinate singularities and SO(3) gauge obstructions and is empirically essential for convergence.
  2. Thorough Analysis of Joint Coupling and Low-Rank Feasibility: The paper backs its design with strong analytical and empirical studies, including an integration error decomposition, SVD spectral analysis showing that both the 331-light reflectance field and learned LoRA updates have very low effective rank, and controlled ablations against split-window generation and FramePack context compression.
Critical weaknesses
  1. Weak Baselines and Missing Open-Source Competitors: Table 1 compares against only a single baseline (3DPR, which is bottlenecked by 3D GAN inversion) while omitting POLAR (CVPR 2026 Oral), a directly comparable 2D single-portrait OLAT generator with publicly released training/inference code and a 16-bit HDR OLAT dataset. Table 2 relies on zero-shot checkpoints of weak or mismatched baselines (2021 CNN RITW, object-level NeuralGaffer, 3D-GAN-based Lite2Relight/3DPR), while omitting much stronger open-source video-diffusion relighting models with released weights such as DiffusionRenderer (CVPR 2025), UniRelight (NeurIPS 2025), and Relit-LiVE (SIGGRAPH 2026).
  2. Incomplete Related Work: The 17-line Related Work section omits closely related OLAT and conditional video-diffusion portrait/human relighting literature, including Lux Post Facto (CVPR 2025) and BodyReLux (SIGGRAPH 2026), and only mentions DifFRelight (SIGGRAPH Asia 2024) in Appendix Table 4 rather than discussing it in the main text.
  3. Hard-Locked 2:1 Aspect Ratio and Incomplete RoPE Rebinding: Concatenating the 1:2 latlong environment map feature uSu_S with the portrait latent zσz_\sigma via a spatial Transpose (Fig. 2, undocumented in text) locks the model to a 2:1 vertical input (512 x 256), requiring 512x512 evaluations to be resampled from 256x512 outputs. Meanwhile, keeping 48 channels of standard frame-index Temporal RoPE prevents permutation invariance and degrades length extrapolation (Fig. 5).
  4. Restricted to Static Single-View Portraits & High Inference Overhead: Consuming the video backbone's temporal axis for illumination restricts SimStage to static, fixed-view portraits. It cannot relight dynamic talking-head videos with temporal consistency. Moreover, because single-frame generation collapses (F=1F=1 in Fig. 5), relighting even one target environment requires denoising all 333 frames with a 14B dual-expert model, which also makes extending to dynamic humans very expensive. Additionally, strong directional shadows/highlights in the input portrait might be partially baked into the outputs (Fig. 9); the authors failed to explicitly discuss this.
Questions for authors
  1. Generalization Beyond Static Portraits: Since the video backbone is pretrained on general scenes, how well does SimStage generalize beyond cropped heads—for instance, to full-body human captures or general objects? Also, could this light-axis formulation be extended to dynamic portrait videos without losing temporal consistency?
  2. Architecture & Input Sensitivity: Could the authors also clarify how non-2:1 aspect ratios can be handled without cropping/resampling, why 48 temporal RoPE channels were left un-rebound, and how the input lighting, camera parameters p, and brightness scalar b are handled for in-the-wild inputs?
11 comments
AuthorsOpenReview ↗

Reply to Weakness 1|Weak baselines and missing open-source competitors

Reply to Weakness 1:

We did not compare our results with POLAR because its official pretrained weights were not released.

We admire the work from the POLAR team and we were very willing to compare our OLAT results with POLAR, yet, as of November 9, its official GitHub repository stated "Note: Pretrained checkpoints are not bundled with this repository. Checkpoint distribution details will be added in a later update." We also reached out to some of the authors of POLAR by email for available preview weights, but they have not replied yet. We are so sorry that we were not able to add POLAR to our OLAT comparison before our submission.

However we managed to infer 3 cases of the in-the-wild OLAT and relight results comparsion to what POLAR provided in their Figure 6, displayed in the github repo we provided: rebuttal repository Since Simstage has more lights thank POLAR, it performs better than POLAR when light come from behind.

DiffusionRenderer, UniRelight, and Relit-LiVE are not designed specifically for portrait relighting, hence we did not include them in our baseline comparison at first. We have additionally evaluated DiffusionRenderer, UniRelight, and Relit-LiVE alongside the existing baselines under a unified square-domain LDR protocol. Please see our response to Reviewer DzWi, Question 3, for the evaluation protocol and exposure-alignment details. The results are:

MethodPSNR ↑SSIM ↑LPIPS ↓DISTS ↓Median g [p10–p90]
Ours (SimStage)27.880.9750.1020.0870.997 [0.93–1.07]
Relit-LiVE22.870.9040.1760.1271.054
UniRelight20.630.9240.2070.1380.884
DiffusionRenderer19.770.9040.2340.1561.523
IC-Light19.480.8830.2260.1510.790
RITW17.130.8390.2820.2122.740
Lite2Relight17.080.8580.3430.1841.057
3DPR17.070.8680.3720.1780.932 [0.58–1.12]
NeuralGaffer16.980.8270.2890.1901.140

SimStage has the best reported values across all four metrics in this additional evaluation.

Additional rebuttal materials are available in our rebuttal repository.

AuthorsOpenReview ↗

Reply to Weakness 2|Incomplete related work

Reply to Weakness 2:

We have added Lux Post Facto, BodyReLux, and DifFRelight to the main-text Related Work section, and updated the corresponding BibTeX entries and bibliography. Please see the Related Work section and References in the revised PDF. The same revised PDF also includes an updated Figure 2 with enlarged text and adjusted Figure 8 to improve readability.

AuthorsOpenReview ↗

Reply to Weakness 3|Hard-locked 2:1 aspect ratio and incomplete RoPE rebinding

Reply to Weakness 3:

The input aspect ratio is not inherently constrained by that of the OLAT conditioning. A separate patchifier can process the OLAT conditioning, after which the resulting tokens can be flattened and concatenated with the input tokens. Training with this alternative design yields comparable results, and this implementation choice is independent of our main contributions. In this work, however, we used a fixed 2:1 vertical input aspect ratio for both training and inference; therefore, the current model does not support variable input aspect ratios.

Retaining the remaining 48 temporal RoPE channels is intentional, and the model would lose its pretrained knowledge, let alone preserve permutation invariance and length extrapolation, if the remaining 48 temporal RoPE channels were replaced: LiRoPE introduces relative light geometry while preserving part of the backbone’s pretrained positional structure. We do not claim strict permutation equivariance of the complete network, and Fig. 5 does not establish that these retained channels cause length-extrapolation degradation.

In ReRoPE (p. 4, Fig. 3), the authors clearly demonstrated how the Wan model behaves when all temporal RoPE channels are replaced.

Following ReRoPE’s band-selection rationale, we replace the lowest-frequency temporal channels while preserving higher-frequency temporal and spatial bands. This partial rebinding is compatible with our graph-walk training, where neighboring positions generally correspond to nearby light directions.

Fig. 5 changes both inference length and pairwise light-interaction statistics, without isolating the retained temporal bands. Our experiments support partial rebinding and matching training and inference windows; they do not establish whether replacing additional channels would improve extrapolation.

Reviewer_iB23OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method to predict a full One-Light-at-a-Time (OLAT) reflectance field from a single portrait, synthesizing 331 per-light HDR responses for arbitrary environment relighting. By repurposing a pretrained video diffusion model (Wan 2.2), the temporal axis is adapted to represent light directions. Key contributions include LiRoPE, which integrates light-direction geometry into the attention mechanism via modified rotary position embeddings, and a reversible HDR-to-LDR encoding for processing linear radiance with a frozen VAE. Trained on FaceOLAT, the method significantly outperforms prior OLAT prediction and direct relighting baselines, enabling diverse lighting readouts from a single inference pass.

Critical strengths

1.The conceptual reframing of the video diffusion sequence axis to model light directions as a joint generative object is an elegant and well-motivated insight.

2.The LiRoPE mechanism is minimal yet critical for convergence. It effectively injects relative light geometry, complemented by the HDR encoding scheme that seamlessly bridges linear radiance with the frozen video VAE.

3.Empirical comparisons between native full-window and split-window training rigorously validate the necessity of joint denoising. Additionally, the pairwise displacement analysis provides a quantitative explanation for extrapolation failures, offering solid empirical grounding rather than relying on heuristic assumptions.

4.The experimental results are robust, compelling, and demonstrate substantial performance gains across the evaluated metrics.

Critical weaknesses

1.High computational cost relative to performance gains: Fine-tuning a 14B-parameter model on 8×A100-80GB GPUs is resource-intensive. While improvements over 3DPR are evident, the paper does not demonstrate whether a smaller or more targeted architecture could achieve comparable results. For a "single portrait to basis" pipeline, this computational overhead significantly impacts practical viability and efficiency.

Questions for authors

Refer to Critical Weaknesses.

4 comments
AuthorsOpenReview ↗

Reply to Weakness 1|High computational cost relative to performance gains

Reply to Weakness 1:

We thank Reviewer iB23 for raising the concern about model size and computational cost.

The 5B variant substantially outperforms 3DPR across all reported metrics, showing that our approach remains effective with a smaller backbone and that its improvement over 3DPR is not restricted to the 14B model. This provides a lower-cost option for applying our method.

To examine whether our approach requires a 14B backbone, we also trained a smaller variant using Wan 2.2 5B on 4090s and evaluated its relighting performance on our test set:

Method               PSNR ↑  SSIM ↑  LPIPS ↓
3DPR                  18.43   0.691    0.339
Ours (Wan 2.2 5B)     25.05   0.867    0.149

With TeaCache acceleration, our A14B model with BF16 weights generates a complete 331-image OLAT basis in an average of 192 s on two RTX 4090 GPUs, equivalent to 0.580 s per OLAT image. Inference uses a 333-frame window; the 8×A100 configuration was used for fine-tuning. This acceleration incurs virtually no loss in OLAT reconstruction quality: compared with inference without TeaCache, evaluation against ground-truth OLAT images shows essentially unchanged PSNR (+0.06 and +0.09 dB in the two evaluated domains) and an SSIM decrease of at most 0.010.

Once the OLAT basis has been generated for a portrait, relighting under new environments requires only a linear combination of the basis images, with no additional diffusion sampling. (For example, a workstation with an Intel Xeon Platinum 8358P CPU and an RTX 4090 GPU produces 6651 relit images in 25.02 s, or 3.76 ms per frame, measured end to end, including I/O and PNG encoding.) Thus, basis generation is a one-time cost per portrait and can be reused across lighting environments.

We hope the above addresses the reviewer's concerns, and we are happy to provide further details if needed.

I thank the authors for their detailed response. Based on the provided results, the performance gain from scaling the model from 5B to 14B does not appear to be significantly larger than the gain of the 5B model over the 3DPR baseline.

I am curious about the difference in inference speed between the 5B and 14B models in practical deployment scenarios. However, this is merely an additional point of curiosity and should not be considered a weakness or a reason for score deduction.

Regarding the method comparison in Table 2, could the authors confirm whether the experimental settings are strictly consistent across all methods (e.g., backbone or model size)? If there are any discrepancies, they should be explicitly noted in the table or its footnote to ensure a fair comparison. Additionally, I suggest improving the visual clarity of the figures. For instance, thickening the axis lines in all plots would be beneficial. Specifically for Figure 22, enlarging the figure (or its three subplots) would help readers observe the details more clearly.

In summary, I will maintain my original score.

AuthorsOpenReview ↗

Reply to Feedback Comment by Reviewer iB23

Thank you for your feedback!

Simstage with 5B backbone requires 52.3s on average for a full 331-basis (with TeaCache), 0.16s per OLAT image. The A14B model is sharded across two RTX 4090 GPUs without an NVLink connection, introducing inter-GPU communication overhead and resulting in substantially slower inference than on a single A100 GPU.

We compared the outputs of each baseline to the same GT image in each table of Table 2. The 'Simstage' method on each table(OLAT/Relight) in Table 2 is the same checkpoint of our A14B model trained on 333 frames windowsize. We provided a unified exposure metric in addition replying to Weakness 3 by Reviewer DzWi:

To account for global exposure differences across methods, we fit a single scalar gain for each prediction by minimizing the masked RGB squared error to the fixed reference in the decoded linear domain, with the gain constrained to [0.25, 4]. This alignment is applied identically to all methods, including SimStage, and uses one shared gain across RGB channels to preserve color differences.

Specifically, we independently fit one gain for each (method, entry, environment map) triplet, i.e., per frame rather than per method. The gain is computed as

g = clip((∑(p ∈ M) xₚᵀyₚ) / (∑(p ∈ M) ‖xₚ‖₂² + ε), 0.25, 4).

Here, M is the person mask, xₚ and yₚ are the decoded linear RGB values of the prediction and the fixed reference at pixel p, and ε is a small constant for numerical stability. We apply g only to the prediction; the reference remains unchanged. For metric computation, we set the background to zero in both predictions and references using the same person mask, then compute all metrics over the full square canvas. The exposure-aligned LDR results are:

MethodPSNR ↑SSIM ↑LPIPS ↓DISTS ↓Median g [p10–p90]
Ours (SimStage)27.880.9750.1020.0870.997 [0.93–1.07]
Relit-LiVE22.870.9040.1760.1271.054
UniRelight20.630.9240.2070.1380.884
DiffusionRenderer19.770.9040.2340.1561.523
IC-Light19.480.8830.2260.1510.790
RITW17.130.8390.2820.2122.740
Lite2Relight17.080.8580.3430.1841.057
3DPR17.070.8680.3720.1780.932 [0.58–1.12]
NeuralGaffer16.980.8270.2890.1901.140

SimStage achieves 27.88 dB PSNR, 0.975 SSIM, 0.102 LPIPS, and 0.087 DISTS, with the best reported values across all four metrics, supporting our original conclusion.

We are enlarging the figure 22 in appendix to help readers observe the details.

Thank you again for your time and constructive feedback, which have helped us clarify the inference efficiency and evaluation protocol and improve the readability of our figures. We hope these responses address your concerns and would be happy to provide any further clarification.

I sincerely thank the authors for their timely feedback. Their rebuttal has fully addressed my concerns. I maintain a positive evaluation of this work and am inclined to accept the paper. Best of luck with your future research.

Reviewer_DzWiOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SimStage, a video diffusion-based framework that generates a complete set of OLAT images from a single portrait for controllable relighting. Its key innovation is Light-Axis Rebinding, which replaces temporal positional encoding with light-direction geometry through LiRoPE, enabling consistent joint generation of all lighting responses. Combined with HDR radiance encoding, SimStage achieves high-quality relighting through linear composition and significantly outperforms existing methods on the FaceOLAT dataset.

Critical strengths
  1. Each component is well-motivated, with clear and reasonable design choices.
  2. The paper demonstrates novelty, particularly in repurposing the temporal dimension of video diffusion for light directions and introducing a corresponding positional encoding.
  3. The experimental results show visible improvements over existing methods.
  4. The paper is clearly written and easy to follow.
Critical weaknesses
  1. Potential confounding factor in the full-basis generation ablation. In the comparison between 81-frame and 333-frame generation, shorter graph walks tend to focus on nearby light pairs, while longer windows provide better coverage of distant light pairs. I suggest modifying the sampling strategy so that shorter training windows have a light-pair distribution closer to that of full-basis training. Otherwise, it is difficult to determine whether the improvement comes from joint generation or simply from better coverage of distant lighting relationships during training.

  2. Potential information loss caused by the frozen VAE. The pretrained video VAE uses 4× temporal compression, while this paper repurposes the temporal axis as a lighting axis. I am concerned that this compression may smooth out high-frequency lighting variations between neighboring OLAT images, particularly sharp specular highlights and cast shadows. It would be helpful to investigate this potential issue.

  3. Fairness of baseline comparisons. The evaluation protocols for SimStage and the baselines are not fully consistent, particularly in exposure normalization and masking. Since some baselines do not support HDR evaluation, I suggest evaluating all methods under a unified LDR protocol with consistent exposure normalization and masking to ensure a fair comparison.

  4. FramePack ablation with a different backbone. The FramePack comparison uses Wan 2.2 5B rather than the A14B backbone used by SimStage. Given the substantial performance difference between these two models, I wonder why this ablation was not conducted using the same A14B backbone.

  5. Minor presentation issues. Some figures could be improved for readability. In particular, the text in Figure 2 is too small, and the overall layout appears somewhat cluttered.

Questions for authors

See weakness.

7 comments
AuthorsOpenReview ↗

Reply to Weakness 1|Potential confounding factor in the full-basis generation ablation

We thank Reviewer DzWi for the thorough review and for raising several points that helped us sharpen the exposition.

Reply to Weakness 1:

With window length and joint-generation size held fixed at 81 frames, closer light-pair distribution alignment does not by itself improve reconstruction quality. Using the same Wan 2.2 5B backbone, we compare two models trained on fixed spiral windows (spiral81) sampler with random graph walks (walk81) sampler. Spiral81 samples consecutive segments of the inference spiral without randomizing the traversal direction. This comparison removes the change in window length and joint-generation size present in the 81-versus-333-frame comparison. The reconstruction results are:

Training sampler   Window length   Light-pair TV ↓   PSNR-μ ↑   SSIM-μ ↑
spiral81                 81                 0.220      30.60      0.854
walk81                   81                 0.392      31.82      0.885

Although spiral81 better matches the full-basis light-pair distance distribution, its PSNR-μ is 1.22 dB lower and its SSIM-μ is 0.031 lower. Thus, improving distribution alignment through a fixed-direction short-window trajectory does not translate into better image metrics.

The sampling statistics identify the accompanying loss of directional diversity. We measure trajectory turning angles for spiral81, walk81, and forced81, a short-window sampler explicitly constructed to match the full-basis pair-distance distribution. The sampling results are:

Sampler     Light-pair TV ↓   Mean turning angle   P(turn <15°)   P(turn >90°)
walk81              0.392                73.8°          17.0%          40.6%
spiral81            0.220                13.5°          76.6%           0.0%
forced81            0.028                43.2°          36.6%           9.8%
walk333             0.140                73.7°          17.1%          40.4%

Forced81 reduces TV from 0.392 to 0.028, but also reduces the mean turning angle from 73.8° to 43.2° and the fraction of turns above 90° from 40.6% to 9.8%; the fraction below 15° increases from 17.0% to 36.6%. Spiral81 exhibits an even stronger directional restriction, with a mean turn of 13.5° and no turns above 90°. These statistics show that the evaluated short-window alignment strategies also make local trajectories more predictable. The reconstruction evidence comes from spiral81 versus walk81; forced81 is included in the sampling analysis.

Walk333 improves global pair coverage while preserving the directional diversity of walk81. Its TV decreases to 0.140, while its mean turning angle (73.7°) and large-turn fraction (40.4%) remain almost unchanged. Together, the reconstruction control and sampling analysis show why light-pair distribution alignment alone is insufficient: the sampling strategy must also retain diverse directional transitions. Our full-basis design combines broader coverage, preserved directional diversity, and joint generation.

AuthorsOpenReview ↗

Reply to Weakness 2|Potential information loss caused by the frozen VAE

Reply to Weakness 2:

The frozen VAE which uses 4× temporal compression attenuates individual OLAT highlights, but severe intensity loss is rare in the evaluated relit composites, and the VAE is not the main bottleneck in overall reconstruction performance.

The reconstruction upper bound in App. B.4 and our additional tests suggest that the frozen VAE is not the main bottleneck in overall reconstruction performance.

App. B.4 evaluates ground-truth OLAT stacks passed through the complete encoding pipeline, including the VAE encode–decode round-trip, obtaining 47.9 dB PSNR-µ and 0.983 SSIM, compared with our model's 32.68 dB PSNR-µ. This substantial gap points to generation as the primary source of overall reconstruction error.

To examine the specific role of temporal compression, we compared single-frame encoding with encoding in groups of four across 12 windows in the linear HDR domain. In highlight regions (luma ≥ 4), the median reconstructed-to-ground-truth luma ratio is 0.966 for single-frame encoding and 0.483 for four-frame encoding; the corresponding SNRs are 4.0 and 4.1 dB. At shadow boundaries, the median ratio is 0.966 in both modes, with SNRs of 4.4 and 4.3 dB, respectively. Thus, temporal grouping affects local highlight intensities, but does not produce a comparable change in the median shadow-boundary ratio or in regional SNR.

We further tested the downstream effect by linearly composing VAE-reconstructed and ground-truth OLAT bases over 11 entries × 512 environment maps. In these composites, the median reconstructed-to-ground-truth luma ratio is 0.966 over the full portrait, highlight regions, and shadow boundaries. The fractions below 0.5× are only 0.1%, 0.2%, and 0.9%, respectively; the corresponding SNRs are 15.7, 15.6, and 6.9 dB. These results show that severe intensity loss is rare in the evaluated composites, although residual errors remain, particularly at shadow boundaries. Together with App. B.4, they support the conclusion that the VAE is not the main bottleneck in overall performance, without implying lossless preservation of individual OLAT highlights.

AuthorsOpenReview ↗

Reply to Weakness 3|Fairness of baseline comparisons

Reply to Weakness 3:

Following the reviewer's suggestion, we conduct an additional evaluation under a unified square-domain LDR protocol. All methods are evaluated on exactly the same test samples (the same entry–environment map pairs) against the same fixed LDR references, using identical person masks and metric implementations. Our HDR predictions are converted to LDR using the same tone-mapping pipeline as the references. To account for global exposure differences across methods, we fit a single scalar gain for each prediction by minimizing the masked RGB squared error to the fixed reference in the decoded linear domain, with the gain constrained to [0.25, 4]. This alignment is applied identically to all methods, including SimStage, and uses one shared gain across RGB channels to preserve color differences.

Specifically, we independently fit one gain for each (method, entry, environment map) triplet, i.e., per frame rather than per method. The gain is computed as

g = clip((∑(p ∈ M) xₚᵀyₚ) / (∑(p ∈ M) ‖xₚ‖₂² + ε), 0.25, 4).

Here, M is the person mask, xₚ and yₚ are the decoded linear RGB values of the prediction and the fixed reference at pixel p, and ε is a small constant for numerical stability. We apply g only to the prediction; the reference remains unchanged. For metric computation, we set the background to zero in both predictions and references using the same person mask, then compute all metrics over the full square canvas. The exposure-aligned LDR results are:

MethodPSNR ↑SSIM ↑LPIPS ↓DISTS ↓Median g [p10–p90]
Ours (SimStage)27.880.9750.1020.0870.997 [0.93–1.07]
Relit-LiVE22.870.9040.1760.1271.054
UniRelight20.630.9240.2070.1380.884
DiffusionRenderer19.770.9040.2340.1561.523
IC-Light19.480.8830.2260.1510.790
RITW17.130.8390.2820.2122.740
Lite2Relight17.080.8580.3430.1841.057
3DPR17.070.8680.3720.1780.932 [0.58–1.12]
NeuralGaffer16.980.8270.2890.1901.140

SimStage achieves 27.88 dB PSNR, 0.975 SSIM, 0.102 LPIPS, and 0.087 DISTS, with the best reported values across all four metrics, supporting our original conclusion.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of changes in Revised PDF

Dear Reviewers and Area Chair,

We have uploaded a revised PDF with the following updates:

  • Figure 2 (p. 3): Updated the font sized of content that was too tiny to read to improve readability. See comment on Reply to Weakness 5 of Reviewer DzWi
  • Figure 8 (p. 9): Adjusted the layout to improve readability. See comment on Reply to Weakness 5 of Reviewer DzWi
  • Related Work (p. 3): Added discussion of DifFRelight, Lux Post Facto, and BodyReLux. See comment on Reply to Weakness 2 of Reviewer KmkR
  • References (p. 11): Adjusted the corresponding references and corrected existing bibliography entries.

The added Related Work text and the adjusted reference entries are highlighted in blue. The figures retain their original colors.

Thank you for your constructive feedback and time.

The Authors

AuthorsOpenReview ↗

Summary of Contributions and Revisions

Dear Committee,

Thank you all for your careful evaluation. SimStage recovers a reusable lighting basis from a single portrait. We have responded to the main concerns and would like to summarize the evidence we added during the rebuttal.

Additional experiments. We evaluated eight baselines, including Relit-LiVE, UniRelight, and DiffusionRenderer, using the same test pairs, LDR references, person masks, and exposure-alignment procedure. SimStage led on all four metrics, reaching 27.88 dB PSNR versus 22.87 dB for the closest baseline. With the A14B backbone held fixed, SimStage also gained 2.75 dB over FramePack-style generation. We further examined VAE highlight attenuation and light-pair sampling, and reported full-basis inference times with TeaCache (52.3 s for 5B and 192 s for A14B).

Examples in the rebuttal repository. We relit pairs of portraits of the same subject, one lit from the left and one from the right, under identical target OLAT lamps and environment maps. The outputs were similar even where the original shadows fell on opposite sides of the face. We also added three in-the-wild comparisons with POLAR, identifying the rows with comparable environment-map orientations, alongside full-body and object examples and a short talking-head composite. Anonymous rebuttal repository

  • KmkR raised their recommendation to Clear Acceptance after the paired-input examples resolved their concern about persistent input shadows, and asked us to include the stronger baselines in the paper and state the fixed input aspect ratio explicitly.
  • DzWi maintained Weak Accept, finding that the unified evaluation and same-backbone comparison largely resolved their fairness concerns, while asking us to acknowledge highlight attenuation in individual OLAT images and temper our causal claim because the ablations do not isolate joint generation from light-pair coverage and directional diversity.
  • iB23 maintained Weak Accept, confirming that our smaller-model results, inference timings, and clarification of the Table 2 evaluation settings had fully addressed their concerns.

We believe the additional experiments strengthen the empirical support for SimStage’s approach to portrait relighting. We respectfully ask you to consider our point-by-point responses, the new evidence, and the reviewers’ final assessments in reaching your decision.

Thank you for your time and thoughtful consideration.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.