- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper fine-tunes HiDream-O1-Image, an 8B pixel-space unified model, for single-image HDR relighting. The target environment is encoded as three LDR maps. These maps and the source enter the model’s existing pathways, so no new modules are added. The model predicts Log-Gamma-encoded HDR directly in pixel space. A linearity constraint (LC) encourages predictions to be additive in illumination. Training uses only synthetic Objaverse objects. The method outperforms six baselines and generalizes well to real photographs.
Critical strengths
- Results are strong. Shadows follow the target light, and source details are preserved. All 192 real-photo comparisons are released.
- Native HDR output allows re-exposure without clipping. Pixel-space prediction avoids LDR-VAE workarounds.
- The design is simple and adds no new modules.
- The LC derivation is clean, and its limitations are discussed honestly.
- The HDR relighting dataset will be released.
Critical weaknesses
W1. Unclear attribution. The method differs from the baselines in backbone scale, resolution, latent space, and data all at once. The ablations remove inputs but do not test the stated alternatives, such as channel concatenation on the same backbone. The zero-shot base model already relights plausibly (Fig. 2). The causal claims should be tempered.
W2. Evaluation protocol. On OLATverse, only the proposed method receives auto-exposure; baselines are evaluated in native LDR (App. C). This may inflate the 6 dB margin in Table 1. All metrics are also computed in LDR, although HDR is the main claim.
W3. Scope of HDR. Radiance is capped at M = 50, far below sun or specular levels. The absolute scale is undefined for an LDR source. Without LC, the output barely responds to light intensity (Table 12: 0.87 to 1.24 for 0.5× to 2.5×).
W4. LC in Log-Gamma space. LC is exact only in linear HDR. In the encoded space it becomes a soft regularizer (App. H.3). Its error rises from 3.7% to 23.5% (Table 2). Exact mixed-light targets are freely available from synthetic data. The comparison with direct supervision of these targets is limited to a 32×32 toy (App. E.1.5).
W5. Rotation and temporal behavior. Rotating-light videos are shown without ground truth. The authors admit frame-by-frame flicker on videos (App. A). However, the videos show only static images under changing light, not dynamic video. This limits use in video applications.
W6. Presentation. I recommend redrawing Figure 4 for clarity. Some routing lines are bent without need. R₁–R₃ merge into a single junction, which may suggest joint encoding, and the text input is not shown. Wrapped floats mid-paragraph break the reading flow.
Questions for authors
- Why does the natural-image ViT extract useful features from non-natural maps like R₂ and R₃?
- Was the initial noise fixed across frames in the rotation videos?
- How does LC handle scenes with emitters, where additivity fails?
- What is the inference time and memory per 2048² image?
5 comments
Response to Reviewer W7g2 (2/3): W3–W4
W3. Scope of HDR.
Radiance is capped at M = 50, far below sun or specular levels. The absolute scale is undefined for an LDR source. Without LC, the output barely responds to light intensity (Table 12: 0.87 to 1.24 for 0.5× to 2.5×).
Response. We agree that this is a limitation of our current model: pixel values above M = 50 are clipped in the output, and M is far below sun levels and often below specular levels. We will update the paper to mention this limitation. We would like to put the cap into perspective. Clipping can only be seen when a clipped pixel is shown below the display peak, the brightest level that can be displayed; as long as it is shown at the peak, it looks the same as the true, brighter value. What matters is therefore how far the cap lies above the display peak. On a standard display, the peak is 1.0, and the cap is 50 times as high. An image can therefore be made up to 50 times darker, that is, lowered by more than 5 stops, before any clipped pixel drops below the peak and the cap becomes visible. HDR displays can show brighter values, so the cap is closer to their peak: 4 to 25 times as high for the displays in the table below, which leaves 2 to 4.7 stops. In other words, the cap does not affect normal viewing, and it leaves this much room for lowering the exposure afterwards.
| Viewing mapping | EXR value at the peak of a 400-nit display | at the peak of a 1000-nit display |
|---|---|---|
| 1.0 → 203 nits (HDR reference white) | 1.97 | 4.93 |
| 1.0 → 80 nits (Windows scRGB) | 5 | 12.5 |
We also agree with the other two points, and they are related. An LDR source has an unknown exposure and carries no absolute scale, so the absolute level of the output can only come from the intensity of the target lighting. As the reviewer notes, without LC the model barely uses this intensity, so the output is correct only up to a global exposure factor. Because the output is linear HDR, this factor can be adjusted afterwards without loss and the appearance is not affected, but the output is not absolute radiance. Making the output follow the intensity of the light is exactly what the linearity of light transport requires, and this is one reason we add LC as a regularizer.
W4. LC in Log-Gamma space.
LC is exact only in linear HDR. In the encoded space it becomes a soft regularizer (App. H.3). Its error rises from 3.7% to 23.5% (Table 2).
Response. We agree. In our Log-Gamma encoding, LC holds only approximately and acts as a soft regularizer, and its error there is 23.5% (Table 2). We consider this a limitation of our current method, and addressing it is an important direction for our future work. At the same time, Table 2 also shows what the constraint achieves in the setting where it holds exactly: in linear HDR, the model trained with LC has a measured error of only 3.7%. We see this as evidence of the constraint's potential going forward.
Exact mixed-light targets are freely available from synthetic data. The comparison with direct supervision of these targets is limited to a 32×32 toy (App. E.1.5).
Response. We agree that a full-model comparison between LC and direct supervision with the summed targets (data augmentation, DA) is needed, and the submission does not contain one: as stated in Appendix E.1.5, that single-scene study does not establish "a general advantage over DA". Our compute is limited, and we gave priority to the same-backbone conditioning comparison requested in W1. We will add the full-model comparison to the revised version and the final version. DA and LC are not in conflict and can be used together; Appendix E.1.5 includes a DA+LC configuration (Table 9).
Response to Reviewer W7g2 (3/3): W5–W6, Q1–Q4
W5. Rotation and temporal behavior.
Rotating-light videos are shown without ground truth. The authors admit frame-by-frame flicker on videos (App. A). However, the videos show only static images under changing light, not dynamic video. This limits use in video applications.
Response. We agree. Our model is an image model so it uses no information from adjacent frames at all, either as input during inference or as supervision during training. Incorporating inter-frame information and spatio-temporal attention during training and inference should largely alleviate it, and this is an important direction for future work.
W6. Presentation.
I recommend redrawing Figure 4 for clarity. Some routing lines are bent without need. R₁–R₃ merge into a single junction, which may suggest joint encoding, and the text input is not shown. Wrapped floats mid-paragraph break the reading flow.
Response. Thank you. We will simplify Fig. 4's routing, show the R₁–R₃ paths separately, add the text input, and replace mid-paragraph wrapped figures with standard floats.
Q1.
Why does the natural-image ViT extract useful features from non-natural maps like R₂ and R₃?
Response. Empirically, the lighting features from the visual encoder matter: without the lighting Visual tokens, the cast shadow of the car's mirror no longer follows the mirror, and without the raw lighting patches it is small and faint (Sec. 5). This also suggests why a natural-image encoder suffices: what the model takes from it is mainly the layout of the lighting, that is, where the light comes from, while exact intensities come from the raw patches. This layout is well preserved in both maps. R₂ is the same environment panorama as R₁ under a logarithmic tone curve, so it still looks like a photograph of the scene, darker and with the strong light sources standing out. R₃ is a smooth color chart in which the color of each pixel encodes its direction, and it serves as a direction reference for the other two maps. Finally, the encoder is frozen, but the Transformer that reads its features is fine-tuned and learns to use them for relighting.
Q2.
Was the initial noise fixed across frames in the rotation videos?
Response. Yes, the initial noise is fixed across all frames within each rotating-light sequence.
Q3.
How does LC handle scenes with emitters, where additivity fails?
Response. Good question. Light transport itself remains additive in scenes with emitters. The issue is the way LC builds its training target, which adds two complete images.
Let H(E) denote the part of the image that comes from the environment lighting E, and let B denote the part that comes from the scene's own emitters, including their indirect illumination. B does not depend on E. The images under E1 and E2 are then H(E1) + B and H(E2) + B. Adding these two images gives H(E1 + E2) + 2B, because H is additive. The correct image under the combined lighting E1 + E2, however, is H(E1 + E2) + B, since the self-emission is unchanged. The constructed target therefore contains B once too often.
For this reason, we deliberately skipped all assets with self-emissive materials when generating the training data for LC. This is a very real limitation of LC and we will discuss it. To be fully consistent in theory, one would have to separate the emissive part B from the non-emissive part H(E) during training, add the two to obtain the final image for visualization, and apply LC only to the non-emissive part. In practice, LC is only one regularization term with a low weight, and its training data skips emitters, so on scenes with emitters the model still appears to work, and LC at least does not cause the results to fail.
Q4.
What is the inference time and memory per 2048² image?
Response. On an H100, our model produces a 2048×2048 HDR image in 29.2 s with 18.4 GiB of peak memory (50 sampling steps, batch 1, model loading excluded). A more detailed discussion and comparison are in our response to Reviewer xqCU (W4).
Response to Reviewer W7g2 (1/3): W1–W2
We thank the reviewer for the detailed and constructive review. The comments on attribution, the evaluation protocol and the scope of HDR helped us to improve the paper, and we answer each point below.
W1. Unclear attribution.
Response. Please see our common response "Same-backbone comparison of lighting conditioning", which answers this point together with Reviewer xqCU (W2).
W2. Evaluation protocol.
Response. Our model outputs HDR and the baselines output LDR, so our prediction must be converted to a display image. We use a standard conversion that does not use the ground truth: the histogram auto-exposure of Unreal Engine (documentation), followed by ACES 2.0. For HDR, exposure is a lossless choice; LDR outputs have it baked in. We acknowledge, however, that applying auto-exposure only to our output made Table 1 asymmetric and favored our method. We therefore re-scored Table 1 with the same auto-exposure rule applied to every method (baselines re-exposed in linear sRGB; ours unchanged). As a further check, we also fitted a ground-truth gain to every method, an oracle unavailable at test time. In both settings the margin is smaller than in Table 1, but we remain best on all three metrics (table below). We will add this table and clarify App. C. We will update the paper.
PSNR / SSIM / LPIPS on OLATverse (340 pairs, 34 objects weighted equally).
Response. As noted above, the baselines output only LDR, so the comparison with them has to be made in LDR, which is why Table 1 uses LDR metrics. We agree that the accuracy of our HDR output should also be measured in HDR. We now report metrics in linear HDR space for our model on the two benchmarks of the paper that have HDR ground truth, OLATverse and our synthetic Internal170, and additionally on BounceBRDF (Iser et al., 2026), a real HDR capture. The results are in the table below, and we will add this table to the appendix.
PU21-PSNR in dB (higher is better); log2 RMSE in stops (lower is better). Each prediction is aligned to the ground truth with one scalar gain.