acceptodds
Under review as a conference paper at ICLR 2027

Native-HDR Relighting with Pixel-Space Flow Models

Abstract

We present the first native high-dynamic-range (HDR) pixel-space model for single-image relighting. Fine-tuned only on paired synthetic single-object images, it generalizes far beyond this fine-tuning distribution and outperforms previous methods on even complex real-world scenes. This is achieved by three design choices. First, whereas previous methods modify the pretrained model's conditioning pathways to fit the relighting conditions, we instead conform the conditions to the model: express the conditions in a form the model fully accepts, which preserves the model's pretrained knowledge. Second, whereas most previous methods produce only low-dynamic-range (LDR) outputs through LDR-trained variational autoencoders (VAEs), our model predicts native HDR directly in pixel space. HDR more closely represents the physical nature of light transport, and predicting in pixel space avoids the lossy reconstruction and representational constraints of such VAEs, preserving fine source details and simplifying HDR fine-tuning. Third, native HDR output lets us formulate a linearity constraint grounded in the additivity of light transport, which maintains consistency better than previous formulations that operate in the latent space of LDR-trained VAEs.

Primary area:
generative models
Keywords:
image relighting, high dynamic range imaging, flow matching, generative models, light transport

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_W7g2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper fine-tunes HiDream-O1-Image, an 8B pixel-space unified model, for single-image HDR relighting. The target environment is encoded as three LDR maps. These maps and the source enter the model’s existing pathways, so no new modules are added. The model predicts Log-Gamma-encoded HDR directly in pixel space. A linearity constraint (LC) encourages predictions to be additive in illumination. Training uses only synthetic Objaverse objects. The method outperforms six baselines and generalizes well to real photographs.

Critical strengths
  1. Results are strong. Shadows follow the target light, and source details are preserved. All 192 real-photo comparisons are released.
  2. Native HDR output allows re-exposure without clipping. Pixel-space prediction avoids LDR-VAE workarounds.
  3. The design is simple and adds no new modules.
  4. The LC derivation is clean, and its limitations are discussed honestly.
  5. The HDR relighting dataset will be released.
Critical weaknesses

W1. Unclear attribution. The method differs from the baselines in backbone scale, resolution, latent space, and data all at once. The ablations remove inputs but do not test the stated alternatives, such as channel concatenation on the same backbone. The zero-shot base model already relights plausibly (Fig. 2). The causal claims should be tempered.

W2. Evaluation protocol. On OLATverse, only the proposed method receives auto-exposure; baselines are evaluated in native LDR (App. C). This may inflate the 6 dB margin in Table 1. All metrics are also computed in LDR, although HDR is the main claim.

W3. Scope of HDR. Radiance is capped at M = 50, far below sun or specular levels. The absolute scale is undefined for an LDR source. Without LC, the output barely responds to light intensity (Table 12: 0.87 to 1.24 for 0.5× to 2.5×).

W4. LC in Log-Gamma space. LC is exact only in linear HDR. In the encoded space it becomes a soft regularizer (App. H.3). Its error rises from 3.7% to 23.5% (Table 2). Exact mixed-light targets are freely available from synthetic data. The comparison with direct supervision of these targets is limited to a 32×32 toy (App. E.1.5).

W5. Rotation and temporal behavior. Rotating-light videos are shown without ground truth. The authors admit frame-by-frame flicker on videos (App. A). However, the videos show only static images under changing light, not dynamic video. This limits use in video applications.

W6. Presentation. I recommend redrawing Figure 4 for clarity. Some routing lines are bent without need. R₁–R₃ merge into a single junction, which may suggest joint encoding, and the text input is not shown. Wrapped floats mid-paragraph break the reading flow.

Questions for authors
  1. Why does the natural-image ViT extract useful features from non-natural maps like R₂ and R₃?
  2. Was the initial noise fixed across frames in the rotation videos?
  3. How does LC handle scenes with emitters, where additivity fails?
  4. What is the inference time and memory per 2048² image?
5 comments
AuthorsOpenReview ↗

Response to Reviewer W7g2 (1/3): W1–W2

We thank the reviewer for the detailed and constructive review. The comments on attribution, the evaluation protocol and the scope of HDR helped us to improve the paper, and we answer each point below.

W1. Unclear attribution.

The method differs from the baselines in backbone scale, resolution, latent space, and data all at once. The ablations remove inputs but do not test the stated alternatives, such as channel concatenation on the same backbone. The zero-shot base model already relights plausibly (Fig. 2). The causal claims should be tempered.

Response. Please see our common response "Same-backbone comparison of lighting conditioning", which answers this point together with Reviewer xqCU (W2).

W2. Evaluation protocol.

On OLATverse, only the proposed method receives auto-exposure; baselines are evaluated in native LDR (App. C). This may inflate the 6 dB margin in Table 1.

Response. Our model outputs HDR and the baselines output LDR, so our prediction must be converted to a display image. We use a standard conversion that does not use the ground truth: the histogram auto-exposure of Unreal Engine (documentation), followed by ACES 2.0. For HDR, exposure is a lossless choice; LDR outputs have it baked in. We acknowledge, however, that applying auto-exposure only to our output made Table 1 asymmetric and favored our method. We therefore re-scored Table 1 with the same auto-exposure rule applied to every method (baselines re-exposed in linear sRGB; ours unchanged). As a further check, we also fitted a ground-truth gain to every method, an oracle unavailable at test time. In both settings the margin is smaller than in Table 1, but we remain best on all three metrics (table below). We will add this table and clarify App. C. We will update the paper.

MethodTable 1Same auto-exposureGT-fitted gain (oracle)
Ours23.55 / 0.802 / 0.14523.55 / 0.802 / 0.14522.90 / 0.790 / 0.149
UniRelight17.23 / 0.733 / 0.21520.58 / 0.758 / 0.20619.93 / 0.744 / 0.220
DiffusionRenderer-SVD15.57 / 0.590 / 0.28518.99 / 0.623 / 0.26818.64 / 0.610 / 0.280
Neural Gaffer16.16 / 0.551 / 0.34017.62 / 0.561 / 0.33517.53 / 0.549 / 0.339
IC-Light12.73 / 0.536 / 0.33117.06 / 0.599 / 0.29517.21 / 0.596 / 0.299
DiLightNet15.82 / 0.493 / 0.34017.30 / 0.496 / 0.34017.17 / 0.491 / 0.343
Light-X13.74 / 0.550 / 0.42315.58 / 0.568 / 0.40915.92 / 0.564 / 0.414

PSNR / SSIM / LPIPS on OLATverse (340 pairs, 34 objects weighted equally).

All metrics are also computed in LDR, although HDR is the main claim.

Response. As noted above, the baselines output only LDR, so the comparison with them has to be made in LDR, which is why Table 1 uses LDR metrics. We agree that the accuracy of our HDR output should also be measured in HDR. We now report metrics in linear HDR space for our model on the two benchmarks of the paper that have HDR ground truth, OLATverse and our synthetic Internal170, and additionally on BounceBRDF (Iser et al., 2026), a real HDR capture. The results are in the table below, and we will add this table to the appendix.

BenchmarkAll pixels: PU21-PSNR / log2 RMSEBrightest 5%: PU21-PSNR / log2 RMSE
OLATverse (340 pairs)29.11 / 0.6728.23 / 0.70
Internal170 (170 pairs)27.82 / 0.9426.57 / 1.07
BounceBRDF (51 pairs)32.08 / 0.5125.37 / 1.09

PU21-PSNR in dB (higher is better); log2 RMSE in stops (lower is better). Each prediction is aligned to the ground truth with one scalar gain.

AuthorsOpenReview ↗

Response to Reviewer W7g2 (2/3): W3–W4

W3. Scope of HDR.

Radiance is capped at M = 50, far below sun or specular levels. The absolute scale is undefined for an LDR source. Without LC, the output barely responds to light intensity (Table 12: 0.87 to 1.24 for 0.5× to 2.5×).

Response. We agree that this is a limitation of our current model: pixel values above M = 50 are clipped in the output, and M is far below sun levels and often below specular levels. We will update the paper to mention this limitation. We would like to put the cap into perspective. Clipping can only be seen when a clipped pixel is shown below the display peak, the brightest level that can be displayed; as long as it is shown at the peak, it looks the same as the true, brighter value. What matters is therefore how far the cap lies above the display peak. On a standard display, the peak is 1.0, and the cap is 50 times as high. An image can therefore be made up to 50 times darker, that is, lowered by more than 5 stops, before any clipped pixel drops below the peak and the cap becomes visible. HDR displays can show brighter values, so the cap is closer to their peak: 4 to 25 times as high for the displays in the table below, which leaves 2 to 4.7 stops. In other words, the cap does not affect normal viewing, and it leaves this much room for lowering the exposure afterwards.

Viewing mappingEXR value at the peak of a 400-nit displayat the peak of a 1000-nit display
1.0 → 203 nits (HDR reference white)1.974.93
1.0 → 80 nits (Windows scRGB)512.5

We also agree with the other two points, and they are related. An LDR source has an unknown exposure and carries no absolute scale, so the absolute level of the output can only come from the intensity of the target lighting. As the reviewer notes, without LC the model barely uses this intensity, so the output is correct only up to a global exposure factor. Because the output is linear HDR, this factor can be adjusted afterwards without loss and the appearance is not affected, but the output is not absolute radiance. Making the output follow the intensity of the light is exactly what the linearity of light transport requires, and this is one reason we add LC as a regularizer.

W4. LC in Log-Gamma space.

LC is exact only in linear HDR. In the encoded space it becomes a soft regularizer (App. H.3). Its error rises from 3.7% to 23.5% (Table 2).

Response. We agree. In our Log-Gamma encoding, LC holds only approximately and acts as a soft regularizer, and its error there is 23.5% (Table 2). We consider this a limitation of our current method, and addressing it is an important direction for our future work. At the same time, Table 2 also shows what the constraint achieves in the setting where it holds exactly: in linear HDR, the model trained with LC has a measured error of only 3.7%. We see this as evidence of the constraint's potential going forward.

Exact mixed-light targets are freely available from synthetic data. The comparison with direct supervision of these targets is limited to a 32×32 toy (App. E.1.5).

Response. We agree that a full-model comparison between LC and direct supervision with the summed targets (data augmentation, DA) is needed, and the submission does not contain one: as stated in Appendix E.1.5, that single-scene study does not establish "a general advantage over DA". Our compute is limited, and we gave priority to the same-backbone conditioning comparison requested in W1. We will add the full-model comparison to the revised version and the final version. DA and LC are not in conflict and can be used together; Appendix E.1.5 includes a DA+LC configuration (Table 9).

AuthorsOpenReview ↗

Response to Reviewer W7g2 (3/3): W5–W6, Q1–Q4

W5. Rotation and temporal behavior.

Rotating-light videos are shown without ground truth. The authors admit frame-by-frame flicker on videos (App. A). However, the videos show only static images under changing light, not dynamic video. This limits use in video applications.

Response. We agree. Our model is an image model so it uses no information from adjacent frames at all, either as input during inference or as supervision during training. Incorporating inter-frame information and spatio-temporal attention during training and inference should largely alleviate it, and this is an important direction for future work.

W6. Presentation.

I recommend redrawing Figure 4 for clarity. Some routing lines are bent without need. R₁–R₃ merge into a single junction, which may suggest joint encoding, and the text input is not shown. Wrapped floats mid-paragraph break the reading flow.

Response. Thank you. We will simplify Fig. 4's routing, show the R₁–R₃ paths separately, add the text input, and replace mid-paragraph wrapped figures with standard floats.

Q1.

Why does the natural-image ViT extract useful features from non-natural maps like R₂ and R₃?

Response. Empirically, the lighting features from the visual encoder matter: without the lighting Visual tokens, the cast shadow of the car's mirror no longer follows the mirror, and without the raw lighting patches it is small and faint (Sec. 5). This also suggests why a natural-image encoder suffices: what the model takes from it is mainly the layout of the lighting, that is, where the light comes from, while exact intensities come from the raw patches. This layout is well preserved in both maps. R₂ is the same environment panorama as R₁ under a logarithmic tone curve, so it still looks like a photograph of the scene, darker and with the strong light sources standing out. R₃ is a smooth color chart in which the color of each pixel encodes its direction, and it serves as a direction reference for the other two maps. Finally, the encoder is frozen, but the Transformer that reads its features is fine-tuned and learns to use them for relighting.

Q2.

Was the initial noise fixed across frames in the rotation videos?

Response. Yes, the initial noise is fixed across all frames within each rotating-light sequence.

Q3.

How does LC handle scenes with emitters, where additivity fails?

Response. Good question. Light transport itself remains additive in scenes with emitters. The issue is the way LC builds its training target, which adds two complete images.

Let H(E) denote the part of the image that comes from the environment lighting E, and let B denote the part that comes from the scene's own emitters, including their indirect illumination. B does not depend on E. The images under E1 and E2 are then H(E1) + B and H(E2) + B. Adding these two images gives H(E1 + E2) + 2B, because H is additive. The correct image under the combined lighting E1 + E2, however, is H(E1 + E2) + B, since the self-emission is unchanged. The constructed target therefore contains B once too often.

For this reason, we deliberately skipped all assets with self-emissive materials when generating the training data for LC. This is a very real limitation of LC and we will discuss it. To be fully consistent in theory, one would have to separate the emissive part B from the non-emissive part H(E) during training, add the two to obtain the final image for visualization, and apply LC only to the non-emissive part. In practice, LC is only one regularization term with a low weight, and its training data skips emitters, so on scenes with emitters the model still appears to work, and LC at least does not cause the results to fail.

Q4.

What is the inference time and memory per 2048² image?

Response. On an H100, our model produces a 2048×2048 HDR image in 29.2 s with 18.4 GiB of peak memory (50 sampling steps, batch 1, model loading excluded). A more detailed discussion and comparison are in our response to Reviewer xqCU (W4).

Reviewer_2bQtOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents a native high-dynamic-range (HDR) single-image relighting framework that operates directly in pixel space, bypassing the detail loss and dynamic-range clipping of LDR-trained VAEs. Adapting the 8B multimodal model HiDream-O1-Image at native 2048 x 2048 resolution, the authors route lighting conditions through existing semantic and raw-patch pathways without adding external modules, paired with an analytical light-transport linearity constraint (LC) derived under independent Gaussian noises via dynamic space and time rescaling. Fine-tuned solely on synthetic single-object renders, the model demonstrates strong zero-shot generalization to diverse real-world photographs and achieves superior quantitative relighting accuracy on real-object captures alongside consistent multi-light superposition.

Critical strengths
  1. Analytical Linearity Formulation under Independent Noises: Unlike heuristic noise-sharing in prior work, the paper derives an exact flow composition under independent Gaussian noises via dynamic space and time rescaling (γt\gamma_t and sts_t). This translates physical light additivity into an analytical velocity-field constraint with zero parameter or inference overhead, offering broad potential for physical superposition tasks beyond relighting.
  2. Native Pixel-Space HDR Generation without VAE Bottlenecks: Bypassing the lossy compression and highlight clipping of LDR-trained VAEs, the model generates native HDR radiance directly in pixel space at 2048 x 2048 resolution via Log-Gamma flow matching. This preserves fine high-frequency textures that latent diffusion baselines blur, while enabling physically grounded re-exposure across wide dynamic ranges.
Critical weaknesses

While the core contributions of this paper are compelling, I highlight the following relatively minor limitations:

  1. Ground Shadows and Temporal Consistency: As acknowledged in Appendix A and visible in the supplementary video, frame-by-frame relighting under rotating sunlight exhibits noticeable shadow popping and temporal discontinuities. This stems from relying entirely on base model priors without ground-plane or multi-frame supervision during synthetic fine-tuning.
  2. Approximations in Log-Gamma Linearity: As discussed in Appendices H.3 and I, operating in non-linear Log-Gamma coordinates distorts Gaussian noise and induces component gradient bias. Consequently, LC relies on empirical value clipping (-1 < x_t < 3) and heuristic time gating (t > 0.3), functioning in practice as a soft regularizer rather than an exact physical constraint.
  3. Undocumented Positional Encodings Across Aspect Ratios: While attention masking is detailed, the manuscript omits how rotary position embeddings (RoPE) are parameterized across different types of token patches. Clarifying this is necessary for reproducibility.
Questions for authors

Two questions related to MIIW:

  1. Specular and Metallic Materials: Other than the MIIW data, how well does the synthetic-trained model generalize to highly specular, mirror-like materials (e.g., chrome spheres or polished metals) under various illuminations?
  2. Zero-Shot MIIW Performance: How does the pure synthetic-trained model perform on MIIW without target-domain adaptation, and could zero-shot metrics be reported alongside the adapted checkpoint for direct comparison with zero-shot baselines?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 2bQt (1/2): W1–W3

We thank the reviewer for the careful review and the constructive questions. We are glad that the reviewer finds the core contributions compelling, and we answer each point below.

W1. Ground Shadows and Temporal Consistency.

… frame-by-frame relighting under rotating sunlight exhibits noticeable shadow popping and temporal discontinuities. This stems from relying entirely on base model priors without ground-plane or multi-frame supervision during synthetic fine-tuning.

Response. We agree with this analysis. Our synthetic fine-tuning data are single objects rendered on black backgrounds, without a ground plane, and our model is an image model: it uses no information from adjacent frames at all, either as input during inference or as supervision during training. As discussed in our limitations (App. A), a source image and an environment map admit many possible relightings, so neighboring frames can end up with noticeably different solutions, especially for sharp shadows and specular highlights. This is a practical limitation for video applications. Incorporating inter-frame information and spatio-temporal attention during training and inference should largely alleviate it, and this is a practical direction for future work. As an intuition, which is not validated: since the task admits many plausible solutions, some frame-to-frame variation may reflect this diversity.

W2. Approximations in Log-Gamma Linearity.

… operating in non-linear Log-Gamma coordinates distorts Gaussian noise and induces component gradient bias. Consequently, LC relies on empirical value clipping (-1 < x_t < 3) and heuristic time gating (t > 0.3), functioning in practice as a soft regularizer rather than an exact physical constraint.

Response. In Log-Gamma coordinates, LC acts as a soft regularizer rather than an exact constraint. We consider this a major limitation of our current method, and addressing it is an important direction for our future work. At the same time, in linear HDR, where LC holds exactly, the model trained with LC has a measured error of only 3.7% (Table 2), which we see as evidence of the constraint's potential going forward.

W3. Undocumented Positional Encodings Across Aspect Ratios.

While attention masking is detailed, the manuscript omits how rotary position embeddings (RoPE) are parameterized across different types of token patches. Clarifying this is necessary for reproducibility.

Response. We agree with this and will clarify this. We will explicitly describe how RoPE is parameterized for each type of token patch in the final version and will release the training and inference code, the dataset generation code and also the dataset. We retain the pretrained HiDream-O1-Image model’s three-axis RoPE unchanged. Each token has position indices (t, h, w); each attention head uses 64 rotary frequency channels, allocated to these axes in an interleaved pattern with 24, 20, and 20 channels, respectively, and base θ = 5×10⁶. Text tokens, including the time token, use the same running 1-D index on all three axes. For an image grid starting at offset o, the token at row r and column c receives (t, h, w) = (o, o + r, o + c). These are integer grid coordinates without normalization or interpolation, so different aspect ratios simply produce different row and column ranges.

We use nine grids: four visual-encoder token grids for the source image S and lighting representations R1, R2, R3, and five pixel-space grids for the HDR target and the four conditioning inputs. The pixel-space grids use 32×32-pixel patches. The visual-encoder spans continue the running text index. At our 2048×2048 output resolution, the HDR target forms a 64×64 grid anchored at position offset 4096. The raw-patch grids of S, R1, R2, R3 follow, each beginning one index after the largest index used by the preceding grid.

AuthorsOpenReview ↗

Response to Reviewer 2bQt (2/2): Q1–Q2

Q1. Specular and Metallic Materials.

Other than the MIIW data, how well does the synthetic-trained model generalize to highly specular, mirror-like materials (e.g., chrome spheres or polished metals) under various illuminations?

Response. We tested our synthetic-only model on mirror-like and metallic materials without MIIW data, and show all results at https://anonymous.4open.science/w/pixelhdr-rebuttal-3FFD/specular/index.html. The test set has 8 synthetic scenes rendered at 2048² with physical metal reflectance (a chrome sphere, a faceted chrome gem, silver, gold, copper, brushed aluminium, steel, and a row of four spheres from mirror to diffuse), each under 19 lightings: 16 Poly Haven environment maps that are not in the training set and 3 single-sun maps. Reflections generally follow the new environment. On the four spheres, the reflection changes to the new environment under all 19 lightings and blurs with increasing roughness. Outdoors, metal objects reflect the sky in their upper half and the ground in their lower half. We also show real captures with a real mirror sphere and ground truth (BounceBRDF).

Q2. Zero-Shot MIIW Performance.

How does the pure synthetic-trained model perform on MIIW without target-domain adaptation, and could zero-shot metrics be reported alongside the adapted checkpoint for direct comparison with zero-shot baselines?

Response. We report the zero-shot result in the table below, next to the adapted checkpoint and the baselines. Our model is trained on full-sphere HDR environment maps, which capture light from every direction (360°). MIIW's lighting comes from a photo of a chrome ball, which reliably captures only the hemisphere on the camera's side (180°). In the zero-shot setting, the model takes this hemisphere as if it were a full sphere, without reliable lighting for the other half, so the result is bound to be off. A short animation and real examples of both inputs are at https://anonymous.4open.science/w/pixelhdr-rebuttal-3FFD/miiw-lighting.html. When we add MIIW to training, as only about 5% of the training samples, the adapted checkpoint is best on all three metrics, which shows that our architecture can also handle MIIW's lighting format. We therefore report both results; UniRelight likewise trains on MIIW (14.3% of its second-stage samples).

MethodPSNRSSIMLPIPS
Neural Gaffer16.730.6680.465
DiffusionRenderer-SVD15.780.6300.396
Light-X14.690.6300.518
IC-Light14.260.5730.368
DiLightNet12.970.4520.465
Ours, zero-shot13.320.6420.353
UniRelight (trained with MIIW)19.490.7640.311
Ours, adapted to MIIW21.370.8040.220

MIIW protocol of App. C: 375 pairs, fixed-exposure gamma-2.2 display, no exposure fitting.

Thank you for the authors' detailed responses to my questions and for addressing my concerns. The additional shiny objects are very impressive. I think it is totally fine to show Ours, zero-shot results in the paper just as a useful reference data point. I will keep my positive rating.

P.S. to authors and reviewers: although my review is flagged for "AI use", I did carefully check, with AI assistance, the paper details and equations, especially the LC part. I found it quite inspiring. That's why I'd like to give it a clear acceptance.

Reviewer_xqCUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a native HDR relighting method based on a pixel-space flow model. It preserves the pretrained model’s conditioning pathways and introduces a light-transport linearity constraint to encourage physically consistent relighting, achieving significant improvements over existing methods.

Critical strengths

1.The proposed Light-Transport Linearity Constraint (LC) is the most novel contribution of this paper. It has a clear and well-motivated formulation, and the experiments demonstrate its effectiveness.

2.The paper is clearly written, with detailed appendices and relatively comprehensive experiments.

Critical weaknesses

1.The proposed Light-Transport Linearity Constraint (LC) is an interesting contribution, but its advantages over simpler alternatives are not fully demonstrated. While Appendix E.1.5 compares LC with direct data augmentation using summed HDR targets, this comparison is limited to a simplified setting. It would be helpful to validate this advantage in the full model and provide ablations on the proposed noise and time rescaling to better justify the theoretical formulation.

2.The paper claims that preserving the pretrained conditioning pathways improves generalization. However, the ablations only show that these pathways are useful, without demonstrating their advantages over adding new conditioning modules.Since the compared methods also differ in their backbones and training configurations, it is difficult to isolate the contribution of this design choice. A controlled comparison using the same backbone with alternative conditioning mechanisms would help support this claim.

3.Although native HDR prediction is one of the main contributions, the main quantitative relighting evaluations are conducted in tone-mapped LDR space using PSNR, SSIM, and LPIPS. These metrics may not fully reflect the accuracy of the predicted HDR radiance, particularly in high-intensity regions and specular highlights. Since the paper emphasizes physically meaningful HDR relighting, it would be helpful to include quantitative evaluations directly in linear HDR space on datasets with HDR ground truth.

4.The proposed method operates directly in pixel space at 2048 × 2048 resolution, which may introduce substantial computational overhead. Could the authors report the inference time and peak GPU memory usage, and compare them with latent-space relighting methods? This would help clarify the practical efficiency and computational trade-offs of the proposed approach.

Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Response to Reviewer xqCU (1/2): W1–W2

We thank the reviewer for the constructive comments. The requested comparisons and evaluations helped us to strengthen the paper, and we answer each point below.

W1.

The proposed Light-Transport Linearity Constraint (LC) is an interesting contribution, but its advantages over simpler alternatives are not fully demonstrated. While Appendix E.1.5 compares LC with direct data augmentation using summed HDR targets, this comparison is limited to a simplified setting. It would be helpful to validate this advantage in the full model and provide ablations on the proposed noise and time rescaling to better justify the theoretical formulation.

Response. We agree that a full-model comparison between LC and direct data augmentation (DA) is needed, and the submission does not contain one: as stated in Appendix E.1.5, that single-scene study does not establish "a general advantage over DA". Our compute is limited, and we gave priority to the controlled conditioning-pathway comparison requested in the reviewer's second point. We will do our best to add the full-model comparison to the revised version and the final version.

We would also like to clarify that DA and LC are not in conflict and can be used together. Informally, DA imposes additivity at the sample level: it adds the exact sum H1+H2 as a new training sample for the unchanged flow-matching loss. LC imposes additivity at the level of the velocity field: it adds no new sample, and instead trains the model's velocity under E1+E2 to agree with the composition of its own velocities under E1 and E2 (Section 3.3, Eq. 2). Appendix E.1.5 discusses the difference and includes a DA+LC configuration (Table 9).

On the "noise and time rescaling": an interactive illustration is at https://anonymous.4open.science/w/pixelhdr-rebuttal-3FFD/lc-rescaling.html. The submission has no experiment that isolates this step. The rescaling is derived, not tuned (Appendix H.1, Eqs. 24–25); it is exact in linear HDR and approximate in the Log-Gamma coordinates of the full model (Appendix H.3).

W2.

The paper claims that preserving the pretrained conditioning pathways improves generalization. However, the ablations only show that these pathways are useful, without demonstrating their advantages over adding new conditioning modules.Since the compared methods also differ in their backbones and training configurations, it is difficult to isolate the contribution of this design choice. A controlled comparison using the same backbone with alternative conditioning mechanisms would help support this claim.

Response. Please see our common response "Same-backbone comparison of lighting conditioning", which answers this point together with Reviewer W7g2 (W1).

AuthorsOpenReview ↗

Response to Reviewer xqCU (2/2): W3–W4

W3.

… the main quantitative relighting evaluations are conducted in tone-mapped LDR space using PSNR, SSIM, and LPIPS. These metrics may not fully reflect the accuracy of the predicted HDR radiance, particularly in high-intensity regions and specular highlights. … it would be helpful to include quantitative evaluations directly in linear HDR space on datasets with HDR ground truth.

Response. The main comparison uses LDR metrics because all baselines output only LDR, so the comparison with them has to be made in LDR. We agree that HDR accuracy should also be measured, and we now report metrics in linear HDR space for our model on three datasets with HDR ground truth: OLATverse, our synthetic Internal170, and BounceBRDF (Iser et al., 2026), a real HDR capture (4 scenes, 51 relighting pairs). Because an LDR input does not define absolute scale, each prediction is first aligned to the ground truth with one scalar gain. We report PU21-PSNR (Mantiuk & Azimi, 2021) and log2 RMSE in stops over all evaluated pixels. Because the reviewer specifically mentions high-intensity regions and specular highlights, we also compute both metrics on only the brightest 5% of pixels in each ground-truth image (table below). Baselines are omitted because they output only LDR, so their highlights are clipped and cannot be compared in HDR. We will add this table to the appendix.

BenchmarkAll pixels: PU21-PSNR / log2 RMSEBrightest 5%: PU21-PSNR / log2 RMSE
OLATverse (340 pairs)29.11 / 0.6728.23 / 0.70
Internal170 (170 pairs)27.82 / 0.9426.57 / 1.07
BounceBRDF (51 pairs)32.08 / 0.5125.37 / 1.09

PU21-PSNR in dB (higher is better); log2 RMSE in stops (lower is better). Each prediction is aligned to the ground truth with one scalar gain.

W4.

The proposed method operates directly in pixel space at 2048 × 2048 resolution, which may introduce substantial computational overhead. Could the authors report the inference time and peak GPU memory usage, and compare them with latent-space relighting methods? …

Response. We report inference time and peak GPU memory in the table below, compared with the image-based latent-space baselines on the same GPU.

MethodOutputH100 s / imageH100 s / megapixelPeak memory (GiB)
Ours2048²29.27.018.4
DiLightNet512²17.3666.0
IC-Light768²1.512.563.9
Neural Gaffer256²1.5123.02.6

Inference time and peak GPU memory on one H100 (batch 1, model loading excluded). s / megapixel is the time per image divided by the output megapixels.

Thank you for the detailed responses and additional evaluations. The HDR evaluation and runtime measurements are helpful. While some concerns remain about the relative effectiveness of DA and LC in the full model and the practical effect of rescaling, I maintain my rating of Weak Accept.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common response: same-backbone comparison of lighting conditioning (Reviewers W7g2 and xqCU)

Same-backbone comparison of lighting conditioning (Reviewer W7g2, W1; Reviewer xqCU, W2).

Reviewer W7g2 (W1): … The ablations remove inputs but do not test the stated alternatives, such as channel concatenation on the same backbone. The zero-shot base model already relights plausibly (Fig. 2). The causal claims should be tempered.

Reviewer xqCU (W2): … the ablations only show that these pathways are useful, without demonstrating their advantages over adding new conditioning modules. … A controlled comparison using the same backbone with alternative conditioning mechanisms would help support this claim.

Response. We agree that this comparison is important and have included it in our revision. The comparison supports our claim: keeping the pretrained pathways and conforming the lighting to them, rather than replacing them or adding new modules, preserves generalization. On the in-distribution benchmark below, all variants except channel concatenation give reasonable results. The clear differences appear on real photographs, which are far from the fine-tuning data. We show and discuss them image by image at https://anonymous.4open.science/w/pixelhdr-rebuttal-3FFD/conditioning/index.html. We encourage the reviewers to look at this page: we believe it is the most convincing evidence for keeping the pretrained pathways. We will add these results and the discussion to the appendix in the final version.

In developing the project, we first followed previous work and added a new pathway for lighting, and then realized that this significantly limits generalization, motivating our design. In the controlled comparison, all variants use the same backbone (HiDream-O1-Image, 8B), the same fine-tuning data and the same 2048² resolution, with similar training compute (about 400 GPU-hours each). They differ only in whether the target lighting keeps the two pretrained image-conditioning pathways of the base model. For each variant, we describe the design and the main observation on real photographs:

  • Ours (pretrained pathways kept). The lighting is given as three lighting maps, represented as images. They pass through the two pathways exactly as an input image does in the base model: Visual tokens from the frozen visual encoder, and raw pixel patches.
  • New encoder (one pathway changed). The frozen visual encoder that produces the Visual tokens of the lighting is replaced by another pretrained encoder, the Cosmos tokenizer. On real photographs: Small but intense lights, such as a green fluorescent tube or an orange lamp, have little effect on the result.
  • Channel concatenation (pretrained pathways not used). The alternative named by Reviewer W7g2, following IC-Light and Neural Gaffer. The lighting is encoded by a VAE and concatenated to the image input as extra channels. On real photographs: The result barely responds to the lighting. It is much worse even on the in-distribution benchmark, possibly because this design was developed for latent diffusion models, whose architecture differs from our pixel-space backbone.
  • Dedicated lighting encoder (pretrained pathways not used). Similar to our early experiments. The lighting enters through the environment encoder of DiffusionRenderer and new cross-attention layers. On real photographs: The result follows only the overall color of the environment, and it often does not respond correctly when the light is rotated.
  • Ours + dedicated lighting encoder (pretrained pathways kept, new module added). The lighting passes through both pretrained pathways as in Ours, and the dedicated lighting encoder with its cross-attention layers is added in parallel. On real photographs: The results are worse than with our pathways alone: the response to intense lights becomes weaker, and the environment leaks into the background. This is why we do not add new pathways.

On a scene far out of distribution, such as a forest path, only the pretrained pathways give a plausible result.

Lighting inputPSNRSSIMLPIPS
Ours (pretrained pathways)22.090.7770.166
New encoder19.610.7270.189
Channel concatenation12.830.5400.311
Dedicated lighting encoder20.230.7270.189
Ours + dedicated lighting encoder21.700.7570.172

Internal synthetic benchmark (170 held-out pairs), same protocol as our ablations.

Regarding Fig. 2: as stated in Sec. 1, the base model without fine-tuning "remains very weak at accurately rendering object appearance under the target illumination, in particular plausible shadows and colors, and it leads to substantial identity drift". Fig. 2 shows this starting point: its results keep some properties of the light, but they are not accurate relighting.

We will revise the wording of our causal claims in the abstract and introduction so that they refer to this controlled comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.