acceptodds
Under review as a conference paper at ICLR 2027

LumaGuide: Distribution Shaping for Training-Free HDR Generation in Diffusion Models

Abstract

Pretrained diffusion models generate realistic images, but their outputs remain constrained by the statistical biases of their training data, limiting their ability to produce high dynamic range (HDR) content. In this work, we introduce LumaGuide, a training-free framework for distribution shaping in diffusion models. Instead of modifying model parameters, LumaGuide steers the sampling process to match target feature distributions via differentiable energy-based guidance. We instantiate this framework for HDR generation by controlling luminance distributions in perceptually uniform PQ space. Our results show that aligning luminance histograms can induce HDR-consistent behavior, including coherent highlights and preserved shadow detail, while maintaining semantic fidelity. Beyond HDR, LumaGuide enables flexible specification of target distributions through data-driven presets, reference images, or text-driven predictors, and extends naturally to video generation with temporal consistency constraints. More broadly, our work demonstrates that controllable generation can be achieved by directly shaping output distributions at sampling time, without retraining diffusion models.

Primary area:
generative models
Keywords:
HDR image generation, image generation, generative models, diffusion models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6y47OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LumaGuide, a framework for HDR generation using frozen pretrained diffusion models. It guides sampling by aligning PQ-space luminance histograms with target distributions through a differentiable Wasserstein-1 objective, and extends the approach to video with temporal luminance regularization. The idea is simple yet effective.

Critical strengths

LumaGuide achieves competitive HDR generation quality without fine-tuning the pretrained diffusion backbone or VAE, offering a practical way to reuse existing models.

Critical weaknesses

The idea is simple yet effective, and the experimental evaluation is generally sound. I appreciate the authors’ efforts. Addressing the following points would help clarify the method and strengthen the evaluation.

  1. Histogram alignment alone does not establish plausible highlight placement or local detail. Please provide targeted highlight/shadow analysis and a post-generation histogram-matching baseline using the same backbone and target distribution.

  2. Please clarify the conversions between decoder RGB, linear luminance, and PQ, and explain how baseline outputs are calibrated to absolute luminance in nits.

  3. The MLP’s KL scores are difficult to interpret without a simple baseline, such as the mean training histogram. Predicted-versus-ground-truth histogram plots, percentile errors, and evaluation on external HDR datasets would strengthen validation.

  4. A distribution-level metric, such as FID computed using consistent tone mapping and sufficient samples, would complement the luminance statistics and subjective assessment. Please also specify the display hardware, luminance capabilities, color management, and viewing conditions used in the user study.

  5. TLC and fixed-pixel temporal variance can penalize legitimate motion and illumination changes. Please provide playable HDR videos and motion-aware evaluation. Video comparisons currently cover backbone ablations only; an additional baseline could combine the same SDR video generator with the HDR conversion method in [2].

  6. Please summarize the key limitations discussed in Appendix A.11 in the main text.

  7. As an optional addition, a downstream application such as image-based lighting could better demonstrate the benefits of HDR generation, which are not always evident in the current comparisons with vanilla FLUX.

The related-work discussion should also include:

[1] Linear Image Generation by Synthesizing Exposure Brackets. CVPR 2026.
[2] Generating HDR Video from SDR Video. SIGGRAPH Asia 2026.
[3] GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild. ICCV 2023.

Questions for authors

Please refer to the weaknesses.

5 comments

We thank the reviewer for recognizing the strengths of our approach and for the detailed, constructive feedback. We address the reviewer's comments below. All changes in the revised manuscript are highlighted in blue for ease of reference.

Response to W1: Highlight/Shadow Analysis and Post-hoc Histogram Matching

We thank the reviewer for suggesting targeted highlight/shadow analysis and a post-hoc histogram-matching baseline.

To compare against post-hoc histogram matching, we introduce a nonlinear CDF-based baseline using the same target histograms, prompts, and random seeds as LumaGuide. Although post-hoc matching achieves better histogram alignment (uW1: 0.33 vs. 0.58), LumaGuide achieves higher Q-quality (0.568 vs. 0.521) and Q-alignment (0.814 vs. 0.797). These results suggest that more accurate histogram matching alone does not necessarily translate into higher measured perceptual quality, highlighting the benefit of incorporating luminance guidance during sampling rather than applying post-hoc adjustments.

We further examine local highlight/shadow structure through a targeted region-level analysis using shared spatial masks derived from the vanilla outputs. Compared with post-hoc matching, LumaGuide achieves higher highlight local contrast (0.144 vs. 0.059), with improvements on 93% of the evaluated examples under this metric. It also achieves higher highlight gradient magnitude (0.027 vs. 0.017) and reduces extreme highlight clipping relative to vanilla generation (11.0% to 1.0%). In shadow regions, LumaGuide achieves higher local contrast (0.111) than both vanilla generation (0.039) and post-hoc matching (0.057). These findings provide targeted evidence of stronger local luminance variations beyond global histogram alignment. We additionally include multi-exposure comparisons at EV−4, EV0, and EV+4 to visualize differences in local luminance rendering.

In the revised manuscript, we have added the post-hoc histogram-matching baseline and its quantitative comparison in Section 4.3, with additional region-level analysis and multi-exposure visualizations in the Appendix A.4.

Response to W2: HDR Signal Representation and Luminance Calibration

We thank the reviewer for highlighting the need to clarify HDR signal representation and luminance calibration. Explicitly specifying the conversion between decoder outputs, PQ-coded signals, and absolute luminance is important for reproducibility.

Regarding the conversion pipeline, we first normalize the VAE decoder outputs from [−1,1][-1,1] to [0,1][0,1] and interpret them directly as PQ-coded RGB values. We then compute a PQ-domain luma proxy using the BT.2020 coefficients:

YPQ′=0.2627RPQ′+0.6780GPQ′+0.0593BPQ′.Y'_{\mathrm{PQ}} = 0.2627R'_{\mathrm{PQ}} + 0.6780G'_{\mathrm{PQ}} + 0.0593B'_{\mathrm{PQ}}.

This proxy is used to construct the differentiable histogram for Wasserstein-1 guidance. Our implementation does not first convert the decoder outputs to linear-light RGB or apply an additional PQ transfer function. We have therefore corrected the original notation PQ⁡(Lum⁡(x))\operatorname{PQ}(\operatorname{Lum}(x)), which did not accurately describe the implemented pipeline.

For luminance evaluation, vanilla Flux and LumaGuide follow the same PQ interpretation and evaluation conventions. The PQ-coded RGB channels are converted into linear-light values using the ST.2084 EOTF, from which luminance statistics are computed in nits. No additional luminance normalization is applied to the generated LumaGuide outputs. This procedure defines an absolute-luminance interpretation under the PQ convention, rather than an independent physical calibration of the pretrained decoder.

In the revised manuscript, we have clarified the signal definitions and conversion pipeline in Section 3.1, Algorithm 1, and the evaluation protocol, distinguishing the PQ-domain guidance proxy from linear-light luminance measurements.

Response to W3: Text-to-Histogram Regressor Validation

We thank the reviewer for suggesting a more comprehensive evaluation of the text-to-histogram regressor. We have introduced a mean training histogram baseline and additional distributional and percentile-based metrics, and further evaluated the regressor on an external HDR dataset.

On the held-out test set, comprising 1,123 frames from source-video-disjoint content, the MLP outperforms the mean training histogram baseline, reducing KL divergence from 0.459 to 0.346 (approximately 25%), PQ-domain W1 from 0.098 to 0.068 (approximately 31%), and P50 error from 0.112 to 0.077 (approximately 31%). P99 error also decreases from 0.047 to 0.044. These results demonstrate that text conditioning provides predictive information beyond the dataset-average luminance distribution.

To assess cross-dataset generalization, we additionally evaluate the regressor without further training on an independent dataset, LIVE Paired Comparison HDR vs. SDR Database, of professionally graded HDR10 content (48 videos, 192 frames). The MLP achieves lower median KL divergence (0.355 vs. 0.407) and PQ-domain W1 (0.098 vs. 0.110) than the mean histogram baseline, indicating that text conditioning retains predictive value across domains. While differences in luminance distributions and creative grading practices may affect performance, LumaGuide does not depend on the text-to-histogram regressor: target histograms can also be specified through predefined presets or HDR reference images.

In the revised manuscript, we have expanded Appendix A.7 with the mean training histogram baseline and additional distributional and percentile-based metrics, and clarified the potential domain limitations of training on UGC HDR content.

Response to W4: Distribution-Level Evaluation and Subjective Study Setup

We thank the reviewer for highlighting the importance of distribution-level evaluation beyond target-histogram alignment. While FID and KID measure overall image-distribution similarity in Inception feature space, they are influenced by semantic content and may be less sensitive to HDR-specific luminance characteristics. Since LumaGuide primarily aims to improve luminance distributions rather than alter image semantics, we additionally evaluate the Wasserstein-1 distance between the pooled PQ-luminance distributions of generated images and real HDR references from professionally graded HDR10 content (48 videos, 192 frames), LIVE Paired Comparison HDR vs. SDR Database. LumaGuide achieves a lower distance (0.139) than vanilla Flux (0.250) and X2HDR (0.179), indicating closer agreement with real HDR luminance statistics. We believe this evaluation more directly addresses the reviewer's concern about distribution-level HDR fidelity, while complementing our existing perceptual and subjective assessments.

Regarding the subjective study setup, the study was conducted indoors using the built-in Liquid Retina XDR HDR display of an Apple MacBook Pro, with nominal HDR brightness specifications of 1000 nits sustained full-screen and 1600 nits peak. All stimuli were encoded as 10-bit AVIF using BT.2020 primaries, the ST.2084 PQ transfer function, and the BT.2020 non-constant-luminance matrix (CICP 9/16/9). HDR metadata was verified for each stimulus. Images were presented through Chrome in full-screen mode using a randomized 2×2 ranking interface, with a 600 ms neutral-gray screen between trials and an optional rest after the 20th trial.

In the revised manuscript, we have expanded the Appendix A.8 to specify the HDR display, stimulus color encoding and metadata, and viewing conditions.

Response to W5: Temporal Luminance Consistency and Video Evaluation

We thank the reviewer for the suggestions regarding video evaluation and temporal luminance consistency. Playable HDR videos have been included in the supplementary material.

Our original evaluation in Appendix A.9 includes a controlled ablation using the same CogVideoX backbone. The results show that TLC reduces highlight-region temporal luminance variance from 0.025 to 0.016 relative to W1 guidance alone. Since this evaluation measures luminance variations at fixed image coordinates, it does not explicitly distinguish flickering from changes induced by object or camera motion. We therefore extend the evaluation to motion-compensated temporal luminance measurements to further examine TLC's effectiveness under motion. Specifically, we evaluate optical-flow-aligned first- and second-order luminance differences across the same generated videos, considering both threshold-based highlight regions and the brightest 2% of pixels. W1+TLC consistently achieves lower motion-compensated flicker errors than W1 guidance alone across all four metrics, with 3–7% reductions in geometric-mean error ratios. To examine whether TLC suppresses legitimate temporal changes, we additionally assess the preservation of motion and luminance dynamics relative to W1 guidance alone. W1+TLC also achieves a motion-preservation score of 0.99 relative to W1 guidance, suggesting that the reduction in temporal luminance fluctuations is accompanied by minimal changes to the generated motion. Together, these results indicate that TLC can further reduce temporal luminance fluctuations beyond fixed-coordinate evaluation while largely preserving the generated motion and luminance evolution.

For the suggested SDR-to-HDR video conversion baseline, we agree that combining the same pretrained SDR video generator with a separate HDR conversion method is a relevant alternative. However, this represents a two-stage generation-and-conversion pipeline, whereas LumaGuide integrates luminance control directly into diffusion sampling without a separate conversion model. Such a comparison would assess the relative effectiveness of two different HDR synthesis strategies, rather than specifically isolating the contribution of sampling-time guidance. Our current backbone-controlled evaluation therefore focuses on assessing the contributions of sampling-time guidance and TLC, while avoiding the additional modeling choices introduced by a separate HDR conversion stage.

In the revised manuscript, we have clarified the scope of fixed-coordinate temporal regularization in Section 3.4.

Response to W6: Limitations in the Main Text

We thank the reviewer for this suggestion.

In the revised manuscript, we have added a concise discussion of limitations to the Conclusion, with detailed analyses and visual examples retained in Appendix A.11.

Response to W7: Downstream Applications

We thank the reviewer for suggesting IBL as a potential downstream application. Our work primarily focuses on training-free HDR luminance distribution control, which we evaluate through luminance-distribution metrics, multi-exposure comparisons, and subjective assessments. While IBL is a promising application, it typically requires spatially accurate HDR environment maps, whereas our current work focuses on general-scene HDR image generation rather than environment-map synthesis. We consider exploring such downstream applications an interesting direction for future research.

Response to Related Work Suggestions

We thank the reviewer for suggesting these relevant studies. In the revised manuscript, we have expanded the Related Work section to discuss the suggested papers.

Reviewer_zYZHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

LumaGuide introduces sampling-time luminance guidance for training-free HDR generation using frozen pretrained diffusion models. At each sampling step, it decodes the Tweedie clean-image estimate, computes a differentiable luminance histogram in the Perceptual Quantizer (PQ) domain, and minimizes its 1D Wasserstein-1 (W1W_1) distance to a target distribution obtained from presets, reference images, or a learned text-to-histogram regressor. The method is evaluated across Flux.1, SD3, and SDXL backbones and extended to video generation via a temporal luminance consistency (TLC) penalty, reporting improved histogram alignment and favorable perceptual scores over existing baselines.

Critical strengths
  1. Flexible, plug-and-play luminance control without backbone fine-tuning. Framing 1D histogram guidance around the closed-form W1W_1 distance over cumulative distribution functions provides stable cross-bin gradients during sampling. Supporting preset, reference, and text-predicted target histograms provides convenient inference-time control that carries over to Flux.1, SD3, and SDXL without retraining the generative backbone.

  2. Promising perceptual quality under moderate guidance. The reported Q-quality metrics and subjective user preferences indicate that shaping global luminance statistics during sampling can produce visually appealing tonal adjustments within practical guidance ranges.

Critical weaknesses
  1. The physical HDR formulation is self-contradictory, 1D marginal histogram matching cannot semantically allocate HDR highlights, and the theoretical guarantees do not hold for the implemented latent-space algorithm.

    • Contradictory physical signal definitions mistake SDR darkening for HDR generation: Pretrained VAE decoders in Flux.1, SD3, and SDXL output non-linear display-referred sRGB values in [0,1][0, 1], not calibrated linear luminance or PQ-encoded signals. Algorithm 1 and Section 3.1 define the feature map as PQ(Lum(x0))PQ(Lum(x_0)) using BT.2020 linear luminance weights directly on uncalibrated decoder outputs, whereas Section 4.1 and Table 1 treat vanilla Flux outputs as already PQ-encoded and apply PQ−1PQ^{-1} to compute nits. Because ordinary bright sRGB pixels ([0.8,1.0][0.8, 1.0]) map to thousands of nits under PQ−1PQ^{-1}, vanilla Flux outputs appear artificially "overexposed" (nits99=4867nits_{99} = 4867). Under LumaGuide, increasing guidance to s0=2000s_0 = 2000 simply compresses pixel values downward, reducing nits99nits_{99} from 4867 to 1694 (Table 1) and shrinking dynamic range DRstopsDR_{stops} from 16.37 to 14.99 (Table 3). Rather than eliciting true HDR highlights from an SDR prior, the method merely darkens standard SDR outputs to compensate for an invalid PQ interpretation.

    • Spatial blindness of 1D histogram matching and invalid spatial-preservation claims: True HDR generation requires separating emissive sources (e.g., sun, fire) from bright reflective surfaces (e.g., white walls, paper). Because LumaGuide optimizes a 1D global marginal distribution, equal-luminance pixels receive identical image-space scalar gradients regardless of semantics (Eq. 7). The objective itself is equivalent to a global monotonic 1D lookup table (histogram specification) and cannot selectively push light sources to high luminance while keeping reflective surfaces moderate. Furthermore, Proposition 1's pixel-space permutation invariance does not preserve spatial or geometric structure during optimization. Actual updates backpropagate through the VAE decoder's spatially coupled Jacobian (dDec/dzt)T(d Dec / d z_t)^T in latent space, where pixel-wise gradient properties no longer prevent spatial interference, as directly evidenced by the banding and unnatural textures reported at higher guidance scales.

    • Disconnect between Theorem 1 and the actual sampling algorithm: Theorem 1 assumes LL-smoothness of z→E(x0(z))z \rightarrow E(x_0(z)) and attributes potential energy increases solely to velocity-prediction error. However, this bound ignores base-flow drift across timesteps, the inherent nonsmoothness of the 1D W1W1 objective (L1L1 norm of CDF differences at zero-crossings), and the omitted Jacobian factor (I−tJv)T(I - t J_v)^T when detaching the velocity network during decoder-only backpropagation. Moreover, the theorem's noise penalty scales with t2×dtt^2 \times dt, which implies reducing step sizes at large tt and directly contradicts the constant guidance schedule (s(t)=s0=2000s(t) = s_0 = 2000) claimed as optimal in Section 3.3.

  2. The experimental evaluation relies on circular target-compliance metrics, omits the direct post-hoc histogram matching baseline, obscures high-resolution computational overhead, and introduces a motion-penalizing video loss without main-text validation.

    • Circular distribution metrics and missing post-hoc 1D histogram matching baseline: Under the default setup, target histograms h⋆h^{\star} are generated by the authors' text regressor, and uW1uW1, p50distp50_{dist}, and p99distp99_{dist} measure distance to that exact same h⋆h^{\star}. Optimizing W1(h,h⋆)W1(h, h^{\star}) at sampling time and evaluating uW1(h,h⋆)uW1(h, h^{\star}) at test time only demonstrates optimization compliance, not physical HDR validity, especially since wider dynamic range does not imply better quality (in Table 4, dual-histogram weighting increases DR_stops from 14.99 to 16.84 while collapsing Q-quality from 0.568 to 0.325). More importantly, the paper only proves the trivial limitation of linear/affine brightness scaling (Theorem 2) while omitting the most direct training-free baseline: applying classical post-hoc non-linear 1D histogram matching directly to vanilla Flux outputs to match h⋆h^{\star}, which achieves near-zero W1W1 distance in milliseconds without 28 steps of VAE backpropagation.

    • Obscured computational cost and flawed, unvalidated video extension: LumaGuide requires full VAE decoding and backpropagation at every sampling step, necessitating gradient checkpointing and spatial tiling at 1024x1024. Yet Table 2 benchmarks only the runtime at 512x512 (7.8 s) and omits the vanilla Flux runtime and peak VRAM usage, obscuring the true overhead relative to the unguided backbone. For video, the temporal luminance consistency loss LTLC=∑∣∣(Yτ−Yτ−1)∗Ωτ∣∣22L_{TLC} = \sum ||(Y_{\tau} - Y_{\tau -1}) * \Omega_\tau||_2^2 (Eq. 8) penalizes pixel differences at identical spatial coordinates across adjacent frames without optical flow or motion compensation. Consequently, any moving highlight or camera pan is penalized as temporal flicker, encouraging dimming or motion smearing. Despite claiming zero-shot HDR video generation as a core contribution, the main text provides zero video figures, quantitative metrics, or TLC ablations.

Questions for authors
  1. Clarify the exact physical signal and gradient computation pipeline. Specify the exact conversion chain from raw VAE decoder outputs to the guidance histogram and exported HDR values (color space, transfer function, and nits scaling), reconciling PQ(Lum(x0))PQ(Lum(x_0)) in Algorithm 1 with the PQ−1PQ^{-1} evaluation of vanilla Flux in Section 4.1. Also, clarify whether gradients backpropagate through vθ(zt,t)v_\theta(z_t, t) or only through Dec(⋅)Dec(\cdot), and how Theorem 1 accounts for detached velocity gradients and base-flow drift.

  2. Add a post-hoc non-linear 1D histogram matching baseline. Apply standard 1D monotonic histogram specification directly to unguided Flux outputs using the exact same target histograms h⋆h^{\star}. Compare uW1uW1, Q-quality, Q-alignment, DR_stops, and multi-exposure visualizations (EV-4 / EV+0 / EV+4) to demonstrate what substantive structural benefit 28-step latent backpropagation provides over one-step post-hoc tone mapping.

  3. Report complete overhead benchmarks and main-text video/TLC quantitative results. Provide inference runtime and peak VRAM for both unguided Flux and LumaGuide at 512x512 and 1024x1024 resolutions. In addition, provide quantitative video comparisons and LTLCL_{TLC} ablations specifically on sequences containing moving light sources or camera motion.

8 comments

We thank the reviewer for recognizing the strengths of our approach and for the constructive feedback. All changes in the revised manuscript are highlighted in blue for ease of reference.

Response to W1(a) and Q1(a): HDR Representation and Signal Pipeline

Regarding HDR representation, LumaGuide builds on the observation from X2HDR that pretrained LDR VAEs can represent and reconstruct HDR signals in perceptually uniform domains such as PQ and PU21 without retraining (Appendix A.3). However, the ability to represent HDR signals does not imply that an SDR-pretrained diffusion model naturally generates HDR-calibrated outputs. While X2HDR addresses this gap by fine-tuning the diffusion backbone, LumaGuide instead explores whether HDR-like luminance distributions can be induced through sampling-time guidance while keeping both the backbone and VAE frozen. We interpret the decoded outputs under a specified PQ convention and evaluate their resulting HDR characteristics, rather than claiming that the pretrained model provides independently calibrated physical luminance estimates.

In our implementation, we first map the VAE decoder output from [−1,1][-1,1] to [0,1][0,1] and directly interpret it as PQ-coded RGB values. Guidance uses a PQ-domain luma proxy computed with BT.2020 coefficients:

0.2627R'_{\mathrm{PQ}} + 0.6780G'_{\mathrm{PQ}} + 0.0593B'_{\mathrm{PQ}}.$$ We then construct a 32-bin differentiable histogram and compute the Wasserstein-1 guidance objective, without applying an additional PQ transformation. For HDR export, we apply the ST.2084 EOTF channel-wise to convert PQ-coded RGB into linear-light RGB and save the results in Radiance RGBE (.hdr) format. The exported values are interpreted under BT.2020 primaries, although the RGBE files do not explicitly embed color-space metadata. We have corrected the inaccurate $\operatorname{PQ}(\operatorname{Lum}(x))$ notation in **Section 3.1** and **Algorithm 1** to distinguish the PQ-domain luma proxy from physical linear-light luminance. For gradient computation, the predicted velocity is detached, so guidance gradients propagate through the VAE decoder and differentiable histogram objective, but not through the diffusion backbone. We address the corresponding theoretical implications, including base-flow drift and the correction to **Theorem 1**, in W1c/Q1(b). As for the luminance statistics, LumaGuide aims to match a specified HDR luminance distribution rather than maximize peak luminance or dynamic range. We further address whether LumaGuide is equivalent to global tone mapping in W1b and W2a/Q2 through monotonic-mapping analysis and an additional post-hoc histogram-matching baseline. In the revised manuscript, we have corrected the PQ-domain feature definition in **Section 3.1** and **Algorithm 1**, and clarified the HDR export and luminance evaluation conventions in **Section 4.1**.

Response to W1b: Spatial Awareness and Semantic Highlight Allocation

We thank the reviewer for raising important questions about the spatial awareness of histogram-based guidance and the scope of our theoretical analysis.

We agree that a global 1D histogram contains no explicit spatial or semantic information and cannot distinguish emissive sources from reflective surfaces. LumaGuide does not rely on the histogram objective to determine highlight placement. Instead, luminance distribution control is achieved through histogram guidance, while spatial and semantic organization primarily comes from the pretrained diffusion prior. We therefore do not claim that histogram matching alone guarantees semantically correct highlight placement.

LumaGuide differs from a global monotonic luminance transformation. Its guidance operates on latent representations during sampling and interacts with subsequent diffusion dynamics. To examine this distinction, we conducted a monotonic-mapping analysis using 100 matched prompts and random seeds. We fitted an optimal global monotonic luminance mapping between vanilla Flux.1 outputs and the corresponding method outputs, and measured the normalized unexplained variance. The median residual was 0.158 for LumaGuide, compared with 0.016 for post-hoc histogram matching. These results indicate that LumaGuide's output changes cannot be adequately explained by a single global monotonic transformation of the vanilla output.

We also acknowledge the limitation of Proposition 1. The proposition establishes that the histogram objective is permutation-invariant and that its image-space gradients depend on luminance values rather than pixel locations. However, these properties do not guarantee spatial preservation during latent-space sampling, where guidance gradients propagate through the spatially coupled Jacobian of the VAE decoder. We have therefore revised the discussion in Section 2.3 to clarify that permutation invariance means the histogram objective does not explicitly prescribe spatial rearrangements, rather than preventing spatial changes induced by latent-space optimization. We have also revised the corresponding claim in Section 2 to avoid implying a theoretical guarantee of spatial preservation.

Response to W1c and Q1(b): Theoretical Validity

We thank the reviewer for carefully examining the consistency between Theorem 1 and the actual guidance implementation.

LumaGuide employs stop-gradient energy guidance. At each sampling step, the pretrained model's velocity is computed without gradient tracking and detached when constructing the clean latent estimate:

z^0=zt−t sg⁡[vθ(zt,t)].\hat z_0=z_t-t\,\operatorname{sg}[v_\theta(z_t,t)].

Guidance gradients propagate through the VAE decoder, PQ-domain luma feature, differentiable histogram, and Wasserstein-1 objective, but not through the diffusion backbone. Thus, the implementation computes a local surrogate energy gradient with the current velocity held fixed, rather than the full gradient involving the velocity Jacobian.

We have revised Theorem 1 as Local Energy Improvement under Stop-Gradient Guidance to account for base-flow drift and the nonsmoothness of the Wasserstein-1 objective, establishing a conditional local energy comparison consistent with the implemented guidance procedure.

Specifically, let vˉn=sg⁡[vθ(zn,tn)]\bar v_n=\operatorname{sg}[v_\theta(z_n,t_n)] and define the fixed-velocity surrogate energy

\qquad g_n=\nabla\Phi_n(z_n).$$ For the unguided update $z_b=z_n-h_n\bar v_n$ and guided update $z_g=z_b-\eta_n g_n$, where $h_n=t_n-t_{n+1}>0$ and $\eta_n=h_ns(t_n)>0$, suppose that $\Phi_n$ has an $L$-Lipschitz gradient on the relevant update region. Then, $$\begin{aligned} \Phi_n(z_g)-\Phi_n(z_b) \leq {}& -\eta_n\left(1-\frac{L\eta_n}{2}\right)\|g_n\|^2\\ &+\eta_n Lh_n\|\bar v_n\|\|g_n\|. \end{aligned}$$ This bound accounts for the interaction between the guidance correction and base-flow drift. When $g_n\neq 0$ and $$\left(1-\frac{L\eta_n}{2}\right)\lVert g_n\rVert > Lh_n\lVert\bar{v}_n\rVert$$ the guided update achieves a strictly lower surrogate energy than the unguided update. Importantly, this is a conditional, same-step comparison under a fixed-velocity surrogate, rather than a guarantee of monotonic energy descent along the full sampling trajectory. The smoothness assumption and guidance schedule also require clarification. A differentiable soft histogram does not imply global smoothness of the Wasserstein-1 objective because of the absolute CDF differences. The revised theorem therefore explicitly assumes local smoothness of the complete surrogate energy. Its sufficient condition neither establishes the optimality of a constant guidance schedule nor guarantees that the default guidance scale satisfies the condition at every timestep; the schedule is instead supported by empirical ablations. In the revised manuscript, we have clarified the stop-gradient computation and sampling update in **Section 2.2** and **Algorithm 1**, and revised **Theorem 1** and its proof in **Appendix A.2.6** to state the corresponding local guarantee and its assumptions. ### Response to W2a/Q2: Evaluation Metrics and Post-hoc Histogram Matching We thank the reviewer for suggesting a post-hoc histogram-matching baseline and for raising concerns about the interpretation of our evaluation metrics. To provide a direct comparison with post-hoc tone mapping, we apply nonlinear CDF-based histogram matching to vanilla Flux outputs using the same target histogram $(h^*)$, prompts, and random seeds as LumaGuide. On 100 prompts at 512×512 resolution, post-hoc matching achieves lower uW1 (0.33 vs. 0.58), indicating more accurate distribution alignment. However, LumaGuide achieves higher Q-quality (0.568 vs. 0.521), Q-alignment (0.814 vs. 0.797), and DRstops (14.99 vs. 13.98). The Q-quality advantage is also observed at 1024×1024 resolution. We further examine local luminance structure through targeted highlight/shadow region analysis. Compared with post-hoc matching, LumaGuide achieves higher local contrast (0.144 vs. 0.059) and gradient magnitude (0.027 vs. 0.017) in highlight regions, as well as higher local contrast (0.111 vs. 0.057) in shadow regions. These results provide additional evidence that sampling-time guidance can produce stronger local luminance variations than post-hoc histogram matching. We also agree that uW1, p50dist, and p99dist primarily measure distribution controllability rather than perceptual quality. Our evaluation therefore includes complementary Q-Eval metrics, subjective JOD assessments, and multi-exposure comparisons. The post-hoc baseline further illustrates that more accurate histogram alignment does not necessarily translate into better measured perceptual quality. In the revised manuscript, we have included the post-hoc histogram-matching comparison in **Section 4.3**, with additional quantitative analysis and multi-exposure visualizations in **Appendix A.4**.
Reviewer_sKjgOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LumaGuide, a training-free framework for controlling the output statistics of pretrained diffusion models at sampling time. Rather than modifying model weights, it steers the sampling trajectory so that a designated low-dimensional feature of the output (here, luminance) matches a prescribed target distribution. The authors instantiate it for text-to-HDR image and video generation, showing that shaping luminance histograms in perceptually uniform PQ space induces HDR-consistent rendering while preserving semantic and spatial fidelity.

Contributions:

  1. A general training-free distribution shaping framework via differentiable energy guidance at sampling time.

  2. LumaGuide for text-to-HDR generation, with luminance histogram alignment in PQ space, extended to zero-shot HDR video via TLC.

  3. Theoretical guarantees: controlled energy descent (Thm. 1), permutation-invariant constraints do not explicitly impose spatial rearrangement (Prop. 1), and global affine brightness adjustments cannot generally reproduce target HDR distributions (Thm. 2).

  4. Empirical validation across image and video backbones (Flux.1, SD3, SDXL, CogVideoX).

Validation:

Works as intended: Under identical seeds/prompts, LumaGuide redistributes luminance from clipped highlights into midtones while preserving highlight gradients and shadow detail, yielding histograms closer to the target HDR distribution than the baseline.

Guidance scale matters: Alignment improves up to a moderate scale, then degrades with banding and unnatural textures at very high scales; a default trade-off point is chosen.

Beats or matches baselines: Best distribution alignment, largest dynamic range, and highest subjective preference, with no fine-tuning; perceptual quality slightly below fine-tuned X2HDR, runtime comparable to it and far faster than BracketDiffusion.

Generalizes across backbones: Consistent control on Flux, SD3, and SDXL without modification.

Ablations confirm design choices: PQ space far better than linear; Wasserstein better than Euclidean and KL; perceptual-log weighting best balances alignment and quality.

Extends to video: Frame-wise shaping plus a temporal coherence term suppresses highlight flicker, giving training-free HDR video with preserved motion and structure.

Critical strengths
  1. The paper reframes controllable generation as output-distribution shaping, and shows the reframing is both principled and practically effective.

This is the consideration that most strongly drives acceptance. The method does not just add another HDR pipeline; it identifies a general lever—steering the sampling trajectory so a low-dimensional feature statistic matches a target distribution—and then demonstrates that this lever is sufficient to induce HDR behavior without touching model weights. This matters because it sidesteps the main cost of prior work (fine-tuning the backbone or adding multi-stage reconstruction), and the paper backs the reframing with theory that is appropriately scoped: energy descent under explicit assumptions, a permutation-invariance argument explaining why semantics are preserved, and a negative result showing global affine brightness adjustment cannot reproduce HDR distributions. The reframing is consequential because it makes the approach plug-and-play across backbones (Flux, SD3, SDXL) and naturally extensible to video, which the paper demonstrates rather than merely claims.

  1. The ablation design isolates why the method works, ruling out the trivial-explanation objection.

The most obvious skeptical reading of LumaGuide is that it just brightens or globally rescales the image. The paper preempts this directly: the linear-domain guidance control performs essentially like the unguided baseline, while PQ-space guidance is consistently effective, and within PQ space the Wasserstein metric beats Euclidean and KL. Together with the Theorem 2 negative result on affine adjustments and the histogram-weighting ablation, this builds a tight argument that the gains come from ordered transport of luminance mass in a perceptually uniform space, not from a shortcut transformation. This is consequential because it converts a plausible-sounding claim into a substantiated mechanism, and it is the kind of control that distinguishes a credible methods paper from a demo.

Critical weaknesses

Q-quality slightly below X2HDR. This is an honest trade-off, not a flaw: LumaGuide achieves it without any fine-tuning, while X2HDR requires LoRA tuning of the backbone. The paper reports this transparently and the training-free advantage is quantified via runtime. A method that trades a small perceptual-quality margin for zero training is a legitimate contribution, not a soundness failure.

Artifacts at very large guidance scales. This is a tuning limitation, acknowledged by the authors, with a default operating point selected on evidence. It does not invalidate the mechanism; it bounds the operating regime, which is normal for guidance-based methods.

These two problems are minor, I cannot identify a material weakness that would support rejection.

Questions for authors

The default configuration relies on the CLIP→MLP regressor described only in Appendix A.7. How much do the reported main-text results (Tables 1–5) depend on this regressor versus a preset or reference-based target? Specifically, if the regressor were replaced with a reference histogram, would the alignment and dynamic-range gains hold, and how sensitive are the results to regressor quality?

3 comments

We sincerely thank the reviewer for the positive assessment of our work and for the question regarding target histogram specification. All changes in the revised manuscript are highlighted in blue for ease of reference.

We would like to clarify that Tables 1–5 use the CLIP→MLP regressor as the default target specification to evaluate reference-free text-to-HDR generation. As described in Sections 3 and 4.5, LumaGuide's distribution-shaping mechanism is independent of the target specification method. Whether the target histogram h∗h^* is obtained from a preset, an HDR reference image, or text-driven prediction, the Wasserstein-1 guidance objective and sampling procedure remain unchanged. Replacing the regressor with a reference histogram removes text-prediction error, although the resulting alignment and dynamic-range gains still depend on the compatibility between the target distribution and the generated scene.

Regarding regressor quality, we additionally evaluated a mean training histogram baseline. On the video-disjoint test set, the regressor reduces KL divergence from 0.459 to 0.346 and PQ-domain W1 from 0.098 to 0.068, indicating that text conditioning captures information beyond the average training distribution. While regressor prediction errors can influence the resulting HDR appearance, they do not change the guidance mechanism for a given target histogram.

In the revised manuscript, we have clarified the regressor's optional role in Section 4.1 and expanded its evaluation in Appendix A.7.

Dear Reviewer,

We would like to kindly follow up on our previous responses and ask whether they have adequately addressed your concerns. We would be happy to provide any further clarification if needed. Thank you again for your time and thoughtful feedback.

Reviewer_sKjgOpenReview ↗

response to author rebuttal

I have no major concern to be addressed, I would maintain my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.