- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a method for generating HDR images by integrating prior information from large models with physical priors. Specifically, a frozen large model is employed to generate structural priors, which are then guided by physical priors. Notably, this approach does not require text prompts, which is an interesting and distinctive feature. Additionally, the rationale behind the different modules is clearly defined.
Critical strengths
- The motivation behind the overall network design is clear. 2.Utilizing the perceptual information inherent in images in place of textual information is a compelling approach.
Critical weaknesses
- The overall reconstruction process is designed to be highly complex, but can such a design for video enhancement be applied to real-world scenarios?
- Given the substantial inference time of several seconds per frame, one must question whether this approach yields sufficiently improved generalization performance to justify its cost. At present, a processing time of five seconds per frame is impractical; for a standard 60-frame-per-second video, generating just one second of footage would require 300 seconds, which is equally untenable. Under these constraints, does this methodology remain viable or meaningful?
- If the authors can demonstrate better generalization capability, the current inference time is also acceptable. Additionally, this generalization capability stems from the large model's prior knowledge, or from the designed physics-aware guidance.
Questions for authors
- Have you considered using a smaller version to reduce training time?
- Why does the PGA module modify the value? Wouldn't that alter the features intended for optimization? I am somewhat unclear on this point. 3.Is it meaningful to spend such a long time reasoning over a single frame?
- Is it permissible to make the source code open-source?
2 comments
Response to Reviewer pUjg (2/2)
Q2:
PGA adds a gated residual to the value and leaves the attention weights untouched. We thank the reviewer for the question, we clarify the mechanism here and in Section 4. PGA adds a gated rank-8 residual to the output of the frozen value projection, V' = W_V x + G (alpha A B x + beta x), where G is a per-token, per-head gate computed from physical signals and scaled by Fourier band energies. Changed values are intentional, the frozen query and key projections still decide where each token attends, and PGA changes only what each token contributes, according to its physical statistics. The frozen weights are never modified, and B and beta start at zero, so the residual is zero at the first step and training starts from the frozen model's own values. In the 38 single-stream blocks the text-slot tokens receive the residual too, with the gate fixed at one, We will update the mansucript the discussion it more detail.
Q4:
Yes, model weights and code are available. We will include the links in our final manuiscripts and will opensource all code, weights, variants, and datasets.
https://anonymous.4open.science/r/LumaFlux-demo-B518/
https://anonymous.4open.science/r/LumaFlux-weights-672C/
We thank the reviewer again for recognising the clear motivation and the prompt-free design. We hope the step sweep, the generalization evidence and the shared model resolve the reviewer's concerns about cost and practicality, and we would be grateful if the reviewer would consider raising the score.
Response to Reviewer pUjg (1/2)
We sincerely thank the reviewer for the insightful review. We are glad that the reviewer finds the motivation behind the model design clear, the rationale of each module well defined, and the use of perceptual information from the image in place of text prompts compelling and distinctive. Below we answer the questions:
W1:
We appreciate this concern. The model has different component but the inference is one fixed pipeline: VAE encode, 8 Euler steps of the frozen backbone with its adapters, VAE decode, one monotone spline. The added parts are small adapters (0.57% of the loaded weights). There is no prompt, no classifier-free guidance, no optical flow, no temporal layer and no per-video tuning. For video, frames can be split across GPUs or streamed sequentially. We also evaluate it on CHUG which is real user-generated clips dataset. More recent production and commercial methods use even more complex methods, whereas our method essentially can be wrapped in LoRA/adapter pipelines of all open-source libraries, making it easy to use and deploy.
W2 / Q3:
We thank the reviewer for the concrete arithmetic: one GPU at batch size 1 needs 2.64, 3.08 and 5.29 s per 540p, 720p and 1080p frame, so one second of 1080p60 video costs about 317 GPU-seconds. LumaFlux is not a real-time method, and the we don’t claim it be a real-time one. The setting it targets is offline conversion of legacy archives and user-generated SDR content for HDR presentation, i.e. backfills or post-productions (Appendix N), where each video is converted once and quality across unknown sources matters more than frame rate. Within that setting the cost is meaningful for three reasons.
Four steps match or beat eight on all seven metrics at about half the time, and two steps are 3.3 times faster with better PSNR and HDR-VDP-3. We will update our manuscript with this revision and recommends four steps. We thank the reviewer for this suggestion, as it strengthens our claim and shows the real robustness of our method.
W3:
The adapted prior does heavy work, and physical gating adds ~2 dB over generic adaptation. We thank the reviewer for this question, which goes to the core of the method.
The primary track uses 20 held-out contents under eight operators and two compression levels, and LumaFlux is best on all seven metrics there. In a cross-operator stress test (Appendix I), an analytic inverse matched to the wrong operator falls to 27.21 and 28.73 dB, while LumaFlux stays at 31.34 and 31.04 dB on the same inputs.
he VAE plus spline without transport gives 18.54 dB against 24.23 dB (Table 1). The physics-aware guidance then adds a clear margin over generic adaptation of the same prior, on the ablation holdout, plain LoRA reaches 27.92 dB (see our reply to Reviewer fJP9), attention and MLP LoRA with 33.1M parameters 28.36 dB, and the physically gated PGA with 3.3M parameters 29.56 dB. Physical gating therefore adds about ~2 dB with a tenth of the parameters.
Q1:
Training is already light, and a 3.3M-parameter variant works. Because the backbone is frozen, training takes less time even on one GPU for the main model. The ablation also shows that a much smaller adapter set works, as the reviewer suggests: PGA + spectral gate + RQS has 3.3M trainable parameters and reaches ~30 dB, against 29 dB for the 71.3M full model at the same 20k-step budget (Table 3). We report this smaller configuration as a lightweight variant.