- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper estimates an HDR environment map at a query location from sparse images with known camera poses. A diffusion model generates overlapping LDR views, a second model recovers HDR content, and exposure alignment and blending assemble the panorama. The stages train on panoramic videos and HDR stills, respectively. The contribution combines a generative prior with camera geometry to use observations from different positions. Validation includes three lighting benchmarks, experiments varying view count and camera displacement, and controls removing translation information. Results show benchmark improvements and benefits from additional views.
Critical strengths
-
The method estimates an environment map at a query location from sparse, posed observations, using LDR video with camera translation to learn content generation and HDR panoramas to learn radiometric reconstruction. This decomposition provides a practical way to exploit complementary data sources without requiring every training example to include both camera translation and HDR supervision. The resulting ability to use observations captured away from the query location is potentially valuable for real capture and object-insertion pipelines.
-
The experiments provide meaningful evidence that additional observations improve predictions. When all input views share the same camera position, increasing their number from one to four improves performance by 2.25 dB on a fixed region outside the coverage of all four views. For inputs captured at different positions, the authors include two controls: a model trained only on views from a single location, and the original model evaluated with camera translations set to zero. Although these comparisons do not fully establish the claimed geometric mechanism, they support the value of additional observations and accurate pose information, providing quantitative evidence for the method’s effectiveness.
Critical weaknesses
-
The paper repeatedly describes LuxDiT as accepting only a single image, although LuxDiT explicitly supports video-conditioned lighting estimation and compares its video mode with frame-wise inference. This mischaracterization undermines the broad claim that existing generative methods discard additional frames and leaves the most relevant multi-frame comparison missing. Explicit poses, sparse wide-baseline observations, and arbitrary query locations may still distinguish this work, but those differences need to be established through accurate positioning and comparisons in shared settings. Without this, the paper’s advancement over existing methods remains insufficiently demonstrated.
-
The evaluation establishes improvements in panorama prediction more convincingly than improvements in HDR lighting estimation. The main benchmark tone-maps environment maps before rendering, suppressing the high-intensity illumination that HDR reconstruction is intended to recover. The supplementary evaluation using linear environment maps covers only Laval Indoor and lacks paired significance testing, while the displaced-view experiments evaluate Stage 1 LDR panoramas rather than the complete HDR pipeline. An experiment combining displaced multi-view inputs, HDR ground truth, and linear-light rendering is therefore needed to substantiate the central contribution.
Questions for authors
-
Why is LuxDiT described as supporting only a single input image? Please correct this characterization and clarify how your method differs from its video mode. Could you also provide a comparison in a setting that both methods support?
-
Do the improvements from multiple views translate into better HDR lighting estimation? Could you provide an evaluation using inputs captured at different camera positions and GT HDR environment maps, with rendering performed using the original linear environment maps? Synthetic scenes could provide an initial test.
-
How is the unseen mask in Section 5 constructed? Does it account for depth, occlusion, or correspondences across views? If it relies only on rotation, how do you ensure that content in the masked target region is not already visible in images captured from other camera positions?
3 comments
Controlled synthetic evaluation
We also conducted preliminary experiments in three PBRT scenes using object-centred camera orbits, known poses, and independently rendered target panoramas. Nested input sets contain one view, four adjacent orbit views, and all sixteen orbit views. We evaluate three fixed query locations: above the selected object, at input camera 0, and at an “away” location outside the input orbit, corresponding to the original scene camera. We compare correct translations with zeroed translations while holding images, rotations, intrinsics, weights, and inference settings fixed.
Each cell reports standard-protocol sphere SI-RMSE averaged across three materials: correct translations / zero translations. Lower is better; bold marks the lowest error in each row across both translation settings and all input counts.
| Scene | Query location | 1 input | 4 inputs | 16 inputs |
|---|---|---|---|---|
| Bistro | Above object | 0.4258 / 0.4369 | 0.3888 / 0.4258 | 0.3424 / 0.3435 |
| Input camera 0 | 0.3775 / 0.3775 | 0.3658 / 0.3732 | 0.3021 / 0.2936 | |
| Away | 0.3994 / 0.3814 | 0.3865 / 0.3789 | 0.2924 / 0.2930 | |
| San Miguel | Above object | 0.5847 / 0.6282 | 0.6138 / 0.6395 | 0.5451 / 0.5895 |
| Input camera 0 | 0.6065 / 0.6065 | 0.5589 / 0.5953 | 0.6195 / 0.7086 | |
| Away | 0.7539 / 0.8112 | 0.6953 / 0.7785 | 0.8504 / 0.9378 | |
| Watercolor | Above object | 0.5986 / 0.6292 | 0.4665 / 0.5452 | 0.4602 / 0.4561 |
| Input camera 0 | 0.3284 / 0.3284 | 0.2668 / 0.3161 | 0.2699 / 0.4073 | |
| Away | 0.2362 / 0.2488 | 0.2253 / 0.3035 | 0.3386 / 0.4084 |
Correct translations help: at four inputs, they reduce SI-RMSE at eight of the nine scene–query combinations. At sixteen inputs, the benefit remains consistent across San Miguel's three query locations and substantial at two Watercolor locations, while Bistro shows little benefit.
Additional observations help: with correct translations, increasing from one to four inputs reduces SI-RMSE at eight of nine scene–query combinations. Increasing to sixteen further improves all three Bistro queries, but the trend is not uniformly monotonic: four inputs remain better at the input-camera and away queries in San Miguel and Watercolor.
The results support using posed multi-view evidence while showing that its benefit depends on the scene, query location, and input configuration. Metric agreement is not uniform: San Miguel's SI-RMSE gains are not consistently accompanied by lower RGB angular error. These preliminary results cover three selected scenes, three query locations per scene, and one generation seed.
Thanks to the authors for the clarifications and additional experiments. The LuxDiT video comparison addresses my concern about the missing external baseline, and I appreciate the correction to the interpretation of the angular mask.
My main concern remains whether the improvements translate into better HDR lighting estimation. The new linear rendering results are mixed, particularly on scenes with direct sunlight. The synthetic experiments provide useful evidence that correct translations help, but they still use the standard protocol that compresses dynamic range before rendering, so they do not close the gap between the multiview results and HDR fidelity.
The response strengthens the evidence for using posed observations to improve panorama prediction, but I remain unconvinced that the central HDR lighting contribution is sufficiently validated. I therefore maintain my rating of 2.
LuxDiT video conditioning
You are correct that LuxDiT supports video inputs, and we will correct its characterization as single-image-only. The distinction is explicit pose conditioning and query-location estimation, not simply accepting multiple frames.
We evaluated LuxDiT video on the same 181 Poly Haven scenes using 25 frames spanning a full yaw rotation at 60-degree vertical FoV. All frames enter the model, and we score its final output panorama. Our method receives four posed views with less angular coverage. Both methods use the standard tone-map-before-render protocol of Table 2.
Our method benefits from additional observations across all three materials, while LuxDiT's SI-RMSE changes are mixed. Compared with LuxDiT video, our four-view model reduces SI-RMSE by approximately 43%, 44%, and 31%. The corresponding absolute reductions have paired 95% scene-bootstrap intervals of [0.0265, 0.0390], [0.0782, 0.1005], and [0.1164, 0.1468]. This supplies an external multi-frame comparison in a shared co-located setting.
HDR fidelity
To examine HDR fidelity separately, we extended the single-view Laval Outdoor evaluation to linear-light rendering, removing tone mapping and clipping of HDR environment maps before rendering. We evaluated the full set and a 138-scene direct-sun subset selected from ground truth and visually checked.
The table reports linear SI-RMSE differences, ours minus LuxDiT, and paired 95% confidence intervals, multiplied by 1,000. Negative differences favor ours.
Intervals use 50,000 paired scene-bootstrap draws and are pointwise. After Holm correction across the 68 full-set and subset comparisons, only the small full-set mirror improvement is supported among these six comparisons. The ranking depends on the metric: LuxDiT achieves lower percentile-normalized linear RMSE for matte-silver and diffuse spheres on the direct-sun subset. Thus, the linear-light comparison is mixed rather than showing the consistent advantage observed under the standard protocol.
We also report preliminary controlled synthetic experiments in the follow-up below, including the complete input-count and translation-ablation results.
Angular-mask definition
We construct the mask from input-camera orientations and intrinsics, without depth, occlusion, or correspondence reasoning. The scored region lies outside the angular coverage of all eight input views, excludes the nadir below −45°, and remains fixed across input counts.
For displaced cameras, scene content visible in an input can move into this region through parallax. “Unseen” is therefore too strong; we will use “outside the eight-view angular coverage” and make the distinction explicit. The translation-control comparison remains on identical target pixels with identical input images and viewing directions, changing only the supplied translations.