- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces VideoRNG, a system for generating relightable 3D assets from a text prompr or a single image. First, a video diffusion model generates a turntable video under fixed camera and lighting protocol. Then, a subsequence feed-forward LRM predicts a Gaussian splat per foreground pixel, each with a neural appearance latent, that is decoded by a shared directional MLP and rendered with stochastic Gaussian ray tracing. The proposed contributions are: 1) a pipeline that generates relightable 3D assets, without constraining the geometry/material representations to coarse meshes and PBR, 2) an LRM that reconstructs relightable neural Gaussian splats in a feed-forward pass.
Critical strengths
Well-motivated problem setup: Generating relightable 3D assets with fur, feathers, and thin structures, which are hard to represent with mesh+PBR, is a meaningful direction. The proposed ray-traced Relightable Neural Gaussians (rtRNG) allow the assets to be integrated into a path tracer with inter-asset shadowing.
Critical weaknesses
1. Design choices not justified with insights and evidences
This reviewer understands that the overall pipeline works, but most components are existing techniques, and this reviewer could not find why each design choice should be the proposed one, at both the logical and experimental levels.
- 1.1) Generative part: The main contribution seems to be the feed-forward LRM that reconstructs rtRNG, while the generator, a straightforward fine-tuning of a pre-trained video model, serves as a front end that canonicalizes camera and lighting. Using generative video as observations for neural materials was also proposed in [1]. Would it be better to reposition the contributions toward the reconstruction method? Related to this, why is the LRM restricted to the fixed turntable protocol, although it already takes per-token Plücker rays (Sec. 3.2) like prior LRMs that accept arbitrarily posed views (e.g., GS-LRM [2], LSRM [3])? If the fixed protocol is necessary because the LRM relies on the known studio illumination, then this reviewer thinks that this is a limitation of the design rather than of the task, as ReLi3D [4] already reconstructs relightable assets from posed views under unknown illumination. Also, this reviewer has concerns about how the LRM, trained only on clean renders, handles the lighting and cross-view inconsistency that might be inherent in the generated videos, and this has not been evaluated or discussed.
- 1.2) rtRNG: rtRNG is mostly a combination of RNG [5] and stochastic Gaussian ray tracing [6], without discussion of any technical difficulty in combining them. Moreover, ray-traced Gaussians (e.g., 3DGRUT [7]) and relightable Gaussians with explicit appearance models (e.g., RGCA [8], R3DG [9], TRON [10]) already exist, and the renderer used [6] also supports relightable per-Gaussian shading. The paper does not discuss which effects rtRNG can represent but PBR Gaussians cannot, or the cost of encoding all appearance into a single latent (editability, interpretability). Since the baselines in Tables 2/3 are meshes with PBR or Gaussians with baked appearance, the gains cannot be attributed to rtRNG rather than to the pixel-aligned geometry. Missing comparisons include a PBR-head variant of the same LRM and renderer, per-object PBR-GS [9,11,12] and RNG [5] optimization on the generated frames (as in Appendix C.9; the known studio lighting is even favorable for PBR inverse rendering), and open-source relightable baselines (ReLi3D [4], Pixal3D [13]). The authors should also show cases where rtRNG represents effects that PBR Gaussians cannot (e.g., complex specular or anisotropic appearance on simple geometry, which Sec. 3.4 claims but never shows).
- 1.3) Proposed components: The two components claimed as improvements are not sufficiently validated.
- The transmittance blend (Eq. 7) is compared only against RNA's binary selection (Table 7), although continuous shadow conditioning was already explored in RNG [5]; comparisons with RNG-style continuous conditioning or simple multiplicative visibility are needed.
- For depth anchoring, Table 5 isolates its effect (26.32 → 30.61 dB) but measures fitting on the same 50 assets, while Table 6 evaluates generalization but compares against "rendering loss only," which lacks depth supervision, so its gain (24.49 → 29.22 dB) mixes anchoring with depth supervision. A full-scale comparison between Eq. 4 and Eq. 5 is needed.
- 1.4) Gaussian prediction architecture: Feed-forward Gaussian prediction has been explored with pixel-aligned depths along rays (GS-LRM [2], LGM [14]), 3D-anchored structured latents (TRELLIS [15]), and learnable Gaussian tokens (TokenGS [16]). The paper adopts the pixel-aligned design with only a brief rationale (avoiding free-space placement, Sec. 3.2). Notably, TokenGS argues that ray-aligned depth regression is suboptimal, and that decoupling Gaussians from pixels improves robustness to multi-view inconsistencies, yields more regularized geometry with fewer primitives, and enables TTO without degrading learned priors. These are exactly the issues observed here: duplicated geometry (Weakness 2), elongated splats (Weakness 3), and degraded TTO results (Appendix C.9). A discussion or comparison with these alternatives is needed.
- 1.5) Input view variants: The RGB and orthographic six-view variants (Sec. 3.1) are mentioned without motivation or evaluation. What are the benefits of having orthographic six-view inputs, rather than turntable views?
Overall, the paper reads as a stack of existing components and variants rather than a coherent method with clear insights.
2. Weak relighting evidence and reconstruction fidelity
- The LRM reproduces input views at only 22.11/24.84 dB (Fig. 12) and 22.15 dB (Table 4). Also, it often loses the specular highlights of the generated videos (e.g., "Witch's butter fungus", "Orca", and "Sunglasses" on the project page). This reviewer could also observe hallucinated and duplicated geometry/texture, such as overlapped textures on the "Rocking horse"'s face and three front legs in the side view.
- The ground-truth comparisons on the project page and Fig. 9 only rotate the training environment map, yet PSNR still drops dramatically for assets with high specular effects. Also, the HDRIs are not shown, and seem to be using too simple ones, that are similar to the training lighting (studio lights with one or two directional highlights).
- This reviewer suggests a controlled relighting evaluation in two settings: 1) feeding the GT turntable videos for unseen assets to the LRM, which isolates the relighting quality of rtRNG, and 2) running the full single-image pipeline from a rendered view of the GT asset, which can evaluate the whole system. In both settings, the GT assets should be rendered under multiple complex IBLs, and the LRM predictions should be relit under the same IBLs, preferably with zoomed-in comparisons of specular highlights. The same protocol should be applied to the PBR alternatives in Weakness 1.2, to justify the benefits of using neural Gaussians.
3. Missing geometry evaluation
Even though the paper claims to generate 3D "assets" and the predicted depths are directly measurable, no geometry metrics (depth, normal, or Chamfer error) are reported. This is critical for rtRNG, since the decoder encodes directions in each Gaussian's local frame and the transmittance is traced through the Gaussians, so incoherent geometry leads to misplaced highlights and shadows. The "Selenite towers" on the project page shows this issue: the generated views contain flat crystal faces and sharp edges, but the reconstruction result seems to be consists of thin, elongated Gaussian splats without coherent surfaces, and the satin sheen is lost. To justify the claim of "3D asset" creation, this reviewer recommends the authors to report depth/normal errors, Chamfer distance, normal visualizations, and an analysis of whether each Gaussian's local frame aligns with the surface normal.
4. Unfair evaluation setup
In Tables 2/3, VideoRNG receives 29 frames covering 360°, while baselines receive 1–16 views. Also, while LSRM [3] already made its PBR inverse rendering model publicly available, the authors choose to use its baked textures as albedo, which makes the overall comparison experiments unfair and carelessly designed. Moreover, the multi-view TRELLIS.2 is described only as "our fine-tuning," without any details. Extending TRELLIS.2 to multi-view is non-trivial, so authors should have provided details. Pixal3D [13] is an open-sourced, multi-view extension of TRELLIS.2, so this reviewer recommends the authors to properly cite the paper and include it as a baseline, rather than naively using own multi-view extension of TRELLIS.2.
5. Limited generalization analysis
Although the paper claims generation from a single image, all image-conditioned results use rendered inputs (the "input photo" in Fig. 8 is also a render). Results on real images, e.g., Stanford-ORB [17] or DTC [18] with GT environment maps, would be more compelling. Moreover, since TTO degrades novel views and lighting (Appendix C.9), the final quality seems to be bounded by the feed-forward prediction.
6. Issues with the paper's presentation
Figure 2 contains several presentation issues that look unprofessional and discounts paper's quality. This overall makes the paper look not ready to be properly reviewed, and even feel like AI-slop. Some examples: overlapping texts and shapes, meaningless bolds for "K" and "M", misaligned shapes, legends, barely readable labels in (b), and an MLP labeled “4L × 64” that contradicts the 9-layer, width-512 decoder in Sec. 3.4. This reviewer wonders whether generative AI tools have been used to create this figure, and if so, should this be disclosed in the AI use statement, which mentions only coding, prompt construction, and manuscript suggestions? Also, the title on OpenReview (“VideoRNG: Relightable Neural 3D Asset Extraction from Generative Video Models”) differs from the title in the PDF (“VideoRNG: Relightable Neural Gaussians from Generative Video Models”); which one is intended?
[1] Xue et al., VideoNeuMat: Neural Material Extraction from Generative Video Models, SIGGRAPH 2026
[2] Zhang et al., GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting, ECCV 2024
[3] Li et al., LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windows, ECCV 2026
[4] Dihlmann et al., ReLi3D: Relightable Multi-view 3D Reconstruction with Disentangled Illumination, ICLR 2026
[5] Fan et al., RNG: Relightable Neural Gaussians, CVPR 2025
[6] Xu et al., Stochastic Ray Tracing for the Reconstruction of 3D Gaussian Splatting, CVPR 2026
[7] Wu et al., 3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting, CVPR 2025
[8] Saito et al., Relightable Gaussian Codec Avatars, CVPR 2024
[9] Gao et al., Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing, ECCV 2024
[10] Perel et al., TRON: Tracing Rays to Orchestrate a Neural Renderer for 3D Gaussian Reconstructions, arXiv 2026
[11] Liang et al., GS-IR: 3D Gaussian Splatting for Inverse Rendering, CVPR 2024
[12] Jiang et al., GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces, CVPR 2024
[13] Li et al., Pixal3D: Pixel-Aligned 3D Generation from Images, SIGGRAPH 2026
[14] Tang et al., LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation, ECCV 2024
[15] Xiang et al., Structured 3D Latents for Scalable and Versatile 3D Generation, CVPR 2025
[16] Ren et al., TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokens, CVPR 2026
[17] Kuang et al., Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark, NeurIPS 2023
[18] Dong et al., Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset, CVPR 2025
Questions for authors
(References as in the Weaknesses section.)
Clarification requests
- Is the fixed turntable protocol necessary for the LRM (Weakness 1.1)? If so, is it because the LRM relies on the known studio illumination, and how robust is the LRM, trained only on clean renders, to deviations in the generated videos (lighting, cross-view inconsistency)?
- Which input variant (RGBA turntable, RGB turntable, or orthographic six views) and which decoder (9-layer shared, 6-layer globally distilled, or 4-layer per-asset) were used for each figure and table? For the orthographic variant, how is the lighting defined, and what are its benefits over orbital views (Weakness 1.5)? Please provide a clear insights on these design choices.
- How is the GT depth for depth anchoring defined for fuzzy or semi-transparent objects (fur, feathers, crystals)?
- How was the multi-view TRELLIS.2 fine-tuned (data, number of input views), and why was LSRM [3] relit with its textures as albedo instead of its publicly-available inverse-rendering variant (Weakness 4)?
- What are the trade-offs of the pixel-aligned design against alternatives such as TokenGS [16], especially regarding the duplicated geometry, weirdly enlongated splats from the results (Weakness 1.4)? Will 2DGS, triangle splatting, etc improve the geometric reconstruction for the assets?
Requested evidence that could change this reviewer's assessment
- A PBR-head variant of the same LRM and renderer, and per-object PBR-GS [9,11,12] and RNG [5] optimization on the same generated frames (Weakness 1.2). Please also show cases where rtRNG represents effects that PBR Gaussians cannot, e.g., complex specular or anisotropic appearance on simple geometry, which Sec. 3.4 claims but never shows.
- A controlled relighting evaluation (Weakness 2) in two settings: 1) feeding the GT turntable videos of unseen objects to the LRM, and 2) running the full single-image pipeline from a rendered view of the GT unseen object asset. In both, please relight the prediction under multiple complex IBLs and compare with the GT rendered under the same IBLs, with zoomed-in comparisons of specular highlights, for both VideoRNG and the PBR alternatives in Q7. Please also visualize the HDRIs used, preferably with complex ones that differ from the training lighting distribution.
- Geometry metrics against GT on the synthetic validation set, with normal visualizations, and an analysis of whether each Gaussian's local frame aligns with the surface normal (Weakness 3).
- A full-scale comparison between Eq. 4 (joint training with depth loss) and Eq. 5 (anchoring), and a comparison of the transmittance blend with RNG-style continuous shadow conditioning or simple multiplicative visibility (Weakness 1.3).
- Comparisons under matched input conditions, e.g., multi-view baselines (LSRM [3], ReLi3D [4], Pixal3D [13]) receiving the same number and coverage of input views as VideoRNG (Weakness 4).
- Results on real images, given the claim of generation from a single image. E.g., single photos from Stanford-ORB [17] or DTC [18] passed through the full VideoRNG pipeline and relit under diverse environment maps (Weakness 5).