acceptodds
Under review as a conference paper at ICLR 2027

Luce: Relightable Gaussians for 3D Asset Generation

Abstract

High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. However, preserving fine detail across the physically based rendering (PBR) modalities needed for relighting remains challenging. To address this, we propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for albedo, metallic-roughness, and surface normals. A variational autoencoder compresses this representation into a unified material-aware latent space. A rectified-flow transformer generates this latent from a single image using multi-layer features from a pretrained image encoder that preserve both semantic context and fine spatial detail. The latent is then decoded into relightable PBR Gaussians and an optional textured mesh with a tangent-space normal map. On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline. We further evaluate Luce on a benchmark of AI-generated images depicting diverse subjects and materials, where it improves the CLIP image-alignment score over the best baseline (0.8519 vs. 0.8299). Luce generates relightable, geometrically accurate, and materially faithful assets that preserve fine details such as text, logos, and inscriptions.

Primary area:
generative models
Keywords:
Image-to-3D Generation, 3D Gaussian Splatting, Relightable 3D Assets, Physically Based Rendering, 3D Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_h94UOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper presents Luce, a framework for generating 3D objects with physically based rendering (PBR) materials from a single image. Luce represents each object using three geometrically independent sets of 3D Gaussians that encode albedo, metallic–roughness, and surface normals, respectively. A variational autoencoder (VAE) compresses these Gaussian sets into a unified structured latent representation. An image-conditioned rectified-flow model generates latents that are subsequently decoded into relightable 3D assets.

Critical strengths

The paper has two main strengths:

  • Luce demonstrates strong generation performance on Toys4K and the authors’ benchmark of AI-generated images.
  • Multi-layer image conditioning helps preserve fine surface details, including text and logos, improving fidelity to the input image.
Critical weaknesses

The paper has three main weaknesses:

  • Limited architectural contribution. The overall framework largely follows TRELLIS and TRELLIS.2. The sparse-structure VAE and flow are reused without modification. Although Luce changes the VAE’s input representation and spatial compression scheme, these adaptations appear incremental rather than substantial architectural advances.

  • Limited new methodological insight. Several central ingredients have clear precedents: MGM [1] explores Gaussian-based generation with explicit PBR attributes, and IDArb [2] uses the same albedo, metallic–roughness, and normal grouping for intrinsic decomposition. Multi-layer image features are also an established technique, as acknowledged by the authors. Beyond combining these ingredients, the paper provides limited analysis of why its particular representation is preferable to simpler alternatives, such as Gaussians with shared geometry and multiple PBR attributes.

  • Cross-modality geometric consistency remains a concern. MGM [1] shows that separately reconstructing PBR components can produce inconsistent geometry and degrade relighting quality (Fig. 6). Luce partially mitigates this issue by first generating a shared sparse voxel structure, which constrains all three Gaussian sets to a common spatial scaffold. However, I am concerned that the adopted voxel resolution is too coarse to sufficiently constrain these geometric differences, particularly around fine structures and object boundaries.

  • [1] Ye et al. Large Material Gaussian Model for Relightable 3D Generation, 2025. Paper.

  • [2] Li et al. IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations, ICLR 2025. Paper.

Questions for authors

In addition to the concerns raised above, I would like to clarify the source of the reported performance gains. The conditioning ablation shows substantial gains from multi-layer DINOv2 features, particularly on the AI-generated-image benchmark, where the single-layer Luce variant underperforms TRELLIS GS in CLIP score. What evidence isolates the contribution of the proposed PBR Gaussian representation and VAE adaptations from these conditioning gains? In particular, how does Luce compare with a TRELLIS-based baseline equipped with the same multi-layer conditioning?

4 comments
AuthorsOpenReview ↗

Response to Reviewer h94U

Thank you for suggesting the TRELLIS comparison. We fine-tuned the official TRELLIS flow on our training data with the same multi-layer DINOv2 features used by Luce, at TRELLIS's 518 × 518 input resolution. This improved TRELLIS's Toys4K FID from 30.75 to 27.38, compared with 20.99 for Luce. To test the contribution of the Gaussian representation more directly, we compared two variants within Luce. At matched storage, Luce achieves 20.99 FID with a separate Gaussian set for each PBR modality, compared with 22.79 when all three modalities share one set.

 

Q: Luce vs. a TRELLIS-family baseline with the same conditioning

(a) Luce vs. TRELLIS 2 at comparable conditioning resolution. Table 4's single-layer Luce uses only the final DINOv2 layer on 1036 × 1036 input images. TRELLIS 2 uses DINOv3 on 1024 × 1024 input images for its latent stage. With single-layer conditioning, Luce retains its advantage over TRELLIS 2 on Toys4K FID (25.21 vs. 29.22) and KID×100 (0.101 vs. 0.165). Their AI-image CLIP scores are similar (0.808 vs. 0.811). The two models share TRELLIS 2's sparse-structure stage (Sec. 3.3) but differ in their image encoders, latent representations, and flows. To test the contribution of Luce's PBR Gaussian representation more directly, we compare modality-specific Gaussian sets with a single shared set below.

(b) TRELLIS with our multi-layer conditioning and our data, as suggested in the review. We trained it with the same layers at TRELLIS's 518 × 518 input resolution and the same data as Luce, starting from the official TRELLIS flow (best in bold, second best underlined):

ModelFID ↓KID×100 ↓AI CLIP ↑
TRELLIS GS (official)30.750.2120.830
TRELLIS + multi-layer, our data27.380.0760.849‾\underline{0.849}
Luce, single-layer (Table 4)25.210.1010.808
Luce at 518 × 518 input conditioning22.14‾\underline{22.14}0.051‾\underline{0.051}0.839
Luce20.990.0330.852

Fine-tuning TRELLIS with our data and multi-layer conditioning improves its Toys4K FID from 30.75 to 27.38. With 518 × 518 conditioning for both models, adapted TRELLIS scores 27.38 FID (KID×100 0.076) and Luce scores 22.14 (0.051). On AI-image CLIP, adapted TRELLIS scores 0.849 and 518 × 518 Luce 0.839. The full 1036 × 1036 Luce model scores 20.99 FID and 0.852 AI-image CLIP. Within Luce, switching from single-layer to multi-layer conditioning lowers FID from 25.21 to 20.99 (Table 4).

(c) Decoder fidelity and VAE design. Fine-tuning the TRELLIS flow improves its image conditioning, but the resulting latents are still decoded by the original TRELLIS decoder. Detail loss is already visible in TRELLIS's reconstruction of the TV (Fig. 6), where the lettering is blurred, and its shaded-color reconstruction PSNR is 27.0 dB (Table 2). To test a specific VAE design choice within Luce, we moved pooling from the end of the encoder to the start. Albedo reconstruction PSNR fell from 39.76 to 36.67 dB, a drop of 3.1 dB. We report the full ablation in our response to DL8F.

On AI-image CLIP and Toys4K FID. As the review notes, single-layer Luce scores lower than official TRELLIS GS on AI-image CLIP (0.808 vs. 0.830), while achieving lower Toys4K FID (25.21 vs. 30.75). After fine-tuning TRELLIS with our data and multi-layer conditioning, its AI-image CLIP reaches 0.849, close to Luce's 0.852.

On the AI-image benchmark, we use CLIP to measure overall alignment between the input image and the rendered asset. In Fig. 17, Luce GS preserves more of the lettering in "GRIMNIR'S FORGE", while official TRELLIS GS scrambles the lettering despite a slightly higher CLIP score (0.845 vs. 0.842 for Luce GS). Figs. 11–18 (Appendix G) show the other comparisons with their CLIP scores. Thank you for prompting this discussion about how well image alignment scores reflect fine details such as lettering.

AuthorsOpenReview ↗

Response to Reviewer h94U (Part 2)

W2: Independent vs. shared geometry

Following the review's suggestion, we compared Luce with the shared alternative, one geometry carrying albedo, normal and metallic-roughness (MR), at matched storage and at matched Gaussian count. We did so at every stage: the Gaussian PBR fitting that produces the VAE's inputs, VAE reconstruction and generation (details in our response to DL8F):

VariantFit albedo PSNR ↑Fit normal PSNR ↑Fit MR PSNR ↑VAE albedo PSNR ↑VAE MR PSNR ↑VAE normal PSNR ↑VAE color PSNR ↑Generation FID ↓
Luce (independent)42.2837.9340.5139.7640.2436.8736.9620.99
Shared, matched storage41.4237.8140.7438.4939.1735.7035.8222.79
Shared, matched count41.6638.0240.9739.5540.4836.6136.6222.22

At matched storage, using a separate Gaussian set for each PBR modality improves albedo reconstruction PSNR from 38.49 to 39.76 dB and lowers generation FID from 22.79 to 20.99. The shared set has a small advantage in fitted MR PSNR (0.23 dB). At matched Gaussian count, the shared set also achieves higher MR reconstruction PSNR (40.48 vs. 40.24 dB), while Luce retains lower generation FID (20.99 vs. 22.22). Our motivation is that fine albedo details, such as lettering, can occur where MR is nearly uniform. Separate sets let each modality place its Gaussians where it needs detail.

MGM and IDArb. Thank you for these pointers; we will add IDArb to the related work of the revised version and expand the discussion of MGM.

  • MGM stores albedo, roughness, and metallic values on shared 2D Gaussian surfels, whose orientations determine their normals. Our ablation tests geometry sharing within Luce's own reconstruction pipeline. In this variant, each shared Gaussian carries an explicit shading normal that can differ from its geometric orientation.
  • IDArb performs 2D multi-view intrinsic decomposition. It uses the standard albedo / MR / normal grouping but produces image-space buffers rather than a 3D generative representation.
  • What is new in Luce's representation. To our knowledge, Luce is the first image-to-3D generative model to represent shading normals with a dedicated set of 3D Gaussians whose positions, scales, rotations, and opacities are parameterized independently of the albedo and metallic-roughness sets. We supervise the normal Gaussians using shading normals rendered with each asset's authored normal maps, allowing them to capture detail finer than the underlying geometry. At mesh export, we bake the decoded normals into a tangent-space normal map.

 

W3: Cross-modality consistency and voxel resolution

MGM's Fig. 6 shows how reconstructing PBR components separately can lead to differences in shape. In Luce, all three Gaussian sets are decoded from the same latent on a shared voxel grid. We supervise them with renders of the same asset from the same cameras. Each Gaussian's center offset is limited to ±0.75 voxel relative to its cell center.

To quantify agreement in foreground coverage, we measured pairwise and three-way intersection over union (IoU) across the rendered masks for albedo, shading normals, and metallic-roughness (each from its own Gaussian set, before compositing) after Gaussian fitting, VAE reconstruction, and generation. To stress-test agreement near the silhouette, we also computed three-way IoU within a 10-pixel band around the boundary of the three masks' union:

StagePairwise mask IoU ↑ (alb–nor / alb–MR / nor–MR)Three-way mask IoU ↑Three-way boundary-band IoU (10 pixels) ↑
Input fits (4 views at 1024²)0.9988 / 0.9991 / 0.99890.99840.990
After the VAE (4 views at 1024²)0.9986 / 0.9987 / 0.99850.99790.989
Generated, Toys4K0.9981 / 0.9984 / 0.99790.99730.987
Generated, AI images0.9982 / 0.9982 / 0.99800.99720.984

Within the 10-pixel silhouette band, three-way IoU is 0.987 for generated Toys4K assets and 0.984 on the AI-image benchmark. Across 150 views of the fitted assets, doubling the grid resolution changes pairwise foreground IoU only slightly, from 0.9985–0.9990 at 128³ to 0.9987–0.9992 at 256³. We also compared the rendered depths of the fitted Gaussian sets inside the silhouette. At 128³, the mean depth difference ranges from 1.1 to 1.7 voxels across the three modality pairs, about 1% of the grid width.

We will add these results to the revised version.

AuthorsOpenReview ↗

Response to Reviewer h94U (Part 3)

W1: Architectural contribution

Luce reuses TRELLIS 2's sparse-structure stage to generate the voxel scaffold (Sec. 3.3). Our main contribution is the PBR representation generated on that scaffold. Albedo, metallic-roughness, and shading normals each have their own Gaussian set with independently parameterized geometry. Our VAE encodes all three sets into one structured latent, which a rectified flow generates from a single image. Our goal was to preserve fine detail through Gaussian fitting, VAE reconstruction, and generation. The ablations summarized below examine how fidelity changes when we share Gaussian geometry across modalities, move pooling earlier in the VAE, or condition the flow on features from a single image-encoder layer.

  • Representation. Independent geometry per modality gives +1.3 dB albedo after the VAE vs. shared geometry at matched storage (W2).
  • VAE. Moving spatial pooling from before the encoder to after it raises albedo reconstruction PSNR from 36.67 to 39.76 dB. Metallic-roughness and shading normals also reconstruct more accurately.
  • Flow conditioning. With the same VAE, Luce scores 25.21 Toys4K FID using DINOv2's last layer and 20.99 using multiple layers (Table 4). Fig. 4 illustrates the difference on fine details such as lettering.

What each comparison tests (Toys4K):

FactorExperimentResult
Full systems, single-layer conditioningLuce (DINOv2) vs. TRELLIS 2 (DINOv3), same sparse-structure stageFID ↓ 25.21 vs. 29.22
Full systems, multi-layer conditioningAdapted TRELLIS (518 × 518 input) vs. Luce (1036 × 1036 input)FID ↓ 27.38 vs. 20.99
Independent vs. shared geometryMatched storage; a VAE and flow trained for each variantAlbedo PSNR ↑ 39.76 vs. 38.49; FID ↓ 20.99 vs. 22.79
Multi-layer conditioningMulti- vs. single-layer Luce (Table 4)FID ↓ 20.99 vs. 25.21
Fusion projectorRe-trained single-layer Luce with and without a 10.5M-parameter projectorFID ↓ 23.55 with projector, 24.39 without
Conditioning resolution after fine-tuningLuce fine-tuned to 518 × 518 vs. Luce at 1036 × 1036FID ↓ 22.14 vs. 20.99
Flow size (different training histories)Luce 0.6B / 1.3B / 2.19BFID ↓ 22.47 / 22.41 / 20.99

We hope these results are helpful. If they address the concerns raised in the review, we would be grateful if the assessment could be reconsidered, and we are happy to answer any further questions.

Thank you for the thorough rebuttal. The ablations and experiments address several of my concerns, and I am raising my rating. However, I still see the contribution primarily as an integration of PBR Gaussians into an established image-to-3D pipeline. Also, the ablations show a benefit from independent geometry, but the authors' claim that each modality allocates Gaussians where it needs more detail remains unverified.

Reviewer_mmBFOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Luce, a single-image-to-3D framework that represents geometry and PBR materials using modality-specific Gaussian primitives for albedo, metallic-roughness, and surface normals. A structured latent VAE compresses these Gaussians, while an image-conditioned rectified-flow transformer jointly generates geometry and materials. The representation supports direct PBR Gaussian rendering and optional textured-mesh export with tangent-space normal maps. Although the results are promising, several core design choices require stronger experimental validation.

Critical strengths
  1. Luce explicitly represents multiple PBR modalities and supports both direct Gaussian rendering and production-compatible textured-mesh export.
  2. Multi-layer DINOv2 conditioning effectively preserves text, logos, and surface patterns, with clear quantitative improvements on both evaluation benchmarks.
Critical weaknesses

Refer to the questions.

Questions for authors
  1. A central contribution is the use of geometrically independent Gaussian sets for different PBR modalities. However, the paper does not compare this design against a shared Gaussian set carrying multiple PBR attributes. Consequently, it remains unclear whether the improvements arise specifically from the proposed modality-specific representation, the larger model, or the training configuration.
  2. Albedo, metallic-roughness, and normals are represented by Gaussian sets with independent positions, scales, rotations, and opacities, and are splatted separately before deferred shading. The paper should explain how spatial and visibility consistency across the resulting buffers is ensured. Quantitative analysis of cross-modality alignment would strengthen the physical validity of the representation.
  3. The mesh decoder training and texture-baking stage are important for the claimed production-compatible output, but several details are missing, including the mesh supervision objective, texture resolution, optimization schedule, number of iterations, and total-variation weight. These details are necessary for reproducibility and for interpreting the reported mesh runtime and quality.
2 comments
AuthorsOpenReview ↗

Response to Reviewer mmBF

Thank you for the careful review and for the questions on the representation, visibility across buffers, and mesh export; we answer each below.

 

Q1: Representation, model size, or training configuration?

We compare Luce's modality-specific Gaussian sets with a shared set that carries all three PBR attributes. We then examine how generation quality varies with flow size and how TRELLIS performs after fine-tuning on our data with our multi-layer conditioning.

  1. Representation. We compared Luce with one shared geometry carrying albedo, normal and metallic-roughness (MR), at matched storage and at matched Gaussian count, at each stage of the pipeline:

    VariantFit albedo PSNR ↑Fit normal PSNR ↑Fit MR PSNR ↑VAE albedo PSNR ↑VAE MR PSNR ↑VAE normal PSNR ↑VAE color PSNR ↑Generation FID ↓
    Luce (independent)42.2837.9340.5139.7640.2436.8736.9620.99
    Shared, matched storage41.4237.8140.7438.4939.1735.7035.8222.79
    Shared, matched count41.6638.0240.9739.5540.4836.6136.6222.22

    At matched storage, using a separate Gaussian set for each modality gives Luce higher VAE reconstruction PSNR for all four outputs and 1.80 points lower generation FID than using a single shared set. The shared set has a small advantage in fitted MR PSNR (0.23 dB). At matched Gaussian count, Luce still achieves 1.23 points lower generation FID. Our response to DL8F gives the full comparison.

  2. SLatFlow model size. The 0.6B- and 1.3B-parameter SLatFlow variants achieve Toys4K FIDs of 22.47 and 22.41, respectively, compared with 20.99 for the 2.19B model (Table 1). Even with the smallest SLatFlow, Luce achieves lower FID than TRELLIS 2 (29.22) and TRELLIS (30.75).

    FlowFID ↓KID×100 ↓AI CLIP ↑
    Luce 2.19B (Table 1)20.990.0330.852
    Luce 1.3B22.410.0540.843
    Luce 0.6B (≈ TRELLIS's flow size)22.470.0460.842
    TRELLIS 2 (Table 1)29.220.1650.811
    TRELLIS GS (Table 1)30.750.2120.830
  3. Training configuration. Fine-tuning the official TRELLIS flow with our data and multi-layer features on 518 × 518 input images improves its FID from 30.75 to 27.38, still 6.4 FID behind Luce (20.99; details in our response to h94U).

 

Q2: Spatial and visibility consistency across buffers

We compared the rendered foreground masks for albedo, normals and MR in each camera view (each from its own Gaussian set, before compositing), using the intersection over union (IoU): pairwise, across all three masks, and, as a stress test around the silhouette, across all three masks within 10 pixels of the union silhouette:

StagePairwise mask IoU ↑ (alb–nor / alb–MR / nor–MR)Three-way mask IoU ↑Three-way boundary-band IoU (10 pixels) ↑
Input fits (4 views at 1024²)0.9988 / 0.9991 / 0.99890.99840.990
After the VAE (4 views at 1024²)0.9986 / 0.9987 / 0.99850.99790.989
Generated, Toys4K0.9981 / 0.9984 / 0.99790.99730.987
Generated, AI images0.9982 / 0.9982 / 0.99800.99720.984

All three Gaussian sets are decoded from the same latent on a shared voxel grid. They are supervised with renders from the same cameras. Measured within each evaluated view, the three buffers agree closely in foreground coverage at every stage. Across Gaussian fitting, VAE reconstruction, and generation, their three-way foreground IoU ranges from 0.9972 to 0.9984. To stress-test agreement near object boundaries, we also measure IoU within a 10-pixel band around the silhouette, where it ranges from 0.984 to 0.990. Inside the silhouette, the mean pairwise depth differences between the three fitted Gaussian sets range from 1.1 to 1.7 voxels at 128³ resolution (about 1% of the grid width). We will add this analysis to the revised version.

AuthorsOpenReview ↗

Response to Reviewer mmBF (Part 2)

Q3: Mesh decoder and texture baking

We will add these details to the appendix of the revised version.

  • Mesh decoder (TRELLIS's architecture, retrained on our latent):
    • network: a 12-block sparse transformer (768 channels, 12 heads, Swin attention with window 8) at the 64³ latent resolution, then two sparse-subdivision blocks to 256³ and TRELLIS's FlexiCubes head (DMC extraction), predicting geometry, vertex colors and vertex normals;
    • data: latents sampled from the frozen SLatVAE encoder on 587K assets;
    • supervision (mesh renders at 512²): geometry 1.0 (mask L1, depth Huber ×10, normal L1 + 0.2 LPIPS + 0.2 SSIM), FlexiCubes regularization 1.0, TSDF 0.01, consistency 10, and L1 + 0.2 LPIPS + 0.2 SSIM on the vertex-color and vertex-normal renders (0.1 each);
    • optimization: 300K steps (the checkpoint used for Table 2); Muon (lr 1e-3, weight decay 1e-3) for hidden matrices and AdamW (lr 1e-4, weight decay 1e-2) for the rest; fp16 mixed precision; EMA 0.9999. For generation, we use an intermediate checkpoint with a short 1.4K-step normal-smoothing fine-tune that reduces faceting.
  • Post-processing: we remove 95% of the faces (quadric decimation) and fill holes.
  • Baking, per modality (albedo, MR, normal) on one shared xatlas UV atlas:
    • texture: 1024², fit to multi-view renders of the decoded Gaussians;
    • optimization: Adam (lr 1e-2 → 1e-5, cosine; betas 0.5 / 0.9), 2,500 iterations;
    • loss: masked L1 plus total variation (weight 1e-2);
    • normals: converted to tangent space with the per-texel mesh tangent frame (Appendix B).
  • Runtime (H100, Table 1): 7.2 s per asset for the Gaussian path and 67–76 s for the mesh path, dominated by baking (54 s of a 62 s per-asset total in a separate 60-asset profile); the mesh decoder itself takes 0.09 s.

We hope this addresses the questions raised in the review, and we are happy to provide further details.

Reviewer_V1PSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Luce introduces a multi-modal Gaussian representation method for generating "relightable" 3D assets from a single image. This approach employs separate sets of Gaussians for albedo, metallic–roughness, and surface normals, compressing them into a unified latent space; an image-conditioned flow model is then used to generate these latent variables. Multi-layer DINOv2 features help preserve fine appearance details, while normal-map baking allows the predicted surface details to be transferred onto exported mesh models. While the method delivers excellent rendering quality, the specific advantages of its independent Gaussian representation—as well as the accuracy of its relighting results—warrant further validation.

Critical strengths

An useful unified representation for generation and relighting. The method brings explicit PBR attributes into a structured latent that supports both Gaussian rendering and mesh export.

A flexible design for material detail. Independent Gaussian sets allow different modalities to allocate their representation capacity to different surface details.

Critical weaknesses

The novelty is mainly in the integration, and the key design is not isolated. PBR Gaussian attributes, structured-latent generation, multilayer DINO features, and normal-map baking all build on existing ideas. The potentially distinctive choice is to give each modality independent Gaussian geometry, but the paper does not compare it with a shared-geometry alternative. Its necessity and contribution therefore remain unclear.

Relighting evaluation is limited and comparisons are affected by the rendering protocol. On Toys4K, the evaluation illumination differs from the conditioning illumination, but only one evaluation environment is used for the quantitative results. Moreover, non-PBR baselines retain baked conditioning illumination and cannot be relit in the same way. Stronger comparisons among relightable methods under multiple lighting conditions would better support the central claim.

The current evaluation does not fully explain the source of the gains. Some baselines use lower conditioning resolutions, and the single-layer conditioning ablation has no fusion projection, so its capacity is not fully matched. Independent Gaussian sets may also produce inconsistent material buffers near occlusion boundaries, which is not directly measured.

Questions for authors
  1. Does independent Gaussian geometry outperform a shared-geometry model at matched capacity and Gaussian count?
  2. How consistent are the predicted modalities near silhouettes and across viewpoints?
  3. Can you report relighting errors under several unseen environment maps, focusing on PBR-capable baselines?
  4. Do the gains persist with matched conditioning resolution and fusion-module capacity?
  5. Can text preservation be evaluated quantitatively, for example with OCR accuracy?
2 comments
AuthorsOpenReview ↗

Response to Reviewer V1PS

Thank you for the thoughtful review and for suggesting the shared-geometry and relighting comparisons; we are happy to share the results below.

 

W1: Novelty and isolating the key design

To our knowledge, Luce is the first Gaussian-based 3D generative model to represent albedo, metallic-roughness, and shading normals with three geometrically independent Gaussian sets. Prior approaches discussed in Sec. 2 obtain normals from reconstructed geometry, for example from the shortest axis of a flattened Gaussian or from rendered depth. In Luce, we supervise the normal Gaussians with the asset's authored shading normals, which lets them capture surface detail finer than the base geometry. We bake these normals into tangent-space normal maps for mesh export (Sec. 3.4). A single structured latent encodes all three sets. In Q1, we test what independent geometry contributes by comparing it with a shared-geometry alternative at matched storage and at matched Gaussian count.

 

Q1: Independent vs. shared geometry at matched capacity and count

Luce uses separate Gaussian sets for albedo, normals, and metallic-roughness (MR). In the shared variant, each Gaussian carries all three attributes. At matched storage, Luce achieves 0.9 dB higher fitted albedo PSNR and higher VAE reconstruction PSNR for all three modalities. At matched Gaussian count, Luce achieves lower generation FID (20.99 vs. 22.22), while the shared variant has slightly higher PSNR for fitted normals and MR and for reconstructed MR. The table summarizes all three stages, with full results in our response to DL8F.

VariantFit albedo PSNR ↑Fit normal PSNR ↑Fit MR PSNR ↑VAE albedo PSNR ↑VAE MR PSNR ↑VAE normal PSNR ↑VAE color PSNR ↑Generation FID ↓
Luce (independent)42.2837.9340.5139.7640.2436.8736.9620.99
Shared, matched storage41.4237.8140.7438.4939.1735.7035.8222.79
Shared, matched count41.6638.0240.9739.5540.4836.6136.6222.22

Generation FID is 20.99 for independent geometry, 22.79 for shared geometry at matched storage and 22.22 for shared geometry at matched count.

 

Q2 / W3: Consistency near silhouettes and across views

We compared the rendered foreground masks for albedo, normals and MR in each camera view (each from its own Gaussian set, before compositing), using the intersection over union (IoU): pairwise, across all three masks, and, as a stress test around the silhouette, across all three masks within 10 pixels of the union silhouette:

StagePairwise mask IoU ↑ (alb–nor / alb–MR / nor–MR)Three-way mask IoU ↑Three-way boundary-band IoU (10 pixels) ↑
Input fits (4 views at 1024²)0.9988 / 0.9991 / 0.99890.99840.990
After the VAE (4 views at 1024²)0.9986 / 0.9987 / 0.99850.99790.989
Generated, Toys4K0.9981 / 0.9984 / 0.99790.99730.987
Generated, AI images0.9982 / 0.9982 / 0.99800.99720.984
  • Boundary band (as suggested in the review). Around the GT silhouette, the fits' pairwise IoUs are 0.993–0.994.
  • Across views. We measure foreground agreement within each view and average over all evaluated views. Coverage IoU does not see depth differences inside the silhouette, which we measure next for the fitted assets.
  • Depth. Inside the silhouette, the rendered depths of the three fitted sets differ by 1.1–1.7 voxels on average at 128³ (about 1% of the grid width).
  • Silhouette agreement. Decoded buffers match the GT silhouette with IoU ≥ 0.9975.
  • Rendering. Deferred shading composites all buffers with one coverage mask (the albedo set's).
  • Mesh export. All three modalities share one surface, predicted by the mesh decoder, and one UV atlas.

We will add these results to the revised version.

AuthorsOpenReview ↗

Response to Reviewer V1PS (Part 2)

Q3 / W2: Relighting under several environment maps

We evaluated all methods on the same 309 Toys4K assets under six environment maps. We re-evaluated the Table 1 map (ninomaru teien) on this subset and added five maps shown in supplementary Fig. 8.

  • Lighting. For each map, the ground truth and every relightable output are rendered under that map, while the conditioning images stay under studio_small_01.
  • Methods. As suggested in the review, we compare the PBR-capable methods: Luce (the paper model; Gaussian and mesh outputs), TRELLIS 2 and 3DTopia-XL. TRELLIS GS and LiTo bake the conditioning illumination into their appearance; since the conditioning images do not change, their outputs do not change with the evaluation map either.
  • Metrics. Generated and reference assets can differ in geometry and pose, so for each map we compare the distributions of their renders (FID, KID); Table 2 separately reports per-pixel errors of the buffers that drive relighting.

Table A. FID ↓ / KID×100 ↓ per environment map (Toys4K assets common to every map and method; best in bold, second best underlined):

Methodninomaru teien (FID↓ / KID↓)courtyard (FID↓ / KID↓)hotel room (FID↓ / KID↓)moonless golf (FID↓ / KID↓)spruit sunrise (FID↓ / KID↓)venice sunset (FID↓ / KID↓)
Luce GS25.95‾\underline{25.95} / 0.01726.96‾\underline{26.96} / 0.01427.37‾\underline{27.37} / 0.03226.33‾\underline{26.33} / 0.01826.35‾\underline{26.35} / 0.01826.86‾\underline{26.86} / 0.022
Luce mesh25.74 / 0.027‾\underline{0.027}26.19 / 0.024‾\underline{0.024}26.45 / 0.035‾\underline{0.035}25.78 / 0.022‾\underline{0.022}25.46 / 0.029‾\underline{0.029}26.09 / 0.031‾\underline{0.031}
TRELLIS 234.46 / 0.14034.58 / 0.13134.03 / 0.13033.55 / 0.13733.21 / 0.15133.83 / 0.119
3DTopia-XL89.09 / 2.4189.26 / 2.5988.62 / 2.4891.37 / 2.6686.63 / 2.3191.52 / 2.59

Table B. Mean over the six maps (best in bold, second best underlined):

MethodFID ↓KID×100 ↓FIDdino ↓CLIP ↑SigLIP2 ↑
Luce GS26.64‾\underline{26.64}0.0200.113‾\underline{0.113}0.903‾\underline{0.903}0.925‾\underline{0.925}
Luce mesh25.950.028‾\underline{0.028}0.0990.9050.930
TRELLIS 233.940.1350.1420.8880.919
3DTopia-XL89.422.5070.5680.7710.788

Under every map, both Luce outputs lead TRELLIS 2, by 6.7–8.7 FID with 3.7–9.4× lower KID, and Luce leads on all five metrics averaged over the maps; on the per-map FID and KID, the method ranking is the same under all six maps. Luce's Gaussian FID varies by only 1.4 across maps (25.95–27.37), so its lead holds under all six lighting conditions.

 

Q4 / W3: Matched conditioning resolution and fusion capacity

Multi-layer Luce and the baselines are from Table 1; the other rows are scored on the same Toys4K assets and AI images (Toys4K FID, KID and CLIP; AI-image CLIP in the last column; best in bold, second best underlined):

FlowFID ↓KID×100 ↓CLIP ↑AI CLIP ↑
Luce, multi-layer (Table 1)20.990.0330.9060.852
Luce, single-layer (re-trained, no projector)24.390.0760.9000.814
Luce, single-layer (re-trained) + matched projector23.550.0550.8990.819
Luce at 518 × 518 input conditioning22.14‾\underline{22.14}0.051‾\underline{0.051}0.904‾\underline{0.904}0.839‾\underline{0.839}
TRELLIS 2 (Table 1)29.220.1650.8900.811
TRELLIS GS (Table 1)30.750.2120.8900.830
  • Resolution. Fine-tuned to 518 × 518 input conditioning, Luce scores 22.14 FID (vs. 20.99 at 1036 × 1036), so resolution explains little of its margin over TRELLIS 2 (29.22).
  • Fusion projector. The re-trained single-layer flow scores 24.39 FID. With a matched 10.5M-parameter projector, it scores 23.55. Multi-layer Luce scores 20.99.
  • Margin over baselines. Even single-layer Luce (24.39) scores below TRELLIS 2 (29.22) and TRELLIS GS (30.75).

 

Q5: Quantitative text preservation (OCR)

We agree that a quantitative text measure would be valuable, but OCR on renders of generated 3D assets is hard to make reliable: scores depend strongly on viewpoint, surface curvature, font and render resolution, and few Toys4K assets carry text. We therefore support this claim qualitatively, with many text-bearing examples: Figs. 1, 2, 4 and 5, and the per-sample comparisons against every baseline in Appendix G (Figs. 11–18).

If these results address the concerns raised in the review, we would be grateful if the assessment could be reconsidered; we welcome further questions.

Reviewer_DL8FOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Luce, a generative 3D framework for single-image relightable 3D asset generation. Its core design constructs three completely independent sets of Gaussians under a sparse voxel grid, modeling three categories of PBR material attributes respectively: albedo, metallic-roughness, and surface normals. It compresses the multimodal Gaussian cloud into a compact latent space via a structured latent variational autoencoder (SLatVAE), and achieves end-to-end single-image 3D generation paired with a rectified-flow transformer and multi-layer DINOv2 image conditioning. The model supports dual-path decoding: it can directly output Gaussian clouds that support relighting via standard PBR deferred shading, or export standard PBR mesh assets with tangent-space normal maps. Experiments achieve state-of-the-art generation and reconstruction performance on benchmarks such as Toys4K, with notable advantages in preserving high-frequency surface details such as text and logos.

Critical strengths
  1. This work proposes a three-set independent modal Gaussian scheme, where albedo, metallic-roughness, and normals each have independent geometric parameters (position, scale, rotation, opacity). This allows each material modality to adaptively adjust Gaussian density according to its own detail distribution. Combined with the design of baking normal Gaussians into tangent-space normal maps, the representation can carry high-frequency surface details beyond the voxel resolution, and performs outstandingly in scenarios such as fine textures and embossed text.
  2. The paper fully implements the complete pipeline of "multimodal Gaussian representation – structured latent space compression – flow-based generation – standard PBR deferred shading". The output PBR material parameters fully comply with industrial rendering standards and can be directly integrated into mainstream game and film rendering pipelines, delivering stronger deployment value compared with neural rendering-based relighting solutions.
  3. On both generation and reconstruction tasks of the Toys4K dataset, multiple core metrics significantly outperform strong baselines such as TRELLIS 2. Meanwhile, ablation studies on multi-layer image conditioning and normal baking are provided, which clearly verify the effectiveness of each key module, and the experimental results are highly credible.
Critical weaknesses
  1. Insufficient justification for the core representation and missing engineering cost analysis Three sets of fully independent Gaussians are the most core design difference of this work. However, the paper does not provide ablation comparison with a lighter baseline of "shared geometric base with independent material attribute channels", so it fails to quantify the specific performance gain brought by fully independent geometric parameters compared with the shared-geometry scheme. Meanwhile, the paper does not report engineering metrics such as total Gaussian count, video memory footprint, and rendering frame rate. The storage and computational overhead brought by the three-set design remain unclear, and the cost-effectiveness of the scheme cannot be evaluated.
  2. Incomplete baseline coverage and undiscussed boundaries between technical routes The paper does not cover concurrent relightable Gaussian generation works in the same track, such as NeAR. Both belong to the direction of single-image relightable 3D Gaussian generation, but adopt distinct technical routes: NeAR adopts the design of "single-set geometric Gaussians coupled with a neural renderer", focusing on material disentanglement under unknown lighting input; this work adopts three sets of independent modal Gaussians with a pure standard analytic PBR pipeline, focusing on industrial compatibility and detail fidelity. The paper does not compare or discuss the applicable boundaries, performance and efficiency differences between the two technical routes, which fails to fully demonstrate the unique value of the proposed scheme.
  3. Shortcomings in numerical accuracy of material attributes In reconstruction experiments, the PSNR of albedo and metallic-roughness of the proposed Gaussian scheme is inferior to the TRELLIS 2 baseline. It only leads in final shading quality and normal accuracy, and shows no advantage in the numerical fidelity of material attributes themselves.
Questions for authors
  1. Please supplement ablation experiments to compare the performance difference between the single-set PBR Gaussian scheme (shared geometric parameters with only independent material attributes) and the proposed three-set independent Gaussian scheme. Meanwhile, please quantify the total Gaussian count, video memory occupation and rendering speed of the two schemes, to demonstrate the necessity and engineering rationality of the three-set independent Gaussian design.
  2. Please clarify the core differences, respective applicable scenarios, and advantages-disadvantages boundaries between the proposed scheme of "three sets of independent modal Gaussians + pure analytic PBR deferred shading" and the route represented by NeAR, namely "single-set geometric Gaussians coupled with a neural renderer", to help readers better understand the trade-offs and value of the proposed design.
  3. The numerical accuracy of albedo and metallic-roughness in this work is inferior to that of TRELLIS 2. What is the main reason for this gap? Does the three-set Gaussian design have inherent limitations in the accuracy of material attributes, and is there room for further optimization?
3 comments
AuthorsOpenReview ↗

Response to Reviewer DL8F

Thank you for the detailed review and for highlighting the fine detail in Luce's outputs and their compatibility with PBR pipelines.
 

W1 / Q1: Shared geometry vs. three independent sets, and engineering cost

 

We designed Luce by locating fidelity bottlenecks stage by stage: first the input representation, then VAE reconstruction, then generation. We follow the same order here and compare Luce's three independent Gaussian sets with one shared set in which each Gaussian carries albedo, normal and metallic-roughness (MR). We test it at matched storage (17 shared Gaussians per occupied 128³ voxel: 340 floats vs. Luce's 336) and at matched Gaussian count (24 per voxel, i.e., three sets of eight). At matched storage, independent geometry is better on albedo and normals in Gaussian fitting, which produces the VAE's input representation (+0.9 / +0.1 dB), while shared geometry is slightly better on MR (−0.2 dB). After training, the independent-geometry VAE also reconstructs better on every modality (+1.3 dB albedo, +1.1 dB MR and +1.2 dB normal PSNR). The rendering cost stays real time.

1. Gaussian fits and engineering metrics (Toys4K, fit and scored on all 150 views; A100-80GB with gsplat; FPS and render peak: rasterizing one 1024² view with all three buffers; render peak = extra GPU memory):

RepresentationGaussiansStorage / render peak (MiB) ↓PassesFPS @1024² ↑Albedo PSNR ↑Normal PSNR ↑MR PSNR (metallic + roughness) ↑
Luce: 3 sets × 80.94M50 / 159325842.2837.9340.51
Shared, matched storage (17)0.67M51 / 133150741.4237.8140.74
Shared, matched count (24)0.94M72 / 169142841.6638.0240.97

2. VAE reconstruction PSNR (Table 2 protocol on Toys4K; all variants use the same 64³ × 32 latent; the count-matched shared decoder outputs 96 Gaussians per occupied latent voxel in total, matching Luce's three sets of 32):

VAEDecoded floats / 64³ voxelAlbedo PSNR ↑MR PSNR ↑Normal PSNR ↑Color PSNR ↑
Luce (independent)134439.7640.2436.8736.96
Shared, matched storage (67 / voxel)134038.4939.1735.7035.82
Shared, matched count (96 / voxel)1920 (+43%)39.5540.4836.6136.62

3. Generation. We trained a separate flow on the latents of each shared-geometry VAE. At matched storage, the shared model scored 22.79 FID, compared with 20.99 for Luce. At matched Gaussian count, it scored 22.22. KID×100 followed the same pattern (0.069 at matched storage and 0.059 at matched count, compared with 0.033 for Luce). In a separate experiment, we also trained a flow on the latents of a new VAE that uses independent Gaussian sets fitted at 256³, and this flow reached 20.31 FID.

Cost. At matched storage, rendering all three buffers takes three passes instead of one (258 vs. 507 FPS at 1024²), which stays real time. Luce prioritizes asset quality: in exchange, it gains 0.9 dB albedo PSNR in fitting and 1.3 dB after the VAE.

We will add these tables, with the full training details, to the appendix of the revised version.

AuthorsOpenReview ↗

Response to Reviewer DL8F (Part 2)

W2 / Q2: Relation to NeAR

Thank you for pointing us to NeAR (CVPR 2026). NeAR targets relighting from casually lit images: it predicts intrinsic material properties and renders the final appearance with a view- and illumination-conditioned neural renderer. Luce instead generates explicit PBR materials (Gaussians, or a mesh with tangent-space normal maps) that plug directly into standard renderers and production pipelines. We believe the two routes are complementary, since a learned renderer could be layered on top of Luce's PBR output. We are happy to add NeAR and this discussion to the related work and will update the draft accordingly.

 

W3 / Q3: Albedo and MR accuracy vs. TRELLIS 2

In the submitted Table 2 evaluation, we applied the alpha mask twice at silhouette pixels in Luce's Gaussian renders (TRELLIS 2's Blender renders are unaffected). Correcting this raises the 128³ model's MR PSNR from 39.1 to 40.24 dB, leaving a 2.5 dB gap to TRELLIS 2 (42.7 dB). Sharing Gaussian geometry does little to close this gap. Compared with Luce's three-set model, the shared model gains 0.24 dB in MR PSNR at matched Gaussian count and loses 1.07 dB at matched storage (W1). We therefore examined how much material detail survives VAE reconstruction.

1. The fits retain material detail. Under the four-view evaluation used in Table 2, Luce's 128³ Gaussian fits reach 44.1 dB on albedo and 46.1 dB on MR. Both exceed TRELLIS 2's reconstruction scores of 40.7 and 42.7 dB, respectively. After Luce's VAE reconstructs the 128³ fits, the scores fall to 39.76 dB on albedo and 40.24 dB on MR.

2. TRELLIS 2's encoder sees a higher-fidelity input: the asset's exact PBR textures voxelized at 1024³ (a median of about 2.4M voxels per asset), whereas Luce's VAE sees Gaussian fits on a 128³ grid. Even so, Luce's reconstruction keeps fine detail such as the TV's lettering (Fig. 6), and its albedo LPIPS is close to TRELLIS 2's (0.055 vs. 0.051, Table 2).

3. Ablation: a VAE on 256³ inputs. We trained a VAE on higher-resolution (256³) Gaussian fits, using four Gaussians per occupied voxel per modality and the same 64³ × 32 latent dimensions. This VAE is an ablation only; the paper's model is unchanged. It is also lighter than the paper's VAE (0.79B vs. 1.82B parameters; 64 output Gaussians per modality at each occupied latent voxel). Besides resolution, the 256³ fits carry about twice as many input Gaussians; the number of decoded Gaussians matters little (changing it moves PSNR by at most 0.25 dB, table (i) below).

Reconstruction on Toys4K (TRELLIS 2: the paper's Table 2; Luce rows: corrected evaluation on the same Toys4K assets):

ModelAlbedo PSNR↑ / SSIM↑ / LPIPS↓MR PSNR↑ / SSIM↑ / LPIPS↓Normal PSNR↑ / SSIM↑ / LPIPS↓Color PSNR↑ / SSIM↑ / LPIPS↓
TRELLIS 2 (Table 2)40.7 / 0.976 / 0.05142.7 / 0.986 / 0.03832.1 / 0.940 / 0.09334.5 / 0.961 / 0.074
Luce VAE, 128³ input (paper model)39.76 / 0.972 / 0.05440.24 / 0.978 / 0.04836.87 / 0.965 / 0.03736.96 / 0.968 / 0.057
Luce VAE, 256³ input40.53 / 0.976 / 0.04141.14 / 0.981 / 0.03437.27 / 0.968 / 0.02937.56 / 0.972 / 0.046
+ 3× MR loss weight40.36 / 0.974 / 0.04341.54 / 0.982 / 0.03137.15 / 0.966 / 0.03137.50 / 0.971 / 0.048

With a higher-fidelity input and the same latent size, Luce comes within 0.2 dB of TRELLIS 2's albedo PSNR (40.53 vs. 40.7), with equal SSIM (0.976) and lower LPIPS (0.041 vs. 0.051), while keeping large leads on normals (+5.2 dB) and shaded color (+3.1 dB); the 256³ input improves on the 128³ VAE in every modality (+0.8 dB albedo, +0.9 dB MR, +0.4 dB normal, +0.6 dB color). MR PSNR remains 1.6 dB lower (1.2 dB with a 3× MR loss weight, which costs 0.2 dB albedo), though MR LPIPS is lower (0.031–0.034 vs. 0.038). The 256³ configuration also changes the input Gaussian count and VAE architecture. At 128³, MR loses about 6 dB from fitting to reconstruction, compared with about 4 dB for albedo.

AuthorsOpenReview ↗

Response to Reviewer DL8F (Part 3)

W3 / Q3 (continued)

Other VAE ablations (Table 2 protocol; K and K′ are the input and decoded Gaussians per voxel and modality, as in Appendix B; in brackets, the change from the first row of each group):

(i) Output Gaussians per voxel, K′ (latent: 64³ grid × 32 channels):

Input fits (grid × K)K′Albedo PSNR ↑MR PSNR ↑Normal PSNR ↑
256³ × 46440.5341.1437.27
256³ × 43240.49 (−0.04)41.11 (−0.03)37.04 (−0.23)
256³ × 49640.60 (+0.07)41.15 (+0.01)37.10 (−0.17)
Input fits (grid × K)K′Albedo PSNR ↑MR PSNR ↑Normal PSNR ↑
128³ × 832 (Luce)39.7640.2436.87
128³ × 86439.85 (+0.09)40.25 (+0.01)36.74 (−0.13)

(ii) Latent channels (latent: 64³ grid):

Input fits (grid × K)ChannelsAlbedo PSNR ↑MR PSNR ↑Normal PSNR ↑
256³ × 43240.5341.1437.27
256³ × 46440.48 (−0.05)41.11 (−0.03)37.22 (−0.05)
Input fits (grid × K)ChannelsAlbedo PSNR ↑MR PSNR ↑Normal PSNR ↑
128³ × 832 (Luce)39.7640.2436.87
128³ × 86439.48 (−0.28)40.06 (−0.18)36.64 (−0.23)
128³ × 81638.67 (−1.09)39.11 (−1.13)36.14 (−0.73)

(iii) MR loss weight (256³ × 4 input):

MR loss weightAlbedo PSNR ↑MR PSNR ↑Normal PSNR ↑
1×40.5341.1437.27
3×40.36 (−0.17)41.54 (+0.40)37.15 (−0.12)
10×39.92 (−0.61)41.71 (+0.57)36.91 (−0.36)

(iv) Encoder pooling (128³ × 8 input):

PoolingAlbedo PSNR ↑MR PSNR ↑Normal PSNR ↑
After the encoder (Luce)39.7640.2436.87
Before the encoder36.67 (−3.09)37.92 (−2.32)35.12 (−1.75)

More output Gaussians or latent channels change PSNR by at most 0.3 dB, fewer channels hurt, a higher MR loss weight trades albedo for MR, and pooling before the encoder costs up to 3.1 dB.

Generation with the 256³ VAE. The reconstruction gain carries into generation. A flow trained on this VAE's latent reaches FID 20.31 vs. 20.99 for Luce (Table 1), with KID×100 0.013 vs. 0.033, FIDdino 0.087 vs. 0.100 and AI-image SigLIP2 0.860 vs. 0.851 (AI CLIP 0.852 for both).

We hope these analyses address the questions raised in the review, and we are happy to clarify any further point.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Response Summary

We thank all the reviewers for their careful and constructive feedback. Reviewers highlighted the quality of Luce's generated assets (DL8F, V1PS, h94U), its preservation of fine details such as lettering and logos (DL8F, mmBF, h94U), and the compatibility of its explicit PBR outputs with production pipelines (DL8F, mmBF).

All four reviewers asked what Luce gains by giving each material modality its own Gaussian geometry. Several also asked how much of the improvement in generation quality comes from image conditioning and model size. We summarize these results below and will include them in the revised version. The individual replies give the full comparisons.

 

  1. Independent vs. shared Gaussian geometry at three stages (DL8F, V1PS, mmBF, h94U). We compare Luce with two variants that use one shared set of Gaussians for all three PBR modalities. One matches Luce's representation storage and the other matches its total Gaussian count. At matched storage, independent geometry yields 0.86 dB higher albedo PSNR in fitting, while shared geometry yields 0.23 dB higher metallic-roughness (MR) PSNR. After VAE reconstruction, independent geometry improves PSNR for all three modalities at matched storage (1.27 dB for albedo, 1.07 dB for MR, and 1.17 dB for normals). In generation, Luce achieves 20.99 FID (Table 1), compared with 22.79 for shared geometry at matched storage and 22.22 at matched count.
  2. Effects of conditioning and flow size on generation (h94U, V1PS, mmBF). We evaluate Luce with single-layer conditioning (with and without a fusion projector), with 518 × 518 input images instead of 1036 × 1036, and with smaller flows (0.6B and 1.3B parameters). On Toys4K, the full model and these variants achieve FID scores of 21.0–25.2, all lower than TRELLIS 2 (29.2) and TRELLIS (30.8). In a separate comparison, fine-tuning TRELLIS on our data with multi-layer conditioning improves its FID from 30.8 to 27.4, still 6.4 points behind Luce.
  3. Cross-modality coverage near silhouettes (V1PS, mmBF, h94U). We measure the three-way intersection over union (IoU) of the rendered albedo, normal and MR coverage masks. On Toys4K, the three-way IoU is 0.9984 for the input fits, 0.9979 after VAE reconstruction and 0.9973 for generated assets. Within 10 pixels of the silhouette, generated assets retain high overlap on both Toys4K (0.987) and the AI-image benchmark (0.984).
  4. Relighting under six environment maps (V1PS). Luce leads TRELLIS 2 under every map with both its Gaussian and mesh outputs: FID is 20–25% lower (6.7–8.7 points), and Luce is also better on all five metrics averaged over the six maps.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.