- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Filigree3D, a single image to 3D geometry generator built on O-Voxel representation, SC-VAE, and the sparse flow matching DiT of TRELLIS.2 (Section 3.1, pp. 3-4). It targets an effective voxel resolution of 2048^3 and claims to be scalable to 4096^3. This method has four complements. Structure-aware sparse scaling (Section 3.2, pp. 4-5) combines occupancy-balanced core halo clipping under token budget B (Eq. 2-4), explicit clipping window tuning via temporal embedding injection (Eq. 5-6), and dual regular and shifted window attention with zero-gated coarse global attention paths per K blocks (Eq. 7-9). Other additions are patch-specific fusion of four DINO feature levels (Section 3.3, Equation 10) and a visibility weighted loss using a Laplacian term for invisible voxels (Section 3.4, Equations 11-12).
The resolution gain data management protocol (Section 3.5, page 6) filters assets based on the ratio of worst-case GT to reconstruction distance (1024^3 vs. 2048^3), followed by manual inspection. Appendix A.2 (p.13–14) also describes engineering fixes for 2048^3 and 4096^3 decoding: 64-bit voxel keys, and block-wise flex_gemm encoding using offset assembly.
Experiments (Section 4, pp. 6-9) compare with Direct3D-S2, Hunyuan3D 2.1, TRELLIS.2 and Pixal3D on a manually selected 537-object Toys4K subset and a new 120-example FineGeo3D set. Table 1 reports very large gains: CD 0.0055 compared to 0.0233 for the best baseline on Toys4K, and F1-0.01 0.9969 compared to 0.5847 on FineGeo3D. Table 2 presents four rows of component ablation, and Figure 5 provides qualitative resolution scaling and commercial system comparison.
Critical strengths
The extended design is coherent and well-motivated for budget-sparse training, fully supporting inference. Occupying the adaptive context radius s⋆ (Equation 3) effectively utilizes the fixed token budget. Limiting the core loss (Equation 4) avoids supervision near the boundaries of the artificial crop. Conditioning with actual cropping windows instead of occupying extrema (Equation 5, page 5) resolves the real ambiguity between partially supported training and full object inference. Zero-initialization gates (Equations 6, 9) allow for safe fine-tuning from a pre-trained TRELLIS.2-type backbone. The appendix engineering details (pages 13-14) are concrete and useful for practitioners scaling O-voxel pipelines: 32-bit key collisions result in long-side artifacts at 2048^3, and the 10-bit coordinate limit of flex_gemm processed by 4×4×4 block assembly.
The qualitative results are visually compelling and consistently show a clearer embossing, thinner structure, and less merging than the open-source baseline. This applies to Figures 1 (p.1), 3 (p.7), 4 (p.8), 6 (p.15), and 7 (p.16). Appendix A.2.1 (pages 14-15) details the metric protocol, covering visible surface ray projection, sample counting, chamfering, F-score definitions, and ULIP-2/Uni3D scoring. This improves the transparency of the digital calculation methods.
Critical weaknesses
The quantitative gains in Table 1 (page 7) are implausibly large and are not attributable to the claimed contributions. Ours-1536 has reduced CD from 0.0270 (TRELLIS.2, also reaches 1536 and shares the same backbone) to 0.0058, and improved F1-0.01 from 0.63 to 0.97. Our 2048 only added 0.0003 CD. So the improvement seems to come mostly from factors other than resolution. Reasonable sources include the 200K privately collected training assets (page 7, page 14), the retrained 128^3 sparse structure stage (page 14), the alignment process, or possible train/test overlap. "Similarity alignment" (page 15) is not specified. The origins and GT provenance of FineGeo3D are not described, nor are controls such as TRELLIS.2 provided for fine-tuning the same data.
Evidence for several core claims is scant or internally inconsistent. Table 2 (page 9) omits core halo clipping, clipping window conditioning, dual-window attention, and block-specific ablation with shared fusion. It also reports no seeds or differences. No peak memory numbers support summary memory claims (p.1). The 4096^3 claim is based on a pillar in Figure 5 (page 9), with no measure or statement as to whether DiTs are trained at that scale. The main text shows DINOv2 and K=5 (pp. 5, 7), while the appendix shows DINOv3 ViT-L/16 and K=6 (pp. 13-14). Curation includes non-repeatable manual checks, the number of curated assets is not reported, and λu and λlap are not given.
Questions for authors
Could you report a controlled comparison where the TRELLIS.2 model, or a model with all proposed components removed, was fine-tuned on the exact same training data (including 200,000 collected assets and a selected subset)? This would separate the contribution of the data from the proposed architecture and loss. Also, please confirm whether any Toys4K or FineGeo3D objects or nearly duplicated objects appeared in the training data.
What exactly is the “similarity alignment” used before calculating the CD and F scores (page 15)? Is the ICP scaled, is it applied in the same way to all methods, and is it run with the same initialization for each method? How was the Toys4K subset of 537 objects selected, and where did the FineGeo3D ground reality mesh and input images come from?
Which DINO version, layer set, and global margin K were actually used in the reported experiments (DINOv2 with K=5 on pages 5/7 vs. DINOv3 with K=6 on pages 13–14)? Could you provide details on ablation isolation core aura clipping, clipping window adjustment, specific block and shared fusion, and seed differences?
For 4096^3, was DiT trained or fine-tuned at that resolution? How many labels are involved in fully supporting inference? Please provide peak GPU memory and runtime measured at 1536^3, 2048^3, and 4096^3, and any quantitative fidelity results at 4096^3.
6 comments
Q5. Component ablations, seed variation, and data curation (Table 2, lines 432–443, and Appendix A.2, lines 792–801). Table 2 isolates crop-window conditioning, core–halo cropping, dual-window attention, and block-specific fusion. The core–halo control centers a single budget-matched crop on a uniformly sampled occupied voxel and computes the loss over all cropped tokens. The runs share one seed. The results show that all four individual changes reduce fine-detail fidelity.
| FineGeo3D setting | CD ↓ | [email protected] ↑ | [email protected] ↑ | ULIP-2 ↑ | Uni3D ↑ |
|---|---|---|---|---|---|
| Full | 0.0049 | 0.9969 | 0.4309 | 0.2760 | 0.3126 |
| Without crop-window conditioning | 0.0053 | 0.9854 | 0.4001 | 0.2600 | 0.3000 |
| Single budget-matched crop | 0.0058 | 0.9790 | 0.3974 | 0.2578 | 0.2987 |
| Single-window attention | 0.0055 | 0.9860 | 0.3950 | 0.2554 | 0.2983 |
| Shared fusion across blocks | 0.0053 | 0.9841 | 0.3999 | 0.2601 | 0.2998 |
We retained approximately 64,000 TRELLIS-500K assets and 200,000 private assets. The retention criterion was whether the reconstruction recovered a coherent feature that was missing or merged with nearby geometry at . A vision-language model classified matched local renderings as keep, reject, or uncertain. Human reviewers resolved uncertain cases and audited samples from both the retained and rejected groups. Section 4.3 and Table 2 of the revised manuscript detail the ablations. Appendix A.2 describes how we selected the training assets.
Q6. Scaling to and inference cost (Table 3, lines 486–493). At , we reuse the DiT trained at without further DiT training or fine-tuning. The SC-VAE is fine-tuned separately for each target resolution. Table 3 reports quality, peak GPU memory, and inference time on FineGeo3D. Token counts are the maximum occupied latent tokens observed during full-support inference.
| Resolution (FV) | CD ↓ | [email protected] ↑ | Max. latent tokens | Peak GPU memory (GB, training / inference) | Inference time (s) |
|---|---|---|---|---|---|
| 0.0054 | 0.3679 | ~100K | ~74 / ~10 | 50.9 | |
| 0.0049 | 0.4309 | ~130K | ~74 / ~15 | 84.2 | |
| 0.0045 | 0.5103 | ~340K | N/A / ~31 | 215.3 |
Geometry improves through with the same DiT. From to , [email protected] increases by 14.24 percentage points and CD falls from 0.0054 to 0.0045. The run takes 215.3 s and peaks at approximately 31 GB of inference GPU memory. Sections 4.1 and 4.3 and the SC-VAE paragraph in Appendix A.2 of the revised manuscript explain the resolution settings. Figure 5 and Table 3 give the qualitative and quantitative results.
Brief follow-up on our response
Dear Reviewer Ffrb,
Thank you again for asking us to clarify where the quantitative gains come from. The new TRELLIS.2 controls show that the fine-tuned sparse-structure model and same-data denoiser fine-tuning narrow much of the original CD gap. Filigree3D retains an advantage over the fine-tuned control at the same resolution, and Table 1 separately shows the gains from moving to .
We also documented how we constructed the evaluation sets, checked for training overlap, and applied the same automatic similarity ICP procedure across methods (Appendix A.2). The revised manuscript adds the requested component ablations and reports geometric fidelity, token counts, peak GPU memory, and runtime for inference using the DiT trained at (Tables 2 and 3).
As the discussion period draws to a close, we would appreciate your feedback on whether the new controls and evaluation details address your concerns.
Best regards,
The Authors
Response to Reviewer Ffrb
Thank you for asking us to separate the sources of the gains and clarify the evaluation and training protocols. We highlighted the revisions in blue in the revised manuscript.
Q1. Same-data TRELLIS.2 control and source of the gains (Table 1, lines 346–358). We have included additional comparative experiments in the revised manuscript. TRELLIS.2 (FV) pairs the pretrained TRELLIS.2 denoiser with predicted voxels from our fine-tuned sparse-structure model. TRELLIS.2† (FV) further fine-tunes that denoiser on the exact same training data as ours. It retains the TRELLIS.2 15363 inference cascade and uses the same VAE and fine-tuned sparse-structure model as Ours-1536 (FV).
The fine-tuned sparse-structure model and subsequent denoiser fine-tuning account for most of the original CD gap on both benchmarks. At the same 15363 output resolution, our method still lowers CD and improves [email protected] compared with TRELLIS.2† (FV). On Toys4K, increasing the resolution of our method from 15363 to 20483 lowers CD by 0.0003 (0.0058 to 0.0055) and raises [email protected] from 0.4616 to 0.5202. Section 4.2 and Table 1 of the revised manuscript describe the matched comparison and report the full results.
Q2. Evaluation sets and training-data overlap (Appendix A.2, lines 774–790). We checked the asset IDs of both evaluation sets against the TRELLIS-500K subset and the private collection used for fine-tuning and found no matching IDs. For geometric near-duplicate screening, we used a vision-language model to flag candidate matches from normal maps rendered at matched viewpoints after standardizing mesh orientation. We selected 537 complex, detailed single-object meshes from Toys4K by applying the screening procedure in Section 3.5 and then inspecting surface-normal renderings. The 120 FineGeo3D ground-truth meshes were collected online or commissioned from external 3D artists. We rendered each mesh from 16 viewpoints and selected one near-frontal rendering as input. Appendix A.2 of the revised manuscript details evaluation set construction and the dataset separation checks.
Q3. Similarity alignment before computing CD and F-scores (Appendix A.2.1, lines 860–863 and 893–916). We convert all meshes to a common coordinate convention and align each prediction to its ground-truth (GT) mesh using multi-start similarity iterative closest point (ICP). All methods use the same set of initial rotations and the same initialization procedure. We set the initial scale to the ratio of the GT to predicted 90th-percentile point-to-centroid distances, measured relative to each point cloud’s own centroid. For each initial rotation, we translate the scaled and rotated prediction so that the two centroids coincide. ICP refines the rotation, translation, and uniform scale. We retain the transform with the lowest bidirectional Chamfer distance (CD) on the registration point sets. We visually inspected the automatic alignments across a range of examples and found them satisfactory. All reported CD and F-scores were computed using the transforms returned by the ICP.
Q4. DINO configuration and visibility regularization (Section 4.1, lines 373–377, and Table 4, lines 756–762). The experiments use DINOv3 ViT-L/16 features from layers 5, 7, 11, and 23. The original manuscript incorrectly named DINOv2 as the experimental encoder. We corrected those references to DINOv3 and updated the main text to report the actual coarse-global communication interval of K=6 DiT blocks. The visibility regularization weights are λu=1.5 and λlap=0.1. Figure 2, Sections 3.3 and 4.1, Appendix A.1, and Table 4 of the revised manuscript give the model configuration and regularization settings.