acceptodds
Under review as a conference paper at ICLR 2027

FILIGREE3D: Scaling Sparse Latent Flow Matching for Ultra-High-Resolution Image-to-3D Generation

Abstract

Scaling image-to-3D generation to ultra-high resolutions requires controlling rapidly growing computational costs without sacrificing fine geometric detail. We present Filigree3D, a sparse latent flow-matching framework that generates 3D geometry from a single image at voxel resolutions up to , with straightforward extensibility to . To make training tractable, we introduce Structure-Aware Sparse Scaling, which combines spatial bounding with alternating local-global attention to constrain token growth while preserving both fine-scale details and long-range structural context. To enhance detail reconstruction, we curate training samples based on their high-resolution geometric gains and inject multi-scale image features into a sparse 3D DiT, effectively coupling structural semantics with fine-grained visual cues. Furthermore, a visibility-aware voxel regularization strategy improves robustness against sparse perturbations and facilitates the completion of unobserved geometry. Under our default configuration, Filigree3D maintains peak GPU memory consumption within practical limits for contemporary hardware, enabling the generation of highly intricate 3D geometry in approximately one minute. Extensive experiments demonstrate that our method yields substantial improvements in overall geometric fidelity and fine-detail preservation compared to existing baselines, validating practical, detail-preserving 3D generation at unprecedented resolutions.

Primary area:
generative models
Keywords:
Image-to-3D Generation, Ultra-High-Resolution 3D Generation;Sparse 3D Representations, Flow Matching, Geometric Detail Preservation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_FfrbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Filigree3D, a single image to 3D geometry generator built on O-Voxel representation, SC-VAE, and the sparse flow matching DiT of TRELLIS.2 (Section 3.1, pp. 3-4). It targets an effective voxel resolution of 2048^3 and claims to be scalable to 4096^3. This method has four complements. Structure-aware sparse scaling (Section 3.2, pp. 4-5) combines occupancy-balanced core halo clipping under token budget B (Eq. 2-4), explicit clipping window tuning via temporal embedding injection (Eq. 5-6), and dual regular and shifted window attention with zero-gated coarse global attention paths per K blocks (Eq. 7-9). Other additions are patch-specific fusion of four DINO feature levels (Section 3.3, Equation 10) and a visibility weighted loss using a Laplacian term for invisible voxels (Section 3.4, Equations 11-12).

The resolution gain data management protocol (Section 3.5, page 6) filters assets based on the ratio of worst-case GT to reconstruction distance (1024^3 vs. 2048^3), followed by manual inspection. Appendix A.2 (p.13–14) also describes engineering fixes for 2048^3 and 4096^3 decoding: 64-bit voxel keys, and block-wise flex_gemm encoding using offset assembly.

Experiments (Section 4, pp. 6-9) compare with Direct3D-S2, Hunyuan3D 2.1, TRELLIS.2 and Pixal3D on a manually selected 537-object Toys4K subset and a new 120-example FineGeo3D set. Table 1 reports very large gains: CD 0.0055 compared to 0.0233 for the best baseline on Toys4K, and F1-0.01 0.9969 compared to 0.5847 on FineGeo3D. Table 2 presents four rows of component ablation, and Figure 5 provides qualitative resolution scaling and commercial system comparison.

Critical strengths

The extended design is coherent and well-motivated for budget-sparse training, fully supporting inference. Occupying the adaptive context radius s⋆ (Equation 3) effectively utilizes the fixed token budget. Limiting the core loss (Equation 4) avoids supervision near the boundaries of the artificial crop. Conditioning with actual cropping windows instead of occupying extrema (Equation 5, page 5) resolves the real ambiguity between partially supported training and full object inference. Zero-initialization gates (Equations 6, 9) allow for safe fine-tuning from a pre-trained TRELLIS.2-type backbone. The appendix engineering details (pages 13-14) are concrete and useful for practitioners scaling O-voxel pipelines: 32-bit key collisions result in long-side artifacts at 2048^3, and the 10-bit coordinate limit of flex_gemm processed by 4×4×4 block assembly.

The qualitative results are visually compelling and consistently show a clearer embossing, thinner structure, and less merging than the open-source baseline. This applies to Figures 1 (p.1), 3 (p.7), 4 (p.8), 6 (p.15), and 7 (p.16). Appendix A.2.1 (pages 14-15) details the metric protocol, covering visible surface ray projection, sample counting, chamfering, F-score definitions, and ULIP-2/Uni3D scoring. This improves the transparency of the digital calculation methods.

Critical weaknesses

The quantitative gains in Table 1 (page 7) are implausibly large and are not attributable to the claimed contributions. Ours-1536 has reduced CD from 0.0270 (TRELLIS.2, also reaches 1536 and shares the same backbone) to 0.0058, and improved F1-0.01 from 0.63 to 0.97. Our 2048 only added 0.0003 CD. So the improvement seems to come mostly from factors other than resolution. Reasonable sources include the 200K privately collected training assets (page 7, page 14), the retrained 128^3 sparse structure stage (page 14), the alignment process, or possible train/test overlap. "Similarity alignment" (page 15) is not specified. The origins and GT provenance of FineGeo3D are not described, nor are controls such as TRELLIS.2 provided for fine-tuning the same data.

Evidence for several core claims is scant or internally inconsistent. Table 2 (page 9) omits core halo clipping, clipping window conditioning, dual-window attention, and block-specific ablation with shared fusion. It also reports no seeds or differences. No peak memory numbers support summary memory claims (p.1). The 4096^3 claim is based on a pillar in Figure 5 (page 9), with no measure or statement as to whether DiTs are trained at that scale. The main text shows DINOv2 and K=5 (pp. 5, 7), while the appendix shows DINOv3 ViT-L/16 and K=6 (pp. 13-14). Curation includes non-repeatable manual checks, the number of curated assets is not reported, and λu and λlap are not given.

Questions for authors

Could you report a controlled comparison where the TRELLIS.2 model, or a model with all proposed components removed, was fine-tuned on the exact same training data (including 200,000 collected assets and a selected subset)? This would separate the contribution of the data from the proposed architecture and loss. Also, please confirm whether any Toys4K or FineGeo3D objects or nearly duplicated objects appeared in the training data.

What exactly is the “similarity alignment” used before calculating the CD and F scores (page 15)? Is the ICP scaled, is it applied in the same way to all methods, and is it run with the same initialization for each method? How was the Toys4K subset of 537 objects selected, and where did the FineGeo3D ground reality mesh and input images come from?

Which DINO version, layer set, and global margin K were actually used in the reported experiments (DINOv2 with K=5 on pages 5/7 vs. DINOv3 with K=6 on pages 13–14)? Could you provide details on ablation isolation core aura clipping, clipping window adjustment, specific block and shared fusion, and seed differences?

For 4096^3, was DiT trained or fine-tuned at that resolution? How many labels are involved in fully supporting inference? Please provide peak GPU memory and runtime measured at 1536^3, 2048^3, and 4096^3, and any quantitative fidelity results at 4096^3.

6 comments

Response to Reviewer Ffrb

Thank you for asking us to separate the sources of the gains and clarify the evaluation and training protocols. We highlighted the revisions in blue in the revised manuscript.

Q1. Same-data TRELLIS.2 control and source of the gains (Table 1, lines 346–358). We have included additional comparative experiments in the revised manuscript. TRELLIS.2 (FV) pairs the pretrained TRELLIS.2 denoiser with predicted voxels from our fine-tuned sparse-structure model. TRELLIS.2† (FV) further fine-tunes that denoiser on the exact same training data as ours. It retains the TRELLIS.2 153631536^3 inference cascade and uses the same VAE and fine-tuned sparse-structure model as Ours-1536 (FV).

153631536^3 methodToys4K CD ↓Toys4K [email protected] ↑FineGeo3D CD ↓FineGeo3D [email protected] ↑
TRELLIS.20.02700.09470.02740.0639
TRELLIS.2 (FV)0.00920.21960.00960.1556
TRELLIS.2† (FV)0.00720.32700.00770.2658
Ours-1536 (FV)0.00580.46160.00540.3679

The fine-tuned sparse-structure model and subsequent denoiser fine-tuning account for most of the original CD gap on both benchmarks. At the same 153631536^3 output resolution, our method still lowers CD and improves [email protected] compared with TRELLIS.2† (FV). On Toys4K, increasing the resolution of our method from 153631536^3 to 204832048^3 lowers CD by 0.0003 (0.0058 to 0.0055) and raises [email protected] from 0.4616 to 0.5202. Section 4.2 and Table 1 of the revised manuscript describe the matched comparison and report the full results.

Q2. Evaluation sets and training-data overlap (Appendix A.2, lines 774–790). We checked the asset IDs of both evaluation sets against the TRELLIS-500K subset and the private collection used for fine-tuning and found no matching IDs. For geometric near-duplicate screening, we used a vision-language model to flag candidate matches from normal maps rendered at matched viewpoints after standardizing mesh orientation. We selected 537 complex, detailed single-object meshes from Toys4K by applying the screening procedure in Section 3.5 and then inspecting surface-normal renderings. The 120 FineGeo3D ground-truth meshes were collected online or commissioned from external 3D artists. We rendered each mesh from 16 viewpoints and selected one near-frontal rendering as input. Appendix A.2 of the revised manuscript details evaluation set construction and the dataset separation checks.

Q3. Similarity alignment before computing CD and F-scores (Appendix A.2.1, lines 860–863 and 893–916). We convert all meshes to a common coordinate convention and align each prediction to its ground-truth (GT) mesh using multi-start similarity iterative closest point (ICP). All methods use the same set of initial rotations and the same initialization procedure. We set the initial scale to the ratio of the GT to predicted 90th-percentile point-to-centroid distances, measured relative to each point cloud’s own centroid. For each initial rotation, we translate the scaled and rotated prediction so that the two centroids coincide. ICP refines the rotation, translation, and uniform scale. We retain the transform with the lowest bidirectional Chamfer distance (CD) on the registration point sets. We visually inspected the automatic alignments across a range of examples and found them satisfactory. All reported CD and F-scores were computed using the transforms returned by the ICP.

Q4. DINO configuration and visibility regularization (Section 4.1, lines 373–377, and Table 4, lines 756–762). The experiments use DINOv3 ViT-L/16 features from layers 5, 7, 11, and 23. The original manuscript incorrectly named DINOv2 as the experimental encoder. We corrected those references to DINOv3 and updated the main text to report the actual coarse-global communication interval of K=6K=6 DiT blocks. The visibility regularization weights are λu=1.5\lambda_u=1.5 and λlap=0.1\lambda_{\mathrm{lap}}=0.1. Figure 2, Sections 3.3 and 4.1, Appendix A.1, and Table 4 of the revised manuscript give the model configuration and regularization settings.

Q5. Component ablations, seed variation, and data curation (Table 2, lines 432–443, and Appendix A.2, lines 792–801). Table 2 isolates crop-window conditioning, core–halo cropping, dual-window attention, and block-specific fusion. The core–halo control centers a single budget-matched crop on a uniformly sampled occupied voxel and computes the loss over all cropped tokens. The runs share one seed. The results show that all four individual changes reduce fine-detail fidelity.

FineGeo3D settingCD ↓[email protected] ↑[email protected] ↑ULIP-2 ↑Uni3D ↑
Full0.00490.99690.43090.27600.3126
Without crop-window conditioning0.00530.98540.40010.26000.3000
Single budget-matched crop0.00580.97900.39740.25780.2987
Single-window attention0.00550.98600.39500.25540.2983
Shared fusion across blocks0.00530.98410.39990.26010.2998

We retained approximately 64,000 TRELLIS-500K assets and 200,000 private assets. The retention criterion was whether the 204832048^3 reconstruction recovered a coherent feature that was missing or merged with nearby geometry at 102431024^3. A vision-language model classified matched local renderings as keep, reject, or uncertain. Human reviewers resolved uncertain cases and audited samples from both the retained and rejected groups. Section 4.3 and Table 2 of the revised manuscript detail the ablations. Appendix A.2 describes how we selected the training assets.

Q6. Scaling to 409634096^3 and inference cost (Table 3, lines 486–493). At 409634096^3, we reuse the DiT trained at 204832048^3 without further DiT training or fine-tuning. The SC-VAE is fine-tuned separately for each target resolution. Table 3 reports quality, peak GPU memory, and inference time on FineGeo3D. Token counts are the maximum occupied latent tokens observed during full-support inference.

Resolution (FV)CD ↓[email protected] ↑Max. latent tokensPeak GPU memory (GB, training / inference)Inference time (s)
153631536^30.00540.3679~100K~74 / ~1050.9
204832048^30.00490.4309~130K~74 / ~1584.2
409634096^30.00450.5103~340KN/A / ~31215.3

Geometry improves through 409634096^3 with the same 204832048^3 DiT. From 153631536^3 to 409634096^3, [email protected] increases by 14.24 percentage points and CD falls from 0.0054 to 0.0045. The 409634096^3 run takes 215.3 s and peaks at approximately 31 GB of inference GPU memory. Sections 4.1 and 4.3 and the SC-VAE paragraph in Appendix A.2 of the revised manuscript explain the resolution settings. Figure 5 and Table 3 give the qualitative and quantitative results.

AuthorsOpenReview ↗

Brief follow-up on our response

Dear Reviewer Ffrb,

Thank you again for asking us to clarify where the quantitative gains come from. The new TRELLIS.2 controls show that the fine-tuned sparse-structure model and same-data denoiser fine-tuning narrow much of the original CD gap. Filigree3D retains an advantage over the fine-tuned control at the same 153631536^3 resolution, and Table 1 separately shows the gains from moving to 204832048^3.

We also documented how we constructed the evaluation sets, checked for training overlap, and applied the same automatic similarity ICP procedure across methods (Appendix A.2). The revised manuscript adds the requested component ablations and reports geometric fidelity, token counts, peak GPU memory, and runtime for 409634096^3 inference using the DiT trained at 204832048^3 (Tables 2 and 3).

As the discussion period draws to a close, we would appreciate your feedback on whether the new controls and evaluation details address your concerns.

Best regards,
The Authors

Reviewer_Qi2JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

FILIGREE3D extends TRELLIS.2-based sparse latent flow matching for single-image 3D geometry generation at 2048³, with a qualitative demonstration at 4096³. It combines occupancy-balanced core-halo cropping, crop-window conditioning, local attention with periodic coarse-global communication, multi-level image features, visibility-aware regularization, and resolution-gain-based data curation. Evaluation on 537 selected Toys4K objects and 120 FineGeo3D cases reports substantial geometric improvements. The visual results are promising, but controlled comparisons and resource measurements are insufficient to establish the source of the gains and practical scalability.

Critical strengths
  1. The method addresses a practical scaling bottleneck coherently. Core-halo cropping preserves context around supervised tokens, crop-window conditioning distinguishes partial support from full support, and coarse-global attention enables communication beyond local windows (Section 3.2).
  2. Figures 3–6 show sharper relief and better separated thin structures than the displayed baselines. Table 1 reports strong gains on two evaluation sets, including F1-0.002 of 0.5202 and 0.4309 at 2048³. Table 2 provides supporting evidence that all four tested components improve performance.
Critical weaknesses
  1. The large gains are not adequately isolated from changes in data and upstream models. The system uses an additional 200K objects and fine-tunes resolution-specific VAEs and the sparse-structure model (Section 4.1 and Appendix A.2), but lacks a TRELLIS.2-based control with matched data, training budget, and representation settings. The “w/o DC” ablation does not explain whether data quantity and training exposure are controlled. Test-set construction and separation from training sources are also insufficiently documented. Appendix A.2.1 mentions similarity alignment without specifying the procedure, defines two Chamfer variants without identifying the reported variant, and selects faces using 20,000 rays without testing sensitivity to surface coverage. These omissions limit confidence in the interpretation of Table 1; they do not establish data overlap or incorrect evaluation.
  2. The central scalability claim lacks direct measurements. The 50,000-token budget bounds training crops, while inference processes full support and global attention still depends on occupied coarse-token count. No peak-memory or token-count measurements are reported. Table 1 gives 84.2 seconds at 2048³, but the 4096³ claim rests on a narrow qualitative example without runtime, memory, or quantitative quality results. Table 2 does not isolate core-halo cropping or crop-window conditioning. Reproducibility is further limited by DINOv2 versus DINOv3 and five-block versus six-block global-communication intervals in the main text and appendix.
Questions for authors
  1. How does a TRELLIS.2-based control perform with matched training data, training budget, target resolution, VAE, and sparse-structure settings? What exactly changes in the “w/o DC” ablation?
  2. How were the evaluation sets constructed and checked for overlap or near-duplicates with both training collections? Which similarity-alignment procedure and Chamfer variant were used? Are the geometric rankings stable with denser ray coverage or complete-mesh evaluation?
  3. What are the training and inference peak memory, token counts, runtime, and failure rates at 1536³, 2048³, and 4096³? What is the effect of replacing core-halo cropping and crop-window conditioning with a simpler matched crop strategy?
  4. Which image encoder and global-communication interval produced Tables 1–2? Please specify the visibility-regularization weights, perturbation and masking strengths, and the final training objective.
4 comments

Response to Reviewer Qi2J

Thank you for requesting stronger controls and cost measurements. We highlighted the revisions in blue in the revised manuscript.

Q1. TRELLIS.2 control and data-curation ablation (Tables 1–2, lines 346–358 and 432–443). We have included additional comparative experiments in the revised manuscript. TRELLIS.2 (FV) pairs the pretrained TRELLIS.2 denoiser with predicted voxels from our fine-tuned sparse-structure model. TRELLIS.2† (FV) further fine-tunes that denoiser on the exact same training data as ours. It retains the TRELLIS.2 153631536^3 inference cascade and uses the same VAE and fine-tuned sparse-structure model as Ours-1536 (FV).

153631536^3 methodToys4K CD ↓Toys4K [email protected] ↑FineGeo3D CD ↓FineGeo3D [email protected] ↑
TRELLIS.20.02700.09470.02740.0639
TRELLIS.2 (FV)0.00920.21960.00960.1556
TRELLIS.2† (FV)0.00720.32700.00770.2658
Ours-1536 (FV)0.00580.46160.00540.3679

Incorporating voxel support and fine-tuning on our data both narrow the gap on both benchmarks. Yet our method still outperforms the strongest fine-tuned control by 10.21 percentage points in FineGeo3D [email protected].

In the data-curation ablation, we replace the curated assets with an equal number of randomly sampled assets while keeping the training schedule fixed. [email protected] drops from 0.4309 to 0.3941 (Table 2). Sections 4.2 and 4.3 of the revised manuscript describe the comparison and ablation setups. Tables 1 and 2 report the full results.

Q2. Evaluation sets, alignment, and ray sensitivity (Appendix A.2, lines 774–790, and Table 5, lines 918–933). We selected 537 detailed single-object meshes from Toys4K by applying the screening procedure in Section 3.5 and inspecting surface-normal renderings. FineGeo3D contains 120 online or commissioned meshes. We rendered each mesh from 16 viewpoints and chose one near-frontal rendering as input.

We checked the asset IDs of both evaluation sets against the TRELLIS-500K subset and the private collection used for fine-tuning and found no matching IDs. For geometric near-duplicate screening, we used a vision-language model to flag candidate matches from normal maps rendered at matched viewpoints after standardizing mesh orientation.

All methods use automatic multi-start similarity ICP with the same initial rotations and initialization rule. It fits a single uniform scale, a rotation, and a translation. Scale and translation are initialized separately for each prediction from the 90th-percentile point-to-centroid distances and the centroids, respectively. CD and F-scores use the ICP transforms without manual adjustment. Table 1 reports CDℓ1\mathrm{CD}_{\ell_1} on visible surfaces, the sum of the two directional mean unsquared nearest-neighbor distances in Equation (15).

The three tested configurations retain the same geometric ranking with 20K, 50K, and 100K Fibonacci-sphere rays and with complete meshes. The complete-mesh protocol lowers the absolute [email protected] values. Selected values from Table 5 are shown below.

FineGeo3D [email protected] ↑20K rays50K rays100K raysComplete mesh
TRELLIS.2 (FV)0.15560.15510.15450.1350
Ours-1536 (FV)0.36790.36720.36590.3168
Ours-2048 (FV)0.43090.42990.42860.3736

The method rankings remain identical for all metrics, and the absolute values are nearly unchanged. These results confirm that our default 20K-ray visible-surface sampling is both stable and geometrically meaningful. Appendix A.2 of the revised manuscript details dataset construction and separation. Appendix A.2.1 explains the metric protocol, and Table 5 gives the full sensitivity results.

Q3. Scalability and cropping (Tables 2–3, lines 432–443 and 486–493). The 50K-token cap applies only to training crops; inference uses the full predicted voxel support. At 409634096^3, we reuse the 204832048^3 DiT together with a resolution-specific SC-VAE. Figure 5 (lines 445–464) expands the resolution comparison to additional objects, and Figure 6 (lines 495–512) presents the commercial comparisons separately. Table 3 reports quality and cost on FineGeo3D.

Resolution (FV)CD ↓[email protected] ↑Max. latent tokensPeak GPU memory (GB, training / inference)Inference time (s)
153631536^30.00540.3679~100K~74 / ~1050.9
204832048^30.00490.4309~130K~74 / ~1584.2
409634096^30.00450.5103~340KN/A / ~31215.3

Full-support inference improves fine-detail fidelity through 409634096^3 at an acceptable runtime and memory cost. Relative to 204832048^3, [email protected] increases by 7.94 percentage points while inference time grows by roughly 2.6×2.6\times, with peak inference memory around 31 GB. All evaluations completed without execution failure.

The two crop ablations in Table 2 isolate each design choice independently. Introducing a budget-matched cube centered on a random occupied voxel, with loss computed over all cropped tokens, reduces [email protected] from 0.4309 to 0.3974. Separately, removing crop-window conditioning while retaining core–halo cropping lowers it to 0.4001. Sections 4.1 and 4.3 of the revised manuscript explain the resolution settings and cropping controls. Appendix A.2 describes the resolution-specific SC-VAE. Tables 2 and 3 give the full ablation and resource results.

Q4. Encoder, communication interval, and training objective (Table 4, lines 756–762, and Appendix A.2, lines 848–853). Our Filigree3D runs use DINOv3 ViT-L/16, layers 5, 7, 11, and 23, and coarse-global communication every six blocks (K=6K=6). The original manuscript incorrectly named DINOv2 as the experimental encoder. We fixed the DINOv2 typo in Figure 2 (lines 162–181) and the main text. “Every five” referred to the intervening blocks and misstated the interval.

Equation (12) is the final training objective. It weights unseen voxels by λu=1.5\lambda_u=1.5 in velocity regression and the Laplacian term by λlap=0.1\lambda_{\mathrm{lap}}=0.1. Voxel coordinates stay fixed. The Gaussian noise standard deviation is 1.0 for visible voxels and 1.5 for unseen voxels. We zero unseen input features with probability 0.10 and exclude masked voxels from velocity regression, encouraging the model to use surrounding structure and the conditioning image. Sections 3.3 and 4.1 and Table 4 of the revised manuscript give the encoder and communication settings. Section 3.4 and Appendix A.2 detail the training objective and perturbation settings.

AuthorsOpenReview ↗

Brief follow-up on our response

Dear Reviewer Qi2J,

Thank you again for your detailed review. We added a same-data TRELLIS.2 fine-tuning control at 153631536^3, using the same VAE and fine-tuned sparse-structure model as Filigree3D. We also clarified how the data-curation ablation keeps the number of training assets fixed (Tables 1 and 2).

We expanded the evaluation protocol to describe the dataset overlap checks, automatic similarity alignment, and Chamfer distance variant. The new ray-sensitivity results show that the ranking of the three tested configurations holds with denser ray sampling and with complete-mesh evaluation (Appendix A.2 and Table 5). We also report quality, peak GPU memory, and runtime at all three resolutions and detail the visibility-aware training objective (Table 3, Section 3.4, and Appendix A.2).

As the discussion period draws to a close, we would be grateful to hear whether these additions address your concerns or whether any point would benefit from further clarification.

Best regards,
The Authors

Dear Reviewer Qi2J,

Thank you for your thoughtful review and constructive feedback. We have provided detailed responses and additional experiments to address each of your concerns. As the discussion period is drawing to a close, we would be grateful to know whether any points remain unclear or would benefit from further clarification. If there are additional experiments or analyses that would help update your assessment, we are happy to provide them.

Best regards, The Authors

Reviewer_xTJ5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents Filigree3D, a sparse latent flow-matching framework for single-image 3D geometry generation at resolutions up to 204832048^3, with extensions to 409634096^3. Building on TRELLIS.2, the method combines token-budgeted core–halo cropping, explicit crop-window conditioning, and local attention with periodic coarse-grained global communication to support high-resolution training and full-object inference. It further introduces block-specific multi-level image conditioning, visibility-aware regularization, and resolution-gain-aware data curation to improve geometric detail and completion of unobserved regions.

Critical strengths

The paper has three main strengths:

  • The core–halo cropping strategy provides a practical approach to learning from complex, high-resolution meshes. Its adaptive spatial extent controls computational cost, while the surrounding context reduces the impact of artificial crop boundaries.
  • The visibility-aware regularization is an interesting design that introduces view-dependent supervision while retaining the model’s existing 3D coordinate frame. Compared with previous work's explicit feature back-projection, it offers an indirect way to encourage image-consistent geometry and plausible completion of unobserved regions.
  • The data curation is well motivated and closely coupled to the learned representation. By comparing O-Voxel and SC-VAE reconstructions across resolutions, it prioritizes assets whose geometric details genuinely benefit from higher-resolution modeling, aligning training-data selection with the representation’s reconstruction capabilities.
Critical weaknesses

The paper has two main limitations:

  • The core–halo cropping strategy involves several important hyperparameters, including the halo width, the coarse-grid cell size used for anchor sampling, and the token budget. However, their sensitivity is not evaluated, leaving it unclear how robust the chosen configuration is across different resolutions and geometric complexities.
  • The crop-window conditioning is not independently ablated. Although it is intended to bridge cropped training and full-support inference, the current experiments do not isolate its contribution, making it difficult to assess whether explicitly encoding the crop bounds provides a meaningful benefit.
Questions for authors

Two implementation details require clarification:

  • Sections 3.3 and 4.1, as well as Figure 2, describe DINOv2 as the image encoder, whereas Appendix A.1 and Table 3 specify DINOv3 ViT-L/16. The released TRELLIS.2 model also uses DINOv3. Could the authors clarify which encoder and checkpoint were used in the reported experiments?
  • Section 4.1 states that global information flow is applied every five Transformer blocks, whereas Appendix A.1 and Table 3 specify an interval of six blocks. Which setting was used for the reported results?
3 comments

Response to Reviewer xTJ5

Thank you for highlighting where the evidence for core–halo cropping and crop-window conditioning could be stronger. We added the requested experiments and marked the revisions in blue.

Q1. Core–halo cropping sensitivity (Table 2, lines 432–443). On FineGeo3D at 204832048^3, we varied one parameter at a time using a shared training seed. Here, gg is the anchor sampling grid width, hh is the halo width, and BB is the training crop token budget. The table shows how each setting affects geometric fidelity.

SettingCD ↓[email protected] ↑
Default (g,h,B)=(8,2,50,000)(g,h,B)=(8,2,50{,}000)0.00490.4309
g=4g=40.00490.4304
g=6g=60.00490.4307
g=10g=100.00500.4305
h=1h=10.00520.4300
h=4h=40.00560.4178
B=20,000B=20{,}0000.00620.3950
B=30,000B=30{,}0000.00540.4130
B=40,000B=40{,}0000.00500.4298

The default is robust to modest changes, with the largest drop in fine-detail fidelity at the smallest token budget. [email protected] changes by at most 0.0011 across the tested grid widths, with a narrower halo, or with a 40K budget. A wider halo (h=4h=4) lowers the score to 0.4178, and a 20K budget lowers it to 0.3950. The default settings support flexible resolutions ranging from 153631536^3 to 409634096^3, yielding improved geometry scores (Table 3, lines 486–493). Section 4.3 of the revised manuscript explains the sensitivity setup, and Table 2 reports the full results.

Q2. Crop-window conditioning (Table 2, lines 432–443). To isolate the effect of crop-window conditioning, we remove it while retaining core–halo cropping during training and full-support inference. On FineGeo3D, CD rises from 0.0049 to 0.0053 and [email protected] drops from 0.4309 to 0.4001. This supports the benefit of encoding the crop bounds. Section 4.3 and Table 2 of the revised manuscript give the crop-window ablation setup and results.

Q3. Image encoder and checkpoint (Table 4, lines 756–762). We used the same DINOv3 ViT-L/16 checkpoint as the released TRELLIS.2 pipeline (facebook/dinov3-vitl16-pretrain-lvd1689m) and extracted features from layers 5, 7, 11, and 23. The DINOv2 references were typographical errors. We corrected the encoder label in Figure 2 (lines 162–181) and the text in Sections 3.3 and 4.1. Figure 2, Sections 3.3 and 4.1, Appendix A.1, and Table 4 of the revised manuscript give the encoder and feature-layer configuration.

Q4. Global communication interval (Section 4.1, lines 375–377). The reported experiments use coarse-global communication every six DiT blocks (K=6K=6). “Every five Transformer blocks” referred to the five intervening blocks and misstated the communication interval. Section 4.1 and Appendix A.1 of the revised manuscript describe the corrected interval, which is also listed in Table 4.

AuthorsOpenReview ↗

Brief follow-up on our response

Dear Reviewer xTJ5,

Thank you again for your thoughtful review. In response to your comments, we added a sensitivity analysis of the core–halo cropping parameters and a separate ablation of crop-window conditioning. The ablation keeps core–halo cropping and full-support inference in place, so it isolates the effect of crop-window conditioning (Section 4.3 and Table 2).

We also corrected the encoder and global communication settings in the manuscript. The reported experiments use DINOv3 ViT-L/16 features from layers 5, 7, 11, and 23, with coarse-global communication every six DiT blocks (Sections 3.3 and 4.1, Appendix A.1, and Table 4).

As the discussion period draws to a close, we would appreciate any feedback on whether the new experiments and clarifications address your concerns.

Best regards,
The Authors

Thank the authors for the detailed responses and additional experiments, which adequately address my concerns. Considering the contributions of the paper, I maintain my original recommendation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive feedback and for taking the time to review our revisions. Filigree3D generates high-fidelity 3D geometry from a single image at 204832048^3, preserving thin structures and fine surface relief. It substantially outperforms the evaluated open-source methods on geometric fidelity and preserves finer details than several commercial systems in our qualitative comparisons. It also scales to 409634096^3 while keeping peak GPU memory and inference time within practical limits. Extensive experiments support our design and parameter choices and confirm the reliability of our evaluation protocol.

We have responded to all reviewer questions, including follow-ups, in the individual discussions. The manuscript revisions are highlighted in blue.

  • Resolution scaling and inference cost (Table 3). We report geometric fidelity, peak GPU memory, and runtime across resolutions, and explain how the same denoiser extends to 409634096^3 with a resolution-specific SC-VAE.
  • Component ablations and robustness (Table 2 and reviewer follow-up). We add controls for cropping, attention, feature fusion, and data curation, and examine sensitivity to cropping parameters and inference seeds.
  • Evaluation reliability and dataset scope (Appendix A.2 and Table 5). We clarify benchmark construction, near-duplicate screening, and the shared ICP protocol. We verify that the ranking of the tested configurations holds under denser ray sampling and full-mesh evaluation, and clarify that the Toys4K subset evaluates detail-rich single objects.
  • Implementation details (Appendices A.1–A.2 and Table 4). We resolve inconsistencies in the reported image encoder and global communication settings and document the visibility-aware training objective.
  • Source of the geometric gains (Table 1). We quantify the contribution of the fine-tuned sparse-structure model and add a same-data TRELLIS.2 control. Filigree3D retains a geometric fidelity advantage when voxel support, VAE, and output resolution are matched.

The individual reviewer responses provide the full experimental settings, additional results, and references to the corresponding manuscript revisions. We appreciate the reviewers’ time and consideration and hope the revised manuscript and our responses address their concerns and clarify Filigree3D’s contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.