acceptodds
Under review as a conference paper at ICLR 2027

Ink3D: Sculpting 3D Assets with Extremely Complex Textures via Video Generative Models

Abstract

Recent 3D generative models can synthesize high-quality geometry but often struggle to reproduce intricate textures from reference images, largely due to the scarcity of large-scale 3D training data with rich surface appearance. In contrast, visual generative models are trained on datasets several orders of magnitude larger and excel at modeling complex visual patterns. Motivated by this gap, we introduce Ink3D, a framework that bridges 3D generation with large-scale video generative models to synthesize extremely complex textures. Ink3D first reconstructs a white-mesh geometry using an off-the-shelf 3D generation model. It then employs OrbitPainter, a conditional video generative model, to produce dense orbit-scan videos capturing object appearance across viewpoints. To convert these views into coherent textures, we introduce TextureOptimizer, a neural baking module that integrates dense multi-view observations while mitigating geometry inconsistencies arising from video generation. By decoupling geometry and texture synthesis and leveraging large-scale pretrained video priors, Ink3D enables significantly richer and more faithful texture generation than prior approaches.

Primary area:
generative models
Keywords:
3D Texture Generation, Video Generative Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YAa5OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses image-conditioned texturing of 3D meshes with very complex appearance (text, plaid, ornaments). Ink3D takes geometry from an off-the-shelf 3D generator and fine-tunes Wan-2.2-14B into OrbitPainter, which generates orbit-scan videos conditioned on the reference image and rendered normal and position videos. A geometry-aware sparse attention restricts attention to tokens at nearby 3D locations. TextureOptimizer then bakes the frames onto the mesh with MRF-based view selection and refines the result with the TRELLIS.2 VAE and a few denoising steps. On 85 TexVerse meshes with generated reference images, Ink3D beats Paint3D, TEXGen, SeqTex and TRELLIS.2 on front-view FID, CLIP-FID, LPIPS and CLIP-I. Ablations cover geometry conditions, attention, video resolution and frame count, baking strategy, trajectory and refinement noise.

Critical strengths
  • Most design choices are backed by ablations (geometry conditions, sparse attention, video resolution and frame count, baking strategy, orbit trajectory, refinement noise). Readers can see what each component contributes, which makes the pipeline credible as engineering.
  • The baking design is practical. Graph-cut view selection avoids blurry averaging, and mild TRELLIS.2 denoising fixes color bleeding without retraining. It is modular and training-free, and Table 4 shows clear gains over averaging and differentiable-rendering baking.
Critical weaknesses
  • Limited novelty. The method combines existing pieces: a fine-tuned video model for geometry-conditioned multi-view painting (SeqTex already textures meshes with a video model, and Hunyuan3D-Paint and Step1X-3D do geometry-conditioned painting), a 3D-proximity sparse attention, and MRF view selection followed by TRELLIS.2 refinement. Each piece is incremental, and the paper does not give a new insight beyond the combination.
  • The evaluation does not fully support the claims. The main Table 1 compares front-view renders against the conditioning reference image, so it largely rewards reproducing the input view, while the claim is about quality across views. The multi-view protocol appears only in the appendix.
  • The metrics are FID and CLIP-FID on 85 images with no seeds, variance, significance tests or user study, yet the text says significantly outperforms. The strongest geometry-conditioned painters (Hunyuan3D-2.1 Paint, Step1X-3D) appear only in appendix Table 5, where the margin is smaller (FID 144.1 vs 153.9) and HY3D-Paint wins CLIP-I. The references are synthetic edits, and real photos are only shown qualitatively. Table 3 also does not clearly support the claim that too few frames degrade quality (41 frames has better LPIPS and CLIP-I than the 61-frame default).
Questions for authors
  1. What are the Table 1 results under the appendix multi-view protocol for all baselines, including TRELLIS.2, TEXGen, SeqTex and Paint3D, with variance over seeds? This would show whether the gains hold beyond the conditioning view.
  2. How much does TRELLIS.2 refinement alone contribute? Please give a quantitative ablation of MRF-only, refinement-only (on average-baked voxels) and both, to separate the video prior's contribution from TRELLIS.2's native 3D prior.
Reviewer_LmkEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper textures a given untextured mesh from a single reference image, with a focus on very complex appearance such as text and dense patterns. Ink3D first uses a fine-tuned video model (Wan2.2 with LoRA), conditioned on rendered normal and position videos, to generate dense orbit videos around the mesh. A geometry-aware sparse attention (GASA) is added to help consistency across frames. Then TextureOptimizer selects one view per voxel with graph cut and refines the result with a few TRELLIS.2 denoising steps. Experiments compare with Paint3D, TEXGen, SeqTex and TRELLIS.2 on a TexVerse benchmark, with a multi-view comparison and ablations in the appendix.

Critical strengths
  • The results on text and dense patterns look clearly better than TRELLIS.2 and Hunyuan3D-2.1 on the side shown by the reference. Using a large video prior with light fine-tuning is a practical way to get detailed textures.
  • The ablations are broad, and the supplementary provides the raw orbit videos and outputs, which makes the claims easy to check.
Critical weaknesses
  • The evaluation does not really test the main claim, and the visual results are hard to check. The main table only scores the front view, which is what the reference already shows, and the multi-view comparison in the appendix does not include TRELLIS.2. The qualitative figures are also too small to verify the results. In Fig. 4, the reference photos are tiny and placed around a composed scene, so it is hard to see their details and check that each generated texture really matches its reference. Figs. 5 and 6 are also small and low resolution, and Fig. 6 does not even show the reference image. I would like to see larger side-by-side comparisons that also include the input geometry (e.g., an untextured or normal render), so we can see what the conditioning mesh looks like and how much of the detail really comes from the generated texture. In the supplementary and Fig. 5, I also noticed some cross-view failures, such as mirrored text on the back of a sign and clothing that changes color between the front and back views.
  • The novelty is limited, and it is unclear where the gain comes from. To my knowledge, generating orbit videos of a mesh with a video model and lifting them to a texture has been explored before (e.g., VideoMat, SeqTex), and graph-cut view selection and partial re-noising are known techniques. With simple average baking, the result is close to TRELLIS.2 alone on FID, so much of the gain seems to come from TextureOptimizer, which itself uses TRELLIS.2. However, the graph cut and the TRELLIS.2 refinement are not separated in the ablation.
Questions for authors
  1. Could the authors add TRELLIS.2 to the multi-view comparison, and report front and back views separately? A user study or a text-legibility metric would also help support the "extremely complex textures" claim.
  2. Could the authors provide larger, higher-resolution versions of Figs. 4 to 6, showing for each asset the reference image, the input geometry (untextured or normal render), and the result from several views?
  3. Could the authors separate the graph cut and the TRELLIS.2 refinement in the ablation?
  4. Is the final output an O-voxel color grid or a UV texture on the input mesh? How are the methods rendered for the metrics?
  5. Were the test assets checked for overlap with the training data?
  6. Ink3D takes about 13 minutes per asset on an H100, mostly for video generation (App. G), while most baselines take seconds to about a minute. Could the authors report the runtime and memory of all methods on the same hardware? The supplementary videos also use a higher resolution and longer orbits than the default; what is the cost of that setting?

Overall: The results on complex textures look impressive on the reference side. However, the full-surface consistency is not well evaluated, the figures are too small to verify that the textures really match the references, and the pipeline mainly combines existing parts without showing which part brings the gain. I currently lean toward a weak rejection, but I am open to raising my score if the authors add the multi-view comparison with TRELLIS.2, clearer figures with the input geometry, and the component ablation.

Reviewer_hm7oOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To tackle the blurriness and distortion that arise when generating complex textures, this paper argues that a major cause lies in the scarcity of high-quality 3D texture training data. In contrast, large-scale video generative models have acquired abundant appearance knowledge. Motivated by this observation, the authors first generate an untextured white mesh using off-the-shelf models, synthesize an orbiting video of the object, and fuse the colors and patterns from the video onto the 3D surface.

Critical strengths
  1. The video is leveraged to provide multi-view consistent appearance references, while geometric conditioning resolves cross-view correspondences. 2. 3D geometric information is incorporated into both input conditions and the attention architecture to enforce texture correspondence across different views. 3. A dedicated texture fusion method is designed to mitigate multi-view misalignment, preventing fine-grained details from being averaged out.
Critical weaknesses
  1. The evaluation of complex texture generation is not direct or comprehensive enough. Qualitative experiments lack sufficient and clear demonstrations showing the resolution of this core issue. 2. The capability to generate material properties beyond color and texture is not adequately discussed. The paper mainly focuses on colors and patterns, while practical 3D assets contain additional material information. 3. The overall presentation of the method fails to highlight its contributions. The mechanism for establishing 3D correspondences is under-explained: how does the model associate regions that are spatially close yet lie on distinct surfaces based on spatial distance?
Questions for authors
  1. For complex texture generation, do the authors report quantitative metrics, such as the accuracy of generated text characters and the alignment fidelity of patterns? 2. Could the authors show rendering results under diverse lighting conditions? Different material attributes are sensitive to lighting variations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.