- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes PBR material reconstruction and generation in UV texture space. Given a mesh, a valid UV atlas, camera poses, and one or more RGB views, it rasterizes each view back into a sparse UV observation and stores world position and normal maps at the same texels. It then concatenates these observation frames and the target material frames in the latent/frame interface of a fine-tuned Wan2.1-1.3B video DiT. The model must demodulate lighting, merge observed evidence, and inpaint unseen texels to produce basecolor plus height/roughness/metallic maps. The use of a video backbone supplies pretrained multi-frame transformer and VAE priors; the generated object is not a temporal video. A 3D-aware RoPE helps connect UV-distant but surface-near texels, while progressive noise rolling and coverage-weighted expert fusion extend inference to higher resolutions and more views. Experiments cover multi-view reconstruction, relighting, and single-image/text-conditioned material generation. The shared UV output removes one source of cross-view bake/fusion inconsistency, but does not by itself guarantee seam-free chart boundaries or pixel-perfect projection.
Critical strengths
- Useful problem formulation. Generating directly in a shared UV atlas produces deployable PBR maps and avoids reconciling independently generated values for the same surface location. Extending this formulation from diffuse texture to basecolor, height, roughness, and metallicity is practically useful.
- Coherent use of video diffusion priors. The method adapts a pretrained video DiT to texture-space observations, uses world-position and normal conditions, and adds a 3D-aware RoPE to connect UV tokens that are distant in the atlas but close on the surface. The high-resolution and many-view extensions are also practically motivated. The high-resolution procedure is an adaptation of prior tiled/noise-rolling inference ideas, rather than an entirely new diffusion algorithm.
- Practical inference extensions and useful evaluation components. Relighting is a useful test of material estimation beyond matching input illumination. The progressive noise-rolling/tiled inference and coverage-weighted multi-view fusion are practical ways to extend a model trained at limited resolution and view count, although they build on prior tiled-diffusion ideas rather than constituting entirely new diffusion algorithms. The diffuse-only TEXGen comparison and runtime reporting are also helpful.
Critical weaknesses
- The evaluation does not use task-matched metrics for the two problem settings. The paper evaluates multi-view-to-material reconstruction/relighting and single-view-to-material generation in separate tables, but the multi-view setting is still assessed mainly through rendered-image reconstruction and relighting scores. Since densely covered multi-view inputs primarily test illumination removal and PBR-channel decomposition, this setting should be evaluated with channel/material-estimation metrics under the same protocol as decomposition methods. The single-view setting is a different material-generation problem and should retain generation metrics and generation baselines. Without this task-specific separation of metrics and comparisons, the reported numbers do not cleanly establish performance on either capability.
- The method is insufficiently positioned against the closest and strongest relevant work. SeqTex is a closely related use of video diffusion for UV texture generation with geometry-informed correspondence, but it is neither discussed nor shown in a qualitative comparison. SeqTex predicts RGB/albedo rather than full PBR, so it is not a fair quantitative baseline for full image-to-material generation; its UV/albedo and seam behavior is nevertheless directly relevant. The multi-view decomposition setting is not compared with task-matched methods such as RGB↔X or SuperMat, and the single-view full-material setting is not compared with recent methods such as NaTex and LumiTex. These omissions make the relative standing of the proposed method unclear.
- Transfer of video priors to fragmented UV layouts and cross-chart alignment are insufficiently validated. UV charts can be independently rotated, cut, and packed, so atlas adjacency and orientation need not reflect coherent surface structure or natural-image composition. Position/normal conditioning and 3D-aware RoPE are reasonable responses, and the paper includes a RoPE ablation, but it does not test robustness to alternative valid unwraps or directly measure cross-chart continuity. A shared atlas avoids independently generated view predictions, but does not itself ensure seam-free surface appearance. The paper also provides only limited controlled evidence for complex, multicolored patterns.
Questions for authors
My current assessment is weak reject. The following evidence could materially change my assessment; the main issue is whether the method is competitive on each task under a fair, task-matched protocol.
- Use task-matched metrics and baselines. For the densely covered multi-view setting, please evaluate illumination removal and PBR-channel/material decomposition with metrics and protocols used by RGB↔X, SuperMat, or comparable decomposition methods, using matched views, geometry, lighting, and outputs. For the single-view full-surface generation setting, please use generation metrics and compare with TRELLIS.2, NaTex, LumiTex, and other full-material methods under the same mesh, camera, and resolution. The two settings should not be judged by the same notion of success.
- Relationship to SeqTex. Please discuss SeqTex explicitly and include qualitative comparisons of its shared RGB/albedo output, UV correspondence, and seam behavior under the same mesh and input image. SeqTex does not predict roughness, metallicity, or the other full PBR channels, so it should be used as a qualitative baseline to compare the albedo itself.
- Robustness to different UV unwraps and seam alignment. If the same mesh and input images are paired with different valid UV maps—for example, UVAtlas, Smart UV Project, or a learned unwrap—does the method produce similar material quality? For patterns that continue across a UV cut, please show close-ups on both sides of the cut and report a metric (if possible) on the corresponding 3D surface points. This would clarify whether world position, normals, and 3D-aware RoPE actually compensate for the lack of natural 2D adjacency in UV space.
- Does the high-resolution procedure actually resolve small-chart ambiguity? Figure 8demonstrates the practical 2K-to-8K, more-view setting, but it reports only one set ofimage-quality numbers. Could the authors report PSNR/SSIM/LPIPS for the same batch of test assets under the 2K and 8K settings, together with the corresponding number of input views and surface coverage? This would clarify the quality change associated with the high-resolution procedure, including whether it improves only fine detail or also reduces errors caused by multiple UV islands falling within one attention patch. The current 4× upscaler comparison in Table S4 does not answer this question because it evaluates a separate upscaling task.
- More diverse materials and the role of geometric detail. Please show more examples with rich, multicolored materials and complex patterns, comparable in diversity to the teapot example, rather than mostly monochromatic or visually simple cases. In addition, can the authors test whether high-frequency texture and material details remain correctly aligned on low-frequency geometry, such as a smooth sphere? A controlled comparison between detailed and smooth versions of the same shape, with the same reference appearance or prompt, would help determine whether the method can align texture from image evidence when geometry provides little local structure. This would separate the contribution of geometric cues from the model's ability to generate and align appearance itself.