- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Style2Shape transfers material appearance from a reference image to a target mesh. An image editing model first produces an image aligned with the target geometry. The method retrieves procedural materials, blends their albedo with generated textures, and optimizes lighting, UV transforms, material parameters, and blending weights through differentiable rendering. The outputs are PBR maps with editable procedural parameters. Experiments show transfers of reference colors and patterns, with quantitative results measuring agreement with the generated target images.
Critical strengths
The paper presents a pipeline for generating procedural materials for a target mesh from a reference image, combining material retrieval, generated albedo, and differentiable optimization. Several examples show good transfers of reference colors and patterns, and the resulting PBR maps can be rendered under different views and lighting.
Critical weaknesses
The methodological novelty is limited. MaPa already combines generated image supervision, procedural material initialization, and differentiable fitting. Style2Shape combines reference-image guidance with generated albedo, but does not compare with MaPa. A comparison using the same aligned target image would help establish the benefit of the proposed material representation. The pipeline appears to be an engineering integration of existing components, and the experiments do not establish a substantial contribution beyond this combination.
The component ablations in Figure 5 are limited to qualitative results on a chair and a bag. They do not establish the benefits of retrieval, generated albedo, and staged optimization across the test set, and the style loss is not ablated.
Only albedo is generated; roughness, metallicity, and normals come from the retrieved procedural material. The paper does not show whether these channels remain compatible with the generated patterns. Please add a controlled material-recovery experiment using a reference rendered from the same object with known materials, and report rendering errors under held-out views and lighting, especially for glossy and metallic surfaces.
The method depends on image editing and PBR map prediction, but does not test how errors from these models affect retrieval and the final materials. Please show the generated textures under repeated tiling to check for visible seams.
Questions for authors
-
Please compare with MaPa using the same aligned target image. What changes to the material representation or optimization are responsible for the improvement? Please also discuss newer related work such as MultiMat and Generative Detail Enhancement for Physically Based Materials.
-
Please report quantitative ablations of generated albedo, retrieval, staged optimization, view selection, and the style loss across the test set.
-
Please evaluate material recovery on objects with known materials, using rendered reference images, and report rendering errors under held-out views and lighting, especially for glossy and metallic surfaces. Does the generated albedo remain consistent with the procedural roughness, metallicity, and normals? Please also show the generated textures under repeated tiling to check for visible seams.
-
How sensitive is the result to errors in the edited target image and PBR map prediction? Please test an alternative image editor or controlled errors in the intermediate predictions, and show whether the optimization can recover.
2 comments
I appreciate the new ablations and the comparison with LumiTex. The three examples help explain what the generated albedo branch adds to the pipeline. My assessment still rests on two points: the methodological step beyond existing procedural material fitting seems modest, and the evaluation leaves open how well the recovered materials render under new lighting. Given these remaining concerns, I will keep my score of 2.
Thank you for the concrete suggestions on representation, component isolation, and material validation. We added quantitative component ablations and an original LumiTex comparison.
1. MaPa and the representation contribution. MaPa uses generated supervision and procedural material fitting. Its official project page and author repository currently provide no runnable implementation, so we could not perform the requested direct comparison under the same Itr. Our within-pipeline ablation measures the benefit of adding the generated-albedo branch to procedural fitting: mean PSNR improves by 2.392 dB and mean LPIPS decreases by 16.5%. PSNR and LPIPS improve on each tested pair under the same generated target.
2. Component ablations. We evaluate three reference–mesh pairs: a bag, a sofa, and a table. Configurations share targets, cameras, rendering settings, and 200 optimization iterations. Metrics against Itr are foreground PSNR, foreground-averaged SSIM, and VGG LPIPS with a common black background.
Generated albedo is removed throughout fitting; the style ablation removes its loss only in final material refinement while retaining UV alignment. Retrieval and the final-stage style term improve all three mean metrics. Staging improves PSNR/SSIM, whereas joint optimization has lower LPIPS. View selection is addressed by the paper's existing qualitative ablation.
3. Original LumiTex comparison. We run LumiTex's AL/MR generation and LVSM completion with the same original reference images and meshes. Both methods' exported materials are frozen and receive the same additional 50-step illumination-only fit to Itr with a common renderer, camera, and mask. Unlike the ablation table, this table therefore includes additional lighting fitting.
In this target-image fitting evaluation, Style2Shape improves mean PSNR by 1.882 dB and reduces mean LPIPS by 15.2%; LumiTex has higher SSIM.
4. Material compatibility, intermediate errors, and tiling. Generated albedo and procedural roughness, metallicity, and normals are fitted through one PBR renderer. The graph still determines the available reflectance and normal variations: an added albedo pattern does not automatically introduce corresponding variations in other channels. The requested known-material recovery under held-out views and lighting has not been evaluated.
Inspecting the sofa intermediate target shows that white pillows become purple only after fitting; their shared material segment limits separate control. This observation locates an error downstream of editing, rather than measuring recovery from controlled prediction errors. Texture generation requests horizontal and vertical boundary continuity, and UV repetition tiles the resulting maps; these mechanisms alone do not establish seam-free output.
5. Related work. MultiMat synthesizes procedural graphs using visual/textual graph representations and constrained search; our method fits retrieved graphs together with generated albedo to target geometry. Generative Detail Enhancement for Physically Based Materials enhances existing asset appearance through multiview diffusion and inverse rendering. Our setting transfers a reference photograph to a target mesh while retaining editable procedural parameters alongside generated albedo.