- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents an agentic framework for procedural material generation conditioned on image and text, i.e., the system input is a reference image or text description, and the output is a Blender program that matches the reference as close as possible. The pipeline is divided into 2 main parts: first, a video generation model converts the input to multiple images from different lighting and camera pose conditions. Then, a loop iterates between discrete compute graph generation using an agent and a differentiable renderer for parameter optimization, with error feedback to the agent. The authors tested their systems on existing benchmarks (both synthetic and real images), and showed improvements over baselines.
Critical strengths
- The quality of the procedural material looks impressive, faithfully capturing the structure and details of the input images. In the appendix, the authors also show meaningful editing by controlling the interpretable procedural parameters, making the system practically useful.
- The paper is well-presented and easy to understand. The overall design of separating the discrete and continuous optimization makes sense.
Critical weaknesses
- My main concern is whether and how much the system is better than baselines, and under what conditions one would perform better than the other. While the supplementary websites provide a lot of visualizations, there are no direct comparisons with baselines. I find the media format of the website is much easier to judge and provides much more diversity than the few figures shown in the pdf, so it would be helpful to provide comparisons with baselines in this format. This is especially important because, numerically, the method is not strictly better than baselines (Tab1, on real photos, the proposed method is ranked 3rd under the VLM judge).
- The novelty of this method is somewhat limited. Both stages of the method have been presented in previous works: VideoNeuMat uses the exact same method to promote input images into videos, and the 2nd stage of agent-based procedural material generation is not new either. It is not surprising that stitching these two together will work, especially given the capability of top LLM agents today. I think this paper brings limited new knowledge to the community, although it may be useful as a well-engineered system.
- The meaning of the ablation terms is vague and is not explained in this paper. What does Pure-Agent, No diff-opt, Post-opt mean and how are they different?
- The paper emphasizes 81-frame video since the abstract, but then L218 says you use 12 reference images. This is confusing in 1) Why does 81-frame worth emphasizing, and 2) how do you select 12 frames from 81?
Questions for authors
- Is the system able to handle more complex materials such as trasparent/traslucent materials (e.g., glass, skin)? I don't think there is a fundamental blocker from my understanding of the system, so it would be interesting to see some examples.
7 comments
Author Responses (2/2)
W4. The 81-frame video and selection of 12 reference images
The 81 frames and 12 reference images correspond to video generation and material reconstruction, respectively. These stages use different sampling densities to serve their respective purposes:
-
Video generation uses 81 frames to represent continuous changes in lighting and viewpoint. We adopt the 81-frame output configuration of the pretrained VideoNeuMat model. Dense sampling along the prescribed trajectories produces gradual appearance changes between neighboring frames, allowing us to exploit the video model’s temporal coherence and material appearance priors to generate related observations under different lighting and viewing conditions. The specific sequence length of 81 frames follows the pretrained model’s configuration.
-
Material reconstruction uses 12 sparsely sampled reference images to balance coverage and computational cost. Adjacent video frames have similar lighting, viewpoints, and appearance, so their information substantially overlaps. We therefore select six frames distributed across each of the two trajectories. This sampling covers different lighting and viewing directions while reducing redundant image processing and the costs of agent image inputs, rendering, and evaluation.
As detailed in the camera and lighting configuration section of the supplementary material, we apply the same fixed frame-selection rule to all samples, using zero-based frame indices:
-
Lighting phase, frames 0-39: The camera remains at a fixed frontal viewpoint while the light orbits the material. We select frames 0, 7, 13, 20, 26, and 33, yielding six frontal reference images under different lighting conditions.
-
Viewing phase, frames 40-80: The light remains fixed above the material while the camera orbits it. We select frames 40, 47, 53, 60, 67, and 73, yielding six oblique reference images from different viewing directions.
-
Both phases are sampled at approximately 60° azimuth intervals. Frames 39 and 80 close their respective orbits and are excluded to avoid repeating the initial poses.
Q1. Support for transparent and translucent materials
Our existing results already include materials with translucency and subsurface scattering effects, such as beeswax and a jelly-like material. We would be happy to add further examples, including rough glass and skin.
Thank the authors for the additional comparisons and explanation of the details. The qualitative comparisons shown in the website are helpful; now I'm convinced that the proposed method generates higher-quality materials than the baselines. My concern about the novelty is not fully addressed. The authors claim their contribution to be a "unified framework", which is more a systems engineering work that chains existing models and systems together rather than interesting research findings. The interleaved feedback makes sense and could be a contribution, but it sounds like a standard way to formulate this problem and is not that exciting. "Ablation evidence" can barely be called a contribution; every paper should do experiments to prove it works. Considering all the factors, I think this paper has good value as an engineering work that provides good results. So I keep weak accept.
Author Responses (1/2)
We thank the reviewer for the careful review and constructive feedback. We particularly appreciate the positive assessment of the visual quality of our materials, the practical usefulness of interpretable parameter editing, and the clarity of the paper. Below, we address the concerns about baseline comparisons, methodological novelty, ablation definitions, and frame selection, followed by the question about more complex materials.
W1. Direct visual comparisons with baselines
Thanks for this suggestion, we have updated these results to the website.
W2. Novelty beyond combining existing components
Our contribution is a unified framework that combines video generation, agent-based program synthesis, and differentiable parameter optimization for editable procedural material generation:
Three complementary components. The video model provides references under multiple lighting and viewing conditions; the agent generates and revises program structure; and differentiable optimization fits continuous parameters. To our knowledge, prior work has not combined these three components through the proposed interleaved feedback mechanism to generate executable, editable Blender material programs.
Interleaved feedback between the agent and differentiable optimization. After each optimization round, the selected program, rendered images, and remaining errors are fed back to the agent to guide subsequent program revisions. We implement Blender-compatible differentiable operators and a rendering proxy that enable parameter fitting for agent-generated shaders and write the fitted values back into their source code. This allows the outcome of continuous fitting to inform subsequent discrete program search.
Ablation evidence for the value of this interaction. Post-opt optimizes candidates from all rounds after agent search, matching the full pipeline’s numbers of agent rounds and optimization calls. On MatSynth-I2V, the full method outperforms Post-opt on all five metrics, reducing LPIPS from 0.3570 to 0.3195. This supports the value of interleaved feedback: parameter optimization can both improve individual candidates and provide useful information for subsequent program revisions.
W3. Definitions and differences of the ablation settings
Thanks for pointing this out. As described in the paper’s ablation experiments: