- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces MatLoom, a compact, layer-oriented language for text-to-material generation. Shared spatial expressions define layer coverage and material channels such as base color, roughness, and height, while a standalone interpreter evaluates programs into renderable material maps. The synthesis pipeline uses pretrained language models without task-specific fine-tuning, combining parser-guided repair, render-based refinement, and text–image scoring for program selection and noise-seed search.
The authors evaluate six language-model backbones on 141 material descriptions against three diffusion-based material generators. The strongest configuration achieves the highest mean on all four flat-layout alignment metrics and receives 59.2% of choices in a blind study involving 30 participants and 20 prompts. A manual editing case additionally illustrates parameter control and explicit cross-channel dependencies.
Critical strengths
The representation is coherent and retains meaningful program-level control. MatLoom organizes layers, shared spatial expressions, and material channels into a compact executable representation. Dependencies between coverage, color, and relief remain explicit in the source. For example, a shared tile mask can control both exposed grout regions and relief boundaries. This provides a concrete mechanism for inspection, parameter modification, and procedural resampling. The editing example in Section 5.4 verifies these mechanisms for the demonstrated material.
The complete system’s appearance results are supported by both automated evaluation and a human study, with appropriately qualified reporting. The evaluation covers 141 prompts, six language-model backbones, and three diffusion baselines. Beyond automated metrics, the blind study with 30 participants provides evidence of appearance preference on the evaluated subset. The paper also reports variation across backbones and prompt sources, disagreement between selection and evaluation metrics, and a nonsignificant comparison. These analyses help readers assess the scope of the demonstrated gains.
Critical weaknesses
The practical value of the proposed representation relative to existing procedural representations is insufficiently established. The central contribution is a restricted, layer-oriented field language, but the main quantitative comparisons are against diffusion models that output raster maps. These comparisons establish the competitiveness of the complete system, but do not determine whether the proposed representation is easier for language models to generate correctly, or more useful for subsequent editing, than existing procedural material approaches or direct Blender program generation. The related-work discussion and Appendix A already acknowledge these distinctions and the absence of direct controls; nevertheless, its practical advantages over alternative procedural representations remain insufficiently established. Furthermore, Section 5.4 demonstrates manual parameter edits for only one material, which is insufficient to assess editing utility across a broader set of generated programs. Given substantial prior work on layered materials and procedural generation, this gap limits the assessment of the contribution’s incremental significance.
Uncontrolled inference budgets and rendering differences limit the interpretation of the baseline comparisons. MatLoom uses five refinement rounds and seed search over up to 3,003 candidates, while the diffusion baselines do not receive a matched search budget. The rendering routes also differ materially: MatLoom’s height drives geometric displacement, StableMaterials’ height drives bump shading, the other two baselines provide no height map, and only MatLoom uses transmission. Consequently, some appearance gains may reflect a combination of generation quality, inference expenditure, and rendering treatment. The initial programs already outperform the baselines on some metrics, and the human study supports the complete pipeline, so these concerns do not invalidate the results. However, the current experiments do not establish how much of the advantage remains under comparable computational budgets and shared rendering channels. Section 5.6 transparently acknowledges this limitation, but its impact is not experimentally quantified.
Questions for authors
The main quantitative comparisons focus on diffusion-based material-map generators. Why were closer procedural material-generation methods, particularly Material Apprentice, which also supports text-conditioned generation, not included? The related-work discussion explains differences in representation and execution, but what practical advantages do these differences provide—for example, higher program-generation success, a better quality–cost trade-off, or more reliable editing control? Could the authors provide a direct comparison or other evidence supporting these benefits?
How much of the performance gap remains when comparable material channels and height treatments are used? Could the authors report quality–cost curves using the final evaluation metrics, rather than only improvements in the internal selection score as the seed-search budget increases?
4 comments
Response to Reviewer KGqv (2/4): points 3–4 (quality and cost, editing control)
3. Practical advantages: quality and cost per successful program (Q1)
This uses the same controlled swap between MatLoom and Blender-Python (BP) as point 2.
| Metric | MatLoom | BP scaffolded | BP unrestrained | MatLoom − scaffolded, paired [95% CI] |
|---|---|---|---|---|
| Flat BLIP | 47.6 | 28.8 | 15.6 | +19.7 [+8.1, +31.1] |
| Flat CLIP | 29.4 | 24.7 | 22.5 | +4.9 [+3.4, +6.5] |
| Flat VQA | 54.0 | 42.1 | 35.7 | +12.6 [+6.8, +18.5] |
| Flat judge | 76.5 | 47.1 | 40.4 | +29.3 [+20.6, +38.3] |
| Staged BLIP | 27.6 | 7.0 | (c) | +21.4 [+12.6, +31.2] |
| Staged CLIP | 25.0 | 20.8 | (c) | +4.5 [+2.9, +6.2] |
| Staged VQA | 52.0 | 35.6 | (c) | +16.9 [+12.0, +22.1] |
| Staged judge | 61.3 | 30.7 | (c) | +30.0 [+21.7, +38.7] |
| Mean program size | 31.3 lines, 2,559 bytes | 88.4 lines, 3,541 bytes | 2,729 bytes | |
| API cost per success | about 0.08 USD (est.) | 0.09 USD | 0.06 USD |
Means are over the runs that delivered a render. All paired differences have p < 0.001 except flat BLIP (p = 0.001). The judge columns were scored in one session. (c) The unrestrained variant renders its own scene, so it has no comparable staged render.
API cost per material is about the same on both sides. The differences are quality, program length (about a third as long), and preview time: about 70 ms in our interpreter against about 12.5 s for a Blender attempt. The scope is one backbone and one seed, because the second backbone did not finish before we froze scoring.
4. Practical advantages: editing control (Q1)
Sec. 5.4 showed one material, as you noted. We built a benchmark of 172 edit tasks on 40 generated materials. A task succeeds only if the requested change is made and every channel outside the intended change stays within 1/255. The task list, tolerances, and masks were fixed before any editor ran. An LLM edits the MatLoom program (2.6 calls per task on average). MatFuse and StableMaterials are re-prompted with the edit instruction and keep the best of three seeds.
Success rate in %, with the rate of making the requested change (ignoring preservation) in parentheses:
| Edit type | n | MatLoom program edit | MatFuse re-prompt | StableMaterials re-prompt |
|---|---|---|---|---|
| Roughness shift | 25 | 80.0 (84.0) | 0.0 (84.0) | 0.0 (76.0) |
| Base-color shift | 38 | 84.2 (84.2) | 0.0 (57.9) | 0.0 (78.9) |
| Widen a shared mask feature (e.g. grout) | 31 | 80.6 (83.9) | 54.8 (61.3) | 22.6 (61.3) |
| Noise-seed swap | 40 | 77.5 (100.0) | n/a | n/a |
| Layer removal | 38 | 97.4 (100.0) | n/a | n/a |
| All tasks | 172 | 84.3 (91.3) | 9.9 (36.0) | 4.1 (39.5) |
| First three types only | 94 | 81.9 (84.0) | 18.1 (66.0) | 7.4 (72.3) |
| Mean change on channels that should not move (of 255 levels) | 0.08 | 34.8 | 68.1 |
Seed swap and layer removal are defined on the program and count as failures for the raster baselines in the "All tasks" row, so the fair comparison is the "first three types" row. Re-prompting often makes the requested change but regenerates everything else. In all 145 successful MatLoom edits the untouched channels are byte-identical. The original benchmark compared only raster re-prompting, and its criteria are computed on the maps, so they measure exactness and not perceived quality.
We have also run the Blender-Python editing comparison: the same five edit types frozen over the 40 matched Blender-Python materials (199 tasks), each arm editing its own program under the matched backbone, judged by one before/after MLLM. Joint success is 54.8% (MatLoom) vs. 44.7% (Blender-Python), paired p = 0.057 (48 vs. 28 discordant successes), execution 98.5% vs. 92.0%, and the largest gap is on the parameter edit (widen: 45.0% vs. 17.5%). The visual judge is stricter than the map-level checks above, so the 84.3% and the 54.8% are not comparable levels. This is now App. R of the revised PDF.
Response to Reviewer KGqv (3/4): point 5 (channels and height treatment)
5. Comparable channels and height treatment (Q2)
"How much of the performance gap remains when comparable material channels and height treatments are used?"
We separated two things that the submission mixed together: which channels a system produces, and how a height map is used at render time.
5.1 Channels. We re-rendered the stored outputs of all four systems through one path with only the channels every system produces: base color, roughness, and normal. MatLoom gives up height, transmission, metallic, ambient occlusion, IOR, and emission; StableMaterials and IntrinsiX give up metallic; MatFuse gives up specular. Nothing was regenerated. Each entry is the shared-channel score, with the loss relative to that system's native render in parentheses (negative means the shared render scored higher).
| Method | Flat BLIP | Flat CLIP | Flat VQA | Flat judge | Staged BLIP | Staged CLIP | Staged VQA | Staged judge |
|---|---|---|---|---|---|---|---|---|
| IntrinsiX | 28.80 (1.1) | 24.06 (0.2) | 43.09 (0.5) | 38.13 (0.9) | 19.40 (1.6) | 21.36 (0.4) | 41.25 (0.7) | 31.55 (0.5) |
| MatFuse | 8.23 (0.0) | 20.09 (0.0) | 30.57 (0.0) | 19.66 (1.9) | 8.61 (0.0) | 19.19 (0.1) | 30.84 (−0.3) | 17.41 (−0.1) |
| StableMaterials | 27.77 (0.8) | 25.36 (0.3) | 44.96 (0.4) | 43.82 (0.4) | 20.07 (1.3) | 23.34 (0.3) | 44.45 (1.1) | 37.92 (2.1) |
| MatLoom | 50.55 (5.5) | 27.74 (1.1) | 51.77 (2.9) | 66.73 (6.1) | 35.54 (0.6) | 25.60 (−0.3) | 52.64 (1.6) | 58.35 (3.2) |
So the gap that remains is large: +22.8 flat and +15.5 staged BLIPScore over StableMaterials, with MatLoom ahead on every metric in both layouts and the contrasts against the baselines significant after Holm correction (p < 0.001). The extra channels do contribute. Removing them costs MatLoom 5.5 flat BLIPScore points and, on the judge, 6.1 flat (p < 0.001) and 3.2 staged (p = 0.057). No baseline loses more than 2.1.
A caution on the judge columns. They come from a new scoring session, in which StableMaterials' flat renders score about 13 points lower than in Table 1 and ours about 6 points higher, with or without the channel restriction. This is the session effect described in point 4 of the general response. Judge differences inside this table are valid, but the judge gap between systems is larger here than in the original evaluation, so we put more weight on the three deterministic metrics.
5.2 Height treatment. We rendered MatLoom with its height as bump (the StableMaterials default: normalized height, distance 0.1) and StableMaterials with its height as true displacement (midlevel 0.5, scale 0.1).
| Condition | Flat BLIP | Flat CLIP | Flat VQA | Staged BLIP | Staged CLIP | Staged VQA |
|---|---|---|---|---|---|---|
| MatLoom, displacement (native) | 56.06 | 28.80 | 54.71 | 36.14 | 25.30 | 54.26 |
| MatLoom, bump | 48.60 | 27.54 | 53.01 | 34.98 | 24.60 | 53.64 |
| StableMaterials, bump (native) | 28.57 | 25.66 | 45.36 | 21.33 | 23.59 | 45.53 |
| StableMaterials, displacement | 30.79 | 25.84 | 46.31 | 21.21 | 23.59 | 46.78 |
Displacement helps MatLoom (7.5 flat BLIPScore, and 8.2 flat and 5.1 staged judge points within one session) and changes StableMaterials by about 2 BLIPScore and 1 judge point. With bump shading on our side and displacement on theirs, MatLoom still leads on every metric, by 19.2 flat and 16.6 staged judge points within one session. No swap changes the ranking. The full table with judge columns is Table 26. This control covers our best configuration only, and displacement at scale 0.1 is not StableMaterials' released setting, so we do not interpret its displacement row further.
Response to Reviewer KGqv (4/4): point 6 (quality against cost)
6. Quality–cost curves in the final metrics (Q2)
"Could the authors report quality–cost curves using the final evaluation metrics, rather than only improvements in the internal selection score…?"
Yes. Fig. 7 in the revision plots final metrics against budget, and nothing in it uses the internal selector score. The same data as tables (flat layout, means over 141 prompts and 3 runs):
6.1 Operating points
| Operating point | LLM calls | Candidates scored | BLIP | CLIP | VQA | Judge |
|---|---|---|---|---|---|---|
| MatLoom, first program | 1 | 1 | 48.31 | 27.70 | 53.34 | 61.35 |
| MatLoom, after 5 critique rounds | 11 | 6 | 51.86 | 28.25 | 52.93 | 67.61 |
| MatLoom, selected program | 12 | 6 | 54.17 | 28.71 | 53.97 | 66.56 |
| MatLoom, with seed search (full) | 12 | 6 + up to 3,003 | 56.06 | 28.80 | 54.71 | 67.01 |
| StableMaterials, 1 sample | 0 | 0 | 28.57 | 25.66 | 45.36 | 57.41 |
| StableMaterials, best of 3 | 0 | 3 | 36.21 | 27.02 | 47.91 | 62.09 |
| StableMaterials, best of 10 | 0 | 10 | 42.63 | 28.18 | 52.37 | (a) |
| IntrinsiX, 1 sample | 0 | 0 | 29.94 | 24.29 | 43.61 | 56.10 |
| IntrinsiX, best of 3 | 0 | 3 | 40.39 | 26.33 | 49.23 | 64.70 |
| MatFuse, 1 sample | 0 | 0 | 8.27 | 20.12 | 30.61 | 36.36 |
| MatFuse, best of 9 | 0 | 9 | 10.07 | 21.70 | 34.47 | 45.16 |
Baseline best-of-N is selected by the same MobileCLIP2 scorer our pipeline uses. (a) Judged in a separate session; see 6.3.
6.2 Our seed search at truncated budgets (K seed variants per candidate, winners re-scored on final metrics)
| K | Flat BLIP | Flat VQA | Flat judge (one session) | Share of wall-clock time |
|---|---|---|---|---|
| 1 | 55.47 | 53.97 | 70.81 | 1% |
| 10 | 56.44 | 54.17 | 70.35 | 3% |
| 50 | 55.99 | 53.94 | 70.22 | 13% |
| 100 | 56.35 | 54.37 | 70.95 | 23% |
| 250 | 56.42 | 54.36 | 71.22 | 43% |
| 500 | 56.37 | 54.71 | 70.25 | 60% |
| 1000 (full) | 56.06 | 54.71 | n/a | 75% |
The curve is flat. One variant per candidate already gives 99% of the full search's BLIPScore, and the judge stays within one point from K = 1 to 500. Larger K keeps improving the internal selector score, which is what the submission reported, but on final metrics it adds about 0.6 BLIPScore. Nearly all of the measured quality comes from the first call, the critique rounds, and a minimal search.
6.3 Paired differences against StableMaterials best-of-N (MatLoom full pipeline minus baseline)
| Selection rule | Flat BLIP | Flat CLIP | Flat VQA | Flat judge | Staged BLIP | Staged CLIP | Staged VQA | Staged judge |
|---|---|---|---|---|---|---|---|---|
| Best of 3, MobileCLIP2 | +19.8*** | +1.8*** | +6.8*** | +4.9 [+0, +10] | +9.9*** | +0.8 [−0, +2] | +6.3*** | −1.1 [−6, +3] |
| Best of 10, MobileCLIP2 | +13.4*** | +0.6 [−0, +1] | +2.3 [−1, +5] | +15.6*** (b) | +5.3 [−0, +11] | −0.2 [−1, +1] | +3.1 [+0, +6] | +12.2*** (b) |
| Best of 3, per-metric oracle | +14.7*** | +1.2* | +2.7* | −3.5 [−8, +1] | +4.0 [−2, +10] | −0.1 [−1, +1] | +2.3 [−1, +5] | −9.2*** |
| Best of 10, per-metric oracle | +0.9 [−5, +7] | −0.4 [−1, +0] | −3.5** | +6.1* (b) | −6.9** | −1.7*** | −4.9*** | +1.2 [−3, +6] (b) |
Stars: * p < 0.05, ** p < 0.01, *** p < 0.001 (two-sided Wilcoxon). Brackets: 95% bootstrap intervals (not significant). (b) Scored in a separate judge session together with our re-judged outputs.
With a deployable selector, the flat BLIPScore lead holds at ten samples, and CLIPScore, VQAScore, and staged BLIPScore become ties. The best-of-10 judge entries favor us but come from the later session discussed in point 5, so we read them next to the best-of-3 judge entries, which are ties. The oracle rows pick the baseline's best sample separately for each metric using that metric. No real selector can do this, so it is an upper bound: at ten samples, flat BLIPScore is a tie and four entries move to StableMaterials. The abstract and conclusion now state the feasible result together with this bound. Tables 28 to 30 give the same analysis for IntrinsiX and MatFuse.
6.4 Dependence on the backbone. At one call, three of the five other backbones beat StableMaterials on flat BLIPScore (by 11.6 to 12.7 points), and Gemma and Qwen fall below it (Table 27).
Two of these results changed our own view of the system. Scaling the seed search from 1 to 1,000 variants, which is where 75% of our wall-clock time goes, adds about 0.6 BLIPScore on final metrics. And the same loop writing Blender-Python executes almost as reliably as MatLoom yet produces much weaker materials, so execution rate alone says little about a program target. Both findings are now in the paper, and every limitation that Sec. 5.6 listed is still there, with the number that measures it. If part of Q2 is still open, please tell us which comparison would settle it and we will try to run it. We would be grateful if you would reconsider your rating in light of these results.
Response to Reviewer KGqv (1/4): points 1–2 (Material Apprentice, generation success)
Thank you for the detailed and fair review. Your two main points were that the value of the representation against other procedural approaches was not established, and that Sec. 5.6 acknowledged the budget and rendering confounds without quantifying them. We agree with both. Below we report the experiments we ran in response, including where the outcome is less favorable to us. Our answers are numbered 1 to 6 across four comments: (1) Material Apprentice, (2) program-generation success, (3) quality and cost, (4) editing control, (5) channels and height treatment, (6) quality against cost in the final metrics.
1. Why Material Apprentice was not included, and the comparison now (Q1)
Material Apprentice (MA) was published at ECCV 2026 and we treated it as concurrent work. The submission discussed it in Sec. 2 and Appendix A but did not run it. We have now run its released code (commit 6811f66, default gpt-5 configuration) and rendered and scored its outputs with our stack. One MA run takes about an hour and 1.5M prompt tokens, so in the rebuttal period we could cover a stratified 45-prompt subset, not all 141 prompts.
Two caveats matter for reading this table, and App. U spells out both. First, MA was designed and evaluated with an image plus text as input. We run its text-only mode, in which it first generates its own reference image, so this is outside the setting its authors evaluated. Second, our API proxy does not support the image tool that MA calls, so we rerouted that call to the equivalent images endpoint with the same parameters and capped concurrency at two. For both reasons we treat MA's appearance scores as a lower bound on what the system can do and do not use them as a headline. For completeness, the paired difference is +41.0 flat BLIPScore, and +40.1 judge points within one judge session.
What we rely on is execution and cost. Both systems completed all 45 prompts, and MA needed roughly nine times as many LLM calls per material. Its cost comes from its multi-module design (retrieval, compilation, three rounds of best-of-10 sampling) and not from failed runs.
MatLayerNet and MultiMat have not released code, and MultiMat is not text-conditioned, so we could not run them.
2. Practical advantages: program-generation success (Q1)
We measured all three; they are points 2, 3, and 4. To isolate the representation, we ran a controlled swap of the output language: same backbone (gemini-3.6-flash), same 45 prompts, repair budget, 5 critique rounds, selection rule, renderer, and metrics, with seed search off on both sides. The model writes either MatLoom or Blender-Python. For Blender we tested a scaffolded variant (the model fills in a
build_materialfunction on our scene) and an unrestrained one (it writes the whole script, 30 prompts). The Blender reference document in the prompt is 24k characters, at least as long as our DSL prompt.Program-generation success turned out not to be the advantage.
(a) Per-attempt statistics were recorded for the Blender variants; a MatLoom program that parses always executes, and initial generations over the same 45 prompts parse on the first attempt in 37/45 (82%, 95% CI 69-91). (b) Pixel standard deviation of the flat render of at least 10, computed after the runs.
Blender first attempts fail in 20% and 43% of runs, in line with the roughly 30% first-shot failure rate that MA reports, but the repair loop recovers nearly all of them. A comparison of execution rates would therefore miss the difference, which is in what the successful programs produce (point 3).