- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents BlenderLore, which converts Blender tutorial videos into tested, reusable procedures packaged as an agent skill. This pipeline extracts instructions, reconstructs executable assets, and distills methods for adaptation and composition. Main contributions include approximately 22,000 project instances, a knowledge library, and a 100-task benchmark. Across three coding models, the skill improves average rubric scores and reduces task time.
Critical strengths
-
The paper addresses a timely and important problem given the growing interest in agents for 3D content creation and the scarcity of high-quality procedural 3D code data. Blender tutorial videos are a valuable source of expert workflows, including intermediate operations and implementation details that are often absent from collections of finished assets. Turning these demonstrations into executable reconstructions and reusable procedural knowledge is a meaningful contribution. The substantial reconstruction and expert-validation effort is also a strength.
-
The extracted procedures provide a practical way to improve existing coding agents without updating their parameters. The reported improvements across the three evaluated models, together with reduced task runtime, suggest that the library provides useful guidance for new 3D tasks. The gains in challenging capabilities such as rigging and simulation are particularly encouraging.
Critical weaknesses
-
The current experiments demonstrate the usefulness of the complete BlenderLore Skill package, but do not sufficiently isolate the contributions of its individual components. Comparisons against workflow-only guidance, direct retrieval of tutorial text or code examples, and knowledge extracted without executable reconstruction would help establish the value of the proposed construction pipeline. Evaluation beyond the authors’ benchmark and additional agent configurations would also strengthen the evidence for generalization.
-
Although the collection effort is substantial, the source collection remains relatively modest given the breadth of Blender capabilities the paper aims to cover. Moreover, the approximately 22K project instances include adaptations and combinations of existing methods, while the number of distinct reusable skills is not clearly reported. More detailed statistics on skill diversity, coverage, and redundancy would help establish the library’s effective scale.
Questions for authors
-
Can you report construction costs, reconstruction success rates, and the amount of human intervention needed to create this dataset?
-
It seems that the dataset mainly contain atomic skills level demonstrations like Luminous silhouette, Emissive star details, etc. Those skills cover only a basic operation of a complete artist creation process. How well can the model combine those skills and maybe make an asset from beginning?
-
Do you think the 3d ability of frontier llm just emerge or is trained using some demonstration data like those you made?(Just curious, do not affect my final score)
4 comments
Reply to Reviewer fsn6, Part 2
W2: Statistics on the distinct reusable skills: scale, coverage, and redundancy.
We thank you for this suggestion. We added descriptive statistics of the procedural knowledge library.
1. Size and coverage. Our procedural knowledge library retains 9,411 distinct entries, covering all eight capabilities of the benchmark:
| GEO | PROC | SURF | SCN | RIG | ANM | SIM | PIPE | Total |
|---|---|---|---|---|---|---|---|---|
| 2,430 | 1,225 | 2,347 | 1,594 | 440 | 456 | 175 | 744 | 9,411 |
These statistics are now in of the revised PDF.
2. Redundancy. The loop of reconstruction and expert review yields 13,733 entries. We then run several rounds of heuristic deduplication and merging of similar entries with Codex (GPT-6 Astra), which leaves 9,411. To further address your concern, we embedded each entry (title, Applies when, and Rule, concatenated) with Qwen3-Embedding-0.6B and ranked all entry pairs by cosine similarity. Four human experts examined the 100 most similar pairs and found no redundant pair.
Q1: Construction costs, reconstruction success rates, and the amount of human intervention.
We thank you for this question. Building the dataset cost about USD 40K in API usage and about USD 9K in expert review. We processed 3,475 videos with 9,504 assets; experts accepted 8,918 assets (93.8%) from 3,003 videos (86.4%). Experts helped refine the workflow early on, which is a one-time effort. With the workflow fixed, our data pipeline is highly automated, and experts only accept or reject its outputs. On our annotation platform, an expert compares the result images extracted from the raw tutorial with an interactive 3D view of the asset built by the pipeline, and accepts or rejects in batches. This takes about 30 seconds per asset, about 1% of the total time to produce an asset.
Q2: The dataset seems to contain mainly atomic skills; can the model combine them to make an asset from the beginning?
The unit of our dataset is a complete asset, not an atomic skill. Each of the 8,918 reconstructed assets has a build.py that builds it from an empty scene to the final result. Entries such as Luminous silhouette are the reusable methods we extract from these complete builds.
Agents do combine them into complete assets. Every benchmark task asks for a complete asset built from scratch. We automatically inspected the 100 evaluation trajectories of GPT-6 Astra using Claude Code with Claude Opus 5.5. Per task, retrieval hits 27.1 entries on average, the agent reads 11.4 of them in full, and applies 6.8 in its final solution. These 6.8 entries come from 2.7 different raw tutorials on average, and 87% of the trajectories apply entries from two or more tutorials.
Q3: Does the 3D ability of frontier LLMs just emerge, or is it trained with demonstration data?
Our observations suggest that frontier LLMs such as GPT-6 Astra and the recent Claude Fable 5.2 have been trained on at least some demonstration data. On the assets of some tutorial videos they perform far better than their general level would predict: given only a reference image, without the tutorial, they reproduce the asset almost exactly.
We sincerely thank you for your exceptionally careful and constructive review. We hope that our rebuttal has addressed your concerns, and we would be happy to answer any further questions.
[1] VIGA: Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning.
Thank you for your additional experiments and clarification. My concerns have been addressed and i will keep my positive rating.
Appreciation for the Reviewer’s Feedback
We again thank you for the time you have devoted to improving our manuscript. We will carefully incorporate the above suggestions into the next revision.
Best regards,
Authors
Reply to Reviewer fsn6, Part 1
Dear Reviewer fsn6
We sincerely appreciate your insightful feedback and your recognition of our work. Below, we address each concern:
To directly address your concern, we added an ablation study that isolates the contribution of each component.
Setup. BlenderLore Skill has two parts: generic workflow guidance, refined with human experts, and a procedural knowledge library that the agent retrieves from on demand. We separate the two. Each configuration adds exactly one retrieval resource to the generic workflow guidance: (i) tutorial transcripts, in the form of our illustrated tutorials, where each step transcribed from the raw tutorial video records the narration, key frames, on-screen text, parameters, and intermediate result; (ii) executable code examples, namely the
build.pyscripts of the 8,918 expert-accepted projects; (iii) unverified procedural knowledge: an early version of our pipeline extracted 21,344 runnable entries from all 3,475 videos in one pass, without reconstruction or expert review; (iv) the procedural knowledge library (BlenderLore Skill). These configurations use Codex with GPT-6 Astra on all 100 tasks, with the same judge as the main experiments. Every resource is mounted the same way, as Markdown files with a manifest that the agent searches, and the agent is told the data structure. Following your suggestion, we also evaluate VIGA [1], with GPT-6 Astra as well, as a representative 3D code agent.build.py)Findings.
We have added this ablation study and its analysis to Sec. 6.4 of the revised PDF.