- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes to estimate the properties of a digital material model using frozen video models. The idea is to probe the video model intermediate representation to extract the typical channels of a material model, namely albedo, normal, roughness and conductivity. The key message is that large general purpose video models are more efficient than specialized feed forward methods trained from annoted data sets.
The paper give a comprehensive description of the proposed architecture in its main text, completed well by the apprendices. The validation, although often narrowed to "DINO vs video features" is quite complete.
I think the main contribution is in the use of frozen video features, optionally fused with DINO feature, to estimate material properties. I think this is an interesting research direction. But I lack elements to really assess how the proposed approach compare to direct specialized methods, both in speed/resolution and quality.
Critical strengths
1/ The paper makes an interesting point in that video features can be very informative for material recognition and estimation. 2/ The fusion of Dino features with the video ones is quite interesting and could certainly be pushed further in the future, although not so critical in the experiment numbers for now.
Critical weaknesses
1/ I think the specialized e.g., feed forward material estimation methods are too quickly discarded. First, this a vast body of work that would deserve a bit more of discussion. Second, although there is certainly interest in knowing how far video models features can go to solve the task, it would still be very informative so see a more comprehensive comparison to specialized models.
2/ While I like the idea of fusing video features, I wonder whether exploiting the training data to adapt Dino feature (fine tuning, LoRA, etc) would help making the them more useful in the fused case.
I believe there are some citation issues (Zeng/Zheng?)
Questions for authors
1/ Would DINOv3 provide any improvement? 2/ Instead of keeping DINO frozen, would an adaptation performed using the same data used to train heads, help improving the DINO side of the feature extraction? 3/ There are other ways to use video models for the task, e.g. VideoNeuMat; how do they compare (at least at a high level) to the proposed method in term of final material appearance (beyond the actual PBR vs neural format)?