- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
A nice fine-tuned model that allow designers to "rotate" an input image as if it is made in 3D.
It is nice because artist sketches are in 2D, yet we'd like to see them from different angles. This method preserves the creative expression of the artist, and having AI to augment it to different view points.
Critical strengths
It works.
nice initial synthetic data generation pipeline and the following verifiable reward during an RL phase. very clever.
Critical weaknesses
In their human study, they don't ask the designers to use the tool in the manner of Figure 2.
In a proper study, they should get desginers who actually spend time drawing a novel design, along with a prompt, and have their tools make it "rotatable". Then, they can ask the designers some in-depth questions on if the tool was useful in any way, and what are some rooms for improvement.
More technically, since the designers did not give the initial sketch by hand, we do not know if the proposed method handles out of domain generalization well. For example, if I give it a very rough pen/paper sketch, would the proopsed method even work? Can it turn a concept sketch like this into rotatable form? https://thereader.mitpress.mit.edu/frank-gehry-and-the-art-of-drawing/
Ultimately, for a project that's so intimately related to a designer's self expression, this project failed to capture the human interaction in its studies. Its participants are reduced to robotically labeling either they preferred A or B. I find this extremely dehumanizing and disappointing.
Questions for authors
Do a proper human study where you give designers creative prompts and have them actually create a novel design in 2D, using whichever tools they like. Then, run this through your tool to make it rotatable in 3D. Have designers play with your tools for 1 hour in free-form, and get detailed feedbacks from them. Report these findings as well.
It would cost you 40 hours of labeling time (assuming you hire 40 designers), which would be very cheap in comparison to fine-tun a 14B model. It would do your work justice.
Do you have an IRB for your subject study?