- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes OrigamiBench, a new benchmark for testing whether vision-language models can understand origami as a sequence of physically grounded actions that transform a sheet of paper into a target shape. To support this, the authors introduce a high-level executable language that represents origami as sequences of formal folding operations. The benchmark contains manually annotated, randomly generated, and LLM-generated folding trajectories, with all visual observations rendered synthetically. Models are evaluated on tasks ranging from predicting or inferring individual folds and resulting states to reasoning over multi-step folding sequences and generating a complete sequence of folds that transforms an initial state into a target state. The results show that while current models can often reason about individual folding steps or infer plausible local transformations, they lack a robust causal understanding of the folding process and struggle to plan or execute sequences of folds over multiple steps.
Critical strengths
-
The problem is interesting, and the paper introduces a distinctive evaluation of leading VLMs on executable sequential origami reasoning, including grounding visual states in fold-action programs and inducing folding procedures. While prior benchmarks have already evaluated VLMs on origami-based spatial reasoning, this work appears novel in its emphasis on executable action-sequence reasoning and synthesis.
-
The benchmark contains a diverse set of trajectories, including manually annotated human-designed origami procedures as well as randomly and LLM-generated procedures, although all visual observations are synthetically rendered and limited to 2D.
-
The evaluation is broad, testing models across increasingly demanding levels of reasoning, from single-step state/action understanding to multi-step execution and ordering, plan completion and steering, and finally generation of a complete folding program from initial and target visual states.
Critical weaknesses
-
The scope of the benchmark is relatively narrow, focusing on sequential action reasoning within 2D flat-foldable origami.
-
The work focuses exclusively on evaluating existing VLMs and does not investigate methods for training models on the task. Hence, while the paper identifies limitations of current models, it does not propose methods for improving performance on the tasks it introduces.
-
While Origami is the art of turning 2D sheets to 3D structure, the benchmark is restricted to flat-foldable, essentially 2D origami and does not evaluate genuinely 3D folded structures. Although the authors mention 3D origami as future work, it remains unclear whether the proposed language and tests and results naturally extend to non-flat 3D configurations.
-
All images presented to the VLMs are synthetically rendered. This raises questions about how well the results transfer to real photographs or videos. Moreover, the renderings appear relatively schematic rather than photorealistic, so the benchmark may emphasize reasoning over an abstract visual representation that does not translate to real world visual conditions.
-
The paper does not include a human baseline. Human evaluation would help calibrate task difficulty and clarify whether low model performance reflects limitations of current VLMs or intrinsic difficulty or ambiguity in the benchmark itself.
Questions for authors
See weaknesses section