- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes IROHA, a training-free agentic harness that integrates planning, executable tools, video generation, and verification to improve visual reasoning with frozen backbones. Experiments across VBVR-Bench, RBench, and LIBERO-Plus demonstrate consistent performance improvements.
Critical strengths
The paper addresses an interesting problem and demonstrates promising improvements across multiple benchmarks and generative backbones. The experiments are relatively comprehensive, including ablation studies and comparisons with existing methods.
Critical weaknesses
-
The proposed IROHA framework largely follows established agentic workflows, including planning, tool execution, verification, and feedback-driven refinement, which are already standard capabilities of existing agent frameworks and harnesses. The paper mainly applies these existing components to video generation without introducing a substantially new algorithm or mechanism. In particular, it remains unclear what technical challenges specific to visual reasoning are addressed by IROHA that cannot be handled by existing agent frameworks.
-
Despite emphasizing long-horizon reasoning as its central contribution, the main experiments do not convincingly evaluate this capability. For instance, the RBench evaluation primarily relies on prompt rewriting and best-of-three candidate selection, rather than sequential reasoning with state tracking and error recovery. Although the paper includes multistep demonstrations and a rule-based planner benchmark, these do not provide sufficient quantitative evidence of long-horizon visual reasoning in generated videos.
Questions for authors
I think this paper should provide a more explicit comparison with existing agent frameworks and harnesses, ideally through a controlled baseline using a general-purpose agent framework with the same tools and backbones, to demonstrate what IROHA uniquely contributes beyond existing agentic workflows.