- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DragForge with 18 task families and 60 configurations spanning placement, path-constrained manipulation, shape production, selection, and dynamic control. Policies observe screenshots and execute bounded waypoint sequences
Critical strengths
- The writing is clear.
Critical weaknesses
-
The task settings in this paper rarely appear in real GUI operations. In the introduction, the authors use dragging operating-system files as an example, but agents such as Claude Code can accomplish this operation simply through bash commands. The paper lacks a justification of its motivation, especially why are drag operations worth evaluating?
-
Lack of benchmark calibratio. Specifically, the authors should add a condition that removes the visual input and includes only the text brief, asking the model to respond, to test whether the model’s performance truly comes from visual understanding of the screenshot.
-
The author lack the analysis of Outliers. Such as, why Kimi K3 is 0 on ping-pong, but Qwen 3,8-27B got 9.3, also why claude opus 5.5 only got the 10.5 but Kimi K3 is 37.4 on flappy. The author should provide the case study of this outliers. not just show how the task looks like.
Questions for authors
1: Could the author provide the motivation of drag bench? I don't understand why this little problem have any constribution to the Agents development.
2: Based on the paper’s current presentation, I am confident that this benchmark has no potential to become a mainstream benchmark for LLMs or GUI agents. I hope the authors can rebut my view, and I am happy to discuss it with them.