- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a benchmark (HumanPhys) and a corresponding corpus for evaluating human-centered physical perception in MLLMs. It covers 13 subtasks across six dimensions: spatial state, kinematics, blink events, deformation, illumination, and camera pose. The authors also evaluate multiple MLLMs and fine-tune Qwen3-VL-4B-Instruct on the proposed corpus.
Critical strengths
None
Critical weaknesses
- The abstract on the ICLR page is inconsistent with the abstract in the official submission, and the two claims conflict. The page states that "we further construct an accompanying training corpus of approximately 760K samples", whereas the main text states that "together with a 386K-sample training corpus generated by the same pipeline". The huge discrepancy of this magnitude raises doubts about the paper's credibility.
- The motivation is questionable. The authors build the benchmark because existing work lacks coverage of human-centered physical quantities in scenes, but the Introduction does not make it clear why evaluating these events is valuable. The authors should justify why MLLMs need to understand such events (e.g., blink counts and illumination on the face).
- The strength of MLLMs lies in complex linguistic reasoning or multi-step abstract reasoning. However, the proposed benchmark mainly targets the ability to recover physical quantities from vision (e.g., distance or trajectory), which makes it unlikely to be adopted as a benchmark for mainstream MLLMs. In other words, one would rather evaluate MLLMs on spatial reasoning than on counting blinks.
- As a dataset paper, it should at least present a case study for each sub-task to help readers build intuition. However, only Fig. 5 shows one of the 13 sub-tasks. It is also unclear why a small sample of the benchmark data was not included in the supplementary material.
Questions for authors
Response to the questions in the weaknesses.