- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper utilizes the assessment systems from developmental psychology, including Carroll's Three-Stratum Theory of Cognitive Abilities and the Motor-Free Visual Perception Test (MVPT-3), to construct a benchmark KidVis for evaluating the perceptual capabilities of MLLMs on fundamental visual primitives. It defines six atomic visual primitives and maps them in pairs to 10 tasks, with each task consisting of 50 questions, totaling 500 questions. A zero-shot evaluation is conducted on 22 MLLMs, and the experimental results show that the composite score of ten 6–7-year-old children reached 95.31, while the average score of MLLMs is only 38.67. Even the strongest evaluated model, GPT-5, scores only 67.33, far below human children. This benchmark reveals that although MLLMs have demonstrated strong abilities in high-level semantic reasoning, they remain fragile on low-level visual tasks, and this weakness is not consistently alleviated by increasing model parameters.
Critical strengths
- The results of the 10 human children are highly valuable. Compared with this baseline, the weakness of the models is shown more clearly, and such a comparison is relatively rare in similar work.
- The experimental design is reasonable, and the result analysis is solid, including sensitivity analysis and confidence intervals.
- The writing is clear, and the task examples and failure cases are also comprehensive and easy to understand.
Critical weaknesses
- The related work is severely lacking. Table 1 only compares five general benchmarks, including MME, MMBench, MMMU, MathVista, and HallusionBench. It highlights the novelty of KidVis in terms of isolated perception, primitive decomposition, child-level, and low semantic. However, these four aspects already have multiple existing works. The concurrent work BabyVision (ICML 2026) [1] also takes the development of children's visual perception as its framework, and tasks such as maze tracing, block counting, and pattern completion heavily overlap with those in this paper. In 2025, VisFactor [2], Do You See Me [3], and Percept-V [4] directly build low-semantic perception benchmarks based on standardized visual perception tests such as FRCT, TVPS, and MVPT, the same tests this paper builds on. In 2024, CoreCognition [5] evaluates 230 models based on developmental cognitive science and reports that low-level abilities do not scale with parameter size. Among these, this paper only cites CoreCognition and merely mentions it in passing as a study of hallucination.
- The evidence for the scaling paradox is insufficient. The differences among the three sizes of Qwen3-VL (4B/8B/30B) are less than 5 points, making it difficult to say that there is a real difference. Moreover, the 30B version is MoE with approximately 3B active parameters, and it should not be compared based on the total parameter count. Also, the range of sizes is not large enough and does not cover the dense 32B and MoE 235B versions.
- Section 5.5 contains many mechanistic explanations, including that MLLMs lack an inherent inhibition mechanism to suppress background distractors, that ViT patch discretization fragments continuous paths into discrete visual tokens, and that MLLMs treat visual history as a static read-only buffer. None of these is supported by any experiment in the paper.
[1] Liang Chen et al. BabyVision: Visual Reasoning Beyond Language. ICML 2026. https://openreview.net/forum?id=ZD3BniFPto
[2] Jen-Tse Huang et al. Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs. arXiv:2502.16435, 2025. https://arxiv.org/abs/2502.16435
[3] Aditya Sanjiv Kanade and Tanuja Ganu. Do You See Me: A Multidimensional Benchmark for Evaluating Visual Perception in Multimodal LLMs. EACL 2026. https://aclanthology.org/2026.eacl-long.343/
[4] Samrajnee Ghosh et al. The Percept-V Challenge: Can Multimodal LLMs Crack Simple Perception Problems? COLM 2026. https://arxiv.org/abs/2508.21143
[5] Yijiang Li et al. Core Knowledge Deficits in Multi-Modal Language Models. ICML 2025. https://arxiv.org/abs/2410.10855
Questions for authors
- Could you expand the related work and discuss the differences between this paper and those works, in particular the works mentioned in weakness 1? This is important for judging the novelty.
- For weakness 2, could you provide the results of Qwen3-VL 32B and 235B? Does the non-monotonic trend still hold?
- For weakness 3, could you provide at least one piece of evidence to support these mechanistic explanations?