- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Currently, Vision-language models exhibit sensitivity to quantity, but their ability to answer numerical questions remains uneven across visual scenes and tasks. To solve this issue, in this paper, the author introduces a two-phase post-training framework to develop more accurate and reusable visual number sense in pretrained models. In addition, they also construct a controlled synthetic dataset for training and evaluation across visual configurations and count ranges. In experiment, besides the general comparison studies to demostrate the effectiveness of the model, in ablation studies, the author find an interesting fact: Representation analyses reveal more accurate intermediate-layer count readout and stronger cross-configuration readout transfer.
Critical strengths
- author construct a controlled synthetic dataset for training and evaluation across visual configurations and count ranges for further use\
- the findings-representation analyses reveal more accurate intermediate-layer count readout and stronger cross-configuration readout transfer-might be insteresting and enlightening
Critical weaknesses
- The gains on synthetic images are much larger than on natural images. In Table 3, Qwen improves by 19.42 percentage points on Synthetic OOD and 5.21 on PixMo-Count. The corresponding gains for InternVL are 39.88 and 0.57 percentage points. These differences do not by themselves establish overfitting. However, the paper should explain why improvements are large on the authors’ generated images but more limited on natural images, and why transfer differs so much between the two models.
- I think the internal diagnostics are narrow. Because there is only one model, one layer, and one probe type in the experiment
Questions for authors
Could the authors provide actual dataset examples, side-by-side comparisons of training and OOD images, and failure cases, together with an analysis of the synthetic-to-real performance gap? Appendix A.2 provides LoRA settings, but are the vision encoder and multimodal projector frozen during both training phases? Which modules are actually trained?
6 comments
Comment 2
A4. Training/test examples and natural-image cases
Yes. We provide actual training and evaluation examples, together with matched original and highlighted natural-image cases.
Figure 2 in the main text already illustrates the visual configurations and numerical tasks. Appendix C, Figure 5 adds actual enumerative and factorized Coverage images, an RNS difference-counting pair, local counting, and Synthetic OOD examples at 10 and 14 objects, with their queries and reference answers. Appendix A.4.5, Figure 4 provides paired PixMo-Count examples.
A5. Which modules are trained in each phase
Yes. The vision encoder and multimodal projector remain frozen in both phases; training updates only language-decoder LoRA parameters.
For Qwen3-VL-8B, LoRA targets q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj. For InternVL3-9B, it targets wqkv, wo, w1, w2, and w3. Both use rank 16, alpha 32, and dropout 0.05. The pretrained decoder weights, token embeddings, output head, and biases also remain frozen. Full initializes the joint RNS+Interface phase from the Coverage adapter and continues optimizing its LoRA matrices. Appendix B.1, Table 14 lists the backbone-specific trainable modules and the freezing policy for both phases.
References
[1] Xingzhou Pang, Yifan Hou, Junling Wang, and Mrinmaya Sachan. Unveiling the Visual Counting Bottleneck in Vision-Language Models. Proceedings of the 43rd International Conference on Machine Learning (ICML), PMLR 306:96335–96355, 2026.
Dear Reviewers, as the discussion period comes to a close, please let us know if you have any remaining concerns or questions. We would be happy to address them. Thank you again for your time and consideration!
## Response — Reviewer a9oc
Thank you for recognizing the controlled dataset and the representation findings. We address the transfer gap, differences between backbones, diagnostic scope, concrete examples, and trainable modules below.
A1. Why synthetic gains exceed natural-image gains
The target-highlighting experiment shows that natural-image errors partly depend on target presentation and visual context.
Our synthetic training controls object configurations and count labels, while natural-image questions additionally require selecting and separating the queried objects. On fixed PixMo-Count cases, we preserve pixels within target masks, object positions, and image dimensions, then either darken the background to 20% brightness or replace it with gray. Table A1 reports the gray-background results for Base and the selected Full checkpoint on 20 historical shared-failure cases and nine jointly correct cases (Appendix A.4.5, Table 13).
After gray-background extraction, Full answers 6/20 shared-failure cases correctly, compared with 4/20 for Base, and retains 8/9 correct answers in the jointly correct group, compared with 7/9 for Base. Full performs better under the same intervention in both selected groups. This supports improved use of clarified target information for numerical answers. The remaining errors and disrupted correct answers also show that target selection and visual context continue to affect natural-image counting.
Table A1. Qwen target-highlighting diagnostics, using the same Base/Full summary as Appendix A.4.5, Table 13. Full denotes the selected seed13 checkpoint. Original-image columns show the historical results used for group selection; gray-background columns show intervention results. Entries are numbers correct.
A2. Why the backbones show different transfer gains
We added Gemma as a third backbone, with Synthetic OOD performance between Qwen and InternVL.
The selected Gemma Full setting achieves 51.75% on Synthetic OOD 9–16, between Qwen’s 49.42% and InternVL’s 54.13%. Table A2 reports Gemma’s Base, Coverage-only, Interface-only, RNS+Interface, and Full results across synthetic and external tasks. All three backbones improve over their pretrained baselines on Synthetic OOD, with different gain magnitudes. The revised main Table 3 and Appendix D.6, Table 22 include the Gemma results.
Table A2. Gemma results across synthetic count ranges and external tasks. Accuracy (%).
A3. Scope of the representation and behavioral diagnostics
The diagnostic study combines two representation readouts with three numerical-response interfaces.
The main Qwen representation analysis uses the last prompt-token representation at language block 17 across Base, Coverage, RNS+Interface, and Full. Ridge regression measures count readout, while prototype classification tests transfer from Grid to Scattered and Clustered layouts without refitting. The added behavioral study uses a separate set of 384 images for free counting and candidate-number scoring, together with 192 image pairs evaluated in both orders for quantity comparison. The original analysis is detailed in Appendix A.4, and the added diagnostic settings in Appendix A.4.4.
Coverage achieves 100% on comparison and 65.63% on both counting interfaces; Full reaches 86.20% on free counting and 86.98% on candidate scoring, compared with 82.81% for Base (Table A3). Full improves exact-count responses while comparison remains near ceiling. The readout–generation dissociation is consistent with the distinction between available quantity information and symbolic output discussed by Pang et al. [1]. The new behavioral results are included in Appendix A.4.4, Table 12.
Table A3. Qwen behavioral diagnostic. Free counting and candidate scoring use 384 images; quantity comparison uses 192 pairs in both orders. Accuracy (%).