acceptodds
Withdrawn from ICLR 2027

What to Count Before How Many: Text-Responsive Zero-Shot Object Counting via Two-Stage Regression

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Integrating text guidance into class-agnostic object counting has drawn growing interest. Historically, lightweight regression-based paradigms (e.g., CLIP-Count) have dominated the field due to their high inference efficiency and accuracy on dense, overlapped objects. However, standard regression architectures suffer from a fundamental, overlooked flaw: they tend to fall into visual pattern recognition, marginalizing text constraints entirely. Due to conventional evaluation metrics (e.g., Mean Absolute Error) assess only counting accuracy, this severe text-ignoring issue has long been obscured by over-counting on visual textures. To serve the need of overall counting accuracy, recent mainstream research has increasingly shifted toward heavy detection-based frameworks, which happens to be text-aware by its original purpose. Nevertheless, such detection frameworks enforce explicit target localization at the cost of high computational overhead. To bridge this gap, we revisit the regression-based paradigm and propose a two-stage framework that restores robust text-awareness without compromising efficiency. Stage 1 extracts fine-grained cross-modal features to generate a prompt-compliant mask corresponding to the specified objects. Stage 2 utilizes this mask as a spatial prior to generate a similarity map for density map regression, the density map will be used for the final counting process. Extensive experiments demonstrate that our approach eliminates text-ignoring issue while maintaining high computational efficiency and competitive counting accuracy.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Counting

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_y8JXOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a two-stage framework for text-guided zero-shot counting. A text-conditioned localization module first predicts a spatial mask, which is then used to select positive and negative visual exemplars for density regression. The method is evaluated on FSC-147 and a small diagnostic set designed to test text responsiveness.

Critical strengths

The paper addresses a meaningful problem: obtaining a plausible count does not necessarily mean counting the objects specified by the prompt. The reported improvement on the diagnostic set is substantial, and the component ablations provide some evidence for the usefulness of gating and positive–negative matching.

Critical weaknesses
  1. The implementation details do not describe a consistent model. Section 3.2 uses layers 9, 18, and 27 with feature dimension 1152, while Appendix A.1 specifies SigLIP-2 Base, which has 12 layers and hidden dimension 768. The reported 92M total parameters also appear incompatible with the stated full image–text backbone. More directly, Algorithm 1 applies ReLU immediately before sigmoid, so all mask values must be at least 0.5. Algorithm 2 then selects background exemplars using a threshold strictly below 0.5. As written, the background set is always empty and the background matching operation is undefined. These may be documentation errors, but they need to be reconciled with the implementation used for the reported experiments.

  2. The most relevant comparison is missing. T2ICount (CVPR 2025) directly addresses text insensitivity and introduces FSC-147-S to evaluate text-guided counting beyond dominant-category bias. Without comparison to T2ICount, the paper does not establish its advance on its central research question. This matters especially because the main improvement comes from a custom set of only 60 images, while the FSC-147 test MAE improvement over LGCount is just 0.05, with worse validation MAE and test RMSE.

Questions for authors
  1. Which exact SigLIP-2 checkpoint was used? Please reconcile the layer indices and feature dimensions with Appendix A.1, and clarify whether the reported parameter count includes both frozen encoders and the second-stage backbone.

  2. Does the implemented mask head apply ReLU immediately before sigmoid? If so, how are background exemplars with mask values below 0.5 obtained? If not, please specify the actual operation sequence.

  3. Can the authors provide a matched comparison with T2ICount on text responsiveness and counting accuracy? This comparison would materially affect my assessment of the contribution.

Reviewer_NtcMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses text-ignoring behavior in regression-based zero-shot object counting. The authors argue that density-regression models can achieve reasonable MAE by exploiting salient visual patterns while failing to follow fine-grained textual prompts. To address this, they propose a two-stage framework. Stage 1 uses frozen SigLIP-2 encoders and T2PRG to produce a text-conditioned spatial mask. Stage 2 uses this mask as a prior to mine positive and negative exemplars, builds object-versus-background similarity maps via dual-channel contrastive matching, and regresses them into a density map. The method is evaluated on a custom diagnostic suite of 60 images and 150 image-text pairs, FSC-147, CARPK, and ShanghaiTech. The authors report lower diagnostic counting errors than the compared regression baselines, competitive FSC-147 accuracy, and efficient inference.

Critical strengths
  1. The problem is reasonable. The observation that "counting the wrong objects correctly" can be invisible to aggregate counting error is useful. Evaluating contrasting prompts on the same image is a relevant diagnostic direction.
  2. The framework is coherent and reasonably well-engineered. Using a mask as a semantic bottleneck between localization and density regression is an understandable design.
  3. Great efficiency. With 92 M parameters and 10.79 FPS, the method is reported as the smallest and fastest among the compared approaches, and it is substantially faster than the detection-based baselines.
Critical weaknesses
  1. The motivation for the proposed regression framework is insufficiently established. The paper motivates its design through text-ignoring in regression methods, while acknowledging that detection methods already provide explicit target localization. In Table 1, all three detection baselines achieve substantially lower MAE than the proposed method. Additionally, the results do not establish an unmet prompt-following need relative to these alternatives. The reported speed advantage provides a practical motivation, but the paper does not convincingly establish when its accuracy-efficiency tradeoff makes this particular two-stage regression design preferable.

  2. The experiments do not adequately validate its claimed mechanism. The central claim still relies on MAE/RMSE, and the advantage in performance/efficiency is narrow. No dataset-level paired measure quantifies whether predicted changes match ground-truth changes or whether density is assigned to the requested objects. The evidence does not sufficiently distinguish better counting from a demonstrated correction of text-ignoring, or establish that the proposed decomposition is responsible. This gap affects the paper's main contribution, rather than a peripheral implementation detail.

  3. The novelty and comparison are insufficiently convinced. The text-ignoring problem is not novel and has been studied by other works such as T2ICount[1], which is listed in the references but omitted from the experiments; However, neither T2ICount method nor its proposed larger public FSC-147-S diagnostic subset is evaluated. This omission leaves open whether the reported improvement holds against a particularly relevant existing solution. The two-stage decomposition also has clear precedents, including Shi et al.[2] and Zou et al.[3], which appear in the bibliography but receive no substantive comparison in the main discussion.

  4. The diagnostic benchmark is self-built, small, and under-specified. The paper does not adequately document image provenance and selection, overlap with training data or categories, annotation and verification procedures, prompt construction, or a release plan. Without these details or image-level uncertainty estimates, the large diagnostic gain is difficult to assess for reproducibility and generality.

References:

[1] Qian, Y., et al. T2ICount: Enhancing cross-modal understanding for zero-shot counting[C]. CVPR, 2025.

[2] Shi, Z., et al. Training-free object counting with prompts[C]. WACV, 2024.

[3] Zou, Z., et al. Decoupling what to count and where to see for referring expression counting[C]. AAAI, 2026.

Questions for authors
  1. Validation of the central claim. On the same-image contrasting-prompt pairs, can you report whether changes in predicted count match changes in ground-truth count, together with quantitative target/background localization or density leakage? Can matched-backbone and matched-supervision controls separate the effect of the two-stage design from PhraseCut mask supervision?

  2. Statistical significance. Are the reported numbers averaged over multiple runs? Is the 0.05 test MAE gap over LGCount statistically significant?

  3. Robustness. How sensitive is performance to τobj\tau_{obj} and τbg\tau_{bg}? How is the case handled where a poor Stage-1 mask yields an empty positive exemplar set? Can error propagation from Stage 1 to Stage 2 be quantified?

  4. Some details. How are the MdotM_{dot}, GG, FimgF_{img} concatenated, as their dimensions seem to be totally different? Besides, the BCE in Eq. (6) seem to lack a negative sign.

Reviewer_emQJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses text insensitivity in regression-based object counting, where models may predict plausible counts without reliably selecting the objects specified by the prompt. The proposed framework separates text-guided localization from density estimation, combining token-to-patch relevance gating with positive and negative exemplar matching. The results show improved performance on a targeted diagnostic set while maintaining competitive counting accuracy on FSC-147.

Critical strengths

Relevant problem. The paper highlights an important limitation of conventional counting evaluation: accurate counts alone may conceal weak dependence on the text prompt.

Coherent method. The two-stage design explicitly connects textual grounding to density estimation. Positive and negative exemplar matching provides an intuitive mechanism for suppressing irrelevant visual responses.

Encouraging empirical evidence. On the diagnostic set, the method achieves an MAE of 18.29 versus 39.34 for the strongest compared regression baseline. Component and backbone ablations further support the proposed design.

Critical weaknesses

The central evidence for improved text responsiveness comes from a custom diagnostic set containing only 60 images and 150 image–text pairs. Its limited scale leaves uncertainty about generalization across categories, attributes, and scene conditions. Meanwhile, standard FSC-147 results are largely comparable to existing methods: test MAE improves only slightly over LGCount, while validation MAE and RMSE do not consistently improve. Consequently, the breadth of the paper’s main contribution depends heavily on this small diagnostic evaluation.

Questions for authors

How were the diagnostic images and contrasting prompts selected and annotated, and will the evaluation set be publicly released?

Could you provide image-level confidence intervals for the diagnostic results, accounting for multiple prompts associated with the same image?

Does the improvement remain consistent across different object categories, attribute types, and density levels?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.