acceptodds
Under review as a conference paper at ICLR 2027

The MixCount Dataset: Exact Annotations at Scale for Open-Vocabulary Object Counting

Abstract

Object counting is a foundational vision task with over a decade of dedicated research, yet state-of-the-art models still fail systematically in the mixed-object setting that dominates real-world applications such as industrial inspection and product sorting. We show that this gap is strongly driven by limitations in existing training and evaluation data. Counting annotation is hardest exactly where it matters most, since a dense scene has to be labelled instance by instance, and existing datasets contain numerous errors. Existing synthetic alternatives remove the annotation cost but lack diversity and realism. We address this with MixCount, a dataset and benchmark for mixed-object counting whose annotations are exact by construction. An automatic pipeline synthesizes images, fine-grained textual descriptions and counting annotations at scale, eliminating the labeling ambiguity that plagues prior datasets. Evaluating state-of-the-art counting models on MixCount exposes severe degradation in the mixed-object setting. More importantly, training these models on our synthesized data yields substantial gains on real-world benchmarks, reducing MAE by 20.14% on FSC-147 and by 18.3% on PairTally. These results establish MixCount as both a benchmark and a training dataset for fine-grained counting, and demonstrate that exact annotations at scale help address a long-standing data bottleneck in counting models.

Primary area:
datasets and benchmarks
Keywords:
Counting, Mixed Objects, Machine Learning, Dataset, Counting Dataset

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_BfgpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces MixCount, a synthetic dataset and generation pipeline for open-vocabulary object counting. It contains 58,000 images generated from scanned 3D assets, with instance-level annotations, visibility information, text descriptions, and visual exemplars. The authors evaluate existing counters on MixCount and show that adding synthetic training images improves performance on real-world benchmarks.

Critical strengths

The strongest result is the improvement on real-world benchmarks after adding only 1,000 MixCount images. The gains on PairTally are substantial for both GeCo2 and CountGD++, suggesting that the dataset has value beyond evaluation on its own synthetic distribution. The combination of instance annotations, visibility measurements, and multiple prompt types also makes it a potentially useful research resource.

Critical weaknesses
  1. The evaluation omits a directly relevant method, T2ICount. T2ICount (CVPR 2025) explicitly addresses text insensitivity and the dominant-category bias of existing counting benchmarks, and introduces FSC-147-S to evaluate this behavior. These issues substantially overlap with the motivation of MixCount. Testing CountSE, CountGD, and CountGD++ without a method specifically designed to address text sensitivity leaves the benchmark evaluation incomplete, particularly in the text-only setting. It remains unclear whether MixCount exposes limitations that persist in such methods or primarily highlights weaknesses of the selected models. This omission also weakens the argument that the observed failures are principally a training-data problem.

  2. The claims about “exact annotations” and the data bottleneck go beyond the evidence provided. Rendering provides precise instance identities and masks, but it does not guarantee that a text description unambiguously identifies the intended objects in the final image. Distinct descriptions can overlap semantically, and distinguishing attributes may become invisible under occlusion or changes in viewpoint. The manual auditing is useful, but the paper does not establish the reliability of these image–text associations. Moreover, MixCount changes realism, scene composition, prompt detail, visibility filtering, and data volume simultaneously. The transfer results show that this combination helps; they do not establish that annotation errors in existing datasets are a principal cause of counting failures. Controlled ablations are needed to support that explanation.

Questions for authors
  1. Can the authors evaluate T2ICount on MixCount under the same text-only protocol? Does it exhibit the same failure patterns, and does MixCount training provide additional gains?

  2. How are descriptions checked for semantic ambiguity against other objects in the same scene, rather than only for correctness on an isolated asset? Is there a quantitative audit of the final image–text–count annotations?

  3. What evidence separates the benefit of annotation accuracy and visibility filtering from that of more diverse scenes or additional training data? Were the data-augmentation comparisons matched for training budget and sampling protocol?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Bfgp 1/2

We thank the reviewer for the careful reading and for highlighting the important gains on real-world benchmarks. Following your suggestions, we added T2ICount to our evaluation and added controlled ablations. This confirmed that the text-only architecture of T2ICount greatly benefits from MixCount data, and quantified the importance of non-noisy labels and visibility filtering with two new ablations.

 

W1/Q1. T2ICount benefits from MixCount data

As suggested, we now evaluate T2ICount under the same text-only protocol, and train it with additional MixCount data (Tab. 4, p. 10). Our revised paper includes the additional Fig. 11 (App. A.5, p. 18), which highlights the failure modes of the original pretrained T2ICount on self-similarity, repeating backgrounds, and mixed objects images. Even though T2ICount was designed to explicitly address text insensitivity, it is still challenged by the settings in MixCount.

To train T2ICount on MixCount, we initialize it from its pretrained checkpoint and train it for 1 epoch on 1k randomly sampled MixCount training images using the same training procedure as in the published T2ICount GitHub repository. Training with MixCount further improves T2ICount’s MAE by -48.13% on MixCount and -37.5% on PairTally:

MethodPairTally MAE ↓PairTally RMSE ↓MixCount MAE ↓MixCount RMSE ↓
T2ICount61.4689.2935.1158.70
T2ICount + 1K MixCount38.41 (-37.50%)64.65 (-27.60%)18.21 (-48.13%)24.65 (-58.01%)

 

W2/Q2. Text annotations

We agree that the guarantee differs in kind for the two annotation types: MixCount guarantees correct counts, masks, class identities, and that objects are sufficiently visible. In contrast, image-text annotations are verified. We now make this distinction explicit in our response and will state it in the final version.

Text annotation accuracy. The manual and automated verification protocol of text annotations is now described in full in App. Sec. B.4 and B.5, p. 20–21:

  • Description review. The generated descriptions are manually checked against a render of their asset, verifying that each one matches the object and does not omit a detail needed to distinguish it from a visually similar one. The vast majority are correct on the first attempt.
  • Cross-asset disambiguation. To verify that distinct descriptions do not overlap semantically, a vision-language model is asked to match each description to the correct render among several candidates, and every mismatch is flagged.
  • Conflict resolution. All flagged mismatches are inspected manually to separate VLM errors from genuine ambiguity, and ambiguous assets are removed. Crucially, this protocol operates on the 2399 3D assets rather than on the 58,000 rendered scenes, which is what keeps it tractable. Ambiguous assets are discarded at every stage, which is why MixCount contains 1522 of the 2399 objects initially present in DTC. The quality of the resulting annotations is further supported by the significant improvements obtained in the text-only setting with CountGD++ and T2ICount.

Ambiguity under occlusion and viewpoint. Distinguishing attributes that become invisible are, in our view, a strength of MixCount. For example, some scenes contain wooden toy cubes, each showing a letter and matching pictures on its faces (K with a koala, kiwi and key, M with a mouse, mushroom and mango). When the letter faces away from the camera, a cube showing a koala still belongs to the K class. A strong counting model should reason about the scene as a whole to infer this, as a human would. Existing datasets do not test this ability, and MixCount provides both training and evaluation data for it.

AuthorsOpenReview ↗

Response to Reviewer Bfgp 2/2

W2/Q3. Controlled ablations supporting exact annotations

All comparisons are matched. Every run adds 1,000 images to the same real training set, with model, schedule, sampling and hardware fixed (CountGD++: 30 epochs, batch size 4, LR of 10−410^{-4} decaying by a factor of 10 every 10 epochs, trained jointly. GeCo2: 1 epoch starting from pretrained FSC-147 checkpoint, batch size 4, LR of 10−410^{-4}, weight decay 10−510^{-5}). We now state this in Sec. 4.2, and add the following controlled ablations for GeCo2 to Tab. 8 on p. 19:

  • MixCount (no visibility filtering): we generate a new dataset of 1000 MixCount images without applying visibility filtering to isolate its effect.
  • MixCount (noisy): to controllably simulate the missing annotations in real-world datasets, each annotation has a 5% dropout chance.

Removing visibility filtering or injecting label noise both increase error on FSCD-LVIS and PairTally, further demonstrating the need for a dataset with exact image annotations, and they are now added to our revised paper in Tab. 8.

Training DataFSCD-LVIS MAE ↓PairTally MAE ↓
Real only (baseline)
FSC-147 [real]38.3352.36
Real + synthetic (ablating dataset factors)
FSC-147 [real] + 1K MixCount (single-category) [synthetic]33.8551.14
FSC-147 [real] + 1K MixCount (no visibility filtering) [synthetic]24.4036.95
FSC-147 [real] + 1K MixCount (noisy) [synthetic]22.8434.27
FSC-147 [real] + 1K MixCount (ours) [synthetic]20.1131.88
Synthetic only (testing scale)
1K MixCount (ours) [synthetic]37.6049.33
56K MixCount (ours) [synthetic]18.8935.85

We hope this addresses the reviewer's concerns and would be glad to answer any follow-up.

Thank you for the additional experiments and clarifications. The T2ICount evaluation addresses my first concern, and the new ablations strengthen the evidence for the benefits of visibility filtering and cleaner labels. I also appreciate the clarification of the matched training settings.

My main remaining concern is the validity of the final image–text–count associations. The described verification operates at the asset level, whereas my question concerns whether the target remains identifiable from the actual rendered image and prompt. In the cube example, reasoning is appropriate if the relevant correspondence can be inferred from the available input; the renderer’s knowledge of asset identity alone does not establish this. A quantitative audit of representative final scenes would help distinguish challenging examples from underdetermined ones.

The noise ablation supports the benefit of cleaner labels under the tested corruption, but does not establish that annotation errors in existing datasets are a principal cause of counting failures. Moreover, visibility filtering removes objects before rendering, so its ablation also changes scene content.

AuthorsOpenReview ↗

Quantitative audit of the final image-text-count annotations

Thank you for your response and for recognizing that our T2ICount evaluation addresses your first concern and that the new ablations strengthen the evidence for the benefits of visibility filtering and cleaner labels.

Identifiability of final image-text-count associations. Following the reviewer’s suggestion, we manually audited representative final scenes to precisely distinguish challenging examples from underdetermined ones. We sampled 100 (image, target class) pairs in MixCount. Given only the rendered image and the concise text prompt, a human annotator placed a point on every instance of the target. Points were matched to the ground-truth instance masks. We include the results of the manual audit below:

  • Instance level: recall 93.6%, precision 93.8% overall.
  • Count level: MAE 1.54 (about 7% relative error). 68/100 counts are exact, and 86/100 are within ±1 with at most one missed instance.
  • Where errors occur: they concentrate in the densest scenes (MAE 4.71 for scenes with 91+ objects, versus 0.20–1.40 below that) and in fine-grained classes such as apples of similar shades (recall about 80%). These differences are specified in the text and visible in the image, but require careful inspection. The median annotation time was 42 s per image.
  • Underdetermined cases: in 3/100 images, the text prompt was deemed ambiguous. In each, scene lighting made the colour cue in the prompt indistinguishable from another class in the same scene (e.g. a “white” figurine rendered greenish under dim lighting). The audit therefore indicates that about 97% of sampled image-text pairs are unambiguous, with residual errors mostly reflecting the difficulty of fine-grained, dense counting rather than missing information. We thank the reviewer for prompting this analysis. We will add the audit results and our analysis to the paper.

Noise ablation. We agree, and will revise the claim. The noise ablation shows that cleaner labels improve counting performance under controlled corruption, but it does not establish that annotation errors in previous datasets specifically are the principal cause of counting failures. We will replace “this gap is strongly driven by limitations in existing training and evaluation data” with “training data with exact annotations substantially reduces these failures”, and present annotation quality as one contributing factor alongside scale and diversity.

Visibility filtering. We agree that this ablation also changes scene content. It is designed to evaluate a step of our generator as a whole, and shows that including this step improves downstream counting performance.

We thank the reviewer again for comments that have made the paper’s claims more precise.

Reviewer_jYRbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MixCount, a large-scale synthetic counting dataset that has exact annotation and fine-grained text description. This paper contributes i) a high-quality, photorealistic dataset for open-vocabulary counting, ii) an automatic generation pipeline. Experimental results show that models trained with MixCount achieve further improvements on realistic datasets. Experimental results show that training on MixCount improve the performance of GountGD++ on real-world counting benchmarks, and demonstrated the transferability for real-world scenarios on GeCo2.

Critical strengths
  1. Clear Motivation: Current large-scale counting datasets either contain numerous errors or lack diversity and realism. Counting models trained on these datasets fail in many scenarios, such as when there are similar objects and repetitive backgrounds.
  2. High-quality Dataset and Clarified Generation Pipeline: The proposed MixCount dataset covers diverse classes and backgrounds. The synthetic images approximate real image. The generation pipeline takes into full consideration of both the foreground and the background.
  3. Significant Performance Improvement: By incorporating MixCount into the training process of the model on the realistic dataset, the performance improve on the challenging PairTally and FSC-147.
Critical weaknesses

Major Weaknesses

  1. Motivation–evidence gap: The paper highlights difficulties in high total amount, distinguishing similar objects, self-similarity components, and repetitive background as failure modes, and attributes these bad cases to data gap, as shown in Fig. 2. Although these challenges are partially considered in the design of generation pipeline, it remains unexplored how, and to what extend can these designs alleviate the corresponding failure modes. Actually in Fig. 2 (b), CountGD++ trained with MixCount still double counts in the case of sunglasses (right up), and even missed one in the left bottom. A further analysis of how these challenging cases are represented in MixCount, or a quantitative evaluation of how effectively MixCount mitigates the corresponding failure modes, would be helpful.
  2. In-sufficient technical details: Several key designs of the generation pipeline require clarification. For example, what’s the exemplar score s and how it is computed in Line 248-249/Fig. 5, what’s the VLM prompting protocol in Line 335. In line 290-296, what’s the criteria of ‘too similar’? This raises a relevant question: how does MixCount represents variations in appearance within the same counting category? In real-world scenarios, objects of the same classes often exhibit natural variations in shape, texture and other aspects, which is far beyond just copying, rotating or scaling the same 3D asset.
  3. Limited experiments: Table 1/2 validated the benefits of additional training on 1K MixCount images on CountGD++, and verified the transferring ability from synthetic to real data on GeCo2, respectively. However, these improvements rely on single models respectively; it remains unclear whether the performance gains and the transfer value can be reliably generalized to other counting models.

Minor Weaknesses

  1. A concurrent work, Count Anything at Any Granularity, also construct a synthetic counting dataset of high diversity based on 3D assets. A discussion about this work would help clarify the distinctive contributions of this paper.
  2. In-sufficient experimental details. For example, for additional training on 1K Mixcount/MCAC/OminiCount samples in Table 1/2, is there any class/category leakage compared to the corresponding test set? This is particularly important, given that the performance trends for MCAC in Table 1/2 are completely opposite, but there seems to be no discussion of this discrepancy.
Questions for authors

My initial rating is Weak Reject, as there is no borderline scores. I appreciate the construction of the synthetic dataset and its engineering value. My main concern is that the experimental design lacks sufficient rigor to convincingly quantify the dataset's contributions. If the authors can address these experimental concerns during the rebuttal, I would be willing to raise my rating.

  1. How and to what extend can the MixCount help to mitigate the failure mode described in the abstract and intro?
  2. Can the performance gains and transfer value generalized to other counting model expect GountGD++/GeCo2? Please provide finer experiments concerning Major Weakness 3.
  3. Please provide more technical and experimental details concerning Major/Minor Weakness.
4 comments
AuthorsOpenReview ↗

Response to Reviewer jYRb 1/2

We thank the reviewer for the careful reading and for the positive assessment of the motivation, the dataset quality, and the gains on real-world benchmarks. ICLR allows a 10th page in the revised paper, and following your suggestion, we

  • added experiments to better demonstrate the claims and motivation behind MixCount (p. 9, Table 3),
  • provided all the technical details to clearly show that all comparisons and ablations are controlled and meaningful (p. 8, Sec. 4.2),
  • demonstrated performance gains on a third model, T2ICount (p. 10, Table 4).

 

W1/Q1. Motivation and evidence

Thank you for the interesting suggestion to better quantify the impact of MixCount on the failure modes identified in Fig. 2, a key motivation behind our dataset. We now perform this experiment in Sec. 4.2 (p. 9) and measure a reduction of -17% to -35% in these specific settings. The added text and results read:

“We build datasets corresponding to each failure mode of Fig. 2 to exactly quantify the added value of MixCount in each scenario. To find images with repetitive backgrounds from the test set of FSC-147, we first mask out every annotated instance of the counted objects and consider the remaining unmasked patches at varying resolutions. We take each patch’s dominant 2D Fourier frequency, which encodes the spacing and orientation of its texture. Images score highly when many patches agree on this frequency, signaling a repeating background pattern. This acts as an initial filter, and the final 100 images are manually chosen by visual inspection of the top candidates. Finally, for similar objects, we use the PairTally benchmark, and for self-repetitions, we evaluate on the “sunglasses category” of FSC-147. The results in Table 3 demonstrate a reduction of -17% to -35% in these specific settings.”

ModelTraining dataSimilar objects MAE ↓Self-repetitions MAE ↓Repetitive backgrounds MAE ↓
CountGD++FSC-14735.2739.754.77
CountGD++FSC-147 + 1k MixCount22.94 (−34.96%)32.86 (−17.33%)3.53 (−25.99%)

 

W3/Q2. Generalization beyond CountGD++ and GeCo2

We now validate the benefits of additional MixCount training data on three models: CountGD++, GeCo2, and T2ICount. T2ICount's comparison is performed under the training protocol of the T2ICount paper, and the results are reported below.

MethodPairTally MAE ↓MixCount MAE ↓
CountGD++35.278.12
CountGD++ + 1K MixCount22.94 (-34.96%)2.11 (-74.01%)
GeCo252.3629.98
GeCo2 + 1K MixCount31.88 (-39.11%)13.52 (-54.90%)
T2ICount61.4635.11
T2ICount + 1K MixCount38.41 (-37.50%)18.21 (-48.13%)
AuthorsOpenReview ↗

Response to Reviewer jYRb 2/2

W2/Q3/MW2. Technical details of the pipeline

We added all requested information regarding:

  • The exemplar score s (p. 5, Sec. 3.1), which is now defined explicitly as “si=∣Mi∣/max⁡∣Mj∣∈(0,1]s_i = |M_{i}| / \max |M_{j}| \in (0,1], the visible area of the instance normalized by the largest visible area of its class in that image. This score is low as soon as the instance is small or heavily occluded.”
  • The VLM prompting protocol is now more detailed (p. 20–21, App. B.4), and describes the renderings used, the exact prompt, and the automatic and manual verification stages in great detail.
  • Descriptions of the manual and automatic checks applied to discard duplicate objects from DTC (p. 21, App. B.5). If either check fails to distinguish between a pair of objects, a single one is kept.

Intra-class variations. Regarding intra-class variation, it is modeled in two ways in MixCount: Within a given class, each object is applied a random scale jitter, such that each instance is geometrically slightly different. Variations in texture and aspect are naturally present in our dataset. For example in Figure 5, the “gold paneled volleyball” and “yellow white volleyball” both belong to the FSC-147 supercategory “ball”, such that the model being trained must count both types when given “ball” as a prompt. We now clarify this point in p.5, L 240 of our revised paper.

MCAC discrepancy. By design, there is strictly no class leakage in MixCount's added-training comparison. As described in p. 5–6 (Sec. 3.1), we assign each MixCount object to an FSC-147 category (e.g. "Light orange football" becomes "ball") and inherit that category's FSC-147 split. Since "book" is an FSC-147 validation class, "A Christmas Carol hardcover book" appears only in MixCount-val. This keeps asset variants within a single split and guarantees that no FSC-147 test class appears in MixCount-train or MixCount-val, so MixCount-trained models remain evaluable on FSC-147 without leakage.

The poor performance of MCAC on FSC-147 can instead be explained by:

  • Lack of occlusion modeling. Objects hidden in the image remain in the ground truth (Fig. 3), which wrongly penalizes models during training.
  • Lack of photorealism and diversity. As seen in Fig. 4 (p. 4), MCAC only includes top-down views of a white box and very limited assets.

These are issues that MixCount is explicitly built to address and their respective impact on downstream performance is quantified in our ablations. We now explain more explicitly this surprising performance gap in our experiments (p. 8, L406-408). While adding MCAC hurts performance significantly on FSC-147, it helps on PairTally, since both PairTally and MCAC have mixed-object scenes. Still, the improvements from MixCount on PairTally are larger.

 

MW1. Discussion of Count Anything at Any Granularity

Thank you for this additional reference, we have added a discussion in Sec. 2 (p. 4):

“Concurrent work (Liu et al. arXiv 2026) generates counting data with text-to-3D models, but the resulting 3D assets lack visual quality. To improve realism, they feed the simulated images to a diffusion model, which may alter the number of objects and break the correspondence between the generated images and the original annotations. In contrast, MixCount uses real-world captures and physical simulation, reading object locations directly from the 3D scene to guarantee both visual fidelity and exact image–annotation alignment.”

We hope these additions address the reviewer's concerns, and we would be glad to run further analyses during the discussion period.

Thank you for rebuttal. I remain one concern about how MixCount addresses the failure modes in intro. The rebuttal quantifies the performance reduction in the corresponding settings, but it does not yet make fully explicit how each failure mode is represented in MixCount or how the corresponding generation choices are expected to mitigate it. Nevertheless, I consider this remaining concern acceptable, and the authors have largely addressed my other concerns. I will therefore raise my rating by one point, while still hoping for a clearer explanation of the underlying mechanisms.

AuthorsOpenReview ↗

Representation of model failure modes in MixCount and generation choices that mitigate them

We thank the reviewer for raising their rating and for this suggestion. For each failure mode, we clarify how it is represented in MixCount, which generation choice produces it, and its measured effect (Tab. 3). We will add this to the revised paper:

  • Similar objects: The generator deliberately builds scenes of similar classes: after the first class is sampled, the additional classes are produced by resizing it or by sampling another object from the same DTC category (Sec. 3.2), e.g. mixed fruits of different sizes or balls with different patterns (Fig. 5). These act as hard negatives, so the model must learn the fine-grained differences (size, colour, pattern, shape) that separate the target from similar objects.
  • Self-repetitions: Many DTC assets contain repeated parts, such as calculators with repeated keys, the two identical halves of a dumbbell, or bowls with symmetric patterns. Each object is labelled as exactly one instance, so the model learns to count whole objects rather than their parts.
  • Repeating backgrounds: We purposefully texture the floor with periodic VasTextures and Polyhaven materials, such as paving stones, tiles, brick, parquet and woven fabric (Sec. 3.2). These patterns are never annotated, which teaches the model to ignore background repetition.
Reviewer_A3P1OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MixCount, a synthetic dataset and generation framework for open-vocabulary object counting, comprising 58,000 images and approximately four million annotated instances with rich visual and textual annotations. Its generation pipeline integrates 3D assets, physical simulation, realistic rendering, and visibility filtering to create complex mixed-object scenes. Experiments with GeCo2 and CountGD++ demonstrate improvements on real-world benchmarks while revealing the limitations of existing models in challenging mixed-object scenarios.

Critical strengths
  1. A scalable dataset with dense annotations, multimodal prompts, and explicit visibility control.

  2. Promising improvements on real-world counting benchmarks across two representative models.

  3. The visualization experiments are comprehensive, and the figures are clear, well-designed, and visually appealing.

Critical weaknesses
  1. Controlled ablations, training comparability, and statistical variability require clearer reporting.

  2. Category overlap between splits and the treatment of asset variants remains unclear.

As my research expertise lies outside this field and I have limited experience in this area, I can only evaluate this paper from the perspective of a non-expert reviewer.

Questions for authors

Please see Critical Weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer A3P1

We thank the reviewer for the careful reading and for the positive assessment of the dataset's scale, annotation richness, and the clarity of our figures. We address the two weaknesses below.

 

W1. Controlled ablations

We now describe the training protocol more explicitly in Sec. 4.2 (p. 8) to make it clear that all comparisons are indeed controlled. It reads:

"In Tab. 1 and Tab. 2, every run adds exactly 1,000 images on top of the same real training set, with the model, optimizer, schedule, and hardware held fixed. For CountGD++, we train jointly on the FSC-147 and MixCount training images using the same hyperparameters as the original paper (30 epochs, batch size of 4, LR of 10−410^{-4} decaying every 10 epochs). For GeCo2, we start with the pretrained FSC-147 checkpoint and train for one additional epoch on the MixCount images with the same hyperparameters as the published paper (batch size of 4, LR of 10−410^{-4}, weight decay of 10−510^{-5}). Please refer to the CountGD++ and GeCo2 papers for further details." This makes MCAC (8.32 MAE on FSC-147), OmniCount-191 (11.77), and MixCount (4.68) directly comparable against the real-only baseline (5.86) for CountGD++ as only the source of added data varies, while the training procedure is fixed.

 

W2. Category splits and asset variants

We now describe the split construction more explicitly in Sec. 3.1 (p. 5–6). In particular, categories match pre-existing FSC-147 taxonomy and do not overlap across splits. Asset variants also all go to the same split. The clarifications read:

“Splits. We divide the MixCount dataset into train-val-test split, including 56,000, 1000 and 1000 images, respectively. Instead of assigning splits to the final images, we guarantee that no object found in the test set can be seen in the train set by splitting the objects to be counted directly, such that each object can only be found in a single split. To ensure compatibility and fairness with models trained or evaluated on FSC-147 (Ranjan et al., 2021), we first match each object in our dataset with a category in FSC-147 and use the same train-test split. This is achieved by first assigning an FSC-147 category to all our objects. For example, “Light orange football” and “Camo patterned basketball” both get assigned to the FSC-147 category “ball”. Then, since “ball” is a train class in FSC-147, the object “Light orange football” becomes a train class in MixCount and will only be found in training images. On the other hand, “Deep Learning textbook hardcover book” and “A Christmas Carol hardcover book” both fall under the category “book”, which is a validation class. This ensures asset variants go to the same splits, and also that MixCount-trained models remain evaluable on FSC-147 without leakage. 70% of objects in our dataset find a match in FSC-147 categories, and the remaining are all very distinct (“Dark blue salt shaker”, “Rubik's cube toy”) and get assigned to a split to match a target distribution of 80/10/10.”

We hope this addresses the reviewer's concerns and would be glad to answer any follow-up.

Reviewer_A3P1OpenReview ↗

Thanks

Thank you for your rebuttal. Due to differences in research expertise, I can‘t provide a sufficiently assessment. I kindly ask the AC to disregard my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised paper and additions

We thank all reviewers for their insightful feedback. We have submitted a revised version with all changes highlighted in red. The main additions and updates include:

  • Generalization across architectures: Extended evaluation to T2ICount to show consistent performance gains when training with MixCount data (p. 10, Tab. 4 and p. 18, Tab. 7 & Fig. 11).
  • Quantification of improvements on identified failure modes: Added a dedicated experiment quantifying performance improvements on the previously identified failure modes in Sec. 4.2 (p. 9, Tab. 3).
  • Additional controlled ablations: Added ablation studies measuring the importance of visibility filtering and label accuracy in App. A.6 (p. 19, Tab. 8).
  • Split construction & disambiguation: Detailed the category matching and split construction in Sec. 3.1 (p. 5–6) to clarify the absence of data leakage, and described in detail the text verification and asset filtering protocols in App. B.4 & B.5 (p. 20–21).
  • Technical details: Clarified the training protocol and baseline evaluation settings in Sec. 4.2 (p. 8, L395–401 & L408–410).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.