- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AnimeText, a dataset of 735,060 images and 4,360,393 text boxes spanning comics, illustrations, posters, and edited images, which includes four kinds of annotation. Experiments demonstrate improvements over training on an equally sized dataset, particularly for non-dialogue text, and show further performance gains as the training scales.
Critical strengths
- AnimeText surpasses most existing manga and scene-text datasets considered in this study in both size and diversity.
- The study examines different training-set scales and detection architectures, with detailed evaluations across diverse text attributes.
Critical weaknesses
- Insufficient dataset transparency and quality assurance. This paper lacks essential annotation details, including annotator qualifications, agreement measures, quality audits. These issues weaken confidence in the dataset’s usability and reliability.
- Confounded comparisons and unclear data separation. Evaluating models on AnimeText favors training data that matches its domain, text categories, and annotation format, making it difficult to isolate the dataset’s actual advantage.
Questions for authors
1.What are the origins and licensing terms of the 735,060 images, and are the authors authorized to publicly distribute them? 2. How are the 985 duplicated images distributed among the training, validation, and test sets, and were they excluded from every reported experiment, particularly those in Tables 5, 6, 16, 19, and 20? 3. Please also explain the approach used to identify exact duplicates and visually similar copies. 4. Could the authors conduct an independent assessment of the quality of each annotation type? 5. Please report the human-validated accuracy of the labels generated by Qwen and provide the specific model version, classification prompt, and inference parameters. 6. Could the authors further disentangle the effects of domain diversity, annotation format, and dataset scale through size-matched training, standardized bounding-box definitions, and testing on an external anime or comic dataset?
We thank the reviewer for the detailed questions.
Q1. Origins and licensing. The images come from Danbooru, and we have obtained permission for their use. The dataset will be released on Hugging Face.
Q2. The 985 images. They are distinct training images with an exact duplicate in another split, forming 1,065 cross-split groups: 643 training–validation, 333 training–test, 87 validation–test, and 2 shared by all three. The matched-budget and scaling experiments train on the 513,159-image pool, which excludes all 985 and shares no exact duplicate with the test split. On the test split, removing all 4,572 duplicate-linked images leaves the matched-budget advantage at 19.3 points or more in the worst case.
Tables 5, 16, 19, and 20 use the validation split (protocols in Table 6). Re-evaluating our full-data YOLO11m checkpoint on all 147,191 validation images:
F1 changes by at most 0.06 points.
Q3. Duplicate identification. Exact duplicates: SHA256 of the original image bytes. Near duplicates: 64-bit DCT perceptual hash, Hamming distance ≤ 4, aspect-ratio factor ≤ 1.1, comparing every test image with every training and validation image. Two human reviewers confirmed 97.6% (164/168) and 99.5% (199/200) of sampled candidate pairs as genuine duplicates and agreed on 98.2% of shared pairs.
Q4. Assessment of each annotation type. Five reviewers took part, two of them external to dataset construction.
The two external reviewers agree on 88.7% of candidate decisions.
Q5. Qwen.
Qwen/Qwen3.8-27B(snapshot1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0), temperature 0, 1,536-token limit, thinking disabled, strict JSON schema. Each request supplied the full image with numbered boxes and matching crops for up to six regions. The sample holds all 1,423 fine regions of 256 random test images. The prompt defines 10 function, 5 lettering, and 5 layout classes and will appear verbatim in the appendix. An external reviewer labeled all 1,423 regions:Under human labels, non-dialogue text is 61.5% of the 581 gained regions (95% CI 55.6–67.4%), versus 63.0% under Qwen labels.
Q6. Domain, format, and scale.