- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces WildBioWiki, a species-centered benchmark for wildlife question answering. It links 264,563 images of 392 vertebrate species to 3,136 unique species-level question–answer pairs, supporting evidence, and image-specific captions. The benchmark enables controlled comparisons that vary access to images, species names, and knowledge while keeping questions and reference answers fixed. The authors evaluate twelve zero-shot models and conduct targeted analyses using balanced claims, matched facts, and human review.
Critical strengths
The dataset comprises 392 vertebrate species, 264,563 images, and 3,136 distinct species-level question–answer pairs covering species identity, habitat, diet, distribution, and behavior. The reference answers retain supporting text and provenance information, facilitating verification of their evidential basis. This resource provides a foundation for studying question answering that integrates visual recognition with biological knowledge.
Critical weaknesses
(1) The manuscript contains formatting and cross-referencing issues, including unresolved “Appendix ??” references in the Related Work and Reproducibility Statement sections.
(2) I recommend redrawing Figure 1, as many text elements extend beyond the figure boundaries.
(3) Parts of the manuscript’s writing and figure preparation rely on AI assistance. The writing lacks clarity, and the manuscript’s overall structure is poorly organized.
Questions for authors
(1) The paper should provide a more detailed discussion of existing related datasets. A comparison table should also be included to clarify the advantages of the proposed dataset.
(2) Was the entire dataset verified by domain experts? Were the reference answers cross-checked by multiple reviewers? If manual verification was conducted, what was the level of inter-rater agreement?
(3) The dataset primarily covers common vertebrate species with well-documented biological knowledge. Do the reported findings also hold for rare species and species that are easily confused? Additional generalization experiments on long-tail species and complex field scenes would help establish the scope of the conclusions.
(4) The dataset is the paper’s main contribution, yet only a small subset is provided in the appendix. I recommend that the authors make the complete dataset publicly available.