- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper introduces Animal World, a "scene-centric" cross-species 2D animal pose dataset with 10,036 images and 19,132 instances, reportedly covering 124 species under a unified 30-keypoint taxonomy, with SAM3-derived and manually verified masks and boxes, and Qwen3-VL-8B-generated behaviour descriptions and image-level group-interaction labels. Images are sparsely sampled frames from more than 500 YouTube wildlife videos. Four human-pose top-down backbones (HRNet-W48, HRFormer-B, ViTPose-B, RTMPose) are benchmarked in MMPose with ground-truth boxes, followed by diagnostic analyses by occlusion level, social scenes, juveniles, rare species and instance count, plus cross-family, cross-dataset (APTv2), long-tail and pHash analyses in the appendix. Keypoints from a ViTPose trained on Animal World are then used as prompts for PRIMA post-optimisation and yield a small 3D reconstruction gain.
Critical strengths
1. The dataset addresses a genuine gap. Existing animal pose benchmarks such as AP-10K, APT-36K and Animal Kingdom contain few multi-animal or cross-species scenes and offer limited coverage of juveniles and endangered species. The proposed dataset targets these cases directly, and its 30-keypoint annotation is denser than the 17-keypoint schemes in common use.
2. The experimental analysis is organised around the properties the dataset is meant to capture. Rather than reporting a single aggregate score, the paper breaks results down by occlusion level, social context, age group and species rarity, and examines transfer to and from an existing dataset. This structure is appropriate for a dataset paper, although, as discussed below, the interpretation of several of these analyses depends on the annotation protocol and on the ground-truth-box evaluation setting.
3. The paper examines downstream utility for 3D reconstruction. The post-optimisation experiment with PRIMA (Table 8) is a reasonable attempt to show that the 2D annotations have value beyond 2D pose estimation, which is not commonly done in dataset papers of this kind.
Critical weaknesses
1. The central claim of the dataset concerns multi-animal and social-interaction scenes, but the evaluation protocol does not test this setting. According to Sec. 4, all baselines are evaluated conditioned on ground-truth bounding boxes, which the Limitations section also acknowledges. The four baselines are top-down methods that operate on a single cropped instance at a time. Under this protocol, the aspects that distinguish multi-animal scenes from single-animal ones, namely detecting all individuals, associating keypoints with the correct individual, and resolving instances whose boxes overlap substantially, are not evaluated. CrowdPose, which the paper cites as the human counterpart for crowded scenes, evaluates on full images without ground-truth boxes, and adopting an analogous protocol here would be natural. Pose estimators that operate on full images, as well as multi-animal tools such as multi-animal DeepLabCut and SLEAP, are available for this purpose. Given that the dataset already provides boxes and masks, a full-image evaluation appears feasible, and it would be needed to support the paper's claims regarding social and multi-animal scenes.
2. The annotation protocol is not described, which makes the reliability of the ground truth difficult to assess, in particular for the virtual surface points and for occluded keypoints. The annotation process is described only as "manually annotated by experts". Three aspects require clarification. (a) Several of the 30 keypoints (anterior, middle and posterior back, abdomen midpoint, tail midpoint, left and right upper forelimb) are not anatomical joints but reference points on a smooth body surface. Their placement is inherently subjective even when fully visible, and no inter-annotator agreement is reported. (b) The treatment of occluded keypoints is not specified: it is unclear whether annotators inferred the positions of invisible joints and whether such inferred coordinates are included in the AP computation. The occlusion analysis in Sec. 4.2 depends on this choice. If inferred coordinates are scored, label noise increases with the degree of occlusion, and the finding that models trained on Animal World perform better at low visibility may partly reflect agreement with the annotators' inferences; if they are not scored, the analysis measures robustness only on the remaining visible keypoints. (c) COCO-style AP depends on per-keypoint OKS sigmas, which are not reported for the 30 keypoints. Stating them would make the reported AP values easier to interpret and to compare with other datasets.
3. The benchmark includes only four generic human-pose backbones; no method developed for animal pose estimation, cross-species transfer or multi-animal scenes is evaluated. HRNet, HRFormer, ViTPose and RTMPose were designed for human pose estimation or generic dense prediction and are trained from scratch on Animal World. The animal pose literature offers natural candidates, such as the cross-species pretrained SuperAnimal models, the prompt-based CLAMP, the open-vocabulary keypoint detector X-Pose, and multi-animal tools such as DeepLabCut and SLEAP, but none of them is included. In addition, all diagnostic analyses are conducted with a single model (ViTPose-B). As a result, the paper does not show how existing animal pose methods perform on this data, and the evidence that the dataset poses challenges beyond those of existing benchmarks rests on comparisons among the authors' own models on their own split. Including animal-specific baselines would strengthen this argument considerably.
Questions for authors
- Please clarify how occluded keypoints were annotated. When a joint is not visible, were annotators asked to infer its position, and if so, what quality-control procedures (annotation guidelines, redundant annotation, inter-annotator agreement) were in place for these inferred labels? Are inferred coordinates included in the AP computation?
- Which OKS sigma values were used for the 30 keypoints, and how were they determined? Could the authors report inter-annotator agreement, particularly for the virtual surface points?
- The central claim of the dataset concerns multi-animal and social-interaction scenes, yet all reported results are conditioned on ground-truth bounding boxes, so no model is ever required to localise the animals or to associate joints with the correct individual. Could the authors provide an evaluation on full images without ground-truth boxes, so that the multi-animal setting the dataset is designed for is assessed directly?
- Could the authors include methods developed specifically for animal pose estimation in the benchmark, in addition to the human-pose backbones currently evaluated?