- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work proposes a new benchmark for MAT. This benchmark is featured by the large scale of video and frame numbers, broad animal category coverage, extensive animal instance annotations, and high quality annotations. They build seen-category and unseen-category protocols for evaluation. They propose CDA to address the low-overlap association challenge, which combines IoU with center similarity normalized by the box scales. Multiple MOT methods are evaluated on the benchmark.
Critical strengths
- This is a large and diverse dataset for MAT, especially on the number of bounding boxes and identity trajectories. Although SAM3 is used for initial labeling, expert review and correction are conducted, and quality is assessed through the independent audit. This improves the annotation quality.
- They establish seen and unseeen category protocols, and evaluate multiple MOT methods under both protocols. These comprehensive experiments are good baselines for following research.
Critical weaknesses
The tracking performance degeneration on unseen-category protocol needs furthey analysis. It's unclear whether the main bottleneck is the detection or the association stage. One more experiment where Ground Truth detections are provided should be conducted. Then we can check the association performance under such situation and get a more clear understanding of the tracking performance degeneration.
Questions for authors
Please check the weakness part.
2 comments
Post Rebuttal Comments
The author's rebuttal has addressed my concerns, and I will maintain the positive rating.
**Response to Reviewer ceZh**
Thank you for suggesting the GT-detection control.
Detection versus association. We evaluated TrackTrack and DiffMOT with normal detections and GT boxes under both protocols. Within each pair, the tracker, checkpoint, and association settings are fixed. GT boxes use one foreground class and confidence 1, without GT identities; appearance features are recomputed from the corresponding crops.
Normal → GT detections; scores in %.
The P1–P2 HOTA gap shrinks from 12.49 to −0.43 points for TrackTrack and 13.52 to 1.38 points for DiffMOT. Their AssA gaps also shrink from 8.29/9.35 to −0.57/0.22 points. This identifies detection quality as the dominant contributor to the observed protocol gap for these trackers.
The table reports paired diagnostic runs. DiffMOT's normal P2 score is 52.852 for the checkpoint used in this control; its original Table 5 result remains 52.90. We will include the control in the revised manuscript.