- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AFTD (Airport Foreground Target Detection), a dataset of 10,050 images for movable target detection in airports, covering airplanes, vehicles, and persons. The images come from cameras at five airports and online sources. The dataset is designed to support research and benchmarking under super multiscale, occlusion change, and viewangle change conditions. It also provides more detailed annotations for airplanes, including scale level, occlusion degree, and approximate viewangle. The authors evaluate 22 detectors and interpret their lower AP on AFTD compared with COCO val2017 as evidence that foundational algorithms lack robustness in real-world scenes. Then they proposes a scene-specific prior based approach, implemented through NSAD with nonparametric scale modeling, dynamic anchor initialization, and scale-normalized regression. The authors report performance gains over D-FINE on both AFTD and COCO val2017, with scale priors refitted for COCO. Based on these results, they conclude that the approach is effective and has cross-scenario generalizability.
Critical strengths
-
The paper collects real-world data from an application where data access is difficult and provides detailed annotations that require substantial effort. The authors collected data at five airports using fixed and PTZ cameras over about twenty months. They also included images from online sources to cover changes in scale, occlusion, viewangle, weather, and illumination. The airplane attribute labels go beyond standard bounding boxes. In particular, occlusion degree is estimated by aligning a 3D airplane model with each instance. These annotations help distinguish AFTD from other datasets, add practical value, and reflect the substantial annotation effort. Given the barriers to data collection at airports, the data and its detailed annotations are the paper’s most important contribution.
-
The paper evaluates a broad range of baselines and provides public links to the data and code. The authors evaluate 22 detectors on AFTD and public datasets, covering YOLO, DETR, two-stage, and anchor-free methods. These results help show how different detection architectures perform in airport scenes. Together with the publicly available data and code, they provide a useful reference for future research on airport surveillance.
Critical weaknesses
-
The benchmark protocol is incomplete, and the key attribute annotations and challenge subsets lack quantitative evaluation. The paper does not clearly state the sizes of the AFTD training, validation, and test sets or how they are split. It is unclear whether any airports are held out for testing or how near-duplicate images from the same camera or nearby time points are handled. AFTD provides airplane annotations for scale level, occlusion degree, and approximate viewangle. It also highlights six challenges: super multiscale, occlusion change, viewangle change, illumination change, weather change, and complex background. However, the paper lacks quantitative results grouped by these attributes and challenges. The criteria and sizes of subsets for weather, illumination, and other conditions are also unclear. The authors mainly interpret the lower AP on AFTD compared with COCO val2017 as evidence that methods lack robustness for real-world scenarios, without further examining performance or differences between methods under each challenge. Although the dataset is the paper’s main contribution, the current experiments mainly show its overall difficulty. They do not fully demonstrate the value of its detailed annotations and challenge design for robustness research.
-
NSAD lacks quantitative results from ablations and a clear baseline comparison. The paper does not report complete quantitative ablation results for dynamic anchor initialization and scale-normalized regression. It also does not clearly identify the detector and model version used as the basis for NSAD or explain how its configuration differs from D-FINE. This makes it difficult to assess the actual gains from each design. The gains on COCO after refitting the scale distribution of all classes support the approach’s use across datasets. However, the claim of cross-scenario generalizability needs to be stated more clearly. The experiments show that the approach remains effective after its priors are re-estimated on a new dataset. They do not establish that a trained model can generalize to unseen scenes.
Questions for authors
- A more complete description of the data split would help readers understand the benchmark results. Please report the sizes of the AFTD training, validation, and test sets and explain how they are split. This should include whether any airports are held out for testing and how near-duplicate images from the same camera or nearby time points are handled.
- The detailed annotations are an important contribution of the dataset. Reporting quantitative results by scale level, occlusion degree, and approximate viewangle would therefore be valuable. Please also clarify the definitions, overlap, and sample counts of the six subsets, along with whether their labels are publicly available. This would help demonstrate the dataset’s value for robustness research and make the link between the conclusions and experimental results clearer.
- The appendix already describes the hyperparameter search and selection process in detail. It would help to identify the detector and model version used as the basis for NSAD and explain how its configuration relates to D-FINE. Reporting separate quantitative ablation results for dynamic anchor initialization and scale-normalized regression would also help clarify the gains from each design.
- The meaning of approximate viewangle needs further clarification. Under the current definition, it describes the orientation of the airplane’s axis in the image plane. At low depression angles, projection effects may cause airplanes viewed from an oblique angle to be labeled as Left or Right. Discussing this limitation and distinguishing the 2D orientation distribution in Figure 9b from 3D viewing-angle coverage would make the meaning of the annotations clearer.
- AI use statement was not found in the provided main text or appendix. Please check the submission requirements and add a statement on whether generative AI tools were used and for what purposes, as required. Please also make the in-text citations and reference list consistent with the conference format.