acceptodds
Under review as a conference paper at ICLR 2027

A New Benchmark Dataset and an Imprecise Supervised Learning Approach for Airport Semantic Segmentation

Abstract

Airport semantic segmentation is fundamental to airport surveillance, yet there currently lacks a specialized benchmark and algorithms for this task. In this paper, we introduce ASS, the first large-scale Airport Semantic Segmentation dataset tailored for airport scenarios. ASS comprises 18 common semantic categories, 250 videos, and over 140,000 frames with precise manual annotations. The dataset covers a wide spectrum of challenges inherent to airport surveillance, including intra-class diversity, unusual shapes, inter-class similarity, extreme multi-scale, class imbalance, and varying weather and illumination conditions. We evaluate 20 state-of-the-art semantic segmentation methods on ASS and observe substantial performance drops, indicating that existing models remain far from practical deployment. The dataset is available at www.agvs-caac.com/ASS/ASS.html. Furthermore, we discuss future directions for airport semantic segmentation and propose a research approach based on imprecise supervised learning, with an initial implementation named SPWSeg (https://github.com/SPWSeg/SPWSeg). Additional experiments show that this research approach generalizes effectively beyond airport scenes.

Primary area:
datasets and benchmarks
Keywords:
airport surveillance, semantic segmentation, dataset, benchmark.

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 4 reviews, above 0% of the others.
Reviewer_DUntOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ASS, a large-scale dataset for airport semantic segmentation comprising 250 video sequences and over 140,000 pixel-annotated frames across 18 semantic categories. The authors document various challenges specific to airport environments—extreme multi-scale objects, non-compact shapes (e.g., aircraft wings), inter-class similarity, and varying weather/lighting conditions—and benchmark 20 existing segmentation methods, observing substantial performance degradation. Additionally, they propose SPWSeg, a weakly supervised approach that leverages 3D shape models to generate synthetic training samples with image-level labels, using these to regularize segmentation predictions. The method is evaluated on ASS and Cityscapes with reported improvements over IRNet.

Critical strengths

1.Airport surveillance is a practically important yet underexplored setting for semantic segmentation. The collection effort represents a substantial investment. If accompanied by clear evaluation protocols and public release, this dataset could enable research on segmentation in long-range surveillance contexts that differ markedly from standard street-scene benchmarks.

  1. The paper correctly identifies that standard segmentation methods struggle with non-compact object geometries that are prevalent in airport scenes. The proposed direction of incorporating shape priors addresses a real limitation of purely appearance-based learning, particularly when annotations are weak or incomplete.
Critical weaknesses

1.The paper conflates two contributions—synthetic training samples and the shape-prior regularization—without ablation to separate their effects.

2.Critical details are missing. The train/val/test split construction is undescribed—whether frames from the same video can appear in different sets is unclear. The dataset claims 18 categories, yet Table 1 reports only 12. Annotation quality metrics (number of annotators, inter-annotator agreement) are absent, and challenges like "extreme multi-scale" are illustrated qualitatively but never quantified.

Questions for authors

1.The paper lists multiple challenges (scale variation, shape, similarity, weather) but does not establish their relative importance. Can you provide a controlled study to quantify which factors contribute most to performance degradation?

2.For SPWSeg, the shape constraint is applied via a loss term that encourages predicted masks to match shape prototypes. How is this constraint computed when a class appears multiple times in an image (e.g., several aircraft), when objects are partially occluded, or when only truncated views are available?

Reviewer_3qxSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces ASS, a semantic segmentation dataset for airport surveillance. The dataset contains 250 videos, more than 140K annotated frames, and 18 semantic categories. The authors evaluate several image-based and video-based semantic segmentation methods on ASS and discuss the challenges of airport scenes. The paper also proposes SPWSeg, an imprecise supervised segmentation method that uses 3D shape priors, and evaluates it on ASS and Cityscapes.

Critical strengths
  1. Airport surveillance is an important real-world setting. Studying semantic segmentation in this domain can be useful, especially because airport scenes contain objects and visual conditions that are different from common street-scene benchmarks.

  2. The paper includes several qualitative examples of airport-specific difficulties. The examples are helpful for understanding the motivation of the dataset. They illustrate issues such as unusual object shapes, large scale variation, inter-class similarity, class imbalance, and changing weather or illumination conditions.

Critical weaknesses
  1. The validation of SPWSeg is too weak for a main methodological contribution. SPWSeg is presented as one of the main contributions of the paper, but its experimental evaluation is limited. The comparison uses only a small number of methods and does not include enough strong and recent alternatives. There are also not enough ablation studies to clearly show which part of SPWSeg is responsible for the improvement. In addition, SPWSeg uses 3D shape priors as extra information. It is therefore difficult to tell whether the improvement mainly comes from the proposed learning method or from the additional prior. Since SPWSeg is an important part of the paper, I think the current experiments are not sufficient to support its effectiveness.

  2. The experiments used to demonstrate that existing methods fail on airport scenes is not convincing. The paper uses the lower performance of existing segmentation methods on ASS compared with their performance on other datasets to illustrate that airport scenes are more challenging. However, results from different datasets are not directly comparable. Different datasets have different categories, data distributions, image resolutions, training sizes, and evaluation settings. Therefore, a lower mIoU on ASS alone does not clearly show that the performance drop is caused by the inherent difficulty of airport scenes. A more controlled analysis is needed to support this claim.

Questions for authors

No

Reviewer_rWsaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces ASS, an airport semantic segmentation dataset reportedly containing 250 video sequences, over 140,000 annotated frames, and 18 semantic categories. The authors evaluate some existing segmentation methods on the dataset and propose SPWSeg, a weakly supervised segmentation method that incorporates shape priors derived from 3D models. While airport scene understanding is a potentially useful research direction, the paper does not sufficiently establish the novelty of the dataset or the technical contribution of the proposed method.

Critical strengths

This paper introduces ASS, a large-scale airport semantic segmentation dataset, effectively filling a critical domain gap in airport surveillance research.

Critical weaknesses
  1. The relationship between ASS and the previously submitted AVSS[1] airport segmentation benchmark (ICLR 2025) is not discussed. Given their apparent similarities in dataset scale and category coverage, the authors must clearly establish what is novel in the current submission.

  2. Although the paper claims 18 categories, the main benchmark table reports only 12, without adequate explanation. Some category definitions are ambiguous, and the dataset construction, annotation quality control, and video-level data splitting protocols are insufficiently documented.

  3. The evaluation relies largely on older segmentation architectures and lacks comparisons with recent state-of-the-art models and segmentation foundation models, such as SAM 3[2]. Evaluating suitable modern models under a clearly defined protocol would provide a more meaningful assessment of the benchmark's difficulty and relevance to current segmentation research.

  4. Shown in Table. 2, the cross-dataset mIoU comparisons are not directly comparable because the datasets differ in label spaces and evaluation settings. These results do not convincingly establish that airport scenes are inherently more challenging.

  5. The manuscript also suffers from serious writing and organizational problems. Most notably, the introduction contains no citations, while excessive parenthetical explanations, inconsistent punctuation, grammatical errors, and unclear section organization substantially impair readability.

[1] https://openreview.net/forum?id=3nkIRKh3Sk

[2] https://arxiv.org/abs/2511.16719

Questions for authors

Same as Weaknesses.

Reviewer_EqbZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  • The paper introduces ASS, a semantic segmentation dataset collected from airport surveillance footage, and SPWSeg, a weakly supervised method that incorporates shape priors.
  • ASS contains 250 clips of approximately 15 seconds each, comprising more than 140k frames from five airports. Footage was captured using four fixed cameras and one PTZ camera, covering clear, rainy, and foggy conditions across daytime, twilight, and night. The dataset has 18 classes. Annotations were created as polygons in AnyLabeling, using labels from the preceding frame as a starting point.
  • The benchmark evaluates 15 image and five video segmentation models under a shared fine-tuning schedule of 20k iterations at 1280×960 resolution. The authors also compare their results with published scores on five public datasets, arguing that airport segmentation is substantially harder.
  • SPWSeg extends IRNet with a shape-prior loss. A shape autoencoder embeds regions extracted from class activation maps, and these embeddings are encouraged to match K-means prototypes derived from silhouettes rendered from 3D models. The reported mIoU improves from 11.33 to 17.62 on ASS and from 14.22 to 22.06 on Cityscapes.
Critical strengths

The dataset addresses a real and relatively underexplored application. Collecting and densely annotating surveillance video from five operational airports is a substantial undertaking. The variation in weather and lighting could make this a useful resource, provided the release conditions and evaluation protocol are clearly documented. The annotation guidelines, particularly the emphasis on recognizability and boundary priorities, are sensible.

Critical weaknesses
  • I could not reconcile the benchmark results or establish the evaluation protocol. The ASS scores in Table 2 do not agree with the per-class results in Table 1. Averaging the 12 reported class IoUs gives approximately: HRNet: 40.9, compared with 20.27 in Table 2, BiSeNetV2: 28.6, compared with 15.52, FCN: 42.1, compared with 37.62. More fundamentally, I could not find a defined train/validation/test split. With more than 140k frames drawn from approximately 250 short clips, nearby frames will be highly redundant. A frame-level split would risk substantial leakage, so the paper needs to specify whether clips, cameras, or airports are kept disjoint. No cross-camera or cross-airport evaluation is reported.
  • Annotation quality is also insufficiently documented. It is unclear how many frames were annotated manually, how many were initialized or propagated from previous frames, and how those annotations were checked. There is no agreement measurement from an independently annotated subset.
  • Several table entries warrant checking. DeepLabV3 reports exactly 43.09 for both airplane and high-pole light; CFFM and CFFM++ both report 54.44 for security post; and DVIS++ scores 0.00 on nine of the 12 reported classes. These values do not by themselves establish an error, but the paper should clarify whether they reflect valid runs, training failures, or transcription issues.
  • IRNet (2019) is the only comparison. More recent weakly supervised segmentation methods exist.
Questions for authors
  • How is ASS mIoU computed in Table 2? Please specify the classes, split, and runs used, and reconcile these scores with Table 1.
  • What are the official train/validation/test splits? Are they disjoint by clip, camera, or airport? Which split is used for Table 3, and how does it relate to the validation set used for the Table 5 search?
  • How many frames were manually annotated versus initialized or propagated from previous frames? Can you provide annotation-agreement results on a doubly annotated subset?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.