- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper focuses on the task of semantic segmentation under visual degradation and proposes DevoSeg, a self-evolving framework that learns how to deploy a frozen segmentor from completed segmentations. Its core mechanism comprises three key aspects: (1) estimating the complementarity of the strategy set and allocating an adaptive execution budget for each image; (2) utilizing post-execution image-mask evidence to contrast the masks generated by the selected strategies against independently generated direct-inference masks; and (3) compiling the observed effects and costs into contextual knowledge, and proposing a disjoint-group verification rule for decision state updates. Experimental results demonstrate that the mask selector of DevoSeg improves the mIoU on the Cityscapes dataset by 2.084 percentage points.
Critical strengths
The mathematical formulation in the methodology section is academical.
Critical weaknesses
-
To ensure a fair comparison in Table 4, SAM 3 should be fine-tuned or adapted using the identical set of 920 annotations to compare its performance against DevoSeg.
-
The paper is motivated by local, category-dependent failure issues, stating that "an image may preserve the overall layout but lose the boundaries of thin or partially occluded objects." However, the evaluation metric adopted in this paper is Image IoU, which is dominated by large-object categories and is therefore largely insensitive to the fine structures and boundary effects the paper claims to address. Tables 1, 2, and 4 should provide per-category (class-wise) results to evaluate whether the empirical evidence supports the objectives argued in the introduction.
-
The dataset usage in the experimental evaluation is non-standard. While the ACDC validation set contains 406 images and the test set contains 2,000 images, the experimental section only evaluates on 200 ACDC images, and the paper lacks any explanation for this discrepancy.
-
The literature review is insufficient. There is currently a substantial body of literature in the field of semantic segmentation under adverse weather conditions, such as [1-5]. Both the related work section and the comparative experiments in Table 4 lack descriptions of and comparisons with these relevant studies.
[1] Brüggemann D, Sakaridis C, Truong P, et al. Refign: Align and refine for adaptation of semantic segmentation to adverse conditions[C]//2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2023: 3173-3183.
[2] Xu Q, Wu Q, Hongtao L, et al. CroPe: Cross-Modal Semantic Compensation Adaptation for All Adverse Scene Understanding[J]. Advances in Neural Information Processing Systems, 2026, 38: 136226-136252.
[3] Lee S, Kim N, Kim S, et al. Frest: Feature restoration for semantic segmentation under multiple adverse conditions[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2024: 1-18.
[4] Li F, Gong Z, Deng Y, et al. Parsing all adverse scenes: Severity-aware semantic segmentation with mask-enhanced cross-domain consistency[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(12): 13483-13491.
[5] Pan Y, Sun R, Li W, et al. Exploring weather-aware aggregation and adaptation for semantic segmentation under adverse conditions[C]//2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025: 13952-13962.
Questions for authors
Q1: What are the experimental results in Table 4 when SAM 3 is fine-tuned or adapted using the same set of 920 annotations?
Q2: What are the per-category (class-wise) results in Tables 1, 2, and 4?
Q3: What is the rationale behind selecting only 200 ACDC images for the evaluation?
Q4: What are the advantages of the proposed method compared to existing literature on semantic segmentation under adverse weather conditions, and how does it compare in terms of performance?
9 comments
2. Class-wise metrics and fine-structure evaluation
2. Class-wise results for Tables 1, 2, and 4 and evaluation of fine structures
Supplementary Table 1 below provides the complete 19-class region IoU and Boundary IoU results for the candidate-input ablations. Supplementary Table 2 provides the corresponding decision-component results, and Supplementary Table 3 provides the results for the other segmentation methods in Table 4 and the appendix. We refer to the paper’s Image IoU as “mean per-image IoU” to make its image-level averaging explicit. It first averages classes with nonempty union within each image and then averages across images:
Here, contains the classes whose prediction–ground-truth union is nonempty in image , and , , and are the true-positive, false-positive, and false-negative pixel counts for class in that image. Each valid class within an image receives equal weight, regardless of its pixel area. To address your concern about fine structures and boundaries, we additionally report Boundary IoU and provide the complete 19-class results for Tables 1, 2, and 4, evaluating region recognition and contour quality separately.
Dataset-level mIoU is the metric called Pooled mIoU in the paper. For each class, we sum its intersection and union pixel counts across all images and take their ratio, then average the resulting class IoUs with equal class weights. Thus, mean per-image IoU averages scores calculated separately for each image, whereas dataset-level mIoU calculates class scores from counts pooled across the dataset. The revised name makes this distinction explicit; the calculation and reported values are unchanged.
For the full-input Comparator in Table 1, the two boundary metrics are:
| Boundary metric | Direct | Full-input Comparator |
|---|---|---|
| Mean per-image Boundary IoU (%) | 52.530 | 53.480 |
| Dataset-level Boundary IoU (%) | 59.372 | 61.807 |
Mean per-image Boundary IoU improves by 0.950 percentage points, with a 99.5% paired interval of [−0.021, 1.960] adjusted for ten comparisons. Results for thin structures and road users are shown below; Supplementary Tables 1 and 2 provide the complete 19-class results for the input ablations and decision-component comparisons, respectively.
| Class | Direct Region IoU | Full-input Comparator Region IoU | Direct Boundary IoU | Full-input Comparator Boundary IoU |
|---|---|---|---|---|
| Pole | 59.52 | 60.15 | 61.40 | 63.39 |
| Rider | 19.25 | 47.35 | 36.53 | 56.91 |
| Car | 82.22 | 89.41 | 69.69 | 82.92 |
For example, pole Boundary IoU increases from 61.40% to 63.39%; for riders, region IoU and Boundary IoU increase from 19.25% and 36.53% to 47.35% and 56.91%, respectively. Joint decisions in Table 2 also increase car and bus region IoU from 82.22% and 62.66% to 86.20% and 66.34%. The pole and rider results above correspond to the full-input Comparator in Supplementary Table 1. The car and bus results for joint decisions are provided in Supplementary Table 2.
Boundary IoU uses an inner boundary band with a width of 2% of the image diagonal and excludes neighborhoods of invalid labels. The class-support table accompanying Supplementary Table 3 gives the ground-truth pixel count and number of images containing each class, relating each class score to its actual support.
3. Evaluation cohort and complete validation results
3. The 200-image component evaluation and the complete validation results
The component comparisons require multiple candidates and multi-step operations for each image, making them computationally demanding. To complete systematic comparisons within the available computational resources, we use 200 ACDC images covering fog, night, rain, and snow, with 50 images per condition. This evaluation cohort was established using early model observations. We also compare Direct and DevoSeg on all 406 images in the official ACDC validation set, with the following results.
| Metric | Direct | DevoSeg | Gain (pp) | 97.5% scene-paired interval |
|---|---|---|---|---|
| Mean per-image IoU (%) | 53.448 | 54.812 | +1.363 | [0.734, 2.370] |
| Dataset-level mIoU (%) | 63.870 | 66.008 | +2.138 | [0.622, 4.683] |
On the complete validation set, DevoSeg outperforms Direct on both metrics, with both confidence intervals for the gains above zero. These results support the overall segmentation benefit of the workflow on ACDC. The evaluation uses ACDC images with the Cityscapes 19-class semantic taxonomy throughout.
1. Segmentor adaptation with the same annotations
Thank you for recognizing the mathematical formulation of the method and for your specific suggestions on comparisons with the same annotations, class-wise metrics, and adverse-weather segmentation research. To make the adaptation and class-wise evaluations reproducible, we will release the implementation, same-annotation adaptation configurations, class-wise and boundary evaluation scripts, data lists, and model sources upon acceptance.
1. Results after adapting SAM 3 with the same 920 annotations
We adapt SAM 3 using the same 920 annotations used to learn the decision modules, then fix that checkpoint and run Direct and DevoSeg separately. Adaptation keeps the image and text backbones frozen and updates the remaining parameters. This compares the use of the same annotations for different learning targets and tests whether external operation selection and candidate comparison can still improve an adapted segmentor.
Adapting SAM 3 with the same annotations substantially improves the segmentor itself. With this stronger segmentor, DevoSeg further increases dataset-level mIoU from 75.180% to 77.413%, a gain of 2.233 percentage points, with a 95% scene-paired confidence interval of [0.188, 4.654]. Mean per-image IoU increases from 67.780% to 67.865%, with a difference interval of [−0.137, 0.295]. These results show how segmentor adaptation and external decision learning can work together: the former improves the underlying predictions, while the latter learns how to make use of candidates produced by the operations.
Using annotations for segmentor adaptation and for decision learning are therefore complementary directions. Reviewer bVbt describes this positioning as “policy learning as a complement to adapting segmentor weights”. The same-annotation comparison above provides a concrete result: the external workflow further improves dataset-level mIoU after the segmentor has been adapted.