- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a detector-guided generative augmentation framework for UAV detection under adverse conditions. The method first generates different environmental conditions while preserving the original UAV through a two-stage generation process. It then measures feature changes using a frozen detector to estimate similarities between conditions. Based on these similarities, a greedy selection algorithm identifies a subset of conditions that balances coverage and redundancy. The experiments demonstrate that selecting a subset of training set can improve the detection performance compared to using all datasets.
Critical strengths
-
This paper addresses a practical problem. Generating more diverse conditions does not always improve detection performance. The proposed method seems simple and easy to understand and implement.
-
The authors test their method on two different detectors, RT-DETRv2-S and YOLO11m. This helps show whether the improvements generalize across different detection models.
Critical weaknesses
-
Insufficient validation of the proposed condition selection method. The paper shows that selecting a subset of conditions can improve detection performance compared to using all conditions. However, the relationship between condition similarity and detection performance is not sufficiently analyzed. It is unclear how performance changes as more conditions are added, and whether any improvements come from having more training data or greater diversity in augmentation conditions. The proposed method is also not directly compared with simpler alternatives such as random condition selection. Since the experiments only consider five predefined conditions, it is difficult to assess how well the approach generalizes to more diverse augmentation conditions.
-
Limited consideration of image-dependent effects and experimental reliability. The proposed method considers augmentation conditions as the main factor for selection, but the effectiveness of augmentation may depend on both the condition and the source image. It is unclear whether selecting conditions globally is appropriate when the same condition may have different effects across images. Since the method averages similarities across source images and removes selected conditions from the entire dataset, such image-dependent differences are not considered or analyzed. In addition, the performance improvements are relatively small, with noticeable variation across training seeds. The experiments also use a relatively small dataset, with only 102 source images for measuring condition similarity and 247 images for external evaluation. This makes it difficult to assess the reliability of the reported improvements.
Questions for authors
-
How does the proposed method compare with random condition selection? Is there a clear relationship between condition similarity and downstream performance gains?
-
How does the method distinguish improvements from increased training data and those from greater condition diversity?
-
How much do condition similarities vary across source images, and how might this affect the proposed global condition selection?
-
How reliable are the reported improvements given the relatively small dataset?
3 comments
Additional Results and Manuscript Revisions (2/2)
Q3. Image dependence and global selection
Sections 3 and 4.2 specify the dataset-level selection objective and paired measurement: each condition is evaluated on the same training sources, similarities are computed within each source from generated-minus-clean responses, and these similarities are then averaged.
We now directly quantify image dependence for YOLO11m in Appendix A.2 and Figure 7. Across the 102 sources, the sample standard deviations of pairwise similarities range from 0.063 to 0.386; dust–low-light similarity is positive for all 102 sources. We further draw 10,000 bootstrap resamples of the sources, retaining all five variants of each sampled source together. For each resample, we recompute the response matrix and rerun selection at the fixed reported values.
Table 2 and Section 5.4 report recovery rates of 99.29% for RSFD, 100% for RSD, and 71.28% for RD. In the remaining RD resamples, snow is retained alongside rain and dust. These results quantify both the variation across images and its effect on the global decision, with particularly stable recovery of the three-condition RSD set.
Q4. Reliability with the available dataset
We strengthened the evaluation with ten additional randomly generated training seeds, giving 13 seeds including the original three. Section 5.2, Table 3, and Appendix B.3 report paired external AP differences, individual 95% t confidence intervals, and two-sided paired t-tests with Holm correction across the three comparisons:
| Comparison | Mean gain | 95% CI | Holm-adjusted | Positive runs |
|---|---|---|---|---|
| RT-DETRv2-S: SDL vs Full | 1.293 | [0.474, 2.111] | 0.0118 | 12/13 |
| RT-DETRv2-S: SDL vs matched Full | 1.454 | [0.564, 2.344] | 0.0118 | 13/13 |
| YOLO11m: RSD vs Full | 0.742 | [0.095, 1.390] | 0.0280 | 11/13 |
The gains are positive in 12/13, 13/13, and 11/13 paired runs, respectively; all three comparisons remain significant after correction. Table 10 lists the additional runs. Together with source-level resampling, these experiments strengthen evidence for repeatable downstream gains and selection stability within the available data. The external test set remains 247 images; the significance tests characterize training-run variability on this fixed benchmark.
Representativeness and extensibility of the candidate conditions
Section 3 now explains that rain, snow, fog, dust, and low light cover common degradations from precipitation, atmospheric obscuration, and reduced illumination. It also specifies how additional conditions enter the framework: generate paired variants on the same sources, extend the response matrix, and apply the same selection procedure.
We hope these additions help address your concerns about condition selection and experimental reliability.
Thank you to the authors for their additional experiments. The authors have addressed most of my concerns, including those regarding experimental reliability through the expanded seed evaluation. However, the limited test dataset size still remains a concern regarding generalizability.
Additional Results and Manuscript Revisions (1/2)
Following up on our earlier response, we have completed the additional analyses and incorporated them into the revised manuscript. Below we highlight the new evidence addressing your concerns and indicate where each addition can be found.
Q1. Random selection and downstream performance
Section 5.3 and Table 4 now compare our method with uniform random selection, validation-AP-based selection, K-means, hierarchical clustering, and farthest-first selection. At each K, methods share the generated-image count, clean/generated sampling proportion, training budget, and seeds. Random performance is calculated exactly over all same-size subsets. All selected subsets in Table 4 exceed their corresponding random expectation. At K=4, our method also exceeds the clustering baselines by 1.26 AP for RT-DETRv2-S and 0.31 AP for YOLO11m. Appendix D provides the protocols and explains how redundancy distinguishes alternatives with equal coverage.
Section 5.4 and Figure 4 bring the response–transfer analysis into the main text. Across ten condition pairs, response similarity correlates with symmetric cross-condition transfer gains (Spearman ρ=0.879; Pearson r=0.908). Appendix C provides the matched duplicated-clean controls and per-seed results. This supports the task relevance of the measured relationships.
Q2. Training quantity diversity and condition count
Section 5.2 and Appendix B.2 isolate the generated sampling proportion from condition composition. For RT-DETRv2-S, SDL and matched Full use 6,000 successful optimizer updates and a generated sampling proportion of 3/8, while matched Full retains all five conditions. The original three-seed gain is 2.11 AP (Table 9); the extended 13-seed comparison gives a significant mean gain of 1.454 AP (Table 3). The same-K comparisons in Table 4 additionally hold training-pool size and condition count constant. Tables 1–2 and the complete subset results in Appendices A.3 and B.1 show performance across condition counts, including cases where adding conditions reduces AP.