- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Rare but important samples often receive weak gradient signals during training, leading models to underfit them. This paper proposes REAP, an adapter module inserted after each stage of a hierarchical backbone, to provide architectural support for learning from rare samples. The method is evaluated across multiple task domains and shows improved tail IoU on ADE20K.
Critical strengths
The evaluation covers a broad range of tasks, including segmentation, classification, and regression.
Critical weaknesses
-
Single-run results and a weak baseline. The paper appears to report only one run per experiment, making it unclear how much run-to-run variance could affect the conclusions. This is especially concerning when the improvements over the baselines are tiny (e.g., Kvasir F1: SegFormer 93.75 vs. REAP+IS 93.76). Moreover, the baseline SegFormer (MiT-B4) achieves only 24.1 mIoU on ADE20K, far below the 50.3 mIoU reported by Xie et al. (2021). Although the experimental protocols differ, gains over such a weak baseline are difficult to interpret, making the improvements less convincing.
-
Limited experimental breadth and insufficient validation of REAP's architectural contribution. The proposed module is evaluated only on variants of SegFormer, despite being intended as a broadly applicable method. Moreover, the current experiments do not sufficiently isolate the effects of different components. I would expect the following ablations on one or two key datasets:
| No adapter | Plain adapter (with approximately the same number of parameters as REAP) | REAP | |
|---|---|---|---|
| Without IS | ✓ | ✓ | ✓ |
| With IS | ✓ | ✓ | ✓ |
The method combines known components (importance sampling, bottleneck layers, and sigmoid gating), with REAP being the main contribution. However, it remains unclear why REAP's architecture would specifically help the network learn more effectively from rare samples.
Questions for authors
-
ImageNet-LT is a standard benchmark for long-tailed learning. Why do you construct a three-class ImageNet subset instead of using ImageNet-LT?
-
How can you demonstrate that the improved downstream performance comes from REAP's design rather than simply from additional trainable parameters? For example, on ADE20K, the backbone learning rate is 5e-7, while the adapter learning rate is 5e-5. This effectively introduces additional trainable parameters that are updated with a much higher learning rate than the backbone. Therefore, a parameter-matched plain adapter baseline would help clarify whether the improvements come from REAP's specific architecture or simply from the additional trainable capacity.
3 comments
Thank you to the authors for the response and the additional experiments. Having reviewed them, I maintain my original rating.
The newly added Table 8 provides the exact comparison I requested; however, it reveals that the gating mechanism offers almost no additional benefit over the Plain Adapter. Under the IS setting, REAP’s PHM08 score is only about 0.2% lower than that of the Plain Adapter (397.50 vs. 398.28 for FD001) and 0.4% lower (13198.49 vs. 13250.65 for FD004). For FD001, the RMSE is nearly identical to that of the Plain Adapter (15.0316 vs. 15.0295)—and it should be noted that these figures represent the results of only a single run. In contrast, for FD004, the PHM08 score improved from 14716.48 (without an adapter) to 13250.65 (with the Plain Adapter). At least on this dataset, the performance gains appear to stem primarily from the introduction of additional trainable parameters, rather than from the gating design itself.
Furthermore, my question regarding the variance across multiple runs remains unaddressed.
Clarification of Spatial Gating and Multi-Seed Evaluation
We thank the reviewer for the careful follow-up and for highlighting the limited incremental performance gains of gating and the lack of multi-run variance estimates. We agree that the original single-run results in Table 8 are insufficient to establish a robust performance advantage of the gating mechanism. We have therefore repeated the experiments using three random seeds (42, 43, and 44).
Multi-seed evaluation. All experiments follow the original IS/Top-K protocol, using the same pretrained checkpoint while varying the randomness of post-training. No subsequent changes to the learning rate, gating architecture, or training procedure were introduced. For each run, we select the checkpoint with the best validation PHM08 score and report the corresponding test results. The mean and sample standard deviation across three seeds are summarized below.
| Dataset | Method | RMSE ↓ | PHM08 ↓ |
|---|---|---|---|
| FD001 | No Adapter | 15.1305 ± 0.0034 | 405.10 ± 0.46 |
| FD001 | Plain Adapter | 15.0351 ± 0.0130 | 398.83 ± 0.55 |
| FD001 | REAP | 15.0389 ± 0.0065 | 398.75 ± 1.27 |
| FD004 | No Adapter | 24.8334 ± 0.0000 | 14716.48 ± 0.00 |
| FD004 | Plain Adapter | 24.7017 ± 0.0049 | 13270.84 ± 39.59 |
| FD004 | REAP | 24.6971 ± 0.0111 | 13241.78 ± 55.90 |
For FD004, all three No-Adapter runs selected the shared initialization checkpoint at step 0, which explains the zero standard deviation.
Across three seeds, REAP consistently improves upon the No-Adapter baseline in both datasets, reducing the mean PHM08 score by 1.57% on FD001 and 10.02% on FD004. However, we acknowledge the reviewer's central observation: the additional PHM08 improvements over the Plain Adapter are modest in the regression tasks. These results indicate that the additional residual adaptation capacity accounts for a substantial portion of the aggregate performance gains in these regression experiments. We do not interpret the multi-seed results as evidence of a statistically robust standalone performance advantage from gating.
Due to time constraints, we have not yet evaluated the visual tasks across multiple seeds. As a complementary qualitative analysis, we visualized gate activations from the trained ISIC checkpoint and observed spatially heterogeneous, stage-dependent responses. These visualizations characterize the learned modulation patterns of the gating mechanism.
Clarification of the gating mechanism. We would also like to clarify the motivation behind the spatial gate. Its primary purpose is not simply to maximize aggregate predictive performance, but to introduce an input-dependent, regularization-like inductive bias into residual adaptation. Specifically, the learned gate continuously modulates the magnitude of adapter residual updates at individual spatial locations. This design encourages selective modifications to pretrained representations, potentially limiting unnecessary perturbations in regions that require little additional adaptation while retaining flexibility for atypical patterns. Unlike explicit regularization, this behavior is learned end-to-end through the task loss, without additional routing supervision or rarity labels.
To provide direct qualitative evidence of this behavior, Figure 9 in Appendix A.4 visualizes the learned spatial gates across feature stages C1–C3 in ISIC 2018 skin lesion classification task. The shallow gate (C1) exhibits relatively broad responses to low-level textures, whereas C2 shows more localized responses near lesion boundaries. At the deeper stage (C3), the relative gate responses become more concentrated around lesion regions, with generally weaker activation over the surrounding background. These stage-dependent patterns emerge without explicit spatial supervision, supporting the interpretation of gating as a learned mechanism for selective residual modulation.
We emphasize that these visualizations characterize the behavior of the learned gate and do not, by themselves, establish a causal improvement in predictive accuracy or training stability. Nevertheless, together with the multi-seed results, they help distinguish the two architectural roles within REAP: the bottleneck adapter provides additional adaptation capacity, while the spatial gate introduces input-conditioned control over how that capacity is expressed.
We appreciate the reviewer's concern and will clarify this distinction in the manuscript to avoid attributing the overall performance gains solely to the gating mechanism.
Response to Baseline Evaluation, Architectural Ablation, and Long-Tailed Benchmark Design
We thank the reviewer for the detailed feedback. We have revised the manuscript with additional ablations and analyses to better clarify the contribution and scope of REAP.
1. ADE20K baseline and evaluation protocol.
Our ADE20K experiments use a crop-based protocol with 256 × 256 crops for both training and validation (Appendix A.2.1), which differs from the original SegFormer evaluation setting. Thus, the absolute mIoU values are not directly comparable. Importantly, all methods in Table 3 follow the same protocol. Within this controlled setting, REAP improves overall mIoU from 24.14% to 30.61% and tail IoU from 9.94% to 20.58%, while preserving head-category performance. We acknowledge that evaluation under the original high-resolution protocol would provide stronger evidence of general applicability.
2. Architectural contribution and component ablation.
We introduce the requested ablation structure in Table 8, comparing No Adapter, Plain Adapter, and REAP under both uniform and importance sampling on C-MAPSS FD001/FD004. The Plain Adapter retains the same residual bottleneck architecture and training protocol but removes input-dependent spatial gating. Under IS, REAP improves PHM08 on both subsets and achieves the lowest RMSE on FD004. This comparison provides evidence of a gating-specific benefit beyond introducing a conventional adapter with additional trainable capacity.
We also provide an instance-level rarity analysis (Figure 3), where gate activation correlates positively with visual rarity (Spearman's ρ = 0.717), including after within-class centering (ρ = 0.494). Figure 1(c) further illustrates distinct adaptation behaviors across feature stages. Together, these analyses provide additional evidence supporting the intended input-dependent adaptation mechanism.
3. Choice of the ImageNet subset instead of ImageNet-LT.
We constructed the three-class ImageNet subset to examine post-training under a fixed, severe class imbalance that persists at test time. It contains one frequent class (tench) and two visually distinct minority classes (conch and Labrador retriever), each representing approximately 1% of both training and test examples (Appendix A.1.2, Table 10).
Preserving this imbalance is important because precision at a fixed high recall depends on class prevalence. Our setting therefore examines whether REAP can recover minority-class examples while limiting false positives under the same imbalance encountered during training (Table 4).
ImageNet-LT addresses a different, broader setting: recognition across 1,000 classes with varying training frequencies and a class-balanced test set. We acknowledge that our three-class experiment does not establish performance at that scale and is intended as a targeted evaluation of high-recall minority-class recognition rather than a replacement for ImageNet-LT.
4. Scope of architectural generalization.
Although our experiments cover classification, segmentation, and regression, the current implementations use variants of the SegFormer family. We therefore distinguish generalization across task domains from generalization across backbone architectures. Extending REAP to other hierarchical backbones remains an important direction for further validation.
We hope these clarifications and the additional analyses address the concerns regarding the experimental setting and the contribution of REAP's architectural design.