- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Search2Adapt, a training-free search framework for adapter architectures in segmentation foundation models. It searches over adapter operations, operation-specific configurations, and layer-wise insertion strengths. Candidate architectures are evaluated using a zero-cost gradient proxy, ZiCo, together with a sparsity penalty, and the search variables are updated through score-weighted perturbations. The selected architecture is then trained normally. Experiments on camouflage, polyp, and nuclei segmentation show that the searched adapters are generally competitive with fixed adapter designs.
Critical strengths
-
The problem is relevant: a fixed adapter architecture may not be equally suitable for different domains, and automatically searching adapter structures is a reasonable direction.
-
The search space is more flexible than simply choosing between a few fixed adapter types. It jointly considers adapter operations, their configurations, and layer-wise insertion patterns. The method is evaluated across multiple datasets, foundation models, and zero-cost proxies.
Critical weaknesses
-
The main assumption behind the method is not sufficiently validated. The framework relies on ZiCo scores to rank untrained adapter candidates, but the paper does not clearly establish the correlation between the proxy score and the final trained performance. It also does not provide a direct downstream comparison with random search or evolutionary search under the same candidate-evaluation budget. Therefore, it is unclear whether the reported gains come from the proposed perturbation search, the proxy, or simply selecting a good candidate from a larger search space.
-
Some of the main claims are overstated. The search still uses labeled data and forward/backward computation, and the selected adapter is trained normally, so “training-free” only applies to candidate evaluation rather than the full adaptation process. In addition, continuous layer-wise coefficients do not necessarily remove adapter parameters or computation, so the claimed sparsity and efficiency benefits are not fully demonstrated. The paper also does not isolate the contributions of operation search, configuration search, layer search, and the sparsity penalty.
Questions for authors
-
Can the authors report the relationship between zero-cost proxy scores and the final performance of the corresponding trained candidates? This would clarify whether the proxy provides a reliable ranking signal.
-
Under the same search space, proxy, and number of candidate evaluations, how does perturbation-based search compare with random search and evolutionary search?
-
How much computation is required for the complete pipeline, including candidate scoring, search, and final training, compared with training a fixed adapter? Do the continuous layer coefficients actually reduce parameter count, FLOPs, or memory?
-
Can the authors provide ablations that separately evaluate operation search, configuration search, layer-wise search, and the sparsity penalty?
These are central issues for assessing whether Search2Adapt is a reliable architecture-search method rather than a heuristic that happens to find competitive adapters on the evaluated tasks. If the authors address these concerns with convincing evidence, I would reconsider my current rating and potentially raise it to 3.
2 comments
The authors have not adequately addressed my concerns. And, too late, I therefore maintain my original rating of Weak Reject.
Regarding the continuous layer coefficients, Table 9 of the supplementary material compares continuous and hard insertion on two datasets. On Kvasir-SEG, hard insertion places adapters in only two of the 12 Transformer layers, whereas continuous insertion uses all 12 layers. Despite this, the architecture obtained with continuous insertion requires only 0.295M adapter parameters and 2.416 GFLOPs per image, compared with 2.370M parameters and 3.222 GFLOPs for hard insertion. These results show that continuous insertion can yield a more compact architecture with lower computational cost. This difference reflects the combinations of the layer-wise insertion pattern and the selected adapter operations.