- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes MSLoRA (Multi-Scale Low-Rank Adaptation), a parameter-efficient fine-tuning (PEFT) method designed specifically for dense vision tasks like object detection and segmentation. The proposed idea is to introduce a spatially varying multiplicative gate alongside the standard additive low-rank residual used in LoRA. Specifically, a frozen feature map is updated by adding a low-rank projection that is element-wise multiplied by a multi-scale spatial attention map computed via parallel depth-wise convolutions. The authors hypothesize that while spatially uniform low-rank updates (like standard LoRA) are sufficient for global image classification, dense prediction tasks benefit heavily from spatial selectivity. Experiments across various architectures (ResNet, Swin, ConvNeXt) and datasets (COCO, ADE20K) demonstrate that MSLoRA achieves competitive or superior performance compared to full fine-tuning and baseline PEFT methods, all while utilizing a small parameter budget.
Critical strengths
The paper provides an insightful analysis of parameter budget allocation in parameter-efficient fine-tuning (PEFT). By identifying that standard LoRA suffers from spatial uniformity, the authors explicitly hypothesize and demonstrate that allocating part of a fixed parameter budget to a multi-scale spatial gate is more valuable for dense prediction than spending it entirely on channel recombination. This budget-allocation principle is well validated in their controlled experiments.
The empirical results include a variety of backbone architectures, including both CNNs and ViTs. The proposed method consistently achieves or exceeds full fine-tuning performance on dense prediction tasks with a very small parameter budget, proving it is an effective improvement for localized representation requirements.
Critical weaknesses
The primary weakness of this submission is its limited architectural novelty and substantial conceptual overlap with existing literature on spatial attention and parameter-efficient fine-tuning. The core operation in MSLoRA combines a low-rank linear projection with a multi-scale depth-wise convolutional attention gate. However, injecting spatial priors via depth-wise convolutions into adapters is already heavily explored in prior works such as ConvAdapter (Chen et al., CVPR 2024) and ViT-Adapter (Chen et al., ICLR 2023). Furthermore, the concept of element-wise spatial modulation of feature maps is a standard paradigm in computer vision, extensively explored in foundational attention mechanisms like the Convolutional Block Attention Module (CBAM) (Woo et al., 2018) and dynamic convolutions (Chen et al., 2020). Modifying the additive branch of a standard adapter with a spatially modulated gate represents a relatively straightforward fusion of two well-established techniques, which limits the theoretical and architectural novelty of the contribution.
Additionally, the efficiency claims require more nuanced qualification. Because MSLoRA relies on an input-dependent, spatially varying multiplicative gate, it fundamentally loses the re-parameterization property of standard LoRA. Meaning the adapter weights cannot be merged back into the frozen backbone during inference. This introduces a persistent computational and memory overhead during deployment. Although the authors acknowledge this limitation and show lower training throughput in Table 5, the lack of an inference latency profile makes it difficult to assess the true deployment cost compared to structurally re-parameterizable approaches like RepAdapter (Luo et al., 2023) for example.
As a result I am borderline leaning on a weak reject for the paper in the current state.
Chen, H., et al. Conv-Adapter: Exploring Parameter Efficient Transfer Learning for Convnets. CVPR 2024
Chen, Z., et al. Vision Transformer Adapter for Dense Predictions. In International Conference on Learning Representations, ICLR 2023.
Woo, S., et al. CBAM: Convolutional Block Attention Module, ECCV 2018.
Chen, Y. et al. Dynamic Convolution: Attention over Convolution Kernels, CVPR 2020.
Luo, G., et al. Towards Efficient Visual Adaption via Structural Re-parameterization. arXiv preprint arXiv:2302.08106.
Questions for authors
Could the authors provide a detailed profile of the inference latency and memory overhead? Since the gating mechanism precludes weight merging, this overhead could be a significant bottleneck for real-time dense prediction tasks compared to standard LoRA.
How does MSLoRA compare empirically against other spatially-aware or dynamic PEFT methods that modulate parameter updates based on input features, such as Dynamic Visual Prompt Tuning (Derakhshani et al., 2023) or mixture-of-expert adapter approaches like ParaX (Wang et al., 2024)? Including these baselines would help contextualize the performance gains of your input-dependent design.
Could you clarify if the hyperparameter configurations, such as the bottleneck dimension and group size, require extensive dataset-specific tuning when migrating across fundamentally different backbones, or is the default configuration robust across all settings?
Derakhshani, M. M., et al. Variational Prompt Tuning Improves Generalization of Vision-Language Foundation Models, ICLR 2023.
Wang, S., Song, et al. Parameters as Experts (ParaX): Adapting Vision Models with Dynamic Parameter Routing and Shared Expert Centers, ICML 2025.
Dear Reviewer,
Thank you again for your valuable feedback. We have provided a detailed response to your concerns and would greatly appreciate it if you could take a look when you have a chance.
We would be grateful to know whether our clarifications have adequately addressed your main concerns, or if there are any remaining questions that we could further clarify during the discussion period.
Thank you for your time and consideration.