- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Universal Taylor Proxy Scoring (UTPS), a lightweight, training-free method that identifies the most task-relevant layers for sparse adaptation using only a single forward-backward pass. UTPS integrates gradient-weight sensitivity, activation-aware scaling, and parameter-density normalization to prioritize layers with the highest performance-per-parameter. The results seems good, but several inconsistencies make the main claims hard to verify.
Critical strengths
-
Selecting useful layers for LoRA is a practical idea, and the method is simple.
-
The experiments cover several tasks and models, with ablations, random selection, and different budgets.
Critical weaknesses
-
The layer counts in Fig. 2 need clarification. InternVL2-2B has 24 vision and 24 language Transformer layers, yet the figure reports 48 selected layers at a 50% budget, with some counts exceeding 24 per modality. If these numbers refer to modules or expert units, please define the counting unit and explain how it relates to the stated budget.
-
Selecting 50% of the layers does not necessarily mean using 50% of the LoRA parameters. Vision and language layers have different parameter costs. Tab. 4 only gives percentages, so the claimed budget matching across methods cannot be checked.
-
Some p-values in Tab. 7 appear inconsistent with the reported means and standard deviations. For example, the Flickr30k results imply a two-sided paired t-test p-value of at least about 0.057 under the stated three-run setup, rather than 0.02. Tab. 1 also reports best results, while the reproducibility statement says averages.
-
The explanation of Taylor-W is not supported by its formula. Taking the norm of the gradient–weight product removes signs and does not measure overall directional alignment. It therefore cannot support the claimed distinction between helpful and harmful updates. Its connection to actual LoRA gains also needs evidence.
Questions for authors
-
What does “layer” mean in Fig. 2? Please explain why 48 layers are selected at a 50% budget and why some counts exceed 24 per modality.
-
Please provide the actual trainable parameter counts for each method and task. How do you ensure equal parameter budgets when vision and language layers have different costs?
-
Please provide the individual results behind Tab. 7 and explain how the p-values were calculated, including whether the tests were one-sided or two-sided. Are the results in Tab. 1 best values or averages?
-
How does Taylor-W distinguish helpful from harmful updates when the norm removes signs? Is there direct evidence that its scores predict actual layer-wise LoRA gains?