- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SURF, a state-based Transformer adaptation framework combining similarity-guided layer grouping with sparse, token-dependent routing over shared rank-one features. The authors theoretically analyze coarse-control expressivity and feature reuse. Experiments across 34 datasets and 10 pretrained models demonstrate improved adaptation performance and inference efficiency compared with existing state-based methods.
Critical strengths
SURF addresses the inference overhead of non-mergeable state-based adapters, achieving decoding latency close to merged LoRA (1.05×) while substantially outperforming MoC in inference efficiency. The paper provides extensive evaluations across 34 datasets and 10 pretrained models covering language, vision, reasoning, and time-series tasks, with ablations supporting the effectiveness of similarity-guided grouping and feature-level routing.
Critical weaknesses
W1. Although Theorem 3.1 establishes the feasibility of coarse control under specific assumptions, it remains unclear whether SURF's fixed shared feature pool and sparse routing can realize the theoretical construction without quantified approximation errors.
W2. Figure 3 and Table 11 demonstrate end-to-end efficiency gains, but lack a breakdown separating intervention reduction, controller computation, activation movement, and routing overhead; the contribution of offline grouping costs is also unclear.
W3. Insufficient justification for similarity-guided grouping. Although cosine similarity measures representation similarity, its relationship with control-merging errors remains unverified, leaving unclear why it is an appropriate criterion for selecting intervention boundaries beyond empirical comparisons with random and uniform grouping.
W4. Lack of statistical robustness in key ablation results. The improvements in Tables 6 and 7 are relatively modest (0.7 and 0.5 percentage points over random grouping and MoC, respectively), but multi-seed results or confidence intervals are not reported.
W5. SURF estimates layer similarity using only 128 calibration examples per task, but the stability of grouping boundaries across calibration subsets and the criterion for selecting similarity thresholds remain insufficiently established.
W6. Incomplete controlled analysis of intervention frequency. Although Tables 6 and 14 and Figure 4(d) examine grouping effects, systematic comparisons across intervention counts under matched controller capacity and training settings are needed to disentangle intervention frequency from grouping locations and quantify accuracy–efficiency trade-offs.
Questions for authors
Please refer to Weaknesses W1–W6 for my questions and concerns.
3 comments
Thank you very much for your detailed response and the additional experiments.
While the underlying reasons why fewer interventions can lead to higher accuracy remain somewhat unclear, I acknowledge that the additional experiments have substantially strengthened the empirical justification for the proposed design.
Thank You for Your Constructive Feedback
Dear Reviewer,
We sincerely thank you for your thoughtful follow-up and for taking the time to carefully consider our additional experiments and clarifications.
We are grateful that you have acknowledged the strengthened empirical evidence, and we greatly appreciate your constructive feedback throughout the discussion, which has helped us substantially improve our empirical analysis and clarify the scope and limitations of our work.
We have made every effort to address your concerns and sincerely hope that these improvements will support a more favorable assessment of our work.
We truly appreciate your time, thoughtful consideration, and valuable contribution to the ICLR 2027 review process!
Sincerely,
The Authors of Paper #19432
Thank you for the additional experiments and clarifications. However, I remain uncertain about why coarse-grained control achieves higher accuracy than fine-grained subblock-level control, beyond its efficiency advantages. Although Table 1 shows that layer-wise control outperforms subblock-level control under matched parameter budgets, the two settings use different per-intervention capacities. Moreover, the additional experiments in W6 show that increasing the number of interventions from 4 to 16 consistently improves accuracy, albeit at a higher computational cost.
I would therefore appreciate a more comprehensive comparison between coarse-grained and fine-grained control using the same controller architecture. In particular, if additional computational overhead or controller capacity is allowed, can fine-grained subblock-level control achieve higher accuracy than SURF?