acceptodds
Under review as a conference paper at ICLR 2027

Which Layers Should Share? Functional Substitutability for Language Model Compression

Abstract

Cross-layer parameter sharing compresses a pretrained language model without shortening its computation graph, but it creates an assignment problem: which position-specialized components should share a physical base, and which member should initialize it? We introduce SUBSHARE, which treats this decision as directed functional role transfer. For each compatible donor-target pair, SUBSHARE measures the next-token loss change caused by substitution in the intact model, converts the resulting asymmetric matrix into medoid group costs, and reuses it across budgets with Greedy or mixed-integer planning. Our local-to-global analysis explains how these costs can select good recovered plans even when interactions affect their absolute loss. Across two Qwen3.5 scales and two Gemma models, SUBSHARE outperforms every topology-, spacing-, and weight-based assignment control in all 16 model-budget settings, beats SharVeT assignments in all 16, and beats Geo-Sharing assignments in 15. The gains extend to zero-shot tasks and persist under Basis Sharing parameterization. Functional substitutability thus provides a reusable decision principle for allocating a compact set of shared bases across a pretrained network.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Language Model Compression, Cross-Layer Parameter Sharing, Functional Substitutability, Budget-Constrained Optimization, Knowledge Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_iFf2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SubShare, which decides which layers should share weights based on how much loss increases when one layer replaces another.

Critical strengths

The paper is very well written, clearly structured, and easy to follow. The motivation and methodology are explained clearly. The experiments are comprehensive, covering different model architectures, compression budgets, and several baselines.

Critical weaknesses
  1. Practical benefits remain unclear. The compressed model still exhibits a performance gap compared with the original teacher. For example, on Qwen3.5-2B at B67, the test NLL remains at 3.238 after 5M recovery tokens, compared with 2.594 for the original model. Moreover, despite reducing the parameter count, decoding throughput decreases from 24.3 to 21.9 tokens/s. The method also introduces additional computational costs for functional substitution measurements, assignment search, and recovery training.

  2. The downstream evaluation is restricted to benchmarks, primarily assessing language understanding and multiple-choice commonsense reasoning. lack of open-ended generation and challenging reasoning evaluations. Table 3 does not report the original teacher model's performance on these benchmarks. Without this reference, it is difficult to determine how much capability is actually retained after compression.

Questions for authors
  1. what are the practical advantages of SubShare over directly using a smaller pretrained model?

  2. Why are the downstream evaluations limited to likelihood-based benchmarks? Could the authors include open-ended generation tasks and report the original teacher's performance to better assess capability retention?

Reviewer_9uNrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SubShare selects which compatible layers share parameters by measuring the next-token loss change when a donor component replaces a target component. It converts these directed costs into medoid group costs and selects budget-constrained partitions with greedy or MILP planning. The main implementation combines shared bases with position-specific low-rank residuals and recovery training. Experiments cover four backbones and four sharing budgets, with additional tests under Basis Sharing and exact tying. The paper also gives a conditional analysis of when local substitution costs preserve the ordering of recovered plans.

Critical strengths
  1. Directed substitution provides a useful criterion for sharing: it measures whether a component can serve another position's role. Table 1 reports the lowest recovered NLL in 15 of 16 model–budget settings, including gains on Gemma with a fixed functional-MILP configuration. These results support the usefulness of the assignment criterion.

  2. The gains extend beyond the main shared-base-plus-residual implementation. Functional grouping improves results under Basis Sharing and exact tying (Tables 7, 21, and 22), supporting a contribution from assignment across different ways of representing shared weights.

Critical weaknesses
  1. The main Qwen comparison does not fully isolate the contribution of functional scoring. In most cells, SubShare selects among four scope/planner combinations, requiring 12 recovery runs, versus three for each fixed baseline; Qwen3.5-2B B67 uses two candidates (Appendix B.2). Calibration also differs: 32,768 tokens at length 2,048 for SubShare versus 10,240 at length 512 for the Geo-Sharing control (Appendices C.1 and D.2). Table 2(c) provides useful fixed-configuration evidence at B50/B67, but the full Qwen grid does not separate the scoring gain from additional candidate selection and calibration.

    The Geo-Sharing comparison also needs qualification. Appendix C.1 uses diagonal Fisher/K-FAC costs with histogram-constrained MILP, whereas the cited method selects shared bases using Hessian-eigenspace information (Zhang et al., Section 3.2). A common parameterization helps compare assignments, but changing the assignment criterion limits what this comparison establishes about the original method. The adequacy of this curvature control as a substitute for Geo-Sharing needs justification.

  2. The advantage over the strongest Qwen baseline is not established beyond short-recovery NLL. Geo-Sharing is the closest comparator throughout the Qwen grid, yet it is absent from both the six-task zero-shot evaluation and the matched 5M-token recovery comparison (Tables 3 and 14). At Qwen3.5-2B B67, the NLL advantage is 0.0375 (Table 8). The larger gains over Random and Adjacent do not establish that this advantage translates to downstream capability or persists with longer recovery.

    The native-model comparison further limits the demonstrated deployment benefit. After 5M recovery tokens, the 1.491B B67 model remains 0.419 NLL worse than native Qwen3.5-0.8B, which has 0.752B parameters (Appendix G). The MILP model also decodes at 21.9 versus 24.7 tokens/s in the reported setup (Table 23). These results leave the practical benefit of preserving the larger model's logical depth unclear in this setting.

I recommend weak rejection. The paper shows that functional grouping is useful, including under alternative sharing parameterizations. My main concern is whether the evidence establishes a sufficient advance over the strongest relevant baseline: the Qwen comparison combines scoring with additional selection and calibration, the curvature control changes the cited method, and the downstream and longer-recovery evaluations omit that control. A consistent fixed-configuration advantage, together with evidence that it extends beyond short-recovery NLL, would materially improve my assessment.

Questions for authors
  1. Using the completed runs, can you report each fixed scope/planner configuration across the Qwen grid alongside the Geo-Sharing control? How much of the Table 1 advantage remains without per-cell candidate selection?

  2. Which parts of the original Geo-Sharing assignment criterion are preserved by the diagonal Fisher/K-FAC implementation? What motivated the different calibration budgets and sequence lengths, and is there existing evidence that these choices do not change the comparison?

  3. Are Geo-Sharing results already available for the six zero-shot tasks or 5M-token recovery? If unavailable, please limit claims of downstream and longer-recovery advantages to the comparators actually evaluated.

1 comment

I maintain my original score.

Reviewer_or6qOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies cross-layer parameter sharing in pretrained language models and proposes SubShare. It measures changes in next-token loss caused by compatible donor–target parameter substitutions to construct a directed cost matrix, then determines sharing assignments through medoid grouping and Greedy/MILP planning. The main implementation uses shared bases with position-specific low-rank residuals, followed by recovery training. The paper also provides a conditional analysis linking local scores to global plan selection and evaluates the assignment criterion across multiple models, budgets, and sharing parameterizations.

Critical strengths

The idea of using actual substitution loss to determine which components are suitable for parameter sharing is valuable. It accounts for the target’s inputs and downstream computation, making it more directly relevant to the problem than relying only on layer distance or weight similarity. The measured cost matrix can be reused across budgets; the additional results with Basis Sharing and exact tying also indicate that this grouping approach does not depend entirely on the paper’s default low-rank residual implementation.

Critical weaknesses
  1. Section 3.3 mentions candidate recovery and development-set selection, but Figure 1 does not show these steps or indicate that Greedy uses the group-size template generated by MILP. The exact group-selection procedure in Greedy and the computation of the recovery losses are also not fully specified, leaving these details to be filled in when reproducing the method.
  2. Section 4 gives results assuming controlled variation in residuals across plans, but does not explain why this condition should hold during actual recovery training. In particular, whole-mixer scoring replaces the local dynamics, whereas the final model retains the target’s dynamics. The existing theory does not adequately explain why this score can still distinguish better plans from worse ones after shared-model construction and recovery.
  3. It is unclear to me what “the same plans” means in Table 2. Section 4 states that a plan includes both its groups and their donors, but does not explicitly state whether the two scoring methods keep the same donors. If each scoring method reselects medoids using Equation 3, the scores may no longer refer to the same complete plan.
  4. The comparisons on public tasks are still limited. Table 3 only evaluates Qwen3.5-2B B67 and includes only Random and Adjacent as baselines, without Geo-Sharing or SharVeT from the main table. It therefore does not show whether SubShare also outperforms these methods on these tasks.
Questions for authors
  1. What rule does Greedy use to select groups at each step? What are the exact formulas for the losses used in recovery training? Please also explain how controls such as Random and Adjacent select donors.
  2. When switching the scoring scope in Table 2, do you retain the donors used in the actual recovered models or reselect the medoids? Please describe the computation procedure actually used here.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.