acceptodds
Under review as a conference paper at ICLR 2027

DESA: Delivered-Energy Spectral Allocation for Merging Reinforced Experts

Abstract

Model merging provides a practical way to combine multiple experts into a single model without retraining or increasing inference cost. However, existing merging methods are largely developed for supervised fine-tuning and do not account for the distinctive structure of reinforcement learning with verifiable rewards (RLVR) updates. We show that RLVR task vectors exhibit two properties that are especially important for merging: (i) apparent sparsity under low-precision representation, and (ii) substantial heterogeneity in update energy across experts. These properties make sparsification-based heuristics unreliable and cause existing spectral methods to impose an uneven, supply-determined allocation of delivered energy, which strongly affects axis-wise recovery after merging. Motivated by this observation, we propose DESA (Delivered-Energy Spectral Allocation), a training-free spectral merging method that explicitly reallocates expert-level delivered energy while preserving a shared spectral frame. DESA introduces controlled budget equalization across experts together with retention floors that prevent under-allocation to experts whose capabilities are sensitive to reduced budget. Across both language and vision-language RLVR expert pools, DESA consistently outperforms strong baselines in average accuracy and recovery. Our results identify explicit delivered-energy allocation as a key ingredient for merging RL-trained experts.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Model merge, RLVR, LLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dZ99OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DESA, a training-free spectral merging method for integrating multiple RLVR-trained experts. The authors identify two characteristics of RLVR task vectors: apparent sparsity under low-precision representations and heterogeneous update energy across experts. They argue that existing spectral merging methods implicitly allocate energy according to the retained spectral magnitude of each expert, which can lead to imbalanced capability recovery. To address this issue, DESA explicitly redistributes expert-level spectral energy through a power-law allocation rule and introduces retention floors to protect sensitive experts. Experiments on language and vision-language expert pools demonstrate improvements over several existing merging baselines.

Critical strengths
  1. The paper highlights expert-level spectral energy allocation as an important factor in model merging.

  2. DESA achieves competitive improvements across both language and vision-language RLVR expert pools without additional training.

Critical weaknesses
  1. The claimed RLVR-specific characteristics are not sufficiently established, and similar magnitude imbalance and spectral calibration problems have been explored in prior work. This weakens the motivation and makes the novelty difficult to assess.

  2. The proposed energy allocation function appears heuristic, and the hyperparameter selection protocol remains unclear. These issues raise concerns about the method's generalizability and practical applicability.

Questions for authors
  1. The reported sparsity largely results from bf16 precision limitations. Could it be mitigated by higher-precision or longer training? Moreover, I think energy heterogeneity also exists in SFT experts. What makes these issues particularly specific to RLVR?

  2. What motivates the specific choice of ui=ei1−βu_i=e_i^{1-\beta}? Is there any theoretical justification, or empirical evidence demonstrating its superiority over alternative allocation functions?

  3. How are β\beta, ϕi\phi_i, and scs_c selected in practice? Does this selection require downstream validation data?

  4. Singular-value calibration and magnitude-aware merging have already been investigated in some related work like SVC [1]、SWIM-Merging [2] and DisTaC [3]. What fundamentally distinguishes DESA from these approaches? A direct comparison and explanation would help clarify the contribution.

[1]. When Shared Knowledge Hurts: Spectral Over-Accumulation in Model Merging (ICML26)

[2]. SWIM-Merging: Reweighting Singular Values of Weight Matrices for Efficient Test-Time Model Merging (Neurocomputing26)

[3]. DisTaC: Conditioning Task Vectors via Distillation for Robust Model Merging (CVPR26)

10 comments
AuthorsOpenReview ↗

Response to Reviewer dZ99 [1/3]

W1 & Q1: Apparent sparsity in bf16.

(a): Under fp32 there is no apparent sparsity

Following your suggestions, we test the experts in our pool using fp32 master weights.

In FP32, the task vector has rare exact zeros, and the exact-zero rate is around 2.9×10−52.9\times10^{-5}. In full precision, the RLVR task vector is dense. The sparsity therefore exists only under bf16, which is exactly why we call it apparent.

Reading of the task vectorExactly-zero coordinates
fp32 master weights2.9×10−52.9\times10^{-5}
bf16, as released72.4872.48 %

And bf16 is not our choice; it is how open-weight LLMs are mainly released. We surveyed the flagship releases of seven major open-weight families (Qwen, Llama, Mistral, DeepSeek, Gemma, Phi, and others). Of the 54 repositories that resolved: 47 BF16 (87.0 %), 3 FP16, 2 FP8, 2 INT8, and 0 in fp32 (0.0 %). Besides, for bf16 checkpoints, upcasting them to fp32 recovers nothing. Apparent sparsity is thus a property of the checkpoints that are actually available for merging, which is the regime the method targets.

(b) Longer training reduces apparent sparsity only marginally, and is not a practical lever for model merging.

We thank the reviewer for this insightful question. Apparent sparsity does decrease as training progresses. However, our empirical measurements show that this decrease is slight and saturates quickly, leaving task vectors firmly in the high-sparsity regime.

Tracking the exact bf16 zero fraction across all weight tensors on our vision-language pool yields a clear flattening trajectory.

Training step4080120160165 (3 epochs)
bf16 zeros87.84%83.20%80.11%77.60%77.30%

The rate of sparsity reduction decelerates monotonically. Moreover, training length is not a viable lever in practice: (1) RLVR steps are set by task convergence, and over-training incurs heavy compute costs and overfitting risks; and (2) model merging typically consolidates off-the-shelf, released checkpoints, where retroactively extending training is impossible.

cc What makes energy heterogeneity particularly specific to RLVR?

We thank the reviewer for this question. While energy variations exist in SFT, relative energy heterogeneity is substantially more pronounced in RLVR primarily due to the vast difference in their update scales:

As shown in Figure 1a, SFT task vectors have a much larger overall displacement, over two orders of magnitude larger than RLVR updates (116×116\times larger in ∥δi∥F\|\delta_i\|_F). Because SFT operates at such a large baseline scale, variations across tasks result in only a small relative ratio. For example, across standard SFT experts fine-tuned from the same Qwen2.5-7B base (e.g., Math 108.4108.4, Medical 114.8114.8, Table 96.296.2, Tool 91.591.5), their Frobenius norms differ by a factor of only 1.25×1.25\times (an energy EiE_i ratio of 1.57×1.57\times), yielding nearly balanced TSV delivered shares.

In contrast, because RLVR updates are extremely small in absolute scale (∥δi∥F≈0.5-1.4\|\delta_i\|_F \approx 0.5\text{-}1.4), even modest differences in update magnitude translate into a large relative disparity across experts. Consequently, our RLVR experts differ by 2.65×2.65\times in norm and 7.02×7.02\times in energy EiE_i, which spectral truncation further amplifies into an 8.8×8.8\times gap in delivered share (4.51%4.51\% vs. 39.62%39.62\%).

AuthorsOpenReview ↗

Response to Reviewer dZ99 [2/3]

W2: Motivation of uiu_i

We derive the form of uiu_i from an entropy-regularized allocation problem. Specifically, maximizing entropy under a linear tilt toward supplied energy. In this formulation, the exponent β\beta emerges naturally as the weight of that tilt. We present the derivation below.

Let q∈ΔNq \in \Delta_N denote a candidate expert-level allocation distribution (where qi≥0q_i \ge 0 and ∑iqi=1\sum_i q_i = 1), and recall that αi=ei/∑jej\alpha_i = e_i/\sum_j e_j is expert ii's retained spectral share ratio (defined in Eq. 2). We show that the normalized reference allocation, defined as qi∗:=ui/Mq^*_i := u_i/M (where M=∑iuiM=\sum_i u_i), is the unique solution to the following entropy-regularized allocation problem:

q∗=arg⁡max⁡q∈ΔN{H(q)+(1−β)∑iqilog⁡αi},H(q)=−∑iqilog⁡qi.q^* = \arg\max_{q\in\Delta_N} \lbrace H(q) + (1-\beta)\sum_i q_i \log \alpha_i \rbrace,\qquad H(q) = -\sum_i q_i \log q_i.

The objective is strictly concave on the simplex because H(q)H(q) has a negative second derivative (−1/qi<0-1/q_i < 0) and the second term is linear. Consequently, the maximizer q∗q^* is unique for any given β\beta. By introducing the Lagrangian L=H(q)+(1−β)∑iqilog⁡αi−ν(∑iqi−1)L = H(q) + (1-\beta)\sum_i q_i\log\alpha_i - \nu(\sum_i q_i - 1), the stationarity condition yields:

∂L∂qi=−log⁡qi−1+(1−β)log⁡αi−ν=0⟹qi∗=αi1−β e−1−ν.\frac{\partial L}{\partial q_i} = -\log q_i - 1 + (1-\beta)\log\alpha_i - \nu = 0 \quad\Longrightarrow\quad q^*_i = \alpha_i^{1-\beta}\,e^{-1-\nu}.

Since e−1−νe^{-1-\nu} is a constant independent of ii, we have qi∗∝αi1−βq^*_i \propto \alpha_i^{1-\beta}, which directly yields ui∝ei1−βu_i \propto e_i^{1-\beta} in Eq. (3). This objective has a clear physical interpretation: H(q)H(q) regularizes the allocation toward uniformity to prevent over-concentration, while ∑iqilog⁡αi\sum_i q_i \log \alpha_i rewards alignment with the spectral supply, with β\beta balancing their trade-off.

We have already added the motivation of uiu_i's form in the revised version.

W3: Hyperparameter choice.

The allocation parameters are selected using a small validation set. This practice aligns with the standard protocol across the model merging literature, such as Task Arithmetic, TIES-Merging, and TSV, which similarly rely on held-out validation splits to tune merging hyperparameters. In our setup, hyperparameters are chosen based on the validation macro-average. We have made this protocol explicit in the revised manuscript.

Moreover, we note that DESA does not rely on brittle hyperparameter tuning: as shown in Figure 2 and Table 9 (Appendix C.1), performance remains stable across a broad range of β\beta, ϕ\phi, and scs_c. Even a hyperparameter-free uniform allocation already outperforms standard TSV (Table 4).

AuthorsOpenReview ↗

Response to Reviewer dZ99 [3/3]

W4: What distinguishes DESA from SVC, SWIM-Merging, and DisTaC?

We thank the reviewer for pointing out these works. While all four methods involve magnitude or singular-value rescaling, they target different training regimes and rescale fundamentally different mathematical objects.

SVC, SWIM-Merging, and DisTaC all target models trained via standard supervised learning across vision and language benchmarks (e.g., ViT, Flan-T5, and SFT-trained LLMs). In contrast, DESA is built for the RLVR post-training regime, where task vectors are tiny, nearly orthogonal (pairwise cosine 0.00180.0018), and highly heterogeneous in energy (7.02×7.02\times spread). Here, the primary bottleneck is how unevenly standard spectral operators divide the merge budget among experts.

MethodRoleObject rescaledGranularityNeeds training
SVCplug-insingular values inflated by over-accumulation of shared directionsdirectionNo
SWIM-Mergingplug-insingular values, learned by entropy minimizationdirectionYes (gradient descent)
DisTaCplug-inthe norm of each task vectorwhole task vectorYes (distillation)
DESAstandalone operatorexpert-block delivered energy in the pooled spectral frameexpert blockNo

Specifically, three substantive distinctions separate DESA from these approaches:

  1. Expert Attribution vs. Post-Summation Scaling (vs. SVC & SWIM): SVC and SWIM apply SVD to the already summed update (Δmerged\Delta_{\mathrm{merged}}). Conversely, DESA decomposes each expert separately and preserves explicit expert blocks (BiB_i) in the pooled orthogonal frame, directly enabling expert-level budget control.

  2. Target Objective: Shared Directions vs. Budget Imbalance (vs. SVC): SVC calibrates singular values inflated by shared-direction over-accumulation, whereas DESA resolves inter-expert budget imbalance caused by RLVR energy heterogeneity (7.02×7.02\times spread) and safeguards vulnerable experts via retention floors.

  3. Resource Assumptions (vs. SWIM & DisTaC): Unlike SWIM (requiring test-time gradient descent) and DisTaC (requiring data-driven distillation), DESA and SVC are strictly one-shot and training-free. We thus focus our empirical comparison on SVC, which shares our setting.

We evaluated the official SVC implementation on our 5-expert language pool across three base operators:

base operatorwithout SVCwith SVCΔ\Delta
Task Arithmetic62.1367.98+5.85
TIES65.6667.49+1.83
TSV66.5767.09+0.52
DESA68.62--

Operating on the exact same spectral frame as TSV, post-hoc direction calibration via SVC yields 67.0967.09 (+0.52+0.52), whereas reallocating expert-level delivered energy via DESA achieves 68.6268.62 (+2.05+2.05), outperforming all baseline+SVC combinations.

We have cited and discussed all three works in the revised manuscript.

Reviewer_7isxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DESA, a spectral-based model merging method for combining RLVR experts. It shows that RLVR task vectors exhibit high energy heterogeneity and sparsity under low-precision representation, making coordinate-based merging heuristics unreliable and existing spectral methods prone to uneven expert allocation. DESA addresses this by explicitly reallocating spectral energy across experts through an equalization parameter and per-expert retention floors. Experiments on language and vision-language expert pools show improvements over a range of merging baselines.

Critical strengths
  • Clear motivation: The paper identifies a challenge in merging RLVR trained experts.
  • Broad baseline comparisons: DESA seems to outperform current model merging methods, including coordinate-based task vectors and spectral methods, for merging RLVR experts.
  • Useful empirical analysis: The allocation sweeps and ablations show that changing expert-level spectral budgets can indeed affect downstream recovery.
Critical weaknesses

Heuristic designs. The rule in Eq. (3) appears heuristic to me. The definition of the reference allocation u_i and its total mass M are not adequately justified why this should be an appropriate design. The exponent β\beta interpolates between supply-proportional allocation and uniform allocation, but this seems to be a design choice rather than a consequence of an optimization objective or a derived model of expert utility. Subsequently, the KL optimization (4) does not explain why the chosen reference allocation uu is optimal. It is also not clear why sallocs^alloc in Eq. (6) is defined that way.

Confusing terminologies. The paper uses supplied energy, retained spectral energy, allocated budget, and delivered energy in ways that are difficult to distinguish. While the appendix does distinguish these quantities, the main text should introduce them more clearly. In particular, the statement “retaining energy is also allocating budget” is vague, and should be supported by relevant equations.

Unclear energy heterogenity analysis. The paper states that: “Consequently, the heterogeneity of ∥δi∥F\|\delta_i\|_F propagates to the budget allocation, modulated only by the degree of concentration of the respective updates”. The phrase “degree of concentration” is not sufficiently defined. The authors should clarify this. Moreover, the definition of efk(i)ef_k(i) is also vague: efk(i)ef_k(i) is defined as the fraction of expert i's energy captured by its top-k directions, but the notation and definition are not sufficiently explicit in the main text, hindering readability.

Relationships between delivered energy and performance. The paper reports a strong correlation between delivered energy and recovery, but this relationship alone does not establish that delivered energy directly affects task performance. Does this relationship generally hold across different domains?

Questions for authors

See Weaknesses.

8 comments
AuthorsOpenReview ↗

Response to Reviewer 7isx [1/3]

We thank the reviewer for the insightful feedback. Below, we provide the missing definitions and equations.

W1: Heuristic designs.

(a) Motivation of the form of reference allocation uiu_i in Eq. (3)

We derive the form of uiu_i from an entropy-regularized allocation problem. Specifically, maximizing entropy under a linear tilt toward supplied energy. In this formulation, the exponent β\beta emerges naturally as the weight of that tilt. We present the derivation below.

Let q∈ΔNq \in \Delta_N denote a candidate expert-level allocation distribution (where qi≥0q_i \ge 0 and ∑iqi=1\sum_i q_i = 1), and recall that αi=ei/∑jej\alpha_i = e_i/\sum_j e_j is expert ii's retained spectral share ratio (defined in Eq. 2). We show that the normalized reference allocation, defined as qi∗:=ui/Mq^*_i := u_i/M (where M=∑iuiM=\sum_i u_i), is the unique solution to the following entropy-regularized allocation problem:

q∗=arg⁡max⁡q∈ΔN{H(q)+(1−β)∑iqilog⁡αi},H(q)=−∑iqilog⁡qi.q^* = \arg\max_{q\in\Delta_N} \lbrace H(q) + (1-\beta)\sum_i q_i \log \alpha_i \rbrace,\qquad H(q) = -\sum_i q_i \log q_i.

The objective is strictly concave on the simplex because H(q)H(q) has a negative second derivative (−1/qi<0-1/q_i < 0) and the second term is linear. Consequently, the maximizer q∗q^* is unique for any given β\beta. By introducing the Lagrangian L=H(q)+(1−β)∑iqilog⁡αi−ν(∑iqi−1)L = H(q) + (1-\beta)\sum_i q_i\log\alpha_i - \nu(\sum_i q_i - 1), the stationarity condition yields:

∂L∂qi=−log⁡qi−1+(1−β)log⁡αi−ν=0⟹qi∗=αi1−β e−1−ν.\frac{\partial L}{\partial q_i} = -\log q_i - 1 + (1-\beta)\log\alpha_i - \nu = 0 \quad\Longrightarrow\quad q^*_i = \alpha_i^{1-\beta}\,e^{-1-\nu}.

Since e−1−νe^{-1-\nu} is a constant independent of ii, we have qi∗∝αi1−βq^*_i \propto \alpha_i^{1-\beta}, which directly yields ui∝ei1−βu_i \propto e_i^{1-\beta} in Eq. (3). This objective has a clear physical interpretation: H(q)H(q) regularizes the allocation toward uniformity to prevent over-concentration, while ∑iqilog⁡αi\sum_i q_i \log \alpha_i rewards alignment with the spectral supply, with β\beta balancing their trade-off.

We have already added the motivation of uiu_i's form in the revised version.

(b) Why M is defined as it is

M:=∑iuiM := \sum_i u_i represents the total mass of the reference allocation uu across all experts. It is just a notation: it is defined by uu, not chosen alongside it. We define MM for the sake of conciseness during the derivation. Nothing depends on its value.

cc KL optimization (4) does not explain why the chosen reference allocation is optimal.

The KL optimization (now Eq. (5) in the revised text) does not claim that the reference allocation uu is optimal; rather, its arg⁡min⁡\arg\min is taken over tt with uu fixed. The reference allocation uu is governed by the hyperparameter β\beta, whose ideal value is pool-dependent. Specifically, how much delivered energy an expert needs depends on how far that domain remains from its standalone expert's performance. As shown in Figure 1c, the marginal return of additional budget varies across domains and saturates at different points, meaning an allocation optimal for one pool may not be optimal for another. Therefore, DESA fixes the parametric family of the allocation rule while selecting the scalar hyperparameter β\beta per pool on held-out validation data.

(d) Eq. (6): Why sallocs^{\mathrm{alloc}} is defined that way

Eq. (6) (now Eq. (7) in the revised text) is mathematically derived to realize the target expert energy ti⋆t^\star_i while preserving intra-expert spectral structure. Recall from Section 4.1–4.2 that sas_a denotes the aa-th retained singular value in the pooled spectrum, and Bi={a:block(a)=i}B_i = \{a : \mathrm{block}(a) = i\} denotes the subset of indices originating from expert ii. Specifically, once the target allocation t⋆t^\star is obtained, its realization on the retained singular values sas_a follows directly from two steps:

  1. Block-wise uniform scaling: DESA reallocates budget across experts without distorting the relative importance of singular directions within each expert. Therefore, all atoms a∈Bia \in B_i belonging to expert ii must be scaled by a single common factor ρi>0\rho_i > 0, i.e., saalloc=ρisas^{\mathrm{alloc}}_a = \rho_i s_a.

  2. Matching the target energy: In the shared orthogonal frame, the spectral energy is quadratic in the singular values. Setting the rescaled block energy of expert ii equal to its target allocation ti⋆t^\star_i gives: ∑a∈Bi(saalloc)2=∑a∈Bi(ρisa)2=ρi2∑a∈Bisa2=ρi2ei=ti⋆.\sum_{a \in B_i} (s^{\mathrm{alloc}}_a )^2 = \sum_{a \in B_i} (\rho_i s_a)^2 = \rho_i^2 \sum_{a \in B_i} s_a^2 = \rho_i^2 e_i = t^\star_i.

    Solving for ρi\rho_i uniquely yields ρi=ti⋆/ei\rho_i = \sqrt{t^\star_i / e_i}, which gives saalloc=tblock(a)⋆/eblock(a) sas_a^{\mathrm{alloc}} = \sqrt{t^{\star}_{\mathrm{block}(a)} / e_{\mathrm{block}(a)}} \thinspace s_a (as formalized in Proposition 4, Appendix A.5).

We have incorporated this explicit derivation into the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 7isx [2/3]

W2: Confusing terminologies of energies.

We thank the reviewer for this constructive feedback. We agree that while Appendix A.1 formally distinguishes these four quantities, introducing them more systematically in the main text significantly improves readability.

1. Disambiguating the four energy quantities. In the revised manuscript, we promote the energy accounting from Appendix A.1 into the main text as a compact summary table using our existing notation:

SymbolNormalized ShareNameDefined as
Ei=∥δi∥F2E_i = \|\delta_i\|_F^2—supplied energyexpert i's whole displacement, before truncation
ei=∑a∈Bisa2e_i = \sum_{a\in B_i} s_a^2αi=ei∑jej\alpha_i = \frac{e_i}{\sum_j e_j}retained spectral energythe part surviving rank-k truncation; ei=Ei efk(i)e_i = E_i\,\mathrm{ef}_k(i)
ti⋆t^\star_iti⋆/Mt^\star_i / Mallocated budgetDESA's target for expert i, Eqs. (4)–(5)
∥ci∥F2\|c_i\|_F^2sharei=∥ci∥F2∑j∥cj∥F2\mathrm{share}_i = \frac{\|c_i\|_F^2}{\sum_j \|c_j\|_F^2}delivered energyexpert-attributed energy actually present in Δm\Delta_m

2. Formalizing "retaining energy is also allocating budget" via equations. This statement in Section 3.2 summarizes Proposition 1 (Appendix A.2).

(1) Standard spectral merging (TSV) equates retained energy with delivered budget. Let ci=U~:,Bidiag(sBi)V~Bi,:c_i = \tilde{U}_{:,B_i}\mathrm{diag}(s_{B_i})\tilde{V}_{B_i,:} be expert ii's attributed contribution in the shared orthogonal frame (U~⊤U~=I,V~V~⊤=I\tilde{U}^\top\tilde{U}=I, \tilde{V}\tilde{V}^\top=I). Because orthonormal multiplication preserves the Frobenius norm, the delivered energy equals the retained spectral energy (∥ci∥F2=∑a∈Bisa2=ei\|c_i\|_F^2 = \sum_{a\in B_i} s_a^2 = e_i). Consequently, the delivered budget share is identical to the retained spectral share:

shareiTSV=∥ci∥F2∑j∥cj∥F2=ei∑jej=αi=Ei efk(i)∑jEj efk(j).\mathrm{share}_i^{\mathrm{TSV}} = \frac{\|c_i\|_F^2}{\sum_j \|c_j\|_F^2} = \frac{e_i}{\sum_j e_j} = \alpha_i = \frac{E_i\,\mathrm{ef}_k(i)}{\sum_j E_j\,\mathrm{ef}_k(j)}.

This identity is the exact mathematical content of our statement: deciding which singular directions to retain implicitly fixes what fraction of the merged update each expert delivers. Because this default split (αi\alpha_i) is dictated purely by each expert's raw update supply rather than domain-specific budget sensitivity (Figure 1c), it remains suboptimal (Table 4).

(2) DESA breaks this identity. By rescaling expert blocks via saalloc=tblock(a)⋆/eblock(a) sas_a^{\mathrm{alloc}} = \sqrt{t^\star_{\mathrm{block}(a)} / e_{\mathrm{block}(a)}}\,s_a followed by global norm restoration γ=∥s∥2/∥salloc∥2\gamma = \|s\|_2 / \|s^{\mathrm{alloc}}\|_2 (Proposition 4, Appendix A.5), DESA decouples delivery from supply:

shareiDESA=γ2ti⋆∑jγ2tj⋆=ti⋆∑jtj⋆≠αi.\mathrm{share}_i^{\mathrm{DESA}} = \frac{\gamma^2 t^\star_i}{\sum_j \gamma^2 t^\star_j} = \frac{t^\star_i}{\sum_j t^\star_j} \neq \alpha_i.

We have updated Section 3.2 to include these explicit identities.

AuthorsOpenReview ↗

Response to Reviewer 7isx [3/3]

W3: Unclear energy heterogenity analysis.

Thank you for the suggestion. In the revised Section 3.2, we now formally define the energy fraction upon its first mention:

efk(i):=eiEi=∑a∈Bisa2∥δi∥F2∈(0,1]\mathrm{ef}_k(i) := \frac{e_i}{E_i} = \frac{\sum_{a \in B_i} s_a^2}{\Vert\delta_i\Vert_F^2} \in (0,1]

which measures the share of expert ii's total update energy (Ei=∥δi∥F2E_i = \Vert\delta_i\Vert_F^2) retained in its top-kk singular directions (ei=∑a∈Bisa2e_i = \sum_{a \in B_i} s_a^2).

Furthermore, the phrase degree of concentration was intended as an informal description of efk(i)\mathrm{ef}_k(i). To eliminate ambiguity, we have replaced the phrase directly with efk(i)\mathrm{ef}_k(i) in the quoted sentence.

W4: Relationships between delivered energy and performance.

We thank the reviewer for this question. We address the relation between delivered energy and task performance, as well as its generality across domains, along two dimensions:

1. Recovery is a direct, positive affine transformation of task accuracy. For any domain ii with a fixed base model score SibaseS_i^{\mathrm{base}} and a fixed standalone expert score SiexpertS_i^{\mathrm{expert}} (Siexpert>SibaseS_i^{\mathrm{expert}} > S_i^{\mathrm{base}}), recovery is defined as:

= a_i\,S_i^{\mathrm{merged}} + b_i,\qquad \text{where } a_i = \frac{100}{S_i^{\mathrm{expert}} - S_i^{\mathrm{base}}} > 0$$ Because $S_i^{\mathrm{base}}$ and $S_i^{\mathrm{expert}}$ are fixed properties of the expert pool, $a_i$ and $b_i$ are strictly constants for a given domain. Consequently, within any domain, "delivered energy predicts recovery" and "delivered energy governs raw task accuracy" are **mathematically equivalent statements about the exact same measurement**, differing only by a constant per-axis linear rescaling. **2. The relationship holds consistently across domains and modalities.** Across the five language domains, our experiments yield a strong pooled Spearman correlation of $\rho = +0.915$ between delivered-share changes and recovered headroom (Figure 3). The four-domain vision-language pool exhibits the same strong positive relationship ($\rho = +0.852$), confirming that this pattern generalizes across diverse domains spanning both language and multimodal settings.
Reviewer_3NLKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the merging of domain-specialized experts obtained by RLVR. The authors identify two properties of RLVR task vectors that complicate existing merging approaches: apparent sparsity under low-precision representation and substantial heterogeneity in update energy across experts. Then they propose DESA, a training-free spectral merging method that explicitly reallocates expert-level delivered energy within a shared spectral frame. Numerical results show performance gain of the proposed method.

Critical strengths
  1. Formulating RLVR expert merging as a delivered-energy allocation problem is intuitive and can inspire future studies. It explains an important limitation of existing model merging methods.

  2. The paper proposes a simple yet effective method, DESA, which is both training-free and data-free, to resolve the issues. The theory part is convincing and the experiments are comprehensive.

  3. The comparison among TSV, uniform allocation, and DESA verify the effect of expert-level allocation. The fixed-budget analysis and renormalization ablation further support the paper’s central claim.

Critical weaknesses
  1. The evidence for apparent sparsity is verified on bf16 checkpoints, and a comparison with fp32 master weights would be better.

  2. The method currently relies on manually specified floors. An automatic way to estimate expert sensitivity would improve the practical applicability of the approach.

Questions for authors

NA

1 comment
AuthorsOpenReview ↗

Response to Reviewer 3NLK

We thank the reviewer for the careful reading and constructive comments, which we have incorporated into the revised manuscript. Below are our point-by-point responses.

W1: Apparent sparsity under bf16 vs. fp32.

Following your suggestions, we test the experts in our pool using fp32 master weights.

In FP32, the task vector has rare exact zeros, and the exactly-zero rate is around 2.9×10−52.9\times10^{-5}. In full precision, the RLVR task vector is dense. The sparsity therefore exists only under bf16, which is exactly why we call it apparent.

Reading of the task vectorExactly-zero coordinates
fp32 master weights2.9×10−52.9\times10^{-5}
bf16, as released72.4872.48 %

And bf16 is not our choice; it is how open-weight LLMs are mainly released. We surveyed the flagship releases of seven major open-weight families (Qwen, Llama, Mistral, DeepSeek, Gemma, Phi, and others). Of the 54 repositories that resolved: 47 BF16 (87.0 %), 3 FP16, 2 FP8, 2 INT8, and 0 in fp32 (0.0 %). Besides, for bf16 checkpoints, upcasting them to fp32 recovers nothing. Apparent sparsity is thus a property of the checkpoints that are actually available for merging, which is the regime the method targets.

W2: An automatic way to estimate expert sensitivity.

Thank you for this constructive suggestion. In this work, our primary contribution is to identify the apparent sparsity and update energy heterogeneity of RLVR task vectors, and to reframe RLVR expert merging as an explicit delivered-energy allocation problem in spectral space. Within this framework, the retention floor serves as a safeguard to protect vulnerable experts from performance collapse under aggressive equalization.

Currently, the floor values are selected on a validation set. As briefly touched upon in Appendix D (Limitations and Future Work), developing more adaptive allocation mechanisms that account for budget sensitivity is indeed a key avenue we envision. In the revised manuscript, we have made this point more explicit by directly noting the floors as a limitation and highlighting automatic expert-sensitivity estimation as a promising direction for future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.