- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DESA, a training-free spectral merging method for integrating multiple RLVR-trained experts. The authors identify two characteristics of RLVR task vectors: apparent sparsity under low-precision representations and heterogeneous update energy across experts. They argue that existing spectral merging methods implicitly allocate energy according to the retained spectral magnitude of each expert, which can lead to imbalanced capability recovery. To address this issue, DESA explicitly redistributes expert-level spectral energy through a power-law allocation rule and introduces retention floors to protect sensitive experts. Experiments on language and vision-language expert pools demonstrate improvements over several existing merging baselines.
Critical strengths
-
The paper highlights expert-level spectral energy allocation as an important factor in model merging.
-
DESA achieves competitive improvements across both language and vision-language RLVR expert pools without additional training.
Critical weaknesses
-
The claimed RLVR-specific characteristics are not sufficiently established, and similar magnitude imbalance and spectral calibration problems have been explored in prior work. This weakens the motivation and makes the novelty difficult to assess.
-
The proposed energy allocation function appears heuristic, and the hyperparameter selection protocol remains unclear. These issues raise concerns about the method's generalizability and practical applicability.
Questions for authors
-
The reported sparsity largely results from bf16 precision limitations. Could it be mitigated by higher-precision or longer training? Moreover, I think energy heterogeneity also exists in SFT experts. What makes these issues particularly specific to RLVR?
-
What motivates the specific choice of ? Is there any theoretical justification, or empirical evidence demonstrating its superiority over alternative allocation functions?
-
How are , , and selected in practice? Does this selection require downstream validation data?
-
Singular-value calibration and magnitude-aware merging have already been investigated in some related work like SVC [1]、SWIM-Merging [2] and DisTaC [3]. What fundamentally distinguishes DESA from these approaches? A direct comparison and explanation would help clarify the contribution.
[1]. When Shared Knowledge Hurts: Spectral Over-Accumulation in Model Merging (ICML26)
[2]. SWIM-Merging: Reweighting Singular Values of Weight Matrices for Efficient Test-Time Model Merging (Neurocomputing26)
[3]. DisTaC: Conditioning Task Vectors via Distillation for Robust Model Merging (CVPR26)
10 comments
Response to Reviewer dZ99 [2/3]
W2: Motivation of
We derive the form of from an entropy-regularized allocation problem. Specifically, maximizing entropy under a linear tilt toward supplied energy. In this formulation, the exponent emerges naturally as the weight of that tilt. We present the derivation below.
Let denote a candidate expert-level allocation distribution (where and ), and recall that is expert 's retained spectral share ratio (defined in Eq. 2). We show that the normalized reference allocation, defined as (where ), is the unique solution to the following entropy-regularized allocation problem:
The objective is strictly concave on the simplex because has a negative second derivative () and the second term is linear. Consequently, the maximizer is unique for any given . By introducing the Lagrangian , the stationarity condition yields:
Since is a constant independent of , we have , which directly yields in Eq. (3). This objective has a clear physical interpretation: regularizes the allocation toward uniformity to prevent over-concentration, while rewards alignment with the spectral supply, with balancing their trade-off.
We have already added the motivation of 's form in the revised version.
W3: Hyperparameter choice.
The allocation parameters are selected using a small validation set. This practice aligns with the standard protocol across the model merging literature, such as Task Arithmetic, TIES-Merging, and TSV, which similarly rely on held-out validation splits to tune merging hyperparameters. In our setup, hyperparameters are chosen based on the validation macro-average. We have made this protocol explicit in the revised manuscript.
Moreover, we note that DESA does not rely on brittle hyperparameter tuning: as shown in Figure 2 and Table 9 (Appendix C.1), performance remains stable across a broad range of , , and . Even a hyperparameter-free uniform allocation already outperforms standard TSV (Table 4).
Response to Reviewer dZ99 [3/3]
W4: What distinguishes DESA from SVC, SWIM-Merging, and DisTaC?
We thank the reviewer for pointing out these works. While all four methods involve magnitude or singular-value rescaling, they target different training regimes and rescale fundamentally different mathematical objects.
SVC, SWIM-Merging, and DisTaC all target models trained via standard supervised learning across vision and language benchmarks (e.g., ViT, Flan-T5, and SFT-trained LLMs). In contrast, DESA is built for the RLVR post-training regime, where task vectors are tiny, nearly orthogonal (pairwise cosine ), and highly heterogeneous in energy ( spread). Here, the primary bottleneck is how unevenly standard spectral operators divide the merge budget among experts.
| Method | Role | Object rescaled | Granularity | Needs training |
|---|---|---|---|---|
| SVC | plug-in | singular values inflated by over-accumulation of shared directions | direction | No |
| SWIM-Merging | plug-in | singular values, learned by entropy minimization | direction | Yes (gradient descent) |
| DisTaC | plug-in | the norm of each task vector | whole task vector | Yes (distillation) |
| DESA | standalone operator | expert-block delivered energy in the pooled spectral frame | expert block | No |
Specifically, three substantive distinctions separate DESA from these approaches:
-
Expert Attribution vs. Post-Summation Scaling (vs. SVC & SWIM): SVC and SWIM apply SVD to the already summed update (). Conversely, DESA decomposes each expert separately and preserves explicit expert blocks () in the pooled orthogonal frame, directly enabling expert-level budget control.
-
Target Objective: Shared Directions vs. Budget Imbalance (vs. SVC): SVC calibrates singular values inflated by shared-direction over-accumulation, whereas DESA resolves inter-expert budget imbalance caused by RLVR energy heterogeneity ( spread) and safeguards vulnerable experts via retention floors.
-
Resource Assumptions (vs. SWIM & DisTaC): Unlike SWIM (requiring test-time gradient descent) and DisTaC (requiring data-driven distillation), DESA and SVC are strictly one-shot and training-free. We thus focus our empirical comparison on SVC, which shares our setting.
We evaluated the official SVC implementation on our 5-expert language pool across three base operators:
| base operator | without SVC | with SVC | |
|---|---|---|---|
| Task Arithmetic | 62.13 | 67.98 | +5.85 |
| TIES | 65.66 | 67.49 | +1.83 |
| TSV | 66.57 | 67.09 | +0.52 |
| DESA | 68.62 | - | - |
Operating on the exact same spectral frame as TSV, post-hoc direction calibration via SVC yields (), whereas reallocating expert-level delivered energy via DESA achieves (), outperforming all baseline+SVC combinations.
We have cited and discussed all three works in the revised manuscript.
Response to Reviewer dZ99 [1/3]
W1 & Q1: Apparent sparsity in bf16.
(a): Under fp32 there is no apparent sparsity
Following your suggestions, we test the experts in our pool using fp32 master weights.
In FP32, the task vector has rare exact zeros, and the exact-zero rate is around 2.9×10−5. In full precision, the RLVR task vector is dense. The sparsity therefore exists only under bf16, which is exactly why we call it apparent.
And bf16 is not our choice; it is how open-weight LLMs are mainly released. We surveyed the flagship releases of seven major open-weight families (Qwen, Llama, Mistral, DeepSeek, Gemma, Phi, and others). Of the 54 repositories that resolved: 47 BF16 (87.0 %), 3 FP16, 2 FP8, 2 INT8, and 0 in fp32 (0.0 %). Besides, for bf16 checkpoints, upcasting them to fp32 recovers nothing. Apparent sparsity is thus a property of the checkpoints that are actually available for merging, which is the regime the method targets.
(b) Longer training reduces apparent sparsity only marginally, and is not a practical lever for model merging.
We thank the reviewer for this insightful question. Apparent sparsity does decrease as training progresses. However, our empirical measurements show that this decrease is slight and saturates quickly, leaving task vectors firmly in the high-sparsity regime.
Tracking the exact bf16 zero fraction across all weight tensors on our vision-language pool yields a clear flattening trajectory.
The rate of sparsity reduction decelerates monotonically. Moreover, training length is not a viable lever in practice: (1) RLVR steps are set by task convergence, and over-training incurs heavy compute costs and overfitting risks; and (2) model merging typically consolidates off-the-shelf, released checkpoints, where retroactively extending training is impossible.
c What makes energy heterogeneity particularly specific to RLVR?
We thank the reviewer for this question. While energy variations exist in SFT, relative energy heterogeneity is substantially more pronounced in RLVR primarily due to the vast difference in their update scales:
As shown in Figure 1a, SFT task vectors have a much larger overall displacement, over two orders of magnitude larger than RLVR updates (116× larger in ∥δi∥F). Because SFT operates at such a large baseline scale, variations across tasks result in only a small relative ratio. For example, across standard SFT experts fine-tuned from the same Qwen2.5-7B base (e.g., Math 108.4, Medical 114.8, Table 96.2, Tool 91.5), their Frobenius norms differ by a factor of only 1.25× (an energy Ei ratio of 1.57×), yielding nearly balanced TSV delivered shares.
In contrast, because RLVR updates are extremely small in absolute scale (∥δi∥F≈0.5-1.4), even modest differences in update magnitude translate into a large relative disparity across experts. Consequently, our RLVR experts differ by 2.65× in norm and 7.02× in energy Ei, which spectral truncation further amplifies into an 8.8× gap in delivered share (4.51% vs. 39.62%).