acceptodds
Under review as a conference paper at ICLR 2027

Weight-Space Graph Signal Processing for Multi-Resolution Model Merging

Abstract

Model merging often suffers from destructive interference arising from conflicting parameter updates across tasks. Prior work has proposed various methods in Euclidean parameter space to mitigate this issue. However, it remains unclear how to identify which components of the parameters cause interference and which can be safely shared. To address this, we study model merging with graph signal processing and propose a novel model merging method, Spectral Graph Merging (SGM), based on a multi-resolution merging process. By viewing weight matrices as bipartite neuron-interaction graphs, we apply graph wavelet transformation to decompose the task vectors into low-frequency components capturing shared global structure and higher-frequency components associated with task-specific conflicts. SGM then reduces the interference in the high-frequency components while aggregating the low-frequency components. Evaluation results on MergeBench and OptMerge benchmarks demonstrate that SGM outperforms state-of-the-art merging methods on both benchmarks, while maintaining robust reasoning and safety under highly conflicting task combinations. Our results also indicate that exploiting spectral structure in weight space provides a principled and effective mechanism for mitigating destructive interference in large-scale model merging.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Model Merging, Large Language Models, Weight Space Learning, Graph Signal Processing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_fvUEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Spectral Graph Merging (SGM), a model-merging method that operates in a graph-spectral domain. For each layer, the delta matrix is lifted to a symmetric bipartite block matrix X_i. Then a shared graph is constructed from the adjacency matrix of the row-normalized mean task vector. Each X_i is decomposed into J frequency bands through a Chebyshev-approximated spectral graph wavelet filter bank. Afterward, the lowest band (j = 0) is aggregated by dense summation, instead, the higher bands (j > 0) are fused with a TIES-style sign election, followed by quantile-based soft-thresholding with a band-dependent sparsity level ρ_j. The fused bands are mapped back to weight space and rescaled to produce the final merged model, that is evaluated on the MergeBench and OptMerge benchmarks.

Critical strengths
  1. Fresh perspective and clear presentation. The manuscript is well written and easy to follow. Treating a layer’s task update as a signal on the neuron-interaction bipartite graph, and performing multi-resolution analysis in that domain, is an interesting idea for model merging. It also links the merging literature to graph signal processing (GSP) and weight-space learning.
  2. Informative ablations. The paper reports a broad set of ablations, covering graph support, band selection, fusion rule and energy rescaling. These help clarify the contribution of each component and support the narrative.
  3. Under-explored direction. Mitigating task interference with tools from GSP appears to be largely unexplored, and this work offers a careful study of it.
Critical weaknesses
  1. Relation to SVD-based methods. My main concern is that spectral graph wavelet decompositions are closely related to SVD-based decompositions, especially on bipartite graphs. This places SGM close to shared-subspace SVD methods such as Iso-C/Iso-CTS, TSV and KnOTS. From the current analysis and evaluation, I cannot determine whether SGM is a genuinely different approach or a reformulation of existing subspace methods in different notation. A more explicit theoretical and empirical comparison with these methods would substantially strengthen the paper.
  2. Inconsistent baseline coverage across benchmarks. The two benchmarks are evaluated against different sets of methods. Table 2 (OptMerge) is considerably more exhaustive, while Table 1 (MergeBench) omits several relevant baselines, including the SVD-based methods most closely related to SGM. The reason for this asymmetry is not explained. Evaluating a consistent and broader set of baselines on both benchmarks would make the empirical claims more convincing.
  3. Possibly misconfigured baseline. In Table 2, Iso-C performs dramatically worse than all other methods. This suggests a possible implementation issue or insufficient hyperparameter tuning. Please double-check this baseline.
  4. Readability of Figure 2. Figure 2 appears almost flat, and the relevant differences are only visible in the small inset. A logarithmic scale for the normalized energy, or a different normalization, would make the separation between models visible at a glance.
  5. Limited evaluation settings. Model-merging papers commonly evaluate on vision benchmarks (e.g., ViT models across 8/14/20 tasks) and NLP benchmarks (e.g., GLUE), in addition to LLM benchmarks. The present evaluation covers only the large (vision-)language model setting. Results in the standard vision and GLUE settings would complement the analysis and ease comparison with prior work.
Questions for authors
  1. Configuration consistency.
    • Which configuration (fixed or adaptive J) produced the results in Tables 1 and 2?
    • Why does the baseline in Figure 8 appear to differ from the full-SGM result in Table 4?
    • Which model and setting produced the results in Table 6?
  2. Hyperparameter selection.
    • What held-out data was used to select the hyperparameters (e.g., α and ζ)?
    • Did the baselines receive an equivalent tuning budget?
    • How were TSV, Iso-C, CART and KnOTS tuned on OptMerge?
    • Is the sharp performance drop of Iso-C due to an implementation error or a configuration issue?
  3. Citations and typos. I noticed several incorrect citation attributions (e.g., DARE and RegMean are attributed to the wrong works), as well as a number of typos and editing leftovers. Please carefully proofread the manuscript and verify all references for the camera-ready version.
Reviewer_3vauOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SGM represents task updates as signals on a shared bipartite graph and decomposes them using a graph-wavelet filter bank. It applies different fusion rules to the approximation and detail bands, then reconstructs and rescales the merged update. The aim is to reduce task interference through topology-aware, multi-resolution processing while producing a static merged model.

Critical strengths

The reported results are promising. In Table 1, SGM improves the best baseline average from 38.1 to 41.3 on Gemma-2-2B and from 61.7 to 65.2 on Llama-3.2-3B-Instruct. These are meaningful improvements in the evaluated settings. The paper also includes useful controls for the graph construction, band decomposition, fusion rule, and energy rescaling. The supplementary code makes several implementation choices inspectable, including the adaptive band count and the sign-preserving soft-thresholding operation. The combination of weight-space topology and spectral fusion is worth investigating.

Critical weaknesses

My main concern is that the spectral method described in the paper does not yet correspond consistently to the supplied implementation. The paper defines the Laplacian as L=I−D−1/2∣A∣D−1/2L = I - D^{-1/2}|A|D^{-1/2}, whereas the code retains signed adjacency entries and uses absolute values only to compute the degrees. These are different graph operators. The detail kernels also have unnormalized centers at 2−j2^{-j}, moving toward lower graph frequencies as the band index increases, while Figure 2 and the "highest-frequency" ablation use the opposite interpretation. Finally, the coefficient calculation already assigns the constant Chebyshev term a 1/M1/M factor, but the filtering routine halves that term again. This appears to change the intended filter responses and undermines the stated tight-frame normalization. Together, these issues make it difficult to assess the proposed spectral explanation.

The experimental procedure also needs a clearer connection to the provided code. Its default path uses adaptive band counts, while Appendix A.6.1 describes the primary method as using static defaults. The supplied command specifies a scaling coefficient of α=0.4\alpha = 0.4, sparsity of ζ=0.1\zeta = 0.1, and order M=20M = 20, but it does not document the held-out calibration described in Appendix A.11.2. Fixed settings at execution time do not establish that their selection was data-free. The paper should identify the implementation and selection procedure behind the reported results. The discrepancies above do not by themselves establish that the benchmark numbers are incorrect.

Questions for authors

Which Laplacian, filter-coefficient convention, and band ordering were used to produce the reported results? Please reconcile the code with the paper, particularly the additional halving of the constant Chebyshev term and the interpretation of the last detail band as the highest-frequency band. Does the implemented filter bank satisfy the stated normalization?

For the main results, were band counts fixed or adaptive? What held-out data and selection procedure were used for the global settings, and how were the comparison methods configured?

Reviewer_AjtDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses genuine interference in model merging and proposes an interesting graph-wavelet representation. Results on language and vision-language benchmarks are encouraging, and the component ablations are useful. However, the claimed data-free protocol conflicts with the calibration procedure in the appendix, the interpretation of spectral variation as interference is insufficiently justified, and mathematical and computational descriptions contain consequential inconsistencies.

Critical strengths
  1. The problem and proposed intervention are clearly connected. Treating different components of task updates differently is a reasonable way to address destructive interference. The graph representation offers a structured alternative to an arbitrary coordinate-grid Fourier basis.

  2. The method produces a deployable static model. Once merging is complete, no router, task identification, or extra inference parameters are required.

  3. The paper includes relevant benchmarks and component controls. Table 4 tests graph support, band selection, fusion, and rescaling. Table 1 reports substantial average improvements on two representative language configurations, and Table 2 extends evaluation to a multimodal model.

Critical weaknesses
  1. The data-free/tuning-free characterization conflicts with the experimental protocol. The main text and Appendix A.6.1 describe zero-data or tuning-free merging, but Appendix A.11.2 states that the scaling coefficient and sparsity were selected using held-out calibration data. This matters for fairness against baselines with different tuning access. The data source, size, selection criterion, transfer across models, and baseline tuning budgets must be specified. A one-time calibration procedure still needs to be included in the stated assumptions.

  2. Spectral variation is not sufficient evidence of destructive interference. Figure 2 and Appendix A.8 measure energy differences or cross-task variance, but specialized useful updates can also have high variance. The analysis does not directly establish that high-frequency bands cause more task loss, or that low-frequency components can safely be summed. Table 4 supports the complete design, but does not fully separate the graph basis from its sign-consensus, sparsity, and scaling rules. A matched fusion comparison across representations would strengthen the central mechanism claim.

  3. Implementation inconsistencies and uncertainty weaken confidence in the results. Eq. (3) multiplies the thresholded magnitude by the original coefficient, while Algorithm 1 uses its sign; these yield different shrinkage rules. The stated O(M∣E∣)O(M|E|) cost also omits the multiple signal columns in filtering the matrix XiX_i, unless an additional implementation strategy is used. Finally, the OptMerge advantage is 63.72 versus 63.30, without uncertainty estimates. The implemented formula, actual matrix-filtering cost, and stability of this small gain require clarification.

Questions for authors
  1. Calibration: Which datasets and splits were used to choose the scaling and sparsity parameters, and were these choices fixed before evaluating every reported model? Did competing baselines receive comparable tuning access?

  2. Implemented method: Does the code use Eq. (3) or Algorithm 1's sign-based shrinkage? Which main results use fixed versus adaptive band counts, and what exact settings produced Table 4?

  3. Cost and stability: How is the full matrix signal filtered, what is the corresponding arithmetic and peak-memory cost, and are paired uncertainty estimates or repeated-run results already available for the 0.42-point OptMerge improvement?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.