- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper builds on ACTMat, a data-free RegMean variant that uses task-vector Gram matrices as a proxy for input covariances. It adds a per-layer "conflict score": the ACTMat loss at its closed-form optimum divided by the loss of the trivial zero merge, so the score lies between 0 and 1. After standardizing the scores across layers, the score sets either (i) a per-layer Ridge damping strength that grows exponentially with the layer's conflict, or (ii) a per-layer step size for a fixed-budget Muon-style solver initialized at the mean task vector.
Critical strengths
Careful attribution of where the gain comes from. Table 1 reports the regularization effect (U vs. ACTMat) and the conflict effect () separately. Table 9 then holds the solver, damping law and pool fixed and changes only the per-layer score. The permutation control (20 draws per cell, with 0 of 20 reaching the true assignment in all four cells, App. A) shows that which layer gets which damping matters, not only the spread of . The replication on independent experts (App. G) found that the task-vector-norm alternative is unstable, and the authors removed a recommendation as a result. This matters because many merging papers give a single headline number; here the reader can see what is actually being claimed.
Critical weaknesses
-
The contribution is narrow, the tuning comparison is uneven, and the dynamic part adds little. The method is a per-layer scaling of a Tikhonov term on ACTMat; ACE-Merging already damps a closed form of this objective with a single constant. The gains hold only for fully fine-tuned CLIP ViT-B and vanish on ViT-L/14 (Table 1), LoRA (Table 10) and T5 (Table 18), as the paper itself notes (L451-452). The comparisons behind "conflict is the signal" favour the conflict variant: the uniform-Ridge baseline reuses the strength tuned for the conflict schedule, the alternative descriptors in Table 2 reuse the conflict-tuned damping law, and external baselines run at published defaults (e.g., Task Arithmetic at 49.04% on B/16, ). In the dynamic setting, the only comparison is the authors' own MASS port, which sometimes falls below the static merge (ViT-L/14 : 91.6 vs. 94.2, Table 3), and a simple "store the MLP projections" rule matches the conflict ranking (Table 15).
-
The claims are systematically overstated, and the presentation is careless. The abstract reports "statistically significant gains when merging 14 and 20 tasks" on ViT-B/32, but Table 8 gives Holm-adjusted (Ridge) and (Muon) at 14 tasks. Figure 1 headlines "More than 2× the uniform ridge's gain" using only the 8-task ViT-B Ridge cells, whose own Table 8 p-values (0.052 and 0.336) are not significant, while the same Table 1 shows no gain on ViT-L/14 and a negative Muon at B/16, ; the figure also does not say whether its conflict heatmap and bars (panels 2-3) are real data or schematic. The figures are not of submission quality: in Fig. 3 (left) the legend sits on top of the MLP-in curve, the panel carrying the main evidence, and the right panel's y-axis label collides with the left panel. The text has many errors ("ours MASS baseline" in the abstract, "haven performance" L97, "We minimizes" L233, "didn't compared" L158-159, a sign error at L370-371), and the appendix is largely narrative. Together with Weakness 1, the paper overstates what the evidence supports and is not ready for publication.
Questions for authors
- Was the uniform-Ridge (and the uniform-Muon ) selected on its own validation set with , or reused from the conflict variant's joint selection? Please report the best uniform on a finer grid, selected on validation and evaluated on test, for all ViT-B cells. If the gain shrinks below about 0.5 points, my assessment of the main claim changes.
- What is the wall-clock and memory cost of the method compared with ACTMat and TSV, in particular of 500 Muon steps per layer and of the per-input factor-space re-merge (App. J)? The paper mentions per-job wall-clock in its manifest but reports no numbers.
- Are panels 2-3 of Figure 1 (per-layer conflict heatmap and bars from to ) computed from the actual ViT-B experts, or schematic? If real, which backbone and task set; if schematic, please label them as such.
3 comments
Thank you for the time and effort you have dedicated to reviewing our paper. As the discussion period is ending soon, we kindly ask you to look at the new results and clarifications in our rebuttal. We believe they address your main concerns, and we hope you will take them into account in your final assessment. We have also updated the manuscript to reflect these changes. If any concerns remain, we would be happy to provide further explanations.
Follow-up on the revised comparisons and claims
Thank you for the detailed response and revision. The revised statistical claims, figure provenance, and total-cost reporting also resolve the corresponding issues in my review.
The revised controls address several important concerns. Although I still find the contribution limited, I will raise my rating from 1 to 2.
Response to reviewer: Additional experiments and clarifications
We thank the reviewer for the detailed assessment and for recognizing our careful attribution of the gains. Conflict-guided Ridge outperforms independently tuned uniform Ridge by +0.77 to +1.15 points across all six ViT-B settings. All six point estimates exceed your 0.5-point threshold, and four gains are significant (Holm-adjusted p < 0.05).
Q1/W1: Uniform Ridge. Uniform Ridge tunes on validation (18 values, 1e-5–1e-1 at 8 tasks; 14 values, 1e-4–1e-1 at 14/20), then tests. Conflict Ridge remains fixed (α=5e-4, κ=1).
C-U gains (original in brackets). Holm p (B/16): .040, .009, <.001; (B/32): .051, .051, .016. Tasks won: B/16 7/8, 11/14, 17/20; B/32 6/8, 12/14, 16/20. Largest drop: 0.45 (B/32, 14 tasks); all three B/16 gains held or rose. Muon moves to the appendix as a secondary solver.
W1: Descriptor controls. Each descriptor (conflict, type, depth, width, update norm) tunes κ, α over 18 settings (κ ∈ {−1, −0.5, 0.25, 0.5, 1, 2}, α ∈ {2.5e-4, 5e-4, 1e-3}). Conflict leads all six ViT-B settings (+0.20 to +0.55); five nominally significant, none after Holm (min adjusted p=.071). It beats all 20 score shuffles per setting (p=.048, minimum with 20 draws).
W1: Tuned baselines (TA was 49.04%). Each baseline tunes on its published validation grid: TA 0.1–1.0, TIES 0.8–2.5, TSV-M/Iso-C 0.25–3.0, WUDI four variants, ACE eps 1e-7–1e-2. TA rises 49.0→70.6 (B/16, 14), still 14 points below ACTMat. Fixed conflict Ridge beats all tuned baselines except Iso-C at B/16, 8 tasks (+1.55 vs. +1.52 over ACTMat).
W1: Beyond ACTMat. Conflict guidance complements ACE's tuned global damping, improving ACE without re-tuning in all six ViT-B settings (+0.64 to +1.06). Applied to each method's tuned baseline, it also improves Iso-C (+2.04; 6/6, p<.0001) and TSV-M (+0.44; p=.039). Shuffling removes the gain, so the gain depends on which layers are damped. RegMean (real input data) does not benefit (−0.9 to −1.1); we will report this.
W1: Generalization limits. Claims focus on fully fine-tuned ViT-B. On ViT-L/14 (8 tasks), a label-fitted schedule gains +0.06 over uniform (CI −0.05 to +0.17), vs. +0.03 for conflict; Whisper-small is similar. LoRA conflict is near zero (mean 0.0004 vs. 0.167 for full fine-tuning). No significant aggregate gain outside ViT-B: ViT-L/14 +0.07 to +0.13 (3 settings); Whisper mean +0.23 (CI −0.07 to +0.54, 18 settings); T5 mean +0.21 (CI −0.16 to +0.58, 10 settings). See Reviewer vj1z (W4).
W1: Dynamic merging. We remove our MASS reimplementation/comparisons. The 91.6 vs. 94.2 gap (L/14, 8) mixes true-head static and routed-head dynamic; official MASS reproduces 90.60 vs. 90.6 (B/16, 8). Whole-layer selection matches random/MLP (Table 15). At 5–10% of MASS storage, within-layer conflict selection beats random by 2.4–3.1 points and all 10 draws (p=.09, minimum with 10), matching MASS truncation; we report this in the appendix.
W2: Claims/presentation. We drop abstract 14-task significance (Holm p=.080 for 18 printed tests; .051 vs. tuned uniform; .111 came from 27 tests including an unprinted row). Figure 1 replaces “more than 2×” with Ridge gains vs. tuned uniform in all nine ViT settings (+0.77 to +1.15 on ViT-B, +0.07 to +0.13 on ViT-L/14). We fix Figure 3 labels, typos/sign wording, and appendix tables.
Q2: Cost. Merging is a one-time offline step; inference cost is unchanged.
Reported merge-only times are medians of 3 idle-GPU runs. Ridge matches ACTMat peak memory (±0.02 GB): 0.7–1.8 GB (B/16), 1.3–3.2 GB (L/14) for 8–20 tasks. The one-off score computation takes ~11 s (B/16), ~60 s (L/14). The table gives Muon at ~2.1× and ~1.8× the Ridge times, respectively. MASS per-input re-merge (App. J) is removed.
Q3: Figure 1 provenance. Panels 2–3 show real data, not schematic: panel 2 plots r_ℓ for B/16 attention/MLP matrices over 8 tasks (12 blocks × 4); panel 3 plots 10^(κz_ℓ) for 12 MLP-input projections, ×0.04 (block 11) to ×100 (block 3). Panel 4 plots uniform/conflict Ridge gains vs. ACTMat at 8 tasks. Caption will clarify.
We thank the reviewer again; We believe the changes following the rebuttal have helped us improve the paper (which we are still updating), and we hope these improvements are reflected in your final assessment.