- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes ReForge, a post-hoc refinement method for model merging. For each linear layer, it solves a ridge regression problem regularized toward an existing merged model (the anchor), using either calibration activations (ReForge-DA) or normalized task-vector Gram matrices (ReForge-DF). On top of this closed-form inner update, an outer Bayesian optimization loop tunes block- and group-wise regularization weights and scaling factors on a validation set. Experiments on ViT (up to 20 tasks) and Llama-3B/8B (5 tasks) show consistent gains over seven anchor baselines.
Critical strengths
Treating existing merged models as priors in a closed-form ridge regression step is clean and practical. It connects activation-based merging like RegMean, weight-Gram merging like WUDI, and existing subspace anchors under one framework. I also appreciate the control experiments in the appendix (replacing the task-vector Gram with random PSD or identity matrices, and testing pure anchor rescaling under the same BO budget), which clearly show the contribution of each component.
Critical weaknesses
ReForge-DF replaces iterative WUDI optimization with a ridge-regularized closed-form solve. It would be helpful to cite and discuss its connection to SWUDI (Closed-Form Spectral Regularization for Multi-Task Model Merging), which also studies the ill-posed WUDI normal equation and solves it in closed form via spectral filtering. In addition, the appendix ablations show that on strong anchors like ISO-CTS, rescaling the anchor via BO already provides a large portion of the gain, while the task-vector Gram adds a smaller margin (0.4% to 1.0%). Discussing this breakdown in the main text would give readers a clearer picture.
Since the outer BO loop evaluates candidates on a labeled validation set across all tasks, the term "data-free" for ReForge-DF (and 1-shot or 5-shot in the few-shot experiments) may confuse some readers, and it would help to clarify how validation data availability differs from calibration data in practice. Moreover, on weaker anchors (Pretrained, TA, TIES, RegMean) and on Llama-3.2-3B, ReForge-DF outperforms activation-based ReForge-DA.
Questions for authors
Could you discuss how the ridge regularization in ReForge-DF relates to the closed-form spectral filtering in SWUDI?
Why does ReForge-DF outperform ReForge-DA on weaker anchors and on Llama-3.2-3B, and could the validation inputs used in BO also serve as unlabeled calibration samples for the inner loop?
Could you briefly explain the choice of block numbers (B=1 for Llama vs. B=3 for ViTs), and why 20 BO trials on Llama-3.2-3B perform slightly below the initial TSV anchor?
5 comments
Response to Reviewer v986 (2/2)
(2/2)
(a) What the names mean. The new names only indicate whether calibration activations are used. Both variants select hyperparameters on labeled validation data, as do tuned baselines such as TSV and ISO-CTS, and "1-shot" and "5-shot" refer only to the number of unlabeled calibration examples per task. The abstract and introduction now state this, and Appendix J (Table 19) lists the validation-set sizes. The dependence on validation data is mild: on 20-task ViT-B/32, using only 10% of the validation set gives relatively stable results (Figure 3).
(b) Validation vs. calibration data in practice. A validation set only has to score a candidate model; in our Llama experiments it consists of small benchmark splits scored by task metrics such as IFEval prompt accuracy, GSM8k exact match and MBPP pass@1. A calibration set instead supplies unlabeled inputs that represent the experts' task domains, whose activations define the regression in every linear layer. A setting may therefore have a scored validation benchmark but no suitable inputs from the experts' domains, or the reverse; ReForge-CF requires only the former.
Revision: the variants are renamed ReForge-CF and ReForge-CA throughout the paper; the abstract and introduction state that both use labeled validation data for hyperparameter selection; Appendix J (Table 19) lists the validation-set sizes.
Why does ReForge-CF outperform ReForge-CA on weak anchors (Q2)? The same pattern appears in our new matched comparison: ReForge-CA is best on the strong anchors (ISO-CTS, TSV, WUDI), and ReForge-CF on the weak ones (RegMean, TA, TIES) (Table 4). A possible explanation is that a weak anchor is far from the experts and needs corrections in many input directions. With 128 calibration images per task, the activation Gram concentrates its energy in a few directions (on 20-task ViT-B/32, the top 10% of directions hold about 73% of its energy, versus 46% for the task-vector Gram), so it carries little information about the remaining directions, whereas the task-vector Gram spreads its information more evenly. In practice, the two variants are complementary: ReForge-CF suits weak anchors, while ReForge-CA suits strong ones.
Can validation inputs serve as calibration samples (Q2)? Yes. We used the unlabeled validation inputs as the calibration samples of ReForge-CA (Appendix F, Table 15). Test accuracy (%) of ReForge-CA on ViT-B/32, mean over five seeds:
| Anchor | Calibration set | 8 tasks | 14 tasks | 20 tasks |
|---|---|---|---|---|
| TA | separate (paper) | 84.98 | 79.20 | 75.54 |
| TA | validation inputs | 84.89 | 79.32 | 75.67 |
| TSV | separate (paper) | 88.90 | 84.33 | 82.01 |
| TSV | validation inputs | 89.03 | 84.35 | 82.23 |
| ISO-CTS | separate (paper) | 90.15 | 85.49 | 82.77 |
| ISO-CTS | validation inputs | 90.14 | 85.45 | 82.85 |
In all nine settings the two choices differ by less than 0.25 points, so ReForge-CA can reuse the validation set that hyperparameter selection already requires instead of a separate calibration set.
Revision: new Appendix F (Table 15) and a sentence in Section 5.4.
Block numbers (Q3). For ViT, B = 3 (15 search dimensions) was a design choice, and later results suggest that more hyperparameters can help further: the 24-dimensional ReForge of our new matched comparison is more accurate than the 15-dimensional one in Table 1, e.g., 83.3% versus 81.5% (ReForge-CF) on 20-task ViT-B/32 with ISO-CTS (Section 5.3, Table 4; response to Reviewer jmsx, W1(b)). For Llama, each validation evaluation is far more expensive, so we limit the search to K = 100 evaluations; to search sufficiently within this budget, we use B = 1, which gives 5 dimensions.
K = 20 on Llama-3.2-3B (Q3). With K = 20, ReForge reaches 0.465, slightly below the TSV anchor's 0.471 (Table 18; Table 10 of the original submission). Twenty evaluations in a five-dimensional space leave the search still exploring, and the configuration is selected on validation data, which does not guarantee a gain on the test set. For this reason, our main Llama experiments use a larger budget of K = 100 (Table 2).
Dear Reviewer v986, thank you again for your feedback. We would appreciate hearing whether our response and the revised paper have addressed your main concerns. If anything remains unclear, we would be happy to provide further clarification.
Response to Reviewer v986 (1/2)
(1/2)
Thank you for the positive assessment and the precise questions. We now cite SWUDI and discuss it as concurrent work.
Relation to SWUDI (Q1).
(a) How Eq. (14) relates to SWUDI. For one linear module, let U(t)=W(t)−Wpre be the task vector of expert t, U(0) the task vector of the anchor, and Bt=U(t)⊤U(t)/∥U(t)∥F2 the unit-trace Gram of U(t) (Eq. 13). Let C=∑tBt and D=∑tU(t)Bt, and let C=∑kμkvkvk⊤ be the eigendecomposition of C. The general estimator of SWUDI (its Eq. 11) is
τ^=τinit+(D−τinitC)Ch†,
where τ^ is the merged task vector, τinit is a base point, and Ch†=∑k(hk/μk)vkvk⊤ applies a filter hk to each nonzero eigenvalue μk of C. Choosing the Tikhonov filter hk=μk/(μk+λ~) with regularization strength λ~>0 and the anchor as the base point, τinit=U(0), gives
τ^=(D+λ~U(0))(C+λ~I)−1,
which is our Eq. (14); the two also agree on the null space of C, where D vanishes and both keep the anchor component. The methods differ in what they design. SWUDI designs the filter, using a soft exponential filter with hard spectral truncation around a fixed base point. ReForge keeps the Tikhonov filter, uses any existing merged model as the base point, selects the regularization strength of each module group on validation data, and also has an activation-based variant, ReForge-CA. We derived this closed form independently: our preprint appeared on arXiv about three weeks before SWUDI, and we give the dates and the link to the Area Chair in a confidential comment (Section 4.3 and Appendix E).
(b) SWUDI as an anchor. Because ReForge accepts any merged model as its anchor, we ran SWUDI and its adaptive variant ASWUDI with the authors' code (unmodified) on the same fine-tuned experts as in all our experiments and applied ReForge on top (ViT-B/32, five seeds; change relative to the anchor in parentheses; Table 14):
ReForge-CA improves both anchors by 1.8 to 4.0 points. ReForge-CF adds at most 0.5 points, as expected because it lies in the same filtering family as SWUDI.
Revision: Section 4.3 now states this relation and cites SWUDI as concurrent work; Appendix E gives the derivation and the SWUDI-anchor results (Table 14).
Rescaling breakdown. We agree and now report this breakdown in the main text (Section 5.3, Table 4). We compare ReForge with a rescaling control that uses the same 24 parameters and 200 BO evaluations but spends all of them on rescaling (see our response to Reviewer jmsx, W1(b)). The gain has two parts, and both come from ReForge. First, searching module-specific hyperparameters with BO is itself part of our design: rescaling alone raises the 20-task ISO-CTS anchor from 77.6% to 81.8%, and BO also outperforms a shared setting or random search (Table 3). Second, the regression adds a further gain on top of it: 1.6 points for ReForge-CF and 2.4 points for ReForge-CA in this setting, and overall ReForge-CA outperforms the rescaling control on all six anchors and ReForge-CF on five, all except WUDI.
Revision: this comparison replaces Table 7 of the original submission and is now in the main text (Section 5.3, Table 4), with ViT-L/14 results in Appendix B (Table 9).
"Data-free" terminology. We agree and, as you suggested, have renamed ReForge-DF and ReForge-DA to ReForge-CF (calibration-free) and ReForge-CA (calibration-assisted) throughout the revised paper and this response.