- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes DoST, a post-hoc method for combining independently trained SFT and RLVR checkpoints from a shared base model. It applies magnitude pruning, restores each update’s original norm, and searches two merging coefficients using verifier accuracy. The evaluation covers seven math benchmarks, OOD QA and coding, and three backbones. A stylized theoretical construction illustrates when complementary source updates can outperform either individually rescaled source.
Critical strengths
Post-hoc calibration is practically useful when source checkpoints are already available. On Qwen2.5-Math-7B, DoST achieves an average accuracy of 46.8, compared with 44.3 and 40.9 for the SFT and on-policy RL source models, respectively (Table 1), with a reported search cost of 20 GPU-hours. The OOD results obtained without retuning the coefficients (Table 3) provide further evidence that the selected mixture can generalize beyond the calibration domain.
Critical weaknesses
W1. The comparisons do not isolate DoST’s contribution beyond established model merging. Appendix A.4 reports default settings for TIES and DARE, whereas DoST receives 100 accuracy-guided search trials. Their poor performance therefore does not establish that their failures stem specifically from SFT–RLVR geometry. The main SFT/on-policy RL comparison also lacks a dense-merging baseline with the same search budget. For ExGRPO/ReLIFT, the available dense-search control already achieves 46.6, compared with 47.1 for DoST (Table 4). The empirical evaluation also omits single-source controls with individually optimized coefficients, despite their inclusion in the theoretical comparator. Budget-matched merging baselines and optimized single-source controls are needed to distinguish the benefit of the proposed synthesis procedure from that of coefficient tuning. This distinction matters because pruning and learned merging coefficients are already established in TIES-Merging and AdaMerging, respectively, both of which the paper cites. W2. The proposed conflict-control mechanism has limited direct validation. Figure 1 uses global top-10% selection, whereas the algorithm retains 30% of the coordinates within each parameter tensor. Opposite-sign overlap can decrease simply because pruning reduces the support, so this statistic alone does not establish that harmful interference has been removed. The pruning ablation shows an average gain of only 0.5 percentage points, with lower scores on four of the seven benchmarks (Table 4). The paper provides neither a comparison against random masks at the same retention rate nor an analysis of how accuracy varies with the retention fraction. The theory does not resolve this gap: Appendix D.4.1 removes coordinates that do not affect correctness and establishes preservation, rather than improvement over optimal dense merging. The central design rationale therefore remains insufficiently supported. Controls that use the algorithm’s actual pruning operator are needed to establish why this processing is beneficial.
Questions for authors
For W1, what scores do budget-matched dense, TIES/DARE, and individually rescaled source baselines achieve on the same source pairs and calibration set? For W2, does magnitude pruning outperform matched-retention random pruning across retention fractions under the same coefficient-search budget? These comparisons would materially affect my assessment of the contribution.