- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether the geometry of pre-SFT weights can guide improvements to a completed SFT checkpoint. It proposes SPACE, which uses the leading right singular vectors of each pre-SFT weight matrix to define a principal input subspace. SPACE removes the SFT update within this subspace and retains the update in its orthogonal complement. The method requires the pre-SFT and post-SFT checkpoints, without training data or additional optimization. The authors evaluate SPACE across five model families, model sizes from 2B to 14B, and mathematical, coding, and medical fine-tuning. They report improvements in average in-domain and out-of-domain performance over the corresponding uncalibrated checkpoints.
Critical strengths
I find this paper valuable, particularly its method. Selectively reverting SFT update components within the principal subspace of the pre-SFT weights while retaining those in the orthogonal complement is a well-motivated approach. The experimental evaluation is also extensive, covering both older and newer backbones and a range of datasets. It is also reasonable that the OOD performance of the method will be greatly improved.
Critical weaknesses
My main concern is the experimental evidence for improvements in in-domain (ID) performance. On Qwen2.5-7B, SPACE increases the MATH500 score from 44.84 after standard SFT to 49.36. I do not understand why this calibration improves mathematical ability on ID benchmarks, and the paper does not explain this clearly. This leaves me with a strong concern that the improvement may reflect an insufficiently trained or tuned SFT baseline.
I therefore checked results from other papers using the same model and dataset. Anchored Supervised Fine-Tuning (ASFT) also uses the Qwen2.5-7B base model and 100K samples from NuminaMath-CoT:
| Paper and method | MATH500 |
|---|---|
| SPACE: Base model | 28.79 |
| SPACE: Standard SFT | 44.84 |
| SPACE: SFT + SPACE | 49.36 |
| ASFT: Base model | 28.79 |
| ASFT: Standard SFT | 47.30 |
ASFT’s standard SFT score is 3 points higher than SPACE’s standard SFT score and fairly close to the result after SPACE calibration. Of course, the two papers may use different sampled training subsets, which could explain some of the difference. Nevertheless, this discrepancy warrants an explanation.
Table 10 raises a related concern. With four training epochs, standard SFT reaches 56.04 on MATH500, and SPACE provides no additional improvement. Although gains reappear at eight epochs, the dependence on training duration needs to be explained. My main concern is whether the ID gains persist with a well-tuned SFT baseline. The current results do not establish this clearly.
Questions for authors
Section 5.1 suggests that reverting updates within the principal subspace improves generalization to unseen ID examples. Could you clarify why this would improve ID performance, and what evidence in the paper supports this explanation?
4 comments
Response to Reviewer zp6G (Part 2)
Comment W1 (Continued)
2. Why does the ASFT paper report a higher standard SFT score?
Thank you for your careful review. We also noticed this discrepancy during our experiments. After a detailed investigation, we identified a formatting bias in the training data used by ASFT.
Formatting bias. Specifically, around 13K of the 100K responses in the training data begin with a line break character \n, while the rest do not. These responses are approximately twice as long as the others:
| Training responses | Number of examples | Mean response length |
|---|---|---|
Beginning with \n | ~13K | ~766 tokens |
Without a leading \n | ~87K | ~375 tokens |
| All responses | 100K | ~426 tokens |
Also, ASFT’s evaluation template ends with ### Response:\n. This \n can cue the longer response style associated with the newline-prefixed training examples. We hypothesize that these longer reasoning chains contribute to the higher evaluation scores. However, this advantage partly arises from the interaction between the formatting bias in the training data and the evaluation prompt. Therefore, to better isolate the effect of SFT from this bias, in our experiments, we simply remove leading line break characters \n from the training responses.
Controlled experiments support this explanation. To clarify this effect, we conducted controlled experiments using ASFT’s official open-source code and training data, varying only whether to keep the leading line break character (\n) in training responses and whether to include it at the end of the evaluation prompt. We evaluated all 500 MATH500 questions with 16 samples per question:
Leading \n in training responses | Evaluation prompt ending | Mean output length (tokens) | MATH500 (Avg@16) |
|---|---|---|---|
| Kept | ### Response: | 335.75 | 44.61 |
| Kept | ### Response:\n | 572.23 | 47.08 |
| Removed | ### Response: | 336.11 | 44.66 |
| Removed | ### Response:\n | 333.25 | 45.00 |
When training responses retain their leading \n, evaluation prompts ending with \n produce substantially longer answers and higher accuracy. After removing the leading \n from training responses, answer length and accuracy remain largely unchanged regardless of whether evaluation prompts end with \n. These results support formatting bias as an explanation for the score discrepancy. We hope this addresses your concern.
SPACE also remains effective using ASFT’s official code and data. We applied SPACE to a standard SFT checkpoint trained and evaluated using ASFT’s official code, retaining the original training data and evaluation prompt template:
| Setup | Standard SFT | SFT + SPACE |
|---|---|---|
| Our original setup | 44.84 | 49.36 |
| ASFT official code (our reproduction) | 47.08 | 55.95 |
Our reproduced standard SFT score closely matches the result reported in the ASFT paper. SPACE substantially improves MATH500 even with the formatting bias present, supporting its robustness across these experimental settings. We hope this addresses your concern.
3. Training duration
Combining the one-epoch result in Table 1 with the longer training runs in Table 10 gives:
| Training epochs | SFT MATH500 | SPACE MATH500 | Average ID gain |
|---|---|---|---|
| 1 | 44.84 | 49.36 | +2.85 |
| 2 | 53.06 | 53.55 | +0.74 |
| 4 | 56.04 | 56.04 | +0.10 |
| 8 | 55.49 | 58.43 | +2.08 |
SPACE’s gains persist after longer SFT training. Standard SFT improves substantially from one to four epochs, then largely plateaus. SPACE still improves the eight-epoch checkpoint, exceeding the best uncalibrated SFT result among the reported training durations. At four epochs, gains are smaller and benchmark-dependent: MATH500 remains unchanged, while improvements are observed on OlympiadBench and AMC23 (ID), and MMLU-Pro (OOD).
Our training settings follow established recipes. Our main experiments adopt training datasets and hyper-parameters from previous work ("On the generalization of sft: A reinforcement learning perspective with reward rectification" and "Rethinking generalization in reasoning SFT: A conditional analysis on optimization, data, and model capability"). These choices were made for comparability with prior work, rather than selected to favor SPACE.
Overall, these results show that SPACE’s benefits extend beyond a single favorable experimental setting and are not limited to a short-training or weaker SFT baseline.
Comment Q1
Section 5.1 suggests that reverting updates within the principal subspace improves generalization to unseen ID examples. Could you clarify why this would improve ID performance, and what evidence in the paper supports this explanation?
Thank you for raising this question. Please see Section 1 ("Why can SPACE improve ID performance?") of our response to Comment W1 for our explanation and supporting evidence.
Thank you for your detailed reply, which has solved my previous concerns.
Thank you for your follow-up. We are glad that our response has addressed your concerns. Please let us know if you have any other questions. We would also appreciate it if you could reconsider your rating.
Response to Reviewer zp6G (Part 1)
Comment W1
Thank you for raising this concern. We address the explanation for ID gains, the SFT score discrepancy, and the dependence on training duration below.
1. Why can SPACE improve ID performance?
Prior work provides a useful perspective. In Weight Ensembling Improves Reasoning in Language Models, Dang et al. (2025) observe that SFT can improve Pass@1 while reducing generation diversity and degrading Pass@k: repeated samples increasingly concentrate on a limited set of solutions, which can be incorrect. Interpolating early and late SFT checkpoints mitigates this diversity collapse and improves both Pass@1 and Pass@k.
Our results show a similar pattern. We compare Pre-SFT, SFT, and SPACE on mathematical benchmarks using Qwen3-14B-Base:
Here, k denotes samples per question. Pass@1 averages correctness over all samples; answer diversity averages the fraction of distinct final answers among each question’s k samples. No-success questions have no correct sample.
SFT reduces answer diversity relative to Pre-SFT, while SPACE partially restores diversity and improves Pass@1. On AIME25 and Minerva Math, SPACE also improves Pass@k, while reducing the number of no-success questions. On AIME24, Pass@16 decreases slightly despite higher diversity and Pass@1, indicating that increased diversity does not uniformly improve coverage within a finite sampling budget. Overall, these observations are consistent with the findings of Dang et al. (2025) and suggest that SPACE can mitigate diversity collapse, thereby helping improve ID performance.
A complementary explanation comes from Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods. Hao et al. (2025) show that merging pretrained and fine-tuned weights can improve target-domain performance. Their analysis in an overparameterized linear setting explains how merging can balance bias from insufficient adaptation against variance from fitting finite fine-tuning data.
Both studies discussed above (Dang et al. (2025); Hao et al. (2025)) analyze weight ensembling/merging. SPACE can likewise be interpreted as a form of model merging: WSPACE=W0P0+WSFT(I−P0), where P0 projects onto the principal input subspace of W0. This provides a plausible connection to the ID improvements observed in weight ensembling.
Finally, ID improvements also appear among the model merging baselines in Table 1. For example, WiSE-FT yields an average ID gain of +0.78 on Qwen2.5-7B, and LiNeS yields +1.70 on Qwen3-14B-Base, consistent with prior observations that weight ensembling can improve ID performance. SPACE achieves larger average ID gains. We hypothesize that useful task adaptation and updates detrimental to generalization may lie in different directions. SPACE’s larger gains over global (WiSE-FT) or layer-wise merging (LiNeS) may therefore stem from selectively reverting detrimental updates while preserving more useful adaptation.
We have incorporated this discussion and analysis into Appendix C.7 of the uploaded revised manuscript, with a brief mention in Section 5.1 of the main text. These changes are highlighted in blue.
References:
Dang et al. (2025), Weight Ensembling Improves Reasoning in Language Models.
Hao et al. (2025), Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods.