- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how to transfer the capabilities of post-trained teacher models to a base model without requiring additional training. The authors observe that functional responses, derived from the base model’s activations, better reflect the computational impact of teacher updates than the magnitude of parameter changes; furthermore, post-training tends to approximately preserve the response scales across channels. Based on these insights, the authors propose Functional Geometric Calibration (FGC), a method that facilitates capability transfer through channel-wise update scaling and the restoration of response scales. FGC also allows for the direct combination of independently calibrated updates from multiple teachers. Experiments in multimodal mathematical reasoning and visual perception demonstrate that this approach can efficiently integrate complementary capabilities using only a small number of calibration samples.
Critical strengths
-
The motivation is clear. For a post-trained teacher model that has already been obtained, the paper explores whether the capabilities embedded in its weights can be directly reused, thereby avoiding the costs associated with regenerating training data and performing iterative optimization.
-
The FGC method is simple, computationally inexpensive, and easy to implement. It primarily consists of channel-wise scaling and response magnitude restoration, requiring no gradient-based training, matrix decomposition, or matrix inversion.
Critical weaknesses
-
The experiments utilize only Qwen2.5-VL-7B-Instruct, along with two teacher models derived from the same base model, focusing primarily on mathematical reasoning and visual perception. The authors are encouraged to extend this work to other model architectures, parameter scales, post-training methodologies, and capability domains.
-
The paper proposes "Post-training preserves the Base functional radius.". It remains unclear whether this is specific to individual models or a universal principle.
-
The response magnitudes of the teacher model and the base model are very similar, a fact that may be linked to the relatively small scale of the post-training updates themselves, and which does not necessarily imply the existence of specific functional-geometric constraints imposed by the post-training process.
Questions for authors
See Weakness
3 comments
Thanks for the author's response. I noticed that the MathVerse average and five subset scores for Qwen3-VL-4B-Instruct in the newly added Table 6 (42.3 / 53.0 / 42.0 / 44.7 / 41.9 / 29.9) are exactly the same as the corresponding results for FGC–ZwZTeacher in Table 3. Could the authors please explain why these results are the same?
We sincerely thank the reviewer for identifying this inconsistency. Upon rechecking the results, we found that the MathVerse scores for FGC–ZwZ Teacher in Table 3 were mistakenly copied from the Qwen3-VL-4B-Instruct results during table preparation. We apologize for this reporting error.
The correct MathVerse scores are 46.8 (avg.), 56.1 (TD), 44.3 (VD), 47.6 (TL), 44.5 (VI), and 41.2 (VO). The results in Table 6 and the visual perception results in Table 3 are unaffected. We have corrected Table 3 in the updated manuscript.
We sincerely thank the reviewer for the thoughtful questions and suggestions. We have extended our evaluation to a different model generation and scale and conducted further analyses of functional-radius preservation, as detailed below.
To further evaluate the generalizability of FGC, we conduct additional experiments on Qwen3-VL-4B-Instruct with two new post-trained Teachers, MMFineReason-4B and P2R-4B. This extends our evaluation across model generations (Qwen2.5-VL and Qwen3-VL) and parameter scales (7B and 4B).
Performance comparison on Qwen3-VL-4B across mathematical reasoning and visual perception benchmarks.
FGC outperforms OrthoMerge and ISO-Merger across all six aggregate benchmark metrics. In particular, FGC achieves 71.2 on MathVerse, compared with 53.0 and 51.9 for the two baselines, approaching the reasoning Teacher's 73.1. This advantage extends across all five MathVerse subsets, supporting the effectiveness of FGC in preserving specialized capabilities across different model generations and scales.
We have included these results in Section 5.5 (Cross-Model Generalization) of the revised manuscript.
We clarify that functional-radius preservation is an empirical observation rather than a universal principle or theoretical guarantee. To examine its generality, we extend the analysis from Qwen2.5-VL-7B to Qwen3-VL-4B, covering two model generations and parameter scales with four post-trained Teachers. For each Base--Teacher pair, we measure the functional-radius ratio ρj across all language FFN layers and Gate, Up, and Down projections using the corresponding Base activations.
Median absolute log-deviation of functional-radius ratios across Base--Teacher pairs.
Approximate functional-radius preservation is observed across both model settings, with median deviations below 10−3 for three Teachers. MMFineReason-4B exhibits larger deviations of approximately 0.005--0.009, indicating that the degree of preservation varies across Teachers. These findings extend the observation beyond the original Qwen2.5-VL-7B setting without establishing a universal property of post-training.
We have clarified its empirical nature in Section 3.2 and included the cross-model analysis in Appendix B.2 of the revised manuscript.
We agree that small post-training updates can contribute to functional-radius preservation, which we interpret as an empirical observation rather than a geometric constraint imposed by post-training. To examine this explanation, we measure the relative functional update magnitude Ej=∥ΔWj∥A/∥WB,j∥A and radius deviation Dj=∣logρj∣ across all 28 language FFN layers of ZwZ-7B and VL-Rethinker-7B. By the reverse triangle inequality, ∣ρj−1∣≤Ej, showing that sufficiently small functional updates naturally lead to similar Base and Teacher response magnitudes.
Relative functional update magnitude and radius deviation.
The median Ej ranges from 0.0015 to 0.0045, confirming that the typical functional updates are small relative to the Base responses. Together with the reverse triangle inequality, these results support the reviewer's explanation. However, Functional Rebase does not assume a constrained post-training trajectory; it restores the Base response scale after Functional Calibration. Its effectiveness is independently supported by the ablation study, where MathVision improves from 33.2 without rebase to 37.5 with Functional Rebase.
We have clarified the empirical nature of functional-radius preservation in Section 3.2 of the revised manuscript.