acceptodds
Under review as a conference paper at ICLR 2027

LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation

11 views

Abstract

Multimodal on-policy distillation (OPD) aims to improve visual grounding while preserving the strong reasoning capabilities of language models. Recent multi-teacher approaches combine LLM and VLM teachers to provide complementary supervision. However, directly using a VLM's full predictive distribution entangles its visual grounding signal with its own language prior, preventing the grounding information from being transferred independently. Conversely, increasing the strength of visual supervision can improve perception but may overemphasize visual evidence and degrade language reasoning. To address this trade-off, we introduce LEGO-OPD, which selectively composes factors from a Language Expert and a Grounding expert into One teacher distribution for multimodal OPD. Under a generalized Bayesian formulation, the language expert provides a prior over candidate tokens, while the grounding expert contributes a visual likelihood that updates this prior, rather than transferring its complete predictive distribution. This factorized composition allows language reasoning and visual grounding to be controlled independently. We further introduce adaptive calibration to determine how strongly the visual likelihood should update the language prior at each decoding prefix. Specifically, LEGO-OPD uses the grounding expert's image-induced prediction shift as a prefix-dependent reference, preventing both insufficient and excessive visual supervision. Experiments with Qwen3 models show that LEGO-OPD consistently outperforms the evaluated single- and multi-teacher OPD baselines on both multimodal and text-only reasoning tasks. Moreover, it improves the initial student's visual perception while preserving text-only reasoning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Vision-Language Models, Multimodal Reasoning, On-Policy Distillation, Model Composition, Multi-Teacher Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_d9WGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LEGO-OPD constructs a multimodal distillation teacher by combining an LLM language prior with a VLM’s image-induced prediction residual. A prefix-dependent calibration rule controls the strength of the visual update using a source-referenced KL budget. Experiments on Qwen3 models show improvements in both multimodal and text-only reasoning, supported by ablations on calibration and expert selection.

Critical strengths
  1. The factorized teacher construction is well motivated: it separates the language prior from the VLM’s image-induced prediction change rather than combining complete teacher distributions. The generalized Bayesian formulation makes this construction clear, while acknowledging that the residual is a model-dependent proxy for visual evidence.
  2. The empirical gains are encouraging. With the same expert pair, LEGO-OPD improves the multimodal average over H-OPD from 43.47 to 46.28 and the text average from 55.57 to 56.68. The comparison against fixed coefficients in Table 2 provides additional evidence for adaptive calibration.
Critical weaknesses
  1. The practical calibration only approximates the stated KL target. Figure 2 usefully documents deviations for larger updates, but the diagnostic covers ρ=1\rho=1, whereas the strongest results use ρ=2\rho=2. The calibration error and coefficient-clipping frequency at this setting remain unclear, limiting the interpretation of why the method works.
  2. The main comparison leaves some uncertainty about attribution and robustness. H-OPD and LEGO-OPD use different distillation supports, so their comparison does not fully isolate teacher construction from the update procedure. Moreover, the reported standard deviations measure evaluation sampling variability rather than variation across independent training runs.
Questions for authors
  1. For ρ=2\rho=2, what is the distribution of achieved versus target teacher KL, and how often does the calibration coefficient reach its cap?
  2. Are matched-support comparisons with H-OPD and independent-training-seed results available? These would materially strengthen the evidence for the source and reliability of the gains.
4 comments
AuthorsOpenReview ↗

Response to Reviewer d9WG [1/2]

Thank you for highlighting the distinction between the intended KL target and its practical approximation, and between evaluation variability and training variability. We agree that these distinctions are essential for interpreting the results.

Q1 & W1 Calibration diagnostics at ρ=2\rho=2

We extended the expert-composition diagnostic in Mainpaper Figure 2 to ρ=2\rho=2 using exactly the same sampled sequences. The coefficient comparisons and achieved-versus-target teacher KL curves are provided in Appendix Figure 8, and the cap-hit rates in Appendix Table 16, both in Section D.6 of the revised manuscript. Figure 8 reports source-KL-binned means. In both settings, achieved KL tracks the target for small source KL and shows upward deviations over an intermediate-to-large source-KL range.

To complement the KL curves, Table R5 reports how often the practical approximate coefficient reaches its cap.

Table R5. Overall cap-hit rate of the practical approximate coefficient (αmax⁡=6\alpha_{\max}=6, 24,576 diagnostic prefixes).

ρ\rhoOverall cap-hit rate (%)
12.14
23.14

The overall cap-hit rate increases modestly from 2.14% at ρ=1\rho=1 to 3.14% at ρ=2\rho=2, remaining low in both settings.

Under Mainpaper Eq. (11), increasing ρ\rho from 1 to 2 doubles the target KL budget. With our approximate calibration rule, this globally scales the visual evidence coefficient α\alpha by 2\sqrt{2} before capping, consistent with the modest increase in the cap-hit rate. These diagnostics show that a larger budget strengthens the contribution of visual evidence as intended. This stronger visual emphasis is reflected in modest gains in multimodal and text-only reasoning averages at ρ=2\rho=2 in Mainpaper Table 1.

Q2 & W2 H-OPD comparison with aligned support selection

We added an H-OPD control that selects the student's top-16 tokens, following LEGO-OPD's support-selection rule. The original H-OPD uses the union of the two teachers' top-8 sets. We compare both H-OPD settings with Single VLM OPD and LEGO-OPD at ρ=1\rho=1 and ρ=2\rho=2 using training seed 42. All models use the 2B student and are evaluated at step 200. Single VLM OPD uses the 4B VLM teacher, while H-OPD and LEGO-OPD use the same 4B language and grounding experts.

Tables R6 and R7 report mean accuracy ± SD across inference seeds for individual benchmarks. Category averages are reported as means only.

Table R6. Multimodal reasoning comparison at training seed 42 (%).

MethodMATH-VisionMathVerseMathVistaMMMU-ProRBench-MOlympiad-MMMM Avg.
Single VLM OPD27.43 ± 0.8952.70 ± 0.7668.60 ± 0.3936.98 ± 0.8916.90 ± 0.5239.60 ± 0.3740.37
H-OPD, original top-832.37 ± 1.8355.41 ± 0.6567.70 ± 0.7939.36 ± 0.6926.92 ± 1.5939.07 ± 3.0843.47
H-OPD, student top-1631.45 ± 3.2456.25 ± 0.6068.02 ± 0.5039.73 ± 0.3926.74 ± 2.0940.80 ± 1.4543.83
LEGO-OPD, ρ=1\rho=133.36 ± 2.5656.96 ± 0.2069.62 ± 0.7240.98 ± 1.1429.17 ± 1.2044.93 ± 2.2445.84
LEGO-OPD, ρ=2\rho=234.28 ± 2.0657.19 ± 0.4470.58 ± 1.0241.38 ± 0.7229.20 ± 1.4945.07 ± 3.8546.28

Table R7. Text-only reasoning comparison at training seed 42 (%).

MethodMATH-500MathVerse-TOMMLU-ProRBench-TOlympiad-TextText Avg.
Single VLM OPD78.72 ± 0.8257.44 ± 0.8245.52 ± 0.1628.59 ± 1.0648.31 ± 0.7051.72
H-OPD, original top-881.88 ± 1.0859.44 ± 0.3946.94 ± 0.1737.17 ± 1.1252.40 ± 0.9555.57
H-OPD, student top-1681.96 ± 0.7859.95 ± 0.4546.84 ± 0.2237.06 ± 0.8052.20 ± 1.4955.60
LEGO-OPD, ρ=1\rho=182.40 ± 0.9959.59 ± 0.9947.15 ± 0.0538.59 ± 1.1952.31 ± 0.8956.01
LEGO-OPD, ρ=2\rho=282.36 ± 0.8460.23 ± 0.4047.54 ± 0.1840.27 ± 1.9953.00 ± 0.7756.68

Changing H-OPD's support raises its multimodal/text-only averages modestly from 43.47/55.57 to 43.83/55.60. Single VLM OPD achieves 40.37/51.72, while LEGO-OPD reaches 45.84/56.01 at ρ=1\rho=1 and 46.28/56.68 at ρ=2\rho=2. LEGO-OPD's gains persist after aligning support selection. This comparison is included in Appendix Tables 28 and 29 in Section E.6 of the revised manuscript. The Single VLM OPD, original H-OPD, and LEGO-OPD results match Mainpaper Table 1. The student-top-16 H-OPD checkpoint is an additional control.

AuthorsOpenReview ↗

Response to Reviewer d9WG [2/2]

Q2 & W2 Robustness across independent training seeds

Computational constraints prevented us from repeating every baseline across three training seeds. We selected H-OPD with student-top-16 support as a representative multi-teacher comparison with aligned support selection. We evaluated this control and LEGO-OPD at ρ=1\rho=1 and ρ=2\rho=2.

The results below use three training seeds (42–44), adding two seeds to the training seed used in the main paper. Each model is trained for 200 steps and evaluated with five inference seeds. Within each training seed, we first average the results over the five inference seeds and compute the category averages. We then report the mean and sample standard deviation across the three training seeds.

Table R8. Multimodal reasoning averaged across three training seeds (%).

MethodMATH-VisionMathVerseMathVistaMMMU-ProRBench-MOlympiad-MMMM Avg.
H-OPD, student top-1631.73 ± 0.4956.60 ± 0.3068.08 ± 0.1839.82 ± 0.3125.68 ± 1.0740.45 ± 0.8643.73 ± 0.10
LEGO-OPD, ρ=1\rho=134.83 ± 1.8557.68 ± 0.6769.92 ± 0.3941.12 ± 0.2528.77 ± 0.4444.71 ± 0.6346.17 ± 0.38
LEGO-OPD, ρ=2\rho=235.16 ± 0.7658.21 ± 0.8970.11 ± 0.5141.41 ± 0.3828.86 ± 1.0644.00 ± 0.9246.29 ± 0.25

Table R9. Text-only reasoning averaged across three training seeds (%).

MethodMATH-500MathVerse-TOMMLU-ProRBench-TOlympiad-TextText Avg.
H-OPD, student top-1681.77 ± 0.2660.03 ± 0.1346.79 ± 0.0537.23 ± 0.5652.29 ± 0.1555.62 ± 0.16
LEGO-OPD, ρ=1\rho=182.28 ± 0.1160.07 ± 0.5647.30 ± 0.2138.85 ± 0.3552.51 ± 0.3056.20 ± 0.22
LEGO-OPD, ρ=2\rho=282.23 ± 0.4560.30 ± 0.0847.46 ± 0.0739.12 ± 1.0152.56 ± 0.4056.33 ± 0.30

Both LEGO-OPD settings achieve higher multimodal and text-only averages than the H-OPD control at each of the three seeds. Across the three seeds, the gains are +2.44/+0.58 pp at ρ=1\rho=1 and +2.56/+0.71 pp at ρ=2\rho=2 for multimodal/text-only reasoning. This consistency supports repeatability beyond inference-seed variability. Detailed benchmark results for each training seed are provided in Appendix Tables 26 and 27 in Section E.5 of the revised manuscript.

Thank you for the detailed response and the additional experiments. The extended calibration diagnostics, matched-support comparison with H-OPD, and results across independent training seeds largely address my main concerns and strengthen the evidence for the method’s robustness.

We thank the reviewer for taking the time to examine our additional analyses and experiments. We appreciate their constructive feedback, which helped us clarify the calibration behavior and strengthen the evidence for the method’s robustness.

Reviewer_Kc8JOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes LEGO-OPD, which combines the reasoning capabilities of language models with the visual understanding capabilities of vision-language models to train stronger small vision-language models. A language expert provides a base prediction, while a grounding expert supplies a visual correction derived by comparing its predictions with and without the image. To avoid excessively strong or weak corrections, the method dynamically computes a scaling coefficient based on the image-induced prediction shift and the language expert’s sensitivity to the correction. These factors are composed into a unified teacher distribution, which supervises the student through on-policy distillation. In experiments with Qwen3 models, the best reported configuration improves average accuracy over H-OPD by 2.81 percentage points on multimodal reasoning and 1.11 percentage points on text-only reasoning, demonstrating improved image-grounded reasoning while maintaining text-only reasoning performance under the evaluated settings.

Critical strengths

The paper introduces source-referenced calibration, which uses the grounding expert’s image-induced prediction shift as a reference to adaptively determine how strongly the visual signal should update the language prior at each student-generated prefix. This mechanism accounts for both the source-side visual shift and its potential impact on the receiving language distribution, providing a concrete, computable way to control cross-expert composition strength and reducing reliance on fixed composition coefficients. This design directly addresses the problem that the same visual signal can induce different prediction shifts under different language priors, establishing a clear connection between the identified problem and the proposed method. With the student, expert pair, and other training settings held constant, Table 2 shows that the default adaptive method achieves an average multimodal reasoning accuracy of 45.84%, compared with 43.22% for the best tested fixed coefficient. This combination of a clear methodological rationale and direct experimental support is an important basis for recognizing the paper’s technical contribution and leaning toward acceptance.

Critical weaknesses

The necessity of source-referenced calibration, the paper’s central design, is not sufficiently validated. The authors use the grounding expert’s image-induced KL divergence as the calibration budget, but do not sufficiently establish why this quantity should serve as the reference or whether it is preferable to simpler adaptive rules. The experiments mainly compare the complete calibration formula against fixed composition coefficients, without key controls such as a fixed KL budget or adaptation based solely on the receiving language distribution’s sensitivity. Thus, the experiments provide stronger support for the complete method’s performance gains than for why source-referenced calibration works or whether it is essential to those gains. This limits confidence in the distinctive value of the core design, without directly undermining the observed improvements.

Questions for authors
  1. Why is the source KL divergence chosen as the calibration budget?** Why should the grounding expert’s image-induced prediction shift serve as a reference for how much the language prior should be modified? What theoretical justification or direct empirical evidence supports this choice?
  2. Does source-referenced calibration outperform simpler adaptive rules?** Could the authors include controls using a fixed KL budget, adaptation based solely on the source KL, and adaptation based solely on the receiving language distribution’s sensitivity, to clarify the contribution of each component of the full formula?
  3. Why is the method evaluated with only Qwen3-VL-2B as the student model?** Could the authors validate it on additional student models to assess whether its effectiveness extends beyond this configuration?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Kc8J [1/3]

We thank the reviewer for the careful assessment and constructive suggestions. We appreciate the positive evaluation of our motivation for adapting visual corrections when the prior changes and of the effectiveness of adaptive calibration. We agree that fixed-coefficient comparisons alone do not sufficiently justify the source KL reference or isolate each component's contribution. Below, we explain our rationale and provide the requested controls and additional student results.

Q1 & W1 Choice of the source KL budget

Our teacher is designed to retain the language expert's prior as its default prediction while allocating larger updates to positions requiring more visual correction. We therefore measure how much the image changes the grounding expert's prediction at the same prefix using the source KL.

Here, qV,tIq_{V,t}^{I} and qV,t0q_{V,t}^{0} denote the grounding expert's predictive distributions at token position tt with and without the image, respectively:

DKL(qV,tI∥qV,t0)=Ev∼qV,tI[log⁡qV,tI(v)−log⁡qV,t0(v)].D_{\mathrm{KL}}(q_{V,t}^{I}\Vert q_{V,t}^{0})=\mathbb{E}_{v\sim q_{V,t}^{I}}\left[\log q_{V,t}^{I}(v)-\log q_{V,t}^{0}(v)\right].

This can also be interpreted as the expected increase in log score from using the image, evaluated under the image-conditioned distribution. We use this model-relative predictive change as a proxy for the strength of visual supervision needed at each position.

Following this design objective, we determine the target predictive change from the source and use KL matching, which recovers the original grounding expert when the prior is unchanged, as our reference (Appendix A.4). We apply this reference to other language priors to preserve the predictive change observed at the source.

Here, qL,tq_{L,t} is the receiving language prior, and rtr_t is the visual residual: the grounding expert's token-wise log-probability difference with versus without the image. The teacher distribution qαt,tq_{\alpha_t,t} incorporates this residual with coefficient αt\alpha_t. The calibration budget is the target KL divergence between the teacher distribution and the language prior.

With the default ρ=1\rho=1, the matching target and the coefficient obtained using the paper's quadratic approximation are

DKL(qαt,t∥qL,t)=DKL(qV,tI∥qV,t0)D_{\mathrm{KL}}(q_{\alpha_t,t}\Vert q_{L,t})=D_{\mathrm{KL}}(q_{V,t}^{I}\Vert q_{V,t}^{0})

α^t=2DKL(qV,tI∥qV,t0)Var⁡qL,t[rt].\widehat{\alpha}_t=\sqrt{\frac{2D_{\mathrm{KL}}(q_{V,t}^{I}\Vert q_{V,t}^{0})}{\operatorname{Var}_{q_{L,t}}[r_t]}}.

Here, the source KL provides the target predictive change, while recipient sensitivity (the residual variance under its prior) determines the coefficient needed to implement that change. Unlike choosing the coefficient from the source KL alone, this accounts for the fact that the same visual residual can induce different predictive changes depending on the recipient.

Matching is a reference for preserving the source's predictive change under a new prior, and we agree that it is not a theoretically guaranteed optimal choice. Accordingly, we allow the amount of visual correction relative to this reference to be adjusted through the budget ρDKL(qV,tI∥qV,t0)\rho D_{\mathrm{KL}}(q_{V,t}^{I}\Vert q_{V,t}^{0}). Table 2 also reports higher performance with ρ=2\rho=2 than with ρ=1\rho=1.

We have revised the discussion in Section 3.2 of the revised manuscript to clarify the rationale for the source-KL reference. To strengthen the empirical evidence for this design, we compare it with simpler adaptive rules below and evaluate the effectiveness of source-referenced calibration and the contribution of each component.

AuthorsOpenReview ↗

Response to Reviewer Kc8J [2/3]

Q2 & W1 Comparison with simpler adaptive rules

We compare LEGO-OPD with the default ρ=1\rho=1 against source-only and recipient-only controls. Source-only determines the coefficient using only the source KL, while recipient-only uses only the residual variance under the recipient prior. All training settings are identical except for the coefficient calculation.

Following Mainpaper Table 3, dtd_t and VtV_t denote the support-restricted source KL and recipient residual variance, respectively. The two controls are

αtS=cSdt,αtR=cRVt+ϵ.\alpha_t^{S}=c_S\sqrt{d_t},\qquad \alpha_t^{R}=\frac{c_R}{\sqrt{V_t+\epsilon}}.

Here, ϵ\epsilon is the regularizer, and cS,cRc_S,c_R are shared scalar multipliers held fixed during training. Following Mainpaper Table 3, Unit sets the corresponding multiplier to one, while Matched chooses it to match LEGO-OPD's mean KL divergence from the language prior on the common prefix set described below.

We determine these constants using the same 24,576 shared prefixes as in Figure 2, obtained from 768 rollouts for 256 MMMU questions and covering student checkpoints 0, 100, and 200. We compute the KL divergence between the teacher distribution and the language prior at each prefix, then average these values. For the Matched controls, this gives cS=2/(0.4395+ϵ)c_S=\sqrt{2/(0.4395+\epsilon)} and cR=2(0.2238)c_R=\sqrt{2(0.2238)}, equivalent to replacing VtV_t by 0.4395 and dtd_t by 0.2238, respectively (Appendix D.3).

The results below use three training seeds (42–44), adding two seeds to the training seed used in the main paper. Each model is trained for 200 steps and evaluated with five inference seeds. Within each training seed, we first average the results over the five inference seeds and compute the category averages. We then report the mean and sample standard deviation across the three training seeds.

Table R3. Comparison with simpler adaptive rules (%).

MethodCoefficientMM Avg.Text Avg.
LEGO-OPD, ρ=1\rho=12dt/(Vt+ϵ)\sqrt{2d_t/(V_t+\epsilon)}46.17 ± 0.3856.20 ± 0.22
Source-only - Unitdt\sqrt{d_t}44.79 ± 0.1756.20 ± 0.11
Source-only - MatchedcSdtc_S\sqrt{d_t}45.92 ± 0.2856.26 ± 0.20
Recipient-only - Unit1/Vt+ϵ1/\sqrt{V_t+\epsilon}45.52 ± 0.3056.00 ± 0.36
Recipient-only - MatchedcR/Vt+ϵc_R/\sqrt{V_t+\epsilon}45.13 ± 0.0955.66 ± 0.16

Per-benchmark means and sample standard deviations across the three training seeds are provided in Appendix D.3, Table 9 of the revised manuscript. Detailed results for each training seed are reported in Appendix D.3, Tables 10–13 for the source-only and recipient-only controls, and in Appendix E.5, Tables 26 and 27 for LEGO-OPD.

For source-only, the improvement from Unit to Matched shows a clear benefit from adjusting the scalar multiplier. Choosing cSc_S to match LEGO-OPD's mean predictive change on the common prefix set increases multimodal accuracy from 44.79% to 45.92%. LEGO-OPD, which additionally accounts for the residual variance at each position, achieves 46.17%, a further increase of 0.25 percentage points. This supports using LEGO-OPD's mean predictive change as a useful reference for choosing the multiplier, with position-dependent variance correction providing a modest additional gain.

For recipient-only, moving from Unit to Matched reduces both the fixed budget and performance. Under the quadratic approximation, the budget is b0=cR2/2b_0=c_R^2/2, equal to 0.5 for Unit and 0.2238 for Matched. Multimodal accuracy decreases from 45.52% to 45.13%. In contrast, LEGO-OPD achieves 46.17% with the same mean predictive change as this control on the common prefix set. This is also 0.65 percentage points higher than recipient-only Unit, despite a smaller mean predictive change on that set.

Compared with recipient-only Matched, LEGO-OPD improves multimodal accuracy by 1.04 percentage points and text-only accuracy by 0.54 percentage points. The methods differ in which positions receive the budget and how much each receives. Recipient-only assigns the same budget to every position, whereas LEGO-OPD assigns larger budgets to positions with larger source KL. These results support the usefulness of allocating position-dependent budgets using the source KL at the same mean predictive change on the common prefix set. We have added these controls and their analysis to Section 4.3 and Table 3 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Kc8J [3/3]

Q3 Additional student size and generalization

We agree that the original 2B-student results alone do not establish effectiveness under other student configurations. We therefore conduct an additional experiment with Qwen3-VL-4B-Instruct as the student.

Official results show that Qwen3-VL-4B-Instruct outperforms Qwen3-8B in non-thinking mode on MMLU-Pro (67.1 vs. 63.4) and AIME25 (46.6 vs. 20.9) [1, Table 9]. To provide a stronger language prior than the student, we select Qwen3-30B-A3B-Instruct-2507, which reports 78.4 and 61.3 on these benchmarks, respectively [2].

LEGO-OPD and H-OPD use the same Qwen3-30B-A3B-Instruct-2507 language expert and Qwen3-VL-8B-Instruct grounding expert. Standard OPD uses Qwen3-VL-8B-Instruct as its teacher. All three methods train the 4B student for 200 steps, and LEGO-OPD uses the default ρ=1\rho=1.

Table R4. Reasoning results for the 4B student (%).

MethodLanguage expertVLM teacherMM Avg.Text Avg.
Qwen3-VL-4B-Instruct, before training——54.83 ± 0.5868.81 ± 0.28
Standard OPD—Qwen3-VL-8B-Instruct55.95 ± 0.5368.25 ± 0.40
H-OPDQwen3-30B-A3B-Instruct-2507Qwen3-VL-8B-Instruct56.67 ± 0.4468.50 ± 0.45
LEGO-OPD, ρ=1\rho=1Qwen3-30B-A3B-Instruct-2507Qwen3-VL-8B-Instruct57.76 ± 0.6069.23 ± 0.36

With the same expert pair, LEGO-OPD outperforms H-OPD by 1.09 percentage points on multimodal benchmarks and 0.73 percentage points on text-only benchmarks. The gains over Standard OPD are 1.81 and 0.98 percentage points, respectively. These results support the effectiveness of our design in improving visual reasoning while preserving text-only performance when scaling the student from 2B to 4B within the Qwen3-VL family.

Detailed benchmark results, together with the initial 8B VLM teacher reference, are provided in Appendix E.4, Tables 22 and 23 of the revised manuscript.

References

[1] Shuai Bai et al. Qwen3-VL Technical Report. arXiv preprint arXiv:2511.21631v1, 2025.

[2] Qwen Team. Qwen3-30B-A3B-Instruct-2507. Official model card, Hugging Face, 2025. URL https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507.

Dear Reviewer, thank you again for your time and constructive feedback. In our responses above, we clarified the rationale for the source-KL reference, added source-only and recipient-only calibration controls across three training seeds, and extended the evaluation to a 4B student. We would appreciate your feedback on whether these clarifications and additional experiments address your concerns. Please let us know if any further clarification would be helpful.

Reviewer_ji7yOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes LEGO-OPD, a multimodal on-policy distillation method that combines an LLM’s language prior with a visual signal derived from a VLM’s image-conditioned and image-free predictions. Its main contribution is a prefix-dependent calibration rule that adjusts the visual signal’s strength using the VLM’s image-induced KL divergence as a reference. Experiments with a Qwen3-VL-2B student show improvements over the evaluated single- and multi-teacher baselines on multimodal and text-only reasoning benchmarks. Ablations examine adaptive versus fixed composition, different expert configurations, and the calibration approximation.

Critical strengths

1.Adaptive calibration is well motivated and supported by controlled ablations. LEGO-OPD adjusts visual guidance at each prefix rather than using a fixed composition coefficient. Table 2 shows improvements over all tested fixed coefficients under otherwise matched settings, supporting the value of this design.

2.The method improves multimodal reasoning while preserving text-only reasoning. Using the same expert pair, LEGO-OPD outperforms H-OPD on all six multimodal reasoning benchmarks and improves the text-only average (Table 1). These results support its effectiveness in combining complementary teacher capabilities.

Critical weaknesses

None

Questions for authors

1.Does LEGO-OPD degrade perception on vision-centric tasks? Additional evaluations on MMVP, V*Bench, and HallusionBench would help clarify this.

2.Could the authors evaluate LEGO-OPD across more model sizes to assess its generalizability beyond the current 2B-student setting?

3 comments
AuthorsOpenReview ↗

Response to Reviewer ji7y

Thank you for recognizing the motivation for adaptive calibration and the improvements in multimodal and text-only reasoning. Following your suggestions, we broadened the perception evaluation and extended the experiments to a larger student.

Q1 Visual perception

Multimodal reasoning performance reflects both language reasoning supported by the language prior and visual grounding. To further assess the transfer of visual capabilities, we added MMVP, V*Bench, and HallusionBench to the BLINK and QBench1,2 evaluations reported in Mainpaper Table 1. The combined results are shown below.

Table R1. Perception results for the 2B student (%).

MethodMMVPV*BenchHallusionBenchBLINKQBench1,2AVG
Initial 2B student45.3372.7731.8751.3469.1854.10
Single VLM OPD46.0072.2531.1652.1369.5454.22
H-OPD, original k=8k=8 setting46.6772.2530.9551.7668.9854.12
M-OPD44.6774.8730.3951.8768.8654.13
VA-OPD45.3371.7331.0152.8169.7054.12
LEGO-OPD, ρ=1\rho=146.0074.3533.2552.0369.3454.99
LEGO-OPD, ρ=2\rho=249.3375.9233.4752.5569.6256.18

LEGO-OPD benefits from a stronger language prior for reasoning while improving the average perception score over Single VLM OPD, from 54.22 to 54.99 at ρ=1\rho=1 and 56.18 at ρ=2\rho=2. Both settings improve all five scores over the initial student. At ρ=2\rho=2, all five also exceed Single VLM OPD. The higher perception scores at ρ=2\rho=2 are consistent with its larger target KL budget and stronger visual emphasis (Mainpaper Section 4.2). These results support effective transfer of visual capabilities alongside improved reasoning. We have included this expanded evaluation in Mainpaper Table 1 of the revised manuscript and added the evaluation protocols in Appendix C.3.2 and Table 7.

Q2 Additional student size and generalization

We agree that the original 2B-student results alone do not establish effectiveness under other student configurations. We therefore conduct an additional experiment with Qwen3-VL-4B-Instruct as the student.

Official results show that Qwen3-VL-4B-Instruct outperforms Qwen3-8B in non-thinking mode on MMLU-Pro (67.1 vs. 63.4) and AIME25 (46.6 vs. 20.9) [1, Table 9]. To provide a stronger language prior than the student, we select Qwen3-30B-A3B-Instruct-2507, which reports 78.4 and 61.3 on these benchmarks, respectively [2].

LEGO-OPD and H-OPD use the same Qwen3-30B-A3B-Instruct-2507 language expert and Qwen3-VL-8B-Instruct grounding expert. Standard OPD uses Qwen3-VL-8B-Instruct as its teacher. All three methods train the 4B student for 200 steps, and LEGO-OPD uses the default ρ=1\rho=1.

Table R2. Reasoning results for the 4B student (%).

MethodLanguage expertVLM teacherMM Avg.Text Avg.
Qwen3-VL-4B-Instruct, before training——54.83 ± 0.5868.81 ± 0.28
Standard OPD—Qwen3-VL-8B-Instruct55.95 ± 0.5368.25 ± 0.40
H-OPDQwen3-30B-A3B-Instruct-2507Qwen3-VL-8B-Instruct56.67 ± 0.4468.50 ± 0.45
LEGO-OPD, ρ=1\rho=1Qwen3-30B-A3B-Instruct-2507Qwen3-VL-8B-Instruct57.76 ± 0.6069.23 ± 0.36

With the same expert pair, LEGO-OPD outperforms H-OPD by 1.09 percentage points on multimodal benchmarks and 0.73 percentage points on text-only benchmarks. The gains over Standard OPD are 1.81 and 0.98 percentage points, respectively. These results support the effectiveness of our design in improving visual reasoning while preserving text-only performance when scaling the student from 2B to 4B within the Qwen3-VL family.

Detailed benchmark results, together with the initial 8B VLM teacher reference, are provided in Appendix E.4, Tables 22 and 23 of the revised manuscript.

References

[1] Shuai Bai et al. Qwen3-VL Technical Report. arXiv preprint arXiv:2511.21631v1, 2025.

[2] Qwen Team. Qwen3-30B-A3B-Instruct-2507. Official model card, Hugging Face, 2025. URL https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507.

The authors’ response has adequately addressed my concerns. I am satisfied with the clarification and maintain my original positive score.

We sincerely thank the reviewer for their positive feedback and for confirming that our response addressed their concerns. We appreciate the reviewer’s constructive suggestions, which helped us strengthen the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.