- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work presents -OPD, a grounded‑and‑graspable on‑policy distillation framework to transfer dynamic spatial‑reasoning capabilities from large vision‑language teacher models into compact student models. To address the uniform‑weight limitation of standard on‑policy distillation, -OPD adaptively re‑allocates reverse‑KL supervision over the student’s self‑generated trajectories using two complementary signals: a grounding score derived from counterfactual video probes that identifies predictions relying on spatial, temporal‑order and motion evidence, and a graspability score combining teacher‑candidate support and local teacher‑student disagreement to locate learnable corrections. Without introducing extra inference‑time modules or task‑specific rewards, the 2B‑parameter student trained with -OPD achieves substantial gains on VLM4D and DSR‑Bench benchmarks compared to baseline distillation and reinforcement‑learning approaches.
Critical strengths
-
This work brings an important issue from the DPO field into the multimodal cognition field, namely that all tokens share the same training weights. This is indeed a problem that methods of this kind need to address.
-
To distinguish the weights of different tokens, the paper suggests two signals that are quite interpretable.
-
A lot of experimental results show that the 2B qwen model using this method has achieved significant performance gains.
Critical weaknesses
-
The training costs increase noticeably. In fact, the DPO method is often accepted as a more efficient version of the GRPO method. But the approach proposed in the paper significantly raises the training costs of DPO (the teacher network has to run 4 forward passes for each sample), which weakens the original advantage of DPO methods.
-
The proposed method introduces a large number of hyperparameter designs (evidence weights (), graspability's (gamma), candidate set size K, and quantile normalization thresholds), which to some extent limits the further widespread application of this method. The author should further analyze how sensitive the method is to the hyperparameters.
-
Counterfactual probes are heuristically constructed, so it's hard to say if they'd still work in more general scenarios.
Questions for authors
-
In fact, within the broader field of multimodal reasoning, there are quite a few ways to optimize DPO using path information. So what advantages does our approach have compared to others?
-
As far as I know (the author's main experiment also proves this), Qwen3 itself is a really excellent multimodal baseline model. This makes me curious whether the method would still work on other multimodal baseline models (LLava, Qwen3.5, Qwen2.5-VL).
-
Actually, I have a relative understanding of work in the MLLM4D field. I know that the outputs of MLLM4D include both text and physical-side data. I want to know which tokens are the most critical in your setup. Does this phenomenon also occur with other multimodal models that could act as teachers?
6 comments
Response to Reviewer kkf4 (2/3)
Hyperparameter sensitivity:
| Parameter | Setting | VLM4D | DSR-Bench |
|---|---|---|---|
| 0.10 | 50.61 | 33.59 | |
| 0.25 (default) | 51.38 | 34.49 | |
| 0.50 | 50.83 | 33.87 | |
| 8 | 50.55 | 33.45 | |
| 16 (default) | 51.38 | 34.49 | |
| 32 | 51.05 | 34.08 | |
| (all channels) | 0.125 | 50.17 | 33.24 |
| (all channels) | 0.25 (default) | 51.38 | 34.49 |
| (all channels) | 0.50 | 50.44 | 33.38 |
Across the tested ranges, maximum variation is 1.21 points on VLM4D and 1.25 on DSR-Bench. The response-local weight rankings are also stable under changes in and : Spearman correlations range from 0.9841–0.9944 at initialization and 0.9889–0.9955 at the trained checkpoint; Jaccard similarities range from 0.9059–0.9778.
Theoretical concentration is controlled as well. Corollary 1 shows that, with and ,
Since and , the raw priority lies in , which yields the bound. The effective number of supervised positions therefore remains at least 81.6% of the response length, characterizing a controlled reallocation of dense supervision.
W3. Counterfactual probe semantics and generality
The three probes isolate distinct evidence requirements of dynamic spatial reasoning:
- Spatial masking (V): masks the same spatial patches across frames, weakening object-identity and reference-point cues.
- Temporal-order perturbation (O): reverses or locally reorders observations, testing temporal direction and event order.
- Local-motion suppression (M): repeats a representative frame within four-frame windows, preserving coarse scene progression while suppressing fine displacement, speed, and orientation changes.
The semantic-by-channel results in Figure 3(a) show content-dependent responses: forward has spatial/order evidence (+0.44/+0.26), moving local-motion evidence (+0.38), closer spatial evidence (+0.94), and before order evidence (+0.46). Relational judgments use multiple channels: approach has motion/order evidence (+1.10/+1.24), and turn has (+0.63/+0.88). These channel-specific responses reveal semantic differentiation across token categories.
Figure 4 quantifies each channel's contribution to response-local allocation: removing the spatial, motion, or order channel decreases the G-only KL allocation uplift by 2.762, 2.084, and 1.445 percentage points, respectively. Figure 6 provides a word-level check: grounding enrichment over matched format tokens is +0.15 for moving, +0.17 for static, and +0.27 for both forward and turn, with positive 95% bootstrap intervals.
We further compare spatial masking against mild photometric perturbations:
| Group | Spatial | Photometric | 95% CI | |
|---|---|---|---|---|
| All 90 | 0.1367 | 0.0314 | +0.1053 | [0.0695, 0.1459] |
| New 32 | 0.1583 | 0.0344 | +0.1240 | [0.0665, 0.1930] |
Spatial masking produces a larger evidence response than mild photometric changes across both sample groups, with positive confidence intervals.
To test transfer beyond specialized spatial benchmarks, we additionally train G²-OPD using general video data and evaluate the resulting Qwen3-VL-2B on Video-MME:
| Benchmark | Base Qwen3-VL-2B | G²-OPD-trained Qwen3-VL-2B | Improvement |
|---|---|---|---|
| Video-MME | 61.9% | 66.7% | +4.8 pp |
This result provides additional evidence that the training strategy can benefit broader video understanding, beyond the specialized dynamic-spatial tasks. Together, semantic-by-channel responses, word-level enrichment, channel ablation, photometric controls, and Video-MME support the probes as structured measures of spatiotemporal evidence dependence.
Response to Reviewer kkf4 (3/3)
Q1. Methodological distinction and simpler weighting baselines
G²-OPD uses the student's own reasoning trajectory to distinguish disagreements about the video from disagreements about wording. Consider two positions with the same teacher–student KL divergence. One concerns whether an object is approaching or departing. The other concerns how to phrase the explanation. KL weighting gives the two positions the same priority based on disagreement, and entropy weighting gives them the same priority if the student is equally uncertain. Random weighting can assign different weights, but the difference carries no information about which judgment depends on the video.
The frame-order probe gives G²-OPD a way to distinguish the two positions even when their KL divergence is the same. If disrupting frame order affects the teacher's score at the direction judgment more than at the wording choice, that difference can raise the relative grounding weight of the direction judgment. Candidate support then helps determine the strength of the correction. Two equally uncertain positions can receive different graspability weights if the teacher's probability mass overlaps more with the student's candidates at one position. The joint weight uses both distinctions, retaining full-vocabulary reverse-KL supervision at every position and keeping the total weight per response fixed.
Comparison with simpler weighting baselines
| Method | VLM4D | DSR-Bench |
|---|---|---|
| Uniform OPD | 46.20 | 28.64 |
| Random weighting | 46.70 | 29.20 |
| Entropy weighting | 47.19 | 29.69 |
| KL weighting | 47.96 | 30.31 |
| Grounded-OPD | 49.17 | 31.92 |
| Graspable-OPD | 48.46 | 31.29 |
| G²-OPD | 51.38 | 34.49 |
Under matched response-wise normalization and weight-concentration constraints, G²-OPD improves over KL weighting by 3.42 and 4.18 percentage points on VLM4D and DSR-Bench, respectively. It also improves over Grounded-OPD by 2.21 and 2.57 points and over Graspable-OPD by 2.92 and 3.20 points. The KL comparison measures the gain over weighting by disagreement alone, and the single-component comparisons show the benefit of using both signals on the same trajectory.
Q2 & Q3. Generalization across teacher–student pairs and token categories
We evaluate an additional pair, Qwen2.5-VL-32B → Qwen2.5-VL-3B, with verified vocabulary alignment:
| Model | Role | VLM4D Overall | DSR-Bench Micro |
|---|---|---|---|
| Qwen2.5-VL-32B | Teacher reference | 53.03% | 30.31% |
| Qwen2.5-VL-3B | Base student | 42.29% | 19.79% |
| Qwen2.5-VL-3B | G²-OPD-trained student | 47.58% | 24.88% |
| Improvement over base | +5.29 pp | +5.09 pp |
G²-OPD consistently improves both benchmarks with this additional teacher–student pair, extending the evidence beyond the original MLLM-4D/Qwen3-VL setting to Qwen2.5-VL.
The objective is defined over aligned next-token distributions in the shared vocabulary. Physical information serialized in the response—direction, distance, depth, and numerical descriptions—is supervised within this same full-vocabulary objective.
The original semantic-by-channel analysis identifies differentiated evidence for direction/trajectory (forward: V +0.44, O +0.26), motion state (moving: M +0.38), distance/depth (closer: V +0.94), speed (faster: M +0.12, O +0.54), temporal order (before: O +0.46), relative motion (approach: M +1.10, O +1.24), and rotation (turn: M +0.63, O +0.88). At the complete-word level, moving, static, forward, and turn have positive enrichment over matched format tokens (+0.15, +0.17, +0.27, and +0.27, respectively).
We also check token-level transfer on the Qwen2.5-VL-32B → Qwen2.5-VL-3B pair. Direction/trajectory tokens again show positive enrichment under spatial evidence (+0.484, 95% CI [0.233, 0.772]) and temporal-order evidence (+0.301, 95% CI [0.094, 0.514]). Motion-state tokens have a positive local-motion estimate (+0.208, 95% CI [−0.003, 0.448]), although this interval slightly overlaps zero. These statistics indicate which token categories receive greater supervision priority.
Revision: The revised manuscript adds cost and teacher-forward-budget comparisons (Section 5.8, Tables 7 and 11), weighting controls (Section 5.3, Table 5), parameter sensitivity (Appendix D.5, Tables 13–14), and cross-model accuracy and token analyses (Appendix D.6, Tables 15–16).
Response to Reviewer kkf4 (1/3)
Thank you for recognizing the importance of dynamic spatial reasoning, the interpretability of our weighting signals, and the gains achieved by G²-OPD. We clarify the computational trade-off, quantify hyperparameter sensitivity, and provide additional evidence for probe selectivity, broader video understanding, and cross-model generalization. Counterfactual probes shape the training-time allocation; deployment retains the original student inference architecture and the original full-vocabulary reverse-KL teacher target.
W1. Computational overhead and matched teacher-compute comparison
G²-OPD evaluates four frozen-teacher conditions (the original video and three counterfactual views) on the same student-generated response and prefixes. The student generates only once; the additional computation is teacher-side scoring. The student still performs one backward pass per microbatch, with 42 gradient-accumulation microbatches per rank per optimizer update.
Compared with Uniform OPD, G²-OPD improves by 5.18 points on VLM4D and 5.85 points on DSR-Bench. Teacher-specific FLOPs increase 4×, while total training and rollout GPU-hours increase 47.37% (220.73 → 325.30), because student generation and optimization are shared. Peak memory increases 4.19%: the frozen teacher is reused sequentially under no-grad execution, and intermediate full-vocabulary logits are staged on CPU, avoiding simultaneous GPU residency of four teacher outputs. Teacher FLOPs and peak memory are measured on four fixed paired samples.
To isolate the benefit of allocation from the extra teacher-forward budget, we compare with 4×-rollout Uniform OPD:
Under the matched teacher-forward budget, G²-OPD retains gains of 2.70 points on VLM4D and 3.97 points on DSR-Bench, quantifying the benefit of allocating supervision across the student-generated trajectory.
W2. Hyperparameter sensitivity and concentration control
We use one shared configuration across VLM4D and DSR-Bench: αspa=αord=αdyn=0.25, γ=0.25, K=16, and gmax=2. The three α's control spatial, order, and motion evidence; γ controls the extra graspability emphasis; K defines the local candidate region. Grounding scores are clipped at their response-wise 95th percentile before positive-score normalization. Teacher support and disagreement are separately scaled by their 5th and 95th percentiles and clipped to [0,1], reducing outlier influence and putting the two signals on comparable scales.