- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Goal-Driven Data Optimization (GDO) for multimodal instruction tuning. GDO computes six descriptors for each training example and constructs goal-specific subsets through a shared scoring function and different feasibility constraints. The method is evaluated with Qwen3-VL-8B-Instruct on several video understanding benchmarks.
Critical strengths
-
The paper studies an important practical problem: allocating a limited multimodal instruction-tuning budget across heterogeneous image and video data.
-
The paper is unusually transparent about its experimental protocol and includes subtask analyses and component ablations.
Critical weaknesses
-
The main baseline is not a matched control. For each benchmark, the paper uses the lowest-scoring endpoint among four historical 512k-sample uniform runs as the reference, which systematically favors GDO and makes the reported gains difficult to interpret.
-
The “faster convergence” claim is not based on directly evaluated sample counts. The reported Peak Match values use interpolated, post-processed trajectories with one checkpoint removed and pseudo-sample-count positions, so the claimed 14–19× reductions are not directly measured.
Questions for authors
-
Can the authors compare GDO against fresh uniform/random subsets under exactly the same training budget and random seeds?
-
Can the authors report actual evaluated checkpoints at the claimed Peak Match sample counts instead of interpolated values?
-
How do the conclusions change if a single fixed uniform baseline is used for all benchmarks rather than the weakest branch selected separately for each benchmark?