- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes DOVAS, a dynamic data-allocation framework for multimodal instruction tuning. It first partitions the training corpus into fixed regions using instruction semantics and offline scores measuring answer compatibility, visual dependence, and correct-image specificity. During training, it periodically estimates how regional optimizer updates would affect held-out answer and visual counterfactual objectives, then adjusts regional sampling probabilities accordingly.
Experiments on LLaVA-665K and Vision-Flan-186K, with additional evaluations on Qwen models, report competitive or better aggregate performance than full-data training under a 50% online compute budget that includes dynamic selection but excludes offline preparation. Ablations examine region construction, proxy objectives, optimizer-aligned estimation, and dynamic versus fixed allocation, while repeated runs support the method’s consistency relative to random sampling and its static variant.
Critical strengths
A distinctive and well-motivated allocation design. DOVAS combines fixed visual-evidence regions with optimizer-aligned utility estimates based on answer and visual counterfactual objectives. This provides a practical way to adapt the training mixture without repeatedly scoring the entire corpus. The combination offers a meaningful methodological contribution beyond static visual-dependence scoring.
Strong empirical results with substantive supporting controls. The method achieves competitive or better aggregate performance than full-data training under half the online compute budget, with improvements across many individual benchmarks. Comparisons against fixed mixtures, raw-gradient estimation, and reduced objective sets support the main design choices, while repeated runs demonstrate consistent gains over random sampling and the static variant. These results make the approach promising beyond its conceptual motivation.
Critical weaknesses
-
The benefit of real-time feedback is not fully isolated from that of a time-varying curriculum. The initialized and average-mixture controls support changing the allocation during training, but both remove temporal scheduling as well as feedback from the current model state. They therefore do not establish whether ongoing utility estimation is needed, or whether replaying a previously obtained allocation schedule would achieve similar gains. This distinction matters because state-dependent allocation is a central contribution and incurs additional online computation.
-
The connection between proxy utility and generalization remains only partially validated. The update-level diagnostics show that predicted utility tracks changes in a proxy objective, while the downstream ablations support the usefulness of the proposed objectives overall. However, neither directly establishes that regions ranked more highly at a given checkpoint produce larger improvements on independent downstream tasks. Such evidence would strengthen the interpretation that the allocator tracks changing training value, rather than simply optimizing a useful but imperfect surrogate.
Questions for authors
I would like to understand whether the gains require feedback from the current model state, or whether a predetermined, time-varying curriculum would suffice. Could you record the regional sampling probabilities from one DOVAS run and replay that schedule under different training seeds, without online utility estimation? Comparing replay with live DOVAS at matched parameter-update counts would help isolate the contribution of feedback; a matched-online-compute comparison would show whether that contribution justifies its overhead.
The utility diagnostics establish agreement with changes in the proxy objective, but I am less certain how well the resulting regional rankings predict generalization gains. At a few checkpoints, could you start from the same model and optimizer states, briefly train on each region separately, and compare the observed improvements on independent validation tasks with the predicted utilities? This would directly test whether regions assigned higher utility are also more beneficial beyond the proxy objectives.
2 comments
Thank you for the detailed response. My concerns are partially addressed, I will remain my positive score.
Thank you for dedicating your time and effort to reviewing our paper. We are deeply encouraged by your positive comments and recognition of our work. Below, we provide detailed responses to address your concerns.
We thank the reviewer for this comment. The initialized-mixture and average-mixture controls in Table 9 show that fixed allocations do not recover the performance of DOVAS. Following the reviewer’s suggestion, we additionally conduct a cross-seed sampling-schedule replay experiment.
Replay retains the time-varying regional probabilities recorded from a DOVAS run, but does not estimate utility from the current state of the target run. It therefore preserves the temporal curriculum while removing current-state feedback.
Let K denote the number of parameter updates performed by DOVAS. Both target runs perform K updates with a global batch size of 16, yielding 16K cumulative training-example visits, and use the same learning-rate schedule.
This result supports the additional value of current-state feedback in the evaluated cross-seed comparison.
We thank the reviewer for this suggestion. We introduce a regional short-training experiment that starts each branch from the same model and optimizer states and directly measures the gains obtained on independent GQA and TextVQA validation sets.
The experiment uses LLaVA-v1.5-7B and LLaVA-665K, with checkpoints selected at approximately 10%, 50%, and 90% of the online compute budget. At each checkpoint, we record the six regional utility scores actually used for allocation. We then initialize separate branches from the same model, AdamW, and learning-rate scheduler states, and train each branch exclusively on one region for 100 parameter updates. All branches are evaluated before and after training on the same fixed, independent GQA and TextVQA validation sets.
Let θt denote the checkpoint model and θt,b100 the model obtained after 100 updates on region b. Let SG and ST denote GQA accuracy and the official TextVQA VQA accuracy, respectively, both expressed on a 0–100 scale. The observed gain is defined as:
Δt,b=21[SG(θt,b100)−SG(θt)+ST(θt,b100)−ST(θt)].
Thus, Δt,b is the arithmetic mean of the accuracy improvements on the two tasks, measured in percentage points.
We use the actual allocation score:
Ut,b=At,b+Vt,b+St,b,
and calculate the within-checkpoint rank correlation:
ρt=Spearman([Ut,0,…,Ut,5],[Δt,0,…,Δt,5]).
Strong positive rank correlations are observed at all three checkpoints, indicating that regions assigned higher predicted utility generally produce larger gains on the independent tasks. The highest-predicted-utility region also outperforms the random control at each stage, providing further empirical support for the ability of the utility score to identify regions with greater local training value.