- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a closed-loop data curation framework for MLLM mid-training, combining coverage-preserving semantic compression with learning-guided data expansion. The semantic compression stage constructs an initial training set by jointly considering image-text pair specificity, local semantic diversity, and cluster-level coverage. The subsequent expansion stage dynamically adjusts training data selection based on model feedback, using prediction loss and reference-relative loss gaps to identify semantic regions that may benefit from additional training. By iteratively performing model diagnosis, data acquisition, and continued mid-training, the framework aims to adapt the training data distribution to the model's evolving learning needs. Experiments demonstrate improvements over existing data selection methods under the same training sample budget.
Critical strengths
The paper addresses an important problem in efficient MLLM training: how to reduce redundant training data while adapting data selection to the model's evolving learning needs. The proposed framework combines semantic coverage with model-dependent feedback in a systematic manner. The overall methodology is clearly presented, and the integration of static semantic compression and dynamic data acquisition provides a reasonable approach to improving training data efficiency.
Critical weaknesses
-
The proposed framework introduces considerable computational overhead beyond standard mid-training, including multimodal embedding extraction over millions of samples, semantic clustering and nearest-neighbor retrieval, training two LoRA-based reference models, and repeatedly evaluating a 500K-sample probe set during training. Despite this complexity, the overall performance improvement is relatively modest: the proposed closed-loop method achieves an average score of 64.7, compared with 64.0 for semantic compression alone and 63.2 for random sampling. Without accounting for the additional data curation costs, it remains unclear whether the proposed framework offers a meaningful advantage over simpler data selection strategies or allocating the same computational resources to additional training.
-
The proposed closed-loop framework repeatedly evaluates the model on a fixed probe set and uses the resulting losses to guide subsequent data acquisition. Although the probe samples are excluded from direct training, repeatedly optimizing the training data distribution based on the same probe set may lead to adaptive overfitting to its particular semantic composition. Furthermore, the reference-relative loss gap is not sufficiently justified as an indicator of future learning potential. The reference models are adapted on different folds of the probe set, while the target model has already undergone mid-training on other samples from the same semantic clusters. Consequently, it is unclear whether the observed loss gaps genuinely reflect the marginal benefit of additional training data, rather than differences in model adaptation, training distributions, or reference-model capacity.
Questions for authors
-
How do the authors prevent adaptive overfitting to the fixed probe set during closed-loop data selection? Although the probe samples and their near-duplicates are excluded from target-model training, the same probe set is repeatedly used to determine the semantic distribution of subsequently acquired training data. This creates a risk that the data selection strategy progressively adapts to the specific characteristics of the fixed probe set rather than the model's general learning needs.
-
Why should the reference-relative loss gap be considered a reliable measure of future learning potential? The reference models are adapted using two-fold cross-fitting on the probe set, whereas the target model has already been trained on other samples belonging to the same semantic clusters. Therefore, the reference-relative loss gap does not necessarily represent the additional improvement achievable through further mid-training.
4 comments
Dear Reviewer 1wpd,
We greatly appreciate your careful review and constructive criticism. In particular, your concerns about the fixed probe set and the reliability of the reference-relative loss gap prompted us to provide further analysis and controlled experiments. We hope these additions have helped clarify our approach, and we would be glad to hear if there are still any unresolved issues worth discussing. Should you consider our responses sufficient to address your main concerns, we would be grateful if you could consider raising your score.
Many thanks for your continued consideration of our work.
The authors.
After reviewing the authors' responses, I have decided to maintain my original rating.
Regarding Q1 and Q2, the responses largely reiterate the explanations already presented in the paper, without providing new evidence that directly addresses my concerns.
Regarding Q3, a meaningful compute-matched comparison should at least consider simpler data selection strategies rather than random sampling. Moreover, it is unreasonable to conclude that having additional computational resources would actually cause a method's results to deteriorate. Therefore, I do not find this additional experiment convincing evidence of a favorable compute–performance trade-off.
We thank the reviewer for the further feedback. Your comments have helped us identify which concerns in our initial response require more direct experimental evidence. We address each point below.
Q1: Does repeatedly using a fixed probe set lead to adaptive overfitting?
Response: The design mechanisms described in our initial response primarily aim to prevent direct data leakage and limit excessive concentration of data selection driven by probe feedback. To examine whether the improvements guided by such feedback generalize to new samples, we evaluated the same 3M, 4M, and 5M checkpoints on both the original probe set and an independent held-out probe set. We constructed the held-out probe set using samples from the same semantic clusters that were not used for diagnosis, reference model adaptation, or target model training.
| Expansion round | Original probe mean ΔLoss | Independent probe mean ΔLoss | Retained reduction |
|---|---|---|---|
| 3M → 4M | 0.0548 | 0.0536 | 97.8% |
| 4M → 5M | 0.0223 | 0.0219 | 98.2% |
Across the two expansion rounds, the independent probes retained 97.8% and 98.2% of the mean loss reductions observed on the original probes, respectively. These results provide more direct evidence that the improvements observed on the fixed probes are not limited to the repeatedly used probe samples, but largely transfer to unseen samples. They also indicate that we did not observe substantial overfitting to the original probe samples in the two evaluated expansion rounds.
Q2: Can the reference-relative loss gap indicate subsequent learning gains?
Response: In our rebuttal to iVKz, we compared high/low gap clusters at similar initial mean prediction difficulty (). Each group received the same number of additional samples, selected using the same within cluster semantic ranking rule. The mean loss reductions illustrate a 25.8% larger reduction in the high gap group. This comparison provides that the gap is associated with subsequent loss reduction. We therefore treat the gap as an empirical signal of learning potential.
Q3: Does the closed-loop method offer a favorable compute–performance trade-off compared with simpler data selection strategies?
Response: We acknowledge that the experiment involving additional randomly selected data does not isolate a single variable and is therefore insufficient to establish the compute–performance trade-off of the closed-loop method. In our response to reviewer iVKz, we also compared our method with Herding, a simpler selection baseline. Herding incurs an end-to-end cost of 611.6 GPU·h, approximately 1.6% higher than the 602.1 GPU·h by our closed-loop method. Their Overall scores are 63.33 and 64.73, respectively. Since the measured computational costs are comparable, these results provide a useful reference for evaluating the compute–performance trade-off.
Meanwhile, to more directly compare two different uses of the same computational budget, we are conducting an additional experiment starting from the mid-training checkpoint of the original 5M Compression setting. We continue training with additional data selected using the same selection algorithm and principles as the Semantic Compression stage described in the paper. We keep the SFT and evaluation settings unchanged and use the end-to-end GPU cost of the closed-loop method as the target computational budget. Due to time and computational resource limits, this experiment may not be completed before the rebuttal deadline. We will include the corresponding experimental results and conclusions in the paper's appendix.
We thank the reviewer again for pointing out the limitations of these comparisons. We appreciate the opportunity to clarify the scope of the existing results and provide additional evidence addressing each concern.
Response to Reviewer 1wpd
We sincerely appreciate your positive comments for our work! We set out below our responses to each of the questions.
Q1: How do the authors prevent adaptive overfitting to the fixed probe set during closed-loop data selection?
Response: The probe-set design considers two risks: data leakage and potential overfitting to a fixed probe set. To address the reviewer’s concern about overfitting, we summarize the safeguards already described in the paper:
The ablation in Figure 5 shows that combining diagnosis-guided selection (setting C) with coverage-preserving selection (setting B) yields better downstream performance than either strategy alone.
Q2: Why should the reference-relative loss gap be considered a reliable measure of future learning potential?
Response: Firstly, a high loss for the target model may reflect either samples that the model has not yet learned and could benefit from further training, or samples that are difficult to learn effectively under the current training conditions, such as noisy samples. Inspired by [1, 2], we introduce the reference-relative loss gap as an additional signal to help distinguish these cases. Two design choices support the reliability of this signal: the reference model is adapted from the base checkpoint and then kept fixed, reducing the influence of the mid-training data distribution on the baseline; and two-fold cross-fitting ensures that each probe’s reference loss comes from a model that was not trained directly on that probe, reducing direct-fitting bias. Figure 5 and Appendix Table 7 provide empirical evidence for the effectiveness of this design: using only the current loss (setting D) or only the reference gap (setting E) performs worse than combining both (setting F). Setting F outperforms D and E across General, Text-rich, Robustness, and MME Total.
Q3: Does the proposed framework still offer an advantage over using the same computational resources for additional training?
Response: Based on our computational cost measurements, our method requires 65.9 GPU·h on a single A800 GPU for data selection and model diagnosis (see iVKz-Q1 for the detailed cost breakdown). This is approximately equivalent to the GPU time required to train on 2M additional samples. To test whether using this compute for more training data yields comparable gains, we added a random-data baseline: starting from the 5M-sample Compression setting, we randomly add 2M samples to obtain a 7M-sample mid-training setting, while keeping the subsequent SFT setup unchanged. The results are shown below.
Adding 2M random samples reduces Overall performance from 63.96 to 63.48 relative to Compression alone, suggesting that the additional data may contain noisy or less useful samples. Under an approximately equal GPU budget, Closed loop also achieves better overall performance than the 7M setting. These results indicate that how additional compute is used matters more than simply increasing the training sample count in this setting. We thank the reviewer for suggesting this valuable comparison and will add the results and discussion to the experimental section.
We will continue to monitor the follow-up questions and discussions of the reviewer.
Reference
[1] Sören Mindermann et al. “Prioritized Training on Points That Are Learnable, Worth Learning, and Not Yet Learnt.” ICML, 2022.
[2] Sang Michael Xie et al. “DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining.” NeurIPS, 2023.