acceptodds
Withdrawn from ICLR 2027

MM-DataLoop: Closing the Data-Side Loop for Recursive Self-Improvement in MLLM Post-Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recursive self-improvement (RSI) offers a path toward automated training loops for large models. Existing research focuses primarily on the model side, while data preparation remains a labor-intensive part of MLLM post-training. Data-side methods typically address individual operations, leaving the composition of heterogeneous interventions and the accumulation of cross-round experience less explicitly organized. We present MM-DataLoop, an end-to-end iteration loop for MLLM post-training data. It groups data operations into five families and runs an Observe Diagnose Plan Execute Verify Reflect cycle. A global rule memory carries experience into subsequent plans. We hold the base-model initialization, training hyperparameters, and evaluation protocol fixed while varying the training data and its mixture. This contract makes data interventions comparable and auditable across rounds. We run 30+ rounds on Qwen3.5-0.8B across MMBench, ChartQA, MMMU, MathVista, and RealWorldQA. The five-benchmark average improves from 57.72% zero-shot to 62.95% (5.32 pp). The study also yields reusable empirical findings about filtering and data composition. For example, removing different subsets flagged by one groundedness filter produces opposite effects. Some subsets judged low-quality also provide useful visual diversity, and removing them reduces performance in the tested recipes. We will release the code, data, and empirical findings to support further data-side research.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Recursive Self-Improvement, Post-Training

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zt1GOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes MM-DataLoop, a framework for iteratively improving multimodal post-training data through five operation families and an Observe → Diagnose → Plan → Execute → Verify → Reflect cycle. A global rule memory preserves experience from successful and unsuccessful experiments. Importantly, each candidate is trained from the same base checkpoint with a fresh LoRA adapter; the accumulated state consists of data recipes, experiment history, and rules, rather than progressively updated model weights. The paper reports a 31-round study on Qwen3.5-0.8B, with the best configuration increasing the five-benchmark average from 57.72% to 62.95%. It also documents recipe-dependent findings concerning filtering, synthetic supervision, and data composition. The contribution is primarily an integrated experimental workflow and an empirical case study of data-side iteration.

Critical strengths
  1. Restarting each round from the same base model helps distinguish data-recipe changes from cumulative weight updates. The explicit acceptance rules, integrity checks, and experiment records provide a useful structure for auditing the search.
  2. Removing different groups identified by the same groundedness filter produces opposite effects, illustrating why a filtering criterion should not be treated as a universally reliable measure of training utility.
  3. The paper acknowledges that many round-to-round comparisons are not single-variable causal tests and that repeated evaluation of a checkpoint does not measure training or data-generation variability.
Critical weaknesses
  1. My main concern is the distinction between finding a better data recipe and demonstrating that MM-DataLoop is a better way to find that recipe. The improvement over the zero-shot model supports the usefulness of the resulting training data, but does not isolate the benefit of diagnosis, operation composition, reflection, or global memory.
  2. The same benchmarks guide planning and evaluate the resulting candidates. Without an untouched final evaluation, the reported gains may partly reflect adaptation to the benchmarks driving the search.
  3. Same-checkpoint reevaluations do not capture training or generation variability. Moreover, changing dataset sizes under fixed epochs changes training exposure, making filtering effects difficult to isolate without matched-size and matched-compute controls.
Questions for authors
  1. What does the closed-loop framework contribute beyond ordinary data-recipe search?
  2. How is generalization evaluated independently of the benchmarks driving the loop?
Reviewer_AZJCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MM-DataLoop proposes an iterative framework for improving multimodal post-training data. It organizes data operations into five families and connects them through an Observe–Diagnose–Plan–Execute–Verify–Reflect cycle, with persistent rule memory and verification gates. Each round trains a fresh LoRA adapter from the same base checkpoint while keeping training hyperparameters and evaluation procedures fixed. A 31-round study on Qwen3.5-0.8B improves the best five-benchmark average from 57.72% to 62.95%. Improvements concentrate in MMBench, ChartQA, and RealWorldQA; MathVista remains approximately unchanged, and MMMU declines. The paper also reports observations about selective filtering, synthetic supervision, and mixture composition.

Critical strengths
  1. Transparent reporting of a substantial experimental trajectory.

The paper records unsuccessful interventions and benchmark regressions alongside improvements. Restarting each training run from the same checkpoint and maintaining consistent evaluation procedures makes recipe comparisons easier to interpret. The authors also explicitly distinguish evaluation repetitions from independent training runs and acknowledge several attribution limitations.

  1. Useful practical observations about filtering granularity.

The comparisons suggest that selective removal and whole-subset exclusion can have substantially different effects. For example, removing flagged ScienceQA samples improves the average relative to R23, whereas subsequently removing the remaining ScienceQA samples reduces performance relative to R29. These observations provide useful hypotheses for data curation and caution against treating teacher judgments as universally reliable indicators of downstream utility.

Critical weaknesses

1.The methodological novelty is insufficiently established relative to close prior work.

The six-stage cycle largely decomposes an established experimentation workflow into evaluation, error analysis, planning, execution, validation, and experience retention. DataEnvGym[ICLR 2025] already connects student weaknesses to iterative data planning, generation, training, and evaluation, including VQA. MM-DataLoop’s operation taxonomy, diagnostic vocabulary, and scoped rule lifecycle may provide useful organization, but the paper does not clearly identify a distinct technical mechanism or demonstrate why this particular integration advances existing approaches.

  1. The evaluation does not convincingly establish the effectiveness or generalization of the proposed framework.

The main evidence is one search trajectory compared against the original checkpoint, without a budget-matched alternative search policy or controlled ablations of diagnosis and memory. Consequently, the reported improvement cannot establish the framework’s added value beyond post-training and recipe optimization. Moreover, the same benchmarks guide diagnosis and planning and determine the best reported configuration; no untouched final evaluation is provided. Repeated evaluation of existing checkpoints does not quantify training or generation variability, leaving smaller gains—R23 over R17 (+0.42 points) and R29 over R23 (+0.49 points)—without adequate uncertainty estimates. Changing dataset sizes under fixed epochs also changes training exposure, complicating attribution. Although the authors acknowledge these limitations, they remain central gaps in validating the paper’s principal contribution.

Questions for authors

What specific mechanism in the planning or rule-update procedure distinguishes MM-DataLoop from existing feedback-driven data workflows and reflection-based agents?

Reviewer_5FBoOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents MM-DataLoop, an iterative framework for multimodal LLM post-training data. It organizes five operation families within an Observe, Diagnose, Plan, Execute, Verify, Reflect cycle. Persistent rule memory is intended to carry experience into subsequent planning. The empirical validation consists of a 31-round study using Qwen3.5-0.8B on five multimodal benchmarks. The best observed configuration improves the average from 57.72% to 62.95%. The paper also reports setting-specific findings about targeted synthesis, filtering granularity, and data-mixture trade-offs. Its principal contributions are the data-side iteration framework and an empirical record of how successive data interventions affect performance.

Critical strengths
  1. The paper conducts a relatively long sequence of 31 iterations and provides interesting observations about how different data interventions affect model performance. These findings offer useful insights into iterative data optimization.

  2. The framework maintains a unified evaluation setting across iterations, making it easier to compare different interventions and analyze performance changes.

Critical weaknesses
  1. The framework appears to use feedback from the target evaluation benchmarks to guide data selection and synthesis. Repeated optimization against these benchmarks may lead to benchmark-specific overfitting rather than genuine generalization. Evaluation on held-out benchmarks would strengthen the conclusions.

  2. Although this paper focuses on data optimization, the effectiveness of data is often closely related to the underlying model and training method. Using only one model and a fixed LoRA-SFT approach limits the generalizability of the findings and the flexibility of the framework. Moreover, all iterations start from the same base model, without exploring whether continuing training from previously improved checkpoints could enable more effective and cumulative self-improvement.

Questions for authors
  1. Which specific LLM is used for Diagnose and Plan? What are the total token consumption and associated costs?
  2. How does the system determine when to stop iterating? Is this decided automatically or manually?
  3. How much human intervention is required throughout the framework? Are there any examples where the system autonomously discovered effective strategies or insights that would be difficult for human researchers to come up with?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.