acceptodds
Withdrawn from ICLR 2027

CLIVO: Differentiable Cluster-Based Instruction Mixing with Learned Validation Objectives

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal instruction tuning depends on both the quality of individual examples and their composition in the training set. Target-specific data selection must therefore decide which examples are useful and how much of each type of supervision to include. Sample-level rankings leave this composition implicit, while group-level optimization faces three connected challenges: fine-grained units enlarge the search space, predefined validation sets may obscure target-relevant feedback, and fitted mixture-response predictors introduce additional approximation error. We propose CLIVO, which connects semantic allocation, automatically learned validation objectives, and differentiable mixture search. Semantic clusters give distinct instruction contents independently adjustable training shares. To evaluate these choices, CLIVO relates individual examples' merged-proxy losses to real-training probe scores, then selects and weights validation examples to form a target-specific objective. Differentiating this objective through reusable expert-merged proxies updates all shares jointly without a separate response predictor. Selecting 100K instructions from an 8M pool, CLIVO improves the four-target average over the best-performing state-of-the-art data selection baseline by 1.55 points; combining its quotas with ICONS increases this advantage to 1.94 points. Ablations show that moderate-budget experts preserve useful feedback, finer allocation benefits most targets, and sample-level weighting exploits target-relevant response differences obscured by cluster averaging. Semantic visualizations and reusable, modular components further support interpretable and flexible target-specific data selection.

Primary area:
foundation or frontier models, including LLMs
Keywords:
vision-language models, multimodal instruction tuning, data selection, data mixture optimization, learned validation objectives, model merging

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DoH5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper uses the CLIVO method to combine semantic allocation, automatically learned validation objectives, and differentiable mixture search, enabling useful feedback to be retained with a moderate budget and thereby improving model performance.

Critical strengths
  1. The paper has a comprehensive framework, is written transparently, and provides detailed experimental records
  2. The paper divides data selection into Unit, Signal, and Learning, clearly breaks down the problem, and can be combined with existing methods
Critical weaknesses
  1. It is unreasonable for the paper to use the final evaluation benchmark as the basis for data selection, as this may cause training to overfit the benchmark scores. The method’s effectiveness in real-world applications therefore remains to be confirmed.
  2. The overall data-selection process is complex and involves extensive probe training. The choice of parameters, such as the value of K in K-means, also lacks detailed analysis.
  3. The total training compute is unclear. Final training consists of 100 optimization steps, and the reported s0 experiment has an effective global batch size of 32. If this applies to all final models, each model sees only about 3,200 samples, far fewer than 100K; the recipe for Full(8M) is not specified.
  4. The writing is unclear, and some passages and explanations of the formulas are lengthy and difficult to understand.
Questions for authors

See Weakness above.

Reviewer_gUmGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a data selection and mixing framework, named CLIVO, for VLM instruction tuning. The framework partitions the instruction pool into hundreds of semantic clusters and uses linear combinations of a set of lightweight Region Experts as a proxy model. It demonstrates that through a differentiable objective, it can find the optimal mixture weights of datasets to improve performance on downstream tasks. Experiments show that under a 100K data budget, CLIVO outperforms existing methods across four downstream tasks.

Critical strengths

Valid Motivation: The paper observes that in data selection, macro-level allocation plays a more important role than the micro-level ranking of individual data samples.

Innovative Differentiable Framework: The design of the differentiable search framework is novel and clever.

Good Extensibility: The framework is flexible and can be seamlessly integrated with existing sample-level methods like ICONS.

Critical weaknesses

Weaknesses Although the method is neat, the paper fails to demonstrate its generalizability, scalability, and feasibility for large-scale adoption. Moreover, the current evaluation has several notable limitations:

Lack of Multi-Model Evaluation: The method is evaluated exclusively on the Qwen2.5-VL-7B model. Results on a single model are insufficient to prove generalizability across different architectures (such as LLaVA or InternVL) or varying parameter scales (e.g., 13B/34B). It remains unclear whether the first-order model merging assumption holds in those settings.

High Cold-Start Cost: Although gradient descent reduces search time, the method incurs a heavy upfront initialization cost to train 100 cluster experts and 128 probe models. Furthermore, all these cluster experts and probe models must be retrained whenever the base model evolves. Such high initialization overhead significantly limits its practicality for large-scale industrial adoption.

Validity of Linear Merging: There is insufficient theoretical grounding for linear parameter merging. The paper lacks analysis explaining why linear merging is appropriate here; on the contrary, Figure 2 shows that expert models can only be trained for very few steps (e.g., 10 steps). If trained longer (e.g., a full schedule), parameters deviate from the local tangent space, causing the ranking fidelity to drop sharply. This indicates that linear merging may break down when applied to larger models or longer SFT schedules.

Lack of Data Scaling Analysis: The evaluation is restricted to a fixed 100K dataset budget. It remains unknown whether scaling the budget to 1M or more would yield consistent accuracy gains without saturating the core semantic clusters.

Questions for authors

Multi-Model Evaluation: Do the authors plan to provide evaluation results across diverse model architectures (e.g., LLaVA/InternVL) and larger parameter sizes to verify whether the first-order linear merging assumption still holds?

Data Scaling Validation: Can the authors validate whether CLIVO maintains its performance superiority under larger data budgets (e.g., 500K, 1M, or full pools) where core semantic clusters may saturate?

Longer Training Validation: Given that experts are restricted to short schedules (10 steps) to maintain merging fidelity, how does the approach perform under longer training steps and extended SFT schedules?

Reviewer_MkNqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CLIVO, a target-specific data selection method for multimodal instruction tuning. The authors partition a self-constructed 8M instruction pool into 100 semantic clusters via K-means, with each cluster’s training portion adjustable independently. They then SFT 128 real probe models and use the probes’ downstream scores to screen validation examples by their per-example losses under merged-expert proxies. Learned weights over these examples form a target-specific validation objective. Finally, the cluster shares are optimized by projected gradient descent through the merged proxy, and the resulting quotas are filled by within-cluster sampling to produce a 100K subset.

All experiments use a 7B base model that the authors pretrained and mid-trained themselves, and results are reported as target-level aggregates rather than per-benchmark metrics.

Critical strengths
  • The paper decomposes data selection into three coupled dimensions, namely the allocation unit, the evaluation signal, and the mixture learning rule, and it clearly separates quota allocation from within-cluster selection.
  • Despite its considerable cost, using real-training feedback to automatically select and weight validation examples, instead of relying on a manually specified validation set, is interesting.
Critical weaknesses

The submission discloses that "generative AI assisted with literature search, experiment planning, interpretation of results, and theoretical exposition." The presentation raises concerns about how thoroughly the content was verified: the main-text claims are noticeably stronger than what the appendix supports, and the appendix repeatedly cites "supplied" result summaries.

  1. The fully self-constructed setting is very unique for this line of work, where public pools such as LLaVA-665K or LLaVA-Onevision with public pretrained models are standard, and it is not justified. The authors build their own data pool and pretrain their own base model, with undisclosed initialization and pretraining data, and they train 128 SFT probes and a bank of cluster experts. Yet the evaluation of selection itself is narrow: a single 100K budget, one pool with no deduplication or decontamination, no transfer experiments, no sensitivity analysis for hyperparameters, or the clustering. With no stated release of code, data, or checkpoints, the results cannot be reproduced or verified by the community.

  2. The core components' contributions are not established. Gradient search barely lowers the proxy objective (D_J = 0.056% for Knowledge and 0.293% for Parsing), and the gains over the best probe compare models trained under different budgets. Without training w0 under the same final recipe, the gains may come simply from picking the best of 128 probes.

  3. The comparisons are uneven. CLIVO uses real-training feedback and the evaluation benchmarks themselves as supervision, while most baselines are target-agnostic, re-implemented by the authors, run once, and insufficiently specified. The most relevant mixture-search methods, eg CLIMB, DoReMi, RegMix, MergeMix, are missing.

  4. The efficiency comparison also excludes the costs of training the experts and probes, which is important for this line of research.

  5. The statistical evidence is weak. Most results come from single runs, and CLIVO's standard deviations are comparable to many of the reported gaps.

Questions for authors

Section Appendix B2 states that the s0 allocation runs use an effective global batch size of 32 for 100 optimizer steps, which would expose the model to only about 3.2K of the 100K selected examples. What is the global batch size of the main runs, and is sequence packing or multi-epoch training used? How many steps is the Full (8M) reference trained for?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.