- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes Query-based Submodular functions with Gradient-guided Synthesis (QSGS), a framework for efficient, target-aware instruction tuning that addresses the redundancy of conventional gradient-based data selection. QSGS first computes compact gradient representations and influence scores using a warm-up model, then employs a submodular graph-cut objective to select examples that are relevant to the target task while minimizing redundancy among selected samples. To further improve coverage, it identifies underrepresented target-gradient directions and generates synthetic examples aligned with these residual directions. Experiments on MMLU, BBH, and TyDiQA using Llama-2-7B, Llama-3-8B, and Mistral-7B show that QSGS consistently outperforms existing selection baselines at a 0.5% data budget, using approximately 1,350 examples from a pool of 270,000. Notably, it matches full-data performance on MMLU for Mistral-7B and achieves near-full-data performance across the evaluated benchmarks, demonstrating the benefits of combining target relevance, diversity-aware selection, and gradient-guided synthesis for data-efficient language model fine-tuning.
Critical strengths
- The problem setup and motivation is well presented.
- The writing is overall clear and succinct. I didnt find much AI use or sloppy writings which was positive.
- The methodology for coreset selection using a simple graph cut objective is simple with less costly as compared to other sophisticated methods.
- Strong empirical experimental setups across multiple language models with varying scale of parameters.
Critical weaknesses
- I think this is a very similar repackaged solution to a widely popular problem for SFT Task selection (see etc. TSDS [1], GREATS [2], PartitionSEL [3], etc.). Even LESS has a very similar idea. Hence, the current method employs the same style of tradeoff of relevance / diversity etc. without introducing any new novel ideas.
- One important thing to note is the current empirical results are only on a smaller fraction of the training examples (0.5%) which is a loss less training samples to train on. It
- Appendix is limited to only 1 page, so very less details on further different varying subset sizes are missing.
- The warm up training model still adds in a training overhead towards gpu training hours. The scalability discussions to more future incoming SFT tasks and order of computation details is missing.
[1] TSDS: Data Selection for Task-Specific Model Finetuning
[2] GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration
[3] Minibatch selection for Language Models via Partition Matroid Constrained Gradient Matching
Questions for authors
- How do you choose (diversity/representative tradeoff) for the final table results?
- What is the training time for each task for the warm up model to get the gradients per task samples?
Hi all,
Reviewers: please respond to the authors.