acceptodds
Under review as a conference paper at ICLR 2027

Query-based Submodular Data Selection for Efficient Instruction Tuning

Abstract

As large language models are fine-tuned for increasingly specialized tasks, their performance depends more on which training examples are selected than on how many. Given a large heterogeneous training pool and a small target set specifying the desired capability, targeted data selection aims to find a compact subset that maximizes downstream performance. Existing methods score each candidate independently against the target and select the top-ranked examples, yet this point-wise strategy ignores inter-example interactions: at tight budgets the selected subset is dominated by near-redundant points that waste capacity on overlapping information. We propose Query-based Submodular functions with Gradient-guided Synthesis (QSGS), which formulates targeted selection as a submodular graph cut over a gradient similarity graph. A modular relevance term pulls selected examples toward the target task, while a pairwise dispersion penalty suppresses intra-subset redundancy. The resulting objective balances target relevance and intra-subset redundancy using the same formulation across different selection budgets. Empirically, the benefit of diversity-aware selection becomes increasingly pronounced as the data budget decreases. To further close coverage gaps that no selection from existing data can bridge, we introduce a gradient-guided synthesis step that identifies under-covered target directions via residual subspace analysis and generates synthetic examples to fill them. Experiments on MMLU, BBH, and TydiQA across Llama-2, Llama-3, and Mistral, show that QSGS at a 0.5% budget matches or exceeds full-data performance on Mistral-7B, outperforms the strongest baseline across all benchmarks, and yields increasing gains as the budget shrinks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Data selection, Instruction Tuning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dxhLOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Query-based Submodular functions with Gradient-guided Synthesis (QSGS), a framework for efficient, target-aware instruction tuning that addresses the redundancy of conventional gradient-based data selection. QSGS first computes compact gradient representations and influence scores using a warm-up model, then employs a submodular graph-cut objective to select examples that are relevant to the target task while minimizing redundancy among selected samples. To further improve coverage, it identifies underrepresented target-gradient directions and generates synthetic examples aligned with these residual directions. Experiments on MMLU, BBH, and TyDiQA using Llama-2-7B, Llama-3-8B, and Mistral-7B show that QSGS consistently outperforms existing selection baselines at a 0.5% data budget, using approximately 1,350 examples from a pool of 270,000. Notably, it matches full-data performance on MMLU for Mistral-7B and achieves near-full-data performance across the evaluated benchmarks, demonstrating the benefits of combining target relevance, diversity-aware selection, and gradient-guided synthesis for data-efficient language model fine-tuning.

Critical strengths
  • The problem setup and motivation is well presented.
  • The writing is overall clear and succinct. I didnt find much AI use or sloppy writings which was positive.
  • The methodology for coreset selection using a simple graph cut objective is simple with less costly as compared to other sophisticated methods.
  • Strong empirical experimental setups across multiple language models with varying scale of parameters.
Critical weaknesses
  • I think this is a very similar repackaged solution to a widely popular problem for SFT Task selection (see etc. TSDS [1], GREATS [2], PartitionSEL [3], etc.). Even LESS has a very similar idea. Hence, the current method employs the same style of tradeoff of relevance / diversity etc. without introducing any new novel ideas.
  • One important thing to note is the current empirical results are only on a smaller fraction of the training examples (0.5%) which is a loss less training samples to train on. It
  • Appendix is limited to only 1 page, so very less details on further different varying subset sizes are missing.
  • The warm up training model still adds in a training overhead towards gpu training hours. The scalability discussions to more future incoming SFT tasks and order of computation details is missing.

[1] TSDS: Data Selection for Task-Specific Model Finetuning

[2] GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration

[3] Minibatch selection for Language Models via Partition Matroid Constrained Gradient Matching

Questions for authors
  • How do you choose λ\lambda (diversity/representative tradeoff) for the final table results?
  • What is the training time for each task for the warm up model to get the gradients per task samples?
Reviewer_s88VOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes QSGS (Query-based Submodular Functions with Gradient-guided Synthesis), a data selection framework for efficient LLM instruction tuning. The authors argue that existing importance-based selection methods, such as LESS, independently rank training examples without sufficiently considering redundancy among selected samples, especially under limited data budgets. To address this issue, QSGS consists of three components: (1) gradient-based influence estimation to measure candidate relevance to target tasks; (2) a query-based submodular Graph Cut objective that jointly considers target relevance and inter-sample redundancy; and (3) gradient-guided synthesis that generates additional examples to cover underrepresented target gradient directions. Experiments on MMLU, BBH, and TyDiQA across Llama-2, Llama-3, and Mistral demonstrate promising results. With only 0.5% of the candidate data, QSGS consistently outperforms the reported selection baselines and matches or approaches full-data performance on Mistral-7B.

Critical strengths
  1. Clear motivation: The paper identifies an important limitation of conventional importance-based data selection methods: independently selecting high-scoring examples may introduce substantial redundancy, particularly under tight data budgets. The motivation is clear and practically relevant to efficient instruction tuning.
  2. Reasonable framework design: QSGS combines gradient-based influence estimation with submodular optimization to balance target relevance and sample diversity. The additional gradient-guided synthesis component provides an intuitive way to address potential coverage gaps in the selected subset.
  3. Promising empirical results: The method achieves consistent improvements across three model families and three benchmarks, particularly under extremely limited data budgets. The paper also provides component-wise ablations, comparisons with alternative subset selection strategies, and sensitivity analyses of key hyperparameters, which help support the effectiveness of the proposed approach.
Critical weaknesses
  • W1. Limited novelty and insufficient comparisons with related work: QSGS mainly extends LESS-style gradient importance scoring with a Graph Cut objective to reduce redundancy among selected examples. However, prior studies such as G2IS (ACL 2025), PRISM (AAAI 2022), and SPICE (ICLR 2026) have already explored gradient-based sample interactions, submodular optimization, and information-aware subset selection. The technical distinctions and advantages of QSGS over these methods remain insufficiently established. Nevertheless, Gradient-guided Synthesis is an interesting component of the proposed framework. Investigating whether it can consistently improve subsets selected by other existing methods would help demonstrate its independent contribution and generalizability.
  • W2. Limited experimental coverage: Although the paper evaluates three model families and three benchmarks, the experiments are mainly restricted to 7B–8B models and knowledge or reasoning-oriented tasks. Broader evaluations on code generation, retrieval-oriented QA, and more complex instruction-following tasks would strengthen the generalizability claims. Moreover, more systematic comparisons across different models, selection methods, and data budgets would provide stronger empirical evidence.
  • W3. Insufficient computational efficiency evaluation: Although QSGS achieves strong performance using only 0.5% of the training data, its preprocessing overhead is substantial. According to Appendix A, warm-up training and gradient computation require approximately 352 GPU hours, compared with only 32 GPU hours for final fine-tuning. While these costs may be amortized across multiple target tasks, the paper does not report the end-to-end computational costs of full-data training or competing selection methods. Given that the paper explicitly emphasizes efficient instruction tuning, the experimental evaluation appears to focus more on downstream performance than on computational efficiency. Without comprehensive end-to-end cost comparisons, I remain unconvinced that the claimed efficiency advantages are sufficiently demonstrated.
  • W4. Insufficient validation of synthetic data quality: Gradient-guided Synthesis identifies under-covered target directions through gradient residuals. However, gradient alignment does not necessarily guarantee the semantic correctness or usefulness of generated examples. The current evaluation provides limited evidence of its effectiveness across different tasks, and the quality of the generated data is not sufficiently investigated.
Questions for authors
  • Q1. Could the authors further clarify the key differences between QSGS and existing gradient-based or submodular selection methods, such as G2IS, PRISM, and SPICE? Additional comparisons under matched data budgets would help establish the effectiveness of the proposed approach.
  • Q2. Could the authors investigate whether Gradient-guided Synthesis can generalize to subsets selected by other methods, such as LESS or TAGCOS? Applying the same synthesis strategy to different selected subsets would help isolate its independent contribution and assess its broader applicability.
  • Q3. Could the authors extend the evaluation to more diverse tasks, such as code generation and retrieval-oriented QA, as well as larger models? More systematic comparisons across different selection budgets (e.g., 0.5%, 1%, 5%, and 10%) would also be valuable.
  • Q4. Could the authors provide a comprehensive comparison of end-to-end computational costs, including warm-up training, gradient computation, data selection, and final fine-tuning? In particular, how does QSGS compare with full-data training and representative baselines such as LESS and TAGCOS in terms of total GPU hours and downstream performance?
  • Q5. Could the authors further evaluate the semantic correctness and actual contribution of the generated examples? Comparisons with random synthesis or standard target-guided synthesis across different tasks would help justify the effectiveness of the proposed gradient-guided strategy.
Reviewer_ukyiOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes QSGS for targeted instruction tuning at very small selection budgets. It constructs optimizer-aware gradient features from LoRA warmup checkpoints and uses a graph-cut objective combining target relevance with pairwise redundancy penalties. A subsequent synthesis stage projects the mean target gradient outside the span of selected gradients and selects generated examples aligned with the residual. Experiments cover MMLU, BBH, and TyDiQA with three model families. At a 0.5% budget, the reported results improve on LESS in all nine model–benchmark settings and approximately recover full-data performance on Mistral-7B.

Critical strengths
  1. The paper studies a consequential operating regime: selecting roughly 1,350 examples from a 270K pool. Table 1 reports improvements over LESS across all three models and three benchmarks, with three-seed standard deviations. On Mistral, the reported MMLU, TyDiQA, and BBH results are within 0.2 percentage points of full-data training, making the results practically interesting.
  2. The ablations help identify the graph-cut selector as the main empirical contribution. Figure 3(a) improves the top-K control from 45.9 to 47.4 before synthesis raises it to 47.9. Figure 4 compares objectives using shared features and a common candidate buffer. The paper also explicitly limits its theoretical claim: it does not apply a randomized nonmonotone approximation guarantee to its deterministic greedy algorithm.
Critical weaknesses
  1. The claimed contribution of gradient-guided synthesis is not isolated or specified sufficiently to reproduce. Equations 7–8 use a target-gradient residual, but do not specify the numerical span/rank tolerance, precisely which checkpoint representation is projected, or how a global residual identifies individual under-covered subtasks. Section 4.1 gives the candidate count and ten-example cap, but the generation prompt and an equal-count unguided synthesis control are absent. Figure 3(a) therefore shows that adding selected generated examples helps, without establishing that residual guidance causes the gain or that the uncovered direction is absent from the original pool.
  2. The supporting analysis and comparisons contain unresolved inconsistencies. Section 3.2 argues that retaining the full candidate–target influence matrix is essential, but Equations 3 and 6 use only its row sums. Section 4.3 explains projection gains as recovery of similarities lost to high-dimensional concentration, although random projection is introduced as preserving geometry; no kernel diagnostic supports that explanation. The top-K MMLU control is 45.9 in Figure 3 versus LESS's 45.2 in Table 1, without an explanation of the changed setting. These gaps make it difficult to determine which implementation changes account for the reported gain.
Questions for authors
  1. Can you provide the exact residual representation, projection/rank rule, subtask-selection rule, generation prompt, and handling of a near-zero residual? Please compare residual-ranked synthetics with the same number of generated samples selected without residual guidance, keeping the final training count fixed.
  2. Can you explain the 45.9 versus 45.2 top-K discrepancy, identify where the full influence matrix is used beyond row sums, and provide before/after projection similarity statistics? Please state the candidate-buffer rule and the chosen checkpoint weights, projection dimension, and relevance–diversity weight for the main comparisons so that these results can be reconstructed.

Comments on the paper

1 comment

Hi all,

Reviewers: please respond to the authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.