acceptodds
Under review as a conference paper at ICLR 2027

LESSER: Post-Training Data Selection with Output-Layer Gradients

Abstract

The choice of post-training data for large language models substantially affects downstream performance. Gradient-based data selection is a popular approach that ranks training data by how well their gradients align with those of a small validation set. However, ranking with full-parameter gradients requires an expensive backward pass on every sample, making computation intractable for large candidate pools. This raises a natural question: can we approximate full-gradient features at a fraction of the cost? Conveniently, we find that output-layer gradients suffice for effective data selection, yet require only the cheaper forward pass. We implement this as LESSER, a drop-in wrapper for selection methods that reduces the feature-extraction FLOP cost by for SFT and for RL benchmarks, while tracking full-gradient performance on downstream tasks. Empirically, we find that even when output-layer and full gradients rank individual samples differently, they select batches with aligned gradients.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Data Selection, Supervised Fine-tuning, Instruction-Tuning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iP4EOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates an efficient way for filtering a LLM post-training dataset using a test-task-aware query set. It extends the exiting method, LESS, to use the gradients at the final layer only, which can be more efficiently computed than those at the whole model. The efficiency is evaluated in FLOPs and wall clock settings, and the performance is demonstrated in SFT and RL.

Critical strengths
  • While some performance decrease exists, the method seems a good choice for tuning a tradeoff between performance and efficiency for LESS-like gradient-based data selection of LLM post-training.
Critical weaknesses
  • This paper lacks AI USE STATEMENT, which may be a violation of the conference policy.
  • The technical novelty is limited. Using the gradients only at the last layer, instead of all the layer, for data analysis is not novel as the related work subsection cites. Transferring such successes to the LESS method for LLM post-training might not be a large contribution.
Questions for authors

How about AI USE STATEMENT?


Another caveat is that LESSER uses only the base model, while LESS uses Adam-preconditioned LoRA (Hu et al., 2021) gradients at four warmup checkpoints for computational tractability and empirical improvements.

It would be nice to add apple-to-apple comparison between LESSER and LESS for clarifying the claimed difference and the improvement.


The effectiveness or ineffectiveness of LESSER seems different depending on tasks. This makes me to worry about the generality of the effectiveness. How do you think why such a large variance exists?

5 comments

Thank you for the thoughtful comments and questions which we address below.

W1. We wish to clarify that we do have an AI usage statement, please see line 1341. We will add a pointer to it from the main text so that it is easier to find.

W2. We wish to clarify our contributions, which extend beyond the surface-level similarity to other works. Specifically, we make novel contributions in our problem application, our technical methodology, and our scientific understanding:

  • Applications: To our knowledge, we are the first to investigate and demonstrate the effectiveness of output-layer gradients across modern LLM post-training settings. While our related work highlights the use of gradient-based selection for SFT [2], RL [3], and teacher distillation [4], those works are not focused on computationally efficient alternatives. While the closest in methodology are [8][9], they focus on data attribution or valuation, i.e., explaining an already-trained model or filtering out junk data, which is a different focus from targeted selection, which aims to improve downstream performance. As a result, they may omit design choices that selection needs: for example, For-Value [8] does not normalize its features, and normalization is known to be necessary for selection [2].

  • Methodology: Our methodological contribution is a gradient-based selection feature at the cost of a forward pass. In targeted data selection, gradient-based selectors such as LESS [2] require backpropagation at several warmup checkpoints, and the area has long sought efficient alternatives (e.g., the analysis of Nayak et al. [1]). The efficient alternative so far, RDS+ [5], replaces gradients with hidden states, and it does not behave like LESS: across all 60 settings in our tables, its gains over Random are only weakly related to those of LESS (Spearman = 0.15). LESSER has the same cost as RDS+ and follows LESS much more closely (Spearman = 0.61), while being 17.8x cheaper than LESS. That is, efficiency does not have to come at the cost of the gradient signal.

  • Understanding: We also explain why output-layer gradients are sufficient (Section 5). Prior work [8][9] judges a feature by how well it scores individual examples, and evaluates it with per-example detection metrics such as AUROC. We find that this is not required. In SFT, LESSER and LESS select largely different examples (Jaccard similarity 0.037) and rank individual examples differently (median Spearman 0.52). Yet their training batches move the model in similar directions: the gradient alignment between LESSER and LESS batches is 0.29, compared with 0.03 for RDS+, and about zero for Random.

References

[1] Nayak et al. "A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)" (ICML 2026)

[2] Xia et al. "LESS: Selecting Influential Data for Targeted Instruction Tuning" (ICML 2024)

[3] Yang et al. "GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning" (arXiv 2026)

[4] Panigrahi et al. "In Good GRACEs: Principled Teacher Selection for Knowledge Distillation" (ICLR 2026)

[5] Ivison et al. "Large-Scale Data Selection for Instruction Tuning" (arXiv 2025)

[6] Madaan et al. "Quantifying Variance in Evaluation Benchmarks" (arXiv 2024)

[7] Dodge et al. "Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping" (arXiv 2020)

[8] Deng et al. "Efficient Forward-Only Data Valuation for Pretrained LLMs and VLMs" (ACL 2026)

[9] Ran et al. "Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation" (NeurIPS 2026)

Q1. Please see W1.

Q2. Thank you for suggesting this; it is an interesting experiment to run. We first note that the Adam-preconditioned LoRA gradients at four warmup checkpoints are exactly the machinery that LESSER removes, contributing to our cost savings.

As a direct apples-to-apples comparison, we consider LESS with its LoRA gradients computed at the base model at initialization, without warmup checkpoints and without Adam. We call this LESS-init. We report results at k=1,000 and k=5,000 on two models and five benchmarks, averaged over three seeds:

Model, taskk=1,000: LESSERLESS-initLESSk=5,000: LESSERLESS-initLESS
Llama-3.2-3B, TyDiQA63.361.666.464.363.866.3
Llama-3.2-3B, MMLU-Pro22.112.822.826.927.226.1
Llama-3.2-3B, GSM8K32.332.134.234.634.934.5
Llama-3.2-3B, Codex61.356.356.357.455.958.1
Llama-3.2-3B, BBH48.047.547.648.549.249.6
Llama-2-7B, TyDiQA50.251.654.651.751.252.9
Llama-2-7B, MMLU-Pro19.719.719.220.220.620.2
Llama-2-7B, GSM8K19.220.619.221.720.721.3
Llama-2-7B, Codex29.526.625.029.128.424.5
Llama-2-7B, BBH38.739.840.541.041.342.9

We observe that at the same checkpoint, LESSER performs on par with LESS-init. At both budgets, LESSER tracks the performance of LESS-init, while LESS-init scores slightly worse than LESS in some cases. We also find that on MMLU-Pro at k=1,000, LESS-init has a big performance drop on Llama-3.2-3B. This is because the fine-tuned model can’t generate coherent answers. Furthermore, LESS’s lead on TyDiQA drops after we remove the checkpoints.

On cost, LESS-init is faster than LESS since it avoids warmup training and 3 extra backward passes, but it still needs a backward pass per example. The wallclock of LESS-init remains about 4.45x slower than LESSER.

Q3. We wish to point out that task-dependence is a well-known phenomenon of selection methods [1], and is not unique to LESSER. For instance, even established works like LESS and RDS+ also display similar variance (see Tables 9, 10, 11). The broad consensus in the literature attributes this to two main factors: (1) task-headroom and (2) seed variance.

(1) Task headroom: Selection can only matter where fine-tuning on the pool changes the result. Nayak et al. [1] restrict their conclusion about LESS to "target tasks that benefit from additional training (i.e., zero-shot performance is not saturated)" and report that the gains of targeted selection "diminish at larger budgets". Our results show similar findings and reinforce our claims that LESSER is a computationally efficient alternative to LESS. The tasks where LESSER appears less effective are coincidentally also the tasks where every selector also degrades.

(2) Seed variance: Prior work has shown that fine-tuning results vary with the random seed [6] [7]. While we already run multiple seeds for our results, outcomes can still be noisier in certain cases. For example, Codex uses the 164 HumanEval problems scored by sampled pass@10, which gives evaluation numbers that fluctuate (standard deviation 1.5 pp) for LESS and LESSER.

We appreciate the reviewer for surfacing this insightful question, and we will include further discussion of this in the revised manuscript.

References

[1] Nayak et al. "A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)" (ICML 2026)

[2] Xia et al. "LESS: Selecting Influential Data for Targeted Instruction Tuning" (ICML 2024)

[3] Yang et al. "GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning" (arXiv 2026)

[4] Panigrahi et al. "In Good GRACEs: Principled Teacher Selection for Knowledge Distillation" (ICLR 2026)

[5] Ivison et al. "Large-Scale Data Selection for Instruction Tuning" (arXiv 2025)

[6] Madaan et al. "Quantifying Variance in Evaluation Benchmarks" (arXiv 2024)

[7] Dodge et al. "Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping" (arXiv 2020)

[8] Deng et al. "Efficient Forward-Only Data Valuation for Pretrained LLMs and VLMs" (ACL 2026)

[9] Ran et al. "Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation" (NeurIPS 2026)

Dear Reviewer iP4E, as the discussion period ends tomorrow, we would like to check whether our responses have addressed your concerns. We are happy to answer any further questions before the deadline.

Reviewer_2dkzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether output-layer gradients can serve as inexpensive features within existing LLM post-training selection pipelines. With fixed token weights, the output-layer gradient is computed from outer products of prediction residuals and final hidden states available from a forward pass. In SFT and the sketched RL comparison, the authors use a 64×128 two-sided projection to obtain 8,192-dimensional features. For GRACE teacher ranking, Appendix A.3 instead states that the existing 512-dimensional projection configuration is retained.

The reported SFT grid covers four models, five tasks, and three subset budgets, with means and sample standard deviations over three training seeds per cell. Most baseline results are reused from Nayak et al.; MMLU-Pro is retrained and evaluated five-shot for all four methods, and selected pipeline replication checks are reported in Appendix A.1. Other experiments evaluate corrupted-reward filtering, online RL on Countdown, and three teacher-ranking settings.

The reported outcomes support a useful performance–cost tradeoff for the tested pipelines. In SFT, the deployed LESSER and LESS subsets have little overlap, yet batches sampled from them have positive centered full-model gradient alignment at the base model. A separate native-gradient audit finds only moderate agreement between output-layer and full-gradient example rankings. These observations concern distinct experimental objects.

The paper's main contributions are empirical validation across post-training settings, efficiency accounting, and diagnostics based on aggregate gradient geometry. The authors explicitly build on earlier uses of output-layer gradients.

Critical strengths

S1. The reported experiments support a practical performance–cost tradeoff across different selection settings. The SFT grid contains 4 models × 5 tasks × 3 budgets, with three training seeds per cell (Appendix C.1, Tables 9–11). From the reported means, the signed average LESSER-minus-LESS difference is approximately −0.19 points and the average absolute difference is 1.33 points. These are descriptive averages across different task metrics, rather than an equivalence result. Baseline reuse and the selected replication checks are explicitly documented in A.1. SFT feature-extraction timing on a 10,000-example prefix implies a 17.8× ratio against four-checkpoint LESS and 4.45× against one checkpoint; the full-pool times are extrapolated. In RL, measured additional scoring time for one round falls from 63.3 to 18.1 minutes, approximately 3.50×. Online RL and GRACE teacher ranking extend the evidence to different aggregation mechanisms. The GRACE result is preservation of teacher rankings/top choices in three fixed settings, rather than a new full distillation-training experiment.

S2. Centering batch gradients is an informative diagnostic beyond raw similarity. Appendix F.1 reports a median Spearman correlation of 0.52 between native output-layer and native full-gradient cosine scores at the base model, over sampled candidates. This is a separate audit from the deployed LESSER/LESS subset comparison in Appendix E. F.1 also reports that candidates in the top 10% under output-layer cosine scores have a positive shift in the body-gradient contribution in 19 of 20 model–task pairs, providing additional evidence that the output-layer signal is informative about the rest of the network. Appendix F.2 then measures full-model gradients at the base model for batches drawn from the deployed selections. Uncentered random batches already have a median cosine similarity of 0.81. Using the centered statistic in Equation (5), with independent estimates of the population mean, median LESSER–LESS alignment at batch size 128 is 0.29, compared with 0.03 for RDS+ and approximately zero for Random; the different-LESS-batch reference is 0.53. The paper's centered statistic averages inner products and second-order terms as specified in F.2; it is not simply the average of pairwise centered cosines. This analysis is useful evidence about aggregate geometry, although its relationship to downstream differences remains descriptive.

Critical weaknesses

W1. The positioning relative to prior forward-only selection work needs a more precise boundary. The paper already cites and explains For-Value and RISE, and explicitly acknowledges earlier last-layer subset-selection methods (p. 3, lines 120–132). My concern is therefore not missing citations or a claim that output-layer gradients are entirely new. The narrower issue is that the valuation-versus-selection distinction is incomplete: For-Value's published §5.2 and Tables 3–5 already include subset selection followed by fine-tuning, and RISE §3.3 includes selection followed by continued pretraining. Their settings are not identical to this submission: for example, For-Value's GSM8K diagnostic uses test reasoning examples for valuation, whereas this paper uses small query sets and held-out test splits; For-Value also reports held-out medical evaluations. The authors should state more explicitly what is added by preserving the existing selectors across SFT, RL, and teacher ranking, and by the aggregate-gradient analysis. A limited matched comparison could quantify the practical relationship, but I do not presume algorithmic equivalence or demand that every implementation choice outperform its closest counterpart. RISE's absence as an experimental baseline is not itself grounds for rejection. For-Value paper, RISE paper.

W2. The SFT design supports a complete-pipeline comparison more directly than an isolated output-layer substitution claim. As disclosed on p. 5, lines 263–267 and p. 9, lines 462–468, LESSER uses base-model output-layer gradients, while LESS uses Adam-preconditioned LoRA gradients at four warmup checkpoints. Thus model state, parameterization, and checkpoint/optimizer processing change together. The authors have already acknowledged this; the concern is the stronger feature-only wording in §1.1 and the strength of the explanatory link to downstream behavior. F.1 does provide a controlled base-model audit of native head versus full-gradient scores, so it would be incorrect to say that no such gradient comparison exists. It does not report a matched downstream subset-training comparison. F.2 directly studies batches from the deployed selections, providing meaningful empirical support, while F.3 studies query-loss trajectories. These complementary observations do not by themselves isolate why the full pipelines have similar held-out performance. A small downstream comparison with checkpoint/objective held fixed would strengthen that attribution; alternatively, the contribution statement can consistently remain at the complete-pipeline level. The reported mean proximity should also retain the task-specific exceptions already shown in the paper, such as TyDiQA at k=1,000: 50.2±0.4 versus 54.6±0.4 for Llama-2-7B, and 63.3±0.5 versus 66.4±0.2 for Llama-3.2-3B.

Questions for authors
  1. Contribution boundary: Given For-Value's existing subset-training experiments and the different evaluation settings, could you clarify the additional empirical question answered by retaining the existing selectors across these three regimes? In particular, which conclusions concern the representation/normalization choices, and which concern the aggregate-gradient diagnostics? A concise algorithmic mapping may suffice to clarify the relationship; a limited matched comparison would further quantify it.
  2. Scope of the SFT claim: I appreciate the complete-pipeline limitation already stated in the paper. Can the feature-only language in §1.1 be qualified consistently with it? If an existing or limited matched downstream comparison is available, it would help separate the effect of output-layer restriction from selection-model warmup and checkpoint/optimizer processing. I am not requesting a full factorial ablation of all components.
  3. Connection between diagnostics and outcomes: F.1's controlled native-gradient audit and F.2's analysis of deployed subsets answer different questions. Across the existing 20 model–task pairs, do lower centered alignments in F.2 correspond to larger held-out LESSER–LESS performance gaps? A descriptive correspondence would clarify how far the proposed explanation is supported; I do not require a general predictive theorem.

These questions concern the contribution boundary and interpretation of existing results. More RL targets, larger models, or stronger theory can remain future work.

5 comments

W1. Thank you for the recommendations on precise framing. We will adjust our exposition and related work accordingly based on the following points:

(1) Distinction in use-case.

For-Value [1] and RISE [2] use output-layer gradients for data valuation and attribution, and their settings differ meaningfully from ours. For-Value is validated mainly by identifying same-class and mislabeled examples in pools of fewer than 1,000 examples. Its selection experiments (Deng et al. [1]’s Section 5.2, Tables 3–5) each use a single in-domain dataset of 5,000 to 10,000 examples as the pool, which differs from selecting out of a general instruction-tuning pool, and they do not include Random, LESS, or RDS+ as baselines. RISE is validated mainly by retrieving backdoored examples from pools of 5,000 and by filtering a pool that is 90% junk for continued pretraining. Neither compares against targeted selection algorithms such as LESS and RDS+, and neither explores formulations and applications to RL and distillation teacher selection.

(2) Distinction in methodology.

Both papers propose their own scoring algorithms. For-Value sums its score over all token positions without normalization, so the score depends on sequence length and longer training samples are favored. Length bias in unnormalized gradient scores is a known concern in data selection, and LESS normalizes for this reason [3]. RISE proposes its own influence-scoring method, combining two output-layer-derived features with additional weighting and normalization choices tailored to its attribution and valuation tasks.

LESSER, in contrast, shows that output-layer gradients can directly replace the gradient representations in existing selectors while preserving their normalization, scoring, and selection rules. We demonstrate this across SFT, RL, and distillation teacher ranking. This is a separate and surprising result: these selectors were designed around backpropagated gradients, and it was not guaranteed for output-layer gradients to preserve their effectiveness, especially when they can rank individual examples very differently. Yet in RL, training on LESSER-selected data produces nearly identical learning curves to selection using full gradients. In GRACE, LESSER picks the same top-ranked teachers as full-gradient selection.

(3) Distinction in scientific contribution.

We further investigate a counterintuitive phenomenon in which LESSER and LESS select different subsets for SFT, yet achieve similar downstream performance. In Section 5, we explain this by examining the full-gradient update directions of the selected batches. LESSER achieves a centered gradient alignment of 0.29 with LESS, compared with just 0.03 for RDS+. This is consistent with aggregate update directions being what selection needs to preserve. Neither For-Value nor RISE investigates this question by comparing its selected data across different selection algorithms. Their analyses instead focus on properties such as gradient energy or the extent to which output-layer gradients approximate full gradients. However, neither establishes that different selections can produce similar full-model updates and downstream performance, which is the phenomenon we investigate.

Nevertheless, the reviewer points to an interesting theme: the fact that LM-head gradients are useful across many different task types show that this feature can be robust. We will include an extended discussion of this in the revised manuscript.

References

[1] Deng et al. "Efficient Forward-Only Data Valuation for Pretrained LLMs and VLMs" (ACL 2026)

[2] Ran et al. "Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation" (NeurIPS 2026)

[3] Xia et al. "LESS: Selecting Influential Data for Targeted Instruction Tuning" (ICML 2024)

W2. We will clarify the language around Section 1.1 to better qualify the experiment setups.

We also ran the matched downstream comparison the reviewer suggests, with the checkpoint and objective held fixed. We consider LESS with its LoRA gradients computed at the base model at initialization, without warmup checkpoints and without Adam. We call this LESS-init. We report results at k=1,000 and k=5,000 on two models and five benchmarks, averaged over three seeds:

Model, taskk=1,000: LESSERLESS-initLESSk=5,000: LESSERLESS-initLESS
Llama-3.2-3B, TyDiQA63.361.666.464.363.866.3
Llama-3.2-3B, MMLU-Pro22.112.822.826.927.226.1
Llama-3.2-3B, GSM8K32.332.134.234.634.934.5
Llama-3.2-3B, Codex61.356.356.357.455.958.1
Llama-3.2-3B, BBH48.047.547.648.549.249.6
Llama-2-7B, TyDiQA50.251.654.651.751.252.9
Llama-2-7B, MMLU-Pro19.719.719.220.220.620.2
Llama-2-7B, GSM8K19.220.619.221.720.721.3
Llama-2-7B, Codex29.526.625.029.128.424.5
Llama-2-7B, BBH38.739.840.541.041.342.9

We observe that at the same checkpoint, LESSER performs on par with LESS-init. At both budgets, LESSER tracks the performance of LESS-init, while LESS-init scores slightly worse than LESS in some cases. We also observe that on MMLU-Pro at k=1,000, LESS-init has a big performance drop on Llama-3.2-3B. This is because the fine-tuned model can’t generate coherent answers. Furthermore, LESS’s lead on TyDiQA drops after we remove the checkpoints.

We will report this comparison in the appendix.

Q1. We address these points in W1.

Q2. We address these points in W2.

Q3. This is an interesting idea to try, and here is our preliminary evidence. Across the 20 (model, task) settings at k=5,000, the F.2 alignment gives Spearman correlation −0.1 with the LESSER − LESS gap. Therefore, bigger gradient alignment doesn’t necessarily predict better performance at the granularity of each fine-tuning run. We suspect this is because there is noise across runs and the margins can be small.

On a positive note, this prediction holds better at the level of different selectors. We show that LESSER batches align with LESS batches far more than RDS+ batches, with medians 0.29 and 0.03. Correspondingly, the gains of LESSER over Random follow those of LESS across all 60 settings (Spearman = 0.61), while those of RDS+ do so only weakly (Spearman = 0.15). That is, when LESS works, we can also expect LESSER to work. We will state in Appendix F.2 the scope of our diagnostic.

References

[1] Deng et al. "Efficient Forward-Only Data Valuation for Pretrained LLMs and VLMs" (ACL 2026)

[2] Ran et al. "Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation" (NeurIPS 2026)

[3] Xia et al. "LESS: Selecting Influential Data for Targeted Instruction Tuning" (ICML 2024)

Dear Reviewer 2dkz, as the discussion period ends tomorrow, we would like to check whether our responses have addressed your concerns. We are happy to answer any further questions before the deadline.

Reviewer_syhpOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies the data selection in post-trianing stage. It tries to reduce the computation cost of method LESS by utilizing only the output-layer gradients. The experiments shows that the selected data highly align with that selected by the full gradients. This method also accelerate the computation process of gradient inner product in LESS.

Critical strengths
  1. The presentation is clear and easy to be understood. It first motives the high computation cost issue existing in LESS, then proposes the solution and empirically demonstrates the advantages of proposed method.
  2. This method shows the great speedup in post training data selection.
Critical weaknesses
  1. This paper focuses on post-training, so its goal is to select data to fine-tune model (LoRA adapter). But actually, this method seems not work on the typical fine-tuning method LoRA. They study on how to select data to fully fine tune models but the compared baseline LESS [1] focuses on how to fine tune model by LoRA not full parameters. That means the data selected by LESS essentially works better for LoRA fine-tuning rather than fully fine tuning. That's probably the reason why LESSER can select almost different data from LESS in SFT setting (line 387). So the comparison experiments don't make sense to me.

[1] Xia, Mengzhou, et al. "Less: Selecting influential data for targeted instruction tuning." arXiv preprint arXiv:2402.04333 (2024).

  1. No significant performance gain is demonstrated in the experiments though it saves lots of selection time. The performance gap even increases when selection size inceases (Table 9-11).
  2. What if we don't know what the target data is? In some case, the target is unknown due to privacy. Then How does this method deal with that?
Questions for authors

Please check the critical weaknesses.

6 comments

Thank you for your thoughtful comments, we respond to the points below.

W1. We wish to first point out that there is nothing fundamentally stopping LESS from being used for both full and LoRA fine-tuning (This is also claimed by Xia et al [1] in their Figure 1 caption). Nevertheless, the reviewer has inspired an interesting question: how would full vs LoRA fine-tuning compare for LESS vs LESSER? To test this, we train Llama-2-7B, the model in LESS, using LESS's own LoRA training recipe. We compare LESSER, LESS and Random selections under both full fine-tuning and LoRA, averaging over three seeds.

Llama-2-7BkFull FT: LESSERLESSRandomLoRA: LESSERLESSRandom
TyDiQA1,00050.254.649.449.451.049.1
MMLU-Pro1,00019.719.216.319.519.616.8
GSM8K1,00019.219.215.020.219.514.2
Codex1,00029.525.026.426.824.327.5
BBH1,00038.740.539.139.940.639.0
TyDiQA5,00051.752.951.251.452.150.7
MMLU-Pro5,00020.220.220.020.120.519.0
GSM8K5,00021.721.316.721.420.916.9
Codex5,00029.124.527.327.326.430.0
BBH5,00041.042.940.339.742.240.3

We see that under both LoRA and full fine-tuning, the performance of LESSER tracks that of LESS, which affirms our core claim. LoRA therefore does not favor LESS's selections over LESSER's.

We next address the reviewer’s concerns on LESS and LESSER selecting different data on SFT (e.g., line 387). One factor is that the candidate pool is very large relative to the selection size: we select only 1,000 (0.5%) to 10,000 (~5.1%) training samples from a pool of 197,196 examples, yet many samples can have similar effects on the model. The table above shows this directly, since the selected data differ and still perform closely under both LoRA and full fine-tuning.

In fact, LESS itself is also brittle in its choice of data! To see this, we compute LESS at the base model vs. warmup checkpoints, with the same LoRA gradients on Llama-3.2-3B at k=1,000.

Jaccard similarity with LESS dataTyDiQAMMLU-ProGSM8KCodexBBH
LESS, at base model0.020.050.040.020.01
LESSER0.010.030.030.02<0.01

Notably, the low overlap between LESS and LESSER is similar to the overlap between LESS at different checkpoints, suggesting that using LoRA vs. output-layer gradients isn’t the only factor leading to this difference.

In summary, one of our key understanding is why gradient-based selectors can choose different data, yet behave similarly (see Appendix F/2). In particular, we claim that different selections are acceptable, provided that the aggregation retains a similar effect, which explains an otherwise counter-intuitive phenomenon that arises in practice.

Reference

[1] Xia et al. "LESS: Selecting Influential Data for Targeted Instruction Tuning" (ICML 2024)

[2] Yang et al. "GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning" (COLM 2026)

[3] Ivison et al. "Large-Scale Data Selection for Instruction Tuning" (arXiv 2025)

[4] Yu et al. "Selective Pre-training for Private Fine-tuning" (TMLR 2024)

[5] Xie et al. "Differentially Private Synthetic Data via Foundation Model APIs 2: Text" (ICML 2024)

[6] Chen et al. "AlpaGasus: Training a Better Alpaca with Fewer Data" (ICLR 2024)

[7] Liu et al. "What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning" (ICLR 2024)

[8] Nayak et al. "A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)" (ICML 2026)

W2. We wish to clarify that this is not a weakness. Our significant savings in selection time is precisely the intended effect, and has been sought after in this literature [1][3][8]. Notably, our effects persist through SFT, RL, and teacher distillation. We do not claim that LESSER outperforms LESS, and we are not certain what the reviewer means by "performance gap", so we give our best-effort interpretation below. Specifically, it is acceptable for LESS to have more bold entries in Tables 9--11 because we claim that LESSER tracks LESS. Instead, the meaningful metric to track is LESS-minus-Random vs LESSER-minus-Random, which we summarize in the following table (average over four models and five tasks, three seeds each):

Budget kLESS − RandomLESSER − RandomDifferenceStandard error95% interval
1,000+2.78+2.700.080.55(−0.99, +1.10)
5,000+3.08+2.870.210.34(−0.49, +0.79)
10,000+1.03+0.750.280.35(−0.43, +0.89)

The two gains differ by 0.08 (+/- 0.55), 0.21 (+/- 0.34), and 0.28 (+/- 0.35) points, where the parentheses denote the standard errors. The difference rises nominally with budget, but each value is within one standard error of zero and all three 95% bootstrap intervals include zero. The data therefore do not show a statistical gap between LESS and LESSER, nor one that grows with selection size.

W3. We wish to clarify that untargeted selection is not our intended setting. Prior work in this field such as LESS [1], RDS+ [3] and GradAlign [2] all take a query set, which is assumed under the standard fine-tuning settings where the user specifies the specific capability with a few examples. The query sets are also usually small: 8 to 81 examples per task (Table 2).

Nevertheless, when the target is unknown for a general fine-tuning setting, it is reasonable to assume that the training pool is sufficiently representative of the target task. One can then form a validation set by sampling uniformly at random from the training pool and run LESSER as usual. One can also repeat this for several random draws and combine the resulting selections.

Selection with a private target [4], differentially private synthetic text that could stand in for private queries [5], and untargeted curation [6, 7] are studied as separate problems and are complementary to our work. We will add discussions of these.

References

[1] Xia et al. "LESS: Selecting Influential Data for Targeted Instruction Tuning" (ICML 2024)

[2] Yang et al. "GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning" (COLM 2026)

[3] Ivison et al. "Large-Scale Data Selection for Instruction Tuning" (arXiv 2025)

[4] Yu et al. "Selective Pre-training for Private Fine-tuning" (TMLR 2024)

[5] Xie et al. "Differentially Private Synthetic Data via Foundation Model APIs 2: Text" (ICML 2024)

[6] Chen et al. "AlpaGasus: Training a Better Alpaca with Fewer Data" (ICLR 2024)

[7] Liu et al. "What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning" (ICLR 2024)

[8] Nayak et al. "A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)" (ICML 2026)

Thank you to the authors for their responses. However, I still have several concerns.

Regarding the first question, the authors have already conducted experiments with LoRA, but the results do not demonstrate an improvement over LESS. I am also unsure whether the proposed method retains its advantage in data selection efficiency under the LoRA setting, where the baselines no longer require full gradient computation.

Regarding the second experiment, I am not convinced by the authors' explanation. From my understanding, the optimal data selection can vary significantly across different training checkpoints. For example, when training a model on mathematical reasoning, basic math problems may be more beneficial during the early stages of training, while more challenging problems may become increasingly valuable as training progresses. This naturally explains why the selected data could differ substantially between LESS using the base model and LESS using a warm-up checkpoint.

Therefore, the authors' observation that " the low overlap between LESS and LESSER is similar to the overlap between LESS at different checkpoints" does not adequately address my concern. These two comparisons reflect fundamentally different factors: one captures changes in data selection across training stages, while the other reflects differences between selection methods. Similar overlap rates alone do not establish that these differences arise from the same underlying mechanism or justify the authors' conclusion.

Finally, to clarify my second question, the "performance gap" I referred to is specifically the performance difference between LESSER and LESS.

So I'm inclined to keep my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Discussions

Summary of Discussions

For convenience, we summarize the reviewers’ main concerns, our responses, as well as the new experiments and analyses that have strengthened our paper.

Summary of main concerns

  1. Further experiments with selection features at different warmup checkpoints (R1, R2, R3). Reviewers noted that LESSER uses base-model output-layer gradients whereas LESS uses Adam-preconditioned LoRA gradients across four warmup checkpoints, potentially confounding the effect of changing the gradient feature. To isolate the effect, we introduced LESS-init, which applies LoRA gradients at the exact same base-model checkpoint as LESSER. Across two models, five tasks, and two budgets, LESSER matches LESS-init’s performance while running 4.45× faster. We further trained Llama-2-7B using LESS's original LoRA recipe and verified that LESSER tracks LESS under both LoRA and full fine-tuning training settings, confirming that our findings generalize to both LoRA and full fine-tuning settings.
  2. Further statistical analysis to confirm performance (R1, R3). Reviewers asked about variation across tasks and the absence of consistent gains over LESS. We claim that LESSER aims to match—rather than surpass—full-gradient selection at a fraction of the cost. Across 20 model–task pairs, mean LESS-minus-LESSER gaps at budgets of 1,000, 5,000, and 10,000 are 0.08, 0.21, and 0.28 percentage points, with all 95% bootstrap confidence intervals including zero. Across all 60 settings, LESSER's gains over Random strongly track LESS's, supporting our claims (Spearman 0.61 vs. 0.15 for RDS+).
  3. Relation to other works with last-layer gradients (R2, R3). Reviewers questioned how LESSER differs from prior output-layer gradient methods such as For-Value and RISE. Regarding RISE (NeurIPS 2026), the ICLR Reviewer Guide explicitly states that comparisons with contemporaneous work are not required and their absence cannot be grounds for rejection. Nevertheless, we clarify our distinct contributions. Rather than proposing a new valuation score, we show that output-layer gradients serve as a drop-in replacement inside existing gradient-based selectors across three distinct post-training settings. This was not guaranteed: output-layer gradients rank individual examples differently from full gradients. Yet LESSER matches LESS at 17.8× lower feature-extraction cost, reproduces full-gradient RL learning curves, and identifies the same top-ranked teachers in distillation. Another distinct contribution is our scientific understanding of selection phenomena, which we describe next.
  4. Mechanistic insight: batch-level update alignment vs. data overlap (R1, R2). Reviewers asked how far aggregate gradient alignment explains downstream results. Although LESSER and LESS select largely disjoint examples in SFT (Jaccard similarity 0.037), their selected batches yield much closer full-gradient update directions than RDS+ (median centered alignment 0.29 vs. 0.03).

Together, these strengthen our core claim that data selection with output-layer gradients is comparable to full-gradients at substantially lower cost.

We thank the reviewers and AC for their time and constructive feedback, and we are happy to address any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.