acceptodds
Under review as a conference paper at ICLR 2027

HARP: Efficient Data Selection for Finetuning Large Language Models

Abstract

Finetuning data selection requires balancing two competing goals: selecting examples that improve the downstream objective, and doing so without repeatedly finetuning models. Train-free selectors are scalable but rely on proxies such as embedding similarity or clustering, which may not match the target objective. Train-based selectors better reflect downstream utility through gradient signals, subset evaluation, or Shapley attribution, but require many costly train–evaluate iterations. We propose Hierarchical Active Region Pruning (HARP), an efficient train-based selector that preserves downstream alignment while reducing selection cost. HARP organizes the training pool into a node–leaf hierarchy, evaluates only representative leaves, and infers unmeasured utilities with empirical Bayes posteriors. It then selects data using two complementary envelopes: HARP-C, which conservatively controls redundancy, and HARP-E, which additively rewards complementary regions. We theoretically show that, under local smoothness and bounded estimation error, HARP controls selection error while reducing train–evaluate cost. Across base models, finetuning datasets and reasoning benchmarks ( settings), HARP variants take the per-cell best among ranked methods on cells and achieve -setting means of / , compared with for the strongest baseline, NICE. HARP-C trains on roughly fewer examples than the budgeted baselines and fewer than full finetuning, combining higher mean accuracy with a smaller mean training-set size than every ranked baseline.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Data Selection, Finetuning, Large language model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_46bkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HARP, a training-based data selection method that provides a middle ground between existing training-free and training-based data selection approaches. It clusters training data into node-leaf groupings, after which some leaves are selected for evaluation and creating Bayes posteriors for the unmeasured data. Then, two algorithms HARP-C and HARP-E are proposed to select the final training set based on these measurements. The paper evaluates across a range of datasets and models, showing that HARP is more performant and efficient in aggregate compared to strong existing baselines. Ablations are also provided for the various hyperparameters involved in the approach.

Critical strengths
  • The biggest strength of the approach lies in it's performance and efficiency. The approach performs better on aggregate compared to presented baselines, while resulting in a much smaller final training set.
  • The approach itself is quite intuitive and sound, and hyperparameter ablations are provided, which is a good reference for practical use of this method.
Critical weaknesses
  • My biggest concern with this paper lies in it's presentation. Section 3 as a whole is quite jargon-heavy and hard to follow. I would strongly recommend the authors to provide simpler, natural language descriptions of the approach in the main section of the paper, and improve the presentation of the mathematical details. For example, proper equations can be used to separate text and mathematical terminology. The main results table is also quite difficult to understand.
  • While it is true that HARP-C finds a training subset with 9x fewer training examples than the 10k example baselines presented, the "leaf-level queries" are themselves more expensive than individual-example queries that just train on one example. To establish the practicality of the approach, a comparison of computation incurred during the entire HARP data selection process, as compared to the baselines, should be included. It might be the case that the end-to-end runtimes or GPU hours for HARP exceed even full fine-tuning, due to repeated evaluation on the full validation set.
  • A lot of implementation details are missing in the manuscript. For example, how are the train set embeddings computed? What is the target number of child groups, kernel temperature and threshold used in the experiments?
Questions for authors
  • Have the authors tried comparing against other standard choices for their design decisions, eg. k-means clustering for the "node-leaf groupings"?
  • Do the authors have a recommendation about which of HARP-C and HARP-E is recommended for practical use? Are these approaches complementary in any way?
  • The baselines are fixed at 10k examples, whereas HARP has the freedom to choose to select a smaller training set. I would be interested in seeing how well the baselines perform at smaller scale (eg. 1k examples) as well.
5 comments
AuthorsOpenReview ↗

Response to Reviewer 46bk (Part 1/3)

We thank the reviewer for the positive assessment of HARP's effectiveness, efficiency, and methodology, and for the constructive suggestions.

Weakness 1.

My biggest concern with this paper lies in it's presentation. Section 3 as a whole is quite jargon-heavy and hard to follow. I would strongly recommend the authors to provide simpler, natural language descriptions of the approach in the main section of the paper, and improve the presentation of the mathematical details. For example, proper equations can be used to separate text and mathematical terminology. The main results table is also quite difficult to understand.

A1. Thank you for the suggestion. We will revise Section 3 and Table 1 accordingly to improve their readability.

Weakness 2.

While it is true that HARP-C finds a training subset with 9x fewer training examples than the 10k example baselines presented, the "leaf-level queries" are themselves more expensive than individual-example queries that just train on one example. To establish the practicality of the approach, a comparison of computation incurred during the entire HARP data selection process, as compared to the baselines, should be included. It might be the case that the end-to-end runtimes or GPU hours for HARP exceed even full fine-tuning, due to repeated evaluation on the full validation set.

A2. Thank you for requesting the full cost comparison. A leaf-level query does cost more than a single-example query, but HARP needs far fewer train–evaluate iterations. For the same MM training examples, evaluating them individually requires MM full-validation evaluations, while grouping them into Q≪MQ \ll M leaves requires only QQ: Cindividual≈Ttrain(M)+M⋅TevalC_{\mathrm{individual}} \approx T_{\mathrm{train}}(M) + M \cdot T_{\mathrm{eval}} and Cleaf≈Ttrain(M)+Q⋅TevalC_{\mathrm{leaf}} \approx T_{\mathrm{train}}(M) + Q \cdot T_{\mathrm{eval}}. As shown in Table R9, validation accounts for 22.0% of HARP-C's total cost and 17.2% of HARP-E's, so evaluating examples individually would multiply the total cost by about 1+0.220(M/Q−1)1+0.220(M/Q-1) and 1+0.172(M/Q−1)1+0.172(M/Q-1), respectively; with M/Q=10M/Q=10, this is about 3.0× and 2.5×. Reducing the number of train–evaluate iterations therefore matters more than the cost of each individual query.

We conducted additional experiments that measure the end-to-end cost of each method in two parts: data selection and final fine-tuning. Costs are averaged over the 36 settings (3 models × 3 training sets × 4 target benchmarks) and reported in units of 101710^{17} FLOPs, as shown in Table R10. Our cost breakdown further confirms HARP's efficiency. Including final fine-tuning, HARP-C/E cost 3.9/4.9, compared with 9.8 for SHED, 33.6 for LESS, 36.6 for NICE, and 27.4 for full fine-tuning. Thus, even after accounting for repeated full-validation evaluations, HARP remains substantially cheaper than the other train-based baselines and full fine-tuning.

Table R9. HARP cost breakdown (% of HARP's total cost)

StageHARP-CHARP-E
Leaf training (representative leaves)68.9%53.8%
Leaf utility estimation (validation)22.0%17.2%
Final ranking (CPU)≈ 0≈ 0
Final training9.1%28.9%
Total100%100%

Table R10. Mean selection and training costs (×1017\times 10^{17} FLOPs)

MethodMean SelectMean TrainMean Total
Random0.03.33.3
DQ0.03.33.3
DSIR0.01.81.8
Full FT0.027.427.4
LESS31.81.833.6
NICE33.82.836.6
SHED6.63.29.8
HARP-C3.50.43.9
HARP-E3.51.44.9

(Continued in Part 2/3.)

AuthorsOpenReview ↗

Response to Reviewer 46bk (Part 2/3)

Weakness 3.

A lot of implementation details are missing in the manuscript. For example, how are the train set embeddings computed? What is the target number of child groups, kernel temperature and threshold used in the experiments?

A3. Thank you for pointing this out. Sections 3.2–3.3 describe the hierarchy construction and utility interpolation, while Appendix Table 4 reports the main hyperparameters. We will add the following implementation details to the main paper.

We use all-MiniLM-L6-v2 to compute normalized embeddings of instruction–response pairs. The target number of child groups is MQ=max⁡(2,⌈∣Q∣/L⌉)M_Q=\max(2,\lceil |Q|/L\rceil), where LL controls group granularity. Section 5.3, Figure 4 evaluates L∈{256,512,1024,2048,4096}L\in\lbrace 256,512,1024,2048,4096\rbrace, showing that HARP maintains stable performance across different group sizes. We set λ=1\lambda=1 to provide smooth similarity weights across representatives and ϵdom=0.005\epsilon_{\mathrm{dom}}=0.005 to filter domains with negligible estimated utility changes.

Question 1.

Have the authors tried comparing against other standard choices for their design decisions, eg. k-means clustering for the "node-leaf groupings"?

A4. Thank you for the suggestion. HARP estimates the utility of unmeasured leaves from representative leaves, so keeping these leaves close helps reduce interpolation error. This motivates our use of farthest-first to cover diverse regions of the training pool, whereas k-means minimizes the average squared distance to cluster centers and may leave some regions poorly covered. Farthest-first is also computationally efficient, requiring O(nmk)O(nmk) time for distance calculations over nn embeddings of dimension mm and kk groups, compared with O(Inmk)O(Inmk) for II full iterations of Lloyd's k-means.

Question 2.

Do the authors have a recommendation about which of HARP-C and HARP-E is recommended for practical use? Are these approaches complementary in any way?

A5. Thank you for asking for a practical choice. We conducted an additional analysis of the subsets selected by HARP-C and HARP-E in nine MMLU settings with seed 42, reported in Table R11. HARP-C selects fewer leaves and fewer training examples than HARP-E in all nine settings, and in six of them all of its leaves are also selected by HARP-E. Even so, HARP-C achieves higher accuracy in six settings, so the extra leaves selected by HARP-E mostly do not improve accuracy. This is consistent with HARP-C's design of limiting redundant positive contributions. For example, on Qwen3-4B/Self-Instruct, HARP-C selects 5 leaves and 2,162 examples, compared with HARP-E's 18 leaves and 10,000 examples. HARP-E includes 80% of HARP-C's selected leaves, while HARP-C achieves higher accuracy, 71.4 versus 70.3, with substantially less training data.

Table R11. Selected training data and MMLU accuracy for seed 42. Values are HARP-C / HARP-E. Bold marks the higher accuracy.

Model / DatasetTrain RowsLeavesC in E Leaves (%)Acc. (%)
Llama-3.1-8B / SI968 / 1,7331 / 310063.0 / 63.2
Llama-3.1-8B / Wiz825 / 5,2371 / 710064.6 / 63.1
Llama-3.1-8B / Tulu445 / 3,5082 / 710064.3 / 64.4
Qwen3-8B / SI3,063 / 7,0055 / 1010075.4 / 74.3
Qwen3-8B / Wiz3,123 / 10,0004 / 1310075.3 / 74.6
Qwen3-8B / Tulu5,467 / 10,0008 / 4062.575.8 / 76.0
Qwen3-4B / SI2,162 / 10,0005 / 1880.071.4 / 70.3
Qwen3-4B / Wiz647 / 2,7503 / 910073.0 / 72.2
Qwen3-4B / Tulu4,729 / 7,8646 / 1283.372.5 / 72.0

Based on these results, we recommend HARP-C as the default when dataset redundancy or quality is unknown, since it avoids overcounting redundant gains and is less likely to select noisy examples. HARP-E is more suitable when the dataset has low redundancy and reliable quality, because it can capture complementary gains from multiple useful leaves within each domain. We will add this analysis and these recommendations to the revision.

(Continued in Part 3/3.)

AuthorsOpenReview ↗

Response to Reviewer 46bk (Part 3/3)

Question 3.

The baselines are fixed at 10k examples, whereas HARP has the freedom to choose to select a smaller training set. I would be interested in seeing how well the baselines perform at smaller scale (eg. 1k examples) as well.

A6. Thank you for suggesting the 1k comparison. We use 10k as the default budget for all baselines for a fair comparison. The baselines use a fixed number of training examples, but the optimal budget for each method cannot be determined before selection. In contrast, HARP dynamically selects the subset size, treating the budget as an upper bound, and achieves better performance with fewer training examples.

We further evaluate LESS, NICE, and SHED at 1k, HARP-C's size, and HARP-E's size, as shown in Table R12. Compared with 10k examples, LESS decreases from 51.6 to 49.9 at 1k, 49.7 at HARP-C's size, and 50.5 at HARP-E's size. NICE improves from 59.3 to 62.1, 61.8, and 60.9, respectively, and SHED improves from 56.4 to 60.7, 61.5, and 59.8, respectively. However, all these results remain below HARP-C's 64.2 and HARP-E's 63.5, so HARP's advantage holds at smaller budgets as well. For the computational cost, please refer to A2.

Table R12. Dataset-size comparisons (bold: best; italic: second best)

MethodRows trained onMMLUARCGSM8KMATHAvg
LESS1k68.9 ± 0.484.7 ± 0.429.9 ± 1.415.9 ± 3.149.9 ± 1.3
LESSHARP-C's size68.0 ± 0.485.0 ± 0.430.5 ± 2.415.2 ± 3.149.7 ± 1.6
LESSHARP-E's size67.5 ± 0.683.8 ± 0.729.1 ± 1.821.4 ± 1.950.5 ± 1.3
LESS10k66.1 ± 0.783.0 ± 0.833.6 ± 6.123.8 ± 1.751.6 ± 2.3
NICE1k69.9 ± 0.586.2 ± 0.463.8 ± 2.028.3 ± 1.662.1 ± 1.1
NICEHARP-C's size69.9 ± 0.586.4 ± 0.462.1 ± 2.229.0 ± 1.961.8 ± 1.2
NICEHARP-E's size68.8 ± 0.785.7 ± 0.560.3 ± 3.928.7 ± 2.160.9 ± 1.8
NICE10k68.6 ± 0.785.9 ± 0.455.1 ± 2.827.9 ± 1.559.3 ± 1.3
SHED1k69.7 ± 0.985.0 ± 0.959.8 ± 5.028.3 ± 1.860.7 ± 2.1
SHEDHARP-C's size69.6 ± 0.785.1 ± 1.062.5 ± 3.029.0 ± 2.161.5 ± 1.7
SHEDHARP-E's size68.6 ± 0.984.5 ± 1.357.0 ± 5.629.1 ± 2.159.8 ± 2.5
SHED10k67.8 ± 0.983.8 ± 1.147.3 ± 3.226.8 ± 1.356.4 ± 1.6
HARP-COwn70.6 ± 0.285.7 ± 0.568.2 ± 2.132.1 ± 1.964.2 ± 1.2
HARP-EOwn69.9 ± 0.385.8 ± 0.564.1 ± 3.234.4 ± 1.563.5 ± 1.4

We thank the reviewer again for the constructive feedback.

Reviewer_J4mKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HARP, an offline data selection framework that aims to improve downstream task performance while reducing costly train-evaluate iterations. To reduce the cost of example level utility estimation, the method partitions the training pool into a two level hierarchy based on embedding similarity. Rather than evaluating the fine tuning effect of every leaf on each domain, it measures only representative leaves and estimate the utility gains of unmeasured leaves within the same parent node through interpolation and empirical-Bayes shrinkage. It then greedily adds the leaf that maximizes the estimated marginal utility gain under one of two surrogate objectives, HARP-C and HARP-E. The method is evaluated on four reasoning benchmarks and compared with strong data selection baselines and full-data fine-tuning.

Critical strengths

HARP provides a clever way to reduce costly train-evaluate iterations by grouping examples based on embedding similarity and treating leaves as the atomic selection units, while using representative leaves to estimate the utility of unmeasured leaves rather than evaluating every leaf individually. The paper also recognizes that gains from separately fine tuning on individual leaves do not directly reflect the effect of joint training, and introduces two surrogate objectives to account for potential redundancy or complementarity between leaves. The experimental results are encouraging: HARP achieves competitive accuracy with strong data selection baselines using substantially fewer selected examples, while consistently outperforming full-data fine-tuning.

Critical weaknesses

The computational efficiency claim is not sufficiently supported by the empirical evaluation, although section 3.6 provides complexity expressions. A reduction in the final select subset size does not establish end to end computational savings, since the selection stage also requires finetuning on representative leaves and then evaluating the resulting models on the validation set for each domain. Besides, the definition of a domain is also unclear: does each reasoning benchmark count as a domain or are domains defined by subjects within a benchmark? I would therefore like to see end to end runtime or GPU hours, including both data selection and final fine-tuning, compared with the baselines. It would also be helpful to clarify which costs can be reused when the downstream tasks change. These measurements would help clarify the practical computational benefits of the proposed selection procedure.

Questions for authors
  1. HARP-E provides a straightforward additive surrogate, whereas HARP-C is motivated by avoiding redundant positive contributions. However, the reported differences in accuracy and the final selected subset size provide only indirect evidence for this motivation. For example, section 5.1 relates the differences in performance to how they aggregate positive contributions. Could the authors provide additional analysis, such as examples illustrating whether HARP-C selects leaves with less overlapping training benefits? It would also be helpful to clarify how practitioners should choose between HARP-C and HARP-E for a new downstream task. Is there a practical criterion for choosing between them, or would both selected subsets need to be fine-tuned and compared on validation data? If so, how would this affect the overall computational cost?
  2. The shrinkage weight ρg(d)\rho_g(d) depends on the across node variance τ2(d)\tau^2(d), which is set to be 0.01 in table 4. Could the authors clarify how the value of 0.01 was chosen? How sensitive is the final downstream performance to this parameter choice?
  3. Is a separate training subset selected for each benchmark, or is a single subset selected using all four validation sets?
  4. Which embedding model is used, and does it encode the instruction alone or the instruction–response pair? How sensitive are the final results to the choice of embedding model?
  5. Could the authors detail how size stratification is implemented when choosing representative leaves?
  6. NICE uses the full 10k budget, whereas HARP can select a smaller subset. since selecting more data does not necessarily improve performance, could the authors compare NICE with HARP at matched subset sizes?
  7. The data selection methods use additional downstream validation data to guide selection. Is this part of data also included in training for the full FT baseline?
4 comments
AuthorsOpenReview ↗

Response to Reviewer J4mK (Part 1/4)

We thank the reviewer for the positive assessment of HARP's methodology and experimental results and for the constructive suggestions.

Weakness.

The computational efficiency claim is not sufficiently supported by the empirical evaluation, although section 3.6 provides complexity expressions. A reduction in the final select subset size does not establish end to end computational savings, since the selection stage also requires finetuning on representative leaves and then evaluating the resulting models on the validation set for each domain. Besides, the definition of a domain is also unclear: does each reasoning benchmark count as a domain or are domains defined by subjects within a benchmark? I would therefore like to see end to end runtime or GPU hours, including both data selection and final fine-tuning, compared with the baselines. It would also be helpful to clarify which costs can be reused when the downstream tasks change. These measurements would help clarify the practical computational benefits of the proposed selection procedure.

A1. Thanks for your helpful comment! Regarding how domains are defined, we group validation examples by subject labels when available. Otherwise, we use normalized embeddings to form KK groups. We select centers using kk-means++ and assign each example to its nearest center by cosine similarity. Each group is treated as a validation domain.

Regarding end-to-end cost, we conducted additional experiments that measure the cost of each method in two parts: data selection and final fine-tuning. Costs are averaged over the 36 settings (3 models × 3 training sets × 4 target benchmarks) and reported in units of 101710^{17} FLOPs.

Table R3. Mean selection and training costs (×1017\times 10^{17} FLOPs)

MethodMean SelectMean TrainMean Total
Random0.03.33.3
DQ0.03.33.3
DSIR0.01.81.8
Full FT0.027.427.4
LESS31.81.833.6
NICE33.82.836.6
SHED6.63.29.8
HARP-C3.50.43.9
HARP-E3.51.44.9

HARP is much cheaper than train-based baselines while achieving higher accuracy. Compared with LESS and NICE, whose average accuracies are 51.6 and 59.3, HARP-C/E achieve 64.2/63.5 average accuracy, with 8.7–9.5×/6.8–7.4× lower total costs. Selection costs are 9.1–9.6× lower, while final training costs are 5.3–8.1× lower for HARP-C and 1.3–2.0× lower for HARP-E. Compared with SHED, whose average accuracy is 56.4, HARP-C/E also achieve higher accuracy with 2.5×/2.0× lower total costs, 1.9× lower selection costs, and 9.2×/2.2× lower training costs. Compared with Random and DQ, HARP-C/E cost only about 1.2×/1.5× as much while achieving higher average accuracy, 64.2/63.5 versus 56.7 for Random and 45.7 for DQ. DSIR is cheaper because it selects shorter examples, but HARP-C/E still achieve higher accuracy, 64.2/63.5 versus 56.0, at 2.2×/2.8× its total cost. Compared with full fine-tuning, HARP-C/E achieve higher accuracy, 64.2/63.5 versus 54.7, with 7.1×/5.5× lower total costs. These results show that HARP achieves a better accuracy–efficiency trade-off than the baselines.

Table R4. HARP cost breakdown (% of HARP's total cost)

StageHARP-CHARP-E
Leaf training (representative leaves)68.9%53.8%
Leaf utility estimation (validation)22.0%17.2%
Final ranking (CPU)≈ 0≈ 0
Final training9.1%28.9%
Total100%100%

Regarding which costs can be reused when the downstream task changes, HARP builds its leaf hierarchy from the training data, independently of the validation domains. When domain weights change, we can reuse the measured utilities. As Table R4 shows, obtaining these utilities, through representative-leaf training and validation, accounts for about 91% of HARP-C's total cost and 71% of HARP-E's. For new domains, we only need to evaluate the saved representative-leaf models and update their utilities. Final ranking needs to be recomputed, and its computational cost is negligible.

(Continued in Part 2/4.)

AuthorsOpenReview ↗

Response to Reviewer J4mK (Part 2/4)

Question 1.

HARP-E provides a straightforward additive surrogate, whereas HARP-C is motivated by avoiding redundant positive contributions. However, the reported differences in accuracy and the final selected subset size provide only indirect evidence for this motivation. For example, section 5.1 relates the differences in performance to how they aggregate positive contributions. Could the authors provide additional analysis, such as examples illustrating whether HARP-C selects leaves with less overlapping training benefits? It would also be helpful to clarify how practitioners should choose between HARP-C and HARP-E for a new downstream task. Is there a practical criterion for choosing between them, or would both selected subsets need to be fine-tuned and compared on validation data? If so, how would this affect the overall computational cost?

A2. Thank you for asking for evidence about the envelopes themselves. We conducted an additional analysis of the subsets selected by HARP-C and HARP-E in nine MMLU settings with seed 42, reported in Table R5. HARP-C selects fewer leaves and fewer training examples than HARP-E in all nine settings, and in six of them all of its leaves are also selected by HARP-E. Even so, HARP-C achieves higher accuracy in six settings, so the extra leaves selected by HARP-E mostly do not improve accuracy. This is consistent with HARP-C's design of limiting redundant positive contributions. For example, on Qwen3-4B/Self-Instruct, HARP-C selects 5 leaves and 2,162 examples, compared with HARP-E's 18 leaves and 10,000 examples. HARP-E includes 80% of HARP-C's selected leaves, while HARP-C achieves higher accuracy, 71.4 versus 70.3, with substantially less training data.

Table R5. Selected training data and MMLU accuracy for seed 42. Values are HARP-C / HARP-E. Bold marks the higher accuracy.

Model / DatasetTrain RowsLeavesC in E Leaves (%)Acc. (%)
Llama-3.1-8B / SI968 / 1,7331 / 310063.0 / 63.2
Llama-3.1-8B / Wiz825 / 5,2371 / 710064.6 / 63.1
Llama-3.1-8B / Tulu445 / 3,5082 / 710064.3 / 64.4
Qwen3-8B / SI3,063 / 7,0055 / 1010075.4 / 74.3
Qwen3-8B / Wiz3,123 / 10,0004 / 1310075.3 / 74.6
Qwen3-8B / Tulu5,467 / 10,0008 / 4062.575.8 / 76.0
Qwen3-4B / SI2,162 / 10,0005 / 1880.071.4 / 70.3
Qwen3-4B / Wiz647 / 2,7503 / 910073.0 / 72.2
Qwen3-4B / Tulu4,729 / 7,8646 / 1283.372.5 / 72.0

Based on these results, we recommend HARP-C as the default when dataset redundancy or quality is unknown, since it avoids overcounting redundant gains and is less likely to select noisy examples. HARP-E is more suitable when the dataset has low redundancy and reliable quality, because it can capture complementary gains from multiple useful leaves within each domain. We will add this analysis and these recommendations to the revision.

Question 2.

The shrinkage weight ρg(d)\rho_g(d) depends on the across node variance τ2(d)\tau^2(d), which is set to be 0.01 in table 4. Could the authors clarify how the value of 0.01 was chosen? How sensitive is the final downstream performance to this parameter choice?

A3. Thank you for asking how the prior variance was set. We use a small prior variance, τ2=0.01\tau^2=0.01, to reduce the influence of noisy utility estimates. In HARP, the local-estimate weight is ρ=τ2τ2+V\rho=\frac{\tau^2}{\tau^2+V}. When the observed variance VV is high, more weight is placed on the global mean, preventing uncertain leaf estimates from misleading selection. A small prior variance encourages stronger shrinkage toward the global mean, reducing the influence of noisy leaf estimates on data selection.

For example, varying τ2\tau^2 from 10−310^{-3} to 10−110^{-1} across six settings produces highly consistent leaf rankings, with Spearman correlation at least 0.986. HARP-C selects identical training examples in all six settings, while HARP-E does so in five. These results support the choice of τ2=0.01\tau^2=0.01 and show that HARP's data selection remains robust. We will add this analysis in the revision.

Table D1. Sensitivity to the prior variance τ2∈[10−3,10−1]\tau^2 \in [10^{-3},10^{-1}].

SettingHARP-C unchangedHARP-E unchanged
Llama-3.1-8B / ARC4/44/4
Qwen3-8B / ARC4/44/4
Qwen3-4B / ARC4/42/4
Llama-3.1-8B / MATH4/44/4
Qwen3-8B / MATH4/44/4
Qwen3-4B / MATH4/44/4

(Continued in Part 3/4.)

AuthorsOpenReview ↗

Response to Reviewer J4mK (Part 3/4)

Question 3.

Is a separate training subset selected for each benchmark, or is a single subset selected using all four validation sets?

A4. Thank you for the question. Yes, each training subset corresponds to one validation set: for every model and training dataset, we select a separate subset for each target benchmark using that benchmark's validation set, rather than one common subset for all four benchmarks. HARP is a train-based method whose selection is aligned with the downstream objective, so when the downstream objective changes, the selection target changes accordingly, and so does the selected subset.

Question 4.

Which embedding model is used, and does it encode the instruction alone or the instruction–response pair? How sensitive are the final results to the choice of embedding model?

A5. Thank you for these questions. We encode instruction–response pairs with all-MiniLM-L6-v2 to build the hierarchy and use the same embedding model in all 36 settings.

To test the sensitivity to this choice, we conducted additional experiments that replace MiniLM with four other embedding models: all-mpnet-base-v2, bge-small, GTE, and mxbai. For each embedding model, we run HARP-C and HARP-E on ARC and MATH with Llama-3.1-8B, Qwen3-8B, and Qwen3-4B, using seeds 42, 1, and 2; the results are reported in Tables R6 and R7. Across the five embedding models, the average accuracy over these six settings differs by at most 1.3 points for HARP-C and 1.0 point for HARP-E. The embedding model therefore does not significantly affect performance, showing that HARP is robust to the choice of embedding model. We will add this ablation in the revision.

Table R6. HARP-C accuracy with five embedding models

ModelEvalMiniLM (paper)MPNetBGEGTEmxbaimax−min
Llama-3.1-8BARC77.9 ± 0.578.4 ± 0.578.0 ± 0.377.7 ± 0.677.0 ± 1.11.4
Qwen3-8BARC92.9 ± 0.392.0 ± 0.292.7 ± 0.192.4 ± 0.292.6 ± 0.10.9
Qwen3-4BARC88.7 ± 0.288.4 ± 0.288.3 ± 0.389.0 ± 0.188.5 ± 0.30.7
Llama-3.1-8BMATH8.5 ± 2.111.7 ± 0.611.8 ± 1.111.8 ± 1.28.6 ± 1.53.4
Qwen3-8BMATH48.2 ± 0.249.3 ± 0.251.6 ± 0.650.0 ± 1.149.2 ± 0.73.4
Qwen3-4BMATH46.9 ± 1.344.6 ± 0.448.3 ± 1.245.2 ± 1.647.7 ± 0.63.7
Avg (6)60.5 ± 0.860.7 ± 0.461.8 ± 0.661.0 ± 0.860.6 ± 0.71.3

Table R7. HARP-E accuracy with five embedding models

ModelEvalMiniLM (paper)MPNetBGEGTEmxbaimax−min
Llama-3.1-8BARC77.6 ± 0.476.1 ± 0.677.7 ± 1.277.0 ± 0.276.5 ± 1.01.6
Qwen3-8BARC93.1 ± 0.492.8 ± 0.492.4 ± 0.492.3 ± 0.192.2 ± 0.40.8
Qwen3-4BARC88.6 ± 0.289.0 ± 0.388.8 ± 0.288.6 ± 0.288.7 ± 0.50.4
Llama-3.1-8BMATH12.3 ± 1.311.0 ± 2.912.9 ± 1.810.4 ± 0.79.4 ± 0.83.5
Qwen3-8BMATH51.6 ± 1.249.5 ± 1.748.0 ± 1.848.8 ± 0.851.2 ± 0.83.6
Qwen3-4BMATH45.2 ± 0.745.5 ± 0.947.4 ± 0.245.2 ± 0.247.0 ± 1.02.2
Avg (6)61.4 ± 0.760.7 ± 1.161.2 ± 1.060.4 ± 0.460.9 ± 0.81.0

(Continued in Part 4/4.)

AuthorsOpenReview ↗

Response to Reviewer J4mK (Part 4/4)

Question 5.

Could the authors detail how size stratification is implemented when choosing representative leaves?

A6. Thank you for the question. We use size stratification to ensure that representative leaves cover different training sizes rather than relying only on embedding similarity. This is important because downstream utility depends on both the number of training examples and their content. For each parent, we select up to R=3R=3 representatives. We group leaves by size using 1D k-means with a BIC-like criterion, allocate representative slots across groups, and select embedding-diverse leaves within them. If a parent has at most RR leaves, we measure all of them.

Question 6.

NICE uses the full 10k budget, whereas HARP can select a smaller subset. since selecting more data does not necessarily improve performance, could the authors compare NICE with HARP at matched subset sizes?

A7. Thank you for the suggestion. We use 10k as the default budget for all baselines for a fair comparison. The baselines use a fixed number of training examples, but the optimal budget for each method cannot be determined before selection. In contrast, HARP dynamically selects the subset size, treating the budget as an upper bound, and achieves better performance with fewer training examples.

We further evaluate NICE, together with LESS and SHED, at 1k, HARP-C's size, and HARP-E's size, as shown in Table R8. Compared with 10k examples, NICE improves from 59.3 to 62.1 at 1k, 61.8 at HARP-C's size, and 60.9 at HARP-E's size. LESS decreases from 51.6 to 49.9, 49.7, and 50.5, respectively, and SHED improves from 56.4 to 60.7, 61.5, and 59.8, respectively. However, all these results remain below HARP-C's 64.2 and HARP-E's 63.5, so HARP's advantage holds at matched subset sizes. For the computational cost, please refer to A1.

Table R8. Dataset-size comparisons (bold: best; italic: second best)

MethodRows trained onMMLUARCGSM8KMATHAvg
LESS1k68.9 ± 0.484.7 ± 0.429.9 ± 1.415.9 ± 3.149.9 ± 1.3
LESSHARP-C's size68.0 ± 0.485.0 ± 0.430.5 ± 2.415.2 ± 3.149.7 ± 1.6
LESSHARP-E's size67.5 ± 0.683.8 ± 0.729.1 ± 1.821.4 ± 1.950.5 ± 1.3
LESS10k66.1 ± 0.783.0 ± 0.833.6 ± 6.123.8 ± 1.751.6 ± 2.3
NICE1k69.9 ± 0.586.2 ± 0.463.8 ± 2.028.3 ± 1.662.1 ± 1.1
NICEHARP-C's size69.9 ± 0.586.4 ± 0.462.1 ± 2.229.0 ± 1.961.8 ± 1.2
NICEHARP-E's size68.8 ± 0.785.7 ± 0.560.3 ± 3.928.7 ± 2.160.9 ± 1.8
NICE10k68.6 ± 0.785.9 ± 0.455.1 ± 2.827.9 ± 1.559.3 ± 1.3
SHED1k69.7 ± 0.985.0 ± 0.959.8 ± 5.028.3 ± 1.860.7 ± 2.1
SHEDHARP-C's size69.6 ± 0.785.1 ± 1.062.5 ± 3.029.0 ± 2.161.5 ± 1.7
SHEDHARP-E's size68.6 ± 0.984.5 ± 1.357.0 ± 5.629.1 ± 2.159.8 ± 2.5
SHED10k67.8 ± 0.983.8 ± 1.147.3 ± 3.226.8 ± 1.356.4 ± 1.6
HARP-COwn70.6 ± 0.285.7 ± 0.568.2 ± 2.132.1 ± 1.964.2 ± 1.2
HARP-EOwn69.9 ± 0.385.8 ± 0.564.1 ± 3.234.4 ± 1.563.5 ± 1.4

Question 7.

The data selection methods use additional downstream validation data to guide selection. Is this part of data also included in training for the full FT baseline?

A8. Thank you for the question. No. For each downstream benchmark, we use the validation set provided with the benchmark; if a benchmark does not provide one, we hold out part of its training set as the validation set. These validation examples are used only to guide selection. Full FT trains on the SFT dataset (SI, WizardLM, or Tulu), so its training data does not contain any validation examples, and the final test sets remain separate from selection.

Reviewer_Ech7OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies data selection for supervised finetuning of LLMs. Existing train-free methods rely on proxy signals that may be misaligned with the downstream objective, while train-based methods are computationally expensive. To address these limitations, the paper proposes HARP. HARP organizes the training pool into a node–leaf hierarchy, measures the utility of only a few representative leaves, and applies empirical Bayes shrinkage to infer the utility of unmeasured leaves. Building on these estimates, it selects data using two complementary envelopes: HARP-C and HARP-E. Experiments across 36 settings demonstrate the effectiveness of the proposed method.

Critical strengths
  1. Finetuning data selection is a practical problem in LLM post-training. The proposed HARP preserves downstream alignment while reducing selection cost, which makes it a useful solution for data selection.
  2. The experiments are comprehensive. The proposed method is evaluated on 3 base models, 3 finetuning datasets and 4 reasoning benchmarks, and the paper also provides a corresponding theoretical analysis.
Critical weaknesses
  1. As also noted in [1], random selection is a strong strategy. In Table 1, for Qwen3-8B on Tulu, Random outperforms HARP on several metrics. The paper should discuss why Random is excluded from the ranking in Table 1.

  2. In Table 1,some baselines perform worse than the Base model on several evaluation benchmarks and models. The reason for this needs to be discussed.

  3. The paper states that "All budgeted methods share a 10,000 example maximum: the baselines use the full budget, whereas HARP treats it as a cap." Is this setting reliable? Are all the other methods statically truncated at this budget? Some tasks may not need 10,000 examples in the first place.

  4. The first paragraph of the Introduction could be split in three. The first paragraph would introduce the background and motivation, and a second paragraph would then cover recent work, and the third would present the proposed method, HARP. I think this would make the Introduction easier to follow.

[1] Rethinking Data Selection at Scale: Random Selection is Almost All You Need

Questions for authors
  1. Why is Random treated as a reference column and excluded from the ranking in Table 1 (together with Base and Full FT)? Unlike Base and Full FT, Random uses the same 10k budget as the other selectors and this is not discussed.

  2. In several settings, almost all baselines, including Random and Full FT, fall far below the untuned Base model. Could the authors explain this degradation? In these cells HARP roughly matches Base. Is HARP's advantage there due to selecting very few examples, so that the base model is left largely unchanged, rather than to selecting useful data?

  3. The paper states that "the baselines use the full budget, whereas HARP treats it as a cap and dynamically selects its best prefix." Why is this setting used? Some of the baselines also rank training examples, so the same best-prefix rule may be applied to them. Is it fair to allow only HARP to adapt its subset size?

9 comments
AuthorsOpenReview ↗

Response to Reviewer Ech7 (Part 1/3)

We thank the reviewer for the constructive feedback and for recognizing HARP's practical value and comprehensive evaluation.

Weakness 1.

As also noted in [1], random selection is a strong strategy. In Table 1, for Qwen3-8B on Tulu, Random outperforms HARP on several metrics. The paper should discuss why Random is excluded from the ranking in Table 1.

[1] Rethinking Data Selection at Scale: Random Selection is Almost All You Need

A1. Thank you for highlighting this point. On average, HARP-C and HARP-E reach 64.2 and 63.5, which is 7.5 and 6.8 points above Random's 56.7. They are also 14.2 and 13.5 points above Base's 50.0, and 9.5 and 8.8 points above Full FT's 54.7. In Table 1, our intent was to highlight the performance of the data selection methods, so the bold and underline marks compare only those methods. Besides, adding the comparison with Random, Base, and Full FT does not change our conclusion. In the revision, we will add a discussion of the comparison with Random. We will also mark the best and second-best results in Table 1 across all methods, including Base, Random, and Full FT.

Weakness 2.

In Table 1,some baselines perform worse than the Base model on several evaluation benchmarks and models. The reason for this needs to be discussed.

A2. Thank you for raising this question. The training datasets are general, which cover diverse tasks and distributions with varying data quality. They may contain irrelevant, noisy, or incorrect examples for a particular downstream task.

Some prior selectors use proxies that may not align with downstream objectives. For example, DQ uses embedding-based selection to select diverse data, but embedding diversity is not directly aligned with downstream accuracy: DQ averages 45.7, below Base's 50.0.

In contrast, HARP directly estimates performance from downstream objectives, uses empirical-Bayes shrinkage to reduce estimation noise, and selects data based on estimated gains and harms (§3.3–3.5). This better aligns data selection with objectives and supports HARP's robustness across the evaluated training datasets and tasks. HARP outperforms the Base model on average, with HARP-C and HARP-E improving accuracy by 14.2 and 13.5 percentage points, respectively.

(Continued in Part 2/3.)

AuthorsOpenReview ↗

Response to Reviewer Ech7 (Part 2/3)

Weakness 3.

The paper states that "All budgeted methods share a 10,000 example maximum: the baselines use the full budget, whereas HARP treats it as a cap." Is this setting reliable? Are all the other methods statically truncated at this budget? Some tasks may not need 10,000 examples in the first place.

A3. Thank you for the question. We use the 10k as the default budget across all the baselines for the fair comparison. The baselines use a fixed number of training examples. However, the optimal budget for each method cannot be defined before the selection. In contrast, HARP dynamically selects the subset size, treating the budget as an upper bound, and achieves better performance with fewer training examples.

We further evaluate LESS, NICE, and SHED at 1k, HARP-C's size, and HARP-E's size, as shown in Table R1. HARP still achieves the best average performance compared with all baselines across different budgets, showing that HARP is robust under different budget settings. Compared with 10k examples, LESS decreases from 51.6 to 49.9 at 1k, 49.7 at HARP-C's size, and 50.5 at HARP-E's size. NICE improves from 59.3 to 62.1, 61.8, and 60.9, respectively. SHED also improves from 56.4 to 60.7, 61.5, and 59.8, respectively. However, all these results remain below HARP-C's 64.2 and HARP-E's 63.5.

Table R1. Dataset-size comparisons (bold: best; italic: second best)

MethodRows trained onMMLUARCGSM8KMATHAvg
LESS1k68.9 ± 0.484.7 ± 0.429.9 ± 1.415.9 ± 3.149.9 ± 1.3
LESSHARP-C's size68.0 ± 0.485.0 ± 0.430.5 ± 2.415.2 ± 3.149.7 ± 1.6
LESSHARP-E's size67.5 ± 0.683.8 ± 0.729.1 ± 1.821.4 ± 1.950.5 ± 1.3
LESS10k66.1 ± 0.783.0 ± 0.833.6 ± 6.123.8 ± 1.751.6 ± 2.3
NICE1k69.9 ± 0.586.2 ± 0.463.8 ± 2.028.3 ± 1.662.1 ± 1.1
NICEHARP-C's size69.9 ± 0.586.4 ± 0.462.1 ± 2.229.0 ± 1.961.8 ± 1.2
NICEHARP-E's size68.8 ± 0.785.7 ± 0.560.3 ± 3.928.7 ± 2.160.9 ± 1.8
NICE10k68.6 ± 0.785.9 ± 0.455.1 ± 2.827.9 ± 1.559.3 ± 1.3
SHED1k69.7 ± 0.985.0 ± 0.959.8 ± 5.028.3 ± 1.860.7 ± 2.1
SHEDHARP-C's size69.6 ± 0.785.1 ± 1.062.5 ± 3.029.0 ± 2.161.5 ± 1.7
SHEDHARP-E's size68.6 ± 0.984.5 ± 1.357.0 ± 5.629.1 ± 2.159.8 ± 2.5
SHED10k67.8 ± 0.983.8 ± 1.147.3 ± 3.226.8 ± 1.356.4 ± 1.6
HARP-COwn70.6 ± 0.285.7 ± 0.568.2 ± 2.132.1 ± 1.964.2 ± 1.2
HARP-EOwn69.9 ± 0.385.8 ± 0.564.1 ± 3.234.4 ± 1.563.5 ± 1.4

(Continued in Part 3/3.)

AuthorsOpenReview ↗

Response to Reviewer Ech7 (Part 3/3)

A3 (continued). Moreover, we conducted additional experiments on training efficiency, reported in Table R2. HARP is much cheaper than train-based baselines while achieving higher accuracy. Compared with LESS and NICE, whose average accuracies are 51.6 and 59.3, HARP-C/E achieve 64.2/63.5 average accuracy, with 8.7–9.5×/6.8–7.4× lower total costs. Selection costs are 9.1–9.6× lower, while final training costs are 5.3–8.1× lower for HARP-C and 1.3–2.0× lower for HARP-E. Compared with SHED, whose average accuracy is 56.4, HARP-C/E also achieve higher accuracy with 2.5×/2.0× lower total costs, 1.9× lower selection costs, and 9.2×/2.2× lower training costs. Compared with Random and DQ, HARP-C/E cost only about 1.2×/1.5× as much while achieving higher average accuracy, 64.2/63.5 versus 56.7 for Random and 45.7 for DQ. DSIR is cheaper because it selects shorter examples, but HARP-C/E still achieve higher accuracy, 64.2/63.5 versus 56.0, at 2.2×/2.8× its total cost. Compared with full fine-tuning, HARP-C/E achieve higher accuracy, 64.2/63.5 versus 54.7, with 7.1×/5.5× lower total costs. These results show that HARP achieves a better accuracy–efficiency trade-off than the baselines. All costs are in units of 101710^{17} flops.

Table R2. Mean selection and training costs (×1017\times 10^{17} FLOPs)

MethodMean SelectMean TrainMean Total
Random0.03.33.3
DQ0.03.33.3
DSIR0.01.81.8
Full0.027.427.4
LESS31.81.833.6
NICE33.82.836.6
SHED6.63.29.8
HARP-C3.50.43.9
HARP-E3.51.44.9

Selection includes embedding computation, hierarchy construction, representative-leaf training, validation evaluation, utility estimation, and selection search.

Weakness 4.

The first paragraph of the Introduction could be split in three. The first paragraph would introduce the background and motivation, and a second paragraph would then cover recent work, and the third would present the proposed method, HARP. I think this would make the Introduction easier to follow.

A4. Thank you for the suggestion. We will revise the Introduction accordingly.

Question 1.

Why is Random treated as a reference column and excluded from the ranking in Table 1 (together with Base and Full FT)? Unlike Base and Full FT, Random uses the same 10k budget as the other selectors and this is not discussed.

A5. Thank you for the question. Please refer to A1.

Question 2.

In several settings, almost all baselines, including Random and Full FT, fall far below the untuned Base model. Could the authors explain this degradation? In these cells HARP roughly matches Base. Is HARP's advantage there due to selecting very few examples, so that the base model is left largely unchanged, rather than to selecting useful data?

A6. Thank you for the question. Please refer to A2 for why the baselines fall below Base, and to A3 for the comparison across different budgets.

Question 3.

The paper states that "the baselines use the full budget, whereas HARP treats it as a cap and dynamically selects its best prefix." Why is this setting used? Some of the baselines also rank training examples, so the same best-prefix rule may be applied to them. Is it fair to allow only HARP to adapt its subset size?

A7. Thank you for the question. Please refer to A3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions

We thank all reviewers for their constructive feedback. We have uploaded a revised paper in which the changes are marked in purple. Below we summarize the changes and the comments they address.

Main results (Section 5.1, Table 1)

  • Table 1 now marks the best and second-best results across all methods, including Base, Random, and Full FT, and groups its columns into reference, train-free, and train-based methods. (Ech7 W1, Q1; 46bk W1)
  • Section 5.1 now compares HARP with Random, Full FT, and Base, and explains why baselines can fall below Base. (Ech7 W1, W2, Q1, Q2)

Data and compute efficiency (Section 5.2)

  • New Table 2 reports the end-to-end cost of every method, split into data selection and final finetuning. Appendix F adds HARP's cost breakdown by stage, the comparison between leaf-level and example-level queries, and the costs that can be reused when the downstream task changes. (J4mK W; 46bk W2)
  • Appendix G (Table 8) evaluates the baselines at 1k examples and at the subset sizes selected by HARP-C and HARP-E. (Ech7 W3, Q3; J4mK Q6; 46bk Q3)

HARP-C versus HARP-E

  • Section 5.1 and Appendix H (Table 9) analyze the subsets selected by the two variants and recommend how to choose between them. (J4mK Q1; 46bk Q2)

Presentation (Sections 1 and 3)

  • Section 3 now includes Algorithm 1, which summarizes the pipeline, displays the HARP-C and HARP-E envelope equations, and uses shorter paragraphs. (46bk W1)
  • The Introduction is split into separate paragraphs for background, prior work, and HARP. (Ech7 W4)

Implementation details (Section 4, Appendix E)

  • Section 4 now states the embedding model and the HARP hyperparameters. Appendix E describes how validation domains are defined, that a separate subset is selected for each benchmark, where the validation data comes from, the target number of child groups, why we use farthest-first instead of k-means, and how representative leaves are selected. (J4mK W, Q3, Q4, Q5, Q7; 46bk W3, Q1)

Additional ablations (Appendix I)

  • Appendix I reports sensitivity to the embedding model (Tables 10 and 11) and to the prior variance (Table 12). (J4mK Q2, Q4)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.