acceptodds
Under review as a conference paper at ICLR 2027

Convex Multi-Target Dataset Selection for Post-Training via Portfolio Optimization

Abstract

Post-training multiple target capabilities under a fixed auxiliary-data budget is harder than single-target selection because one auxiliary mixture must account for transfer across potentially conflicting targets. Inspired by single-target data selection with Kernel Mean Matching (KMM), we view this problem through the lens of Markowitz portfolio optimization: auxiliary datasets are assets, target-auxiliary alignment is return, and the similarity matrix among auxiliary datasets is the risk in the capital asset pricing model. This view naturally extends to multiple targets by letting a user-specified preference vector define the aggregate return across target capabilities, while the redundancy penalty among auxiliary datasets remains shared. Based on this formulation, we introduce Multi-Target KMM (MT-KMM), which uses linear scalarization to compute continuous dataset scores, then post-processes them into a common auxiliary training mixture for multiple targets. Motivated by Adam's first update from zero moments, we represent each dataset by its normalized signed gradient at a frozen reference model, which coincides with Adam's normalized first-step features from zero moments in the zero-stabilizer limit. The gradients are computed only once, and the resulting selection can be reused to train models up to 70B. Theoretically, we show that finite-preview MT-KMM valuations converge to their population counterparts at the rate for raw gradients and, under a linear near-zero condition, for SignGD, where is the size of the preview gradients. Empirically, MT-KMM achieves the highest mean target accuracy among the compared methods at every tested Llama scale and auxiliary-data budget for SFT, with gains extending to Qwen models from 0.6B to 32B and to RL post-training. By varying the target preference vector across different target capabilities, we show that MT-KMM can adapt the selected auxiliary mixture to different downstream priorities.

Primary area:
foundation or frontier models, including LLMs
Keywords:
data valuation, large language model post-training, data marketplace, kernel mean matching, gradient-based attribution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_o6KuOpenReview ↗
Rating
1(−1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1(−1)
Read review
Paper summary

The paper studies which auxiliary datasets to mix into post-training, and in what proportions, when several target tasks matter at once. It takes the single-target-task dataset selector of prior work, which scores candidate datasets by the alignment of their loss gradients with the target loss gradient, while penalizing redundancy among candidates, and applies it to a preference-weighted scalarized combination of the target task losses. The experiments show that the resulting score-weighted mixtures reach higher target-task accuracy than baselines across model sizes.

Critical strengths

The problem is relevant, since post-training has to serve several tasks at once. The experiments are extensive and at a reasonable scale.

Critical weaknesses

The proposed method is incremental: it is that of [1] applied to a scalarization of the target task losses instead of a single one, plus a modification already explored in [2], taking the gradients’ signs as features while discarding their magnitudes. The paper says so itself, yet lists the formulation and the justification of the sign features as its contributions.

Even taken as a modification of [1], the method is never shown to improve on [1]: the paper does not even run [1] unchanged on the scalarized loss. Its gains over the non-redundancy-accounting alignment-based baselines appear only once the budget is split unevenly across the selected datasets, according to their scores. This rule is never justified, and [1] tried it and found it harmful, which the paper does not discuss.

The remaining listed contributions are weak. The theoretical bounds seem vacuous at the paper’s own sample sizes, and the claimed transfer of selections across model families and post-training algorithms is never tested.

What seems novel is the benchmark and the budget-splitting rule.

[1] Zeng et al. Convex dataset valuation for post-training. ICML, 2026.

[2] Xia et al. LESS: Selecting influential data for targeted instruction tuning. ICML, 2024.

Questions for authors
  1. Does gradient-alignment selection match the proposed method once its budget shares are made as uneven as the method’s, rather than proportional to alignment as in the no-K control?
  2. What do the bounds of Theorems 4.1 and 4.2 evaluate to for the Llama-3.2-1B setting considered in the paper?
  3. [1] found softmax weighting of the selected datasets by their scores harmful, while here proportional weighting seems to carry much of the gain. Why?
5 comments
AuthorsOpenReview ↗

Response to Reviewer o6Ku

Thanks for your time and constructive feedback on our manuscript. Please find our response to your comments below.

Comparison with Zeng et al.'s method

Reviewer:

Even taken as a modification of [1], the method is never shown to improve on [1]: the paper does not even run [1] unchanged on the scalarized loss.

This rule is never justified, and [1] tried it and found it harmful, which the paper does not discuss. [1] found softmax weighting of the selected datasets by their scores harmful, while here proportional weighting seems to carry much of the gain. Why?

Response:

Thanks for pointing it out. In Table R7 (now Table 31 in the paper), we included Zeng et al.’s two setups, 1) equal allocation of auxiliary datasets 2) non-uniform allocation with softmax weighting, both using raw gradients as dataset representations. MT-KMM outperforms all variants in Zeng et al. Weighted allocation follows from the portfolio view, in which the KMM scores are themselves the allocation (Sec. 4).

Table R7. From Zeng et al.'s method to MT-KMM, one change at a time.

MethodFeaturesAllocationAccuracy
Zeng et al., unchangedraw gradientsequal39.51
Zeng et al., softmax allocationraw gradientssoftmax40.11
Zeng et al., weighted allocation (paper Table 3a, "Raw")raw gradientsweighted42.62
MT-KMMSignGDweighted43.42

Why normalization, not softmax? First, in Table R7, as [1] already suggested, softmax is suboptimal. Our normalization is not the same as softmax, although both are non-uniform allocation rules. One potential advantage of our method is that it preserves the relative magnitudes of original very informative KMM scores. Softmax preserves exponentiated score difference instead, which may substantially flatten allocations when score differences are small.

 


 

Is the gain only the uneven split?

Reviewer:

Its gains over the non-redundancy-accounting alignment-based baselines appear only once the budget is split unevenly across the selected datasets, according to their scores. Does gradient-alignment selection match the proposed method once its budget shares are made as uneven as the method's, rather than proportional to alignment as in the no-K control?

Response:

Thanks for the question. We would like to clarify that our no-K ablation already uses nonuniform budget allocation, as described in App. C.2 in the original manuscript. The difference between no-K and MT-KMM lies in how scores are computed, rather than the allocation rule.

To “match budget shares made as uneven as the method's”, we could artificially sharpen the allocation. However, such matching would require calibrating the no-K allocation using MT-KMM's solution, which itself depends on K. It would not be a fair and independent gradient alignment only baseline.

Vacuous theoretical bounds

Reviewer:

The theoretical bounds seem vacuous at the paper's own sample sizes.

Response:

We agree that the bounds are conservative at our experimental sample size. As already noted in Remark 4.2, our bounds characterize asymptotic convergence rather than providing tight numerical certificates at the finite preview sizes used in our experiments. Building on Zeng et al. (2026), our main theoretical contribution is extending finite-preview convergence guarantees to the multi-target setting, uniformly over target preferences, and further establishing convergence for SignGD representations under a near-zero condition.

Our empirical results complement these asymptotic guarantees by demonstrating stability at practical preview sizes. Appendix D.9 shows that increasing the preview size from 256 to 512 or 1024 preserves an average of 9.8/10 selected datasets, with only 2.9–3.5% of the budget reallocated and similar downstream accuracy.

 


 

Transfer across model families and algorithms

Reviewer:

The claimed transfer of selections across model families and post-training algorithms is never tested.

Response:

Sorry for the confusion. We meant to say the empirical results are strong for different model families and post-training algorithms. We reused selections only across model scales within each family and each algorithm in the submitted version. We have updated our conclusion in the updated manuscript.

Following the reviewer's suggestion, we tested both kinds of transfer in new experiments. The cross-family result adds to the practical value of MT-KMM.

  • Across families: For SFT experiments, we compute budget allocation with KMM using gradients from Llama-1B and apply this same allocation for Qwen-0.6B. We also run the experiments the other way around, computing w~\tilde{w} on Qwen and applying it to Llama.

  • Across training algorithms: For the GRPO setting, we computed the MT-KMM selection from SFT gradients instead of GRPO policy gradients, with the same targets and auxiliary pool, and trained GRPO on it.

In Tables R8–R9, MT-KMM selections do transfer across model families but not across post-training algorithms. We include the results in App. E.2 in the updated manuscript.

Table R8. Training each model family on the other family's MT-KMM mixture (new).

Model trainedIts own One StepIts own MT-KMMOther family's MT-KMM mixture
Llama-3.2-1B39.5143.4242.83 (from Qwen3-0.6B)
Qwen3-0.6B45.8146.6047.38 (from Llama-3.2-1B)

Table R9. GRPO trained on selections computed with different gradients (Llama-3.2-1B).

Selection used for GRPO trainingAccuracy
random (paper Table 2c)38.44
One Step, GRPO policy gradients (paper Table 2c)40.82
MT-KMM, GRPO policy gradients (paper Table 2c)42.33
MT-KMM, SFT gradients (new)39.01

I thank the authors for the response and the new experiments. Three of my concerns have been resolved: the paper now runs [1] unchanged, motivates the budget-splitting rule, and corrects and tests the transfer claim.

I maintain my rating, however, because my main concern remains unresolved. The paper still claims as main contributions the multi-target formulation and the justification of the sign features. Both are ported from prior work [1, 2], and neither accounts for the method's practical gain: in the response to Reviewer UNbG, the multi-target formulation performs comparably to solving for each target separately, and in the new Table 31, most of the gain over [1] comes from the budget-splitting rule and little from the sign features. These claims therefore remain unsupported.

What seems novel, then, is the budget-splitting rule and the benchmark. The rule is a proportional variant of the score weighting already tested in [1], and it remains unclear whether its gain comes from the method's scores or simply from uneven shares. While extensive and useful, the benchmark cannot carry this paper, whose claims are about the method.

Unsupported main claims and useful but modest novel parts do not warrant acceptance.

Reviewer_UNbGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to select and weight auxiliary datasets for several target tasks under a fixed data budget. MT-KMM combines target features according to user preferences and fits this composite feature with a weighted combination of auxiliary features. These features are normalized signs of average gradients at a frozen model. The method then keeps the largest positive dataset scores and normalizes them into training proportions. The paper gives conditional stability bounds for the estimated scores and evaluates the method on Llama and Qwen models, with additional GRPO experiments.

Critical strengths
  1. I find the method practically appealing. Selection is inexpensive, and a mixture computed on a small model can be reused within the same model family. In Table~2, MT-KMM improves over One Step by 3.91, 2.40, and 1.59 percentage points on the three Llama scales. The Qwen and GRPO results provide further evidence that the method is useful beyond one model configuration.

  2. I appreciate the experimental controls. The paper checks the effects of extra target preview labels and same-target auxiliary datasets (Tables ~ 14 -- 15), and separates selection from allocation ( Table~3 ). These experiments address plausible alternative explanations for the gains. In particular, the allocation ablation helps explain where much of the improvement comes from.

Critical weaknesses
  1. My main concern is the gap between the matching objective and the actual training mixture. Even under the first-step Adam approximation, updating on a mixture is generally different from combining dataset-wise signed updates:
\neq \sum_i q_i\operatorname{sign}(g_i).$$ The optimizer also permits negative scores, which are discarded before truncation and normalization. The stability results do not establish that the resulting mixture still achieves good matching. Since allocation contributes substantially to the gains in Table 3, I would appreciate a comparison with directly optimizing nonnegative mixture weights to better justify this design. 2. **I find the additional multi-target contribution limited.** With fixed representations, Equation (5) is equivalent to single-target KMM using a composite target feature: $$\phi_{\mathrm{tar},\alpha} =\sum_r\alpha_r\phi_{\mathrm{tar},r}.$$ The shared kernel and Pareto guarantee follow from this construction, and the guarantee concerns matching objectives rather than downstream performance. Table 7b compares against single-target selection, which uses less target information. I would appreciate a comparison with combining all target-specific KMM mixtures under the same budget and dataset-count limit to clarify the benefit of joint selection.
Questions for authors
  1. How well does the signed matching objective predict measured target-loss improvement under Adam? In the reference setting, how does it compare with retaining raw target gradients or using ℓ2\ell_2-normalized gradients?

  2. How much negative score mass is removed, and how much does mixture extraction change the matching error? How does the method compare with directly optimizing nonnegative weights that sum to one, with the same final dataset-count limit?

  3. How does joint selection compare with the α\alpha-weighted combination of all target-specific KMM mixtures, followed by the same top-kk restriction and normalization? This comparison would help me assess the additional multi-target contribution.

4 comments
AuthorsOpenReview ↗

Response to Reviewer UNbG

Thanks for your time and constructive feedback on our manuscript. Please find our response to your comments below.

Gap between the matching objective and the training mixture

Reviewer:

Even under the first-step Adam approximation, updating on a mixture is generally different from combining dataset-wise signed updates. The optimizer also permits negative scores, which are discarded before truncation and normalization. The stability results do not establish that the resulting mixture still achieves good matching. How much negative score mass is removed, and how much does mixture extraction change the matching error? How does the method compare with directly optimizing nonnegative weights that sum to one, with the same final dataset-count limit?

Response:

Adam approximation. As explicitly noted in Sec. 4.2 (following Eq. 7), our signed-feature objective is a matching surrogate, rather than an exact characterization of the training update. The first-step Adam motivation applies to individual dataset representations; we do not assume that combining these representations is equivalent to applying Adam to the mixed gradient.

How much negative score mass is removed? Although our optimization permits signed coefficients, we observe no negative scores at our default γ=0.05\gamma = 0.05. In contrast, removing the ℓ1\ell_1 penalty (γ=0\gamma = 0) produces 26 and 29 negative coefficients on Llama and Qwen, respectively, suggesting that ℓ1\ell_1 regularization plays an important role in suppressing negative coefficients in our setting, although nonnegativity is not guaranteed in general.

How much does mixture extraction change the matching error? To assess whether postprocessing preserves gradient matching, we measure the scale-invariant matching residual:

ρ(w)=min⁡c≥0∥cΦw−ϕtar,α∥22∥ϕtar,α∥22.\rho(w)=\min_{c\geq 0}\frac{\lVert c\Phi w-\phi_{\mathrm{tar},\alpha}\rVert_2^2}{\lVert\phi_{\mathrm{tar},\alpha}\rVert_2^2}.

Here c≥0c\geq 0 allows positive rescaling to find the best directional match. We focus on matching direction because the overall update magnitude can be adjusted through the learning rate. A smaller ρ\rho indicates better matching. In Table R3, postprocessing barely changes the matching residual, while uniform allocation substantially worsens it. This provides empirical evidence that our final training mixture preserves the matching quality of the continuous solution. We have added this analysis to Appendix B.3.

Table R3. Matching residual of each mixture.

WeightsMatching residual ρ\rho
Continuous QP scores (all 96 datasets)0.31
MT-KMM mixture (top 10, normalized)0.32
Uniform allocation (same 10 datasets)0.50

How does the method compare with directly optimizing nonnegative weights that sum to one? In a new experiment, we solved the QP directly over nonnegative weights that sum to one, with the same limit of 10 datasets (i.e., the simplex QP). As shown in Table R4, the simplex QP and the MT-KMM training mixture tie (difference not significant, p=0.62p=0.62). This is consistent with our observation that the unconstrained KMM solutions already contain no negative coefficients.

Table R4. Accuracy of each mixture.

MixtureAccuracy
MT-KMM training mixture43.42
Simplex QP (new)43.46

Does the score predict the measured loss drop?

Reviewer:

How well does the signed matching objective predict measured target-loss improvement under Adam? In the reference setting, how does it compare with retaining raw target gradients or using ℓ2-normalized gradients?

Response:

We measured the reduction in target loss after one Adam step per auxiliary dataset, starting from the same frozen model (Table R5). For each target, we computed Spearman correlations across all 96 auxiliary datasets between their observed loss reductions and their alignment scores with the target, using different feature representations. A higher correlation indicates that the alignment scores better predict which auxiliary datasets yield greater target-loss improvement under Adam.

Table R5. Spearman correlation between alignment score and measured one-step loss drop, mean over the 8 targets (reference setting).

FeatureSpearman
Raw gradients0.58
SignGD (ours)0.85
ℓ2-normalized gradients0.83

Both normalized features predict the loss drop far better than raw gradients, and about equally well. We use SignGD because it coincides with Adam's first update from zero moments and our actual optimizer, on which our stability analysis relies (Sec. 4.2, App. B).

 


 

Joint selection vs combining single-target KMM solutions

Reviewer:

I would appreciate a comparison with combining all target-specific KMM mixtures under the same budget and dataset-count limit to clarify the benefit of joint selection. How does joint selection compare with the α-weighted combination of all target-specific KMM mixtures, followed by the same top-k restriction and normalization?

Response: We thank the reviewer for suggesting this comparison. We added the requested baseline by taking the α\alpha-weighted average of normalized single-target KMM solutions, followed by the same top-k selection and allocation procedure under identical training budgets.

Table R6. Joint QP vs α\alpha-weighted average of single-target KMM solutions: mean accuracy with 2 targets (setting of paper Table 20a) and 8 targets (reference setting).

Method2 targets8 targets
MT-KMM (joint QP, ours)47.4943.42
α\alpha-weighted average of single-target KMM solutions47.1544.01

The two approaches achieve comparable performance, with neither consistently outperforming the other. Our motivation for joint optimization is its direct derivation from linear scalarization of the target-specific matching objectives, yielding a single convex QP with a shared redundancy matrix KK. Joint optimization also requires only one QP solve, rather than one per target.

Thank you for the additional experiments. The Adam results support SignGD for ranking individual auxiliary datasets, and the residual and simplex-QP comparisons address my concern about mixture extraction in the reference setting. The joint and per-target approaches achieve comparable accuracy, with the joint method requiring only one QP solve. How well the score predicts the Adam loss reduction for the final mixture remains open. I would welcome including the new comparisons in the manuscript. My overall rating remains unchanged

Thank you very much for your comments and feedback, we really appreciate them. We will add those experiments to our paper.

Reviewer_er2fOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper extends gradient-space kernel mean matching to auxiliary-dataset selection for multiple target tasks. A preference vector combines target representations, and a convex quadratic objective balances target alignment against similarity among auxiliary datasets. Normalized signed gradients are used as representations, and the resulting scores are converted into a training mixture. The paper establishes stability of continuous scores under stated conditions and evaluates the method across Llama and Qwen models, data budgets, and post-training procedures.

Critical strengths

The empirical evaluation covers several model scales, two model families, and both SFT and GRPO. The controls for additional preview labels and same-target auxiliary datasets address important alternative explanations.

The ablations separate the effects of signed representations, auxiliary interactions, and allocation weights. In particular, Table 3 shows that learned allocation contributes materially to the full method’s performance.

Critical weaknesses

The methodological novelty is incremental. Equation (5) reduces the multi-target objective to single-target matching with a composite target representation. The underlying alignment-versus-redundancy objective and portfolio interpretation already appear in Zeng et al. (2026). The main additional value is the signed representation, its analysis, and the broader empirical study.

Most performance comparisons use multi-target adaptations of single-target selectors. A direct comparison with an existing multi-target mixture optimizer under a similar selection-compute budget is missing. This leaves the practical trade-off against methods designed for multiple targets less clear.

Questions for authors

Can a compatible multi-target mixture optimizer be compared under a similar selection-compute budget? Section 2 discusses these methods, but the reported comparisons mainly adapt single-target selectors.

For the nonuniform preferences in Table 20, how much do the mixture weights and per-target accuracy trade-offs vary when only the preview samples are redrawn? Appendix D.9 studies preview variation in the standard eight-target setting, while Section E.3 fixes the preview for the reported training-seed comparisons.

3 comments
AuthorsOpenReview ↗

Response to Reviewer er2f

Thanks for your time and constructive feedback on our manuscript. Please find our response to your comments below.

Comparison with a multi-target mixture optimizer

Reviewer:

Most performance comparisons use multi-target adaptations of single-target selectors. A direct comparison with an existing multi-target mixture optimizer under a similar selection-compute budget is missing. Can a compatible multi-target mixture optimizer be compared under a similar selection-compute budget?

Response:

We thank the reviewer for this suggestion. As discussed in Sec.2, existing multi-target methods differ from our setting substantially, which considers one fixed dataset-level auxiliary mixture under user-specified target preferences and a limited selection-compute budget.

Specifically, GRAPE, TiKMiX, CAMEL, and OP-Mix focus on pretraining mixture optimization; AutoMixAlign targets DPO; and BIDS, ParetoHqD, NAG, and G2IS operate at the instance level rather than selecting entire datasets. DynaMiCS and MOSAIC are more closely related, but present substantial adaptation challenges:

  • DynaMiCS optimizes target improvement under explicit capability-preservation constraints, requiring target/constraint assignments and thresholds not specified by our preference vector α\alpha. Moreover, its experiments involve only 3-6 fine-tuning datasets, compared with our 96, making its repeated per-dataset training probes substantially more costly in our setting.
  • MOSAIC relies on task-specific failure annotations to guide mixture optimizations. Such annotations are not available in our 8-target benchmark and would require designing a new diagnostic framework.

Thus, while these methods are relevant, none directly supports our setting under a comparable selection compute budget without substantial modifications to its original procedure.

The closest applicable method is FastMix (Tan et al., ICLR 2026), which learns dataset-level mixture weights during one training run. Optimized on the average of the eight target losses, it is 8.0–8.7 points below MT-KMM in all five seeds, at higher selection cost (Table R1). We have added it to Sec. 2, App. C.2 and Tables 6 and 18.

Table R1. FastMix vs MT-KMM on Llama-3.2-1B, eight targets, 96 auxiliary datasets (new).

MethodAccuracyvs MT-KMMSeeds wonSelection GPU-h
MT-KMM43.42——0.7
FastMix, short search34.71−8.710/51.6
FastMix, default search35.42−8.000/511.2

 


 

Stability of the preference sweep under redrawn previews

Reviewer:

For the nonuniform preferences in Table 20, how much do the mixture weights and per-target accuracy trade-offs vary when only the preview samples are redrawn?

Response:

We repeated the preference experiments in Table 20a with 10 independent preview draws. Across all 10 draws, increasing LEDGAR preference consistently improves LEDGAR accuracy while reducing RACE accuracy. The mixture weights are also stable, with a typical redraw reallocating only 3-4% of the auxiliary budget. These results confirm that the preference-dependent accuracy trade-offs are robust to selection randomness, rather than artifacts of a particular preview draw. We included this analysis in Appendix E.1, with accuracy in Table 22 and budget shares in Table 23.

Table R2. Per-target accuracy (mean ±\pm std) across 10 preview draws.

PreferenceLEDGAR acc.RACE acc.
α=0.3\alpha = 0.348.56 ±\pm 0.9143.26 ±\pm 0.23
α=0.5\alpha = 0.552.62 ±\pm 0.8442.24 ±\pm 0.42
α=0.7\alpha = 0.753.82 ±\pm 1.0340.99 ±\pm 0.75

Thank you for the response and additional experiments. They address my main questions and I will maintain my weak accept rating.

Thank you very much!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We sincerely thank the reviewers for their detailed feedback, recognizing the practical relevance, computational efficiency, and extensive empirical evaluation of our method. We uploaded a revised PDF, where revised text is in blue. We now provide a summary of all changes to the paper and common questions here.

1. Changes to the paper

We have uploaded a revised manuscript, with changes highlighted in blue. The main revisions are:

  • Multi-target comparisons (er2f): Added FastMix as a multi-target mixture-optimizer baseline (Sec. 2, App. C.2, Tables 6 and 18).
  • Preference robustness (er2f): Repeated the preference experiments over 10 preview draws to evaluate allocation and accuracy stability (App. E.1, Tables 22–23).
  • Matching and allocation (UNbG): Analyzed negative scores and matching preservation after mixture extraction (App. B.3).
  • Comparison with Zeng et al. (o6Ku): Added direct comparisons with the original KMM method and alternative allocation rules, and clarified the no-K baseline and weighted allocation (Sec. 4; Apps. C.2, E.4).
  • Transferability (o6Ku): Added cross-family transfer experiments and corrected claims about transfer across post-training algorithms (Secs. 1, 6; App. E.2).

2. Novelty Concern

We appreciate all reviewers' concerns regarding the incremental novelty of extending single-target KMM to the multi-target setting. We would like to emphasize that our novel contributions go beyond this extension:

  1. A much larger scale benchmark and broader empirical study. Zeng et al.'s main benchmark selects among the language subsets of one multilingual dataset (26 languages of Aya), with additional experiments on seven instruction-tuning sources. We select among 96 distinct public datasets for eight targets, half of them deliberately unrelated to any target, and study each component and hyperparameter: the gradient feature, the Gram matrix, the allocation rule, the number of targets, target and auxiliary data size, pool size and preview size, across Llama and Qwen models from 0.6B to 70B, under both SFT and GRPO.

  2. Making SignGD actually effective. LESS uses SignGD to approximate Adam updates for scoring individual examples within an asymmetric influence formulation that retains raw target gradients. Notably, LESS reports that SignGD underperforms both SGD- and Adam-based influence in its setting (Table 9). In contrast, we apply SignGD symmetrically to whole auxiliary datasets and targets within a shared Gram matrix. Our results (Table 3a) demonstrate that SignGD consistently outperforms raw gradients in this dataset-level matching formulation, highlighting the importance of how signed-gradient representations are used.

  3. A new & better weight allocation rule. Zeng et al. split the budget equally over the selected datasets, and test a heuristic softmax weighting only as an ablation (their App. H), where it did not help. Our specific allocation rule improves target task performance significantly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.