acceptodds
Under review as a conference paper at ICLR 2027

A Helps B While B Hurts A: Directed Transfer in Instruction-Tuning Mixtures

Abstract

Adapting a language model to a specialized corpus means choosing which instruction-tuning tasks to train on under a fixed budget, and testing one choice costs a fine-tuning run. Common heuristics add more source tasks or pick sources similar to the target. The first assumes transfer is never negative; the second, that it is symmetric. We show that both assumptions fail: task can help task while hurts , so helpfulness is a signed property of ordered source–target pairs. We introduce the transfer map, a signed estimate of how much each source helps or hurts each held-out target. We fit the map in hundreds of fine-tuning runs on Qwen3 and Mistral models from 0.6B to 32B parameters, with all sources drawn from one corpus and no training examples from the target. The map predicts a held-out target's accuracy on unseen mixtures: recorded before those runs, its predictions have less than half the error of a mixture-agnostic baseline. The map is specific to its target and corpus but transfers across model scale: a mixture selected in advance at one size beats training on all source tasks at every other size we tested. Transfer is thus a property of the data. The map selects the tasks that help and drops the one that interferes: accuracy on the reasoning targets (causal explanation, multi-hop questions and methodological critique) rises by up to 14 percentage points over training on all source tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
instruction-tuning, data mixtures, data selection, task transfer, multi-task learning, domain adaptation, large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oczQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of selecting data mixtures for domain-adaptive instruction tuning under a fixed sample budget. The authors point out that two commonly used heuristics each rest on an untested assumption: the count heuristic (the more task types, the better) assumes that transfer is never negative, while the similarity heuristic (selecting the sources most similar to the target) assumes that transfer is symmetric. By generating all eight task types from a single corpus, withholding the target task during training, and probing with roughly 94 balanced mixtures, the authors fit an additive linear data model to obtain a signed and directed "transfer map."

Critical strengths
  1. The problem is important and clearly motivated. Choosing the task mixture for synthetic supervision data is a real pain point in domain adaptation, where every trial-and-error attempt costs a fine-tuning run.

  2. The core findings are crisp. Examples include the negative transfer of ERR -> MHOP, the differing directions of DEF -> MHOP versus MHOP -> DEF, and the fact that an interfering source can cancel the benefit of the other sources — results that offer direct takeaways for practitioners.

Critical weaknesses
  1. All scores come from an automated rule, and the gold answers are generated by GPT-5-mini. I recommend adding at least a small-sample human evaluation or stricter format controls, and summarizing in the main text how this affects the main conclusions.

  2. The cost to non-target tasks calls for a more practical discussion. The paper reports that the map-selected mixture scores 2.40 pp lower on average on the non-target macro. Real deployments often require balancing multiple objectives, whereas the current rule selects for a single target. The authors should provide a Pareto frontier, constrained selection, or trade-off results across multiple targets — or at least discuss when the selection rule should not be used.

  3. A mechanistic explanation is absent. Once surface-level explanations such as token volume and answer length are ruled out, there is no diagnostic analysis at all of "what makes a task type an interferer" (e.g., gradient conflicts or representation-level interference). Leaving this as an open question is understandable, but it keeps the method at the level of "measurement," and practitioners cannot tell in advance whether running a probe is worthwhile on a new corpus.

  4. The selector baselines could be stronger. The paper does not compare against gradient-influence methods such as LESS that "measure benefit rather than resemblance," and the comparison with the embedding-similarity selector is not cost-normalized — similarity selection costs zero training runs, whereas the transfer map requires dozens of fine-tuning runs. Showing at what probe budget the transfer-map route beats the free baseline would make the practical case more complete.

  5. The controlled-corpus setup does not yet separate task type, passage content, and dose. The passage sets behind different task types overlap at a Jaccard of only 0.51–0.53; narrowing the mixture increases the number of examples per retained source; and some pools require repeated sampling. The check that answer length does not correlate with the coefficients is no substitute for these two controls. The cross-scale and cross-family correlations are valuable evidence of stability, but all models share the same generation and scoring protocol, and the second corpus changes document granularity, pool size, and evaluation size all at once. This is insufficient to attribute the effects to task format itself, or to support the dichotomous conclusion that transfer is a property of the data rather than of the model.

Questions for authors
  1. Have you conducted any small-scale human evaluation to calibrate the scoring rule? Even a manual correctness judgment on 100 sampled generated answers from each of 2–3 predicted cells would substantially mitigate Weakness 1.

  2. Under dose-matched controls (with the number of examples per source held fixed and the budget growing with d), does the directional asymmetry between DEF→MHOP and MHOP→DEF still hold?

  3. Why did you choose unregularized OLS? Would shrinkage estimation of β (ridge or Bayesian) further reduce the variance of the 24-run probe and improve stability at small probe budgets?

  4. How sensitive is the selected source set to τ? Would a transfer map fitted at τ=0.80 or 0.90 select the same source subsets?

  5. To what extent is the cross-family agreement (r=0.88) driven by the shared generator and shared scoring rule? Does the cross-family consistency survive re-estimation with a different embedding model or with human scoring?

  6. Under what conditions does the transfer-map route outperform the alternative of "spending the probe budget directly on generating training data of the target's own task type" (the Bonito-style route)? This seems to be the most natural alternative for practitioners, yet no such comparison appears in the paper.

2 comments
AuthorsOpenReview ↗

Changes according to oczQ's review

Thank you for the thorough and insightful review. Most of the new experiments came from your questions.

W1 / Q1. Human evaluation.

We did not add a human evaluation; Section 8 lists it as a limitation and future work. Our effects are differences of a few points, each measured on about 2,000 answers per neuroscience target at three seeds. A sample of 100 hand-graded answers would not reliably resolve differences that small.

We therefore added an independent second metric, an LLM judge (Gemma 4 31B), and refitted the map to it. The main trends hold: source identity predicts accuracy better than mixture size, and maps agree across model sizes. Mixtures selected from the judge-scored map beat the uniform mixture in all six large-model cases (three seeds).

→ Sections 7.1 and 8; Appendix H, Tables 14 and 16.

W2. Cost to non-target tasks.

We agree that selecting for one target has trade-offs in deployment. Averaged over the seven non-target tasks, the judge-selected mixtures differ from the uniform mixture by −3.1 to +3.9 pp. The clear loss falls on critiques when they are not the target, since the uniform mixture trains on them directly. Section 7 now notes that balancing several targets requires constrained selection; we did not build a Pareto frontier.

→ Section 7; Appendix H, "Cost and sensitivity".

W3. Mechanism.

We found one concrete mechanism: a model trained with the target withheld copies the answer format of its sources. Without multi-hop questions among the sources, about half of the critique answers restate the question.

In a format control with one example answer in the prompt, the gap to the uniform mixture under the judge closes on Qwen3-32B. It does not close on Mistral-24B, which keeps another source's format. We did not run gradient-conflict or representation-level diagnostics. Predicting which task type will interfere on a new corpus remains open.

→ Appendix H, "Why the selected sources differ".

W4. LESS and probe cost.

We did not run LESS; it scores single examples, while the transfer map scores task types. Similarity selection requires no training runs; the map needs a 24-run probe shared by all targets (11 to 22 GPU-hours).

Over three seeds under the embedding rule, the rule-scored map stays ahead of the embedding-similarity selector in five of six cases and ties with TaskShop, which uses the same probe. Under the judge, the judge-scored map beats both in all six.

With half the probe (12 runs), the judge-scored 8B map still drops its interferer in nearly every random subset, though the rest of the mixture often changes. We did not train those mixtures, so the break-even probe budget remains open.

→ Section 2; Appendix H, "Other selectors" and "Cost and sensitivity".

W5 / Q2. Passages, dose-matched control, and data vs. model.

Under the embedding rule, the map's mixture still beats the uniform mixture at equal examples per source in all four tested cases.

The passage-matched probe uses only passages shared by all four open task types and repeats no example. Under the embedding rule, at 600 examples per source (so the budget grows with the number of sources), DEF→MHOP is +6.70 pp [5.70, 7.73] and MHOP→DEF is −0.67 pp [−1.34, +0.04]. The directions keep their signs, though the negative effect cannot be distinguished from zero.

Section 7 now states that "a property of the data" means stability across models under one shared training and evaluation protocol. It also notes that PubMed changes several factors at once.

→ Sections 7 and 8; Appendix I, Tables 18 and 19.

Q3 / Q4. Unregularized OLS, ridge, and τ.

We used unregularized OLS as the simplest choice, since the rule reads only the signs. Ridge (penalty 0.1) on the judge-scored maps raises out-of-sample R² at all four sizes and changes the selected source subset in half of the cases, but every selection still drops the interferer.

The selection is sensitive to τ: at 0.80 or 0.90 it changes in most cases.

→ Appendix H, "Cost and sensitivity".

Q5. Shared generator and shared scoring rule.

We cannot rule them out as a driver of r = 0.88 (see W5). We had not kept the large models' probe answers, so we could not re-score that correlation. We tested an LLM judge rather than a second embedding model or human scoring. Under the judge, maps at four Qwen3 sizes agree, and mixtures selected at 8B beat the uniform mixture on both Qwen3-32B and Mistral-24B, a different family.

→ Section 7.1; Appendix H, Table 16.

Q6. Direct target supervision (the Bonito-style route).

Direct target supervision works better for critiques: with 2,000 or more examples it beats every mixture that withholds critiques. For WHY and MHOP it stays near or below the untuned model. These are single-seed runs. Section 8 now clarifies that we withhold the target to measure transfer. When target data are available, direct supervision is an option.

→ Section 8; Appendix H, "Cost and sensitivity".

I thank the authors for the response. However, my main concerns aren't resolved. Therefore, I maintain my rating.

Reviewer_VnvNOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper asks the question: given a fixed budget of training examples, how should they be formatted to perform well on downstream tasks. This question is studied in a neuroscience paper question-answering setting and finds that including certain task formats during finetuning can have a negative effect on downstream performance. The work tries to formalize these effects by introducing “transfer maps”, which are the coefficient of an ordinary least squares regression models fit on leave-one-out training runs. They suggest fixing this interference by leaving out datasets that have negative coefficients. This leads to better results than using an embedding similarity threshold to choose the tasks to train on and comparably in some cases to training on the best task, or using methods derived from Skill-It and TaskShop.

Critical strengths

The problem set-up presented in this paper is potentially of interest to the broader community. Selecting what post-training data to use can be difficult, so methods that help would be useful to the community.

Critical weaknesses
  • Some methodological decisions weaken the contribution
    • As described in the related work section, the idea of trying to estimate the effect of data mixtures is not new and neither is the fact that there is negative interference between certain tasks. The related work seems to claim that the main contribution of this work is using a fixed training sample budget, but it is not clear why this is a relevant quantity to fix. One version of the “count hypothesis” is that training on additional tasks is helpful, not for the task format itself but because there are more training examples or views on the data. By fixing a training sample budget, the work does not test this aspect.
    • The experiments are performed on a newly introduced dataset which is automatically generated without any human validation. The techniques described here are not unique to neuroscience question-answering, so it would be more convincing to use existing datasets or provide human validation and release the dataset as an artifact for the community.
    • Finally, this is more minor, but the use of cosine similarity between sentence embeddings to compute answer correctness for open-ended tasks is likely sub-optimal. Just like the paper shows in Table 1 for dataset samples, different answers can have similar embeddings. It’s probably better to evaluate these using LLM-as-a-judge.
  • Lack of clarity. Most of the submission after the the introduction is written in a way that makes reading quite difficult, weakening the contribution. Some specific notes are in the “Questions for Authors” section.
Questions for authors
  • Why is Lemma 1 formatted as a lemma? This does not fit stylistically with the rest of the work as there is no later proof that this lemma is designed to support.
  • Math on line 229 is buggy (I think the “does-not-force” symbol ⊮ should be an indicator function) and potentially overly-complex (why not sum over M directly?)
  • In Table 1, why are differences between the selected mixture minus competitor reported? Readers likely want to see the raw numbers with best performances bolded.
2 comments
AuthorsOpenReview ↗

Changes according to VnvN's review

Thank you for reading the submission carefully and for the candid feedback. Your concerns on the fixed budget, the scoring metric and the presentation led to a new experiment, a second metric and targeted clarity fixes in the revision.

W1. What is new, and why fix the training sample budget?

You are right that estimating mixture effects is not new, and neither is negative interference. The introduction now states our contribution directly. We measure transfer with all task types from one corpus, a fixed training sample budget and the target withheld from training. Proposition 1 shows that evidence for "more tasks help" can only show that the average task helps, so it cannot justify adding every task. In both prediction tests, we wrote down the predictions and selected mixtures before the runs.

We also see this as a first step toward bringing curriculum learning from reinforcement learning (RL) into supervised fine-tuning (SFT). A curriculum chooses which task types to train on and in what order for a target capability. Ours is short and not learned, but we measure how task selection and order matter.

A fixed training sample budget is the practitioner's situation: with limited compute, every added task takes examples from the others. Your alternative version of the count hypothesis—that more tasks help because they bring more training examples—is a fair one, and we now test it. We added the dropped sources back on top of the selected mixture, so the sample budget grows and nothing is displaced. Accuracy on the target falls in all four tested cases (under the embedding rule). The extra tasks hurt even when they come with extra data.

→ Sections 1 and 4.2; Appendix E; Appendix I, Table 18.

W2. A generated dataset without human validation.

The generated dataset and the splits of both corpora are released in the supplement (data/corpus/). Using a different dataset for each task type would change task type and content together. Generating all task types from one corpus reduces that confound.

We did not hand-check the generated items. Section 8 names the generated references as a limitation. Every comparison scores all mixtures on the same items, and the main trends hold under two independent metrics (W3).

→ Sections 2 and 8; supplement, data/corpus/.

W3. LLM-as-a-judge.

We re-scored the open-ended answers with an LLM judge (Gemma 4 31B) and refitted the map to its scores. The main trends hold, although the exact numbers and selected sources change. Source identity predicts accuracy better than mixture size, and maps agree across model sizes. Mixtures selected from the judge-scored map beat the uniform mixture in all six large-model cases. The selected sources change because a model trained with the target withheld copies the answer format of its sources, and the two metrics reward different formats.

→ Section 7.1; Appendix H.

W4 and Q1–Q3. Clarity.

We addressed each point you raised and added a table of terms, including the task abbreviations.

For Q1, Lemma 1 is now Proposition 1 because it is a stand-alone result.

For Q2, Eq. 2 (formerly line 229) now sums over S ∈ M directly, removing the broken indicator symbol.

For Q3, Table 1 now shows raw accuracies, with results within 0.5 pp of the column's best in bold, the map's gain over the uniform mixture, and each selector's mean and worst gap to the best.

→ Eq. 2; Proposition 1 (proof in Appendix B); Table 1; Appendix J.

Thank you for your response. I've updated my score accordingly.

Reviewer_s1c5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates task transfer in instruction-tuning mixtures under a fixed training budget. The authors show that transfer between task types can be asymmetric and negative, challenging the assumptions that including more tasks or selecting similar tasks always improves performance. They propose a Transfer Map that uses linear regression over fine-tuning experiments to estimate directed source-to-target effects and select training mixtures. Extensive experiments across Qwen3 and Mistral models demonstrate that the map can predict the effects of unseen mixtures, improve target-task performance, and exhibit transfer consistency across model scales.

Critical strengths
  • Meaningful research problem. The paper studies how source-task selection affects a held-out target under a fixed training budget. Measuring directed transfer among tasks generated from a shared corpus provides a useful setting for examining when adding tasks or selecting similar ones can fail.

  • Experiments span multiple model sizes and two model families. Particularly valuable is the prospective evaluation, with predictions recorded before the corresponding fine-tuning runs. The cross-scale experiments and explicit reporting of unsuccessful predictions strengthen the empirical contribution.

Critical weaknesses
  1. Lack of robust validation for evaluation metrics:
    The main reasoning results rely on embedding similarity between model-generated reference answers and model predictions, without independent validation of answer correctness. Although the authors provide sensitivity analyses, changes in scoring rules and response formats affect several transfer estimates. This raises concerns about whether the reported performance gains genuinely reflect improvements in reasoning quality.

  2. Unclear incremental value compared to simpler task-selection methods:
    In various settings, the TaskShop-style approach to task selection yields comparable performance, with observed differences based on only single-run comparisons. Furthermore, in some cases, removing a single source of interference recovers most of the performance gains. Without directly comparing against a simpler strategy that removes one source task at a time, the additional value of the Transfer Map remains insufficiently demonstrated.

Questions for authors
  • Is any human evaluation or independent semantic correctness analysis available to verify that the main transfer effects reflect genuine improvements in answer quality rather than scoring or response-format differences?

  • How does the Transfer Map compare with a simpler approach that removes one source task at a time? Could the authors also compare its performance with TaskShop across multiple random seeds?

  • Do the observed transfer effects still hold when different tasks use the same source passages or when the number of training examples per task is controlled?

2 comments
AuthorsOpenReview ↗

Changes according to s1c5's review

Thank you for the thoughtful and constructive review. Each of your three questions led to new experiments, which are now in the paper.

Q1 / W1. Do the gains reflect answer quality rather than scoring rules or response format?

We added an independent correctness check: an LLM judge (Gemma 4 31B) re-scored the answers, and we refitted the map to its scores. The main trends hold: source identity predicts accuracy better than mixture size, and maps agree across model sizes. Mixtures selected from the judge-scored map beat the uniform mixture in all six large-model cases, over three seeds.

We did not add a human study. The effects are differences of a few points, each measured on about 2,000 answers per neuroscience target at three seeds. A small expert-graded sample would not reliably resolve them. Section 8 lists this as a limitation and future work.

You were right about response format. A model trained with the target withheld copies the answer format of its sources. Our original critique mixtures leave out multi-hop questions, the only other open task with yes/no answers, so about half of their critique answers restate the question. The embedding rule credits many of these and the judge does not, so under the judge those mixtures fall below the uniform mixture. This is also why the selected sources change with the scoring rule.

→ Sections 7.1 and 8; Appendix H, Tables 14–16.

Q2 / W2. Removing one source task at a time, and TaskShop across multiple seeds.

Both are done, on Qwen3-32B and Mistral-24B over three seeds.

Under the embedding rule, the TaskShop-style ranker ties with the rule-scored map: no difference can be distinguished from zero in any of the six cases. It uses the same probe and keeps as many sources as the map, so the map still decides how many to keep. The map stays ahead of the embedding-similarity selector in five of six cases. Under the judge, the judge-scored map beats both in all six.

Removing only the interfering source comes within 0.5 pp of the rule-scored map's mixture in three of the four MHOP and ERR cases under the embedding rule. This supports our main point: one source of interference cancels most of what the other sources add. The map identifies that source from one probe shared by all targets; finding it by removal takes seven mixtures per target (for the seven candidate sources). Under the judge, the judge-scored map's mixture is ahead of the best of the seven single removals in all six cases.

→ Appendix H, "Other selectors" and Table 17.

Q3. Same source passages, and controlling the number of training examples per task.

Under the embedding rule, the map's mixture still beats the uniform mixture at a matched number of examples per task in all four tested cases (three seeds). Adding the dropped sources back as extra data lowers accuracy in all four.

We rebuilt the training pools from the source passages that carry all four open task types. At a fixed budget under the embedding rule, the predicted directions keep their signs; one is too small to tell from zero.

→ Section 8; Appendix I, Tables 18 and 19.

Thank you for the response. The embedding metric crediting answers that merely restate the question confirms my evaluation concerns. I appreciate the authors' efforts and will maintain my positive rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Combined improvements based on reviews

We thank all three reviewers for their careful and constructive reviews. Your comments led to a second metric, two new controls, multi-seed baselines and a clearer presentation. In the PDF, new text is red and removed text is struck out in blue; the new material is in Section 7.1, Section 8 and Appendices H–J.

We view this work as a first step toward bringing curriculum learning from reinforcement learning (RL) into supervised fine-tuning (SFT). A curriculum chooses which task types to train on and in what order to build a target capability. Ours is short and not learned, but we quantify why these choices matter.

1. A second metric (all reviewers).

Additionally, we examine the evaluation metric in Section 7.1. The main trends hold, although the exact numbers and selected sources change. We re-scored the answers with an LLM judge (Gemma 4 31B, from a different family than our tested models and data generator) and refitted the map to its scores. Source identity predicts accuracy better than mixture size, and maps agree across model sizes. Mixtures selected from the judge-scored map beat the uniform mixture (training on all seven candidate sources) in all six large-model cases. Under the judge, the uniform mixture often ends below the untuned model, and the map's mixtures avoid that loss.

The selected sources change because a model trained with the target withheld copies the answer format of its sources, and the two metrics reward different formats.

→ Section 7.1; Appendix H.

2. Dose and passage controls (all reviewers).

Under the embedding rule, the map's mixture still beats the uniform mixture at a matched number of examples per source, and adding the dropped sources back as extra data lowers accuracy.

With all tasks built from the same source passages, the predicted directions keep their signs at a fixed budget; one is too small to tell from zero.

→ Section 8; Appendix I.

3. Simpler selectors (s1c5, oczQ).

Under the embedding rule over three seeds, the TaskShop-style ranker on the same probe ties with the map, and the map stays ahead of the embedding-similarity selector in five of six cases. Under the judge, the judge-scored map beats both in all six.

Removing only the interfering source comes close to the rule-scored map's mixture. Both results support our main point: the gain comes from measuring which source hurts the target.

→ Appendix H, "Other selectors".

4. Presentation (VnvN).

We simplified Eq. 2 and renamed Lemma 1 as Proposition 1.

Table 1 now shows raw accuracies, with results within 0.5 pp of the column's best in bold. We also added a table of terms.

→ Eq. 2; Proposition 1; Table 1; Appendix J.

5. Human evaluation (all reviewers).

Our effects are differences of a few points between mixtures, each measured on about 2,000 answers per neuroscience target at three seeds. A small expert-graded sample would not reliably resolve differences that small. We therefore tested the findings under two independent metrics, and the main trends hold under both (point 1). A human study remains a limitation and future work.

→ Section 8.

The supplement has a clean copy of the paper, the code, the results behind every table and the dataset.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.