acceptodds
Under review as a conference paper at ICLR 2027

Let the Target Select for Itself: Data Selection via Target-Aligned Paths

Abstract

Targeted data selection seeks training examples from a candidate pool that improve downstream task performance. While trajectory-based selectors effectively guide this process, existing approaches typically construct reference states by warming up on the candidate pool itself, thereby inheriting pool-dependent distributional biases. To address this coupling, we introduce Target-Aligned Candidate Selection (TACS), which constructs a short, capacity-constrained reference path using a compact target-validation proxy. Candidates are ranked by their normalized loss reduction along this trajectory, requiring only forward passes and allowing the path to be reused across distinct pools for a fixed model-target pair. Empirically, TACS achieves competitive downstream performance across controlled logistic, vision, and NLP benchmarks, while slashing selection compute by up to 69% and candidate storage to less than 10 MB in large-scale instruction tuning.

Primary area:
optimization
Keywords:
targeted data selection, instruction tuning, data attribution, training dynamics, influence functions, LLM fine-tuning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fTmSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TACS, a target-aware data selection method that constructs a short reference trajectory using only a compact target-validation proxy. Candidates are ranked by normalized loss reduction along this trajectory, enabling forward-only scoring and reuse of the same reference path across multiple candidate pools. The method is evaluated on synthetic, vision, and instruction-tuning benchmarks.

Critical strengths
  1. The paper identifies reference-path construction as an important but underexplored factor in trajectory-based data selection and proposes a simple target-aligned alternative.

  2. TACS provides substantial compute and storage savings while remaining competitive with strong trajectory-based baselines.

Critical weaknesses
  1. The large-scale instruction-tuning gains are uneven. Averaged across pools, TACS improves TyDiQA substantially, but does not outperform the untrained base model on MMLU or BBH, making the claimed cross-task benefit less convincing.

  2. TACS critically depends on the target proxy, but robustness to different proxy samples is not evaluated. This is especially concerning for TyDiQA, where the proxy contains only nine examples.

Questions for authors
  1. How sensitive are the selected subsets and downstream results to different samples of the target proxy?

  2. Why does TACS fail to improve over the base model on average for MMLU and BBH?

  3. Do the reported efficiency gains remain similar under compute-matched lower-cost configurations of LESS and ToV?

5 comments

Thank you for the careful review and the precise questions. They led us to test how TACS depends on the proxy sample, why gains over the base model are small on MMLU and BBH, and how its cost compares with cheaper configurations of the baselines. We have added the new experiments below to the revised manuscript in blue.

W2 / Q1: "TACS critically depends on the target proxy, but robustness to different proxy samples is not evaluated. This is especially concerning for TyDiQA, where the proxy contains only nine examples." "How sensitive are the selected subsets and downstream results to different samples of the target proxy?"

Response: We agree that this question matters, because TACS builds its reference path from the proxy alone, and nine examples is a small sample. We add a proxy-sampling study on Dolly → TyDiQA. We draw five new nine-example proxies, one training question per language, with no overlap with the original proxy or the evaluation set. We also build proxies that cover only three languages or one language. For each proxy, we rebuild the reference, rescore the full Dolly pool, select the top 5%, and retrain, with everything else fixed. Test scores use the fine-tuning checkpoint with the highest validation score ("validation-selected") and the last checkpoint ("final"). Across the five new proxies, F1 varies by about one point and stays more than 11 points above Random. The new results appear in Appendix J.4, Table 13.

Proxy, Dolly → TyDiQAF1, validation-selected / final
Original proxy63.35 / 63.63
Five new proxies, mean ± s.d.62.66 ± 0.89 / 63.61 ± 0.70
Proxies covering 3 languages, mean of three60.45 / 62.79
Proxies covering 1 language, mean of three60.95 / 63.28
LESS61.50 / 62.44
Random49.63 / 49.73

The selected subsets have a Jaccard index of 0.38 to 0.54 with the original selection, so 55 to 70% of the selected examples are shared, and different proxies select partly different data of similar value. Proxies that cover part of the target lose at most 3 points, and a language does not score consistently higher when the proxy covers it. On Dolly → TyDiQA, downstream quality therefore stays stable across proxy samples, and even a narrow proxy keeps most of the gain.

W1 / Q2: "Averaged across pools, TACS improves TyDiQA substantially, but does not outperform the untrained base model on MMLU or BBH." "Why does TACS fail to improve over the base model on average for MMLU and BBH?"

Response: This is a fair concern: the strong TyDiQA gains do not by themselves establish improvement across targets. We first compare every selector and full-pool training with the base model, averaged over the four pools of Table 1. None of the methods improves over the base model on average on MMLU or BBH, so the small gains are shared across methods.

Average over four poolsBaseRandomLESSToVTACSFull pool
MMLU accuracy56.2255.7255.6954.9356.0854.42
BBH accuracy48.6147.4047.5547.8647.8547.34

To understand why, we analyzed the base model and 48 trained runs per task, and asked whether these tasks offer the same answer-behavior gains as TyDiQA. On TyDiQA, about 84% of base-model F1 loss is associated with answers that are not passage spans or contain extra text. BBH already provides three demonstrations, and only 10.6% of base errors lack the expected answer marker; training leaves that count almost unchanged (59 versus 58.7). MMLU scores option likelihoods after five demonstrations, so changing generated-answer format cannot improve its metric. On BBH, training corrects 96 base errors but breaks 107 previously correct answers on average, leaving little net gain. At our scale—5% subsets and four epochs of LoRA training—these results suggest that general instruction data more readily improves answer behavior than supplies missing knowledge or symbolic skills. This analysis appears in Appendix J.5, Table 14.

To separate content from answer format, we add a control: training on 1,493 held-out MMLU questions, and on 1,493 puzzles that use the same MMLU answer format. Format-only training gives little change, while matched-content training raises Llama-2-7B accuracy by 4.75 points. The new control appears in Appendix J.5, Table 15.

MMLU accuracyLlama-3.2-3BLlama-2-7B
Base model56.2445.53
Format-only training55.9245.43
TACS on a generic pool55.8245.20
Matched-content training56.9950.28

This supports the distinction between improving answer behavior and supplying task-relevant content. We acknowledge that TACS trails LESS on COT → BBH (46.11 versus 48.12). Across our experiments, we observe a general pattern: TACS is less stable on targets with little headroom for general instruction data and more competitive where that headroom is larger. We have added this finding to the paper's limitations, in the conclusion and in Appendix B, in blue. The paper presents TACS as an efficient selector with competitive downstream performance.

Q3: "Do the reported efficiency gains remain similar under compute-matched lower-cost configurations of LESS and ToV?"

Response: This is an important question, because our reported savings are measured against the standard configurations of LESS and ToV, and a cheaper configuration could narrow them. We add two measurements for a lower-cost LESS configuration that keeps the method intact and uses one warmup checkpoint. First, we measure its selection quality on Dolly → TyDiQA. Second, we time both methods in one job per pool, on the same GPU and the same 3,000 sampled candidates. The cheaper LESS keeps a quality comparable to standard LESS, and TACS remains cheaper per candidate up to about three targets per pool. The new results appear in Appendix J.6, Table 16.

Dolly → TyDiQAF1, validation-selected / final
LESS, 4 checkpoints61.50 / 62.44
LESS, 1 checkpoint61.28 / 64.28
TACS63.35 / 63.63
Compute per candidate (GPU-seconds)DollyCOTFlan v2OASST1
TACS, per target (2 forward passes)0.0390.0420.0560.046
LESS, per checkpoint (1 per-example gradient)0.1710.1590.1870.170
LESS, standard (4 checkpoints)0.680.630.750.68
Adding one 100k-row poolTraining on the poolCandidate compute (GPU-hours)Stored per candidate
TACS, 1 targetnone1.31 score
TACS, 3 targetsnone3.83 scores
LESS, 1 checkpointwarmup on 5% of the pool4.88,192 values
LESS, 4 checkpointswarmup on 5% of the pool19.132,768 values

The advantage comes from how TACS scales with the pool. TACS never trains on the candidate pool: its reference path is built once per target from the proxy and reused for every pool, so a new pool costs two forward passes per candidate and target. LESS must warm up on each new pool and compute a per-example gradient for every candidate at every checkpoint. At three targets, TACS needs 4.5 to 5.9 times less compute per candidate than standard LESS, and 1.1 to 1.5 times less than one-checkpoint LESS, before the LESS warmup. The gap closes as more targets share one pool, because LESS reuses its stored gradients for every target; our limitations section (Appendix B) states this trade-off. For ToV, per-candidate scoring is 120.5 of its 124.7 GPU-hours in our benchmark, at about 2.4 times the TACS cost per candidate and target. Because almost all of its cost is per-candidate scoring, a cheaper warmup would barely change it; only scoring at fewer checkpoints would, and we did not test that configuration. We conclude that, against a cheaper LESS, TACS keeps a lower marginal scoring cost when new pools are screened for a few targets, together with its storage advantage under every configuration.

We thank the reviewer for their thoughtful questions and hope these additions address their concerns. We would be happy to clarify any remaining points and would appreciate their reconsideration in light of the new evidence.

Reviewer_CpC3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TACS, which builds a short, low-capacity reference trajectory from a target proxy and tunes its learning rate and duration by cross-validation. Candidates are ranked by normalized loss reduction between two checkpoints, enabling forward-only scoring and reuse across pools. The theory explains reference sensitivity under smooth population selection and shared-covariance quadratic losses, without finite top-N or retraining guarantees.

Critical strengths
  1. Controlled warmup experiments show substantial changes in rankings and downstream F1, making reference construction a useful design variable beyond TACS.

  2. The compute savings are credible in the measured multi-pool setting: 58.46 selection GPU-hours versus 187.35 for LESS and 124.74 for ToV, including calibration and reference construction. The 6.5 MB figure refers to candidate scoring payload; many-target settings have a different amortization regime.

  3. TyDiQA provides the strongest utility evidence: gains over LESS of 2.12 F1 on Dolly and 3.34 on Oasst1, with paired evaluation-item intervals excluding zero. Fixed-final retraining supports the score-band relationship, reducing reliance on checkpoint selection. Noise experiments and capacity/depth ablations provide further support.

Critical weaknesses
  1. Target-only references are not sufficiently justified. Results are broadly comparable to LESS and worse on COT-BBH (46.11 versus 48.12). Reference sensitivity alone does not show that excluding candidate data helps; a small target proxy may miss useful information about candidate distributions.

  2. With only three targets, I remain unconvinced that target-only construction generally improves the balance between target alignment and distributional coverage. Broader tasks, like TAROT (https://arxiv.org/abs/2412.00420), and reference comparisons matched for capacity and scoring are needed.

  3. Downstream utility is tested only on Llama-3.2-3B. Reference-sensitivity analysis on a second model does not establish selection utility on larger models or other model families.

Questions for authors
  1. With reference capacity, scoring rule, training budget, subset size, and final retraining fixed, how do target-only, diverse-pool, and mixed references compare?
  2. Does the motivation hold across more target tasks and proxy coverage levels?
6 comments

Thank you for carefully reviewing our paper and for your constructive comments. Your questions helped us strengthen the evidence for reference construction and clarify where TACS works well. We have added the new experiments below to the revised manuscript in blue.

W1 / Q1: Target-only references may miss useful information about candidate distributions. With reference capacity, scoring rule, training budget, subset size, and final retraining fixed, how do target-only, diverse-pool, and mixed references compare?

Response: Thanks for raising a good question. A target-only reference can still select useful pool data, but whether mixing pool data into the reference helps requires a direct comparison. We add a matched experiment comparing target-only, 50/50 mixed, and pool-only references with the same capacity, scoring rule, update budget, subset size, and retraining within each setting. We find that target-only is the best of the three references in all four settings, exceeding mixed references by about 3–6 score points. Test scores use the selected-subset fine-tuning checkpoint with the highest validation score (“validation-selected”). We use the standard benchmark metrics, F1 for TyDiQA-GoldP [1] and MC2 for TruthfulQA [2]. The new comparison appears in Appendix J.1, Table 10.

SettingTarget-onlyMixedPool-onlyRandom
Llama-3.2-3B, Dolly → TyDiQA63.2157.1554.7550.60
Llama-2-7B, Dolly → TyDiQA53.3449.6649.8048.80
Llama-2-7B, OASST1 → TruthfulQA50.2747.0243.8147.97
Gemma-2-2B, OASST1 → TruthfulQA45.4441.8742.8445.44

We extend the same experiment to more mixing ratios, where 0 means target-only and 1 means pool-only, holding the total update budget fixed within each setting. We run it on TyDiQA and on TruthfulQA. Final-checkpoint scores are averaged over independently constructed reference draws. On Llama, target-only gives the highest TyDiQA mean on both pools. On TruthfulQA, small pool shares tie target-only, and references with half or more pool updates fall below Random (49.93). Gemma is less sensitive to the ratio, with larger variation across draws. These new results appear in Appendix J.1, Figure 17.

Model, pool → target01/161/81/41/23/41
Llama-3.2-3B, Dolly → TyDiQA64.3763.2062.9661.8060.5359.7260.17
Llama-3.2-3B, COT → TyDiQA59.6258.6458.4057.3256.8657.0356.46
Gemma-2-2B, Dolly → TyDiQA65.1166.5363.6062.8365.8166.0061.50
Llama-3.2-3B, OASST1 → TruthfulQA50.6250.6750.5250.0949.1447.6246.79

We also add a control applying LESS-style gradient scoring and TACS scoring to the same reference paths at each ratio, reporting final-checkpoint TyDiQA F1. LESS shows a much weaker drop on Dolly: from pool share 0 to 1, it falls from 63.83 to 62.64 (1.19 F1), while TACS falls from 63.63 to 58.65 (about 5 F1). On COT → TyDiQA, the corresponding drops are 1.08 and 2.06 F1. Intermediate ratios fluctuate, and the best Dolly LESS-style result uses a mixed reference. Under the same scoring rule, the target-only path therefore also helps gradient matching (63.83 versus 62.64 with a pool-only reference), and TACS reaches the same quality with two forward passes per candidate. The new control appears in Appendix J.2, Figure 18.

Pool, scoring rule00.100.250.500.751
Dolly, LESS-style63.8363.9663.8964.4763.0562.64
Dolly, TACS63.6363.2360.8459.3260.0358.65
COT 10%, LESS-style58.6859.2357.9456.8158.5057.60
COT 10%, TACS58.6758.4157.8556.5056.8256.61

(Continued in the next comment.)

[1] Clark et al. TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages. TACL, 2020.

[2] Lin et al. TruthfulQA: Measuring How Models Mimic Human Falsehoods. ACL, 2022.

W1 / Q1 (continued)

For COT → BBH, we add a LESS-scoring mixture control using an 81-example reference and selecting 500 examples from a fixed 10,000-example COT subsample. The validation-selected accuracy curves are nearly flat under both Adam and SGD candidate gradients, contrasting with the weaker decline on Dolly → TyDiQA. These new results appear in Appendix J.2, Figure 18.

Candidate gradients00.200.400.600.801
Adam (LESS)47.4146.9447.5947.2247.0446.76
SGD46.2046.9447.0446.1146.8546.67

We acknowledge that TACS trails LESS on COT → BBH (46.11 versus 48.12). Our analysis of the base model and 48 trained runs per task gives evidence consistent with the small overall gains above base on MMLU and BBH. On TyDiQA, about 84% of base-model F1 loss is associated with answers that are not passage spans or contain extra text. BBH already provides three demonstrations, and only 10.6% of base errors lack the expected answer marker; training leaves that count almost unchanged (59 versus 58.7). MMLU scores option likelihoods after five demonstrations, so changing generated-answer format cannot improve its metric. This analysis appears in Appendix J.5, Table 14.

At our scale—5% subsets and four epochs of LoRA training—these results suggest that general instruction data more readily improves answer behavior than supplies missing knowledge or symbolic skills. On BBH, training corrects 96 base errors but breaks 107 previously correct answers on average, leaving little net gain. A new MMLU control is consistent with this distinction: format-only training leaves Llama-2-7B at 45.43 versus 45.53 for base, while training on matched MMLU content reaches 50.28 (Appendix J.5, Table 15, added in blue).

Across our experiments, we observe a general pattern: TACS is less stable on targets with little headroom for general instruction data and more competitive where that headroom is larger. We have added this finding to the paper’s limitations, in the conclusion and in Appendix B, in blue. The paper presents TACS as an efficient selector with competitive downstream performance.

W2: With only three targets, the balance between target alignment and distributional coverage is not sufficiently established. Broader tasks and reference comparisons matched for capacity and scoring are needed.

Response: This is a fair concern: three targets provide limited evidence for how broadly the method works. We add a five-target comparison, including GSM8K and TruthfulQA, on three models using the same 15,000-row COT pool and a 10% selection budget. In the nine comparisons below, TACS exceeds Random in eight and has the highest selector score in five. The gains vary by model: Gemma GSM8K favors Random, and LESS remains stronger on TyDiQA for both Llama models. Scores use validation-selected checkpoints; TyDiQA uses F1, GSM8K exact-match accuracy, and TruthfulQA MC2. The full five-target results are added in blue in Appendix J.3, Table 11.

ModelTargetBaseRandomLESSToVTACS
Llama-3.2-3BTyDiQA42.0054.9261.0756.3256.63
GSM8K31.0129.7230.7827.8232.52
TruthfulQA45.5044.3244.9845.5346.47
Llama-2-7BTyDiQA39.0943.9050.8342.0949.84
GSM8K13.1216.0717.2113.6520.62
TruthfulQA38.8237.1638.9637.5738.51
Gemma-2-2BTyDiQA47.4061.1160.9558.0563.14
GSM8K25.9334.3433.1330.1030.48
TruthfulQA36.6138.0837.7737.9038.42

W3: Downstream utility is tested only on Llama-3.2-3B. Reference-sensitivity analysis on a second model does not establish selection utility on larger models or other families.

Response: We agree that reference sensitivity alone does not establish downstream utility on other models. We add the full pipeline—model-specific calibration, selection, retraining, and evaluation—on Llama-2-7B and Gemma-2-2B. TACS improves over Random by 4.5 and 6.2 F1 on TyDiQA, while TruthfulQA scores are close. These results extend selection utility to a larger model and another family, with the strongest gains on TyDiQA. Scores use validation-selected checkpoints; TyDiQA uses F1 and TruthfulQA MC2. The new results appear in Appendix J.3, Table 12, in blue.

ModelSettingBaseRandomTACS
Llama-2-7BDolly → TyDiQA39.0948.8053.34
Gemma-2-2BDolly → TyDiQA47.4059.3865.59
Llama-2-7BOASST1 → TruthfulQA38.8249.2849.71
Gemma-2-2BOASST1 → TruthfulQA36.6145.7645.19

Q2: Does the motivation hold across more target tasks and proxy coverage levels?

Response: This is an important question because a compact proxy may omit parts of the target distribution. We add a coverage experiment that keeps the TyDiQA proxy at nine examples but varies whether it covers nine, three, or one language, with the remaining settings fixed. Narrow proxies retain a large gain over Random: validation-selected F1 drops by about 2 points, while final-checkpoint F1 stays within 1 point of full coverage. We find no consistent average advantage for languages included in the proxy. This supports robustness to reduced language coverage on the mostly English Dolly pool. The new coverage results appear in Appendix J.4, Table 13, in blue.

ProxyValidation-selectedFinal
All 9 languages62.6663.61
3 languages60.4562.79
1 language60.9563.28
Random selection49.6349.73

We hope these results address the concerns about reference construction and evaluation breadth. We would be glad to clarify any remaining concern. In light of these additional results and clarifications, we would appreciate the reviewer revisiting their assessment.

Reviewer_Kb1mOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduced Target-Aligned Candidate Selection (TACS), a target-aware data selection method that constructs a reference trajectory by fine-tuning a model on a small target-validation set. TACS ranks candidate examples by their proposed normalized loss reduction between two checkpoints along this trajectory. The authors also introduce low-capacity adaptation and trajectory-length calibration to improve selection. Experiments on several tasks show competitive downstream performance with reduced selection compute and storage costs compared to existing methods.

Critical strengths

The paper is well written and easy to follow. TACS outperforms several baseline methods on various tasks.

Critical weaknesses

The proposed data selection criterion is quite similar to the criterion proposed in CoLoR-Filter [1]. However, the authors did not compare their method with CoLoR-Filter in the experimental section. Moreover, the incremental contribution of this work should lie in the difference between TACS and CoLoR-Filter. However, the authors present the paper as if the incremental contribution is the difference between LESS [2] and TACS.

[1] CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training, NeurIPS, 2024.

[2] LESS: Selecting Influential Data for Targeted Instruction Tuning, NeurIPS, 2024.

Questions for authors

My questions/ concerns are shown as follows:

  1. Please compare TACS with CoLoR-Filter. Restate TACS's contributions.

  2. Please run further experiments and directly compare TACS's performance with CoLoR-Filter.

  3. As shown in Appendix D, Figure 8, the normalized loss (TACS) and Raw loss (CoLoR-Filter style) have similar performance.

I am happy to change my score if my concerns are addressed.

5 comments

We thank the reviewer for the constructive comments and for noting that the paper is "well written and easy to follow." The main concern is the relation between TACS and CoLoR-Filter. Our contribution relative to CoLoR-Filter is stated in the Related Work section: the "TACS score shares this loss-drop form," while "when candidate pools differ from the target and supervision is limited to a compact proxy, TACS controls reference capacity and calibrates trajectory depth to the target task." We're happy to elaborate on this comparison below and add the requested experiment.

Q1. "Please compare TACS with CoLoR-Filter. Restate TACS's contributions."

Response:

Novelty beyond loss-difference scoring. The loss difference between a base model and a fitted model is a well-known scoring rule in data selection. CoLoR-Filter [1] builds on this principle, previously used in RHO-LOSS [2], and contributes in a different data-selection setting. Similarly, we do not claim the loss-difference form as novelty. Our main contribution is how we construct the reference trajectory under limited target supervision: we control model capacity and select a task-specific training depth.

Different data regime. CoLoR-Filter studies targeted pretraining with millions of target tokens (7.4M–25M). TACS operates in supervised fine-tuning with much more limited target supervision (as few as nine examples). In this regime, the reference is particularly sensitive to its training duration and capacity, so controlling both is an important part of reference construction.

Reference construction. CoLoR-Filter constructs its conditional reference with one epoch of fine-tuning on target data. TACS instead controls reference capacity through low-rank adaptation and calibrates trajectory depth to the target task. Figure 6 (middle) shows that the optimal trajectory depth varies substantially across tasks. A fixed one-epoch reference is therefore not justified in general across target tasks.

The requested experiment (Q2) and Q3 follow in the next comment.

[1] Brandfonbrener et al. CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training. NeurIPS, 2024. [2] Mindermann et al. Prioritized Training on Points that are Learnable, Worth Learning, and not yet Learnt. ICML, 2022.

Q2. "Please run further experiments and directly compare TACS's performance with CoLoR-Filter."

Response: Our main comparisons with LESS and ToV focus on strong, reproducible baselines developed for the same targeted instruction-tuning setting, which allows evaluation under a consistent protocol. Transferring CoLoR-Filter directly to our setting requires additional choices in reference construction. Its one-epoch adaptation was evaluated with millions of target tokens, while our depth ablations show that reference-training duration is critical and target-dependent under scarce supervision.

Following the reviewer's request, we manually adapt CoLoR-Filter to our setting. The reference is a single epoch of fine-tuning on the target validation proxy, followed by the raw loss-difference score. The budget (5%), retraining, and evaluation follow the Table 1 protocol. We note that this is our adaptation, not an authentic CoLoR-Filter reproduction: CoLoR-Filter does not specify a reference for this regime, so we decide the reference construction ourselves (full-parameter or LoRA rank 128, and the reference learning rate).

ModelCoLoR-Filter referenceCoLoR-FilterTACSTACS − CoLoRRandomBase
Llama-3.2-3B, seed 3full-parameter (3.21B), LR = retraining LR62.2763.35+1.0849.6342.00
Gemma-2-2B, 3 seedsfull-parameter (2.61B), LR = retraining LR38.8865.59+26.7159.3847.40
Gemma-2-2B, 3 seedsfull-parameter (2.61B), LR lowered to 1e-554.5065.59+11.0959.3847.40
Llama-3.2-3B, 3 seedsLoRA rank 128, LR = retraining LR62.8163.21+0.4050.6042.00
Llama-2-7B, 3 seedsLoRA rank 128, LR = retraining LR51.8853.34+1.4648.8039.09

TyDiQA F1 after retraining on the Dolly selection, at the checkpoint with the best validation F1. By default the reference uses the retraining learning rate of the cell. On Gemma-2-2B this rate (5e-5) is too high for full-parameter training, so we also report a reference at 1e-5; retraining is unchanged.

TACS outperforms the adapted CoLoR-Filter in all five settings. The gap is small on Llama-3.2-3B (+0.40 to +1.08) and large on Gemma-2-2B. On Gemma-2-2B, the adapted CoLoR-Filter is highly sensitive to the reference learning rate: 38.88 at the retraining rate, below the base model (47.40), against 54.50 at 1e-5. Even at 1e-5, it stays 11.09 F1 below TACS and below Random. This sensitivity to reference construction under scarce supervision is the problem that TACS addresses.

Q3. "As shown in Appendix D, Figure 8, the normalized loss (TACS) and Raw loss (CoLoR-Filter style) have similar performance."

Response: Figure 8 is intended to show that the optimal trajectory depth remains target-dependent and non-monotonic when the rank-1 restriction is lifted to the rank-128 LoRA used in retraining, under both raw and normalized loss-difference scoring. The similar performance of the two scores is therefore consistent with our claim: the importance of trajectory calibration is independent of normalization. Figure 13 shows that normalization mainly affects the preference for shorter examples. These experiments are not a CoLoR-Filter baseline. They help separate the effect of reference construction from that of the scoring rule.

We hope these clarifications distinguish TACS from CoLoR-Filter and clarify the paper’s contributions. We revised Appendix D to state this comparison and to explain how Figure 8 supports depth calibration. The direct comparison, including both Gemma-2-2B reference rates, is Appendix J.7 (Table 17). All changes are in blue, and we would be glad to clarify any remaining point. In light of these clarifications, we would appreciate the reviewer revisiting their assessment.

If you have any further concerns regarding our replies, please let us know. We are happy to discuss further in the remaining few days of the discussion period.

Comments on the paper

1 comment

We thank all reviewers for their careful reading and constructive questions. During the discussion period we ran a set of new experiments, which are added to the revised manuscript in blue, mostly in Appendix J. We summarize them below and discuss each reviewer's questions in the individual responses.

Together, the results support TACS as a cheap and competitive selector in the settings we evaluated. Because its reference is built from the target alone, a team can store lightweight rank-1 trajectories for the set of targets a model must excel at, and screen each new data corpus as it arrives with two forward passes per candidate, without training on the corpus.

New experiments:

  • Target-only, mixed, and pool-only references with matched capacity and update budget, and a sweep over the pool share of the reference (CpC3)
  • TACS and LESS-style scoring on the same reference paths, now also shown in the main text (Figure 6, right) (CpC3)
  • Two new targets, GSM8K and TruthfulQA, and two new models, Llama-2-7B and Gemma-2-2B (CpC3)
  • A direct comparison with CoLoR-Filter adapted to our setting (Kb1m)
  • Resampled and narrower target proxies for TyDiQA (fTmS, CpC3)
  • An analysis of why gains on MMLU and BBH are small, and a lower-cost LESS baseline (fTmS)

What we learned:

  • We find that the target-only reference selects better data than mixed ones, by 3 to 6 points across three models. TACS draws its target signal from the reference path, and trajectory selectors that already use target information in their score, such as LESS, also benefit from a target-only reference.
  • The TyDiQA gains carry over to Llama-2-7B and Gemma-2-2B (+4.5 and +6.2 F1 over random selection), and TACS stays competitive on GSM8K and TruthfulQA.
  • Compared with CoLoR-Filter adapted to our setting, TACS is ahead in all settings we tried, while CoLoR-Filter is sensitive to the learning rate of its reference.
  • Our proxy study shows that downstream quality is stable across resampled proxies, and a proxy covering a single language keeps most of the gain.
  • On MMLU and BBH, every method, including full-pool training, stays close to the base model. Our analysis suggests that at this training scale, general instruction data mainly improves answer behavior, while these targets call for knowledge and skills that the candidate pools rarely contain. Overall, TACS is competitive across targets but less stable where general instruction data leaves little headroom, and we added this to the limitations.
  • Against a lower-cost one-checkpoint LESS, TACS remains cheaper when new pools are screened for a few targets, because it never trains on the candidate pool. For ToV, almost all of the cost is per-candidate scoring, so a cheaper warmup would barely change the comparison.

We hope these additions address the main concerns, and we would be glad to discuss any remaining question.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.