acceptodds
Under review as a conference paper at ICLR 2027

BiTFlow: Efficient Coreset Selection for Instruction Tuning via Balanced Token Flow Matching

Abstract

Coreset selection for instruction tuning aims to reduce fine-tuning computation while preserving downstream performance. Forward-only selection methods avoid costly model updates and backward passes by comparing hidden-state fingerprints, most often from the last layer. However, samples with highly similar last-layer states can induce different intermediate token updates, a phenomenon we call *last-layer similarity collapse*, indicating that intermediate layers carry information beyond the last-layer state. We capture this information with residual flows, the residual updates of each token between selected layers. Building on these flows, we introduce BiTFlow, a forward-only coreset selection algorithm that compares candidate samples with a small set of target-task samples at the token level. BiTFlow summarizes target residual flows into token-level prototypes and aligns candidate tokens to them via one-to-one bipartite matching. A balanced aggregation then scores each sample by both how well its tokens match and how many distinct target token IDs are covered. At a 5% selection budget on the LESS (0.27M samples) and SmolTalk (1.04M samples) pools, BiTFlow achieves the highest average performance and exceeds full-data fine-tuning on multiple benchmarks, and its selections transfer across model scales and families. Scoring requires no backward passes or warmup training and is 18 faster than gradient-based methods.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Coreset Selection, Residual Flow, LLM Post Training

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_NrfTOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Many forward-only data selection methods utilize the final layer of a model. The paper is built around the observation that different data points can create models with different intermediate layers and the same final layer. To address this and select the right data points, the paper proposes BiTFlow, which tracks how data updates across many layers in a token-wise fashion. The paper demonstrates that BiTFlow selects highly effective training subsets and outperforms other selection methods while being efficient.

Critical strengths

The paper identifies and addresses the problem of last-layer similarity collapse by tracking how data updates across middle layers. The algorithm proposed, BitFlow, is also very computationally efficient and shows strong performance in tests. It outperformed other top selection methods using only a strict 5% data budget. Finally, the training data chosen by BiTFlow transfers well to other models.

Critical weaknesses

None.

Questions for authors
  1. The results show that performance drops when you select 15% of the data. Besides general noise, why does this happen? How can a user find the best data size for a new dataset?
  2. The system's success depends heavily on how it weights the prompt versus the response. How should users choose the right weight for a new task? Is there a simple rule to avoid testing every option?
  3. BiTFlow only slightly beat simpler methods on the very clean SmolTalk dataset. Is BiTFlow less useful for data that is already filtered? When is the extra effort worth it?
Reviewer_g2p8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BiTFlow, a forward-only coreset selection method for instruction tuning that uses intermediate-layer residual updates to capture information obscured by similar last-layer representations. It constructs target token prototypes and scores candidate samples through one-to-one bipartite matching and balanced aggregation of matching quality and target coverage. Experiments on the LESS and SmolTalk pools show that selecting 5% of the data achieves the highest average performance among the evaluated selectors, with selected subsets also transferring across model scales and families.

Critical strengths
  • Detailed hyperparameter analysis and study. The paper examines matching cost, prompt–response weighting, and layer configurations across three tasks and multiple training seeds, providing useful insights into the method’s design choices and sensitivity.

  • Broad empirical evaluation. Experiments cover two candidate pools, multiple benchmarks, and diverse baselines. BiTFlow achieves the highest average performance among the evaluated selectors at a 5% selection budget, and its selected subsets transfer across model scales and families.

Critical weaknesses
  • Unclear benefit of residual-flow encoding. In Table 4a, residual flow only slightly outperforms projected multi-layer hidden states on GSM8K and performs slightly worse on MMLU and TyDiQA. These results support using intermediate-layer information but do not clearly establish the additional benefit of residual-flow encoding.

  • Unclear statistical significance of performance gains. The average improvement is largely driven by gains on a few benchmarks (e.g., AlpacaEval on the LESS pool). Multiple-seed evaluations with appropriate statistical significance tests (e.g., paired t-tests) would help establish whether the gains over strong baselines are robust.

  • Lack of detailed computational analysis and missing baselines. The reported timings cover feature extraction and scoring but exclude method-specific preparation costs, such as BiTFlow’s reference fitting and S2L’s proxy-model training, as well as final subset selection. Moreover, for proxy-model using strategies, S2L is included, but its reliance on proxy training and loss trajectories does not fully represent the range of efficient small-model selection methods. Comparisons with efficient alternatives such as Superfiltering [1] and GradFiltering [2], reporting both downstream performance and end-to-end computational cost under matched selection budgets, would better substantiate BiTFlow’s practical efficiency.

  • Unclear relationship between token coverage and task relevance. BiTFlow measures target coverage through matched token IDs, which may favor lexical overlap without necessarily capturing task-relevant concepts or reasoning patterns. Comparisons with token-overlap scoring under the same matching and aggregation framework, together with robustness tests using paraphrased targets, would help clarify what this coverage measure captures.

  • Limited evaluation of cross-model transfer. Table 3 compares transferred subsets only against random selection. While this demonstrates that BiTFlow’s selections remain useful across models, it does not establish whether they transfer better than subsets selected by competitive baselines such as BM25S and RDS+.

References
[1] Li et al., Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning, ACL 2024
[2] Yuan et al., Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning, arXiv 26.01 \

Questions for authors
  • How sensitive is BiTFlow to the size and composition of the target set? Since the reported training seeds reuse a fixed selection, could the authors evaluate different target-set sizes and sampling seeds to assess selection robustness?

Please refer to the weakness section for more details.

Reviewer_9UKROpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

BiTFlow is a forward-only method for targeted instruction coreset selection. It represents tokens using projected, normalized residual updates between selected transformer layers, aggregates target instances into token-ID prototypes, and scores candidates through exact and one-to-one fallback matching. A harmonic aggregation balances candidate support and target-token coverage, with separate prompt and response scores. Experiments on the LESS and SmolTalk pools compare 5% subsets against several selectors and full-data training. Additional analyses study intermediate-layer information, scoring components, selection budgets, cross-model transfer, and candidate-scoring cost.

Critical strengths
  1. The paper develops a coherent token-level selection method with good implementation detail. One-to-one fallback matching controls repeated use of a target prototype, while balanced aggregation accounts for both candidate support and target coverage. Table 4 compares these choices within a shared framework, including a many-to-one control at the same matching cost, which is more informative than an overall comparison alone.
  2. The empirical study is substantial for a forward-only selector. Table 1 compares two source pools and numerous baselines, with three training seeds on LESS; Appendix E specifies disjoint target and evaluation IDs. Table 3 demonstrates useful transfer, particularly on GSM8K. Table 5 reports 30.03 GPU-ms per candidate versus 547.98 for LESS, supporting a large candidate-scoring speed advantage without scorer warmup.
Critical weaknesses
  1. The distinctive benefit of residual-flow encoding remains limited relative to ordinary multi-layer states. In Table 4(a), the matched projected multi-layer control is slightly better on MMLU and TyDiQA, while residual flow is better on GSM8K. Appendix G.1 finds only a 0.83-point probe advantage over last-layer states on the most similar pairs, with a confidence interval spanning zero. The paper acknowledges these results, but they suggest that the evidence supports using intermediate information more strongly than the specific residual-flow explanation of the downstream gains.
  2. The aggregate performance advantage needs more careful qualification. On LESS, BiTFlow's 41.92 average exceeds full-data training's 41.59 by only 0.33 points, while its AlpacaEval score rises by 12.74 points; excluding that benchmark, its mean is lower than full-data training. On SmolTalk, the 0.78-point lead over RDS+ uses one training seed. AlpacaEval reports raw win rate, and selection is fixed across training runs. These choices leave uncertainty about how much of the overall advantage persists beyond preference-score and target/selection variation.
Questions for authors
  1. Given the matched multi-layer-state control, what conclusion should be attributed specifically to residual flow? Can you provide paired uncertainty for the encoding differences in Table 4(a) and state whether the contribution is primarily the matching/aggregation framework or the residual representation?
  2. Can you report uncertainty for the aggregate improvement, repeat the SmolTalk comparison with the nearest baseline, and provide a length-controlled preference result or a response-length diagnostic for the existing AlpacaEval outputs? A small target/reference resampling check would further clarify whether training-seed variation captures the relevant uncertainty.

Comments on the paper

1 comment

Dear reviewers,

Please be aware that little time remains for the author-reviewer discussion. As the deadline is approaching, we kindly request that you refrain from asking for additional experiments. Please summarize your final position concisely.

Sincerely, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.