acceptodds
Under review as a conference paper at ICLR 2027

TokenFlow: Inference-Free Token Selection with Diversity Preservation for LLM Fine-Tuning

Abstract

Supervised fine-tuning (SFT) is critical for aligning large language models (LLMs), yet its efficiency is hindered by the uneven utility of tokens in instruction datasets. Existing token-level selection methods rely on reference models or auxiliary inference, introducing significant computational overhead. In this paper, we propose TokenFlow, a plug-and-play, inference-free framework for token-efficient SFT. TokenFlow exploits intrinsic signals from the standard SFT forward pass and consists of two synergistic modules that determine which tokens to optimize and how to optimize them: MINT (Maximizing INstruction-aware Token utility) and Tempered Cross-Entropy (TCE). MINT reformulates token selection as an instruction-aware subset utility maximization problem. Unlike existing methods that independently rank tokens, MINT considers token interactions and subset-level utility to select complementary token subsets under a fixed budget. TCE preserves response diversity by incorporating semantically valid alternatives from the model's predictive distribution while maintaining correctness supervision. Extensive experiments across diverse LLM families demonstrate the effectiveness of TokenFlow. Training on only 60% of the tokens, TokenFlow outperforms full-dataset fine-tuning by 2.78 points on the OpenLLM Leaderboard and consistently surpasses state-of-the-art selection baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Data Selection, LLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_p3qyOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an inference-free framework for efficient supervised fine-tuning (SFT). This is done by dynamically choosing which tokens need to be trained on. The first component of the framework is MINT, which uses DPP to select a token subset. The second is Tempered Cross-Entropy (TCE), a modified loss function that ensures diversity by considering alternative tokens. The results show that this framework can outperform traditional SFT across multiple LLM families and datasets.

Critical strengths

The paper addresses fine-tuning efficiency, which is becoming increasingly relevant as datasets and models grow. The experiments also cover a variety of model families.

Critical weaknesses

My main concern with this paper is that it does not cite or discuss two very relevant works.

  1. Nagaraj et al., "TRIM: Token-wise Attention-Derived Saliency for Data-Efficient Instruction Tuning." ICML 2026. This paper addresses the same issue similarly - it uses attention-derived signals from a forward pass to evaluate token-level importance. Hence, it is very important for the paper to compare MINT to TRIM: how do they differ, how they are similar, a comparison of results if possible, and where each application lies.

  2. Jia, Yaning, et al. "Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning." 2026. This paper also similarly scales SFT loss using the per-token logit gap, trimming both extremes of model confidence. How is TCE different from this paper? Can there be a comparison?

Questions for authors

Please provide a comparison with the papers mentioned in critical weaknesses and any other relevant papers.

6 comments
AuthorsOpenReview ↗

Response to Reviewer p3qy

W2: Jia, Yaning, et al. "Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning." 2026. This paper also similarly scales SFT loss using the per-token logit gap, trimming both extremes of model confidence. How is TCE different from this paper? Can there be a comparison?

Thank you for highlighting this related work. We clarify how TCE differs from TrimSFT and provide an additional experimental comparison below.

Part 1. Different optimization mechanisms

TrimSFT follows a token-level loss reweighting paradigm: it uses logit gaps to adjust the contribution of individual tokens while retaining the original ground-truth supervision. Thus, it only changes how strongly each ground-truth token is learned, without explicitly preserving plausible alternative continuations in the supervision signal. In contrast, TokenFlow introduces Tempered Cross-Entropy (TCE), which incorporates plausible alternatives from the model's predictive distribution alongside the ground-truth token. By enriching the supervision signal rather than merely reweighting token losses, TCE avoids unnecessarily suppressing semantically valid alternatives, providing an explicit mechanism for preserving response diversity.

Part 2. Additional Experiment

We compare Full-Token SFT with TCE against Full-Token SFT with TrimSFT, with benchmark and response-diversity results reported in Tables 1 and 2, respectively. TCE consistently achieves better average task performance across all three backbones. It also exhibits substantially higher response diversity on both AlpacaEval and GSM8K, as reflected by lexical and semantic diversity metrics. Moreover, its superior GSM8K pass@k performance demonstrates better coverage of correct solutions under repeated sampling. Finally, incorporating MINT further improves benchmark performance and GSM8K pass@k, highlighting the complementary contribution of diversity-aware token selection.

Table 1. Benchmark performance

BackboneMethodHellaSwagBoolQGSM8KTyDiQALogiQATriviaQAARC-COpenBookQAAverage
LLaMA-3.1-8BFull Token + CE60.5783.1154.1348.2128.0662.7856.0745.4554.80
Full Token + TrimSFT52.8166.0948.6260.8426.2063.0148.2341.6250.93
Full Token + TCE63.5285.1855.4459.1428.0663.3355.2145.4556.92
MINT + TCE66.4885.7955.9058.7229.3067.1357.1947.2758.47
Qwen2-7BFull Token + CE58.8384.9678.1051.0731.1656.4357.9745.4558.00
Full Token + TrimSFT51.4785.2175.4267.8130.3955.3646.2538.7956.34
Full Token + TCE61.6683.5475.3461.0631.6357.7457.8045.2559.25
MINT + TCE64.1985.2477.9565.1433.3355.5056.0746.0660.44
OLMo-2-7BFull Token + CE60.5483.4568.5344.8725.8962.2559.0046.0656.32
Full Token + TrimSFT55.3184.5968.0761.9227.4461.8552.4545.0557.09
Full Token + TCE61.9081.4169.6053.5126.0561.9959.4347.8857.72
MINT + TCE63.9383.6971.6759.7425.8962.6259.8747.8859.41

Table 2. Response diversity and repeated-sampling performance

(a) Response diversity on AlpacaEval

MetricFull Token + TrimSFTFull Token + TCEMINT + TCE
Self-BLEU ↓91.2175.6972.02
Distinct-1 ↑0.03620.09260.0916
Distinct-2 ↑0.07060.24010.2501
Distinct-3 ↑0.09430.36090.3828
Cosine distance ↑0.04650.14070.1597
L2 distance ↑0.20950.48110.5229
1-NN ↑0.04840.26700.3087
Unique response rate (%) ↑43.0088.5092.97

(b) Response diversity on GSM8K

MetricFull Token + TrimSFTFull Token + TCEMINT + TCE
Self-BLEU ↓99.7284.4678.45
Distinct-1 ↑0.02990.04330.0722
Distinct-2 ↑0.03720.15060.2223
Distinct-3 ↑0.04240.27310.3656

(c) GSM8K pass@k

Methodpass@1pass@2pass@4pass@8pass@16pass@32
Full Token + TrimSFT13.7715.4516.9318.0518.8319.33
Full Token + TCE20.2833.4050.1567.3181.0389.54
MINT + TCE48.8966.4779.3387.1592.0395.38

We have added the TrimSFT comparison to Section 4.3 (Figure 6(c)) and Appendix D.1 (Figure 10(c)).

AuthorsOpenReview ↗

Response to Reviewer p3qy

Q1: Please provide a comparison with the papers mentioned in critical weaknesses and any other relevant papers.

We discuss the relevant work in the Related Work section and report the experimental results on LLaMA-3.1-8B below. Scores are unweighted averages over the eight downstream benchmarks; ssToken results are taken from Table 1 in our response to Reviewer #2, and Full Token + TCE and TrimSFT results are taken from Table 1 in this response.

Table 3. Average benchmark performance with CE and TCE across selection strategies.

Selection strategyCETCEGain from TCE
Full Token54.8056.92+2.12
Random54.5056.26+1.76
RHO-154.8656.71+1.85
TokenCleaning55.6957.11+1.42
DFT54.9056.74+1.84
EFT54.4156.58+2.17
ssToken55.0056.41+1.41
MINT56.4658.47+2.01

Table 4. Average benchmark performance with different training objectives.

Training objectiveAverageChange from SFT
SFT (CE)54.80—
TrimSFT50.93-3.87
ToFu53.17-1.63
KL55.09+0.29
GEM55.62+0.82
DFT54.90+0.10
NEFT53.97-0.83
TCE56.92+2.12

We have expanded the ssToken baseline comparison in Section 4.2 (Table 1) and the selection-strategy and training-objective comparisons in Section 4.3 (Figure 6) and Appendix D.1 (Figure 10).

W1: Nagaraj et al., "TRIM: Token-wise Attention-Derived Saliency for Data-Efficient Instruction Tuning." ICML 2026. This paper addresses the same issue similarly - it uses attention-derived signals from a forward pass to evaluate token-level importance. Hence, it is very important for the paper to compare MINT to TRIM: how do they differ, how they are similar, a comparison of results if possible, and where each application lies.

We thank the reviewer for pointing out the related work TRIM. Although both methods leverage attention-derived signals, they address fundamentally different data selection problems: TRIM selects training samples, whereas MINT selects response tokens within fixed training samples. Therefore, a direct quantitative comparison is not well-defined due to their different selection units and budget definitions. Nevertheless, we clarify their similarities and differences from three perspectives.

(1) Problem formulation. TRIM evaluates individual token relevance and aggregates scores for sample selection. In contrast, MINT reformulates token selection as an instruction-aware subset utility maximization problem, explicitly modeling token interactions to select informative and complementary tokens under a limited budget. Thus, MINT's contribution lies not merely in exploiting attention signals, but in this new problem formulation.

(2) Task dependency. TRIM primarily targets task-specific selection using target-set examples to construct token fingerprints (with an in-domain variant using an internally sampled target pool). MINT is target-task-agnostic and requires neither external validation data nor task-specific fingerprints.

(3) Selection efficiency. TRIM requires separate LoRA warmup and forward-scoring stages. MINT directly reuses logits and attention signals from the standard SFT forward pass, requiring no separate warmup, external reference model, or additional inference passes.

We have added a discussion distinguishing TRIM’s sample-level selection from TokenFlow’s token-level selection in Appendix B (Related Work); please refer to this addition in the revised manuscript.

Thank you for providing these comparisons. Adding these to the paper will strengthen it. I will revise my score accordingly.

Reviewer_ZDjLOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets the token-level selection problem and proposes TokenFlow to improve training efficiency. Specifically, TokenFlow incorporates two components: MINT, which combines instruction relevance derived from attention with representation-based diversity through DPP subset selection, and TCE, which assigns reward probability to the annotated token and high-probability alternatives. TokenFlow does not require a reference model to obtain scores, making it more efficient and scalable. Experiments are conducted on three base models and across multiple benchmarks to demonstrate the effectiveness of TokenFlow.

Critical strengths
  1. The two components, MINT and TCE, are both technically sound. MINT does not rank tokens independently, but considers the selected subset as a whole. TCE not only learns from the standard ground-truth token, but also leaves room for other plausible answers. The ablation studies confirm that each component contributes to performance.

  2. The experiments are thorough, covering multiple model families and benchmarks. The authors also show the improvements of their method over popular sample-level selection methods.

Critical weaknesses
  1. ssToken [R1] already uses attention to estimate instruction-related token importance, which is closely related to TokenFlow. However, TokenFlow mentions ssToken only in Figure 7, without a formal citation, discussion, reproduction details, or main-table results. This omission makes it difficult to assess the novelty of the attention-based relevance component and the advantage over a closely related approach. ssToken should be compared under matched settings, including with TCE applied to both selectors.

  2. TCE treats high-probability non-annotated tokens as alternatives, but this may introduce incorrect predictions, particularly in factual or math tasks. The experiments in the paper seem more focused on task performance than on measuring response diversity.

[R1] ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuning. ICLR 2026.

Questions for authors
  1. Can the authors formally discuss and cite ssToken, explain its reproduction setup in Figure 7, and provide matched comparisons such as ssToken + CE, MINT + CE, ssToken + TCE, and MINT + TCE? I believe this is an important experiment for verifying the effectiveness of TokenFlow.

  2. Can the authors report the total computational cost of TokenFlow (e.g., attention extraction, similarity construction, DPP selection) and compare it with other token-level selection methods?

5 comments
AuthorsOpenReview ↗

Response to Reviewer ZDjL

W1: ssToken [R1] already uses attention to estimate instruction-related token importance, which is closely related to TokenFlow. However, TokenFlow mentions ssToken only in Figure 7, without a formal citation, discussion, reproduction details, or main-table results. This omission makes it difficult to assess the novelty of the attention-based relevance component and the advantage over a closely related approach. ssToken should be compared under matched settings, including with TCE applied to both selectors.

Q1: Can the authors formally discuss and cite ssToken, explain its reproduction setup in Figure 7, and provide matched comparisons such as ssToken + CE, MINT + CE, ssToken + TCE, and MINT + TCE? I believe this is an important experiment for verifying the effectiveness of TokenFlow.

Thank you for highlighting this related work. We clarify the methodological differences and provide the requested matched comparisons below.

Part 1: Methodological differences.

While both ssToken and TokenFlow use attention to estimate instruction-related token importance, MINT further distinguishes attention to different instruction components. ssToken measures overall attention to the instruction; MINT accounts for the unequal importance of instruction components, guiding selection toward response positions associated with salient instruction content.

More fundamentally, MINT changes the selection objective from individual token ranking to instruction-aware subset utility maximization. ssToken combines attention with retrospective excess loss to rank tokens individually. Such rankings do not explicitly account for redundancy among selected tokens: several highly relevant tokens may provide overlapping information. MINT addresses this limitation through a DPP objective that jointly models relevance and inter-token similarity, making a token's selection value depend on its complementarity with the other selected tokens. The objective therefore allocates a limited supervision budget according to the collective utility of the subset. TCE complements this selection objective by calibrating how the selected positions are supervised, retaining the annotated-target signal while allowing probability mass on plausible alternatives. Together, MINT and TCE address both the selection of complementary supervision positions and the flexibility of their training targets.

An additional practical distinction concerns historical-model scoring. ssToken's retrospective excess loss measures per-token loss reduction relative to a history model. Computing this loss difference requires obtaining per-token losses from a history model, introducing additional inference overhead. MINT reuses relevance and similarity signals from the current SFT forward pass, avoiding this historical-model scoring stage.

Part 2: Experimental comparison.

To assess MINT's advantage over ssToken and the contribution of TCE, we evaluate both selectors with CE and TCE across three backbones and eight benchmarks.

Table 1. ssToken and MINT with CE and TCE. Average is the mean over eight benchmarks.

BackboneMethodHellaSwagBoolQGSM8KTyDiQALogiQATriviaQAARC-COpenBookQAAverage
LLaMA-3.1-8BssToken + CE60.1782.2747.4055.2626.2066.1656.5046.0655.00
MINT + CE63.6583.2954.6755.2627.4464.6356.6846.0656.46
ssToken + TCE60.6485.5553.3760.9224.8164.1955.3046.4656.41
MINT + TCE66.4885.7955.9058.7229.3067.1357.1947.2758.47
Qwen2-7BssToken + CE58.7782.8072.4351.6331.1657.1458.0544.4457.05
MINT + CE59.2483.2974.2753.5131.1659.2059.5246.2658.31
ssToken + TCE61.3485.8973.8959.1029.9258.8257.9745.8659.10
MINT + TCE64.1985.2477.9565.1433.3355.5056.0746.0660.44
OLMo-2-7BssToken + CE60.5376.5663.7851.4324.3463.5957.1146.8755.53
MINT + CE61.1481.9664.6253.4225.7464.4559.4347.4757.28
ssToken + TCE61.0881.3866.4660.5524.9661.3758.8347.2757.74
MINT + TCE63.9383.6971.6759.7425.8962.6259.8747.8859.41

We have added the formal citation and discussion in Section 5 (Related Work) and these experiments are also included in Section 4.2, Table 1 Section 4.3 (Figure 6) and Appendix D.1 (Figure 10).

AuthorsOpenReview ↗

Response to Reviewer ZDjL

W2: TCE treats high-probability non-annotated tokens as alternatives, but this may introduce incorrect predictions, particularly in factual or math tasks. The experiments in the paper seem more focused on task performance than on measuring response diversity.

Thank you for raising this concern. TCE preserves reference supervision through the annotated-target term, weighted by α, while β controls the contribution of candidate continuations. This allows flexibility beyond a single reference, whose wording or solution trajectory need not cover every valid continuation. The ratio β/α controls this relaxation, with β = 0 recovering CE up to an additive constant.

To directly test whether this flexibility produces more diverse qualified answers while maintaining correctness, we added two response-level evaluations: quality-filtered diversity on AlpacaEval and diversity together with final-answer correctness on GSM8K.

(1) Direct response diversity and semantic appropriateness. To assess whether the additional variation includes appropriate answers, we measure lexical and semantic diversity on 100 AlpacaEval prompts and separately evaluate diversity after quality filtering. An anonymized deepseek-v4-pro judge scores correctness, relevance, instruction following, and coherence from 1 to 5; a response qualifies only if all four scores are at least 4. For the qualified-answer comparison, we use four responses per method on the same 80 eligible prompts, averaging over 100 subsampling repetitions.

Table 2. Response diversity and qualified-answer diversity on AlpacaEval.

CategoryMetricFull TokenMINT (w/o TCE)TokenFlow
SemanticCosine distance ↑0.09290.10280.1597
Pairwise L2 distance ↑0.35530.38190.5229
Nearest-neighbor L2 distance ↑0.12940.17230.3087
Qualified-answer cosine distance ↑0.07900.09030.1363
LexicalDistinct-1 ↑0.06720.07850.0916
Distinct-2 ↑0.14880.18670.2501
Distinct-3 ↑0.20440.26290.3828
Self-BLEU ↓87.3579.7172.02
Unique-response rate (%) ↑66.3876.1992.97

(2) Diversity and correctness in mathematical reasoning. We report response diversity and final-answer correctness on all 1,319 GSM8K test questions under matched prompts, decoding settings, and sampling budgets.

Table 3. Response diversity and final-answer correctness on GSM8K.

CategoryMetricFull TokenMINT (w/o TCE)TokenFlow
LexicalSelf-BLEU ↓98.3294.4678.45
Distinct-1 ↑0.04270.05470.0722
Distinct-2 ↑0.07260.13360.2223
Distinct-3 ↑0.09530.20360.3656
Unique-response rate (%) ↑21.3253.4285.78
Correctnesspass@1 (%) ↑17.2217.6948.89
pass@2 (%) ↑21.4526.0866.47
pass@4 (%) ↑25.6936.0279.33
pass@8 (%) ↑29.4846.3787.15
pass@16 (%) ↑32.6755.8892.03
pass@32 (%) ↑35.4863.6895.38

We have added quality-filtered diversity and GSM8K correctness analyses in Section 4.6 (Figures 8–9) and Appendix E (Tables 12–17).

Q2: Can the authors report the total computational cost of TokenFlow and compare it with other token-level selection methods?

Thank you for this suggestion. We have added a computational-cost comparison with other token-level selection methods and a breakdown of TokenFlow's runtime, as shown below. TokenFlow requires no warm-up training for token selection, no additional full-model inference passes, and no reference model, substantially reducing selection-related overhead.

Table 4. Runtime comparison for 50,000 examples and one epoch.

MethodSelectionTrainingTotal
RHO-19,787.447,318.6017,106.04
TokenCleaning10,639.467,281.7817,921.24
EFT6,516.707,302.3913,819.09
ssToken + CE5,798.387,279.4313,077.81
MINT + CE1,008.907,246.238,255.14
TokenFlow (MINT + TCE)1,008.907,344.558,353.45

Table 5. Runtime breakdown of complete TokenFlow for 50,000 examples.

ComponentTime (s)Seconds/sampleShare of total
Attention extraction and instruction-aware relevance scoring184.120.0036822.20%
Similarity and DPP kernel-matrix construction3.530.0000710.04%
Greedy DPP selection and mask construction821.250.0164259.83%
Selection subtotal1,008.900.02017812.08%
Training (including TCE)7,344.550.14689187.92%
Total8,353.450.167069100.00%

We have added the complete computational-cost comparison and component-level breakdown in Appendix D.2 (Tables 7–8) and updated Section 4.4 (Figure 7).

Dear Reviewer ZDjL,

Thank you for your time and for your thoughtful comments on our submission. As the discussion deadline is approaching, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you again for your time and consideration.

Best regards,

Authors

Reviewer_SDvdOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To reduce unnecessary computational cost, this paper investigates whether supervising only part of a response during fine-tuning can outperform full-token supervision. It proposes TokenFlow, which selects supervision positions using information already available during training, without an additional reference model. TokenFlow has two components: MINT selects instruction-relevant positions that provide complementary information, while TCE assigns partial credit to model-predicted alternatives alongside the annotated token, aiming to preserve other reasonable expressions. Experiments across three model families and eight benchmarks show higher average scores than full-token fine-tuning, while directly supervising only about 60% of response positions. Ablation results support the contributions of both the selection strategy and the modified training objective to these gains.

Critical strengths
  1. The paper offers a useful training approach by considering both which positions to supervise and how to supervise them. Unlike methods that select tokens by independent scoring and ranking, MINT accounts for what each position contributes beyond those already selected. It combines instruction relevance with inter-position similarity to identify supervision that is both relevant and complementary. TCE then modifies the training objective so that high-probability alternative predictions are penalized less heavily. Together, these components link the choice of supervision positions to the way those positions are trained, using signals from the model being trained and without an additional reference model.

  2. The experiments provide supporting evidence for this approach. The authors evaluate the selection strategy and the training objective separately, rather than reporting only the complete method’s final performance. On LLaMA-3.1-8B, full-token CE, MINT with CE, full-token TCE, and the complete method achieve reported average scores of 54.80, 56.46, 56.93, and 58.47, respectively (Tables 1–2 and Figure 6). These results indicate gains from each component and further improvement when combined. Under the same TCE objective, the reported figures imply 57.11 for TokenCleaning versus 58.47 for MINT, suggesting that the advantage is not explained by the loss change alone. Higher average scores than full-token SFT across three model families further indicate that the benefits are not limited to a single model.

Critical weaknesses
  1. Response diversity is not directly evaluated. Selecting complementary supervision positions does not establish that the model generates more diverse valid answers, and high-probability alternatives are not necessarily semantically appropriate. The task scores support performance gains, but provide insufficient evidence for the central claim of preserving response diversity.

  2. The efficiency of the complete method remains unclear. Supervising 60% of response positions does not imply proportional computational savings, since the full context is still processed and token selection adds computation. Although the paper reports training-time comparisons, it is unclear whether the measured configuration includes both MINT and TCE. This makes it difficult to connect the reported runtime with the complete method’s performance gains.

Questions for authors
  1. Do you have direct evidence that TCE preserves response diversity while maintaining answer quality, compared with CE under matched token-selection settings, decoding settings, and sampling budgets?

  2. Does “Ours” in Figure 7 correspond to MINT with CE or the complete method with TCE? If it measures MINT with CE, the figure does not establish the cost of obtaining the complete method’s reported performance; if it measures MINT with TCE, the different performance scores remain unexplained.

4 comments
AuthorsOpenReview ↗

Response to Reviewer SDvd

W1: Response diversity is not directly evaluated. Selecting complementary supervision positions does not establish that the model generates more diverse valid answers, and high-probability alternatives are not necessarily semantically appropriate. The task scores support performance gains, but provide insufficient evidence for the central claim of preserving response diversity.

Q1: Do you have direct evidence that TCE preserves response diversity while maintaining answer quality, compared with CE under matched token-selection settings, decoding settings, and sampling budgets?

Thank you for the suggestion. We conducted additional experiments evaluating (1) lexical and semantic diversity, (2) answer quality and qualified-answer diversity, and (3) correctness under repeated sampling.

(1) Lexical and semantic response diversity. To address W1's request for direct response-level evidence, we measure lexical and semantic variation among generated answers. Each method generates 32 responses per prompt using the same tokenizer, prompt format, temperature (0.6), top-p (0.9), top-k (50), and maximum generation length (2,048 tokens).

Table 1. Lexical and semantic response diversity on AlpacaEval.

CategoryMetricFull TokenMINT (w/o TCE)TokenFlow
SemanticCosine distance ↑0.09290.10280.1597
Pairwise L2 distance ↑0.35530.38190.5229
Nearest-neighbor L2 distance ↑0.12940.17230.3087
LexicalDistinct-1 ↑0.06720.07850.0916
Distinct-2 ↑0.14880.18670.2501
Distinct-3 ↑0.20440.26290.3828
Self-BLEU ↓87.3579.7172.02

Table 2. Lexical response diversity on GSM8K.

CategoryMetricFull TokenMINT (w/o TCE)TokenFlow
LexicalSelf-BLEU ↓98.3294.4678.45
Distinct-1 ↑0.04270.05470.0722
Distinct-2 ↑0.07260.13360.2223
Distinct-3 ↑0.09530.20360.3656
Unique-response rate (%) ↑21.3253.4285.78

(2) Answer quality and diversity among qualified answers. To address W1's concern about semantic appropriateness and Q1's request for diverse valid answers, we test whether the diversity gain persists after quality filtering. We evaluate all 9,600 AlpacaEval responses (3,200 per method) with deepseek-v4-pro, using anonymized model identities and the same rubric. Each response receives a score from 1 to 5 for correctness, relevance, instruction following, and coherence. Overall quality is the mean of these four scores; a response qualifies only if every dimension scores at least 4.

We then directly measure semantic diversity among qualified answers, comparing four such responses per method on the same 80 eligible prompts and averaging over 100 subsampling repetitions. We also measure response uniqueness on all 100 prompts. As shown in Table 3, qualified-answer semantic diversity improves by 51% over MINT (w/o TCE), demonstrating that the diversity gain persists after filtering for answer quality. Response uniqueness also increases from 76.19% to 92.97%.

Table 3. Qualified-answer semantic diversity and response uniqueness on AlpacaEval.

CategoryMetric ↑Full TokenMINT (w/o TCE)TokenFlow
SemanticQualified-answer cosine distance (80 prompts)0.07900.09030.1363
LexicalUnique-response rate (%; 100 prompts)66.3876.1992.97

(3) Final-answer correctness and repeated-sampling benefits. To address Q1's requirement of maintaining answer quality under matched sampling budgets, we additionally evaluate final-answer correctness and correct-answer coverage. Each method generates 32 responses per question across all 1,319 GSM8K test questions under the same decoding settings. To quantify the benefit of repeated sampling, we also report the gain over each method's own pass@1:

Table 5. GSM8K pass@k under matched sampling budgets.

Metric (%)Full TokenMINT (w/o TCE)TokenFlow
pass@117.2217.6948.89
pass@221.4526.0866.47
pass@425.6936.0279.33
pass@829.4846.3787.15
pass@1632.6755.8892.03
pass@3235.4863.6895.38

Table 6. GSM8K sampling gains over pass@1 (percentage points).

Sampling gain (percentage points) ↑Full TokenMINT (w/o TCE)TokenFlow
pass@1 − pass@10.000.000.00
pass@2 − pass@14.238.3917.58
pass@4 − pass@18.4818.3330.44
pass@8 − pass@112.2628.6838.26
pass@16 − pass@115.4538.1943.14
pass@32 − pass@118.2645.9946.48

We have added these experiments in Section 4.6 (Tables 4–5 and Figures 8–9) and Appendix E.

AuthorsOpenReview ↗

Response to Reviewer SDvd

W2: The efficiency of the complete method remains unclear. Supervising 60% of response positions does not imply proportional computational savings, since the full context is still processed and token selection adds computation. Although the paper reports training-time comparisons, it is unclear whether the measured configuration includes both MINT and TCE. This makes it difficult to connect the reported runtime with the complete method’s performance gains.

Q2: Does “Ours” in Figure 7 correspond to MINT with CE or the complete method with TCE? If it measures MINT with CE, the figure does not establish the cost of obtaining the complete method’s reported performance; if it measures MINT with TCE, the different performance scores remain unexplained.

Thank you for pointing this out. We apologize that the original Figure 7 reported MINT + CE and omitted the cost of the complete TokenFlow (MINT + TCE) method. Under the same setting, we have additionally profiled the complete method and updated the runtime comparison below to include both MINT + CE and MINT + TCE.

**Table 7. Runtime for 50,000 examples and one epoch. **

MethodSelectionTrainingTotal
RHO-19,787.447,318.6017,106.04
TokenCleaning10,639.467,281.7817,921.24
EFT6,516.707,302.3913,819.09
ssToken + CE5,798.387,279.4313,077.81
MINT + CE1,008.907,246.238,255.14
TokenFlow (MINT + TCE)1,008.907,344.558,353.45

Regarding the 60% supervision ratio, we agree that it does not imply proportional computational savings. In fact, the primary goal of token selection is data-efficient learning: extracting more useful supervision from the available training examples. This motivation is particularly relevant when high-quality training data are scarce or responses contain noisy or uninformative tokens, making it important to prioritize useful training signals within each example. By focusing supervision on informative and complementary response positions, TokenFlow achieves higher average downstream performance than full-token SFT on the same training samples while using a smaller supervised-token budget. Reducing computation or training time is not its primary objective.

We have revised the runtime–performance comparison in Section 4.4 (Figure 7) and added complete-method costs in Appendix D.2 (Tables 7–8).

Thank you for the additional experiments and clarifications. I will raise my score.

Thank you for carefully considering our response and for updating your assessment. We sincerely appreciate your constructive feedback, which has helped us further clarify TokenFlow's contributions and provide a more comprehensive empirical evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.