acceptodds
Under review as a conference paper at ICLR 2027

TokenDistill: Compressing CoT via Iterative Token Distillation with Continuous Prompt Transformation

Abstract

Chain-of-Thought (CoT) prompting can substantially improve the final-answer accuracy of large language models (LLMs) by enabling step-by-step reasoning. However, the intermediate tokens it generates incur additional computational and memory overhead, thereby increasing deployment costs. Existing CoT compression methods either attain limited compression ratios, rely on auxiliary external models, or require extra comparison samples during optimization, which introduces considerable training overhead. To address these limitations, we propose TokenDistill, a novel training pipeline that enables the model itself to iteratively distill CoT tokens in an adaptive manner, achieving aggressive CoT compression while largely preserving final-answer accuracy. At the core of TokenDistill is an iterative prompt transformation mechanism, which enables the model to internalize the requirement for concise outputs from explicit prompting into its own generation behavior. Specifically, we first augment the original input prompts with a concise-output instruction to obtain concise outputs. We then fine-tune the model using the original inputs without the concise-output instruction and the generated concise outputs as supervised input-output pairs, thereby internalizing the concise-output behavior into the model. Furthermore, we repeat the aforementioned procedure over multiple rounds, enabling the model to progressively push its compression limits beyond the previously attained level. Extensive experiments demonstrate that TokenDistill consistently outperforms state-of-the-art methods in CoT compression, incurring only minimal accuracy loss and in some cases even slight accuracy gains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Chain-of-Thought Compression, Large Language Models, Iterative Self-Distillation, Prompt Transformation, Efficient Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LK6BOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TokenDistill, an iterative self-distillation framework for shortening CoT reasoning while preserving answer accuracy. Each iteration generates concise reasoning via a concise-output instruction, filters by answer correctness, fine-tunes on retained responses without the instruction, and decides whether to continue. Its main advantage is avoiding an external teacher, preference comparisons, and RL. Experiments on several Qwen2.5, LLaMA-3.1, and DeepSeek-R1-distilled models over GSM8K, MATH-500, and CommonsenseQA show substantial CoT-length reductions with competitive accuracy. However, the evidence does not establish that iterative data reconstruction is necessary, especially versus simpler repeated SFT under matched compute. And the novelty of the proposed method is limited. Using the test set for stopping and an underspecified checkpoint-selection procedure also raises validity and reproducibility concerns about the reported accuracy-compression trade-offs.

Critical strengths

S1. A simple and practical self-distillation framework without external teacher models The paper proposes an intuitive and computationally accessible approach to CoT compression. Unlike methods requiring external teachers, auxiliary models, or reinforcement learning with specialized reward functions, TokenDistill relies on the model's own generated responses and SFT. By transferring the concise-output requirement from explicit prompting into the model's generation behavior, the proposed framework avoids introducing additional model components or specialized optimization objectives. This design makes the method relatively easy to implement and integrate into existing LLM fine-tuning pipelines. The simplicity and self-contained nature of the framework constitute a meaningful practical contribution, as they reduce implementation complexity and dependence on external models.

S2. Promising empirical results across models and useful analysis of iterative compression The empirical results demonstrate substantial reductions in CoT length across multiple model families and reasoning benchmarks. Particularly encouraging results are reported for DeepSeek-R1-Distill-Qwen-7B. On MATH-500, TokenDistill improves accuracy from 75.6% to 83.0% while reducing average CoT length from 2519.03 to 1221.44 tokens. Similar improvements are observed on GSM8K. Beyond the main comparisons, the authors provide several useful analyses. Appendix A.1 demonstrates that multi-round distillation can achieve stronger compression than single-round training. Section 4.5 further compares two training strategies, showing that the compression improvements do not solely depend on accumulating parameter updates across rounds. The cross-dataset experiments and random-seed analysis also provide additional evidence of transferability and stability. These results support the practical effectiveness of the proposed framework and demonstrate its potential applicability to different reasoning models and tasks.

Critical weaknesses

W1. Insufficient evidence for the additional benefits of iterative data reconstruction My primary concern is that the paper does not sufficiently establish the additional benefits of iterative data reconstruction, which constitutes a central design choice of the proposed framework. I acknowledge that Appendix A.1 demonstrates the benefits of multi-round distillation over single-round training. Furthermore, Section 4.5 provides useful evidence that compression improvements do not solely depend on accumulating parameter updates across rounds. However, these experiments do not directly isolate the contribution of progressively reconstructing the training data. An essential missing baseline is repeated SFT using the same correctness-filtered concise dataset generated in the first round, under a matched training budget. Such a comparison would determine whether the additional compression achieved in later rounds results from progressively reconstructed training data or can be similarly obtained through repeated optimization on existing concise supervision. This distinction is particularly important because TokenDistill requires additional model inference and correctness filtering to reconstruct the dataset at each iteration. Without demonstrating a meaningful advantage over fixed-data repeated SFT, it remains unclear whether this additional computational overhead is justified. While the current experiments support the effectiveness of the overall framework, they do not sufficiently establish the independent benefit of its central iterative data reconstruction mechanism. This limits the strength of the paper's methodological contribution.

W2. Unclear checkpoint selection and potential evaluation bias across distillation rounds The second major concern is the lack of a clearly defined checkpoint-selection strategy across distillation rounds. Comparing Table 1 with Appendix A.1 (Table 4) reveals that the main results use checkpoints from different rounds. For example, on GSM8K, Qwen2.5-3B uses round 2, while Qwen2.5-7B uses round 5. However, the paper does not explain how these checkpoints are selected. This is particularly concerning because the stopping criterion in Section 3.4 (Eq. 12) depends only on average CoT length. While an accuracy constraint is not necessarily required, this criterion does not sufficiently explain why earlier checkpoints are reported even when subsequent rounds achieve further compression. It remains unclear whether the reported checkpoints were selected according to a predefined stopping rule or retrospectively chosen based on their observed accuracy-compression trade-offs. If the latter, selecting checkpoints after inspecting benchmark test accuracy could introduce selection bias and overestimate the effectiveness of the proposed method. Furthermore, Eq. (12) uses the benchmark test set for stopping decisions. Although only response lengths are considered, this still introduces test-set dependence. The authors should clarify the checkpoint-selection protocol and demonstrate that the reported results can be reproduced using predefined criteria and an independent validation set.

Questions for authors

Q1. Necessity of iterative data reconstruction Appendix A.1 demonstrates that multi-round distillation achieves stronger compression than single-round training, while Section 4.5 compares two strategies that both rely on progressively reconstructed datasets. However, have the authors considered a baseline that repeatedly performs SFT on the same correctness-filtered concise dataset generated in the first round, without reconstructing the training data? Could the authors compare this baseline with TokenDistill under matched optimization steps and training-token budgets, reporting the accuracy-compression trade-offs across rounds? This comparison would directly help determine whether iterative data reconstruction provides additional benefits beyond repeated optimization on fixed concise supervision.

Q2. Stopping criteria and checkpoint selection The proposed stopping criterion considers only average CoT length, whereas Appendix A.1 shows that further compression can sometimes cause substantial accuracy degradation. How are the final checkpoints reported in the main experiments selected from different training rounds? In particular, are these selections determined solely by the stopping conditions, or is answer accuracy also considered? Could the authors report the selected round and corresponding stopping-condition values for each experimental setting, and clarify how the proposed procedure ensures a favorable accuracy-compression trade-off?

Q3. Independence of the evaluation protocol Can the authors reproduce the main results using an independent validation set for stopping and checkpoint selection, reserving the test set exclusively for final evaluation? Please also report the selected round and corresponding stopping decisions for each setting.

1 comment

Thank you for the detailed and constructive review. We appreciate your recognition of the framework’s simplicity, self-contained design, and empirical compression results. Your concerns identify two important issues: isolating the contribution of data reconstruction and making checkpoint selection independent of the test set. We address them separately below.

W1 / Q1: Additional benefits of iterative data reconstruction

Thank you for suggesting this comparison. Accordingly, we evaluated fixed-data repeated SFT on Qwen2.5-7B/MATH: all five rounds reuse the same 3,845 correctness-filtered concise examples from Round 1, with each round continuing from the preceding model.

We compare this baseline with the two training strategies described in Sections 3.3 and 4.5 and evaluate all three on MATH-500.

Table R1. Comparison of fixed-data SFT and iterative data reconstruction on MATH-500

StageFixed-data Acc. (%)Fixed-data ATLMkM_k-based Acc. (%)MkM_k-based ATLM0M_0-based Acc. (%)M0M_0-based ATL
r0r_0 (Original)71.2574.6571.2574.6571.2574.65
r1r_172.0523.1072.0523.1072.0523.10
r2r_272.6520.7370.4470.9571.6463.10
r3r_373.4516.9068.8415.4370.4422.21
r4r_474.4514.9368.6362.7067.8396.74
r5r_572.8509.4969.0338.5570.8375.32

MkM_k-based denotes the training in each round based on the previous-round model, while M0M_0-based denotes the alternative strategy that always uses the initial model as the base model. Both strategies reconstruct the training dataset each round. Fixed-data repeated SFT follows the MkM_k-based strategy but reuses the same correctness-filtered concise dataset constructed in the first round.

As shown in Table R1, fixed-data SFT yielded little additional compression after the initial round. Both TokenDistill variants continued to shorten reasoning traces across rounds. These results suggest that iterative data reconstruction helps sustain compression across rounds.

W2 / Q2 / Q3: Stopping criterion and checkpoint selection

Thank you for highlighting this issue. The length-based stopping criterion was intended to guide the exploration of further compression across iterations, rather than to select the checkpoint with the best accuracy–length trade-off.

Our original aim was to examine the best observed trade-offs across training rounds. Accordingly, the checkpoints reported in Table 1 of the submitted paper were selected retrospectively based on test-set accuracy and ATL. For TokenSkip, we also reported results across all evaluated compression ratios to illustrate its accuracy–compression trade-offs.

We used no separate validation set for stopping or checkpoint selection and acknowledge the risk of selection bias. We plan to re-evaluate with predefined criteria on an independent validation set, reserving the test set for final evaluation. Results are not yet available. In the revision, we will document the stopping and checkpoint-selection procedures and report the selected round for each setting.

As supplementary evidence, we compare TokenDistill’s cross-dataset transfer with TokenSkip’s in-domain performance. TokenDistill is trained and its checkpoint selected only on GSM8K; the same fixed checkpoint is then evaluated on MATH-500 and CSQA. In contrast, the TokenSkip baselines are trained separately on the corresponding MATH and CSQA training sets and evaluated on MATH-500 and CSQA, respectively. Their results and selected compression ratios are taken from Table 1 of the submitted paper.

Table R2. Comparison of GSM8K-trained TokenDistill with in-domain TokenSkip

Target datasetModelTokenSkip Acc. (%) ↑TokenSkip ATL ↓TokenDistill Acc. (%) ↑TokenDistill ATL ↓
MATH-500Qwen2.5-3B60.40501.7761.80420.55
MATH-500Qwen2.5-7B71.40496.1772.00398.28
CSQAQwen2.5-3B74.10129.7775.51105.98
CSQAQwen2.5-7B81.70134.7981.41107.27

Compared with in-domain TokenSkip, the fixed GSM8K-trained TokenDistill checkpoints produced 16.2–20.4% shorter outputs in all four settings and achieved higher accuracy in three. The only exception was Qwen2.5-7B on CSQA, where accuracy was 0.29 percentage points lower but outputs were 20.4% shorter. These results suggest that a TokenDistill model trained on a single source dataset can achieve competitive accuracy–compression trade-offs on target tasks using the checkpoint selected on that same source dataset, without further training or checkpoint selection.

Reviewer_KyK7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TokenDistill, an iterative self-distillation framework for compressing CoT reasoning. The method generates concise and correct reasoning traces using an additional prompt instruction, then fine-tunes the model on these traces. Repeating this process progressively reduces output length. Experiments across several language models and reasoning benchmarks demonstrate substantial CoT compression with generally competitive accuracy.

Critical strengths
  1. The method is simple to implement and does not require external teacher models or reinforcement learning.

  2. The experiments cover multiple model families and reasoning tasks, showing promising accuracy–length trade-offs.

Critical weaknesses
  1. Section 3.4 determines whether to continue training solely based on whether the average CoT length decreases on the test set. However, the reported checkpoints do not always correspond to the stopping criterion. For example, Table 1 reports round 2 for Qwen2.5-3B on GSM8K, whereas Appendix Table 4 shows that later rounds achieve further compression, albeit with substantial accuracy degradation. It is unclear how the final checkpoints are selected and whether accuracy is implicitly considered. Could you clarify the checkpoint-selection procedure and whether you use a separate validation set for stopping and model selection.

  2. The proposed method primarily combines several components including concise prompting, fine-tuning, and iterative data regeneration. While effective, these components are relatively conventional, making the methodological novelty limited. More importantly, the experiments do not sufficiently disentangle the benefits of iterative data regeneration from those of additional supervised training on correct reasoning traces. Comparisons with standard correct-CoT SFT and repeated SFT using fixed concise data under matched training budgets would help establish the necessity of the proposed iterative mechanism. Furthermore, the multiple rounds of response generation, evaluation, and fine-tuning introduce additional computational costs, which should be quantified when discussing training efficiency.

Questions for authors

See Weaknesses.

3 comments

W1: Stopping criterion and checkpoint selection

Thank you for highlighting this issue. The length-based stopping criterion in Eq. (12) was intended to guide the exploration of further compression across iterations, rather than to select the checkpoint with the best accuracy–length trade-off.

Our original aim was to examine the best observed trade-offs across training rounds. Accordingly, the TokenDistill checkpoints reported in Table 1 of the submitted paper were selected retrospectively based on both test-set accuracy and average token length (ATL). This is why an earlier checkpoint could be reported even when subsequent rounds produced shorter outputs. For TokenSkip, we reported results across all evaluated compression ratios to illustrate its accuracy–compression trade-offs.

We used no separate validation set for stopping or checkpoint selection and acknowledge the risk of selection bias. In the revision, we will clarify how the original checkpoints were chosen and report the selected round for each setting. We also plan to re-evaluate using predefined stopping and selection criteria on an independent validation set, reserving the test set for final evaluation. Results under this protocol are not yet available.

As supplementary evidence, Table R1 compares TokenDistill’s cross-dataset transfer with TokenSkip’s in-domain performance. For each backbone, we trained TokenDistill and selected its checkpoint on GSM8K, then evaluated that same checkpoint directly on MATH-500 and CSQA without further training or selection. In contrast, the TokenSkip baselines were trained separately on the corresponding MATH and CSQA training sets. Their results and selected compression ratios are taken from Table 1 of the submitted paper.

Table R1. Comparison of GSM8K-trained TokenDistill with in-domain TokenSkip.

Target datasetBackboneTokenSkip ratioTokenSkip Acc. (%) ↑TokenSkip ATL ↓TokenDistill Acc. (%) ↑TokenDistill ATL ↓
MATH-500Qwen2.5-3B0.960.40501.7761.80420.55
MATH-500Qwen2.5-7B0.971.40496.1772.00398.28
CSQAQwen2.5-3B0.574.10129.7775.51105.98
CSQAQwen2.5-7B0.581.70134.7981.41107.27

Compared with in-domain TokenSkip, the fixed GSM8K-trained TokenDistill checkpoints produced 16.2–20.4% shorter outputs in all four settings and achieved higher accuracy in three. The only exception was Qwen2.5-7B on CSQA, where accuracy was 0.29 percentage points lower but outputs were 20.4% shorter. These results suggest that competitive accuracy–compression trade-offs can transfer without target-specific checkpoint selection.

W2(a): Contribution of iterative data reconstruction

We agree that concise prompting and supervised fine-tuning are established techniques. The design choice we investigate is how the training data changes across rounds. In each TokenDistill round, the current model generates new concise responses. We retain responses with correct answers and use them as training targets under the standard prompt, without the conciseness instruction.

To examine what iterative data reconstruction adds, we evaluated fixed-data repeated SFT on Qwen2.5-7B/MATH: all five rounds reuse the same 3,845 correctness-filtered concise examples generated by the original model for the first round, with each round continuing from the preceding model. Table R2 compares this baseline with the two TokenDistill strategies described in Sections 3.3 and 4.5. All three approaches are evaluated on MATH-500.

The MkM_k-based strategy provides the closest comparison: like fixed-data SFT, it continues from the previous-round model, but it reconstructs its training data each round. The M0M_0-based strategy also reconstructs the data but initializes each round from the original model.

Table R2. Fixed-data repeated SFT and iterative data reconstruction on MATH-500.

StageFixed-data Acc. (%)Fixed-data ATLMkM_k-based Acc. (%)MkM_k-based ATLM0M_0-based Acc. (%)M0M_0-based ATL
r0r_0 (Original)71.2574.6571.2574.6571.2574.65
r1r_172.0523.1072.0523.1072.0523.10
r2r_272.6520.7370.4470.9571.6463.10
r3r_373.4516.9068.8415.4370.4422.21
r4r_474.4514.9368.6362.7067.8396.74
r5r_572.8509.4969.0338.5570.8375.32

As shown in Table R2, fixed-data SFT yielded little additional compression after the initial round: its ATL remained above 500 tokens through r5r_5. By contrast, ATL fell in every subsequent round for both reconstruction strategies, reaching 338.55 and 375.32 tokens, respectively. These results suggest that iterative data reconstruction helps sustain compression across rounds.

W2(b): Quantifying the computational cost of iteration

Thank you for raising this point. We agree that a discussion of TokenDistill’s computational efficiency should account for the additional cost of response generation, evaluation, and fine-tuning across rounds.

We therefore profiled five rounds of TokenDistill with Qwen2.5-7B on CSQA. Table R3 reports the time spent generating training-set responses with and without the conciseness instruction, fine-tuning the model, and evaluating each checkpoint under the standard prompt. The “Standard-prompt evaluation” column reports evaluation on the held-out CSQA split without the conciseness instruction: r0r_0 is the original-model baseline, and r1r_1–r5r_5 correspond to the post-training evaluation in Section 3.4.

Table R3. Five-round runtime breakdown for Qwen2.5-7B on CSQA (single-RTX-3090-equivalent minutes).

RoundData construction (concise generation)Pre-training evaluation (standard generation)Training (SFT)Standard-prompt evaluationTotal
r0r_0 (Original)———37.837.8
r1r_1156.8274.0164.021.0615.8
r2r_272.9152.3118.59.6353.3
r3r_352.581.9123.57.9265.8
r4r_451.561.3124.67.4244.8
r5r_555.858.1118.46.9239.2
Five rounds (r1r_1–r5r_5)389.5627.6649.052.81,718.9
Total including baseline389.5627.6649.090.61,756.7

Including the one-time baseline evaluation, the five-round run took the equivalent of 29.28 hours on a single RTX 3090. We will report this computational cost in the revision.

Reviewer_KHhmOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces TokenDistill, an iterative self-distillation framework for compressing chain-of-thought (CoT) reasoning traces in large language models. The method uses concise-output prompting to generate shorter reasoning traces, then fine-tunes the model on these outputs without the original concise instruction. Repeating this process transfers concise reasoning behavior into the model parameters and progressively reduces CoT length. Experiments on three reasoning benchmarks show competitive compression compared with prompting and SFT baselines, with limited accuracy degradation.

Critical strengths
  1. A simple self-distillation framework without an external teacher. TokenDistill uses the model's own concise outputs as training targets, making the method straightforward to implement without relying on a separate teacher model.

  2. Substantial CoT compression with largely preserved accuracy. The reported results show that reasoning traces can be shortened considerably while largely maintaining accuracy on the evaluated benchmarks. This is useful for reducing the token cost of reasoning.

Critical weaknesses
  1. The experimental coverage is insufficient to establish the method's comparative value and generalization. The main experiments omit relevant approaches such as the RL-based method ExpThink and the on-policy self-distillation method CRISP. The analysis also lacks results on the Qwen3 family. In addition, similar average-token-length versus accuracy trajectories across related datasets establish only consistent compression behavior in those settings. They do not sufficiently demonstrate that TokenDistill learns a compression strategy that transfers across diverse domains. These gaps limit the evidence for its advantages over existing compression methods.

  2. The methodological novelty is limited. The approach mainly combines concise-output prompting, self-distillation, and iterative SFT. The paper does not clearly establish what methodological contribution goes beyond this combination. This limits the contribution independently of the reported compression results.

Questions for authors
  1. Can the authors provide comparisons with ExpThink and CRISP, or explain which differences in their training settings prevent a direct comparison? Results on Qwen3 models would also help establish whether the conclusions extend to that model family.

  2. Can the authors evaluate a model compressed on one dataset on other datasets or domains, without further compression training on the target datasets, and compare this transfer with other compression methods? Reporting both accuracy and average token length would help determine whether the learned compression strategy generalizes beyond related tasks.

2 comments

W1/Q1: Qwen3-8B and comparisons with CRISP and ExpThink

Thank you for suggesting these evaluations. Accordingly, we trained Qwen3-8B with TokenDistill for three rounds on the MATH training split. We compared the round-3 checkpoint with the authors’ publicly released CRISP Qwen3-8B checkpoint on MATH-500, GSM8K, and CSQA.

Table R1. Comparison with CRISP on Qwen3-8B

DatasetMethodAcc. (%) ↑ATL ↓
MATH-500CRISP90.002,740.89
MATH-500TokenDistill91.802,068.23
GSM8KCRISP95.53872.06
GSM8KTokenDistill95.45465.16
CSQACRISP83.78778.50
CSQATokenDistill81.98535.99

Compared with CRISP, TokenDistill produces substantially shorter reasoning traces at comparable accuracy across all three benchmarks.

Notably, we used the round-3 checkpoint for the CRISP comparison because the available time and compute allowed us to complete three rounds. The table below compares Qwen3-8B’s accuracy and ATL on MATH-500 at baseline and after each of the three TokenDistill rounds.

Table R2. Qwen3-8B performance on MATH-500 across TokenDistill rounds

Qwen3-8B checkpointAcc. (%) ↑ATL ↓
r0r_0 (Original)81.004,160.88
r1r_185.203,431.19
r2r_289.402,700.85
r3r_391.802,068.23

Accuracy increased and ATL decreased in each completed round, suggesting that additional rounds could further improve the accuracy–length trade-off.

As for ExpThink, its RL training samples a group of full rollouts per prompt at every step, costing far more compute than our SFT pipeline. With a different evaluation protocol and no public checkpoint for that Qwen3-8B run, a matched comparison was not feasible within the rebuttal period. We will include one in the final version.

W1/Q2: Cross-dataset generalization

Thank you for suggesting this evaluation. We evaluated GSM8K-trained TokenDistill checkpoints directly on MATH-500 and CSQA, without any additional training on either target dataset, and compared their transfer performance with that of GSM8K-trained TokenSkip checkpoints. CCoT is included as a reference baseline.

We use the same TokenDistill rounds and TokenSkip compression ratios selected for GSM8K in Table 1 of the submitted paper, keeping them fixed across both target datasets.

Table R3. Comparison of cross-dataset generalization across CoT compression methods

DatasetMethodQwen2.5-3B Acc. (%) ↑Qwen2.5-3B ATL ↓Qwen2.5-7B Acc. (%) ↑Qwen2.5-7B ATL ↓
GSM8KOriginal83.60316.2991.50298.58
GSM8KCCoT83.00252.9891.50251.89
GSM8KTokenSkip80.40233.0387.90178.25
GSM8KTokenDistill82.03145.6890.60169.41
MATH-500Original63.40577.9171.20574.65
MATH-500CCoT66.20524.3370.20533.50
MATH-500TokenSkip59.80568.5368.80478.84
MATH-500TokenDistill61.80420.5572.00398.28
CSQAOriginal73.30253.1079.90265.40
CSQACCoT73.10197.5080.90194.30
CSQATokenSkip70.52216.9079.20158.30
CSQATokenDistill75.51105.9881.41107.27

Relative to the original models, accuracy improves in three of the four transfer settings and drops by 1.6 percentage points for Qwen2.5-3B on MATH-500. TokenDistill produces shorter outputs than TokenSkip and CCoT in all four settings, with higher accuracy in all four comparisons with TokenSkip and in three with CCoT.

Table R1 extends the transfer evaluation to Qwen3-8B. Together, these results provide direct evidence that the learned compression behavior can transfer beyond the training dataset. They therefore support cross-task transfer without further compression training, rather than merely consistent compression under separate task-specific training.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.