acceptodds
Under review as a conference paper at ICLR 2027

TERSE: Trace-Efficiency Reward via Squeezing rEdundancy for Efficient Large Reasoning Models

Abstract

Large reasoning models perform well on complex problems by generating long chains of thought, but these traces are often redundant, raising inference cost and latency without improving accuracy. Reinforcement learning methods curb this by penalizing length, which captures how much a model writes but not how much of it merely repeats: two correct traces of equal length can carry very different amounts of redundancy. Prior methods address this by adding a second signal, derived from external information such as a reflection lexicon, an auxiliary model, or the model's internals. We instead read a second signal from the trace itself, the gzip compressibility of its raw bytes, which depends on bytes rather than meaning and is therefore not tied to a domain or model family. Building on it, we propose Trace-Efficiency Reward via Squeezing rEdundancy (TERSE), a reward for group-relative reinforcement learning. TERSE residualizes compressibility against length so that it rewards only the redundancy length does not already explain, and standardizes both signals within difficulty buckets to remain stable on hard prompts where correct rollouts are scarce. On five mathematical reasoning benchmarks with DeepSeek-R1-Distill-Qwen at 1.5B and 7B, improves accuracy over the base model by 7.8 and 3.4 points while cutting response length by 54.4% and 37.4%, improving the accuracy–length trade-off at both scales.

Primary area:
reinforcement learning
Keywords:
efficient reasoning, large reasoning models, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_mDsVOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TERSE, a GRPO-style RLVR reward for efficient LLM reasoning. The main insight is that length is not the only metric for measuring reasoning conciseness. Reasoning trajectories of the same length may have different degrees of redundancy. Its main contribution is using the gzip compressibility of a trace's raw bytes as a redundancy signal that needs no auxiliary model, lexicon, reference answer, or extra inference. The experiments show that TERSE raises average accuracy across five math benchmarks by 7.8 points while cutting response length by 54.4% at 1.5B, and by 3.4 points with a 37.4% cut at 7B.

Critical strengths
  1. The method is simple and cheap. There is no auxiliary model and no extra forward pass; the overhead is negligible, and it is easy to reproduce and plug into existing pipelines.
  2. Well-motivated design. Residualization and difficulty bucketing each address a specific failure mode, and the ablations confirm both.
Critical weaknesses
  1. Baselines were copied, not rerun. All baseline numbers come from the ARLCP paper rather than being retrained under TERSE's own setup, which weakens the comparison.
  2. gzip cannot catch semantic repetition: it can only recognize literal repetition. If the same meaning is restated in different words, it is very likely unable to detect it, and this limitation is not discussed much in the text. This may lead to reward hacking in the model: the model's reasoning redundancy is not reduced; it merely repeats the same information using different expressions.
Questions for authors
  1. Can the baseline be reproduced and tested again? At the very least, the open-source models from these papers should be tested under the same experimental environment and configuration. Deviations in data across different papers caused by differences in environments, verifiers, etc., may be larger than the improvement margin of the method.
  2. Although the motivation of the method emphasizes redundancy, the experiments only test the model’s accuracy and reasoning length. Could redundancy evaluation metrics be added? In particular, testing semantic redundancy would help address the second point in the weaknesses.
3 comments
AuthorsOpenReview ↗

Response to Reviewer mDsV: Q1

Q1:Can the baseline be reproduced and tested again?

The baseline numbers in Table 2 are taken from ARLCP. To check that the comparison does not depend on their reported values, we have evaluated the public checkpoints of Vanilla, NoThinking, TLMRE, AdaptThink, and LASER under our own pipeline, and trained ARLCP from scratch in our environment. The results agree with the reported values for every method we could rerun (see table below). The paper reports one seed. We have since run a second seed at both scales.

MethodSourceGSM8KMATH-500AMC23AIME24AIME25
VanillaARLCP78.5 / 100980.2 / 458266.7 / 774230.0 / 1225621.4 / 12201
Vanillaours77.0 / 105482.0 / 452467.8 / 784026.2 / 1261022.1 / 12310
NoThinkingARLCP70.0 / 27869.2 / 90849.2 / 137714.4 / 42669.8 / 3630
NoThinkingours72.6 / 29869.0 / 94448.4 / 182912.5 / 439210.0 / 3221
TLMREARLCP84.3 / 53082.1 / 180072.1 / 279825.8 / 538619.6 / 4581
TLMREours79.8 / 37982.0 / 247274.1 / 442826.5 / 904521.1 / 8576
AdaptThinkARLCP84.2 / 48883.2 / 186967.2 / 334230.8 / 686422.5 / 6626
AdaptThinkours84.0 / 50982.0 / 176567.4 / 332331.2 / 685523.5 / 6491
LASERARLCP82.3 / 85084.6 / 236575.9 / 389928.8 / 762925.4 / 7084
LASERours83.1 / 89185.2 / 237074.5 / 394429.4 / 741521.2 / 7087
ARLCPARLCP87.3 / 65884.6 / 181273.3 / 303734.2 / 595126.5 / 5196
ARLCPours85.5 / 68384.8 / 202074.2 / 337536.2 / 725624.6 / 6627
TERSEseed086.8 / 64485.6 / 167580.9 / 277536.9 / 592925.6 / 5305
TERSEseed186.6 / 62784.9 / 170980.0 / 276135.7 / 599226.7 / 5283
MethodSourceGSM8KMATH-500AMC23AIME24AIME25
VanillaARLCP87.3 / 69491.2 / 367387.5 / 598051.5 / 1054735.0 / 11427
Vanillaours87.9 / 65791.2 / 375588.3 / 573552.9 / 1014637.3 / 11212
NoThinkingARLCP84.7 / 28480.6 / 70661.3 / 117924.0 / 267116.3 / 2499
NoThinkingours83.0 / 28680.4 / 73157.5 / 136020.4 / 344916.7 / 2979
TLMREARLCP88.8 / 86691.2 / 241988.1 / 356052.1 / 639335.6 / 6584
TLMREours89.6 / 47990.2 / 245187.8 / 448152.9 / 887936.5 / 9484
AdaptThinkARLCP90.3 / 33091.0 / 186286.9 / 364156.3 / 878437.7 / 9622
AdaptThinkours90.6 / 33190.4 / 180786.4 / 382053.8 / 868238.8 / 9350
LASERARLCP86.7 / 81891.0 / 187189.5 / 289456.5 / 599539.4 / 6410
LASERours87.8 / 80091.6 / 189988.6 / 297556.5 / 610739.2 / 6394
ARLCPARLCP89.3 / 57891.0 / 208689.7 / 326056.7 / 679539.6 / 7560
ARLCPours88.3 / 61190.8 / 239088.4 / 372052.1 / 791737.1 / 8270
TERSEseed091.4 / 57592.6 / 194591.6 / 311054.4 / 647839.6 / 7297
TERSEseed191.2 / 61892.8 / 186292.5 / 303554.0 / 639240.6 / 7283
AuthorsOpenReview ↗

Response to Reviewer mDsV: Weakness2 and Q2

Weakness2: gzip cannot catch semantic repetition: it can only recognize literal repetition. [...]. the model's reasoning redundancy is not reduced; it merely repeats the same information using different expressions.

The reward is not compressibility alone. It spends a fixed budget on two terms, a length penalty and a residual-compressibility penalty, and on any prompt that the model solves more often than not, the length term is the larger of the two. A step that is reworded rather than removed still occupies tokens, so it still pays that larger penalty and avoids only the smaller one, while removing it avoids both. The reward therefore does not favour rewording, and rewording cannot account for the length reductions we report, since restating content in different words leaves the trace about as long as before.

Q2: Although the motivation of the method emphasizes redundancy, the experiments only test the model’s accuracy and reasoning length. Could redundancy evaluation metrics be added? In particular, testing semantic redundancy would help address the second point in the weaknesses.

Two things are worth separating. What the method sets out to remove is content that does not help reach the answer. What the signal is built to detect is one pattern of that content, where the trace goes back over material it has already produced. That pattern is readable from the bytes, and it is the purest case of waste, since a detour at least works on the problem while restating a settled step does not.

The evaluation should measure the first of those, and accuracy and length do measure it when they are read together. Cutting tokens alone shows nothing, as NoThinking cuts 81.0% of them and loses 12.8 points of accuracy, which removes reasoning rather than waste. What is informative is a large reduction with no accuracy loss on any benchmark. At 1.5B we remove 54.4% of the tokens while accuracy rises by 7.8 points and stays at or above the base model on all five benchmarks, and no other method in Table 2 does both. The tokens we removed were not contributing to the answers.

Thank you for your response. The new experiments increase the credibility of the paper. However, the use of gzip is more like a heuristic approach, and its usefulness is only demonstrated by outcome metrics such as accuracy and length, lacking a more detailed experimental analysis of it. I will raise the score to 2

Reviewer_vxprOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TERSE uses gzip compressibility alongside response length to reward efficient reasoning. It residualizes compressibility against length, maintains difficulty-bucketed statistics, and allocates penalty weights according to online success rates. Experiments cover three models, five mathematical benchmarks, and MMLU.

Critical strengths

1.The signal requires no auxiliary model, model internals, or hand-crafted reflection lexicon, making the approach simple and practical.

2.The reported results demonstrate competitive accuracy–length trade-offs across multiple models and benchmarks.

Critical weaknesses
  1. The methodological novelty appears limited. LZ-Penalty has already explored compression-based signals for suppressing repetition, while ARLCP combines reflection-token and length penalties under a shared budget. TERSE introduces a distinct design by using gzip compressibility in an RL reward, together with residualization and difficulty-bucketed statistics. Nevertheless, the contribution appears primarily incremental relative to these existing approaches.

  2. Compressibility is not sufficiently validated as a proxy for unnecessary reasoning. Gzip captures byte-level regularity, which may reflect necessary mathematical notation or repeated phrasing rather than redundant reasoning. Models could also reduce compressibility by rephrasing redundant content. The paper lacks direct evidence that the signal reliably identifies unnecessary reasoning after controlling for length.

  3. The specific advantage of gzip remains unclear. The 4-gram variant achieves reasonably close results, while LZMA produces shorter responses with slightly lower accuracy. These findings support the broader framework, but leave unclear what gzip specifically contributes. Uncertainty estimates and comparisons at matched lengths would help clarify this point.

Questions for authors
  1. Can the authors directly validate the relationship between compressibility and unnecessary reasoning, using correct, approximately equal-length responses to the same problems?

  2. What advantages does gzip offer over 4-gram repetition and LZMA in accuracy–length trade-offs, computational cost, or training stability?

  3. Does residualization remain beneficial when the penalty coefficients are adjusted to match response lengths?

2 comments
AuthorsOpenReview ↗

Response to Reviewer vxpr: Q1, Q2, Q3

Q1: Can the authors directly validate the relationship between compressibility and unnecessary reasoning, using correct, approximately equal-length responses to the same problems?

Yes, and this is the test we ran. Claude Sonnet 4.5 read each correct trace and rated it from 1 to 5 for unproductive repetition and hesitation on 5 test datasets in experiment. We then took pairs of correct traces of the same problem whose generated-token lengths differ by at most 5%, which across these pairs works out to about 2.5% on average, and asked whether the trace with the higher rating is also the more compressible one.

Same problem, correct, length matched, 95% CIDS-1.5BDS-7B
Compressibility picks the more unnecessary trace73.8% [67.9, 79.5]72.3% [66.5, 78.0]
Length picks it instead, on the same pairs58.9% [52.5, 64.9]57.4% [52.2, 62.4]

Both rows use the same pairs, restricted to those whose ratings differ by at least one levels, and random ordering would give 50%. Length still carries some signal inside a 5% window, but compressibility is right about 15 points more often at both scales, with intervals that do not overlap.

Q2: What advantages does gzip offer over 4-gram repetition and LZMA in accuracy–length trade-offs, computational cost, or training stability?

We do not claim an advantage for gzip over the other two. The paper states the reason we use it, which is ubiquity and simplicity rather than superiority, and reports the three estimators behaving alike to argue that what the reward needs is a content-agnostic measure of redundancy rather than one specific estimator. The three differ only in the redundancy measure, with the budget unchanged, so the accuracy and length each one reaches is a consequence of its signal, and the ordering in Table 3 of the paper is already a comparison of the signals themselves.

What does differ is how each one finds repeated content. A 4-gram repetition rate counts sequences of four whitespace tokens that recur exactly, so it misses overlaps shorter than four tokens, a step restated with one number changed, or the same fragments reordered, and it requires a tokenization and a window size to be chosen. gzip encodes a repeated substring of any length from three bytes upward as a reference to its earlier occurrence, so those cases still register, and LZMA is a stronger compressor of the same family. On this reading the two compressors should behave alike and the 4-gram rate should be the weakest of the three.

Q3: Does residualization remain beneficial when the penalty coefficients are adjusted to match response lengths?

The intervention in that ablation is switching residualization on, and the response length it produces is one of its consequences rather than a nuisance to be removed. At a fixed budget, which is how Table 3 of the paper is run, removing residualization costs 1.0 accuracy point and compresses 4.5 points further, +6.8 at −58.9% against +7.8 at −54.4%. Treating the length difference as something to equalize presumes length is a setting, whereas under RL it is an outcome of the reward. Matching lengths requires changing the budget as well, so that comparison differs from the ablation in two ways at once, and what it measures is the composite of residualization and a retuned coefficient. A difference found there could not be attributed to either one, and the absence of a difference would not clear residualization. The two controls are also not interchangeable. The coefficient rescales length pressure uniformly across difficulty, while residualization changes how that pressure is distributed over difficulty. Retuning the coefficient to recover a length therefore cannot restore the profile that removing residualization took away.

AuthorsOpenReview ↗

Response to Reviewer vxpr: methodological novelty

Methodological novelty with LZ-Penalty and ARLCP

On novelty, we would separate the two references, because they act at different levels of the system.

LZ-Penalty uses compression at decoding time. It derives a code-length term from the logits and suppresses repetition as each token is produced, leaving the parameters unchanged and requiring the penalty at every inference. That leaves open the question we are concerned with, which is whether compressibility works as a training signal. The two settings are not variants of one implementation. A decoding penalty shapes the distribution over the next token at a single step and never ranks one complete trace against another, while a reward does exactly that, and it is also exposed to being gamed by the policy it trains.

ARLCP's redundancy signal is a hand-written list of reflection keywords, and its difficulty levels are thresholds on the count of those same keywords, so both parts of its mechanism depend on finding particular words in the text. Such a list has to be assembled for the vocabulary a model actually produces, and it has to be maintained. Our redundancy signal is computed from the bytes of the trace, and our difficulty estimate is the pass rate of the rollout group, so neither one examines which words appear.

What we claim is the class of signal, together with the two components that make such a signal usable as a reward. The claim is not about gzip in particular, which is why the paper also reports an LZMA variant and a 4-gram variant. Residualization is necessary rather than cosmetic. Compressibility rises with length, so a large part of the raw score only reflects how long a trace is, and without residualization the redundancy term charges for length a second time and the reward drifts toward a uniform length penalty. In our ablation, that version gives shorter responses at lower accuracy. The difficulty buckets are what allow the signal to be standardized on hard prompts, where correct rollouts are rare.

Reviewer_FexYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TERSE introduces gzip compressibility as a low-cost signal for reasoning redundancy, combined with residualization against length and difficulty-dependent reward allocation. The approach is simple and practical, and the results across model sizes, together with component ablations, are encouraging.

Critical strengths

1.Simple and practical. TERSE computes compressibility directly from generated text, without an auxiliary model, access to model internals, or additional model forward passes. This makes the approach straightforward to integrate into existing reasoning RL pipelines. 2. Thoughtful treatment of complementary efficiency signals. The method addresses relevant challenges in combining length and repetition penalties: their statistical overlap, the need to preserve reasoning capacity on difficult problems, and the scarcity of correct rollouts for estimating reward statistics.

Critical weaknesses
  1. The motivation for using compressibility to measure reasoning redundancy remains indirect. Compression algorithms operate on byte patterns, whereas the intended target is unnecessary reasoning at the semantic level. Repeated notation may be essential to a valid derivation, while redundant reasoning expressed through paraphrases may remain relatively incompressible. The paper does not sufficiently establish this connection.
  2. The experimental validation does not adequately support the broader framing around group-relative reinforcement learning. All reported experiments use RLOO, with no evaluation of TERSE or representative efficiency baselines under GRPO. Although RLOO is a valid instantiation of the proposed framework, the absence of GRPO experiments leaves its applicability across optimization algorithms unverified. The cited instability of GRPO under length penalties further motivates a direct investigation of whether TERSE’s benefits carry over to this setting.
  3. When clipping is inactive, substituting the residual into the reward gives an effective length penalty of wℓ−ρwew_\ell-\rho w_e, which can become negative on difficult prompts. Consequently, among correct responses to the same prompt with the same compressibility, a longer response can receive a higher reward. Since this behavior is not discussed in the paper, could the authors clarify whether it is intended and explain the rationale behind it?
Questions for authors

As shown in weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer FexY: Weakness1,2

Weakness1: The motivation for using compressibility to measure reasoning redundancy remains indirect. [...] The paper does not sufficiently establish this connection.

Our target is not semantic-level judgment. The paper defines redundancy as a trace going back over content it has already produced, re-deriving a settled step or re-checking an answer already reached, and that pattern leaves repeated bytes. gzip never reads meaning, and we do not claim it does. We establish the connection empirically. Claude Sonnet 4.5 read each correct trace and rated it for repetition severity, from 1 for no unproductive repetition to 5 for severe repeated re-checking or looping. Taking traces of the same problem whose generated-token lengths differ by at most 5% and whose ratings differ by at least two levels, the trace with the higher rating also has the higher residual-gzip score in 73.8% of pairs at 1.5B and 73.1% at 7B, against 50% for random ordering. This design speaks to both cases you raise. Because both traces solve the same problem, the notation a valid derivation requires is held fixed, and what remains is how much more one trace repeats than the other. And because the judge reads the reasoning rather than the bytes, it marks reworded repetition that gzip would miss. If that were the dominant case, the agreement would fall toward chance.

Weakness2: The experimental validation does not adequately support the broader framing around group-relative reinforcement learning. [...] The cited instability of GRPO under length penalties further motivates a direct investigation of whether TERSE's benefits carry over to this setting.

We trained TERSE under GRPO, changing only the advantage estimator. The compression carries over and the accuracy gain does not. At 7B, accuracy is below the RLOO run on all five benchmarks late in training, by about 4 points on average, while GSM8K responses fall from 687 to 404 tokens. We also observed the same pattern at 1.5B. This matches a known property of std-normalized advantages rather than anything specific to our signal. [2] shows that when every rollout for a prompt is correct, normalizing the advantage by the group standard deviation makes it independent of the penalty coefficient, so the length pressure acts at full strength however small that coefficient is set, and reports the resulting sharp drop in accuracy during training. The same cancellation applies to our reward, and since the compressibility weight is zero at pass rate 1, what survives on those groups is the length term alone. This is also why the effect concentrates on easy prompts, which are the ones that routinely produce all-correct groups, while the hard benchmarks are not over-compressed. [1] reports the same failure for its own reward, losing 2.71 points of accuracy at 65.2% shorter length under GRPO against a gain of 5.81 at 53.1% under RLOO, and uses RLOO for this reason. The fix is to not divide by the group standard deviation, which is what [2] does and why RLOO is unaffected. Making the standard GRPO estimator work with an efficiency term would require the shaping to survive the normalization, for instance by applying it to the advantage rather than to the reward, which we leave to future work.

[1] Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty. ICLR 2026.

[2] Training Language Models to Reason Efficiently. NeurIPS, 2025.

AuthorsOpenReview ↗

Response to Reviewer FexY: Weakness3

Weakness3: When clipping is inactive, substituting the residual into the reward gives an effective length penalty of (wℓ−ρwe)(w_\ell - \rho w_e), [...]. Since this behavior is not discussed in the paper, could the authors clarify whether it is intended and explain the rationale behind it?

This is intended. The substitution is as stated and the coefficient does turn negative, at pass rates below ρ/(1+ρ)\rho/(1+\rho). We residualize so that the compressibility term scores redundancy relative to length instead of repeating the length penalty, and the budget moves toward that term as problems get harder, because a longer derivation is often necessary there and length becomes the weaker signal. The sign turns over only where that shift has already taken hold. Wherever the effective penalty is negative, the weight on redundancy is at least 1/ρ1/\rho times the weight on length, which is about two to one at our measured slope and more lopsided the harder the prompt. Equal compressibility is then not a neutral case. Compressibility rises with length in the bucket's typical relation, so when two correct traces have the same compressibility, the longer one is less compressible than its length predicts, and the redundancy term credits it accordingly. The remaining length charge does not override that, and where the budget is weighted that way we would rather let the redundancy comparison settle the ranking. We will add the substituted form and the effective coefficient to the paper.

Reviewer_MDShOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TERSE: gzip compressibility as a second signal for redundancy in reasoning traces. Compressibility is residualized against length; a fixed reward budget is then moved from the length term to the residual as problem difficulty increases, with EMA standardization inside difficulty buckets. The method is claimed not to need a lexicon, an auxiliary model, or model internals. On 1.5B/7B, the paper reports +7.8/+3.4 accuracy and −54.4%/−37.4% length versus the base models.

Critical strengths

• The problem matters, and the method is cheap: gzip is enough; there is no extra model to train. • The overlap between length and compressibility is treated explicitly, and the budget is reallocated by difficulty; the motivation is clear. • Two scales and five math benchmarks, plus Qwen3-1.7B and MMLU. LZMA/4-gram results suggest the effect is not tied to one gzip implementation.

Critical weaknesses

• Gzip is not shown to measure “harmful reasoning redundancy.” It scores repeated bytes, not semantic repetition. Formulas, templates, and paraphrases can move the score. Figure 1 compares traces from two different prompts, mixing content and difficulty. Higher accuracy and shorter length only show that the reward can be used, not that it measures the claimed construct. In Table 3, 4-gram is almost as good, which looks more like surface repetition doing the work. • The reward has an untested exploit. A correct extracted answer counts as correct, and lower residual compressibility is better. On hard problems the length penalty is almost turned off, so the model can gain reward by appending high-entropy text, paraphrases, or even irrelevant tokens. There are no adversarial tests and no process-quality checks of high-reward traces. • The main table does not support “better than baselines” or “the best length trade-off.” Table 2 baselines are taken from Yu et al. (2026); TERSE is evaluated separately. There are no multiple seeds or intervals. The text says TERSE has the largest average length cut, but in the 1.5B table TLMRE (−58.1%) and NoThinking (−81.0%) are both shorter; the same paragraph’s 74.1% for NoThinking also does not match the table. Ablation variants have different lengths, so they compare different operating points.

Questions for authors

• After controlling for prompt and length, how strongly does residual gzip correlate with “unnecessary reasoning”? • Did you test irrelevant suffixes, paraphrases, or random strings? Why is the length penalty turned off on hard problems? • Which Table 2 baselines were rerun locally? How many seeds for TERSE? • At matched length or matched accuracy, what does gzip add over 4-gram? • Is the length penalty in tokens or bytes? Is compression applied to the think span or the full output?

3 comments
AuthorsOpenReview ↗

Response to Reviewer MDSh: Q1, Q2

Q1. After controlling for prompt and length, how strongly does residual gzip correlate with “unnecessary reasoning”?

We test the relation on correct rollouts from the DeepSeek-R1-Distill-Qwen-1.5B and -7B base models on 5 math evaluation testing datasets. For each problem, we sample 16 rollouts. Claude Sonnet 4.5 reads each reasoning trace and assigns a repetition-severity rating from 1, no unproductive repetition, to 5, severe repeated re-checking or looping. For each rollout, we compute a length-residualized gzip score (e). To quantify the association, we compare two correct rollouts from the same problem whose generated-token lengths differ by at most 5%. This controls for the prompt and closely matches response length. We consider pairs whose repetition-severity ratings differ by at least one levels, such as 1 versus 2 or 4 versus 5. In these pairs, the rollout with the higher repetition rating also has the higher residual-gzip score in 73.8% of cases at 1.5B and 73.1% at 7B. Random ordering would yield 50%. Thus, for similarly long solutions to the same problem, residual gzip tends to assign a higher score to traces with more repeated reasoning.

Q2: Did you test irrelevant suffixes, paraphrases, or random strings? Why is the length penalty turned off on hard problems?

We took 100 verifier-correct rollouts from the untrained DeepSeek-R1-Distill-Qwen-1.5B base model and inserted one suffix of about 1,024 tokens before </think>: random alphanumeric text, reasoning copied from a correct rollout of a different problem, or a paraphrase of the trace's own final reasoning that keeps its content and conclusion. Each edited response was re-checked with the same verifier and remained correct, so only the efficiency term moves.

SuffixΔ reward at pr = 1Δ reward at pr = 1/16
Random alphanumeric−0.070+0.225
Reasoning from a different problem−0.070+0.010
Paraphrase of the trace's own reasoning−0.067−0.057

Two of the three do not pay off anywhere in this range. A paraphrase lowers the reward because restating a derivation in new words still leaves fragments that gzip encodes against the text above it. Reasoning from another problem is close to neutral. Random alphanumeric text raises the reward on hard prompts, since it is almost incompressible.

We therefore also checked whether the trained model writes such text on its own. From the validation outputs of our 1.5B checkpoint we took the 488 highest-reward correct traces, selected within each difficulty level so that the hardest prompts are included, together with 488 correct traces drawn at random from the same pool. We removed the labels, shuffled the two sets together, and asked Claude Sonnet 4.5 to mark garbled filler and any text after the final answer. Neither set contained either. Of 20 traces with a random suffix inserted, 19 returned a parseable verdict and all 19 were flagged as garbled, so the judge does detect the pattern when it is present.

On the length penalty, it is reduced on hard prompts, not removed. Its weight is W⋅prW \cdot pr, where prpr is the fraction of correct rollouts for the prompt, and the residual-gzip weight is W(1−pr)W(1-pr). We sample 16 rollouts per prompt, so a prompt with a correct rollout has pr≥1/16pr \geq 1/16 and the length weight stays nonzero. We shift most of the budget on hard prompts because a longer derivation can be necessary there, so length alone cannot separate necessary work from reasoning that repeats steps already completed.

AuthorsOpenReview ↗

Response to Reviewer MDSh: Q3

Q3:Which Table 2 baselines were rerun locally? How many seeds for TERSE?

The baseline numbers in paper are taken from ARLCP. To check that the comparison does not depend on their reported values, we have evaluated the public checkpoints of Vanilla, NoThinking, TLMRE, AdaptThink, and LASER under our own pipeline, and trained ARLCP from scratch in our environment. The results agree with the reported values for every method we could rerun (see table below). The paper reports one seed. We have since run a second seed at both scales.

MethodSourceGSM8KMATH-500AMC23AIME24AIME25
VanillaARLCP78.5 / 100980.2 / 458266.7 / 774230.0 / 1225621.4 / 12201
Vanillaours77.0 / 105482.0 / 452467.8 / 784026.2 / 1261022.1 / 12310
NoThinkingARLCP70.0 / 27869.2 / 90849.2 / 137714.4 / 42669.8 / 3630
NoThinkingours72.6 / 29869.0 / 94448.4 / 182912.5 / 439210.0 / 3221
TLMREARLCP84.3 / 53082.1 / 180072.1 / 279825.8 / 538619.6 / 4581
TLMREours79.8 / 37982.0 / 247274.1 / 442826.5 / 904521.1 / 8576
AdaptThinkARLCP84.2 / 48883.2 / 186967.2 / 334230.8 / 686422.5 / 6626
AdaptThinkours84.0 / 50982.0 / 176567.4 / 332331.2 / 685523.5 / 6491
LASERARLCP82.3 / 85084.6 / 236575.9 / 389928.8 / 762925.4 / 7084
LASERours83.1 / 89185.2 / 237074.5 / 394429.4 / 741521.2 / 7087
ARLCPARLCP87.3 / 65884.6 / 181273.3 / 303734.2 / 595126.5 / 5196
ARLCPours85.5 / 68384.8 / 202074.2 / 337536.2 / 725624.6 / 6627
TERSEseed086.8 / 64485.6 / 167580.9 / 277536.9 / 592925.6 / 5305
TERSEseed186.6 / 62784.9 / 170980.0 / 276135.7 / 599226.7 / 5283
MethodSourceGSM8KMATH-500AMC23AIME24AIME25
VanillaARLCP87.3 / 69491.2 / 367387.5 / 598051.5 / 1054735.0 / 11427
Vanillaours87.9 / 65791.2 / 375588.3 / 573552.9 / 1014637.3 / 11212
NoThinkingARLCP84.7 / 28480.6 / 70661.3 / 117924.0 / 267116.3 / 2499
NoThinkingours83.0 / 28680.4 / 73157.5 / 136020.4 / 344916.7 / 2979
TLMREARLCP88.8 / 86691.2 / 241988.1 / 356052.1 / 639335.6 / 6584
TLMREours89.6 / 47990.2 / 245187.8 / 448152.9 / 887936.5 / 9484
AdaptThinkARLCP90.3 / 33091.0 / 186286.9 / 364156.3 / 878437.7 / 9622
AdaptThinkours90.6 / 33190.4 / 180786.4 / 382053.8 / 868238.8 / 9350
LASERARLCP86.7 / 81891.0 / 187189.5 / 289456.5 / 599539.4 / 6410
LASERours87.8 / 80091.6 / 189988.6 / 297556.5 / 610739.2 / 6394
ARLCPARLCP89.3 / 57891.0 / 208689.7 / 326056.7 / 679539.6 / 7560
ARLCPours88.3 / 61190.8 / 239088.4 / 372052.1 / 791737.1 / 8270
TERSEseed091.4 / 57592.6 / 194591.6 / 311054.4 / 647839.6 / 7297
TERSEseed191.2 / 61892.8 / 186292.5 / 303554.0 / 639240.6 / 7283
AuthorsOpenReview ↗

Response to Reviewer MDSh: Q4, Q5

Q4: At matched length or matched accuracy, what does gzip add over 4-gram?

We do not claim gzip adds an advantage over 4-gram. The paper states the reason we use it, which is ubiquity and simplicity rather than superiority, and reports gzip, LZMA and the 4-gram rate behaving alike to argue that what the reward needs is a content-agnostic measure of redundancy rather than one specific estimator. On matching, the variants in Table 3 of the paper differ only in the redundancy measure, with the budget unchanged, so the length each one reaches is a consequence of the signal rather than a nuisance to be removed. Matching the lengths would mean changing the budget for one of them, and that comparison would differ in two ways rather than one. What does differ between the two is how each finds repeated content. A 4-gram repetition rate counts sequences of four whitespace tokens that recur exactly, so it misses overlaps shorter than four tokens, a step restated with one number changed, or the same fragments reordered, and it requires a tokenization and a window size to be chosen. gzip encodes a repeated substring of any length from three bytes upward as a reference to its earlier occurrence, so those cases still register. On this reading the 4-gram rate would be the weaker of the two, which is the ordering in Table 3 of the paper.

Q5:Is the length penalty in tokens or bytes? Is compression applied to the think span or the full output?

The length penalty is in tokens. It uses the number of response tokens, which is also the length we report in the tables. Compressibility is computed on the full output, by gzip-compressing its UTF-8 bytes at level 5 and taking one minus the compression ratio. Both follow the official ARLCP implementation, where the length term is the response token count and the reflection-keyword count is taken over the full response, so our two signals are measured on the same text as theirs. The notation ℓ=∣o∣\ell = |o| in Eq. 4 of the paper is ambiguous on this point, and the surrounding text describes both quantities in bytes, which is wrong for the length term. We will state the units explicitly in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.