acceptodds
Under review as a conference paper at ICLR 2027

Same Score, Different State: Post-Training Moves Pre-Answer Recoverability under KV Compression

Abstract

Serving a reasoning model means shrinking its key-value (KV) cache while the model is still reasoning. Eviction and a token budget both act on the partial trace before the answer appears, and no benchmark grades that state. We measure it with an answer-marker-audited midpoint protocol: truncate a completed trajectory at 50% of its reasoning tokens, reject prefixes containing answer markers, compress the re-encoded midpoint KV cache, and elicit a fresh boxed answer. The protocol separates midpoint recoverability R, whether a forced answer elicited immediately from the audited pre-answer state is correct, from full-chain accuracy and from conditional eviction survival, with exact KV accounting. Our central finding is that post-training moves R by tens of points while paired full-chain accuracy shows no detectable change, in a recipe-dependent direction. Conditional on a correct source trajectory, the Base 1.5B model recovers 77% of midpoint answers on GSM8K. Sixteen public RL checkpoints post-trained from that base, fifteen by five length-shortening recipes, recover 15-60%, all below it, falling monotonically along each recipe's declared dose and tracking realized trace length (Spearman ρ = 0.96). The deficit is also present at 8B; on the audited trio it survives a change of elicitation wording, question-only re-solving and, in large part, an answer-leakage audit, and it persists when the model re-reasons from the compressed cache. Paired GRPO arms reproduce it in all twenty runs over three Qwen-1.5B and two Llama-8B seeds. By step 200, R falls to 35-42% at 1.5B under a length penalty and to 45-50% under a correctness-only control, and to 58.5-61.0% at 8B; accuracy is undetectably changed in every run, and the penalty decline is deeper in every 1.5B seed and absent at 8B. Correctness-only arms retrained at an 8,192-token cap with the registered fp32 weights train yet stay at Base level, 17-37 points above their 1,536-cap twins in all three seeds, so this manipulation supports training-time length pressure as the mediator. The direction depends on the recipe: a public RL-only 7B checkpoint recovers 96.1% against its registered base's 85.3% (+10.7 paired, p = 3.8 × 10^-5) at indistinguishable accuracy, and a 14B pair repeats it more weakly (+6.8, p = 6.6 × 10^-3). Eviction survival is separately model- and task-dependent: compressor rankings reverse across datasets. Under prespecified logistic predictors, no feature family predicts per-trajectory tolerance better than length does. A grouped Clopper-Pearson calibration nevertheless meets per-cell conditional-survival targets on held-out splits (up to 72% KV savings at the 75% retention target, 48% at 90%), while uniform ratios and cross-task transfer both miss it. Two checkpoints with no detectable benchmark-score difference can thus sit 40 points apart in R: same score, different state, in either direction.

Primary area:
foundation or frontier models, including LLMs
Keywords:
reasoning models, KV cache compression, chain-of-thought, reinforcement learning post-training, evaluation and measurement, efficient inference, calibration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tZMEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates into whether the conventional end-to-end accuracy can well represent the recoverability and compressibility of intermediate states in LLMs, by designing an audited midpoint evaluation protocol that interrupts reasoning trajectories prior to final answer generation and assesses midpoint recoverability (R) by forcing answer elicitation and measures conditional survival (S) with KV-cache compression.The central finding is that post-training can substantially alter midpoint recoverability without detectable changes in final-answer accuracy, with the direction and magnitude depending on the training recipe

Critical strengths
  1. Interesting motivation that connects reasoning models with inference infrastructure. Accuracy on completed chains says little about whether a model's partial reasoning survives KV compression, and the paper turns this into a measurable quantity.
  2. The experiments are carefully controlled. Besides comparing public checkpoints, the authors train GRPO arms from the same base with and without a length penalty over multiple seeds, and use paired tests on the same questions.
  3. The authors are candid about limitations and report results that cut against their own headline, e.g., the fixed-token-budget reversal in Appendix S.
Critical weaknesses
  1. Unclear contribution: the main text and the appendix are inconsistent.
    • Cutting at a fraction of the trace requires knowing the full trace length, which, as Appendix S itself says, "an operator cannot know online". Yet the abstract only says that the length-shortened checkpoints fall below Base. It does not mention that under a token budget, which is what a serving system actually has, these models do better.
    • What does "added post hoc on 2026-09-08" mean?
  2. Inappropriate framing. I do not quite understand how the compression assumptions relate to reasoning-model serving. It also seems that you do not compare with R-KV?
Questions for authors
  1. Please consider rewriting parts of the main text and adjusting the figures for better readability.
    • Panels (a) and (b) of Figure 1 should be aligned; the layout of Figures 3, 4, and 5 looks odd and should be adjusted.
    • Several parts are far too long, e.g., the abstract (L12–46), L298–323, and L330–349.
    • Please clarify: the abstract lists many specific numbers, but what does "Base 1.5B model" refer to? Which model family is it, and where do the public RL checkpoints come from? Further details would help.
  2. Suggestion: following Strength 1, why not also consider agent trajectories?
Reviewer_FzLVOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

This paper shows that models with similar final-answer accuracy can differ substantially in their ability to recover answers from partially completed reasoning traces. It studies how post-training and KV-cache compression affect this recoverability and proposes calibrated compression budgets.

Critical strengths

Extensive experimentation with paired checkpoints, training interventions, multiple seeds, compression methods, and robustness controls. The distinction between recoverability before compression and survival after compression is valuable, as is the conditional budget-calibration procedure.

Critical weaknesses

The central measurement compares models at 50% of their own completed reasoning length, which is not a comparable absolute computational budget. Critically, the paper reports that the model ordering reverses under matched fixed-token budgets, undermining the direct serving interpretation of the main result.

Questions for authors

Is the observed degradation a meaningful loss of recoverable information, or primarily a consequence of changing where the answer appears relative to a shortened trajectory? Which practical conclusions remain under matched computation and realistic online eviction policies?

Reviewer_3owDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this paper, the author investigates how post-training changes the recoverability of answers from intermediate reasoning states, even when final-answer accuracy shows no detectable change. The paper introduces an audited midpoint protocol that truncates reasoning, compresses its KV cache, and elicits an answer. Experiments show that posttraining often reduces midpoint recoverability. although some other RL recipes improve it. The paper also calibrates compression budgets.

Critical strengths

The paper shows that models with similar final-answer accuracy can differ substantially in answer recoverability during reasoning, highlighting behavior that conventional benchmarks overlook.

The paper distinguishes midpoint recoverability from conditional eviction survival and uses calibrated budgets to make the tradeoff between KV savings and answer retention explicit.

Critical weaknesses

The cross-task experiments show that GSM8K-calibrated budgets do not reliably preserve the target retention rate on other datasets. This raises questions about how much task-specific calibration is required and whether the approach remains effective on mixed or changing workloads.

The paper introduces many metrics and experimental variants, with key explanations scattered across the main text and appendices. This makes it difficult to follow the central argument and assess which results support the main claims.

Questions for authors

Given the observed cross-task transfer failures, how much recalibration is needed for mixed or changing workloads?

Can you quantify how reliably forced-answer recoverability predicts final-answer accuracy after continued reasoning under compression?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.