acceptodds
Under review as a conference paper at ICLR 2027

LatentKV: Preserving Reasoning Ability in Continual Latent Reasoning

Abstract

Latent reasoning carries intermediate computation in continuous hidden states rather than generating an explicit chain of thought, solving problems with a few continuous thoughts and far fewer reasoning tokens. Continually adapting such models to new tasks while preserving previously acquired reasoning abilities poses a further challenge: because the states carry the computation, the final output does not directly reveal how internal reasoning changes as the model is updated. Answer-based replay shapes these states only through the answer loss and does not distinguish how different state changes affect it; matching historical states constrains every state component to its historical value and may over-restrict adaptation to the new task. The key question is therefore which state perturbations affect the old-task answer loss locally. Through local perturbation analysis, we find that (i) projecting KV-state changes onto the sensitivity direction of the answer loss yields a signal correlated with the change in old-task loss, and (ii) sensitivity computed on the current model predicts this change more accurately than sensitivity inherited from the historical model. We propose LatentKV, a continual-learning method for latent reasoning that combines historical-KV replay with answer-sensitive state constraints using a single LoRA adapter and a fixed replay budget. For replay, LatentKV guides old samples with a surrogate gradient conditioned on stored historical KV. For constraint, it periodically refreshes both the reference KV and the answer-loss sensitivity, and applies a symmetric projection penalty that limits state change along answer-sensitive directions. When continually adapting from GSM8K to SVAMP, LatentKV raises old-task latent accuracy from 26.80% to 52.13% over sequential fine-tuning, while achieving 69.09% accuracy on the new task, a decrease of 3.89 percentage points. Code is available at https://anonymous.4open.science/r/latentkv/.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, latent reasoning, large language models, catastrophic forgetting, experience replay, key-value states, answer-sensitive preservation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7mW8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how latent reasoning models can preserve previously acquired reasoning abilities during continual learning. The authors argue that conventional answer replay cannot explicitly constrain changes in internal states during latent reasoning, while directly matching historical states may restrict adaptation to new tasks. To address this issue, the authors propose LatentKV, which combines historical-KV-conditioned surrogate-gradient replay with answer-sensitive regularization. The former uses fixed historical KV states to guide backpropagation on old-task samples, while the latter periodically updates reference KV states and answer-loss sensitivities to constrain state changes along directions that are sensitive to old-task answers. The authors also conduct local perturbation analyses, showing that KV sensitivities computed by the current model correlate more strongly with old-task loss changes than historical sensitivities at certain training checkpoints. Experiments are conducted on Llama-3.2-3B and Qwen3 models. On GSM8K→ProsQA (training on GSM8K followed by ProsQA), LatentKV improves the average accuracy across the two tasks from 64.83% with Answer Replay to 69.62%. However, on GSM8K→SVAMP, its average accuracy is slightly lower than that of Answer Replay, suggesting that the performance gains are task-dependent.

Critical strengths
  1. The paper presents a clear research problem concerning internal state changes in latent reasoning, with a straightforward motivation: identifying a correlation and leveraging it. Rather than focusing solely on answer accuracy, the authors also investigate the relationship between KV state changes and old-task answer loss. They further conduct local perturbation experiments and find that sensitivities computed by the current model are correlated with actual loss changes at certain training checkpoints. This provides a meaningful perspective for analyzing continual learning in latent reasoning models.

  2. The proposed method separates historical information replay (incorporating old-task information during new-task training) from dynamic state constraints, achieving improvements on certain tasks. Fixed historical KV states are used for surrogate-gradient replay, while reference KV states and sensitivity directions are updated throughout training. This allows the model to adapt to new tasks while retaining historical context. In addition, the authors claim that the method does not require separate adapters for different tasks and introduces no additional inference-time overhead. Some experimental results provide support for the practical utility and design rationale of the proposed method.

Critical weaknesses
  1. Although the paper introduces a considerable number of design choices, the necessity of its key components is not sufficiently validated, particularly the independent contributions of answer-sensitive directions and surrogate-gradient replay. The authors claim that constraining KV changes along answer-sensitive directions helps preserve old-task reasoning abilities. However, I noticed that in the gradient-norm-matched experiments in Appendix C.2, random directions actually achieve higher average accuracy than the sample's own answer-sensitive directions. Furthermore, under the constraint coefficient selected for the main experiments, the paper lacks a comparison with standard Answer Replay combined with the same projection constraint. Therefore, it is difficult to determine whether the more complex historical-KV surrogate-gradient mechanism provides benefits that cannot be achieved by a simpler alternative. This directly affects the assessment of the effectiveness of the paper's core methodological contributions.

  2. A more significant concern is that I am not convinced by the stability of the experimental results or the specificity of the method to latent reasoning. The main experiments use three random seeds, which is a reasonable setup, but some tasks exhibit particularly large standard deviations. Moreover, LatentKV does not outperform Answer Replay on GSM8K→SVAMP. Under Protocol B with latent-curriculum training, it also fails to surpass standard Answer Replay on GSM8K→ProsQA, and longer task sequences do not demonstrate consistent advantages. In addition, the main experimental setting, Protocol A, uses explicit CoT supervision during adaptation and only performs latent reasoning at evaluation time. Although the method operates on KV states at continuous-thought positions, its core mechanisms are still replay and intermediate-representation regularization, which could in principle be extended to standard autoregressive SFT. The paper does not provide direct comparisons demonstrating that these designs address problems specific to latent reasoning. This limits the conclusions that can be drawn regarding the applicability and novelty of the proposed method.

Questions for authors
  1. Under the hyperparameter configuration selected for the main experiments, can standard Answer Replay combined with the same jointly refreshed KV projection constraint achieve performance comparable to the full LatentKV method? If so, what is the independent contribution of historical-KV-conditioned surrogate-gradient replay?

  2. I am still unclear about why this method specifically needs to be studied in the context of latent reasoning. Could the authors provide more intuition behind this design choice? In my view, the method can certainly be considered a continual learning approach, but its connection to latent reasoning does not seem essential.

  3. The Sequential FT result reported in Table 1 is rather puzzling, with an old-task GSM8K accuracy of only 0.07%. The appendix seems to attribute this to failures in answer formatting or numerical answer extraction. For a fairer comparison, shouldn't the answer formats across the two datasets be standardized? For example, all training data could use a consistent answer format such as "The answer is {...}".

5 comments
AuthorsOpenReview ↗

Response to Reviewer 7mW8: Historical-KV Replay and Continuous Reasoning

Thank you for the review. We added the missing replay control and a direct comparison of latent and explicit reasoning interfaces.

1. What does historical-KV replay contribute beyond the same projection?

Historical-KV replay changes how an old example supplies a learning signal. Ordinary Replay uses the current answer-loss gradient. Our replay retains the current forward prediction but constructs the backward update through a computation conditioned on historical KV.

The projection controls subsequent state changes. We periodically recompute reference KV and answer-sensitive directions for replay examples. Between refreshes, current KV is compared with the most recently recorded reference, and displacement along the cached sensitive direction is penalized. Historical KV supplies the past computation used for replay, while the refreshed reference measures recent state changes.

Using the Llama-3.2-3B model, we conducted component ablations on GSM8K→ProsQA under Protocol A. All variants use matched initialization, the same data and update budget, and 200 replay examples. The replay weight is 0.02; projection variants use coefficient 0.005 and jointly refresh reference KV and sensitivity every 50 updates.

Results are as follows:

MethodGSM8KProsQATwo-task average
Answer Replay43.5272.4757.99
Splice-only48.0874.1361.11
Answer Replay + same projection43.2865.6354.46
LatentKV (ours)48.5682.3065.43

The full method exceeds Replay with the same projection by 10.97 points, with positive paired differences in all six seeds. Both task accuracies improve. Removing projection while retaining historical-KV replay gives 61.11%, 4.32 points below the full method.

Our reading is that the projection's value depends on the replay update it accompanies. Applying the same penalty to ordinary Replay does not recover the combination's performance. This directly supports the historical-KV path at the selected configuration. Component coefficients are fixed here; the main table selects configurations per method, including replay weight 0.01 for Answer Replay.

The random-direction result concerns a separate issue. A direction that predicts local loss changes need not be the best long-horizon regularizer. Our symmetric penalty also restricts potentially beneficial displacement, offering a possible explanation for this distinction. The direction controls remain mixed; the replay ablation holds the projection rule fixed and does not assume answer-sensitive directions are universally best.

2. Why is latent reasoning an important setting for this problem?

We study whether a learned continuous reasoning process remains usable as the model learns new tasks. In latent reasoning, intermediate hidden states become inputs to subsequent reasoning without being decoded into textual steps. Updating the shared model can change both those states and how later computation reads them. This makes the internal computation itself an important object of preservation.

Protocol A tests whether previously learned latent computation survives adaptation with new-task CoT supervision. Protocol B tests adaptation with latent-curriculum training. These are different adaptation conditions and are reported separately; the Protocol A benefit should not be extrapolated directly to Protocol B.

We also intervened on feedback content on GSM8K while keeping all six computation steps unchanged. Results are as follows:

MethodNormal feedbackFixed feedbackChannel perturbation
Answer Replay46.6715.2015.73
LatentKV52.6019.4715.93

These development results show substantial accuracy losses for both models when feedback content is changed. The content carried between steps therefore matters in this arithmetic setting. ProsQA does not show the same consistent decline, indicating task dependence.

The explicit-CoT control gives old-task test gains over Replay of 4.25 points for latent inference and 2.32 for explicit CoT; the difference remains uncertain. We view applicability to CoT as a useful extension of the state-preservation idea. Our research question concerns continuous computation reused across reasoning steps; answering it does not require the protection mechanism to be exclusive to latent reasoning.

AuthorsOpenReview ↗

Response to Reviewer 7mW8: Answer-Format Control

3. How should the 0.07% Sequential result be interpreted?

Your suggestion identifies an important distinction: under changing answer conventions, accuracy reflects both retained task ability and compatibility with the output protocol. A clearer retention comparison uses aligned answer encoding and measures change from a matched starting model.

We therefore reran training with ProsQA's two candidates encoded as 1/2 and numerical answers retained for GSM8K. All methods share the same retrained initialization within each seed.

Results are as follows:

MethodGSM8K accuracy (%)Change from initialization (points)
Initialization45.33—
Sequential FT35.47−9.87
Answer Replay45.87+0.53
LatentKV (ours)48.67+3.33

Under aligned encoding, Sequential retains 35.47% accuracy but still loses 9.87 points from its initialization. LatentKV reaches 48.67%, improving by 3.33 points over initialization and exceeding Replay by 2.80 points. Thus, the old-task retention result in this control does not depend on Sequential's near-zero score under the original protocol.

The control also makes the adaptation tradeoff visible: two-task means are 56.23% for Sequential, 50.10% for Replay, and 49.89% for LatentKV. LatentKV retains the most old-task accuracy here, while this control does not show an overall average-accuracy advantage.

Because both answer encoding and initialization are changed, these development results are reported separately from the original main-table tests. The original 0.07% is strict accuracy under the original output protocol; it should not be equated with almost complete loss of mathematical reasoning.

The format control lets us state the retention result relative to a shared starting model and aligned numerical answer encoding, which provides a more informative comparison than the original near-zero Sequential score alone.

Thanks for the detailed response and the additional experiments! I appreciate the new matched ablation and answer-format control, which address some of my concerns. However, I'm still not fully convinced about the necessity of answer-sensitive directions, especially given the random-direction results. The connection to latent reasoning also remains somewhat unclear to me, as the method seems applicable to more general continual learning settings. Additionally, the improvements still appear inconsistent across tasks and settings. Overall, while the rebuttal has clarified several points, my main concerns about the method's novelty and experimental robustness remain. So I'll keep my original score. Thanks again for your efforts, and good luck!

Reviewer_5St9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LatentKV, a continual learning method designed to preserve previously acquired reasoning abilities in latent reasoning models. The method combines historical KV-conditioned surrogate-gradient replay with a dynamically refreshed, answer-sensitive projection constraint to reduce catastrophic forgetting while adapting to new tasks. Experiments on mathematical and logical reasoning benchmarks demonstrate improvements in the retention–adaptation trade-off under several settings, without introducing additional inference-time parameters or computation.

Critical strengths

1.The paper addresses an interesting and important problem of preserving reasoning abilities during continual adaptation of latent reasoning models. 2.The proposed method is interesting, combining historical KV replay with answer-sensitive constraints to balance knowledge retention and adaptation. 3.The idea of dynamically refreshing KV references and sensitivity directions is intuitive and provides a useful perspective on protecting latent reasoning representations. 4.The paper includes multiple baselines, ablation studies, and experiments with different model sizes, providing a relatively comprehensive empirical evaluation.

Critical weaknesses

1.The effectiveness of the answer-sensitive constraint is not sufficiently established. In Table 8, random directions outperform the proposed sensitivity directions, raising questions about the necessity of the answer-specific design. 2.Some important ablation studies are missing. In particular, the comparison between standard Answer Replay with the projection constraint and the full method under the selected hyperparameters is needed to clarify the contribution of surrogate-gradient replay. 3.The performance improvements are inconsistent across different task orders and settings. In some experiments, LatentKV performs worse than standard Answer Replay, limiting the generalizability of the conclusions. 4.The evaluation mainly focuses on short task sequences, while continual learning typically involves longer sequences of tasks. More extensive experiments would strengthen the practical relevance of the method. 5.The method introduces additional training computation and KV storage overhead. A more detailed analysis of scalability and the trade-off between performance improvements and computational costs would be helpful.

Questions for authors

1.Why do random constraint directions outperform the proposed answer-sensitive directions in Table 8? 2.Can the authors provide a comparison between Answer Replay with the projection constraint and the full LatentKV method using the selected hyperparameters?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 5St9: Component Contributions, Direction Choice, and Practical Cost

Thank you for the review. We added the missing component comparison and revisited the direction results. Below we explain what these experiments tell us about the method.

1. Does historical-KV replay contribute beyond the same projection? (Q2)

Our answer is yes in the main setting: the replay path matters even when the projection rule is held fixed. Historical KV conditions the replay gradient, while the refreshed reference regulates current state displacement.

On GSM8K→ProsQA (3B, Protocol A), all variants share matched initialization, data, updates, and 200 replay examples. The replay weight is 0.02; projection variants use coefficient 0.005 and joint refresh every 50 updates.

Mean test accuracies (%) across six seeds are:

MethodGSM8KProsQATwo-task average
Answer Replay43.5272.4757.99
Splice-only48.0874.1361.11
Answer Replay + same projection43.2865.6354.46
LatentKV (ours)48.5682.3065.43

LatentKV exceeds Replay with the same projection by 10.97 points, with positive paired differences in all six seeds. The projection's mean effect is particularly informative: it lowers ordinary Replay by 3.53 points, but improves historical-KV replay by 4.32 points. Its effect therefore depends on the replay path, supporting the value of the combination.

The six seeds comprise the original three and three new seeds. The full method beats both single-component variants in each additional seed, although three-seed estimates remain uncertain. This table fixes component coefficients; the main table instead selects configurations separately, including replay weight 0.01 for Answer Replay.

2. Why can random directions perform better? (Q1)

Our interpretation is that a locally informative direction need not be the best long-horizon regularizer. Our symmetric penalty restricts displacement of either sign along the answer-sensitive direction, including potentially beneficial changes. It may therefore restrict beneficial updates as well as harmful drift. This is a plausible explanation to investigate, not an experimentally isolated cause of the random-direction result.

We checked that the original Table 8 already matches the constraint gradient norm to 1% of the new-task gradient norm before clipping (observed mean: 0.976%–0.980%). Thus, the random-direction advantage cannot simply be attributed to a smaller gradient norm.

ComparisonAnswer-sensitiveRandom
Original Table 861.0065.44
Fixed main coefficient69.6264.58
Equal-budget coefficient selection55.5057.64

Entries are two-task means (%); equal-budget selection tests five coefficients per direction on the development set.

The ordering changes across these controls. We use answer sensitivity to target changes that affect the old answer loss. These controls show that this local relevance does not, by itself, guarantee the best training outcome.

3. Task order and longer streams

A fixed buffer must represent a broader task history as learning continues. We tested longer streams without increasing the total replay budget beyond 200 examples:

Task sequenceAnswer ReplayLatentKV
GSM8K→AQuA→ProsQA32.4236.27
ProsQA→AQuA→GSM8K48.0748.28
GSM8K→AQuA→ProsQA→SVAMP54.6754.25

These are final average development accuracies (%) over three seeds and all tasks. The first stream extends the positive mean gain to three tasks; the reverse stream has a smaller difference, and the four-task means differ by 0.42 points. Sequence length alone does not determine the benefit: task composition and order matter, as do the supervision differences in Protocol B. We will report stage-wise accuracy and forgetting so readers can see which transitions produce the gains and tradeoffs.

4. What does the extra training cost buy?

We see the practical tradeoff as additional work during adaptation, with deployment kept unchanged. Within the same uninterrupted A100 allocation, complete 250-update training took:

MetricAnswer ReplayLatentKV
Training time34.41 min52.83 min
Relative time1.00×1.54×

For this 3B/B200 setting, active serialized KV/sensitivity storage is 537 MiB. Timing includes reference construction, refreshes, optimization, and model saving, excluding loading and evaluation. Storage grows with the number of examples and recorded state dimensions.

This tradeoff matters for periodic adaptation followed by many inference requests: historical KV and sensitivities are needed only during training. Deployment retains the updated model and a single adapter, without adding replay machinery to inference. We report this training overhead separately from latent-versus-CoT inference efficiency.

AuthorsOpenReview ↗

Additional Experiments: Replay Budget and Joint Refresh Sensitivity

Following our earlier response, we have completed additional experiments on replay budget and joint refresh frequency using Llama-3.2-3B on GSM8K→ProsQA under Protocol A. The main coefficients remain fixed: replay weight 0.02 and projection coefficient 0.005.

1. Joint refresh frequency

With 200 replay examples, we jointly refresh the reference KV and answer sensitivity at different intervals. Results are as follows:

Refresh intervalGSM8KProsQATwo-task average
No refresh after initialization48.1777.6062.88
R=1048.8085.1366.97
R=2549.0885.0767.07
R=50, our default48.9890.2769.62
R=10050.0488.5369.29

Periodic refresh gives positive mean gains over the fixed reference throughout the tested R=10–100 range. Our existing R=50 configuration improves the two-task average by 6.74 percentage points, while R=100 also reaches 69.29%.

We interpret the schedule as controlling the window over which state changes are penalized: refreshing the reference resets the point from which displacement is measured. More frequent refreshing therefore need not produce better learning. The results support a useful range of intervals, with the highest observed mean at the existing default.

2. Replay budget

We also compare LatentKV with Answer Replay at the same replay-example budgets, retaining the fixed coefficients above.

BufferMethodGSM8KProsQATwo-task average
50Answer Replay35.7863.6749.73
50LatentKV (ours)33.6672.8053.23
100Answer Replay39.4558.0748.76
100LatentKV (ours)39.4079.1359.27
200Answer Replay43.1482.3362.74
200LatentKV (ours)48.9890.2769.62
400Answer Replay46.3778.4762.42
400LatentKV (ours)50.1480.7365.44

LatentKV achieves positive mean two-task gains at all four budgets: +3.51, +10.51, +6.89, and +3.02 points, respectively. The task breakdown also shows how the benefit changes: at B=50 and B=100 it is mainly improved new-task learning, whereas at B=200 and B=400 both task means exceed Replay. B=100 has substantial seed variation, so the larger mean there should not be interpreted as an optimal budget.

All entries are matched three-seed mean test accuracies (%). The B=200/R=50 reference conditions reuse existing completed runs. These are fixed-coefficient sensitivity comparisons; the separately tuned Answer Replay in the main table uses replay weight 0.01.

These additional experiments show that positive mean gains are observed across several replay budgets and refresh intervals in this setting, while making the corresponding retention–adaptation tradeoffs explicit. Our main configuration remains unchanged.

Reviewer_MezoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LatentKV studies how to adapt a latent reasoning model to a new task while preserving performance on earlier tasks. It uses a single LoRA adapter and stores KV states for 200 old examples. During replay, the current model produces the answer, while historical KV states guide the backward pass. A separate penalty limits KV changes along directions that affect the old answer loss. On GSM8K to ProsQA, LatentKV improves average test accuracy by a reasonable margin over Answer Replay.

Critical strengths
  • The feedback intervention is informative. With six latent steps in both conditions, replacing the feedback with a fixed vector drops GSM8K accuracy visibly. This shows that the content passed between steps matters for arithmetic reasoning.
  • The ablation study is a strength. With surrogate replay held fixed, the authors show that jointly refreshing the KV reference and the sensitivity direction outperforms refreshing either alone.
Critical weaknesses
  • The contribution of historical-KV replay is unclear.
  • The advantage over ordinary replay varies across task sequences. Answer Replay slightly outperforms LatentKV on GSM8K to SVAMP and achieves higher final average accuracy on both reported three-task streams.
Questions for authors

My main concern is the gap between predicting local changes in answer loss and preventing forgetting over continued training. At the selected penalty weight, how does retention compare when using the latest answer gradient, gradients averaged across checkpoints, or random directions?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Mezo: Direction Choice and End-of-Training Retention

Thank you for your suggestions. Your central question is whether the sensitivity information observed in local perturbation experiments actually helps preserve old-task abilities during subsequent training. Following this suggestion, we added end-of-training comparisons of different constraint directions and further examined the contribution of historical-KV replay.

1. How much old-task ability do different directions preserve after training?

On GSM8K→ProsQA, we held initialization, training budget, and constraint coefficients fixed and compared four alternatives: the most recently refreshed direction, the same example’s average direction over the last three checkpoints, a fixed random direction, and the answer-sensitive direction recomputed at every update. Reference KV was refreshed every 50 updates in all groups. The per-update variant recomputed only the direction, without resetting reference KV at every update.

The following table reports mean held-out test accuracies (%) across three additional seeds:

Method / direction variantOld GSM8KNew ProsQATwo-task average
LatentKV (ours; most recently refreshed direction)48.1474.3361.24
Three-checkpoint-averaged direction45.4169.2057.31
Fixed random direction48.9571.0059.98
Direction recomputed at every update49.5653.2051.38

The first row is our standard LatentKV configuration. The remaining rows replace only its constraint-direction rule.

In this comparison, the recently refreshed direction was more effective than checkpoint averaging: the averaged direction produced lower old-task accuracy in all three seeds. However, more frequent recomputation did not improve overall performance either. Recomputing the direction at every update increased old-task accuracy to 49.56%, while new-task accuracy fell to 53.20%, indicating a substantial adaptation cost.

Our standard configuration achieved the highest two-task average. The random direction retained slightly higher old-task accuracy and produced a similar overall result. These experiments therefore provide the end-of-training comparison you requested, while also showing that local predictive accuracy and long-term retention do not have a simple correspondence. Sensitivity provides local information for preservation, but its training effect also depends on the balance between retention and adaptation. The current results do not establish that answer-sensitive directions provide better long-term retention than random directions.

All groups used the same coefficients, but different directions can still produce different parameter-gradient magnitudes. This remains a limitation when attributing the endpoint differences specifically to the choice of direction.

AuthorsOpenReview ↗

Response to Reviewer Mezo: Historical-KV Replay and Task-Sequence Dependence

2. Could the gains come solely from the projection constraint, rather than historical-KV replay?

We added the missing direct comparison: standard Answer Replay with the same jointly refreshed projection used by the full method. All variants shared matched initialization, training data, update budget, and 200 replay examples. The replay weight was fixed at 0.02, and the projection coefficient was 0.005 where applicable.

The following table reports mean held-out test accuracies (%) across six seeds:

MethodGSM8KProsQATwo-task average
Answer Replay43.5272.4757.99
Splice-only48.0874.1361.11
Answer Replay + same projection43.2865.6354.46
LatentKV (ours)48.5682.3065.43

LatentKV improved average accuracy by 10.97 percentage points over Answer Replay with the same projection, with positive paired differences in all six seeds. It also improved over Splice-only, which uses historical-KV surrogate replay without the projection constraint, by 4.32 percentage points.

This comparison provides more direct evidence of the contribution of historical-KV replay. Adding the same projection to standard answer replay did not recover the full method’s performance; using historical-KV replay alone also fell short of the combined method’s average performance. In the three additional seeds, the full method likewise outperformed both single-component variants in every seed, although the magnitude of the gains varied substantially.

The six-seed results include the original three seeds and three additional seeds. This table fixes the component configuration, whereas the original main table compares configurations selected separately for each method. In the original main table, Answer Replay uses a replay weight of 0.01, which explains why its scores differ between the two tables.

3. Performance across longer task sequences

To address your concern about task-sequence length, we added three-seed experiments on two three-task streams and one four-task stream. The total replay budget remained fixed at 200 throughout, rather than increasing with the number of tasks.

The following table reports average development accuracy (%) across all tasks after completing each sequence:

Task sequenceAnswer ReplayLatentKV
GSM8K→AQuA→ProsQA32.4236.27
ProsQA→AQuA→GSM8K48.0748.28
GSM8K→AQuA→ProsQA→SVAMP54.6754.25

On GSM8K→AQuA→ProsQA, LatentKV increased final average accuracy from 32.42% to 36.27%, showing a positive mean gain in a sequence that includes an additional task. In the reverse three-task stream, the methods performed similarly, with a slightly higher mean for LatentKV. After adding SVAMP as a fourth task, their final averages differed by 0.42 percentage points.

These experiments extend the original two-task evaluation to multi-stage learning under a fixed total replay budget. They also show that the magnitude of the gains varies with task composition and order. We will include stage-wise accuracy and forgetting results in the revision, allowing readers to examine what changes at each task transition rather than only the final average.

AuthorsOpenReview ↗

Additional Experiments: Joint Refresh Frequency

Following our earlier response, we have completed additional experiments on the joint refresh schedule using Llama-3.2-3B on GSM8K→ProsQA under Protocol A. We use 200 replay examples and fix the replay weight at 0.02 and the projection coefficient at 0.005.

The earlier direction comparison kept reference KV refreshed every 50 updates and varied how the direction was computed. This experiment instead varies how often reference KV and sensitivity are refreshed together. The scan uses the main-setting seed group and its corresponding R=50 reference, with paired comparisons across schedules.

Results are as follows:

Joint refresh intervalGSM8KProsQATwo-task average
No refresh after initialization48.1777.6062.88
R=1048.8085.1366.97
R=2549.0885.0767.07
R=50, our default48.9890.2769.62
R=10050.0488.5369.29

Periodic refresh gives positive mean gains over the fixed reference throughout the tested R=10–100 range. Our existing R=50 configuration improves the two-task average by 6.74 percentage points. R=100 also achieves 69.29%, showing that the positive mean gain is not confined to one refresh interval.

These results help clarify the role of reference updates. A fixed reference measures accumulated displacement from the same recorded KV values; periodic updates instead measure changes since the latest reference. At R=50, GSM8K improves from 48.17% to 48.98%, while ProsQA improves from 77.60% to 90.27%. The benefit is mainly improved new-task adaptation, accompanied by a smaller increase in mean old-task accuracy.

We therefore interpret the refresh interval as controlling how much state change accumulates before the reference is reset. More frequent reference updates need not produce a better retention–adaptation balance. The existing R=50 default has the highest two-task mean in this completed comparison, and our main configuration remains unchanged.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Revisions

We sincerely thank all the reviewers for reading our paper so carefully and for taking the time to share their suggestions. We also thank the Area Chair for coordinating the review and discussion. Your feedback helped us add important controls and explain more clearly where our findings apply. We appreciate the opportunity to improve the paper through this discussion.

A check mark indicates that the reviewer raised or discussed the concern.

ConcernMezo5St97mW8Location in the revision or rebuttalMain result or clarification
Contribution of historical KV replay✓✓✓Table 3LatentKV improved two-task test accuracy over Answer Replay with the same projection and training budget.
How to interpret the 0.07% for Sequential FT✓Appendix A, Table 6After retraining with a common answer format, Sequential FT scored 35.47% and LatentKV 48.67% on GSM8K dev.
Choice of direction and old-task retention✓ Refresh timing✓ Random control✓ Random controlAppendix CThe latest refresh gave the highest two-task mean, and fixed random scored slightly higher on the old task.
Connection to latent reasoning✓Table 5 and Appendix F.5With six steps in both conditions, fixing the feedback input lowered GSM8K dev accuracy from 52.60% to 19.47%.
Robustness across settings✓✓✓Rebuttal follow-ups on sensitivity and longer streamsMean gains held across tested budgets and refresh intervals on the main task pair, with mixed results on longer streams.
Training and storage costs✓Appendix F.4On the same A100, LatentKV took 1.54× as long as Answer Replay and used about 537 MiB of active cache.

Motivation and Core Contributions

  1. Can a model still use the latent reasoning it learned for earlier tasks after adapting to a new one? In latent reasoning, the hidden state from one step feeds into the next. Updating the shared model can change both the states it produces and how later steps use them. We study how to preserve the computation that earlier tasks depend on while the model learns new ones.

  2. Bring the old task's computational context into replay. The current model makes the prediction for each replay example. The backward update uses a surrogate gradient conditioned on historical KV. This lets the same replay examples bring context from the earlier model into training alongside answer supervision.

  3. Update the state constraint as the model adapts. LatentKV periodically recomputes the reference KV and answer-sensitive direction with the current model. From that reference, the penalty constrains KV drift along the refreshed direction. The historical KV provides a fixed context for replay. Both operations take place during training, and inference uses the updated model with a single LoRA adapter.

Thank you again for the time and care you have given this work. We would be grateful if the Area Chair could consider the additional comparisons alongside the original reviews when forming the overall assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.