- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how latent reasoning models can preserve previously acquired reasoning abilities during continual learning. The authors argue that conventional answer replay cannot explicitly constrain changes in internal states during latent reasoning, while directly matching historical states may restrict adaptation to new tasks. To address this issue, the authors propose LatentKV, which combines historical-KV-conditioned surrogate-gradient replay with answer-sensitive regularization. The former uses fixed historical KV states to guide backpropagation on old-task samples, while the latter periodically updates reference KV states and answer-loss sensitivities to constrain state changes along directions that are sensitive to old-task answers. The authors also conduct local perturbation analyses, showing that KV sensitivities computed by the current model correlate more strongly with old-task loss changes than historical sensitivities at certain training checkpoints. Experiments are conducted on Llama-3.2-3B and Qwen3 models. On GSM8K→ProsQA (training on GSM8K followed by ProsQA), LatentKV improves the average accuracy across the two tasks from 64.83% with Answer Replay to 69.62%. However, on GSM8K→SVAMP, its average accuracy is slightly lower than that of Answer Replay, suggesting that the performance gains are task-dependent.
Critical strengths
-
The paper presents a clear research problem concerning internal state changes in latent reasoning, with a straightforward motivation: identifying a correlation and leveraging it. Rather than focusing solely on answer accuracy, the authors also investigate the relationship between KV state changes and old-task answer loss. They further conduct local perturbation experiments and find that sensitivities computed by the current model are correlated with actual loss changes at certain training checkpoints. This provides a meaningful perspective for analyzing continual learning in latent reasoning models.
-
The proposed method separates historical information replay (incorporating old-task information during new-task training) from dynamic state constraints, achieving improvements on certain tasks. Fixed historical KV states are used for surrogate-gradient replay, while reference KV states and sensitivity directions are updated throughout training. This allows the model to adapt to new tasks while retaining historical context. In addition, the authors claim that the method does not require separate adapters for different tasks and introduces no additional inference-time overhead. Some experimental results provide support for the practical utility and design rationale of the proposed method.
Critical weaknesses
-
Although the paper introduces a considerable number of design choices, the necessity of its key components is not sufficiently validated, particularly the independent contributions of answer-sensitive directions and surrogate-gradient replay. The authors claim that constraining KV changes along answer-sensitive directions helps preserve old-task reasoning abilities. However, I noticed that in the gradient-norm-matched experiments in Appendix C.2, random directions actually achieve higher average accuracy than the sample's own answer-sensitive directions. Furthermore, under the constraint coefficient selected for the main experiments, the paper lacks a comparison with standard Answer Replay combined with the same projection constraint. Therefore, it is difficult to determine whether the more complex historical-KV surrogate-gradient mechanism provides benefits that cannot be achieved by a simpler alternative. This directly affects the assessment of the effectiveness of the paper's core methodological contributions.
-
A more significant concern is that I am not convinced by the stability of the experimental results or the specificity of the method to latent reasoning. The main experiments use three random seeds, which is a reasonable setup, but some tasks exhibit particularly large standard deviations. Moreover, LatentKV does not outperform Answer Replay on GSM8K→SVAMP. Under Protocol B with latent-curriculum training, it also fails to surpass standard Answer Replay on GSM8K→ProsQA, and longer task sequences do not demonstrate consistent advantages. In addition, the main experimental setting, Protocol A, uses explicit CoT supervision during adaptation and only performs latent reasoning at evaluation time. Although the method operates on KV states at continuous-thought positions, its core mechanisms are still replay and intermediate-representation regularization, which could in principle be extended to standard autoregressive SFT. The paper does not provide direct comparisons demonstrating that these designs address problems specific to latent reasoning. This limits the conclusions that can be drawn regarding the applicability and novelty of the proposed method.
Questions for authors
-
Under the hyperparameter configuration selected for the main experiments, can standard Answer Replay combined with the same jointly refreshed KV projection constraint achieve performance comparable to the full LatentKV method? If so, what is the independent contribution of historical-KV-conditioned surrogate-gradient replay?
-
I am still unclear about why this method specifically needs to be studied in the context of latent reasoning. Could the authors provide more intuition behind this design choice? In my view, the method can certainly be considered a continual learning approach, but its connection to latent reasoning does not seem essential.
-
The Sequential FT result reported in Table 1 is rather puzzling, with an old-task GSM8K accuracy of only 0.07%. The appendix seems to attribute this to failures in answer formatting or numerical answer extraction. For a fairer comparison, shouldn't the answer formats across the two datasets be standardized? For example, all training data could use a consistent answer format such as "The answer is {...}".
5 comments
Response to Reviewer 7mW8: Answer-Format Control
3. How should the 0.07% Sequential result be interpreted?
Your suggestion identifies an important distinction: under changing answer conventions, accuracy reflects both retained task ability and compatibility with the output protocol. A clearer retention comparison uses aligned answer encoding and measures change from a matched starting model.
We therefore reran training with ProsQA's two candidates encoded as 1/2 and numerical answers retained for GSM8K. All methods share the same retrained initialization within each seed.
Results are as follows:
| Method | GSM8K accuracy (%) | Change from initialization (points) |
|---|---|---|
| Initialization | 45.33 | — |
| Sequential FT | 35.47 | −9.87 |
| Answer Replay | 45.87 | +0.53 |
| LatentKV (ours) | 48.67 | +3.33 |
Under aligned encoding, Sequential retains 35.47% accuracy but still loses 9.87 points from its initialization. LatentKV reaches 48.67%, improving by 3.33 points over initialization and exceeding Replay by 2.80 points. Thus, the old-task retention result in this control does not depend on Sequential's near-zero score under the original protocol.
The control also makes the adaptation tradeoff visible: two-task means are 56.23% for Sequential, 50.10% for Replay, and 49.89% for LatentKV. LatentKV retains the most old-task accuracy here, while this control does not show an overall average-accuracy advantage.
Because both answer encoding and initialization are changed, these development results are reported separately from the original main-table tests. The original 0.07% is strict accuracy under the original output protocol; it should not be equated with almost complete loss of mathematical reasoning.
The format control lets us state the retention result relative to a shared starting model and aligned numerical answer encoding, which provides a more informative comparison than the original near-zero Sequential score alone.
Thanks for the detailed response and the additional experiments! I appreciate the new matched ablation and answer-format control, which address some of my concerns. However, I'm still not fully convinced about the necessity of answer-sensitive directions, especially given the random-direction results. The connection to latent reasoning also remains somewhat unclear to me, as the method seems applicable to more general continual learning settings. Additionally, the improvements still appear inconsistent across tasks and settings. Overall, while the rebuttal has clarified several points, my main concerns about the method's novelty and experimental robustness remain. So I'll keep my original score. Thanks again for your efforts, and good luck!
Response to Reviewer 7mW8: Historical-KV Replay and Continuous Reasoning
Thank you for the review. We added the missing replay control and a direct comparison of latent and explicit reasoning interfaces.
1. What does historical-KV replay contribute beyond the same projection?
Historical-KV replay changes how an old example supplies a learning signal. Ordinary Replay uses the current answer-loss gradient. Our replay retains the current forward prediction but constructs the backward update through a computation conditioned on historical KV.
The projection controls subsequent state changes. We periodically recompute reference KV and answer-sensitive directions for replay examples. Between refreshes, current KV is compared with the most recently recorded reference, and displacement along the cached sensitive direction is penalized. Historical KV supplies the past computation used for replay, while the refreshed reference measures recent state changes.
Using the Llama-3.2-3B model, we conducted component ablations on GSM8K→ProsQA under Protocol A. All variants use matched initialization, the same data and update budget, and 200 replay examples. The replay weight is 0.02; projection variants use coefficient 0.005 and jointly refresh reference KV and sensitivity every 50 updates.
Results are as follows:
The full method exceeds Replay with the same projection by 10.97 points, with positive paired differences in all six seeds. Both task accuracies improve. Removing projection while retaining historical-KV replay gives 61.11%, 4.32 points below the full method.
Our reading is that the projection's value depends on the replay update it accompanies. Applying the same penalty to ordinary Replay does not recover the combination's performance. This directly supports the historical-KV path at the selected configuration. Component coefficients are fixed here; the main table selects configurations per method, including replay weight 0.01 for Answer Replay.
The random-direction result concerns a separate issue. A direction that predicts local loss changes need not be the best long-horizon regularizer. Our symmetric penalty also restricts potentially beneficial displacement, offering a possible explanation for this distinction. The direction controls remain mixed; the replay ablation holds the projection rule fixed and does not assume answer-sensitive directions are universally best.
2. Why is latent reasoning an important setting for this problem?
We study whether a learned continuous reasoning process remains usable as the model learns new tasks. In latent reasoning, intermediate hidden states become inputs to subsequent reasoning without being decoded into textual steps. Updating the shared model can change both those states and how later computation reads them. This makes the internal computation itself an important object of preservation.
Protocol A tests whether previously learned latent computation survives adaptation with new-task CoT supervision. Protocol B tests adaptation with latent-curriculum training. These are different adaptation conditions and are reported separately; the Protocol A benefit should not be extrapolated directly to Protocol B.
We also intervened on feedback content on GSM8K while keeping all six computation steps unchanged. Results are as follows:
These development results show substantial accuracy losses for both models when feedback content is changed. The content carried between steps therefore matters in this arithmetic setting. ProsQA does not show the same consistent decline, indicating task dependence.
The explicit-CoT control gives old-task test gains over Replay of 4.25 points for latent inference and 2.32 for explicit CoT; the difference remains uncertain. We view applicability to CoT as a useful extension of the state-preservation idea. Our research question concerns continuous computation reused across reasoning steps; answering it does not require the protection mechanism to be exclusive to latent reasoning.