- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Reusable Latent Correction (RLC) for a setting in which a large model is available during preparation but not at test time. During offline construction, the large model gives feedback on uncertain steps produced by a frozen small model. RLC turns the resulting changes in the small model’s hidden states into entries in a key–value bank. At test time, a router decides whether to activate the bank; if activated, the small model retrieves corrections using its current reasoning state and applies them during generation. The paper reports accuracy gains on five benchmark families with three small-model sizes. Average inference latency is reported to be 3.1–4.2% above native inference.
Critical strengths
- The paper addresses a practical constraint: teacher feedback can be collected offline, while the deployed small model must run without teacher calls or parameter updates. The proposed bank gives a concrete way to reuse that feedback across problems.
- The evaluation goes beyond a native-model comparison. The prefill-only and decode-only ablations, the always-on bank results, and the mean-value control help identify which parts of the system contribute to performance. Several of the reported accuracy gains are large enough to merit further investigation.
Critical weaknesses
- The results do not yet separate the value of the stored corrections from that of the trained router. In Appendix Table 6, always enabling the bank improves average accuracy over native inference by 3.93, 3.52, and 1.93 points for the 3B, 7B, and 14B models. Adding the router yields a further 5.79, 4.45, and 4.00 points. The mean-value control is useful evidence for state-matched retrieval, but the paper gives too little information about router training-set sizes, class sampling, and validation to assess how reliably the routing gains would generalize. A control that keeps the router and intervention schedule fixed while changing how corrections are matched would make the main claim more convincing.
- Some reporting issues make the quantitative conclusions difficult to assess. The paper does not give the numerical uncertainty threshold, final bank sizes, or a complete account of validation and hyperparameter selection. It also reports no uncertainty intervals or paired tests, which matters particularly for AMC24 (45 problems) and AIME (30 problems). Moreover, native-model results differ between Tables 1 and 5 without an explanation: 32B MATH500 accuracy is 83.20 in Table 1 but 82.80 in Table 5, while 14B GPQA output length is 421.87 versus 634.18. These discrepancies affect the interpretation of the cross-scale comparisons. Finally, the 3B experiments use AIME25, whereas the larger-model experiments use AIME24. Averages across those groups should therefore not be treated as directly comparable.
Questions for authors
- What uncertainty threshold and final bank size were used for each model? How many examples trained each router, what were the class counts, and which data were used to select the dataset-specific intervention settings?
- What accounts for the native-model differences between Tables 1 and 5? Were the results obtained from different runs, prompts, or decoding configurations?
- Could you provide paired native, always-on bank, and routed-bank outcomes with uncertainty estimates? A comparison that holds the router fixed but randomly reassigns corrections to retrieved states would also help establish the contribution of the stored key–value associations.
6 comments
Response to Reviewer cfDq(2/4)
W2.The experimental settings and validation procedures are insufficiently described.
(1) Uncertainty threshold: To determine the uncertainty threshold(JSD threshold),we first sample 5% of the data from each batch and calculate their average step-level JSD,which is then used as the threshold.
(2) Final experience bank size: The actual sizes of the Banks constructed separately for the 3 models are shown below.As model capability increases,the corresponding correction experiences,i.e.,the number of experience entries,decrease.However,memory usage per entry still increases because the vector dimension grows.
| Bank | Number of experience entries | Tensor memory(MiB) | KiB per experience entry |
|---|---|---|---|
| 3B | 4,521 | 105.96 | 24 |
| 7B | 4,000 | 164.06 | 42 |
| 14B | 3,549 | 207.95 | 60 |
(3) Hyperparameter selection and validation procedure: For validation and parameter selection on more difficult test sets,such as AIME and AMC,we randomly sample 10% of the level 4 and 5 problems from the MATH training set for tuning.For GSM8K,we sample 10% of its training set for tuning.For MATH500,we likewise sample 10% of the MATH training set for tuning.For GPQA,we tune the hyperparameters on the approximately 250 problems remaining in GPQA Main after excluding the test problems.Regarding the specific tuning procedure,a coarse search of the similarity threshold indicated better performance above 0.9.We first try top-k values of 1,2,and 3 while keeping the other parameters at their default values.We then sweep the similarity threshold over in increments of 0.01.The process-vector injection strength is swept over in increments of 0.1,and the result-vector injection strength over in increments of 0.05.Because the result vector causes larger fluctuations,we use a slightly smaller injection strength and a smaller step size.We report paired results for Native,always enabling the Bank,and using the Bank with the Router on the same set of samples,together with uncertainty estimates.
Response to Reviewer cfDq(3/4)
(4) Confidence intervals and paired statistics: The tables below report the accuracy of the Native,Bank and Router as percentages.""denotes the sample standard deviation(SD)across 5 random seeds, measured in percentage points.Table 6 of the paper already reports the proportions of problems that remain on Native and those that activate the Bank under the Router for different datasets."Only X correct / Only Y correct"gives the number of problems answered correctly only by X and only by Y,respectively,showing the detailed paired outcomes for problems on which the Bank and Native branches disagree.N,B,and R denote Native,always enabling the Bank,and selecting a branch for each problem using the Router,respectively.
| Method(3B)/ Paired comparison | MATH500 | GSM8K | GPQA-Diamond | AMC24 | AIME24 |
|---|---|---|---|---|---|
| Native(N) | 67.00 ± 0.62 | 84.69 ± 0.30 | 30.30 ± 0.80 | 4.44 ± 1.57 | 3.33 ± 2.36 |
| Bank(B) | 65.00 ± 0.68 | 83.70 ± 0.32 | 31.82 ± 0.94 | 22.22 ± 1.57 | 3.33 ± 2.36 |
| Router(R) | 73.00 ± 0.60 | 89.99 ± 0.27 | 40.91 ± 0.51 | 24.44 ± 1.57 | 3.33 ± 2.36 |
| Only B correct / Only N correct | 39 / 49 | 76 / 89 | 30 / 27 | 9 / 1 | 0 / 0 |
| Only R correct / Only B correct | 42 / 2 | 85 / 2 | 25 / 7 | 1 / 0 | 0 / 0 |
| Only R correct / Only N correct | 37 / 7 | 74 / 4 | 23 / 2 | 9 / 0 | 0 / 0 |
| Method(7B)/ Paired comparison | MATH500 | GSM8K | GPQA-Diamond | AMC24 | AIME24 |
|---|---|---|---|---|---|
| Native | 74.20 ± 0.42 | 90.83 ± 0.21 | 38.89 ± 1.01 | 20.00 ± 1.57 | 13.33 ± 2.36 |
| Bank | 74.00 ± 0.55 | 90.75 ± 0.25 | 37.88 ± 1.07 | 35.56 ± 2.22 | 16.67 ± 2.36 |
| Router | 79.80 ± 0.63 | 94.09 ± 0.16 | 45.45 ± 0.62 | 37.78 ± 1.57 | 20.00 ± 3.33 |
| Only B correct / Only N correct | 29 / 30 | 43 / 44 | 28 / 30 | 8 / 1 | 3 / 2 |
| Only R correct / Only B correct | 30 / 1 | 44 / 0 | 21 / 6 | 1 / 0 | 1 / 0 |
| Only R correct / Only N correct | 28 / 0 | 43 / 0 | 22 / 9 | 8 / 0 | 3 / 1 |
| Method(14B)/ Paired comparison | MATH500 | GSM8K | GPQA-Diamond | AMC24 | AIME24 |
|---|---|---|---|---|---|
| Native | 77.00 ± 0.45 | 94.47 ± 0.17 | 42.93 ± 1.13 | 31.11 ± 1.57 | 13.33 ± 2.36 |
| Bank | 79.40 ± 0.51 | 93.25 ± 0.23 | 41.41 ± 0.71 | 37.78 ± 2.22 | 16.67 ± 2.36 |
| Router | 83.00 ± 0.49 | 95.68 ± 0.14 | 52.02 ± 0.87 | 37.78 ± 1.57 | 20.00 ± 2.36 |
| Only B correct / Only N correct | 33 / 21 | 19 / 35 | 22 / 25 | 4 / 1 | 2 / 1 |
| Only R correct / Only B correct | 21 / 3 | 35 / 3 | 25 / 4 | 1 / 1 | 1 / 0 |
| Only R correct / Only N correct | 30 / 0 | 16 / 0 | 18 / 0 | 3 / 0 | 2 / 0 |
Response to Reviewer cfDq(1/4)
We are grateful for your detailed feedback on the experimental analysis and reporting.Your suggestions have guided us in further examining the contributions of the Router and correction vectors and clarifying the evaluation protocol.We respond below with additional control experiments,methodological details,and corrected results.
Thank you for suggesting that we further distinguish the contributions of the Router and correction vectors.In the original paper,we analyzed the role of the Router through ablation experiments and examined the importance of state matching through the Mean Value control.Below,we first clarify the existing results and then present the newly added Shuffled control experiment to further verify the role of correction vectors.
(1)To verify the effectiveness of the Router,we conducted an ablation experiment,as reported in Table 6 of Appendix C.3.Always enabling the Bank without the Router improves average accuracy over Native by 3.93,3.52,and 1.93 percentage points for the 3B,7B,and 14B models,respectively.Adding the Router yields further improvements of 5.79,4.45,and 4.00 percentage points,demonstrating its effectiveness.
(2)We agree that the effects of the correction vectors have not yet been fully isolated.In the original paper,we included a Mean Value control to examine the importance of state-matched corrections.To further verify the effectiveness of the correction vectors,we added a Shuffled experiment:while keeping the Router and retrieval rules fixed,we randomly shuffled the key-value correspondences in the Bank and reran the experiments using this shuffled Bank.The results are shown below.
Table R2.Results after shuffling the pairings in the Bank. Each cell reports accuracy(%)/ average output tokens.Averages assign equal weight to the 5 datasets.
As shown,randomly shuffling the key-value correspondences in the Bank substantially reduces accuracy even when the Router and retrieval conditions remain unchanged.This is because the disrupted correspondences result in randomly mismatched interventions on Bank-routed problems,preventing them from being solved correctly.Together with the Mean Value experiment,this also demonstrates that correction vectors can indeed provide step-level corrections matched to the current reasoning state for problems that Native cannot solve correctly.