acceptodds
Under review as a conference paper at ICLR 2027

MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models

Abstract

Continual learning (CL) is essential for small language models (SLMs) to adapt to evolving real-world needs in resource-constrained deployments. However, directly updating their limited parameter space causes catastrophic forgetting. While memory-based methods naturally address this by decoupling knowledge retention from parameters, existing approaches designed for large language models (LLMs) rely on abundant storage and strong in-context reasoning that SLMs lack. To address these challenges, we propose MIITA, a Memory-Induced Inference-Time Adaptation framework for supervised CL under constrained storage. MIITA stores supervised experiences as compact correction-direction prototypes with semantic anchors, and retrieves them at inference time using semantic and uncertainty-based cues. The retrieved directions are applied through gated temporary hidden-state adaptation, enabling non-destructive reuse of past supervision without backbone updates, prompt extensions, or test-time backpropagation. A local theoretical analysis links this design to first-order loss reduction, uncertainty-guided retrieval, and directional coverage for retaining old-stage knowledge. Extensive experiments across diverse supervised CL settings show that MIITA consistently improves final performance and mitigates forgetting under fixed memory budgets.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Continual Learning, Small Language Models, Inference-Time Adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_95DDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MIITA addresses supervised continual learning for small language models under a fixed persistent-storage budget. It freezes the backbone and converts each supervised example into an averaged negative loss gradient in the pre-LM-head hidden space. Similar correction directions are consolidated into prototypes, with semantic anchors supporting retrieval. At inference, semantic similarity and a predictive-entropy gradient guide prototype selection; the aggregated direction is applied through a state-dependent gate during decoding.

The contributions are a correction-oriented memory representation, budget-aware memory management, and a lightweight inference-time adaptation mechanism. Validation covers four benchmarks spanning classification, multilingual QA, and instruction following, with additional model-size, storage-budget, retrieval, and gating analyses. The paper reports consistently better final performance and backward transfer than its evaluated baselines. Its theory provides local first-order interpretations and a conditional loss-reduction bound.

Critical strengths
  1. Computing correction vectors directly from the LM head avoids backpropagation through the backbone, while inference requires neither demonstrations in the prompt nor parameter optimization. Separating semantic retrieval keys from functional correction values is a useful design choice. This matters because it targets both storage and inference costs, rather than addressing forgetting alone.
  2. Table 1 reports improvements across all four benchmarks and both principal model sizes. Appendix D.1 strengthens the comparison by including byte-matched SAPT-LoRA and kNN-LM: MIITA exceeds SAPT-LoRA by 3.65 OP points on BANKING77 and 3.70 on SUPERNI. Table 2 also shows that combining semantic and uncertainty signals improves over either alone, while the gating and adversarial-retrieval experiments provide additional evidence for the proposed inference mechanism. These results make the approach promising beyond a single dataset or baseline.
Critical weaknesses
  1. The experimental reporting does not sufficiently establish the claimed learning–retention advantage or make the storage-matched comparisons auditable. The frozen backbone is reported as a latency reference, but its task performance is absent. Because MIITA preserves backbone weights, this reference is essential for measuring how much it actually learns beyond existing capabilities. Aggregate OP and BWT also conceal the immediate acquisition and subsequent retention of individual tasks; less-negative BWT can partly reflect lower initial acquisition.
  2. The central comparison depends on extremely small storage allocations, but the manuscript does not report the actual LoRA ranks and target modules, numerical storage precision, or per-method storage breakdowns. Appendix D.1 states that these quantities are controlled without providing the configurations. Task identities/orders, optimization settings, and the source of variability across the three runs are also insufficiently specified. These omissions materially limit confidence that the reported advantage reflects the method rather than particular baseline configurations.
  3. Insufficient positioning against function-vector methods and isolation of the proposed contribution. The paper omits discussion of function-vector-based continual learning methods [1], which directly connect compact activation-space vectors to continual learning, propose function-vector-guided training, and demonstrate that injecting original task function vectors during inference can recover performance after forgetting. It is therefore important prior work for MIITA’s claim of preserving and reusing functional adaptation signals.
  4. The memory-format ablation does not isolate the benefit of storing correction directions. ExampleMem and SummaryMem use prompt insertion, whereas DirectionMem uses hidden-state adaptation. Because both what is stored and how it is applied change, the results do not establish whether the gains come from correction directions, hidden-state adaptation, or their combination.
  5. Forward transfer to unseen tasks is not evaluated. The paper reports final performance and backward transfer (BWT), but not forward transfer (FWT). It therefore remains unclear whether accumulated correction memories improve or impair performance on future tasks before learning them, limiting the assessment of MIITA’s generalization beyond previously seen tasks.

[1] Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning. ICLR 2025.

Questions for authors
  1. What are the frozen backbone’s scores on all four benchmarks? Can you provide the stage-by-task performance matrices, or at least immediate post-learning and final scores for each task, to distinguish acquisition from retention?
  2. What exact storage precision, adapter ranks/target modules, and storage allocations were used for each baseline? Please also specify benchmark task selections/orders, validation-based hyperparameter selection, and what varies across the three runs?
  3. Does the reported MIITA use the union of semantic and directional candidates in Eq. (8), or the semantic-only candidate pool described in Appendix C.4? What values are used for the semantic-key layer, restricted vocabulary size, entropy-probe scale, and stability threshold, and what precisely determines the memory-repair operations?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 95DD (Part 1/3): W1 & Q1

We thank Reviewer 95DD for recognizing the value of separating semantic retrieval keys from functional correction values. Your comments helped us strengthen the evaluation of task acquisition, retention, and forward transfer, and improve the reproducibility of our experiments. We address each concern below.

W1 & Q1. Task acquisition and retention.

We have added the frozen-backbone scores to Table 1 for both model sizes across all four benchmarks, using the same prompts, test sets, decoding settings, and task metrics as MIITA. The table below summarizes acquisition and retention: Immediate is the mean score immediately after each task is learned (the diagonal mean), Final is the final task-averaged score (OP), and Final gain is MIITA final OP minus Frozen OP.

ModelDatasetFrozen OPMIITA immediateMIITA final OPFinal gain
Qwen3-0.6BBanking7732.1482.9677.67+45.53
Qwen3-0.6BDSC46.6783.8278.29+31.62
Qwen3-0.6BPAXQA17.5662.3560.01+42.45
Qwen3-0.6BSuperNI11.0245.1440.58+29.56
Qwen3-4BBanking7737.8985.3581.05+43.16
Qwen3-4BDSC58.9786.1281.66+22.69
Qwen3-4BPAXQA21.4467.0065.21+43.77
Qwen3-4BSuperNI13.4652.2048.21+34.75

MIITA retains final gains of 22.69–45.53 points over the frozen backbone. To present the 426 task-stage scores compactly, we have added all eight performance matrices as heatmaps in Appendix D.9, with a reference in Section 4.2. The diagonal shows immediate acquisition, each column tracks the same task through later stages, and the final row shows its retained performance.

For comparison, SAPT-LoRA's mean immediate scores, derived from its reported OP and BWT using A‾imm=OP−T−1TBWT\overline{A}_{\mathrm{imm}}=\mathrm{OP}-\frac{T-1}{T}\mathrm{BWT}, are 80.09 on Banking77 and 42.13 on SuperNI with Qwen3-0.6B. MIITA reaches 82.96 and 45.14, respectively, while its BWT is −6.17/−4.89 versus −7.08/−5.63 for SAPT-LoRA. Thus, these comparisons show higher initial acquisition together with smaller subsequent average declines. Appendix D.9 includes this analysis alongside the full matrices.

AuthorsOpenReview ↗

Response to Reviewer 95DD (Part 2/3): W2-W3 & Q2

W2 & Q2. Experimental configurations and reproducibility.

(1) Configurations and storage accounting. Appendix C.2 reports the baseline adaptations, adapter configurations, and counted storage components. For SuperNI with Qwen3-0.6B, persistent numerical arrays use FP32 and the shared storage cap is 133,240 bytes:

MethodConfigurationSerialized storage (bytes)
LoRA (sequential)Rank 1; last six layers, q/v projections128,880
Per-task LoRARank 1; last-layer v projection; 15 adapters; oracle task ID122,940
SAPT-LoRARank 1; last-layer v projection; 15 adapters; 32-D keys; 32→8→32 selector129,884
MBPA++Frozen-SLM retrieval; temporary local LoRA129,322
CIS256-token summary cap; k=497,824
InCA64-D tags; k=4; ρ=0.01; 256-token summary cap117,762
CT0Rank 1; last-layer q/v projections; replay129,578
FOREVERLoRA with model-time replay scheduling133,114
DKVB128 subspaces; 16 pairs/subspace; 8-D keys/values133,240
Srinath K et al.Shared replay-adapter; task/domain replay127,896
MIITAMain-paper configuration127,123

Totals include all method-specific persistent state and serialization metadata, excluding the shared frozen backbone. For per-task LoRA and SAPT-LoRA, all retained task adapters count toward the global cap; SAPT-LoRA additionally includes its selection and reflection state. Appendix D.1 references these configurations.

(2) Task protocols, tuning, and repeated runs. Banking77 follows the official categories.json order, grouped consecutively into seven stages of 11 intents. DSC follows DKVB's ten-domain partition with positive/negative labels, ordered as Tools and Home Improvement, Pet Supplies, Office Products, Kindle Store, Movies and TV, Video Games, Musical Instruments, Sports and Outdoors, Patio Lawn and Garden, and Toys and Games. The SuperNI task order is task748, task073, task1590, task639, task1572, task1687, task591, task363, task1510, task1729, task181, task511, task002, task1290, task875. PAXQA uses EN → AR → ZH → RU. Within each benchmark, all methods share the task partition and order.

Optimized baselines use Adam, learning rate 10−410^{-4}, effective batch size 16, and 10 epochs per stage. We hold out 10% of each training stage for validation, filter configurations by the storage cap, and select by mean validation score across observed tasks and checkpoints. Final runs use the full original training stream and fixed configurations; only the random seed varies. Appendices C.1 and C.2 specify these protocols, including the CIS, InCA, and DKVB search ranges.

W3. Relationship to function-vector-based continual learning.

Jiang et al. [1] provide closely related evidence that injecting source-task function vectors can restore performance after continual tuning, and introduce function-vector-guided (FVG) training with consistency and KL losses. Their function vectors summarize demonstration-induced attention-head activations. MIITA instead derives correction directions from supervised prediction errors and continually consolidates and retrieves them under a fixed storage cap, while keeping the backbone frozen. We have expanded Section 2 to explain this connection and the distinct memory construction and adaptation mechanisms.

We have also added LoRA + FVG to Appendix D.8 by augmenting our sequential LoRA baseline with the two FVG losses. The comparison uses Qwen3-0.6B, the same task streams and training settings, and each dataset's 1% persistent-storage cap, including all retained adapters, function vectors, and other method-specific state. Results are means over three independent runs.

MethodBanking77 OP ↑Banking77 BWT ↑SuperNI OP ↑SuperNI BWT ↑
Sequential LoRA65.31-11.4431.24-8.91
LoRA + FVG69.52-9.1433.79-7.21
MIITA77.67-6.1740.58-4.89

Adding FVG improves OP over sequential LoRA by 4.21 and 2.55 points on Banking77 and SuperNI, respectively, with BWT improvements of 2.30 and 1.70 points. MIITA achieves further OP gains of 8.15 and 6.79 points over LoRA + FVG, and BWT gains of 2.97 and 2.32 points, showing higher final task performance and less average forgetting in this storage-constrained comparison.

Appendix D.8 also reports comparisons with other related vector-based methods under the same evaluation protocol and storage budget.

[1] Jiang G, Jiang C, Li Z, et al. Unlocking the power of function vectors for characterizing and mitigating catastrophic forgetting in continual instruction tuning. ICLR 2025.

AuthorsOpenReview ↗

Response to Reviewer 95DD (Part 3/3): W4-W5 & Q3

W4. Isolating the contribution of correction-direction memory.

The relevant controlled comparison in Figure 4(a) is SemanticMem versus DirectionMem. SemanticMem replaces correction-direction vectors with semantic prototype vectors while keeping the retrieval, aggregation, and temporary hidden-state adaptation procedures unchanged. Under the same storage budget, DirectionMem achieves higher OP and less forgetting than SemanticMem. This comparison supports the benefit of storing supervised correction information when the adaptation mechanism is held fixed.

ExampleMem and SummaryMem provide complementary comparisons with prompt-based textual memory. We have revised Section 4.5 to distinguish these comparisons and clarified SemanticMem's shared adaptation pipeline in Appendix C.3.

W5. Forward transfer to unseen tasks.

We evaluated forward transfer on SuperNI with Qwen3-0.6B under the 1% persistent-storage budget. For each task i = 2, ..., 15, MIITA is evaluated immediately before learning that task, using only memory accumulated from preceding tasks. The frozen backbone is evaluated on the same test sets with the same prompts and decoding settings. FWT is the mean difference between these pre-learning scores and the corresponding frozen-backbone scores.

MethodMean pre-learning ROUGE-L ↑FWT ↑
Frozen backbone11.270.00
MIITA12.78+1.51

Both means cover tasks 2–15. The positive FWT of 1.51 points shows that accumulated correction memory provides a modest average benefit on unseen tasks before receiving their supervision. We have added the protocol and results to Appendix D.10 and summarized the finding in Section 4.2.

Q3. Retrieval configuration and memory repair.

MIITA uses the union in Eq. (8): the top Rk=8R_k=8 semantic candidates and top Rd=8R_d=8 directional candidates are combined with duplicates removed, reranked by Eq. (9), and the top M=4M=4 prototypes are aggregated. We have corrected Appendix C.4 to describe this procedure.

Both memory and query semantic keys use the final hidden layer before the LM head, with mean pooling over input tokens and normalization. We use Kv=64K_v=64, probe scale ϵ=1.0\epsilon=1.0 in Eq. (9), and stability threshold ϵr=10−12\epsilon_r=10^{-12}. The directional retrieval branch is enabled when ∥rq∥2>ϵr\|r_q\|_2>\epsilon_r; otherwise, candidates come from semantic retrieval. The key-layer choice is explicit in Section 3.2.2, and Appendix B.2 now specifies these parameter values.

Memory repair follows Section 3.2.3 and Algorithm 1 and is triggered when a memory write exceeds the storage cap. It first removes an anchor from the most similar pair within a prototype when its cosine is at least τk=0.80\tau_k=0.80, retaining at least one anchor per prototype. If no such pair exists and at least two prototypes remain, it merges the pair with the highest directional cosine using their support counts as weights. If the merged prototype alone exceeds the cap, its most redundant anchors are further pruned until it fits or only one anchor remains. Eviction selects the minimum-value prototype under Eq. (5) when neither pruning nor merging is available. The footprint is rechecked after each repair until the cap is satisfied. Appendix B.2 now details these selection and transition rules.

Thank you for the careful review and constructive suggestions. We welcome further discussion of these results and clarifications.

Reviewer_yMVmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MIITA, a memory-based framework for continual learning with small language models. It stores supervised experiences as correction-direction prototypes and retrieves them to adjust hidden states at inference time, while keeping the backbone frozen. Experiments on classification, multilingual QA, and instruction-following benchmarks show improved performance and reduced forgetting under limited storage budgets.

Critical strengths
  • The paper is well written and easy to follow. The motivation and the main components of the method are clearly explained.

  • The experiments are comprehensive, covering multiple benchmarks and model sizes. The additional baseline comparisons and ablation studies help demonstrate the effectiveness of the proposed method.

Critical weaknesses

I have two main concerns:

  1. My first concern is that the proposed method appears closely related to activation-steering methods, but the paper does not discuss or compare with this line of work.

In particular, storing vectors and retrieving them for inference-time intervention has already been explored. I can see differences in how MIITA constructs correction directions, manages its storage budget, and uses uncertainty for retrieval. However, I am not yet convinced how much these choices add over a related vector-memory approach adapted to the same setting.

Relevant work includes, but is not limited to:

[1] Function Vectors in Large Language Models. ICLR 2024.

[2] In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering. ICML 2024.

[3] ELICIT: LLM Augmentation via External In-Context Capability. ICLR 2025.

[4] Dynamic Steering With Episodic Memory For Large Language Models. Findings of ACL 2025.

  1. I also find it difficult to tell how much MIITA learns beyond the frozen backbone.

I could not find the backbone’s task scores without memory, which would help assess the gains attributable to memory and how well those gains are retained throughout the stream. The budget-matched comparisons in Appendix D.1 are helpful, but I would also like to see the actual adapter ranks, target modules, and storage usage. With such a small budget, these choices could strongly affect the baselines’ performance. Without these details, I am less confident in the broader conclusion that MIITA provides a better way to reuse experience than parameter-based methods.

Questions for authors
  1. Which hidden layer is used to extract the semantic keys in Eq. (1)? I could not find the choice of l_k in the implementation details.
  2. Could you clarify how the candidate set is constructed in the full method? Eq. (8) takes the union of semantic and directional retrieval results, while Appendix C.4 describes semantic recall followed by uncertainty-based reranking. Which procedure is used in the experiments?
5 comments
AuthorsOpenReview ↗

Response to Reviewer yMVm (Part 1/2): W1

We thank Reviewer yMVm for recognizing the clarity of our presentation and the breadth of our experiments. Your comments helped us strengthen the comparison with related vector-based methods and clarify learning gains and implementation details. We address each concern below.

W1. Relationship to activation steering and vector-memory methods.

Thank you for highlighting this closely related line of work and suggesting a direct comparison. Activation steering guides model behavior through interventions on internal representations, and related vector-memory methods store and retrieve intervention signals for reuse. MIITA shares this inference-time intervention perspective, while focusing on the continual accumulation and reuse of supervision in small language models under a fixed persistent-storage budget. Our motivation is to retain reusable correction information while reducing historical-text storage and reliance on in-context reasoning. This leads to prediction-error-derived directions, consolidation by correction similarity, and uncertainty-guided retrieval.

We compared MIITA with all four methods cited in the review and an additional LoRA + FVG baseline [5] on Banking77 and SuperNI, using Qwen3-0.6B, the same continual-learning protocols, and each dataset's 1% persistent-storage cap. LoRA + FVG augments sequential LoRA with function-vector consistency and FV-guided KL losses. All retained method-specific state is counted toward the cap. Results are means over three independent runs.

MethodBanking77 OP ↑Banking77 BWT ↑SuperNI OP ↑SuperNI BWT ↑
FV [1]69.23-9.2131.12-8.52
ICV [2]68.17-9.7630.09-8.77
ELICIT [3]71.31-8.7534.55-7.39
DSEM [4]63.22-11.1229.88-10.21
LoRA + FVG [5]69.52-9.1433.79-7.21
MIITA77.67-6.1740.58-4.89

ELICIT achieves the highest OP among the added baselines on both datasets, while LoRA + FVG achieves the least negative baseline BWT on SuperNI. MIITA improves OP over ELICIT by 6.36 and 6.03 points on Banking77 and SuperNI, respectively. Compared with LoRA + FVG, MIITA improves OP by 8.15 and 6.79 points and BWT by 2.97 and 2.32 points. MIITA therefore achieves the highest final task performance and least average forgetting among the evaluated methods in both settings.

We have expanded Section 2 to discuss this research direction, including the four cited works and function-vector-guided continual learning, and added the comparison to Appendix D.8. Because the discussion timeline allowed us to complete this additional comparison on two of the four benchmarks, we present it separately in the appendix for now; we would like to extend its coverage and integrate it into the main experimental section in a subsequent revision.

[5] Jiang G, Jiang C, Li Z, et al. Unlocking the power of function vectors for characterizing and mitigating catastrophic forgetting in continual instruction tuning. ICLR 2025.

AuthorsOpenReview ↗

Response to Reviewer yMVm (Part 2/2): W2 & Q1-Q2

W2. Gains beyond the frozen backbone and adapter configurations.

(1) Gains beyond the frozen backbone.

We have added the frozen-backbone results directly to Table 1 for both model sizes across all four datasets. The frozen models use no memory or hidden-state adaptation and share MIITA's prompts, test sets, decoding settings, and task-specific metrics. OP is averaged over tasks in the same way as for MIITA. Values below are mean ± standard deviation over repeated runs under the shared evaluation protocol; Increase is the difference in mean OP (MIITA − Frozen), in score points.

ModelDatasetFrozen OPMIITA OPIncrease (points)
Qwen3-0.6BBanking7732.14 ± 0.3377.67 ± 0.26+45.53
Qwen3-0.6BDSC46.67 ± 0.2778.29 ± 0.24+31.62
Qwen3-0.6BPAXQA17.56 ± 0.2260.01 ± 0.21+42.45
Qwen3-0.6BSuperNI11.02 ± 0.1940.58 ± 0.27+29.56
Qwen3-4BBanking7737.89 ± 0.2881.05 ± 0.22+43.16
Qwen3-4BDSC58.97 ± 0.3181.66 ± 0.25+22.69
Qwen3-4BPAXQA21.44 ± 0.2365.21 ± 0.19+43.77
Qwen3-4BSuperNI13.46 ± 0.2448.21 ± 0.20+34.75

MIITA improves OP by 22.69–45.53 points across all eight settings, showing that its accumulated correction memory provides substantial gains beyond the frozen backbone's existing task performance. Together with MIITA's BWT of −6.17 to −2.38 in Table 1, these results support its ability to benefit from new supervision while limiting average forgetting. We have added these analyses to Section 4.2.

(2) Adapter configurations and storage accounting.

Appendix C.2 now specifies the adapter configurations and extends the shared configuration/storage table to include the additional adapter baselines from Appendix D.1 on SuperNI with Qwen3-0.6B. Persistent numerical arrays use FP32, and all methods use the same 133,240-byte cap.

MethodConfigurationSerialized storage (bytes)
LoRA (sequential)Rank 1; last six layers, q/v projections128,880
Per-task LoRARank 1; last-layer v projection; 15 adapters; oracle task ID122,940
SAPT-LoRARank 1; last-layer v projection; 15 adapters; 32-D keys; 32→8→32 selector129,884
MBPA++Frozen-SLM retrieval; temporary local LoRA129,322
CIS256-token summary cap; k=497,824
InCA64-D tags; k=4; ρ=0.01; 256-token summary cap117,762
CT0Rank 1; last-layer q/v projections; replay129,578
FOREVERLoRA with model-time replay scheduling133,114
DKVB128 subspaces; 16 pairs/subspace; 8-D keys/values133,240
Srinath K et al.Shared replay-adapter; task/domain replay127,896
MIITAMain-paper configuration127,123

For sequential LoRA, we count the shared adapter carried across stages. For per-task LoRA, we count all 15 task adapters. For SAPT-LoRA, we count all retained adapters together with the selection and reflection state. All reported totals include serialization metadata and exclude the shared frozen backbone. Appendix C.2 specifies the corresponding configurations, counted components, and validation protocol for selecting budget-feasible configurations.

At this storage budget, MIITA exceeds SAPT-LoRA, the strongest non-oracle PEFT baseline in Appendix D.1, by 3.70 OP points and 0.74 BWT points on SuperNI. These results support the benefit of direction memory in the evaluated storage-constrained setting. Appendix D.1 now points to the SuperNI adapter details in Appendix C.2.

Q1. Layer used for semantic keys.

We use the final hidden states before the LM head to construct semantic keys. Specifically, we mean-pool the representations over input-token positions and apply the normalization in Eq. (1). Memory keys and query keys use the same layer and pooling procedure. We have made this choice explicit in Section 3.2.2.

Q2. Candidate construction and reranking.

The experiments use the union-based procedure in Eq. (8). We retrieve the top Rk=8R_k=8 prototypes by semantic relevance and the top Rd=8R_d=8 by alignment with the uncertainty-induced direction, take their union with duplicates removed, and rerank the candidates using λkSk+λuU\lambda_k S_k+\lambda_u U in Eq. (9). The top M=4M=4 prototypes are aggregated using Eq. (10). If the directional proxy is unavailable, candidate construction uses only semantic retrieval.

The description in Appendix C.4 omitted the directional-retrieval branch. We have corrected the SemUncRet description and aligned the related analysis in Appendix C.4 and Section 4.6 with the procedure in Eqs. (8)–(10).

Thank you for your helpful feedback. We welcome further discussion and would be happy to clarify any remaining questions.

AuthorsOpenReview ↗

Follow-up to Reviewer yMVm

Dear Reviewer yMVm,

Thank you for highlighting the connection to activation steering and vector-memory methods. We expanded the related-work discussion and added comparisons with all four suggested methods and LoRA + FVG under the same continual-learning protocols and storage caps. We also provided frozen-backbone scores, adapter configurations, and clarifications of semantic-key extraction and candidate construction.

With the discussion deadline approaching, we would appreciate your assessment of whether these revisions clarify MIITA’s contribution relative to related vector-based approaches and its gains beyond the frozen backbone. We remain available to discuss any outstanding concerns.

Best regards,

The authors of Submission 27508

Reviewer_GQnFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MIITA converts supervised examples into compact correction directions, merges them into a bounded memory, and retrieves directions to steer a frozen language model at inference. Retrieval combines semantic similarity with an uncertainty-derived signal. The storage-constrained continual-learning problem remains consistent throughout the introduction, methodology and conclusion. The empirical study is the main reason to view the contribution favorably.

Critical strengths
  1. The storage-oriented design is well motivated for small models, and the evaluation extends beyond a single favorable benchmark.

  2. Results cover four benchmarks, multiple model sizes, memory budgets and alternative memory representations.

  3. The appendix adds PEFT/kNN comparisons, direction-alignment diagnostics and gate/layer ablations.

  4. Persistent storage is budgeted in bytes, including auxiliary state. Inference costs are measured: the reported SuperNI example increases from 615 to 721 ms/query.

  5. The empirical evidence gives the method value even though several individual components are standard.

Critical weaknesses

The mechanism should be characterized more precisely. Because the selected update is applied immediately before a linear LM head, it is equivalent to a retrieved, token-gated logit bias. A strong output-steering baseline is therefore particularly important. Comparisons with example and semantic memories do not fully isolate this alternative.

The local descent argument requires alignment with the supervised loss gradient, whereas inference uses an entropy proxy. The appendix provides useful average-case diagnostics, but confidently wrong predictions and contradictory memories remain meaningful failure cases.

Finally, adaptations of classification-oriented baselines to generation require sufficient implementation and tuning detail to make the comparisons reproducible.

Questions for authors
  1. Does MIITA outperform a retrieved logit-bias or token-distribution correction baseline at identical storage and retrieval cost?

  2. How reliably does the uncertainty proxy avoid harmful corrections on confidently wrong or conflicting queries?

  3. Can the authors provide exact serialized-byte accounting and tuning protocols for the adapted generative baselines?

2 comments
AuthorsOpenReview ↗

Response to Reviewer GQnF

We thank Reviewer GQnF for the thoughtful assessment and for recognizing the motivation behind storage-constrained continual learning for small models and the breadth of our empirical evaluation. Your comments helped us clarify the inference mechanism and examine uncertainty guidance in difficult cases. We address your comments below.

W1 & Q1. Mechanism characterization and output-correction comparison.

With a linear LM head, Eq. (11) is exactly equivalent to a query-specific, token-gated logit bias:

zt′=zt+ηαtWLMdˉq.z_t^{\prime} = z_t + \eta \alpha_t W_{LM} \bar{d}_q.

Here, zt=WLMhtoutz_t = W_{LM} h_t^{out} denotes the original logits.

MIITA constructs this bias from compressed supervised correction directions, with the gate controlling its strength at each decoding step.

For the token-distribution correction alternative, we supplement the kNN-LM [1] results in Appendix D.1 with measured costs on SuperNI using Qwen3-0.6B under the same 1% persistent-storage cap:

MethodStorage (MB)Retrieval (ms/query)OP ↑BWT ↑
kNN-LM0.13315734.92−7.20
MIITA0.12713140.58−4.89

OP and BWT are the three-run means reported in Appendix D.1. MIITA improves OP by 5.66 points and BWT by 2.31 points while using less storage and lower retrieval latency. We have made the equivalence explicit in Section 3.2.4 and added this cost comparison to Appendix D.1.

[1] Khandelwal U, Levy O, Jurafsky D, et al. Generalization through memorization: Nearest neighbor language models. ICLR 2020.

W2 & Q2. Reliability on confidently wrong and conflicting queries.

The local loss-decrease result assumes alignment with the supervised correction direction. Appendix D.2 provides empirical evidence for uncertainty guidance through U-Err (AUROC 0.72) and U-Gain (Spearman correlation 0.53).

We further evaluate 100 randomly sampled held-out queries per case on SuperNI with Qwen3-0.6B under the 1% storage budget. Confidently wrong queries have low entropy and an incorrect answer-start token prediction. Conflicting queries contain candidate prototypes with opposing effects on reference-answer loss. We compare MIITA with an ablation setting λu=0\lambda_u=0, keeping the candidate pools, aggregation rule, adaptation scale, and gate fixed. Both variants use the same queries. Loss reduction and loss-increase rates are measured relative to the frozen model under teacher forcing.

Query subsetMethodMean loss reduction ↑Loss increasesRate ↓
Confidently wrongw/o uncertainty scoring0.0043636.00%
Confidently wrongMIITA0.0091818.00%
Conflicting effectsw/o uncertainty scoring0.0122828.00%
Conflicting effectsMIITA0.0201010.00%

Uncertainty scoring increases the mean loss reduction and lowers the loss-increase rate by 18 percentage points in each sampled subset. Complementarily, Appendix D.5 shows that gating mitigates adverse retrieved directions: at 25% adversarial prototype replacement, gated adaptation achieves 38.63 OP versus 35.62 without gating. We have added the targeted diagnostic protocol and results in a dedicated subsection, Appendix D.7, and summarized the findings in Section 4.6.

W3 & Q3. Baseline adaptation, tuning, and storage accounting.

We have expanded Appendix C.2 to clarify the generative adaptations and tuning protocols. CIS retrieves task/domain summaries for prompting; InCA selects task/domain memory groups through its external learner; DKVB applies retrieved value vectors before the frozen LM head and trains them with reference-token cross-entropy. Appendix C.2 also describes the LoRA-based adaptations of MBPA++, CT0, and FOREVER, and the task/domain replay adaptation of Srinath K et al.

Hyperparameters are selected on training-derived validation data using the mean validation score across observed tasks and checkpoints, then fixed for the final runs. We have specified the validation split and the search ranges for CIS, InCA, and DKVB.

We report the serialized storage of all methods in the main comparison on SuperNI with Qwen3-0.6B under the same 1% persistent-storage budget:

MethodSerialized storage (bytes)
LoRA128,880
MBPA++129,322
CIS97,824
InCA117,762
CT0129,578
FOREVER133,114
DKVB133,240
Srinath K et al.127,896
MIITA127,123

All methods remain within the same 133,240-byte cap. Totals include method-specific persistent state and serialization metadata; the shared frozen backbone is excluded. Appendix C.2 lists the configurations and counted components.

Thank you again for your constructive feedback; we welcome any further questions or suggestions.

AuthorsOpenReview ↗

Follow-up to Reviewer GQnF

Dear Reviewer GQnF,

Thank you again for your careful assessment. We clarified the logit-bias equivalence, supplemented the kNN-LM comparison with measured storage and retrieval costs, and added diagnostics on confidently wrong and conflicting queries. We also expanded the baseline adaptation and tuning details.

As the discussion period draws to a close, we would particularly value your feedback on whether these additions address your concerns about the inference mechanism and the reliability of uncertainty guidance. We welcome further discussion and would be happy to clarify any remaining questions.

Best regards,

The authors of Submission 27508

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their thoughtful feedback, which helped us clarify the presentation and strengthen the experimental evidence. We summarize the revisions below and indicate the corresponding reviewer comments for each substantive change.

  1. Expanded the related-work discussion in Section 2 and added comparisons with related vector-based methods in Appendix D.8.

    In response to Reviewer yMVm W1 and Reviewer 95DD W3.

  2. Clarified the inference mechanism in Section 3.2.4 and supplemented the output-correction comparison with storage and retrieval-cost measurements in Appendix D.1.

    In response to Reviewer GQnF W1 & Q1.

  3. Added frozen-backbone references to Table 1 and further analysis of task acquisition and retention in Section 4.2 and Appendix D.9.

    In response to Reviewer yMVm W2 and Reviewer 95DD W1 & Q1.

  4. Expanded experimental configurations, baseline adaptations, tuning protocols, and storage accounting in Appendices C.1, C.2, and D.1.

    In response to Reviewer GQnF W3 & Q3, Reviewer yMVm W2, and Reviewer 95DD W2 & Q2.

  5. Added targeted diagnostics of uncertainty guidance on challenging queries in Appendix D.7, with findings summarized in Section 4.6.

    In response to Reviewer GQnF W2 & Q2.

  6. Clarified the design and interpretation of the memory-variant comparison in Section 4.5 and Appendix C.3.

    In response to Reviewer 95DD W4.

  7. Added a forward-transfer evaluation and aligned the accompanying acquisition and retention discussion in Section 4.2 and Appendices D.9-D.10.

    In response to Reviewer 95DD W5 and W1 & Q1.

  8. Clarified the retrieval configuration, candidate construction, and memory-repair procedure in Sections 3.2.2 and 4.6 and Appendices B.2 and C.4.

    In response to Reviewer yMVm Q1 & Q2 and Reviewer 95DD Q3.

For ease of reference, substantive additions and revisions relative to the original submission are highlighted in blue.

AuthorsOpenReview ↗

Thank You and Rebuttal Summary

We thank the reviewers and the Area Chair for their time and constructive feedback. As the discussion draws to a close, we summarize MIITA’s contribution, the additional evidence provided during rebuttal, and the feedback we received on the revision.

Main Contributions

MIITA’s central contribution is to use the correction a frozen small language model needs as the unit of continual memory. The same direction determines which supervised experiences can share storage and gives the model an intervention it can apply to a later prediction. Examples that differ in meaning can share a prototype when they require similar corrections, while distinct semantic anchors keep that prototype accessible from different contexts. This design lets a small model retain supervision under a fixed byte budget and reuse it without having to infer the intended correction from retrieved text. Across four benchmarks spanning classification, multilingual question answering, and instruction following, MIITA achieves higher final performance and less forgetting than the continual-learning baselines in Table 1 at both model sizes.

We are pleased that the reviews recognized the motivation for this design, the breadth of the evaluation, and the clarity of the presentation:

Motivation and Method Design

  • “The storage-oriented design is well motivated for small models, and the evaluation extends beyond a single favorable benchmark.”
  • “Separating semantic retrieval keys from functional correction values is a useful design choice.”

Evaluation

  • “The empirical study is the main reason to view the contribution favorably.”
  • “The experiments are comprehensive, covering multiple benchmarks and model sizes.”
  • “The additional baseline comparisons and ablation studies help demonstrate the effectiveness of the proposed method.”
  • “These results make the approach promising beyond a single dataset or baseline.”

Presentation

  • “The storage-constrained continual-learning problem remains consistent throughout the introduction, methodology and conclusion.”
  • “The paper is well written and easy to follow. The motivation and the main components of the method are clearly explained.”

Main Concerns Addressed

The reviewers’ comments helped us clarify where MIITA contributes to prior work and strengthen the evidence for what it learns and retains. We summarize the main concerns and our responses below.

1. Contribution beyond activation steering and vector memory

We compared MIITA with five related baselines under each dataset’s 1% cap. MIITA scored highest on Banking77 and SuperNI. With retrieval, aggregation, and intervention fixed on SuperNI, correction prototypes outperformed semantic prototypes and reduced forgetting, showing the value of storing how predictions should change.

2. What does MIITA learn beyond the frozen backbone, and could weaker initial learning explain lower forgetting?

We evaluated the frozen backbone under matched conditions and tracked each task through the stream. MIITA’s final gains ranged from 22.69 to 45.53 points over the frozen backbone across eight settings. On Banking77 with Qwen3-0.6B, MIITA scored higher than SAPT-LoRA on average immediately after learning and lost fewer points thereafter, ruling out weaker initial learning in this comparison.

3. Fairness and reproducibility under the storage budget

We documented baseline adaptations, validation splits, tuning ranges, and the selected adapter ranks and target modules. We listed each method’s persistent components and serialized byte total, including metadata, so readers can verify it fits the shared cap.

4. Retrieval reliability on difficult queries

We reranked a fixed union of semantic and directional candidates with and without uncertainty scoring. With scoring, reference-answer loss exceeded the frozen baseline less often in both SuperNI subsets (confidently wrong predictions and conflicting candidate effects), by 18 percentage points each. The signal thus helped avoid harmful corrections.

Reviewers’ Follow-Up Comments

We appreciate that all three reviewers followed up. Reviewer GQnF maintained their positive assessment, and Reviewer 95DD indicated that the reply had “resolved most of my concerns” and raised their assessment. Reviewer yMVm acknowledged that the revisions “have addressed several of my concerns and strengthened the empirical evaluation,” while retaining a concern about the significance of the methodological advance. In response, we examined recent work beyond the examples cited in the review and explained why organizing memory around the corrections a particular small model needs matters for both storage and prediction.

Thank you again for your careful consideration and for the opportunity to strengthen the paper. We hope the revised manuscript and discussion give a clearer account of MIITA’s contribution and the evidence supporting it.

Best regards,
The authors of Submission 27508

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.