acceptodds
Under review as a conference paper at ICLR 2027

Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA

Abstract

Reinforcement learning (RL)-based post-training has become an effective approach for eliciting reasoning capabilities in large language models (LLMs). However, adapting post-RL models to new knowledge domains or behaviors through subsequent supervised fine-tuning (SFT) can severely overwrite these capabilities. Existing approaches mitigate such forgetting through experience replay, specialized initialization, or constrained optimization using gradient projection, but either provide limited preservation or incur substantial training overhead. Our analysis shows that reasoning activations concentrate in low-dimensional subspaces, leaving substantial null-space capacity for adaptation, and that the corresponding approximate null spaces can be reliably estimated from a modest number of examples. Motivated by these observations, we propose Null-Basis Low-Rank Adaptation (NB-LoRA), a parameter-efficient method for adapting post-RL LLMs while preserving their acquired reasoning ability. We formulate reasoning retention as a layer-wise hidden-state preservation constraint and construct a fixed approximate null basis from reasoning activations. LoRA updates are then reparameterized through this basis, enforcing the preservation constraint throughout fine-tuning. Extensive experiments across multiple RL-trained LLMs and diverse downstream tasks show that NB-LoRA matches standard LoRA in adaptation performance, maintains reasoning accuracy near pre-fine-tuning levels, and generalizes this preservation to held-out reasoning benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Null-Basis LoRA, parameter-efficient fine-tuning, reasoning preservation, post-RL large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gsYxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a practical problem: how can a model that has gained reasoning ability through reinforcement learning retain that ability while learning new tasks? The authors propose NB-LoRA. It first records the inputs to each layer while the model solves reasoning problems, then identifies directions with weaker activation signals. Subsequent fine-tuning restricts updates to these directions to reduce interference with the original reasoning process. The authors evaluate Qwen2.5-1.5B and 3B on eight commonsense tasks and tool use. NB-LoRA achieves new-task performance close to standard LoRA while largely maintaining reasoning accuracy at its pre-fine-tuning level. The paper also evaluates other reasoning benchmarks, different parameter settings, and two-stage sequential learning.

Critical strengths
  1. The method strikes a good balance between learning new tasks and retaining existing abilities. In Table 2, the 3B model achieves 84.6% average new-task accuracy, close to the 84.7% achieved by standard LoRA. Its reasoning accuracy remains at 63.2%, compared with 63.8% before fine-tuning, whereas standard LoRA drops to 13.9%. The 1.5B model shows similar results. This is the paper's most convincing contribution.
  2. The method is simple and offers efficiency benefits. NB-LoRA restricts update directions through its structure, avoiding full-gradient projection at every step. Figure 4 reports approximately 6.3× faster training than NSCL on the 3B model, with roughly half the peak GPU memory usage. The authors also examine cross-benchmark retention, anchor-set size, and sequential learning, providing a fairly broad evaluation.
Critical weaknesses
  1. The claim of exact preservation goes beyond what the implemented method guarantees. Section 3.3 uses an approximate null space, but Eq. (5) states an exact equality to zero. The selected directions still contain some activation signal, so updates generally have a nonzero effect on the original inputs. Large updates could amplify this effect. Updating several layers at once also changes the inputs to later layers. The authors should distinguish the ideal mathematical result from the approximate behavior of the actual algorithm. Appendix B addresses optimization convergence, which does not directly establish that reasoning ability will be preserved. Its gradient-descent conditions also differ from the Adam optimizer used in the experiments.
  2. Some experimental details and conclusions need further explanation. Appendix D mainly describes NB-LoRA's settings and gives limited information about baseline tuning, replay ratios, and SDFT generation settings. Table 3 does not identify the downstream adaptation task either. In Table 2, replay scores 0% on tool use for the 1.5B model, while SDFT achieves only 11.9% average new-task accuracy. These low scores need explanation before the fairness of the comparison can be assessed. The threshold results are also not stable across all tasks. In Table 6, the 3B model's reasoning accuracy after HellaSwag adaptation is 53.68% at threshold 0.90, about 10 percentage points below the original 63.8%. Looking only at averages overlooks this substantial drop.
Questions for authors
  1. Approximate constraints: How should Eq. (5) be stated for the actual 99% energy threshold? Could an error analysis or direct measurements show how much each layer's output changes after training? This would help assess the method's protective effect.

  2. Experimental comparisons: Which downstream adaptation task was used in Table 3? Please provide the training settings and evaluation rules for the main baselines, and explain replay's zero tool-use score and SDFT's low scores. In particular, could response formatting or answer extraction have affected these results?

  3. Stability: How do the authors explain the substantial HellaSwag drop at threshold 0.90 in Table 6? Should the claim of threshold insensitivity be qualified? Variation across tasks in Table 2 also does not capture variation across repeated training runs. If results from different random seeds are available, please report them.

  4. Costs: Does Figure 4 include the preparation costs of generating reasoning traces and computing the null-space bases? How much extra time and GPU memory does NB-LoRA require compared with standard LoRA under the same settings? This information would help readers assess its practical cost.

3 comments

We thank the reviewer for their careful feedback.

W1 & Q1. Approximate preservation

Threshold-based approximate subspaces are commonly used in projection-based continual learning to balance preservation and adaptation, including GPM [R1] and Adam-NSCL [R2]. Following this common practice, we protect dominant activation directions using a 99% spectral-energy threshold and adapt in the remaining low-energy directions. This limits interference but does not guarantee unchanged outputs: large updates and activation changes propagated from earlier layers can still affect reasoning.

Our experiments support the strategy's practical effectiveness: across nine adaptation tasks on Qwen2.5-3B, NB-LoRA achieves 84.6% target accuracy and 63.2% retained reasoning accuracy, versus 84.7% and 13.9% for vanilla LoRA (post-RL reasoning reference: 63.8%). This demonstrates effective retention in the evaluated setting, without guaranteeing exact preservation. Appendix B concerns gradient-descent convergence, not Adam convergence or reasoning retention; subspace membership holds independently of the optimizer.

We will correct Eq. (5) and the exact-preservation wording, cite these precedents and clarify the limits of approximate preservation, and distinguish empirical retention from optimization convergence.

[R1] Saha et al. Gradient Projection Memory for Continual Learning. ICLR 2021.

[R2] Wang et al. Training Networks in Null Space of Feature Covariance for Continual Learning. CVPR 2021.

W2 & Q2. Table 3 and baseline reproducibility

We agree that Table 3 must identify its downstream adaptation task and checkpoint, which require verification from the original evaluation records. We will also report method-specific learning rates and tuning ranges, trainable modules, ranks, optimizers, training steps, replay buffer sizes/mixing ratios, and SDFT teacher prompts and generation settings.

Replay's 0% tool-use score on the 1.5B model corresponds to 0/62 correctly scored calls. This and SDFT's 11.9% average adaptation accuracy require checking raw outputs, parsing/extraction, truncation, and teacher-rollout quality. Aggregate scores alone cannot establish whether these reflect task errors or formatting failures.

We will clarify Table 3, supply the baseline configurations and evaluation rules, and audit these outcomes before attributing them to intrinsic method limitations. Confirmed errors will be corrected for all affected methods.

W2 & Q3. Threshold sensitivity and run variation

We agree that threshold sensitivity is task-dependent. After HellaSwag adaptation on Qwen2.5-3B, reasoning accuracy falls from 63.80% to 53.68% at threshold 0.90, whereas 0.99 retains 63.54%. We will explicitly highlight this failure and remove the broad threshold-insensitivity claim. Accuracy need not vary monotonically with threshold because independently trained adapters can differ in update norms and optimization trajectories; the specific cause requires repeated runs and residual measurements.

Variation across tasks also does not measure run-to-run uncertainty or establish the significance of small method differences.

We will label task-wise dispersion accurately and avoid statistical-significance claims unsupported by independent runs.

Q4. Preparation costs and LoRA overhead

Figure 4 compares training efficiency with full-weight NSCL, reporting 627.72 s/11.23 GiB for NB-LoRA versus 3,924.93 s/23.62 GiB for NSCL on 3B. The current implementation excludes rollout and basis preparation from its adaptation timer; the original benchmark boundaries still require verification. These results do not establish an end-to-end speedup or overhead relative to vanilla LoRA.

The full cost is Trollout+Tactivation/basis+TadaptationT_{\mathrm{rollout}}+T_{\mathrm{activation/basis}}+T_{\mathrm{adaptation}}, with stagewise peak memory reported separately. Bases can be reused for independent adaptations from the same checkpoint and protected corpus. Compared with vanilla LoRA, NB-LoRA reduces trainable parameters from mr+rnmr+rn to mr+rkmr+rk, but adds an n×kn\times k fixed basis and basis transformations, so a matched benchmark is needed.

We will clarify Figure 4's timing scope, separate preparation and reuse costs, and limit the current efficiency conclusion to the measured NSCL comparison.

Reviewer_gsYxOpenReview ↗

review

Thank you for clarifying what the 99% spectral-energy threshold guarantees and for agreeing to revise Eq. (5). The comparison with vanilla LoRA remains encouraging: on the 3B model, NB-LoRA retains 63.2% reasoning accuracy versus 13.9% for LoRA, with nearly identical adaptation accuracy. My remaining concern is whether the comparisons with replay and SDFT are reliable. Could you report what you find when checking the raw outputs, especially how many of replay’s 62 tool calls were scored incorrectly because of formatting or parsing, and whether rescoring changes the comparison? It would also help to identify the adaptation task and checkpoint used in Table 3. I am keeping my rating at 2 for now and would reconsider it in light of those checks.

Thank you for the follow-up. We checked the saved tool-use outputs for both model sizes.

Replay. All 62 responses on 1.5B contain additional calls, but 44 begin with the exact correct tool and arguments. The parser extracts these correctly; they fail because the evaluator compares the entire sequence against one gold call. The same first-call diagnostic gives:

ModelMethodFull-sequence exactFirst-call exact
1.5BReplay0/62 (0.0%)44/62 (71.0%)
1.5BNB-LoRA39/62 (62.9%)39/62 (62.9%)
1.5BVanilla LoRA41/62 (66.1%)41/62 (66.1%)
1.5BSDFT10/62 (16.1%)12/62 (19.4%)
3BReplay47/62 (75.8%)49/62 (79.0%)
3BNB-LoRA49/62 (79.0%)49/62 (79.0%)
3BVanilla LoRA48/62 (77.4%)48/62 (77.4%)
3BSDFT40/62 (64.5%)44/62 (71.0%)

First-call scoring places replay above NB-LoRA on 1.5B and ties NB-LoRA on 3B; NB-LoRA scores higher on 3B full-sequence compliance. We will report this distinction while retaining extra calls as a protocol failure.

SDFT. On 1.5B, 44 responses yield no extracted call, including some with correct tool names and arguments in prose. Formatting affects its score, although these tool-use files cannot explain the 11.9% average across all adaptation tasks.

Table 3. The downstream adaptation task is ARC-Challenge. We evaluate the resulting merged checkpoints on the five reasoning benchmarks to assess retention after adaptation. The reasoning anchor corpora—MATH-lighteval for 3B and GSM8K for 1.5B—are used to construct the preservation subspaces; they are distinct from the downstream adaptation task. We will clarify this protocol and specify the adaptation task and checkpoint identifiers in the revised caption and experimental details.

Reviewer_x8axOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies how to adapt language models that acquired reasoning ability through reinforcement learning without losing that ability during subsequent supervised fine-tuning. NB-LoRA collects layer-input hidden states from reasoning rollouts, constructs a fixed approximate null basis from low-energy activation directions, and restricts LoRA updates to that basis. The intended effect is to reduce changes to the original reasoning computations while allowing new-task learning. Experiments on RL-trained Qwen2.5 models compare downstream adaptation and reasoning retention with standard LoRA and other preservation methods. The paper also evaluates held-out reasoning benchmarks and sensitivity to anchor-set size and the spectral-energy threshold.

Critical strengths

Preserving acquired reasoning ability while learning new tasks is an important practical problem. Since subsequent adaptation is needed in many scenarios, a method that reduces reasoning loss while permitting new-task learning would address a consequential need.

Critical weaknesses
  1. The exact-preservation claim does not follow from the implemented approximate null space. A 99% spectral-energy threshold retains small-eigenvalue directions and does not generally imply HNs=0HN_s = 0 or HΔW⊤=0H\Delta W^\top = 0 exactly. Moreover, a constraint on anchor-derived hidden states does not directly establish preservation on unseen reasoning distributions. Without residual measurements and a clearer account of anchor coverage, the central preservation argument is overstated.

  2. The results support competitive adaptation and approximate retention, but do not establish uniform improvement or a consistent retention advantage over strong baselines. On Qwen2.5-1.5B, NB-LoRA's BoolQ accuracy is 68.3%, below ordinary LoRA's 68.6% and several other baselines; NB-LoRA also does not consistently outperform the baselines on WinoGrande and SocialIQA. On Qwen2.5-3B, the average retained reasoning accuracy in Table 2 is 63.2% for NB-LoRA versus 63.3% for NSCL. The Table 3 averages for these methods are 67.1% and 67.0%, respectively, and NB-LoRA's MMLU-STEM accuracy falls from 65.9% before fine-tuning to 63.6%. Without estimates of variability from independent runs, small gains, including the 54.5% to 54.7% change in average accuracy on Qwen2.5-1.5B, cannot substantiate significant improvement.

Questions for authors
  1. With the 99% spectral-energy threshold used in lines 066-070, what are the absolute Frobenius norms ∥HNs∥F\|HN_s\|_F or ∥HΔW⊤∥F\|H\Delta W^\top\|_F, and the corresponding relative residuals? Could you distinguish the strict theoretical null space from the approximate null space used in the implementation and revise the equality signs and "satisfied exactly" claim accordingly?

  2. What does a "small" anchor set in lines 089-091 mean quantitatively for the tested models and data, given the instability with 32 examples and the more stable retention with 64-512? How does the strength of the constraint affect the balance between preserving reasoning and adapting to new tasks, particularly when too little constraint permits forgetting and too much makes adaptation difficult?

  3. The layer-output constraint in line 158 applies only to hidden states produced by the anchor data. Could you stratify anchors by domain, difficulty, reasoning-chain length, and importance, and report retention separately for each subdistribution? This would help establish which reasoning distributions are protected and how far preservation generalizes beyond the anchors.

  4. In Table 2, NB-LoRA's BoolQ accuracy on Qwen2.5-1.5B is 68.3%, compared with 68.6% for ordinary LoRA, 69.2% for LoRA-Null, 69.0% for MiLoRA, 68.9% for NSCL, and 68.9% for experience replay. NB-LoRA also does not consistently outperform the baselines on WinoGrande and SocialIQA. Could you describe the result as competitive overall adaptation without implying improvement on every downstream task, and report results across multiple seeds to show whether these differences are robust?

  5. Are the differences of 0.1-1 percentage points in Tables 2 and 3 larger than the variation caused by random seed, data order, and anchor sampling? On Qwen2.5-3B, Table 2 reports average retained reasoning accuracy of 63.2% for NB-LoRA versus 63.3% for NSCL, while Table 3 reports averages of 67.1% and 67.0%, respectively. On Qwen2.5-1.5B, NB-LoRA's Table 3 average changes only from 54.5% before fine-tuning to 54.7%. Could you provide means, standard deviations, and paired significance tests across independent runs, and explain how the aggregate averages are calculated? Without such evidence, these results support similar retention but do not establish a consistent advantage over NSCL or a significant improvement over the original reasoning ability.

  6. In Table 3, NB-LoRA's MMLU-STEM accuracy on Qwen2.5-3B falls from 65.9% to 63.6%, a loss of 2.3 percentage points. Even on mathematics benchmarks closer to the anchors, NB-LoRA's MATH-lighteval score of 63.5% is below NSCL's 63.7%, and its MATH-500 score of 62.2% is below experience replay's 62.6%. How should the preservation claim be limited in light of these results? Could you break down MMLU-STEM by subject, question type, or similarity to the anchor distribution to determine whether the decline mainly affects knowledge or reasoning types farther from the anchors?

1 comment

We thank the reviewer for their careful feedback.

W1 & Q1. Approximate preservation

Threshold-based approximate subspaces are commonly used in projection-based continual learning to balance preservation and adaptation, including GPM [R1] and Adam-NSCL [R2]. Following this common practice, we protect dominant activation directions using a 99% spectral-energy threshold and adapt in the remaining low-energy directions. The parameterization enforces subspace membership throughout training, but does not guarantee unchanged reasoning outputs. Large updates and activation changes propagated from earlier layers can still affect preservation.

Our experiments support this strategy's practical effectiveness: on Qwen2.5-3B, NB-LoRA achieves 84.6% average target accuracy and 63.2% retained reasoning accuracy, versus 84.7% and 13.9% for vanilla LoRA (post-RL reasoning reference: 63.8%). The threshold alone, however, cannot supply the requested absolute and relative residual measurements or guarantee a 1% output change.

We will correct Eq. (5) and the exact-preservation wording, cite these precedents, and distinguish subspace enforcement from empirical reasoning retention.

[R1] Saha et al. Gradient Projection Memory for Continual Learning. ICLR 2021.

[R2] Wang et al. Training Networks in Null Space of Feature Covariance for Continual Learning. CVPR 2021.

Q2. Anchor size and constraint strength

Our default uses 512 reasoning examples; the ablation tests 32–512. Retention is more stable with 64–512 examples in the evaluated settings, while 32 can be insufficient: after WinoGrande adaptation on Qwen2.5-3B, reasoning accuracy drops from 63.80% to 49.20%. Separately, subspace similarity to a 4,096-example reference largely stabilizes beyond 512 examples for most analyzed modules. These results do not establish a universal minimum anchor count.

A higher energy threshold protects more directions and leaves fewer available for adaptation. Our results show competitive adaptation at 0.99, but sensitivity is task-dependent: after HellaSwag adaptation, reasoning accuracy is 63.54% at 0.99 versus 53.68% at 0.90.

We will state actual anchor counts and replace broad threshold-insensitivity claims with a discussion of the per-task trade-off.

Q3. Anchor coverage and generalization

The anchors are MATH-lighteval for Qwen2.5-3B and GSM8K for Qwen2.5-1.5B. Retention on other benchmarks provides empirical evidence of transfer, but neither anchor count nor subspace similarity establishes coverage across domains, difficulty levels, reasoning-chain lengths, or importance.

The current results do not include the requested stratified analysis. Comparing anchor-selection strategies requires matched example counts, reported token counts, and an explicit definition of importance.

We will distinguish anchor size from coverage and restrict generalization claims to the evaluated distributions.

W2 & Q4. Competitive adaptation

We agree that NB-LoRA does not improve every downstream task. For example, on Qwen2.5-1.5B BoolQ it scores 68.3%, versus 68.6% for vanilla LoRA and 69.2% for LoRA-Null. Our claim is competitive overall adaptation with substantially better reasoning retention, as illustrated by the aggregate results in W1/Q1. Multiple-seed uncertainty is discussed below.

We will remove language implying uniform target-task improvement.

Q5. Run variation and aggregation

The small differences do not establish superiority over NSCL or the original reasoning model. On Qwen2.5-3B, NB-LoRA/NSCL retention is 63.2%/63.3% in Table 2 and 67.1%/67.0% in Table 3. Likewise, the 54.5%-to-54.7% change on Qwen2.5-1.5B cannot establish improvement without repeated runs.

Table 2 averages over nine adaptation tasks; Table 3 averages over five reasoning benchmarks. Both use unweighted means. Reported task-wise dispersion is not run-to-run uncertainty. Matched repeated runs are needed to quantify variation from training seeds, data order, and anchor sampling, using means, standard deviations, and paired uncertainty estimates.

We will clarify aggregation and dispersion, describe retention as comparable to NSCL, and avoid unsupported significance claims.

Q6. MMLU-STEM degradation

The MMLU-STEM decline from 65.9% to 63.6% is a limitation that the aggregate average should not obscure. NB-LoRA also trails NSCL on MATH-lighteval (63.5% versus 63.7%) and replay on MATH-500 (62.2% versus 62.6%). The results support substantial mitigation of forgetting, with remaining task-specific degradation.

Anchor mismatch is a possible explanation, but requires subject-level scores and sample counts, paired correctness changes, and breakdowns by question type or anchor similarity. Aggregate accuracy alone cannot distinguish lost knowledge from degraded reasoning.

We will highlight the MMLU-STEM decline and avoid attributing it to a specific mechanism without supporting analysis.

Reviewer_iPudOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Supervised adaptation after reinforcement learning can degrade reasoning, creating a need for efficient updates that preserve previously acquired capabilities. The paper proposes NB-LoRA, which reparameterizes low-rank updates through a fixed approximate null basis estimated from reasoning activations. Experiments on mathematical reasoning, commonsense adaptation, and tool use report better reasoning retention than unconstrained fine-tuning, competitive adaptation performance, and lower training costs than full-model projection.

Critical strengths
  1. The fixed-basis parameterization maintains an explicit update constraint without repeated full-model gradient projection.

  2. Held-out reasoning evaluations and the sequential adaptation study provide evidence beyond retention on the anchor corpus.

Critical weaknesses

The method extends existing activation-based null-space preservation by maintaining the constraint throughout LoRA training. This is a useful distinction, but the additional insight specific to post-RL reasoning is limited.

  1. The experiments use only two small Qwen2.5 models, mostly with commonsense questions and a very small single-call tool-use test. These settings provide limited evidence for stronger reasoning models, particularly because SDFT is known to perform poorly at small scales.

  2. The preservation argument assumes an exact null space, whereas the implemented basis is approximate. The resulting activation drift is not bounded, and optimization convergence does not establish that reasoning is preserved.

Questions for authors

Does the runtime in Figure 4 include reasoning rollouts and basis construction, and how does the total cost compare with a low-rank adapter using a fixed null-space projector?

2 comments

We thank the reviewer for their careful feedback.

W1. Contribution beyond null-space preservation

Activation-based null-space preservation is an established idea. Our contribution is to enforce this constraint throughout parameter-efficient post-RL adaptation using a fixed activation-derived basis and trainable low-rank factors. Unlike null-space initialization, NB-LoRA constrains subsequent updates; unlike full-weight projection, it trains only the adapter factors and avoids repeated update projection.

Our post-RL analysis shows that useful low-energy capacity remains despite the many token representations in long reasoning traces. Under the 99% energy criterion, the average available dimensions are 34.3% for Q/K/V inputs, 29.2% for MLP-up inputs, and 45.1% for MLP-down inputs on the analyzed Qwen2.5-3B layers. The anchor-size analysis further examines estimation from limited reasoning traces.

The experiments demonstrate the resulting adaptation-retention trade-off: on Qwen2.5-3B, NB-LoRA achieves 84.6% average target accuracy and 63.2% retained reasoning accuracy, versus 84.7% and 13.9% for vanilla LoRA (post-RL reasoning reference: 63.8%). Retention is comparable to NSCL; our measured advantage over full-weight NSCL is primarily training efficiency.

We will clarify the contribution as sustained constraint enforcement, empirical post-RL subspace analysis, and parameter-efficient adaptation.

W2. Model scale and evaluation scope

The null-space approach is applicable to larger models: it uses activation-derived subspaces rather than a model-size-specific design. As model width increases, the additional representation dimensions can provide more low-energy capacity for adaptation. Our comparison from Qwen2.5-1.5B to Qwen2.5-3B shows increased approximate null-space capacity, supporting the potential of this strategy at larger scales.

We agree that SDFT is effective when the model is sufficiently powerful to benefit from in-context demonstrations. NB-LoRA addresses a complementary setting: smaller models whose weaker in-context learning can limit self-distillation quality. It preserves reasoning through an activation-derived constraint without relying on strong in-context learning, making the two approaches complementary.

Our evaluation includes eight commonsense tasks, held-out reasoning benchmarks, sequential adaptation, and a preliminary single-call tool-use task with 2,513 training and 62 evaluation examples. These test adaptation and retention across several settings; more demanding tasks would further strengthen the evaluation.

We will clarify the scaling motivation and the observed increase in null-space capacity, and emphasize how NB-LoRA complements SDFT in smaller models.

W3. Approximate preservation

Threshold-based approximate subspaces are commonly used in projection-based continual learning to balance preservation and adaptation, including GPM [R1] and Adam-NSCL [R2]. Following this common practice, we protect dominant activation directions using a 99% spectral-energy threshold and adapt in the remaining low-energy directions. This enforces subspace membership throughout training, but does not guarantee unchanged reasoning outputs or accuracy.

Large updates can amplify the remaining activation residual, and changes from earlier adapted layers can propagate to later layers. The threshold therefore does not imply a 1% output-change guarantee. Our retention results provide empirical evidence of effectiveness, rather than exact protection. Appendix B concerns gradient-descent convergence under its stated assumptions; it does not establish Adam convergence or reasoning retention.

We will correct Eq. (5) and the exact-preservation wording, cite these precedents, and distinguish subspace enforcement, propagated activation changes, and optimization convergence from empirical retention.

[R1] Saha et al. Gradient Projection Memory for Continual Learning. ICLR 2021.

[R2] Wang et al. Training Networks in Null Space of Feature Covariance for Continual Learning. CVPR 2021.

Q1. Preparation costs and fixed-projector comparison

Figure 4 reports 627.72 s/11.23 GiB for NB-LoRA versus 3,924.93 s/23.62 GiB for full-weight NSCL on 3B. The current implementation excludes rollout generation and basis construction from its adaptation timer; the original benchmark boundaries still require verification. The approximately 6.3x runtime ratio therefore does not establish an end-to-end speedup.

A complete comparison should report rollout, activation/basis preparation, and adaptation costs, with stagewise peak memory. Preparation can be reused for independent adaptations from the same checkpoint and protected corpus. A changed checkpoint or protection target requires refreshing the relevant activation statistics and basis.

A fixed-projector LoRA adapter has the same feasible effective updates as NB-LoRA. NB-LoRA directly trains smaller factors in the selected subspace, but a projected adapter can also use a factored basis without materializing a dense projector. Different parameterizations can produce different Adam trajectories, so the NSCL comparison does not establish an advantage over an efficient projected adapter.

We will clarify Figure 4's timing scope and distinguish full-weight NSCL from projected LoRA. An end-to-end comparison requires preparation costs and matched standard-LoRA and projected-LoRA measurements.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.