- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ECR-LoRA, a class-incremental learning framework that partitions adaptation across transformer depth into an evolving shared core and task-specific residual LoRAs. The core uses subspace-compatible learning and Fisher-guided consolidation, while residual learning uses core-error reweighting (CER) and residual-mix consistency (RMC). At inference, core-only entropy calibrates competition among residual experts.
Critical strengths
The shared/specific organization is clearly motivated, and the paper acknowledges its relationship to CL-LoRA. The All-scan variant already improves over CL-LoRA on several datasets, indicating that the reported benefits are not entirely due to the final inference calibration.
Critical weaknesses
- The exact candidate initialization and Fisher normalization are unclear. If the candidate reuses the previous down-projection’s row space, the merged matrix already has rank at most r, leaving the role of SVD truncation unclear.
- CER updates the full Core–Residual path, and RMC is not compared with generic feature perturbations, leaving residual-specific complementarity insufficiently established.
- Table 2 and Figure A1 disagree for removing CER on ImageNet-R and RMC on ImageNet-A.
- Full residual scanning limits scalability, while the 50-session ImageNet-R gain over CL-LoRA is only 0.02 percentage points. Matched accuracy–latency comparisons are needed.
- The clarity and format of many figs.
Questions for authors
- How are the candidate down-projection and normalized Fisher support constructed, and does SVD discard nonzero singular directions in practice?
- Can the Table 2/Figure A1 discrepancies be reconciled?
- How much improvement remains over CL-LoRA when both methods use the same inference calibration?
9 comments
Response to Reviewer DTFq (2/3): W3 & Q2, W4
W3 & Q2: Numerical consistency. We agree that the numerical differences between Table 2 and Figure A1 require clarification. Table 2 reports component ablations from an earlier experimental batch, while Figure A1 presents a separate three-seed coefficient sweep. Since these experiments were conducted independently, corresponding settings yield slightly different results (ImageNet-A w/o RMC: 60.76 ± 1.29 vs. 61.36 ± 1.02; ImageNet-R w/o CER: 80.29 ± 0.41 vs. 79.92 ± 0.10).
We also identified a plotting error: the default-setting points in Figure A1 were taken from the main-table results rather than the original sweep. After checking the sweep logs, we corrected these points in the revised Figure A1. All other points were consistent with the original records, and each curve now uses results from the same experimental batch. We have also updated the corresponding analysis in Appendix C.2 to clarify the experimental sources and reflect the corrected results.
The qualitative conclusions remain unchanged: removing SAR produces the largest accuracy drop, while CER and RMC show smaller variations across nearby nonzero weights.
W4: Scalability and accuracy–efficiency trade-off.
To assess the accuracy–efficiency trade-off, we conducted matched comparisons of ECR-LoRA and CL-LoRA on 10- and 20-session ImageNet-R. Both methods use the same shared-prefix reuse optimization for latency measurements mentioned in Appendix D.1, ensuring a consistent inference setup.
Table R2. Accuracy–latency comparison on 10- and 20-session ImageNet-R.
| Sessions | Method | (%) | Latency (ms/image) |
|---|---|---|---|
| 10 | CL-LoRA (All-scan) | 79.76 ± 0.28 | 38.5 |
| 10 | ECR-LoRA (All-scan) | 80.28 ± 0.29 | 43.1 |
| 10 | CL-LoRA + CP | 80.47 ± 0.12 | 45.4 |
| 10 | ECR-LoRA + CP | 80.84 ± 0.08 | 48.1 |
| 20 | CL-LoRA (All-scan) | 76.26 ± 0.21 | 74.5 |
| 20 | ECR-LoRA (All-scan) | 77.83 ± 0.49 | 80.8 |
| 20 | CL-LoRA + CP | 76.36 ± 0.74 | 79.9 |
| 20 | ECR-LoRA + CP | 78.56 ± 0.31 | 91.9 |
Accuracy is reported as mean ± SD over three seeds. Latency is measured on the same NVIDIA L20 using FP32, batch size 1, and shared-prefix reuse for both methods, reporting the median of three measurement rounds.
Under All-scan inference, ECR-LoRA improves final accuracy over CL-LoRA by 0.52 and 1.57 pp at 10 and 20 sessions, respectively, with approximately 12% and 8% additional latency. When CP is applied to both methods, ECR-LoRA maintains gains of 0.37 and 2.20 pp, with approximately 6% and 15% additional latency. Notably, although the absolute accuracies differ between experimental batches, the 20-session All-scan improvement is similar to that reported in Table 3 (+1.39 pp).
At 50 sessions, the original three-seed results show comparable final accuracy between ECR-LoRA and CL-LoRA (71.73 ± 0.26 vs. 71.71 ± 0.67), with a difference of only 0.02 pp. However, ECR-LoRA achieves substantially higher average accuracy across sessions (78.63 ± 1.63 vs. 75.89 ± 1.68), an improvement of 2.74 pp. This suggests that, despite similar performance at the final session, ECR-LoRA maintains better classification accuracy throughout the incremental learning process. We additionally conducted a matched 50-session rerun with the same configuration on an L20 GPU. In this single-seed experiment, ECR-LoRA outperforms CL-LoRA in both final accuracy (73.70 vs. 70.65, +3.05 pp) and average accuracy (80.81 vs. 76.49, +4.32 pp). These results provide further evidence of ECR-LoRA's effectiveness in long-sequence continual learning, particularly in maintaining higher accuracy across incremental sessions.
Nevertheless, shared-prefix reuse does not eliminate the linear cost of evaluating task-specific Residuals. We explicitly acknowledge this limitation and discuss sparse expert selection as future work in Appendix D.1.
Response to Reviewer DTFq (3/3): Q3, W5
Q3 — Matched Core-Prior Calibration on CL-LoRA.
Thank you for suggesting a matched calibration comparison. We reran CL-LoRA and ECR-LoRA using matched task splits and three seeds on NVIDIA L20 GPUs. Both methods use the same Core-prior (CP) calibration rule with , without method-specific retuning. CP is a score-calibration rule rather than a separately trained routing module. It can therefore be applied to CL-LoRA using its shared-only pathway as the reference, allowing a matched comparison under the same inference calibration.
Table R3. Matched CP comparison (, mean ± SD, %; three seeds on NVIDIA L20 GPUs). Full results in Table A9, Appendix D.5.
| Method | ImageNet-R | ImageNet-A | CIFAR-100 |
|---|---|---|---|
| CL-LoRA (All-scan) | 79.76 ± 0.28 | 58.09 ± 0.91 | 86.15 ± 0.22 |
| CL-LoRA (+CP) | 80.47 ± 0.12 | 59.96 ± 0.74 | 87.81 ± 0.28 |
| ECR-LoRA (All-scan) | 80.28 ± 0.29 | 61.33 ± 1.37 | 87.40 ± 0.20 |
| ECR-LoRA (+CP) | 80.84 ± 0.08 | 63.49 ± 0.41 | 89.25 ± 0.23 |
CP improves CL-LoRA's final accuracy by 0.71, 1.87, and 1.66 percentage points on ImageNet-R, ImageNet-A, and CIFAR-100, respectively. When both methods use CP, ECR-LoRA retains margins of 0.37, 3.53, and 1.44 points. Thus, its gains are not solely attributable to CP, although the margin on ImageNet-R is modest.
We have clarified this comparison in Sec. 4.4 and added the CL-LoRA implementation, matched protocol, and complete final/average accuracy results to Appendix D.5 (Table A9).
W5: Figures and Presentation.
Thank you for your helpful suggestions and careful attention to these details. We have corrected the plotting error in Figure A1 and revised the corresponding descriptions to improve clarity and consistency. We have also added further explanations and experimental details in the relevant sections and appendix. We sincerely appreciate your feedback and would be happy to provide additional clarification or make further improvements if any points remain unclear.
Response to Reviewer DTFq (1/3): W1 & Q1, W2
W1 & Q1 — Candidate Initialization, Fisher Support, and SVD.
Thank you for highlighting these ambiguities. Each shared Core is initialized with a row-orthonormal A0c obtained via QR decomposition and B0c=0. At task t, the candidate inherits At−1c, keeps it frozen, and optimizes only the zero-initialized Btnew.
We estimate Fisher-inspired support from squared cross-entropy gradients over four current-task mini-batches, averaging across batches and output rows and normalizing by the maximum rank-wise entry. Its mean and the historical EMA determine the absorption coefficient ηt.
Since the candidate and the existing Core share the down-projection, their merged matrix already has rank at most r. SVD therefore preserves the merged mapping exactly in exact arithmetic while providing balanced factors for the next task. Direct factor accumulation would preserve the same mapping but would not provide this reparameterization.
We have clarified these details in Section 3.3, Algorithm A1, and Appendix A.2.
W2: CER/RMC Complementarity and Appropriate Controls.
Thank you for this careful reading. We agree that CER derives detached example weights from shared-only (Core) predictions, while the weighted loss is optimized through the full Core–Residual path (Eq. 6). Thus, CER is Core-conditioned rather than Residual-restricted: it emphasizes examples insufficiently explained by the Core, while Residual specificity comes from the split-depth architecture rather than gradient isolation.
Following your suggestion, we conducted matched controls on 10-session ImageNet-A (NVIDIA L20, three seeds).
Table R1. CER/RMC controls on ImageNet-A. Accuracy (%) is reported as mean ± SD over three seeds. Old-task accuracy is measured after the final session; new-task accuracy is averaged across sessions. Parentheses report seed-paired differences from the full model (mean ± SD) for the four primary comparisons discussed below. Also reported as Table A6 in Appendix D.3.
CER. Removing CER reduces new-task accuracy and All-scan AAvg in all three paired seeds. On a fixed subset of samples misclassified by the Core-only path, its removal also reduces within-task accuracy by 0.46±0.13 pp. Full-path weighting performs comparably, suggesting that CER-style example reweighting benefits new-task learning without establishing the unique superiority of Core-derived weights.
RMC. Removing RMC or replacing historical-Residual shifts with norm-matched random perturbations reduces CP AAvg by 0.35±0.09 and 0.28±0.11 pp, respectively, with decreases in all three seeds. This provides modest evidence for the usefulness of structured historical-Residual exposure. It is also qualitatively consistent with the original coefficient sweep (Figure A1), where setting λmix=0 lowers ALast on both datasets. However, the changes in final accuracy in the matched L20 reruns remain within seed variation.
We focus the detailed comparison on ImageNet-A, where the Core-only path misclassifies a larger proportion of samples (approximately 48% versus 34% on ImageNet-R). On ImageNet-R, the same controls differ from the full model by less than 0.2 pp across the reported metrics. Complete results for both datasets are provided in Appendix D.3.