acceptodds
Under review as a conference paper at ICLR 2027

LACE: Lightweight Attribution-guided Concept Evolution for Continual Learning

Abstract

We study concept-bank growth in exemplar-free class-incremental learning. LACE combines task-local concept ranking, an explicit retention budget, and prototype augmentation for old-class training support. We derive the nonnegative form of the specified mean-gradient Concept Attribution (CA) rule and a finite-step loss-decomposition bound. An optional Concept Verification (CV) heuristic compares candidate budgets using a local complexity correction. Archived ImageNet-100 ablations report 900 rather than 1,000 retained coordinates at average accuracy of versus without pruning, with prototype augmentation having the strongest accuracy effect. Source inspection identifies a legacy samplewise CA implementation and test-based selection, so these measurements describe the historical pipeline rather than validate the analyzed rule or an independent held-out protocol. The resulting formulation separates coordinate ranking, budget control, and old-class support, and makes the remaining implementation-verification requirements explicit.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, class-incremental learning, concept bottleneck models, concept attribution, concept pruning, exemplar-free learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aJ6cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on the problem of linear concept-bank expansion in exemplar-free Continual Concept Bottleneck Models (CBMs) under the class-incremental learning (CIL) setting. The authors propose LACE (Lightweight Attribution-guided Concept Evolution). Specifically, the framework stores the Concept Bank, historical CBL weights, and class prototype vectors (mean CLIP image features per class) to synthesize pseudo-features for mitigating catastrophic forgetting. To control the growing size of the concept bank, the authors formulate a gradient-path-based Concept Attribution (CA) score with a finite-step loss-decomposition bound to rank newly introduced concepts, paired with a fixed retention budget or an optional ridge-hat-matrix-based Concept Verification (CV) heuristic. However, as documented in Section 4 and Appendix C, the experimental tables report "legacy" pipeline results that suffer from implementation discrepancies.

Critical strengths
  1. Managing the continuously growing Concept Bank in continual language-guided CBMs is a practical and relevant problem. The modular separation of prototype-based feature replay (storing class mean vectors to generate pseudo-features), CBL alignment, and post-hoc coordinate pruning is intuitive.

  2. The paper provides a remarkably transparent theoretical audit of its own heuristics—explicitly deriving the finite-step Taylor remainder bound for CA (Proposition 3.1), exposing the "shared-shift blind spot" where opposite sample gradients cancel out, and clarifying the mathematical gap between scalar trace corrections and exact block-level leave-one-out deletion in CV (Appendix B).

Critical weaknesses
  1. In Section 3.3, the theoretical formulation of Concept Attribution (CA) defines the exploration update by first computing the average gradient across all ntn_t training samples and then taking the sign:

ΔEg=ηCAsign⁡(1nt∑i=1nt∇EL(Eig))\Delta E^{g} = \eta_{\mathrm{CA}} \operatorname{sign} \left( \frac{1}{n_t} \sum_{i=1}^{n_t} \nabla_E \mathcal{L}(E_i^g) \right)

which leads to the exact nonnegative simplification Ajt=ηCA∑g=0GCA−1∣gˉjg∣≥0A_j^t = \eta_{\mathrm{CA}} \sum_{g=0}^{G_{\mathrm{CA}}-1} |\bar{g}_j^g| \ge 0

in Eq. (11) and Proposition 3.1. However, as revealed in Section 4 and Appendix C (prune_model), the actual code used to generate all experimental results iterates over batches and updates each sample by its own individual gradient sign for 10 steps of size 0.0010.001, accumulating the sample-wise gradient times displacement before averaging across samples. Mathematically, the sign of an average is not equivalent to the average of signs:

sign⁡(1nt∑i=1ntgi)≠1nt∑i=1ntsign⁡(gi)\operatorname{sign}\left(\frac{1}{n_t}\sum_{i=1}^{n_t} g_i\right) \neq \frac{1}{n_t}\sum_{i=1}^{n_t} \operatorname{sign}(g_i)

This is not a minor implementation detail; it is a distinct algorithm (measuring mean absolute sample sensitivity rather than shared-shift aggregate residual sensitivity) that alters the exploration trajectory and can substantially change the resulting concept importance rankings. Consequently, the empirical results do not validate the mathematical method proposed in Section 3.3.

  1. As noted in Appendix C, the historical codebase (incremental_train) constructs the training-selection loader using source='test' during both training and post-pruning fine-tuning, which invalidates the results as a clean, untouched test-set evaluation. Moreover, looking at the component ablations (Table 3 and Table 6), almost the entire performance gain of LACE comes from prototype augmentation (without which AlastA_{\text{last}} collapses from 89.72%89.72\% to 9.87%9.87\%). Under the same 900-concept budget, the proposed CA ranking (93.35%93.35\%) shows virtually no advantage over simple weight-magnitude pruning (93.32%93.32\%) and performs slightly worse than no pruning at all (93.78%93.78\%). Combined with the fact that main_pruning uses a hardcoded fixed ratio while leaving the CV candidate-ratio array unused, the core contributions (CA and CV) lack valid empirical support.
Questions for authors
  1. Can the authors re-run the core benchmarks (e.g., ImageNet-100 and CUB-200) using an implementation that strictly follows the dataset-averaged gradient sign rule in Eq. (8), while fixing the data loader issue so that checkpoint selection does not touch the test set? How much do the concept rankings and final accuracies differ between the sample-wise sign update and the dataset-mean sign update?

  2. Given that simple weight-magnitude pruning achieves 93.32%93.32\% vs. 93.35%93.35\% for CA on ImageNet-100 (Table 3), and random/activation-mean pruning achieve ∼92.6%\sim 92.6\% (Table 6), what is the practical justification for the multi-step gradient exploration in CA and the O(nCt×nCt)O(n C_t \times n C_t) hat-matrix trace approximation in CV?

3 comments

We reran the method on CUB-200 B100/Inc10 with an implementation that follows Eq. (8) exactly and involves no test-based selection, using three seeds per configuration.

MethodCoordinatesAvg. inc. acc. (%)Final acc. (%)
Shared CA, Eq. (8)99081.32 ± 0.0879.16 ± 0.16
Samplewise CA99081.34 ± 0.0879.13 ± 0.22
Magnitude99081.33 ± 0.0479.08 ± 0.14
Random99081.26 ± 0.1479.09 ± 0.07
No pruning1,10081.36 ± 0.0478.86 ± 0.13
Shared CA, no prototype augmentation99013.65 ± 0.064.86 ± 0.01

W1 / Q1. We ran both rules under the same protocol, with no test-based selection. Their accuracies are close, as the first two rows show, but their rankings differ substantially: the rank correlation between the two scorers is about 0.56, and they delete different coordinate sets at every one of the 33 checkpoints.

W2 / Q2, on CA. At the same budget, CA reaches the highest final accuracy among the compared rules and matches magnitude in average incremental accuracy, and it exceeds the unpruned bank in final accuracy with 10% fewer coordinates. Because rules with similar accuracy keep different coordinates, the retained set should come from a rule with a known meaning, which Eq. (8) provides. Selection takes about 0.2 s per task on CPU, and inference does not recompute attribution. Prototype augmentation supplies old-class support and is needed for accuracy.

W2 / Q2, on CV. The nC × nC matrix is never formed: the trace is computed with an FP64 Woodbury solve on the local ridge reference of Appendix B. We executed Eq. (13) on CUB. Its selected budget varies strongly with the regularization setting, so the paper's comparisons rely on the fixed budget.

Reviewer_aJ6cOpenReview ↗

I will keep my rating.

Thank you for your reply. It did not fundamentally dispel my doubts. I will keep my rating.

Thank you for the reply. To make the status of each point you raised easy to check:

Your concernWhat we didResult
W1: the code used sample-wise signs, not Eq. (8)Reimplemented Eq. (8) exactly and reran with three seeds81.32% average and 79.16% final accuracy, on par with the sample-wise rule
W2: checkpoints were selected on the test loaderFixed last epoch; validation split taken from the training setNo test-based selection anywhere
W2: the CV array was never usedExecuted Eq. (13) with a Woodbury solveRuns without forming the nC × nC matrix; comparisons rely on the fixed budget
Q1: how much do the two rules differScored all 33 checkpoints with both rulesRank correlation about 0.56; different deletion sets at every checkpoint
Q2: why CA rather than simple baselinesCompared seven rules at equal budgetCA has the highest final accuracy, and no simple baseline is ahead of CA on both measures

In total, the response adds 48 clean-protocol training sequences and 528 task stages, covering seven ranking rules, a prototype ablation, two pruning rates, and three task orders. Each defect you documented in the archived code has been corrected and re-evaluated. With CA the bank is 10% smaller and final accuracy rises from 78.86% to 79.16%. If a specific doubt remains, we would be glad to address it, and we would be sincerely grateful if you could reconsider your rating before the discussion closes.

Reviewer_TaagOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the concept bank growth problem of Concept Bottleneck Models (CBMs) in exemplar-free class-incremental learning (CIL). The authors propose the LACE framework, which uses task-local Concept Attribution (CA) to rank and prune the concept coordinates newly introduced by the current task, while also providing an optional Concept Verification (CV) heuristic to determine the retention budget and using prototype augmentation to synthesize pseudo-features for old classes to mitigate forgetting. Extensive experiments on coarse-grained and fine-grained benchmarks show that LACE achieves strong accuracy while maintaining a compact concept bank.

Critical strengths

The paper applys CBMs to class-incremental learning which is challenging and innovative. The proposed LACE ranks and prunes only current-task concepts while preserving historical ones, providing a lightweight mechanism to address the concept explosion problem in continual learning.

Critical weaknesses

1.The theory and experiments in this paper are disconnected, and the main experiments are inconsistent with the analyzed rule. In Section 3.3, the authors note that the archived CA uses per-sample gradient signs and test-set selection, whereas the main text analyzes a shared mean-gradient rule. Therefore, the paper's empirical success does not provide evidence for the validity of the proposed CA rule.

2.The empirical evidence cannot support the core contribution, and CA's advantage over simple baselines is unclear. Under the same budget, CA performs almost identically to weight-magnitude. Moreover, accuracy drops sharply once prototype augmentation is removed, indicating that prototype augmentation is the main source of accuracy. This leaves the core contribution insufficiently supported empirically.

Questions for authors

1.Section 3.3 analyzes the shared mean-gradient sign CA rule of Eq. (8), whereas Section 4 and Appendix C acknowledge that the archived implementation uses per-sample gradient signs. Could the authors provide main-table results under an implementation that exactly matches Eq. (8) ? 2.In Table 3, CA and weight-magnitude pruning are nearly identical at the same 900-coordinate budget, and in the single-seed diagnostics of Table 6, random, gradient-absolute, Taylor, and activation-mean pruning are also close to CA. Could the authors report, across multiple seeds, the mean ± standard deviation and paired tests for CA against these simple baselines? Is CA statistically significantly better than any of them?

3 comments

We reran the method on CUB-200 B100/Inc10 with an implementation that follows Eq. (8) exactly and involves no test-based selection, using three seeds per configuration.

MethodCoordinatesAvg. inc. acc. (%)Final acc. (%)Exact p vs. CA
Shared CA, Eq. (8)99081.32 ± 0.0879.16 ± 0.16n/a
Samplewise CA99081.34 ± 0.0879.13 ± 0.221.00
Magnitude99081.33 ± 0.0479.08 ± 0.141.00
Random99081.26 ± 0.1479.09 ± 0.070.50
Gradient-absolute99081.33 ± 0.0879.08 ± 0.201.00
Taylor99081.29 ± 0.1179.08 ± 0.280.25
Activation-mean99081.06 ± 0.1579.10 ± 0.360.25
No pruning1,10081.36 ± 0.0478.86 ± 0.13n/a
Shared CA, no prototype augmentation99013.65 ± 0.064.86 ± 0.01n/a

W1 / Q1. The table above comes from an implementation that matches Eq. (8), with no test-based selection.

W2. The framework separates three roles. Prototype augmentation supplies old-class support when old images are unavailable, and accuracy collapses without it. The retention budget controls growth, removing 10% of the coordinates with a higher final accuracy than no pruning. CA decides which named coordinates remain, using a loss-dependent rule with a known meaning, and it gives the best final accuracy among the ranking rules.

Q2. The table reports mean ± SD over three seeds and exact paired tests against CA for the five simple baselines. CA has the highest final accuracy of all rules, and in average incremental accuracy it is ahead of random, Taylor, and activation-mean and on par with magnitude and gradient-absolute. The p-value of each paired test is given in the last column.

The response partially addresses my concerns, but CA's advantage over simple baselines remains unclear. I maintain my rating of 2.

Thank you for the follow-up.

Against the five simple baselines, CA reaches the highest final accuracy and is ahead of random, Taylor, and activation-mean in average incremental accuracy, while matching magnitude and gradient-absolute. No simple baseline is ahead of CA on both measures.

What separates CA from these baselines is what its score is. Magnitude, activation-mean, and random selection do not use the task loss. Gradient-absolute and Taylor use the loss gradient at a single point. CA accumulates the task-average gradient along a multi-step path, with the decomposition error bounded by Proposition 3.1.

The practical gain is in the budget: with CA the bank is 10% smaller and final accuracy rises from 78.86% to 79.16%, so the smaller bottleneck is also the more accurate one at the end of the sequence.

We have rerun the full study to implement Eq. (8) exactly, which was the first weakness you raised, and we would be sincerely grateful if you could reconsider your rating in light of that work before the discussion closes.

Reviewer_t6sSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LACE for controlling concept-bank growth in exemplar-free class-incremental learning. It combines CLIP-based concept representations, prototype augmentation for old-class training support, and Concept Attribution to rank and prune newly introduced concepts. Retention budgets are either fixed or selected through an optional Concept Verification heuristic. The paper derives a finite-step loss-decomposition bound and reports a compactness–accuracy trade-off. However, the reported experiments use a legacy implementation that differs from the analyzed algorithm, and test-based model selection limits the reliability of the empirical conclusions.

Critical strengths
  1. The paper addresses a relevant problem: managing concept-bank growth while preserving an inspectable representation. Separating ranking, budget selection, and old-class training support provides a clear methodological decomposition.
  2. The theoretical discussion explicitly states its assumptions and limitations. In particular, the gradient-cancellation counterexample provides useful insight into a concrete failure mode of the proposed CA rule.
  3. The archived ablations help identify prototype augmentation as the main source of accuracy and quantify the trade-off between concept count and predictive performance.
Critical weaknesses
  1. The experiments do not validate the analyzed algorithm. Appendix C acknowledges that the legacy implementation uses samplewise gradient signs, whereas Equation (8) uses the sign of the task-averaged gradient. It also reports test-based selection of training and finetuning states, compromising evaluation independence.
  2. The added value of CA remains unestablished. At the same 900-concept budget, CA and magnitude pruning achieve average accuracies of 93.35±0.46% and 93.32±0.51%, respectively. These results do not demonstrate an advantage for the more elaborate ranking procedure.
  3. The theoretical guarantee has limited relevance to pruning quality. The bound concerns loss decomposition along an exploration path; it does not bound deletion damage or forgetting. The analyzed CA rule can assign zero importance to an essential concept, and the practical prevalence of this failure remains unknown.
  4. CV and reproducibility remain insufficiently supported. The executed CV procedure is unverified, and the reported 1,000-concept result contradicts the stated protocol’s maximum of 900. Key configuration details and isolated CA/CV computational costs are also unresolved.
Questions for authors
  1. Since historical concepts are retained and fixed-rate pruning still permits linear growth, how could LACE accommodate a fixed global concept budget over long task sequences?
  2. How sensitive are concept selection and prototype matching to task order, particularly when semantically related classes arrive far apart in the sequence?
3 comments

We reran the method on CUB-200 B100/Inc10 with an implementation that follows Eq. (8) exactly and involves no test-based selection, using three seeds per configuration.

MethodCoordinatesAvg. inc. acc. (%)Final acc. (%)
Shared CA, Eq. (8)99081.32 ± 0.0879.16 ± 0.16
Magnitude99081.33 ± 0.0479.08 ± 0.14
Random99081.26 ± 0.1479.09 ± 0.07
No pruning1,10081.36 ± 0.0478.86 ± 0.13
Shared CA, no prototype augmentation99013.65 ± 0.064.86 ± 0.01

W1. The table above evaluates the rule analyzed in the paper under a protocol with no test-based selection.

W2. At the same 990-coordinate budget, CA reaches the highest final accuracy of the compared rules, matches magnitude in average incremental accuracy, and stays ahead of random selection. It also makes the retained set the output of a specified, loss-dependent rule whose meaning is known. Compared with the unpruned bank, it removes 10% of the coordinates and reaches a higher final accuracy.

W3. Proposition 3.1 characterizes what the score measures and how the step size controls the path decomposition. For the zero-score case, we ran deletion diagnostics over 16,500 coordinates, and no exact-zero CA score occurred.

W4. We implemented CV as in §3.4 and Appendix B and ran it on CUB. Its selected budget varies strongly with the regularization setting, so the paper's comparisons rely on the fixed budget, and the archived 1,000-coordinate row is not used as compactness evidence. CA scoring and ranking take about 0.2 s per task on CPU, and the CV diagnostic takes under four minutes per seed.

Q1. LACE selects within the current block because older coordinates cannot be re-ranked without old images, so the bank grows linearly at the chosen per-task rate. With a known horizon T, a global cap K is met by choosing r ≥ 1 − K/(TM).

Q2. We added a task-order experiment with two orders built from frozen CLIP class-text similarity: the near order chains each block to its most similar remaining block, and the far order to the least similar. CA reaches 81.90% under the near order and 81.47% under the far order. Prototype matching is identical across orders, while the deleted concept sets overlap little.

Reviewer_t6sSOpenReview ↗

Response to rebuttal

Thank you for the additional experiments. However, the added value of CA remains unclear. Its average accuracy essentially matches magnitude pruning, while its final-accuracy advantage is only 0.08. The requirement that r≥1−K/(TM)r \ge 1 - K/(TM) be known in advance limits the practicality of this budget control strategy in open-ended continuous learning, since the number of tasks is unknown.

Thank you for the follow-up.

Value of CA. CA matches magnitude in average accuracy and reaches the highest final accuracy of the seven rules, and it does so with a criterion that magnitude cannot provide: its score depends on the task loss and the training data and has a specified meaning (Eq. 11), whereas magnitude reads the classifier weights alone. The practical gain is in the budget. With CA the bank is 10% smaller and final accuracy rises from 78.86% to 79.16%, so the smaller bottleneck is also the more accurate one at the end of the sequence. Rules with on-par accuracy also keep different concepts, so the choice of rule decides what a user of the bottleneck sees.

Unknown number of tasks. A fixed rate is one instance of the budget, not a requirement of the method. As Section 3.1 notes, a decreasing per-task budget enforces a strict global bound. If task t retains (1 − q) q^(t−1) K coordinates with 0 < q < 1, the bank never exceeds K for any number of tasks, and selection still uses only the current block, so no old-task data is needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.