acceptodds
Under review as a conference paper at ICLR 2027

Data Selection for Continual Learning: A Theoretical Analysis of Sample Scoring under Task Similarity

Abstract

In continual learning, a training example that helps a model learn a new task may also interfere with knowledge acquired earlier. Yet most theories of data selection assume a stationary learning objective, leaving unclear how sample prioritization affects learning and forgetting across sequential tasks. We develop a theory of score-based data selection in a structured linear teacher–student model, showing how feature and readout similarity between tasks determine a scoring rule's performance relative to uniform sampling. We analyze two representative scoring rules based on feature novelty and residual error. Novelty consistently improves the trade-off between learning the new task and retaining the old one, whereas residual-based selection depends on the relationship between the tasks: high-error samples help when the tasks are sufficiently aligned but increase forgetting when shared features support conflicting outputs. We further show that this qualitative ordering extends from score-proportional reweighting to hard budgeted selection, while tighter selection amplifies the effect. Experiments on CIFAR-100, CUB-200, and AwA2 qualitatively support the predicted dependence on task similarity and show that hard selection can produce larger retention effects than reweighting, while nonlinear CIFAR-100 experiments show a similar score ordering.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, data selection, data pruning, task similarity, sample scoring, teacher-student model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Kq3GOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper theoretically investigates how data selection affects learning and forgetting in continual learning. Using a linear teacher-student model, the authors analyze two scoring rules, feature novelty and residual error, under different levels of task similarity. The results suggest that feature novelty consistently improves knowledge retention, whereas the effectiveness of residual-error-based selection depends on the relationship between feature and readout similarities. The authors further extend their analysis to hard selection and provide empirical evaluations on CIFAR-100, CUB-200, and AwA2.

Critical strengths

The paper provides a theoretical analysis that connects data selection strategies with task similarity in continual learning.

The finding that residual-error-based selection can either improve or worsen knowledge retention depending on task similarity is interesting.

Critical weaknesses

1. Unclear theoretical novelty over prior work. Previous studies, such as Lee et al. [1] and Hiratani [2], have already investigated the effects of feature and readout similarities on knowledge transfer and forgetting. Furthermore, the benefit of preserving directions associated with previous tasks has been explored through gradient projection methods [3, 4]. Although this paper extends these insights to data selection, it remains unclear whether the main theoretical findings represent fundamentally new insights or relatively straightforward extensions of existing results. The authors should clarify the theoretical novelty of their work.

2. Limited scope of data selection strategies. The analysis focuses exclusively on feature novelty and residual error, although various other data selection strategies have been studied in the literature. It remains unclear whether the theoretical conclusions extend to other scoring rules, such as gradient-based or diversity-based selection. A broader discussion of the applicability of the proposed analysis would strengthen the paper.

3. Restrictive theoretical assumptions. The theoretical analysis assumes fixed feature representations, an infinite-sample limit, and a strongly overparameterized linear model. However, feature representations typically evolve during continual learning, which can substantially affect forgetting. It is unclear whether the main theoretical conclusions, particularly the consistent advantage of feature novelty, remain valid under such conditions. The authors should clarify the implications of these assumptions for practical deep continual learning.


[1] Lee, S., Goldt, S., and Saxe, A. (2021). "Continual Learning in the Teacher-Student Setup: Impact of Task Similarity." International Conference on Machine Learning (ICML).

[2] Hiratani, N. (2024). "Disentangling and Mitigating the Impact of Task Similarity for Continual Learning." Advances in Neural Information Processing Systems (NeurIPS).

[3] Farajtabar, M., Azizan, N., Mott, A., and Li, A. (2020). "Orthogonal Gradient Descent for Continual Learning." International Conference on Artificial Intelligence and Statistics (AISTATS).

[4] Saha, G., Garg, I., and Roy, K. (2021). "Gradient Projection Memory for Continual Learning." International Conference on Learning Representations (ICLR).

Questions for authors

Please refer to the weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 3 (Part 1/2)

We thank the reviewer for highlighting the similarity-dependent effects of residual scoring. Our main theoretical contribution is to explain how sample scoring shifts learning between shared and task-specific directions, and when this shift improves or worsens the stability-plasticity trade-off in continual learning. In the revision, we show how the framework applies to other scoring rules. We also find that the predicted retention ordering persists as representations evolve during end-to-end CIFAR-100 training. Revisions are highlighted in blue in the updated manuscript.

W1. Theoretical contribution and relation to prior work.

Lee et al. (2021) and Hiratani (2024) establish how feature and readout similarity affect transfer and forgetting. We build on these insights to address a further question: for a given task relationship, how does prioritizing particular current-task samples change retention at the same current-task loss.

Our model distinguishes shared and task-specific latent directions, so samples differ in their contributions to learning along directions that interfere with the previous task. We derive how a scoring rule changes the latent second moment and, through it, the learning gains in these two blocks. This yields a criterion for whether reweighting improves or worsens retention relative to uniform sampling at matched task-2 loss (Section 4, Appendix A.6).

In particular, within the stated approximation regime, residual scoring changes from harmful to beneficial at ρb=ρa/2\rho_b=\rho_a/2, where ρa\rho_a and ρb\rho_b describe feature and readout similarity, respectively. Thus the analysis predicts when emphasizing high-error examples worsens the learning-retention trade-off. We also connect proportional reweighting and hard selection through their induced second moments, deriving their common retention ordering and different effect sizes under the stated approximations (Section 5).

The connection to gradient projection is useful but does not establish this result by itself. A high-novelty sample can still have components in the past-task input subspace, so selecting it does not eliminate interfering updates. Following the reviewer's suggestion, we added Appendix B.4 to make the distinction explicit: projection constrains update directions, whereas novelty scoring changes which samples are emphasized. In our linear setting, at the start of task-2 training, the squared Frobenius norm of the projected per-sample gradient equals the residual score multiplied by the novelty score. Novelty is therefore the input-space factor determining how large a gradient remains after projection.

W2. Other scoring rules and applicability of the framework.

Following the reviewer's suggestion, we investigated two additional scoring rules, predictive uncertainty and squared-gradient scoring, in newly added Appendix B, “Connections to other scoring rules and gradient projection,”.

Novelty and residual error are representative scores for which we derive explicit retention predictions. The second-moment formulation applies more broadly: for a fixed nonnegative score σ(s)\sigma(\mathbf{s}), where s\mathbf{s} is the latent sample, proportional reweighting induces

C=E[σ(s)ss⊤]E[σ(s)].C=\frac{\mathbb{E}[\sigma(\mathbf{s})\mathbf{s}\mathbf{s}^{\top}]}{\mathbb{E}[\sigma(\mathbf{s})]}.

Substituting this matrix into the update equation gives the trained weights and both task losses. A new score therefore requires characterizing its induced CC; it need not have the same retention behavior as novelty or residual scoring. In particular, the squared-gradient score equals the residual score multiplied by the squared input norm. We use this as a representative gradient-based score, rather than implementing full gradient-based algorithms such as Gradient-based Sample Selection (GSS) or Online Coreset Selection (OCS), which require additional machinery such as replay buffers (Aljundi et al., 2019; Yoon et al., 2021). We clarified this broader applicability in the main text (lines 255–256).

Diversity-based selection generally depends on the selected set rather than on a fixed score for each example. Within our linear squared-loss setting, its effect can also be studied through the selected data's second moment CC. Deriving CC and retention predictions for these methods remains future work.

AuthorsOpenReview ↗

Response to Reviewer 3 (Part 2/2)

W3. Assumptions and evolving representations.

We agree that feature evolution can change forgetting and the effect of selection in end to end deep learning. Nevertheless, our numerical experiments, including newly added end-to-end CIFAR-100 training, qualitatively recover the predicted retention/forgetting behaviors from fixed-feature linear setting.

Appendix A.7 uses the empirical second moment to examine finite-sample effects, and the real-data experiments evaluate selection with finite datasets. Appendix D.5 trains a two-layer MLP sequentially on frozen pretrained features and recovers the predicted score ordering with comparable task-2 accuracy.

We have also added end-to-end CIFAR-100 training of both the backbone and readout (Appendix D.6, Fig. 13). The retention ordering persists, with similar task-2 accuracy at the 70% budget; at 30%, novelty's retention advantage comes with lower task-2 accuracy. Diagnostics confirm that both the representation and readout change, with novelty showing the smallest mean representation drift.

The completed five-task end-to-end CIFAR-100 experiment also recovers the simulation ordering: novelty >> random >> residual for both task-1 retention and task-average retention, with a small cost in task-5 accuracy. These results are reported in Appendix D.6, Fig. 14.

These experiments support the qualitative predictions under evolving representations. Analytically, extending the framework would require tracking how selection changes both the representation and subsequent updates; this remains beyond the present derivation.

We hope these clarifications and additional experiments address your concerns and strengthen the evidence for our main findings. If so, we would be grateful if you would reflect these improvements in your updated assessment and rating. Thank you again for your constructive feedback.

AuthorsOpenReview ↗

Rebuttal Follow-up

Dear Reviewer,

We wanted to kindly follow up on our rebuttal and the revised manuscript. We would be grateful to hear whether our revisions address your concerns, or if there are any remaining questions we could clarify before the discussion period ends.

Thank you again very much for your time and consideration!

Reviewer_MG3MOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper provides a theoretical analysis of continual learning in terms of sample selection for current task learning rather than directly focusing on the architecture or sample gradient update control. The paper builds upon the linear teacher–student framework using task-specific transformation matrices and a Gaussian latent vector. This provides careful control of the task similarity in both feature and label space. The authors provide two ways to compute sample importance and provide theory connecting sample importance, latent variance, and forgetting with respect to uniform sampling. The authors show that novelty-based scoring has an improved and consistent tradeoff compared to residual-based scoring. The paper provides evidence of theoretical results using both simulated and real-world datasets.

Critical strengths
  • This paper provides a theoretical analysis that connects sample difficulty, sample importance, and task similarity, with a special focus on sample selection during current task learning. This is meaningful and important in the CL field.
Critical weaknesses
  • The scope of theoretical contribution is too narrow. The paper only studies the sequence of 2 tasks. The authors are encouraged to extend the analysis to multiple tasks.
  • The paper does not provide an empirical comparison of the two provided selection methods against the existing selection methods for CL (for the current task (>1) selection). Also, other importance scores are mentioned in the literature review but not compared empirically. Authors are encouraged to provide comparisons with SOTA exemplar-free CL methods as well. Examples:
    • "Online Coreset Selection for Rehearsal-based Continual Learning" - ICLR 2022
    • "Active Learning for Continual Learning: Keeping the Past Alive in the Present" - ICLR 2025
    • "Coresets are more than replay: a data-centric view of continual learning" - Neural Computing and Applications 2026
  • Authors are encouraged to improve the writing as well. In several places, the notations are not defined or discussed properly. Some examples are as follows:
    • The same notation xx can sometimes mean a task 1 sample and sometimes mean a task 2 sample, which is confusing.
    • The legend in Figure 3 is also confusing, where a solid line means sampling and a dotted line means reweighting. Simply writing "solid" and "dotted" would be much clearer.
    • ρa\rho_a and ρb\rho_b not defined properly.
    • In line 112: sss_s and sps_p are defined by not used directly. Using dd should be enough to explain and simplify, as done in Figure 1b. Also, in Equation 12, a~\tilde{a} is not defined.
    • In lines 59- 61: what are "feature overlap" and "readout similarity"? What is readout? Please define them.
    • Lines 53-55: What is meant by "at matched task-2 performance"?
    • The introduction does not discuss "Data selection in continual learning" properly. Also refer to the list of past work mentioned above.
    • Lines 104-108, please define the notations properly: NsN_s, "readout matrices", NxN_x, NyN_y,
Questions for authors

Please refer to the weakness section.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 2 (Part 1/2)

We thank the reviewer for finding our work meaningful and important to continual learning, and for the careful reading and constructive suggestions. In response, we extended our evaluation to five tasks in simulations and on CIFAR-100. We also added theoretical connections to other scoring rules, expanded our discussion of the suggested methods, and clarified the notation and presentation. The manuscript revisions are highlighted in blue in the updated PDF.

W1. Extension beyond two tasks.

We focus on two tasks to isolate how feature and readout similarity interact with sample selection. Following the reviewer's suggestion, we have added five-task reweighting simulations. We observed a consistent result: novelty-score-based selection is robust to task similarity, whereas the efficacy of residual-score-based selection is conditional. Specifically, we studied three settings: i) sustained agreement, ii) sustained conflict, and iii) a switch in readout relationships, where tasks 4-5 agree with each other but conflict with tasks 1-3.

We computed residual scores from the preceding model and novelty relative to the combined input subspace of all earlier tasks. At matched current-task loss, novelty improves task-1 retention and average past-task retention in all three settings, while residual scoring improves them under sustained agreement and worsens them under sustained conflict. These findings hold at both uniform-reference regularization values, λ=50\lambda=50 and 750750.

In the switching setting, residual scoring harms retention at task 4 and helps at task 5 under the weaker constraint; under the stronger constraint, its effect at task 4 is small. These results suggest history-dependent effects beyond two tasks. Details are in Appendix A.10 and Fig. 8.

We have also extended the CIFAR-100 experiments to five tasks (Appendix D.6, Fig. 14). Both task-1 retention and average past-task retention follow the same qualitative score ordering as the simulations.

W2. Existing selection methods and broader applicability.

Our goal is to analyze how the effects of sample selection algorithms depend on task relationships, rather than to propose a new selection algorithm. Novelty and residual error serve as representative scoring rules within this framework. To clarify its broader applicability, we have added Appendix B, “Connections to other scoring rules and gradient projection.” It formulates predictive uncertainty and squared-gradient scoring in our setup, relates them to novelty and residual scoring, and explains the connection between novelty and gradient projection.

We appreciate the suggested references and have expanded the Introduction's discussion of current-task selection. OCS (Yoon et al., 2022) and AccuACL (Park et al., 2025) are directly relevant, although their methods use stored past examples: OCS combines current-task selection with rehearsal, while AccuACL uses a memory buffer to estimate accumulated informativeness. Our analysis considers current-task selection without replay. Appendix B captures related ideas rather than reproducing these complete algorithms; empirical comparisons with the full methods remain outside the present evaluation.

Yildirim et al. (2026) evaluate current-task coreset selection across several CL methods. Their finding that uncertainty and GraphCut selection with LwF (Learning without Forgetting) can improve current-task adaptation while worsening retention illustrates the trade-off motivating our analysis, although these criteria differ from our residual score. Our analysis complements their study by isolating the roles of feature and readout similarity.

AuthorsOpenReview ↗

Response to Reviewer 2 (Part 2/2)

W3. Writing and notation.

Thank you for identifying these points. We have revised the explanations and notation as follows:

  • Task identity of inputs and targets. We clarify that (x,y∗)(\mathbf{x},\mathbf{y}^*) denotes a pair from task μ\mu, and task-2 samples in the selection analysis unless stated otherwise (lines 115–116, 168–169).
  • Fig. 3 legend. The caption now specifies solid for selection and dashed for reweighting (lines 335–342).
  • Similarity parameters ρa\rho_a and ρb\rho_b. We define ρa\rho_a as the cross-task correlation of shared feature columns and ρb\rho_b as the correlation of corresponding readout columns in both latent blocks (lines 121–124).
  • Shared and task-specific latent blocks. We retain ss\mathbf{s}_s and sp\mathbf{s}_p to make this distinction explicit, using the same subscripts for associated quantities (lines 119–121).
  • Columns in Eq.(12). We now define the feature and readout matrix columns and the independent Gaussian columns before the coupling equation (lines 717–720).
  • Feature overlap and readout similarity. The Introduction now defines these as similarity in the maps from latent variables to inputs and to targets, respectively (lines 48–50).
  • Matched task-2 performance. We clarify that task-1 retention is compared at the same task-2 loss (lines 51–52).
  • Data selection in CL. We discuss selection of incoming examples for training or labeling and cite the suggested works (lines 37–43 and Section 2).
  • Dimensions and readout matrices. We define NsN_s, NxN_x, and NyN_y before use and introduce the feature and readout matrices as maps to inputs and targets (lines 112–115).

We would appreciate your feedback on whether these revisions provide the strengthening you had in mind. If so, we would be grateful if you would reconsider your assessment and score in light of the revised paper.

AuthorsOpenReview ↗

Rebuttal Follow-up

Dear Reviewer,

We wanted to kindly follow up on our rebuttal and the revised manuscript. We would be grateful to hear whether our revisions address your concerns, or if there are any remaining questions we could clarify before the discussion period ends.

Thank you again very much for your time and consideration!

Reviewer_hHEEOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Authors of this paper study which new-task samples to prioritize so a model learns task 2 without forgetting task 1. Using a linear teacher–student model with “shared” and “task-specific” feature directions, it compares two scoring rules: Novelty and Residual Error.

Critical strengths
  • Asks a clear, under-studied question: how data selection interacts with task similarity.
  • The key sign results are exact and match simulations well.
  • Gives a simple, testable rule for when hard-example selection is harmful.
  • Reports when experiments deviate from the theory.
Critical weaknesses
  • Small effects. In theory, reweighting changes forgetting by only O(1/N_s), and empirical gains are a few percent. The practical relevance is unclear.
  • Results are close to built in. Novelty favors exactly the directions that don’t interfere, so its benefit follows almost by definition. Gradient projection (OGD, GPM) already does this directly, and there is no comparison with it.
  • Strong assumptions. Linear model, frozen features, two tasks, an infinite-sample limit, and an isotropic penalty. The residual analysis assumes large output dimension, but the experiments use 5–8 outputs.
  • No CL baselines. Missing comparisons with GSS, OCS, and influence-based selection. There is no end-to-end training.
  • Minor issue: Some figures, especially heatmaps, are too small to read.
Questions for authors

See the weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 1 (Part 1/2)

We thank the reviewer for recognizing the value of this study and for the constructive suggestions. We clarify why hard selection can produce larger retention effects than reweighting, explain how our scores relate to gradient projection and other selection methods, and show that the predicted retention ordering holds in five-task end-to-end CIFAR-100 experiments. Revisions are highlighted in blue in the updated PDF.

W1. Effect size and practical relevance.

We agree that the reweighting effects are modest. Reweighting provides an analytically tractable way to understand how sample prioritization affects learning and forgetting, while hard selection can produce stronger effects. Within the approximation's validity regime, the leading retention effects scale as O(Ns−1)O(N_s^{-1}) for reweighting and O(Ns−1/2)O(N_s^{-1/2}) for hard selection at a fixed, non-extreme selection budget. The retention effect can also be non-negligible at the finite values of NsN_s we consider (in our setting, NsN_s is much smaller than the feature dimension). We now state this distinction explicitly in Section 5.

Our intended practical contribution is to identify when a chosen scoring rule helps or harms retention. In particular, the analysis explains why prioritizing high-residual samples can worsen forgetting depending on task similarity.

W2. Novelty scoring and gradient projection.

We agree that novelty scoring has an intuitive connection to preserving directions associated with previous tasks. However, prioritizing samples with large components outside the task-1 subspace does not remove their components within that subspace. Their updates can therefore still interfere with task-1. Our analysis shows improved retention at matched task-2 performance, demonstrating that novelty scoring improves the trade-off between learning and forgetting.

We have added Appendix B.4 to make the connection to OGD and GPM explicit. In our linear setting, projection away from the task-1 input subspace preserves task-1 predictions. At W=W1W=W_1, the squared Frobenius norm of the projected per-sample gradient equals the residual score multiplied by the novelty score. Novelty is therefore the input-space factor determining how large a gradient remains after projection. This provides an analytical connection to gradient projection methods.

W3. Assumptions and output dimension.

The fixed-feature linear model, two-task setting, and isotropic penalty allow us to isolate the interaction between task similarity and sample scoring. We agree that these assumptions limit the scope of the theoretical guarantees. The manuscript includes finite-sample and small-output checks, and we have added five-task experiments in response to your feedback:

  • Finite samples: Appendix A.7 replaces the population second moment with its empirical counterpart. For the task pair tested, the retention gaps approach their population values as sample size increases. The real-data experiments provide additional finite-sample tests, including CUB-200 with approximately 300 training images per task.
  • Small output dimension: The large-output assumption approximates the residual score matrix by its block mean. In Appendix A.8, we retain the realized matrix and vary the output dimension. The mean gaps recover the broad agreement–conflict pattern even at small output dimensions, with deviations near the predicted boundary. The supplementary experiments also include CIFAR-100 with 50 outputs (Fig. 11), CUB-200 with 15 outputs, and AwA2 with 85 outputs (Fig. 16), supporting the qualitative findings across a wider range of output dimensions.
  • Five sequential tasks: The added Appendix A.10 (Fig. 8) extends the linear simulations to five tasks. Novelty improves task-1 and average past-task retention at matched current-task loss; residual scoring helps under sustained agreement and harms under sustained conflict. The five-task end-to-end CIFAR-100 experiment also recovers the predicted retention ordering (Appendix D.6, Fig. 14).
AuthorsOpenReview ↗

Response to Reviewer 1 (Part 2/2)

W4. CL baselines and end-to-end training.

Our main objective is to provide a framework for analyzing how sample scoring interacts with task similarity. Novelty and residual error serve as two representative scores within that framework.

To clarify its broader applicability, the added Appendix B discusses predictive uncertainty and squared-gradient scoring, alongside the connection to gradient projection. GSS and OCS use gradient diversity and alignment to select samples; we analyze squared gradient norm as a tractable gradient-based score and relate it to residual error.

Regarding nonlinear training, Appendix D.5 trains a two-layer MLP on frozen pretrained features and recovers the predicted retention ordering with comparable task-2 accuracy. We have now added end-to-end CIFAR-100 fine-tuning of both the backbone and readout (Appendix D.6, Fig. 13). The retention ordering persists, with similar task-2 accuracy at the 70% budget; at 30%, improved retention comes with lower task-2 accuracy.

We have also extended end-to-end CIFAR-100 training to five tasks (Appendix D.6, Fig. 14). Both task-1 retention and average past-task retention follow the simulation ordering: novelty, random, then residual. Current-task accuracy is broadly comparable, though slightly lower for score-based selection than random selection.

W5. Figure readability.

Thank you for pointing this out. We have enlarged the heatmaps in Figs. 5 and 15 to improve their label readability.

We hope these additions strengthen the paper along the directions you identified. If they address your concerns, we would be grateful if you would consider reflecting this in your updated assessment and rating. Thank you again for your review!

AuthorsOpenReview ↗

Rebuttal Follow-up

Dear Reviewer,

We wanted to kindly follow up on our rebuttal and the revised manuscript. We would be grateful to hear whether our revisions address your concerns, or if there are any remaining questions we could clarify before the discussion period ends.

Thank you again very much for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.