acceptodds
Under review as a conference paper at ICLR 2027

Implicit Target Shift in Online Learning: Characterization and Correction

Abstract

Online learning from a stream of data is a defining feature of intelligence, yet modern machine learning systems often struggle in this setting, especially under distributional shift. To understand its basic properties, we study the relationship between online and offline learning in the context of kernel regression by deriving a closed-form expression for the function learned by online kernel regression. We reveal that online kernel regression is equivalent to offline regression with shifted, inaccurate target outputs. Conversely, we show that by compensating for this implicit target shift in the teaching signal through target correction, online kernel-based learning can provably learn the same predictor as its offline counterpart. We derive both a closed-form expression for this target correction and an iterative form that can be applied sequentially. Applying this framework to continual image classification tasks on domain-incremental Split CIFAR-10 and CORe50, we show that online stochastic gradient descent with iteratively corrected targets outperforms learning with the true targets. This work therefore provides a basic framework for analyzing and improving online learning in non-stationary environments from the perspective of implicit target shift.

Primary area:
learning theory
Keywords:
online learning, continual learning, kernel method

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YcSrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the difference in the predictors obtained via SGD and offline kernel regression. They argue that by supplying SGD with modified targets, one can recover the offline estimator exactly. They further demonstrate improvements in continual learning experiments on MNIST, CIFAR-10, and CORe50 using the corrected targets.

Critical strengths

The paper provides an interesting perspective on how the predictors obtained via online SGD and offline kernel regression differs. Their proposed idea of correcting the targets so that online algorithm recovers the offline predictor exactly is also something that I found interesting. Experiments on continual learning benchmarks provides empirical evidence of the benefits of such (iterative) corrections.

Critical weaknesses

My main concern is that why does the fact that online learner finds a different predictor than offline learner posed as a critical limitation of online learning? Online algorithms are already known to enjoy strong regret guarantees and optimal convergence rates on excess risk via online to batch conversion. Therefore, deviation from the offline solution does not necessarily imply statistical suboptimality, and it is unclear why correcting for this discrepancy is necessary.

Also theoretical and experimental comparisons with stronger online algorithms such as online KRR and kernelized Online Newton Step are not studied. This makes it hard to accurately judge if target correction mentioned in the paper is something fundamentally meaningful or whether it is merely a mechanism to correct for the limitations of the particular online update rule they study. Further there are online algorithms with optimal strongly-adaptive / dynamic-regret guarantees which are particularly suited for problems with adversarially changing data distribution. Comparison to such methods will also be insightful.

Questions for authors

Please see the section on Critical Weaknesses.

1 comment

W1: My main concern is that why does the fact that online learner finds a different predictor than offline learner posed as a critical limitation of online learning?...

We agree that a discrepancy between online and offline predictors does not necessarily imply suboptimality. However, we respectfully disagree that existing regret and convergence guarantees eliminate the importance of understanding and correcting this discrepancy, particularly in non-i.i.d. learning settings.

First, classical online-to-batch conversion results (e.g., Cesa-Bianchi et al., 2004) establish statistical generalization guarantees under i.i.d. sampling assumptions. Although some online learning algorithms satisfy regret guarantees even for arbitrary sample sequences, these guarantees do not necessarily imply that the final online predictor matches the offline solution or achieves comparable test performance under distributional shift (see our reply to W2 below). As emphasized in our Introduction, our primary interest is online learning under non-i.i.d. data streams, where distributional shift is common. A prominent example is continual learning, in which different tasks or classes are learned sequentially. By construction, the training data in such settings are not sampled i.i.d.

To illustrate this distinction, consider the class-incremental MNIST experiment in Fig. 2. Here, the network first learns from images of digit 0, then digit 1, and so forth, rather than receiving samples randomly drawn from all classes. Although this benchmark is synthetic, it captures an important aspect of distributional shift encountered in practical applications, including medical diagnosis and face recognition.

Fig. 2D compares the test accuracy of offline learning (blue dashed line), vanilla online learning (orange line), and online learning with target correction (green and red lines), evaluated on the MNIST test set containing all ten digits. Vanilla online learning consistently underperforms its offline counterpart, demonstrating that the discrepancy between online and offline solutions is associated with a substantial performance gap in this setting. By contrast, exact target correction recovers the offline performance at the end of learning, as shown in Theorem 4.1, while iterative target correction also substantially improves performance over vanilla online learning. Moreover, a similar, though smaller, discrepancy is observed even under i.i.d. sampling (Fig. 2C), indicating that the effect is not restricted to non-i.i.d. settings.

Our main contribution is to establish implicit target shift as a framework for understanding the discrepancy between online and offline solutions in feature-based learning, and to develop methods to correct it. Our numerical experiments further demonstrate that this perspective may offer algorithms for practical learning tasks with distributional shift. We therefore believe that the established regret and convergence guarantees for online learning does not contradict with the motivation and contributions of our work.

W2: Also theoretical and experimental comparisons with stronger online algorithms such as online KRR and kernelized Online Newton Step are not studied.

Thank you for raising an interesting point. Online learning algorithms with regret guarantees under arbitrary sample orders, such as Online Newton Step, could potentially be useful for online continual learning, as you suggested. However, to the best of our knowledge, such algorithms have received relatively little attention in the continual learning literature.

In the newly added Figs. 5A and B (p. 27 in Appendix of the revised manuscript), we compared iterative target correction with Kernelized Online Newton Step (KONS) on MNIST under random and class-incremental sample orders, using a ConvNet-based NTK. As before, we optimized the hyperparameters of each algorithm over logarithmically spaced grids. We did not include recursive kernel ridge regression (KRR) in this comparison because exact recursive KRR recovers the corresponding offline KRR solution, rather than approximating it through sequential gradient updates. Thus, unlike iterative target correction, it does not directly address the question of how to improve gradient-based online learning.

KONS outperformed vanilla online kernel regression in both settings, but generally performed worse than iterative target correction, except in the later phase of learning under random sample order (brown vs. red lines in Fig. 5A). Thus, iterative target correction showed an advantage even over this stronger online learning algorithm, particularly in the class-incremental setting. One possible explanation is that, although iterative target correction relies on first-order gradient updates, the correction mechanism incorporates additional information into the learning targets, which helps compensate for the limitations of standard online updates.

Reviewer_43DLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper compares single-pass online learning with offline (joint) learning in kernel regression. It first derives a closed form for the predictor of sample-wise and mini-batch SGD started from zero. In this form, the Gram matrix of KRR is replaced by its strictly upper-triangular part (Lemma 3.1, Eqs. 6–7). From this, the authors define effective targets YeY^e such that offline KRR on YeY^e reproduces the online predictor (Thm 3.2). They also show that each new sample shifts its neighbours' effective targets in proportion to its prediction error and kernel similarity (Eq. 11). Inverting this relation gives corrected targets YcY^c. Online learning on YcY^c recovers the offline predictor exactly. Because YcY^c needs future data, the paper also proposes a causal block-wise variant (Prop. 4.2), with gap bounds (Prop. 4.3; O(1/t)O(1/t) under i.i.d. data, Prop. A.5). Experimentally, the iterative correction keeps a conv-NTK regressor on MNIST close to the offline solution (Fig. 2). Empirical-NTK-based corrected targets improve SGD over vanilla SGD and EWC on Split CIFAR-10 and CORe50, while staying below cumulative replay (Fig. 3; memory-matched results in Table 1).

Critical strengths

S1. Exact, verifiable characterization with an interpretable consequence. I checked the following numerically on random instances: It provides rigorous theoretical analysis on the implicit target shift in online kernel regression. Equation 11, together with Figures 1B and 2A–B, gives a concrete interpretation of how a poorly predicted sample shifts its neighbors’ effective targets. This suggests a useful way to reason about interference in online learning and is the main source of value I see in the paper.

S2. Careful and candid evaluation. The evaluation includes several useful controls: hyperparameters are tuned separately for each method (Figure 7), a soft-label control tests an alternative explanation for the gains (Table 1), and the amount of stored past data is ablated (Figure 3C). The paper also candidly reports that replay outperforms target correction at matched memory (Table 1) and explicitly acknowledges that target correction is not a drop-in state-of-the-art method (p. 9). Finally, the DER+TC experiment suggests a concrete application in improving replay-buffer targets.

Critical weaknesses

W1. When memory is matched, target correction underperforms simple replay. With the same memory budget, target correction falls below plain ER (Table 1: 64.57 vs. 69.49 on CIFAR-10; 22.54 vs. 28.29 on CORe50). With full memory, it also falls below cumulative replay (Fig. 3A). Performance is therefore higher when the stored data are used directly as training samples than when they are used through corrected targets. This limits the practical case for target correction as a way of using stored data in neural networks.

W2. Validity beyond the kernel regime and small-scale benchmarks is not demonstrated. The theory is exact for kernel regression; for networks it is carried over through an empirical-NTK linearization. All experiments are on modest-scale image benchmarks: a conv-NTK regressor on MNIST, Split CIFAR-10, and CORe50. The paper therefore offers no evidence that the correction remains valid or useful for language models or other current large-scale architectures. I recognize that several factors make this extension non-trivial such as linearization for the modeling, and objectives, but the general framing ("online stochastic gradient descent") should be scoped to the regimes tested, and this should be stated as an explicit limitation .

W3. Theorems 3.2 and 4.1 follow by a change of variables. The paper presents these as its two core online-to-offline conversion theorems (§6). Because γI+K\gamma I+K is invertible, any predictor of the form k(⋅,X) αk(\cdot,X)\,\alpha equals the offline KRR predictor for targets (γI+K)α(\gamma I+K)\alpha. Such an equivalence therefore exists regardless of online learning. The informative part is the explicit form of YeY^e and its dynamics in Eq. 11, and the paper should present its contribution accordingly.

W4. The relation to label-synthesis teaching is not discussed. Steering an SGD learner toward a target model by synthesising labels is the idea behind iterative teaching by label synthesis (Liu et al., NeurIPS 2021). For least squares, its greedy teaching label blends the learner's prediction with the target model's prediction, which appears closely related to Eq. 18. This limits the novelty of the claimed "new mathematical lens". The results remain correct, but the paper should say precisely what is new relative to these viewpoints.

Questions for authors

Please refer to the strengths and weakenesses

2 comments

W1: When memory is matched, target correction underperforms simple replay.

Thank you for your feedback and positive assessment of our theoretical framework.

One possible explanation for the performance gap between iterative target correction (TC) and replay-based methods such as ER and DER is the difference in how past samples are used for weight updates. In ER and DER, the network weights are updated using gradients computed from both current and replayed samples. By contrast, TC updates the weights using only the current samples, whose targets are corrected using information from the memory buffer. This interpretation is consistent with the competitive performance of DER + TC (Table 1), where the network is trained on both current and replayed samples, with target correction applied to the latter.

We also conducted an additional experiment combining ER with TC, as suggested by another reviewer (please see our reply to Reviewer vK9D). We implemented ER + TC by using the same memory buffer for both experience replay and target correction of the current samples, retaining only 10% of past data in the buffer. The resulting ER + TC algorithm achieved performance comparable to DER++ on CIFAR-10 and outperformed both DER++ and DER + TC on CORe50 (newly added row in Table 1).

These results suggest that the relatively weak performance of standalone TC is partly due to the absence of direct memory-based weight updates. When combined with experience replay, TC achieved performance comparable to or better than DER++. Thus, while standalone TC is less effective than conventional replay methods, target correction can provide additional benefits when combined with replay, even with a limited memory buffer.

W2: Validity beyond the kernel regime and small-scale benchmarks is not demonstrated.

We agree with this limitation. Our theoretical results are established in the kernel regression setting, and the neural network experiments are limited to relatively small-scale architectures and image classification tasks.

However, we emphasize that the main aim of our work is to establish a conceptual framework for understanding the discrepancy between online and offline learning, rather than to develop a practical continual learning algorithm. The neural network experiments in Section 5 were introduced as a proof of principle for our somewhat counterintuitive theoretical result: an online model can achieve better predictive performance when trained on corrected targets than when trained directly on the ground-truth targets. We also note that the practical limitations of our current formulation are explicitly acknowledged in Section 5: "We emphasize, however, that in its current formulation, iterative target correction is not intended as a practical, drop-in replacement for state-of-the-art continual learning algorithms."

We also agree that its applicability beyond the kernel regime and the relatively small-scale neural networks studied here remains an open question. Developing more practical algorithms based on this framework, including heuristic approximations and applications to settings such as streaming continual learning, knowledge distillation, and machine teaching, is an important direction for future work.

W3: Theorems 3.2 and 4.1 follow by a change of variables.

We agree that the algebraic equivalence underlying Theorems 3.2 and 4.1 follows rather directly from a change of variables, and that the explicit formulation of the implicit target shift and its dynamics is an important part of our contribution. However, we believe that the mathematical simplicity of this equivalence does not diminish the conceptual novelty of interpreting the discrepancy between online and offline learning through the lens of implicit target shift and its correction.

If the notions of implicit target shift and correction had already been established, and our contribution were limited to deriving their exact formulations, we would agree with your assessment. However, to the best of our knowledge, aside from a few heuristic approaches involving target modification, this perspective of interpreting online learning through implicit target shift and recovering offline performance through target correction has not been systematically explored. We therefore view the main contribution as establishing this conceptual framework and deriving its concrete implications for online kernel learning, rather than the mathematical complexity of the underlying equivalence itself.

W4: The relation to label-synthesis teaching is not discussed.

Thank you for pointing out this relevant work, which we had overlooked. We agree that there is a conceptual similarity between the two approaches. However, we believe that the motivation, formulation, and main contributions of the two works are substantially different.

Liu et al. (2021) approach the problem from the perspective of machine teaching: given the true target model, the teacher synthesizes labels to guide the learner toward that model as efficiently as possible. In contrast, our starting point is the discrepancy between online and offline learning. We first derive an explicit expression for how sequential gradient updates implicitly shift the effective target outputs (Theorem 3.2 and Eq. 11). Target correction then arises naturally from reversing this shift, rather than from directly optimizing a teaching policy toward a prescribed target model. In particular, we derive an exact correction that recovers the offline kernel regression predictor (Theorem 4.1), followed by a causal iterative approximation that minimizes the discrepancy between the online and offline predictors in the RKHS.

Our iterative target correction algorithm (Eq. 18) bears a resemblance to the greedy label synthesis rule of Liu et al., as both involve the learner's current prediction and a reference prediction. However, unlike the machine teaching setting employed in Liu et al., where the true target function is used for label synthesis, our algorithm estimate the offline predictor from the accumulated data. This leads to theoretical bounds on the online--offline gap under iterative target correction, including for arbitrary sample orders (Proposition 4.3).

We agree that this connection deserves discussion and will include the work of Liu et al. in the revised manuscript, clarifying both the conceptual similarity and the differences between the two frameworks.

Reviewer_vK9DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines the relationship between online and offline learning in the kernel regression setting. The authors analyze both learning frameworks applied to kernel regression to evaluate how the online learning predictor relates to its offline learning equivalent. Through their analysis, the authors derive a closed-form relationship showing that online learning for kernel regression is equivalent to offline learning on a set of effective new targets transformed from original targets using a derived transformation. This transformation can be interpreted as an effective shift of the original targets in the output space. By inverting this shift, the authors derive corrected targets which allow online learning to exactly match the offline learning predictor. However, in order to utilize these corrected targets, one needs access to the entire dataset in advance, as per the established formula, which violates the causal requirement of online learning, as noted by the authors. The paper addresses this issue by introducing a causal approximation of the target correction process that iteratively adjusts the incoming new targets and providing theoretical bounds on the discrepancy between the offline predictor and the online predictor that utilizes these iteratively corrected targets. Under specific assumptions on the data and kernel structure, the expected squared discrepancy between the online and offline predictors gap converges at a O(1/n)\mathcal{O}(1/n) rate. Finally, experiments with Neural Tangent Kernel (NTK) regression task on MNIST dataset provide support for the established theoretical analysis, demonstrating that online learning with corrected targets can approach performance levels of offline learning under a variety of evaluated settings. Further experiments on sequential CIFAR-10 and CORe50 datasets extend the proposed target correction approach heuristically to neural networks, using empirical NTKs and demonstrate improvements over selected baselines.

Critical strengths

Rigorous theoretical analysis of online and offline learning with kernel regression. The paper provides an extensive and detailed analysis and derivation that characterizes a relation between online and offline learning, providing a closed-form expression relating online and offline learning predictors. The derived relation results in an interpretable representation of online learning as an offline learning problem with transformed, corrected targets. Utilizing these derived transformations and the existing limitations in their practical applications (forcing the use of entire dataset knowledge in advance), the authors further introduce an iterative framework that approximates the target correction process, and provide provable bounds on their gap to offline learning. The theoretical analysis is detailed, thorough, informative, and yields insightful and interpretable results.

Critical weaknesses

Limited scope of experimental validation and performance improvement. While the provided results validate the claims of the paper, their scope is limited, in particular when it comes to validating the proposed iterative framework. The theory of the paper directly relates to the kernel regression problem, but the corresponding experimental results that are directly equivalent are limited to evaluating the approach with NTK on a single dataset (MNIST). The presented result focuses on two things: the implicit shift, and the improvement over vanilla online learning. The implicit shift result, while informative, is a natural consequence of the framework's design, so the main contribution is the accuracy comparison with vanilla online and fully offline approaches. Several experiments that would provide more insight into the applicability and performance of the proposed approach are missing. Some examples include: the framework's robustness to distribution changes (while the results in Figures 2 and 3 demonstrate results for different data distributions, a controlled study that varies the degree of data heterogeneity is not provided), how does changing the architecture and kernel in NTK affect the final performance (Appendix C.2 establishes that only one set of settings of the NTK was considered), do the results generalize to different datasets? Furthermore, the authors provide synthetically generated dataset to illustrate the implicit shift and learning curves, but a more detailed analysis of behavior of the iterative framework in such a controlled environment would be greatly beneficial.

The same criticisms apply to the neural network experiments. Additionally, Table 1 reports comparisons where the proposed iterative framework performs worse than the DER++ baseline (on CIFAR10 even when used in a plug-and-play way and combined with the DER method). Furthermore, the combined DER + TC approach exhibits significant performance improvement over all baselines on CORe50 but underperforms DER++ on CIFAR10. These results raise questions about the effectiveness of the proposed solution.

Finally, a noticeable issue, as pointed out by the authors, is the O(n3)\mathcal{O}(n^3) computational cost of updating and inverting the empirical NTK over past data, which results in needing to use sparse sampling of past data. Figure 3c provides a comparison of how different sampling rates affect the final performance, but a full analysis of a performance-computation trade-off is not available in the paper.

Overall, while the theoretical contributions are considerable, the corresponding experimental validation is limited in scope, especially in establishing the generalizability of the proposed iterative target correction framework.

Questions for authors
  • How do different kernel architectures and kernel choices affect NTK regression performance?
  • How well does the iterative target correction generalize to different kernel regression datasets, including in synthetically-generated controlled data? How do different properties of the data affect the algorithm's performance? (for example different heterogeneity degrees of the incoming data streams)
  • Why does TC underperform several baselines in both CIFAR10 and CORe50, and why does DER+TC underperform on CIFAR10 while leading to a robust performance improvement on CORe50? Furthermore, ER shows robust performance on CIFAR10 and second-best performance on CORe50 as a standalone approach, why not try combining it with TC?
  • Could the authors provide a detailed analysis of the accuracy-computation trade-off, including comparisons with existing methods?
3 comments

W1: Limited scope of experimental validation and performance improvement.

Thank you for your positive assessment of our theoretical results and for the suggestions for additional experiments.

The implicit shift result, while informative, is a natural consequence of the framework's design

We agree that the implicit target shift follows rather directly from the offline formulation of online kernel learning (Lemma 3.1). However, we believe this does not diminish the conceptual significance of interpreting online learning through the lens of implicit target shift. Our main contribution is establishing this perspective as a framework for understanding and correcting feature-based online learning, rather than simply demonstrating performance improvements over vanilla online learning.

how does changing the architecture and kernel in NTK affect the final performance

We conducted additional numerical experiments, presented in the newly added Figure 5 (p. 27 in the revised manuscript). These results support the robustness of the advantage of iterative target correction over vanilla online learning across different architectures, kernels, and datasets. We also included a controlled analysis of the effect of sample heterogeneity. Please see our replies to Q1 and Q2 for details.

We also note that Theorem 4.1 guarantees that online kernel regression with full target correction recovers the offline kernel regression predictor, regardless of the choice of kernel or dataset, under the assumptions of the theorem. Although iterative target correction does not have the same guarantee in general non-i.i.d. settings, Proposition 4.3 provides a limited theoretical guarantee, suggesting favorable behavior when the offline kernel regression error approaches zero.

W2: Neural network experiments: performance comparison.

We agree that the neural network results are mixed and that the proposed method does not consistently outperform existing continual learning methods, particularly DER++. However, as we noted in the manuscript, these neural network experiments are not intended to establish iterative target correction as a practical, drop-in replacement for state-of-the-art continual learning algorithms. Instead, they serve as a proof of principle for our somewhat counterintuitive theoretical result: an online model can achieve better predictive performance when trained on corrected targets than when trained directly on the ground-truth targets.

We believe the presented experiments support this point, although they also make clear that the current implementation is not yet competitive with established replay-based methods in all settings. Developing more practical algorithms based on this framework is an interesting direction for future work. Please also see our reply to Q3 for specific points.

W3: The computational cost of updating and inverting the empirical NTK over past data.

We agree that computational cost is a significant limitation of the current implementation of target correction, as we also noted in the manuscript. Since the neural network experiments were intended as a proof of principle, we used a relatively direct implementation of the target correction derived for online kernel regression. This requires computing and inverting the empirical NTK over past data, resulting in substantial computational overhead. This, however, also indicates a room for computationally cheaper approximations. For instance, we showed that the method remains effective with sparse sampling of previous data (Fig. 3C), corresponding to a limited-memory setting. However, we agree that a more systematic analysis of the performance--computation trade-off would be a useful future direction.

More generally, while the current implementation has limited practicality for standard continual learning benchmarks, we believe the concepts of implicit target shift and its correction could have broader applications, including streaming continual learning, settings where minimizing network weight updates is particularly important, knowledge distillation, and machine teaching. Exploring these possibilities and developing more efficient implementations under the constraints of those problem settings are directions for future work.

Q1: How do different kernel architectures and kernel choices affect NTK regression performance?

In the newly added Figs. 5C and D (in the Appendix), we examined how different network architectures used to construct the NTK affect performance. Specifically, we considered two additional NTKs: one derived from an infinite-width multilayer perceptron (MLP) with two hidden layers, and another from a ResNet with three residual groups (see Appendix C.2 for details). In both cases, we observed qualitatively similar trends to those obtained with the ConvNet-based NTK (Figs. 5C and D vs. Fig. 2D). Under the ResNet-based NTK, both vanilla online learning and iterative target correction showed lower performance, but the relative advantage of iterative target correction was preserved (Fig. 5D).

Q2: How well does the iterative target correction generalize to different kernel regression datasets, including in synthetically-generated controlled data? How do different properties of the data affect the algorithm's performance? (for example different heterogeneity degrees of the incoming data streams)

In the newly added Fig. 5G, we examined the effects of RBF kernel width and observation noise using the synthetic data generated from a Gaussian process (GP), as in Fig. 1. The advantage of iterative target correction over vanilla online learning, measured by generalization error, was robust across the tested kernel widths and noise levels. We also applied the ConvNet-based NTK to class-incremental CIFAR-10 and observed qualitatively similar trends, although the test accuracy remained relatively low even with full offline learning (Fig. 5E).

Regarding the degree of data heterogeneity, we conducted an additional experiment in which we varied the sample uniformity by randomly shuffling a fraction ρsu\rho_{\text{su}} of MNIST samples from the class-incremental presentation order (Fig. 5F). Here, ρsu=0\rho_{\text{su}} = 0 corresponds to the original class-incremental setting, while ρsu=1\rho_{\text{su}} = 1 corresponds to a fully shuffled, approximately i.i.d. presentation order. We found that the advantage of iterative target correction over vanilla online kernel regression was preserved across the tested values of ρsu\rho_{\text{su}}. Interestingly, the largest performance gap appeared at an intermediate value of ρsu\rho_{\text{su}}, rather than at the class-incremental limit.

Q3: Why does TC underperform several baselines in both CIFAR10 and CORe50, and why does DER+TC underperform on CIFAR10 while leading to a robust performance improvement on CORe50? Furthermore, ER shows robust performance on CIFAR10 and second-best performance on CORe50 as a standalone approach, why not try combining it with TC?

Regarding the different effects of combining DER with TC on CIFAR-10 and CORe50, one possible explanation is that CIFAR-10 was implemented as a domain-incremental task, whereas CORe50 was implemented as a class-incremental task. Intuitively, in class-incremental learning, target correction allows the teacher to convey not only that "the correct label is dog," but also that "the correct label is dog, not horse or truck," through the corrected target vector. We suspect that this type of correction may be particularly beneficial in class-incremental settings, although further investigation is needed to establish this explanation.

Regarding the performance of standalone TC relative to ER and DER, one important difference is how past samples are used. ER and DER train on both incoming and replayed examples, combining their losses in a single SGD update. In contrast, TC uses past samples to compute corrected targets, while the actual weight update is performed only on the incoming examples. Thus, although all three methods use a memory buffer, TC uses past samples in a qualitatively different way. This may partly explain why replay-based methods outperform standalone TC.

Following your suggestion, we conducted an additional experiment combining ER with TC by incorporating memory-based weight updates into the TC algorithm. Interestingly, the resulting ER + TC algorithm achieved performance comparable to DER++ on CIFAR-10 (DOMAIN-IL) and outperformed DER++ on CORe50 (CLASS-IL), as shown in the table below (the other results are reproduced from Table 1 in the Appendix). ER + TC was implemented by simply adding experience replay to the weight updates based on target-corrected current samples. We used 10% of past data as a memory buffer for both experience replay and target correction (see Appendix C.3 for details).

These results support our interpretation that the relatively weak performance of standalone TC is partly due to the absence of direct memory-based weight updates. Combining TC with ER achieved performance comparable to or better than DER++. This suggests that target correction can provide additional benefits when combined with conventional experience replay, even though standalone TC does not outperform replay-based methods.

We also note that DER++ uses two memory-based loss terms in addition to the loss on incoming samples, whereas ER + TC uses a single replay loss alongside target-corrected learning on incoming samples. Thus, the two methods differ in how they use past samples during training.

METHODCIFAR-10 / DOMAIN-ILCORe50 / CLASS-IL
Vanilla SGD57.19 ± 2.6113.59 ± 1.27
Soft Label (SL)58.05 ± 2.4511.23 ± 0.74
ER69.49 ± 1.3628.29 ± 1.40
DER66.64 ± 1.3212.04 ± 0.60
DER++69.90 ± 1.4527.59 ± 1.61
TC (with 10% samples)64.57 ± 1.9222.54 ± 1.09
DER+TC67.64 ± 1.4432.04 ± 1.21
ER+TC69.71 ± 1.2534.52 ± 1.57

Q4: Could the authors provide a detailed analysis of the accuracy-computation trade-off, including comparisons with existing methods?

For the CIFAR-10 domain-incremental task, the wall-clock time per run was approximately 17 seconds for DER, compared with 196 seconds for iterative target correction when using all previous data, and 30 seconds when using only 10% of previous data for target correction.

Thus, sparse sampling substantially reduced computational cost, although the current implementation remained slower than DER. We agree that a more systematic analysis of the accuracy--computation trade-off, including comparisons with existing methods, would be useful. However, as discussed in our reply to W3, developing computationally efficient implementations is beyond the scope of the current proof-of-principle study.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.