acceptodds
Under review as a conference paper at ICLR 2027

Continual Learning as Asymmetric Nash Bargaining with Absent Players

Abstract

Gradient-based multi-task learning methods assume that all task gradients are available when an update is selected. Continual learning violates this assumption: past tasks are absent and can only be represented through imperfect proxies, such as replay buffers, regularization statistics, or frozen models. Unlike ordinary minibatch noise, these proxies may introduce systematic, non-vanishing errors in the gradient and curvature information that defines each task utility. We formulate task-incremental continual learning as an asymmetric -player Nash bargaining problem in which absent players are represented by uncertain proxies, utilities are second order, and disagreement points are tolerable-forgetting budgets. Taking the worst case over bounded proxy errors yields an exact robust bargaining objective whose feasible set is nonempty and bounded without a trust region. We first characterize how proxy errors perturb the bargaining solution, deriving a sensitivity bound governed by the proxy error bounds, bargaining slack, and curvature conditioning. We then connect this perturbation to catastrophic forgetting through a bound that separates the ideal bargaining contribution, proxy-induced update error, and a third-order curvature remainder. Finally, we derive a damage certificate that is computable at the returned update and involves no oracle quantity. Together, these results identify bargaining slack as a central source of fragility in continual learning and provide a principled basis for bargaining with imperfect representations of past tasks. Our source code is available at https://anonymous.4open.science/r/CL-Nash-bargaining-8584/

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Nash Bargaining, Catastrophic Forgetting, Proxy Uncertainty, Gradient Conflict

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xhY8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper formulates task-incremental continual learning as an asymmetric Nash bargaining problem with absent players represented by imperfect proxies. It introduces second-order utilities and tolerable-forgetting budgets, and derives a robust bargaining objective under bounded proxy errors. The analysis connects bargaining sensitivity to one-step loss increases and provides a computable damage certificate. Experiments examine proxy dominance, certificate coverage, curvature approximation errors, and task-incremental CIFAR-100 performance. The curvature bargain outperforms the tested first-order bargains but remains below replay at substantially greater computational cost.

Critical strengths

1.The paper presents a novel formulation of continual learning as an asymmetric Nash bargaining problem.

2.The theoretical analysis connects proxy errors, update deviations, and catastrophic forgetting.

3.The experiments include mechanism diagnostics, and the paper candidly reports its practical limitations.

Critical weaknesses

1.The method does not sufficiently address its central motivation. The paper is motivated by systematic, non-vanishing proxy errors that evolve with parameter drift. However, the proposed robust bargain relies on reliable gradient and curvature error bounds, which are precisely the quantities that are difficult to obtain when past tasks are absent. Training sets the gradient uncertainty radius to zero and uses a curvature uncertainty constant far below the measured operator-norm error. Although the analysis explains how to protect past tasks when error bounds are known, it does not sufficiently explain how this protection can be achieved when proxy errors are unknown and continually changing. The central difficulty is therefore partly shifted to reliably estimating these error bounds.

2.The experiments do not sufficiently isolate the contribution of Nash bargaining itself. The method incorporates second-order utilities, tolerable-forgetting budgets, and worst-case uncertainty penalties into the Nash bargaining objective. However, these components could also be incorporated into optimization methods that directly constrain damage to past tasks. The existing analysis does not sufficiently clarify what additional task-protection properties or update behavior Nash bargaining itself provides.

3.The experimental scope and computational efficiency provide insufficient support for broad effectiveness. The real-data experiments cover only one task-incremental CIFAR-100 setting, a small network, and three random seeds. Systematic evaluations across different numbers of tasks, memory capacities, and tolerable-forgetting budgets are missing.

Questions for authors

1.How can the gradient and curvature error bounds be reliably estimated and dynamically updated when past tasks are absent and proxy errors evolve with model updates?

2.Compared with constrained optimization using the same second-order information, uncertainty penalties, and tolerable-forgetting budgets, what additional task-protection properties or update behavior does the Nash bargaining objective provide? Could controlled comparisons that match these components distinguish the contributions of curvature modeling, budget choices, and the bargaining mechanism, and further clarify how the framework captures update-induced proxy drift?

3.Given that 99.99% of proposed updates require shrinkage, could the authors compare retention of past tasks, learning on the current task, and wall-clock time with and without the admission control mechanism? Could its benefits also be evaluated across different budgets and task-sequence lengths to establish whether high certificate coverage and substantial computational overhead translate into effective continual-learning performance?

1 comment
AuthorsOpenReview ↗

Response to Reviewer xhY8

We thank the reviewer for identifying the central weakness of the submission. Please first see the General Response (GR), which corrects errors in the submission; we apologize for them.

W1 / Q1. How can gradient and curvature error bounds be estimated when tasks are absent and proxy errors drift? We agree with the premise: they cannot be estimated reliably, and the submission trained with r = 0 and ζ far below the measured error. Rather than estimating these bounds, the revised approach no longer needs them. Forgetting is certified directly on held-out examples from each task that never influence training, using concentration bounds on paired 0-1 loss differences relative to each task's anchor (revised Section 6.2). Because the certificate measures realized forgetting at the delivered model, proxy drift during training does not affect its validity. With δ = 0.05 fixed in advance we observe no coverage violations, and the bound width is ≈ 0.03. The quadratic model remains only as a proposal mechanism.

W2 / Q2. What does Nash bargaining add over constrained optimization with the same second-order information, penalties, and budgets? We ran exactly this comparison, with the same proxies and budgets. Our first constrained baseline was flawed (the trust radius equaled τ, GR §1c), which produced a misleading 35-point gap. After correcting it, constrained QP attains the highest A_20 on all three seeds (0.672); bargaining does not (0.647, or 0.663 on seed 0 after tuning). The difference between the objectives is behavioral. At three shared checkpoints (same parameters and proxies, both solvers, 3 solves each), at matched step length (‖Δ‖ ≈ 0.55) the bargaining step causes ~3× lower true damage to past tasks (0.0065 vs. 0.0207) and never makes a constraint tight, whereas QP saturates them. However, this lower per-step damage does not yield a lower certified bound at the delivered model (mean UCB 0.122 vs. 0.113 for QP). We no longer claim an advantage for Nash bargaining (GR §4).

Q3. Admission control on vs. off, budgets, and sequence lengths. We now compare training with and without admission control, plus a fixed step-scaling control with the same mean step length: A_20 is 0.593 (off), 0.551 (fixed scaling), and 0.472 ± 0.120 (admission). In some runs the budget is exhausted late in the stream and the last tasks are not learned; the variance comes from kernel nondeterminism amplified ~38× by the line search and vanishes under deterministic kernels. We therefore certify after training rather than enforcing budgets during training. Across budgets τ ∈ {0.05, 0.1, 0.4}, certification succeeds in 0/18, 1/18, and 18/18 cells. We did not evaluate different sequence lengths within the discussion period; we acknowledge this limitation.

W3. Scope. We agree that a single dataset and backbone are insufficient for broad claims, and we have narrowed the claims accordingly (GR §4).

Reviewer_fUxHOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper casts task-incremental continual learning as Nash bargaining with “absent participants”: the current task is optimized directly, while past tasks are represented through limited replay data and curvature proxies. The method uses second-order utilities, a disagreement point that allows forgetting, and penalties for gradient and curvature uncertainty. The authors analyze feasibility, uniqueness, update sensitivity, and one-step loss increase, and propose a damage certificate and update scaling. The experiments include a quadratic sandbox showing that first-order utilities lose conditioning as past tasks converge, a certificate-coverage study and a true-Hessian diagnostic on a reduced ResNet-18, and a 20-task CIFAR-100 benchmark. On that benchmark the curvature bargain beats the first-order bargains (67.4% vs. 62.4%) but stays below plain replay (68.3%) while taking roughly 125× its wall-clock time.

Critical strengths
  1. The motivation makes sense. Near a past task’s optimum, the gradient is almost zero, so a first-order proxy mostly follows noise. Lemmas 1–3 (Appendix E) and Figure 2 support this.
  2. Using a “forgetting budget” instead of a zero disagreement point suits continual learning, and it keeps the feasible set bounded without an extra trust region (Propositions 2–3). The proofs are solid.
  3. The authors are honest about the weak spots: the certificate is too conservative, the Gauss-Newton proxy is far from the true Hessian, and the method doesn’t beat replay.
Critical weaknesses
  1. The presentation needs improvement. Some key assumptions are not clearly stated. The connection between the theoretical results and the actual implementation is also hard to follow.

  2. The experimental evidence is limited. The evaluation uses only one dataset and one small backbone, with little exploration of the forgetting budget. The method is about 125 times slower than replay but has slightly lower average accuracy. It also does not consistently reduce forgetting compared with the first-order bargains, so better protection of past tasks is not clearly demonstrated. More tuning details and comparisons with stronger baselines, such as DER++, would help clarify its benefits.

  3. Some claims are stronger than the evidence supports, and key assumptions need clearer explanations. The quadratic experiments show that second-order utilities can help, but do not prove they are generally necessary. The Hessian analysis uses a different training setup, so it cannot fully explain the main experimental results. The regularity assumptions behind Theorems 3–4 are also not clearly stated, and Appendix A, which should discuss the assumptions, is empty.

  4. The contribution of Nash bargaining remains unclear. The paper lacks a comparison that uses the same curvature information and forgetting budgets without the Nash bargaining objective.

Questions for authors
  1. How do accuracy and forgetting change with nonzero gradient uncertainty and curvature bounds that cover the measured proxy error?
  2. Was admission control used in Table 3? If so, how much did it shrink updates, and how did it affect accuracy?
  3. How does the method compare with DER++ and a constrained baseline using the same curvature and forgetting budgets?
1 comment
AuthorsOpenReview ↗

Response to Reviewer fUxH

We thank the reviewer for the detailed review. Several criticisms are correct. Please first see the General Response (GR), which corrects errors in the submission; we apologize for them.

W1/W3. Presentation, unstated assumptions, empty Appendix A, overclaims. We agree. All assumptions are now stated: the local regularity conditions for Theorems 3–4 (Section 3.3), the condition ‖ĝ₀‖ > r₀ for Proposition 1 (which also lacked a proof; one is now included), and γ > 0 for boundedness and uniqueness, with a map from each result to its assumptions (Appendix A). We removed the overclaims you identified in the former Sections 6.1–6.4 ("structurally necessary", "indispensable", "tangible improvement", "fully explaining", the inconsistent "exponentially") and rewrote the abstract, experiments, and conclusion around the re-measured results. The Hessian diagnostic, which used a different setup, now appears only in Appendix H as motivation.

Q1. Accuracy and forgetting with nonzero gradient uncertainty and curvature bounds covering the measured error. On the measured setup, a valid two-sided curvature bound would require ζ ≥ −λ_min(H) ≈ 9.6–17.6 instead of 0.01. By Proposition 2 the feasible step norm shrinks as γ = λ_min(H̄) + Z grows, so updates would become negligible. We did not train with such constants. Instead, we moved the guarantee off the proxy: forgetting is certified directly on held-out data, with no r, ζ, or ρ (GR §2).

Q2. Was admission control used in Table 3? No (GR §1a). Run for real, it reduces A_20 from 0.593 to 0.472 ± 0.120, versus 0.551 for a fixed step scaling of the same mean length.

Q3 / W2 / W4. DER++, a constrained baseline with matched curvature and budgets, and the contribution of Nash bargaining. Both baselines are added, tuned on calibration data. Over 3 seeds, A_20 of the delivered model is 0.672 (constrained QP), 0.666 (DER++), 0.665 (first-order bargaining), 0.647 (curvature bargaining), and 0.626 (replay). Our initially large advantage over the constrained baseline was an artifact of the submitted code, where the trust radius equaled τ (GR §1c). After correcting it, Nash bargaining does not improve accuracy; a tuned curvature bargain only matches QP (0.663 vs. 0.661 on seed 0). What distinguishes it is behavior: at three shared checkpoints (3 solves each, one trajectory) and matched step length, its step causes ~3× lower true damage to past tasks (0.0065 vs. 0.0207) and never makes a budget constraint tight. This does not yield a lower certified bound. We have revised our claims accordingly (GR §4), and we agree that the original results did not show consistently lower forgetting.

Cost. Measured with one job per machine: 75 min per run for curvature bargaining vs. 1.4 min for replay. Curvature estimation accounts for 67–74% of the time and the bargaining solver for under 1%, so the cost comes from the Gauss-Newton operator, which constrained QP pays equally. We state this in the limitations.

Reviewer_vswWOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper casts task-incremental continual learning as Nash bargaining between the current task and past tasks, which can only be seen through proxies such as a replay buffer. Each task’s utility uses a second-order model of its loss, and the disagreement point is a forgetting budget rather than zero. The authors take the worst case over bounded proxy errors. They prove that the update exists and is unique, and derive a sensitivity bound, a one-step forgetting bound, and a damage certificate.

Critical strengths

The paper identifies two real problems with Nash-MTL style bargaining in continual learning. With first-order utilities, a converged past task is driven mostly by proxy noise. A zero disagreement point can also leave no feasible update. Using curvature together with a forgetting budget is a reasonable answer to both.

Critical weaknesses

The experiments are too limited to show practical value. There is one dataset, one small network, three seeds and one budget value.

Questions for authors
  • Is the shrinkage rule used in the Table 3 runs, and how much does it affect the results?
  • How does the method compare with replay plus a curvature penalty from the same buffer?
  • Can you add results over several budget values and stronger baselines?
  • Does the certificate still hold when the constants are fixed in advance?
1 comment
AuthorsOpenReview ↗

Response to Reviewer vswW

We thank the reviewer for the focused questions. Please first see the General Response (GR), which corrects errors in the submission; we apologize for them.

Q1. Was the shrinkage rule used in Table 3, and how much does it affect results? No. Admission control was never applied in Table 3; the reported 99.99% was computed after each update had already been applied (GR §1a). We have now run it for real: enforcing the budget during training reduces A_20 from 0.593 to 0.472 ± 0.120, whereas a fixed step scaling with the same mean step length gives 0.551 (revised Table 2). The cost is therefore not just smaller steps, and the large variance disappears under deterministic kernels. We consequently certify the budget after training instead of enforcing it during training.

Q2. Comparison with replay plus a curvature penalty from the same buffer. We added this baseline, using the same Gauss-Newton operator built from the same buffer, tuned on calibration data. It improves as the penalty weight decreases: μ = 0.1 gives A_20 = 0.464 over 3 seeds and μ = 0.01 gives 0.606 on seed 0, still below plain replay (0.626). Since μ = 0.01 is the lowest value tried, its optimum may lie lower; we report it as such.

Q3. Several budget values and stronger baselines. We now report certification at τ ∈ {0.05, 0.1, 0.4}: over 6 methods × 3 seeds, average forgetting is certified in 0/18, 1/18, and 18/18 cells respectively (GR §3). We added tuned DER++ and a constrained-optimization baseline with matched curvature and budgets. Over 3 seeds, A_20 of the delivered model is 0.672 (constrained QP), 0.666 (DER++), 0.665 (first-order bargaining), 0.647 (curvature bargaining), and 0.626 (replay) (revised Table 3). The evaluation still covers a single dataset and backbone; extending it could not be completed within the discussion period.

Q4. Does the certificate still hold when the constants are fixed in advance? The new certificate has no model constants to fix: it is computed on held-out data that never influences training, with the confidence level δ = 0.05 fixed in advance (revised Section 6.2). Under this protocol we observe no coverage violations, and a control experiment confirms that bounds hold simultaneously across all candidates. The original model-based certificate requires a two-sided curvature bound ζ ≥ −λ_min(H) ≈ 9.6–17.6 on the measured setup, far above the 0.01 we used, which is why we no longer use it as a practical tool.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Corrections and Summary of Revisions

General Response

We thank all reviewers for their careful reading. Several of the concerns raised are correct, and some of them reveal errors in our submission. We apologize for these errors and correct them first.

1. Corrections

(a) Admission control was not used in Table 3. The 99.99% figure in Section 6.2 of the submission is a shrinkage factor β* computed after each update had already been applied. It never scaled any update and had no effect on training.

(b) The real-stream results of the submission cannot be verified. The machine holding the exact code versions that produced the submitted tables has failed, and we cannot recover them. Running the public code released with the submission, unmodified, gives replay 62.2% A_20 rather than the reported 68.3%. We therefore treat all real-stream results of the submission as superseded. Every method has been re-implemented and re-measured in a single environment, with code, configurations, and seeds released alongside the revision. All numbers below come from this re-measurement.

(c) A design flaw in the submitted code. The solver's trust radius was set equal to the budget τ, so every budget sweep was also an uncontrolled step-size sweep, and the constrained baseline was crippled. We decoupled the two in all new experiments.

2. Main change: certify forgetting on data, not on proxy constants

Reviewer xhY8 correctly observed that our guarantees required gradient and curvature error bounds that are unavailable precisely when tasks are absent, and that training used r = 0 and ζ far below the measured error. We agree, and we changed the design (revised Section 6.2):

  • The forgetting certificate no longer depends on r, ζ, or ρ. Forgetting is certified directly on held-out data from each task that never influences training (gradients, curvature, or step acceptance).
  • The delivered model is selected among 16 candidates fixed in advance, with a union bound, so all bounds hold simultaneously and the guarantee holds for any continual-learning method (a Learn-then-Test procedure).
  • Section 6.2 proves a Hoeffding bound; the numbers we report use a tighter pooled empirical-Bernstein variant, which we label as such.
  • The quadratic bargaining model is used only to propose updates.

3. New results (CIFAR-100, 20 tasks, 3 seeds, re-measured)

  • Certificate validity. With δ = 0.05 fixed in advance, no coverage violations; bound width ≈ 0.03.
  • Certification by budget. Over 6 methods × 3 seeds, average forgetting is certified at τ = 0.4 in 18/18 cells, at τ = 0.1 in 1/18 (DER++, one seed), and at τ = 0.05 in 0/18.
  • Admission control (all reviewers). Enforcing budgets during training reduces A_20 from 0.593 to 0.472 ± 0.120, whereas a fixed step scaling with the same mean step length gives 0.551. Under deterministic kernels the variance vanishes: it comes from kernel nondeterminism amplified ~38× by the line search.
  • Nash vs. constrained optimization (fUxH, xhY8). After the correction in 1(c), constrained QP attains the highest A_20 on all three seeds (0.672), followed by tuned DER++ (0.666), first-order bargaining (0.665), curvature bargaining (0.647), and replay (0.626). A tuned curvature bargain matches QP on the one seed evaluated (0.663 vs. 0.661). At three shared checkpoints with matched step length, the bargaining step causes ~3× lower true damage to past tasks (0.0065 vs. 0.0207) and never makes a budget constraint tight; this does not yield a lower certified bound (mean UCB 0.122 vs. 0.113 for QP).
  • Accuracy vs. certification. The most accurate method (QP, all seeds) does not obtain the lowest certified bound on any seed, although differences among the leading methods are within the bound width.
  • Cost. Measured with one job per machine: 75 min per run for curvature bargaining vs. 1.4 min for replay. Curvature estimation takes 67–74% of this time; the bargaining solver takes under 1%.
  • Cost of certification. Holding out 250 examples per task costs ~3.5 points of A_20.

4. What we no longer claim

We do not claim that Nash bargaining outperforms other methods in accuracy or certification, that model-based certificates are usable on deep networks, or that forgetting below ~0.1 can be certified on this benchmark. The analysis of why first-order bargaining fails and why the second-order program is well posed is unchanged.

5. Revision

The revised PDF (changes in blue) rewrites the abstract, introduction, experiments, and conclusion around the re-measured results; adds the held-out certificate (Section 6.2) and the re-measured experiments (Section 6.3, Appendix G); moves the submitted real-stream results to Appendix H, marked as superseded; states all assumptions (Section 3.3, Appendix A); and removes overclaims. Code: https://anonymous.4open.science/r/CL-Nash-bargaining-8584/.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.