acceptodds
Under review as a conference paper at ICLR 2027

BALANCE: Benchmarking Continual Learning under Changing Deployment Budgets

Abstract

Continual learners adapt to changing tasks under deployment budgets that can vary independently. We introduce Benchmark for Adaptive Lifelonglearning with Network Control and Elasticity (BALANCE) for studying model lifecycle control under these joint changes. External requests reclaim capacity at announced deadlines and return it later. A controller schedules resizing, candidate training and deployment while the current learner serves the task stream. Preparation and updates consume interaction time; ready deployment is instantaneous. Two recurrent prediction families combine locally shared task structure with limited deployment capacity: shared linear rules support analytical capacity calibration, and shared nonlinear features support evaluation with ordinary task-conditioned MLPs. We develop a resource- and deadline-aware model-maintenance algorithm whose rule-based implementations provide reference baselines in BALANCE. An accompanying feasibility study uses imitation-initialized reinforcement learning to learn lifecycle decisions under the same resource protocol. Paired experiments show that maintenance cadence changes the trade-off between prediction performance and model-maintenance work at similar average deployed sizes. A controlled throughput-shift study in the linear family finds that sensitivity to maintenance speed accompanies increased candidate cancellation and reduced deployed capacity despite resource compliance.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Dynamic Resource Constraints, Adaptive Model Capacity, Model Maintenance, Benchmarking

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wHQ1OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The main aim of this work is to formalise and propose a new benchmark based on resource allocation in dynamic environments, where there are multiple types of constraints and deadlines. One of the contributions is to present this problem in an environment where tasks and constraints may change over the course of the problem, turning it into a continual learning problem.

Critical strengths

From what I understand of the work, its main potential contribution lies in the formalisation and creation of a new benchmark. I am not familiar with prior work in this specific area, but I imagine this benchmark could fill a gap in existing benchmarks. Creating a comprehensive benchmark is no easy task and usually requires ongoing feedback in the community.

Critical weaknesses

The biggest weakness lies in the writing, which makes it difficult to understand the depth and details of the work. There is no clear motivation, no identification of the gap this work seeks to fill (Why is this benchmark important?), and no clear description of the problem or the benchmark presented. Perhaps the work contains a very good idea and has valuable contributions to offer the community, but the current version is unable to convey it properly, and that contribution may be lost.

Questions for authors
  • What gaps does this work aim to fill?
  • Could you provide a simple and specific explanation of the problem you are seeking to solve? Let’s leave the mathematical aspects aside for the moment.
  • Concerning the Continual Learning aspect, what sort of distribution shift do you encounter in this problem? What sort of forgetting might the model experience? What sort of knowledge transfer would be beneficial?
  • For the two families of benchmarks presented: what is the relationship between them, and what motivations and gaps are they seeking to address?
4 comments

Thank you for recognizing the potential value of the formulation and benchmark. Below, we address your questions in turn.

Q1. What gaps does this work aim to fill?

Thank you for this question. We focus on the interaction between a model's continual adaptation and the capacity allocated by its platform.

Consider a model deployed on a cluster with limited resources to provide an ongoing service. Other tasks running on the cluster may require additional resources. Can the model autonomously decide to compress its deployed weights, freeing capacity for those tasks, and then adjust its size again to use the capacity returned when they finish?

The model therefore faces two challenges: (1) performing its own deployed task; and (2) adjusting the size of its weights in response to external resource-scheduling notices.

To our knowledge, this joint problem lacks a dedicated benchmark. Providing one is our primary motivation.

More specifically, a learning method may need to shrink at an externally specified time, keep serving while preparing its replacement, and relearn when capacity returns. Knowledge and resource availability therefore evolve differently: the platform can return storage immediately, while recovering predictive knowledge takes learning.

BALANCE brings external reclamation notices, effective deadlines, matched returns and timed model maintenance into a common evaluation interface. Its contribution is to make these coupled decisions comparable across algorithms under the same conditions.

This enables concrete research questions: how early should a smaller model be prepared; how much learning should precede its deployment; how often should an unchanged-size model be refreshed; and how should a policy change when maintenance becomes slower? The benchmark supplies controlled task and resource streams for examining these questions. The rule-based maintenance algorithm provides reference strategies, and the learned-control experiments examine whether decisions can be learned through imitation and reinforcement learning.

Q2. Could you provide a simple and specific explanation of the problem you are seeking to solve? Let’s leave the mathematical aspects aside for the moment.

Thank you for this question. Imagine a continually learning model running on a cluster. It keeps receiving prediction requests and learns from their feedback. Another application starts, and the cluster notifies the model that it must reduce the size of its deployed weights by a specified deadline. A smaller replacement can be trained while the current weights continue serving requests. The model must decide when to begin preparation, how much training to perform, and when to switch weights. Later, the other application finishes and capacity returns. The model can expand again, but adding parameters does not restore knowledge removed during shrinking.

Specifically, a continually learning model must meet two challenges: (1) performing its original continual-learning task; and (2) adjusting the size of its weights in response to external resource-scheduling notices.

BALANCE separates this process into a controller and the continually learning model it manages. Based on external resource-scheduling notices, the controller adjusts the managed model's weight size so that it achieves as high an average score as possible over a long horizon. BALANCE evaluates the controller's ability to manage this process over a continuing task stream.

Q3. Concerning the Continual Learning aspect, what sort of distribution shift do you encounter in this problem? What sort of forgetting might the model experience? What sort of knowledge transfer would be beneficial?

In the deployment setting described in Q1–Q2, continual task adaptation and external resource scheduling create two interacting demands on the same model.

Distribution shift. The problem faced by the deployed service changes over time as operating conditions or the mix of requests change. Consequently, recent observations may follow a different task distribution from earlier ones, and the model needs to adapt using incoming feedback. The cluster's allocation can change independently because of other applications. A capacity change affects the learner's available representation. The setting combines a changing task distribution with an externally changing deployment constraint.

In the linear family, members of a group share input directions but use different coefficients to combine them into a target. For the same input, members A and B therefore generally require different outputs and have different joint distributions of inputs and targets. Switching groups also changes the shared input subspace and the active members, requiring adaptation to another set of linear prediction rules.

In the nonlinear family, members of a group share nonlinear features and combine them using different coefficients. Changing members changes the nonlinear function currently being predicted. Changing groups also changes the input projections and nonlinear responses that generate those features, so the change involves both the shared features and their combinations.

In both families, each member's target rule remains fixed, and the input sampling distribution stays the same across task changes. Members are sampled with fixed probabilities within a group. Thus, members have different conditional task distributions, while the overall mixture remains stable during one group visit. Group changes alter the active member set and its mixture over time, and earlier groups subsequently return. Resource changes are generated independently and can occur within a group visit or coincide with a group change.

Forgetting. Learning from recent data can interfere with knowledge useful under earlier conditions. Shrinking to satisfy a resource request can additionally remove parameters carrying useful knowledge. The controller needs to consider how continual learning and capacity changes jointly affect the model's existing abilities.

Specifically, we ask whether error on a previously learned task rises after subsequent learning or shrinking, although its target rule and input distribution remain unchanged. The default learner trains candidates using a bounded replay buffer. As new samples replace earlier-group samples, subsequent training places fewer constraints on earlier tasks; once a candidate is deployed, this can appear as worse performance on those tasks.

In the linear family, each member has its own output coefficients, but predictions also depend on a low-rank basis shared by all members. Updating this basis while learning new members or groups can change an earlier member's predictions even if its coefficients are retained. When the directions needed by earlier and current groups exceed the available basis capacity, adapting to the current group makes retaining earlier directions harder. Shrinking can directly remove some of those directions.

In the nonlinear family, members have different task embeddings but share the hidden layers and output layer. Learning the current group updates these shared weights and can change the nonlinear responses used by earlier tasks. Adjusting existing hidden units for a new group can impair earlier predictions; shrinking additionally removes hidden units and connections and can cause further loss of learned features.

When an earlier group returns, prediction error and recovery under the same task distribution allow knowledge retention to be assessed. The setting therefore includes forgetting caused by continued updates at a fixed model size, together with additional knowledge loss caused by externally required shrinking.

Beneficial transfer. Useful transfer means preserving and reusing knowledge that remains relevant across task conditions and model sizes. Shared features learned under earlier conditions can reduce the effort needed to adapt to related later conditions. When shrinking, retaining useful behavior in the smaller model can reduce the loss of predictive ability; when expanding, carrying forward the smaller model's learned representation can provide a starting point for further adaptation. These forms of transfer help the model satisfy its original learning objective while responding to the cluster's resource demands.

In the linear family, members of a group share a linear subspace, so representations learned from some members can help learning on others. In the nonlinear family, members combine the same nonlinear features using different coefficients, allowing those features to be reused. When a group returns, retained knowledge can help restore prediction performance; carrying learned weights forward during growth provides a starting point for further adaptation.

BALANCE makes these two challenges jointly evaluable: the controller must respond to external notices while maintaining the managed model's predictive performance over the long task stream.

Q4. For the two families of benchmarks presented: what is the relationship between them, and what motivations and gaps are they seeking to address?

These are two separately constructed families of prediction tasks under the same resource and model-maintenance protocol. Their settings and roles are summarized below:

Shared linear rulesShared nonlinear features
RelatednessMembers share a linear subspace and have distinct coefficients.Members combine shared nonlinear features with distinct coefficients.
Deployed learnerA bilinear predictor with a shared low-rank representation.An ordinary task-conditioned, two-hidden-layer MLP.
Capacity calibrationAn exact fixed-model approximation result within the bilinear family.Local learning, transfer, revisitation and finite-budget joint-fitting diagnostics.
PurposeMake the relation between local learnability and joint capacity analytically transparent.Examine maintenance behavior with nonlinear targets and a conventional neural predictor.

Motivation for shared linear rules: We use simple tasks that are easy to analyze. A bilinear model within the deployment cap can represent one task group, while a single fixed model of that size cannot represent all tasks exactly. This gives clear grounds for the capacity needed by one group and the overall capacity restriction, allowing the model cap to be set accordingly. With these properties specified, it is easier to interpret maintenance strategies under task and resource changes.

Motivation for shared nonlinear features: We use nonlinear tasks and an ordinary MLP to study methods during nonlinear representation learning. Local learning, transfer, revisits and joint fitting check whether the current group is learnable, whether shared knowledge can be reused, and whether the selected model faces capacity pressure across all tasks. This lets us study how resizing interacts with knowledge retention and readaptation using standard neural-network training, and provides common test conditions for different inner learning methods.

Problem addressed: When tasks keep changing and deployment capacity is reclaimed and returned externally, how should a model be resized, trained and deployed to meet resource constraints while keeping its long-term average performance high? Under the same protocol, the two families provide tests that are easy to analyze and tests with nonlinear relationships, allowing controlled comparisons between algorithms.

I appreciate the authors’ detailed answer.

I now have a better understanding of the motivation and the problem the benchmark addresses; however, it is unclear how the authors plan to incorporate these changes into the paper.

One problem that I see is that the benchmark is trying to combine two very complex problems into one (doing continual learning and adjusting the size of the weights). It is difficult to see the relationship between the two problems, which weakens the contribution of the paper. The idea is novel: I have not seen the problem of adjusting the size of the weights in a continual scenario before. However, there is still a significant gap between this version of the paper and a version that can be accepted.

I would recommend simplifying the problem and rewriting the paper more clearly. If, after this step, adding more complexity still makes sense, it can be added as an extra contribution or ablation. It would also help to make the link between the proposal and other benchmarks that may be similar, or that tackle similar problems.

I am keeping my score.

Reviewer_F7DJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BALANCE studies continual learners whose deployment capacity is reclaimed and returned by an external scheduler. A controller prepares, trains, and deploys resized candidates while the current model serves a recurrent task stream. Two synthetic regression families support capacity calibration and comparisons of rule-based and learned scheduling. Experiments demonstrate maintenance–performance trade-offs and sensitivity to maintenance throughput.

Critical strengths
  • External capacity changes and maintenance deadlines provide a useful, promising extension of budgeted continual learning.
  • The deployment contract, timing rules, linear capacity certificate, and paired statistical analyses are carefully specified.
Critical weaknesses
  • Evaluation covers only recurrent synthetic regression. The paper uses shared linear rules or nonlinear features. No real-data stream tests whether the maintenance–performance trade-offs extend beyond this controlled task structure. No evaluation has been made on any external CL methods either.
  • The nonlinear capacity interpretation remains conditional on training. Table 1 and Appendix B.2 report joint MSE of 0.248–0.314 at width 16 versus 0.013–0.023 at width 128 after 65,536 updates, including width-16 Adam/SGD controls. These finite fits do not separate optimization difficulty from representational limits.
Questions for authors
  • What evidence supports transferring the maintenance–performance findings beyond the two recurrent synthetic task families?
  • Does the cadence–work trade-off, including relative controller rankings, persist across the optimizer/replay variants in Table 10 or alternative resizing rules?
  • What evidence separates nonlinear representational limits from optimization difficulty, and is the capacity claim intended only for the tested training recipe?
  • Is it possible to add more CL methods for evaluation under the proposed benchmark?
5 comments

Thank you for recognizing the value of the external-capacity setting, deployment contract and paired analyses. Below, we address each concern and question raised in your review.

W1. Evaluation is restricted to recurrent synthetic regression, without real-data streams or external CL comparisons.

We agree that real-data evaluation can strengthen the benchmark. However, the submitted evaluation focuses on a simplified environment. Real data streams introduce additional factors that may make experiments more complex and the benchmark harder to get started with. We believe the adaptation and scheduling challenges already observed in our evaluation make this simplified setting worth studying.

To directly test the applicability of these findings to real data, we added a handwritten digit classification stream and compared maintenance frequencies under the same outer protocol; Q1 reports the complete paired results. Q4 further explains how existing CL methods can be combined with outer maintenance control.

W2. Nonlinear joint fitting does not separate optimization difficulty from representational limits.

Thank you for highlighting this key issue. We added longer training, independent learning-rate selection, more initializations and known-representable targets to investigate the effects of optimization and model size on joint fitting. Each configuration receives 262,144 updates with batch size 128 and a full long cosine schedule. Adam and SGD each use three learning rates and three initializations on an independent development bank. Confirmation then uses six width-16 initializations and three width-128 initializations on each of the three original task banks. SGD uses .9 momentum.

The known-representable control generates targets with exactly the width-16 student's MLP architecture. Global affine output normalization is absorbed into the final layer, so an exact representation exists. Three teachers and three independent student initializations give nine controls per optimizer. All 81 development and confirmation fits completed. The key results are:

TargetWidthOptimizerLearning rateConfirmation runsMean MSE
Original tasks16Adam.00118.24209
Original tasks16SGD.0118.23900
Original tasks128Adam.0039.00279
Known-representable16Adam.00191.841e-5
Known-representable16SGD.0194.218e-4

After independent learning-rate selection, longer training and multiple initializations, width-16 models still have substantial error when fitting the original tasks jointly, while models of the same width fit known-representable targets. These results strengthen the experimental evidence for capacity pressure in the selected model configuration.

Q1. What evidence supports transferring the maintenance–performance findings beyond the two recurrent synthetic task families?

We added online classification using real images from the UCI Optical Recognition of Handwritten Digits data. Five digit-pair tasks recur. A shared ten-class MLP predicts from the image, and the controller features also omit task IDs. All configurations were determined using the original training writers; the formal online stream uses the original test data from different writers. Each label follows its prediction, and the finite images recur.

This instance retains the same exogenous reclaim/return protocol and deadline guard. Widths 1/2/4 contain 87/166/330 deployed parameters; replay holds 256 examples, batch size is 32, and ordinary jobs request 16 updates. Maintenance throughput is 33,000 units per interaction: maximum-width preparation and updates take 17 interactions once feedback is available, with a minimum reclaim-notice lead of 64. ASAP-64 retains ASAP's capacity-response rule and requires at least 64 interactions since the previous job start before voluntary same-width maintenance; resizing and deadline recovery bypass this wait. Each method uses the same ten task/resource scenarios, with 65,536 interactions per lifetime. All 50 lifetimes completed and were resource-compliant.

ControllerOnline accuracyMean deployed parametersMaintenance work (G)
Minimum87.62%87.001.525
ASAP93.68%294.551.878
ALAP93.72%294.841.878
Periodic-ASAP92.06%294.551.041
ASAP-6486.14%294.52.485

Table entries are means over ten lifetimes. G denotes a billion declared maintenance-work units, including preparation and training actually consumed per lifetime. Relative to ASAP, Periodic-ASAP reduces work by 44.56% with an accuracy difference of −1.62 percentage points; the paired 95% interval is [−2.12, −1.09] points. The sparser ASAP-64 reduces work by 74.16% with an accuracy difference of −7.54 points and interval [−8.53, −6.60]. The three policies have similar mean deployed sizes and occupy different positions on the performance–work relation.

The trajectories also show differences in adaptation after task switches: ASAP and ASAP-64 average 31.78% and 8.82% accuracy in the first 64 interactions of a visit, while both exceed 99% in its final 64. These statistics average complete visits within each scenario before aggregating scenarios, excluding the initial and right-censored final visits. The results extend the maintenance–performance observation to real-image classification with a different task-information setting and show how cadence changes the adaptation process after switches.

Q2. Does the cadence–work trade-off, including relative controller rankings, persist across the optimizer/replay variants in Table 10 or alternative resizing rules?

The new crossed experiments show that work savings persist across all five tested recipes, while performance changes and rule rankings depend on the inner recipe. Periodic-ASAP scores slightly below ASAP with FIFO and improves reward in both reservoir configurations.

We crossed Adam/FIFO, Adam/reservoir, SGD/FIFO, SGD/reservoir and paid SVD shrinking with ASAP, ALAP and Periodic-ASAP. Each cell uses the same ten new linear task/resource scenarios, with 65,536 interactions per lifetime: 150 rule tests. Replay holds 256 examples without an additional recent-data batch, and SGD has no momentum. The optimizer/buffer recipes retain Table 10's development-selected rates, respectively .01/.0003/.3/.0003; SVD inherits Adam/FIFO's .01.

Inner recipeASAP rewardALAP rewardPeriodic-ASAP rewardPeriodic−ASAP [95% interval]
Adam/FIFO.77602.77639.76979−.00623 [−.00786, −.00468]
Adam/reservoir.01492.01483.07580+.06088 [+.05151, +.07210]
SGD/FIFO.77335.77376.76902−.00433 [−.00621, −.00263]
SGD/reservoir.24184.24169.25492+.01308 [+.00635, +.02170]
Adam/FIFO + SVD.77694.77655.76916−.00778 [−.00967, −.00597]

Periodic-ASAP reduces maintenance work by 21.95%–21.96% across all five recipes. For Adam/FIFO, mean lifetime work falls from 164.48M to 128.36M, with mean deployed parameters of 640.08/639.30. M denotes a million declared work units: consumed preparation plus training, retaining work spent on canceled jobs. SVD's extra preparation cost is included. The paired ALAP-minus-ASAP intervals include zero in all five recipes. In contrast, the Periodic-minus-ASAP reward difference becomes positive in both reservoir configurations, showing that the performance cost of cadence and rule rankings are recipe-dependent.

We also evaluated frozen DAgger and Post-PPO weights selected on validation data under Adam/FIFO, SGD/FIFO and SVD: three training seeds per method crossed with the same ten scenarios, totaling 180 transfer tests. Learned entries below average 30 lifetimes; ASAP averages the corresponding ten scenarios. All 330 crossed tests completed and were resource-compliant.

Inner recipeASAP rewardDAgger rewardPost-PPO reward
Adam/FIFO.77602.76920.76220
SGD/FIFO.77335.76576.75980
Adam/FIFO + SVD.77694.76963.76322

The mean ordering is the same in these three transfer recipes. Relative to Adam/FIFO, the four paired intervals for changes in learned-minus-ASAP reward gaps under SGD/FIFO or SVD include zero; all three Post-PPO-minus-DAgger intervals also include zero. Rule intervals resample paired scenarios, while learned intervals resample both training seeds and scenarios. All are pointwise 95% intervals from 20,000 resamples. These results directly support work savings across recipes and show that the performance trade-off should be evaluated for the particular inner learning configuration.

Reviewer_PJGzOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces adaptive lifelong learning with Network Control and Elasticity (BALANCE) for studying model lifecycle control under the changes of tasks or deployment budgets. The paper makes three main contributions: a mathematical formation for lifecycle control, the BALANCE benchmark, and a resource- and deadline-aware maintenance algorithm. Experimental evaluations are conducted to evaluate the performance of the proposed algorithm.

Critical strengths

The paper studies an interesting problem and proposes a benchmark comprising two task families.

Experimental evaluations are conducted under two different profiles, including a feasibility study of assess learned lifecycle decisions.

Critical weaknesses

The paper is difficult to follow since there are many terms not clearly explained or defined. For example, the ‘Dynamic resource-accuracy allocation’ is discussed on page 1 but its meaning is unclear. Furthermore, the term does not appear in the cited paper.

Furthermore, the problem studied and the overall aim of the paper are not clearly presented. Although the paper titled “benchmarking continual learning under changing deployment budgets”, the concept of “benchmarking continual learning” is not explicitly explained in the main the body of the paper. Further the objectives of the evaluation are unclear.

Section 3 is titled problem formulation and deployment protocol. However, the problem inputs and the objective function are not clearly defined. Despite the section title, no protocol is explicitly defined or described in the section.

The experimental evaluation is presented in Section 6. However, the settings for the two families are not sufficiently justified. It is unclear how the settings are determined. The metrics used for evaluation include lifetime mean reward, mean deployed parameters and consumed work. However, none of these are clearly defined in the proceeding sections. Furthermore, no existing methods are chosen as baselines. In addition, the caption of Table 2 introduces “Work” but there is no corresponding column in the table.

For the proposed maintenance algorithm there is no pseudocode presented. Instead, the decision procedure and the schedule rules are described informally.

Questions for authors

How are the two families in Section 4 designed? Why are they suitable as benchmarks?

Why are no existing approaches included as baselines for comparison?

Why are the column headings in Table 2 and 3 different?

5 comments

Thank you for recognizing the research value of the problem and the two task families. Below, we address each concern and question raised in your review.

W1. Explanation and Definition

W1.1. Unclear terminology, including “dynamic resource–accuracy allocation.”

Thank you for the question. We agree that this phrase was insufficiently explained. We intended to describe how changes in the resources available on a shared device can be addressed by adjusting model size or configuration, trading resource use against prediction accuracy. NestDNN (Fang et al., 2018) supports this deployment motivation: its runtime scheduler selects model configurations as resources change.

W1.2. The problem, the meaning of benchmarking continual learning, and the evaluation objectives are unclear.

Our motivation is to let a model continue predicting and learning as external requirements on its deployed model size change. It can use deployment capacity that would otherwise be unavailable, or release some capacity for other tasks.

Consider a cluster or device running a learning service alongside other applications. The service faces a changing mix of prediction tasks; the other applications' resource demands also fluctuate over time. One conservative allocation is to keep a fixed region reserved for the model and leave enough resources for the other applications' peak demand. When those applications use less than their peak allocation or finish running, that headroom remains unavailable to the model. This can be substantial when competing requests are infrequent.

A variable deployment budget lets the model use some of that spare capacity. When another application needs resources, the platform announces how much capacity must be returned and by when. The learning service prepares and trains a smaller replacement while the current model continues serving requests. When an outstanding allocation is released, capacity becomes available again and the controller can consider expanding. Expansion creates room to learn, but does not automatically recover knowledge removed during compression.

Larger models offer additional representational capacity, while resizing and retraining incur work and can affect retained knowledge. BALANCE studies how to manage this trade-off as both the active tasks and the available capacity change. Model size, training effort and replacement timing are decisions made during continuing operation.

“Benchmarking” means providing the same task/resource streams, public information, maintenance actions, cost accounting and compliance checks for competing control methods. We evaluate prediction performance over the entire stream, whether each applicable resource constraint is respected, and how much maintenance work is consumed. The inner predictor continues learning during testing. The outer controller can be a fixed algorithm or a separately trained policy; a trained policy's parameters are frozen during testing.

Our contribution is a formulation and evaluation platform for this problem, together with resource- and deadline-aware rule references. The learned-control study investigates the feasibility of learning the same lifecycle decisions.

W1.3. The inputs, objective and deployment protocol in Section 3 are unclear.

An instance specifies an initial predictor, supported model sizes, a task-stream generator, an external resource process, preparation/update costs, maintenance throughput, and a horizon of HH prediction interactions. The controller observes current deployment and candidate status, recent performance, issued reclamation notices and returns that have occurred. Future returns and hidden task-group switches are not supplied.

Let MtM_t serve interaction tt, let P(Mt)P(M_t) be its complete parameter count, let CtC_t be the available deployment capacity, and let π\pi be the maintenance policy. With JHJ_H denoting the lifetime score defined in Eq. (7), the objective is to maximize Eπ[JH]\mathbb{E}_{\pi}[J_H] subject to P(Mt)≤CtP(M_t)\le C_t at every active boundary. All deployed heads, embeddings and biases count. Actions wait, start a candidate with a target size and update budget, cancel a candidate, or deploy a ready candidate.

We make the execution protocol explicit in the following ordered steps:

  1. Audit the deployed model against the capacity effective at the current boundary.
  2. Execute the maintenance action.
  3. Serve one prediction, then reveal its label.
  4. Advance candidate preparation and training by the declared work allowance.
  5. Perform any requested ready automatic deployment and audit the next active boundary.

External recalls take effect at their announced deadlines without waiting for the controller. Ready manual deployment serves the current prediction; automatic deployment serves the next. Earlier violations remain recorded. A complete pass requires a full lifetime, valid actions and zero resource violations; failed lifetimes remain in the reported results.

Candidate preparation and training use the idealized auxiliary node and consume interaction time. The protocol therefore specifies both what a controller knows and when its decisions affect prediction and compliance.

W1.4. Experimental settings and metrics are insufficiently explained; existing baselines and the work columns are unclear.

The settings are chosen to make local task learning possible while maintaining pressure on joint task representation. Q1 below gives the design and calibration evidence, including the relation between notice and maintenance duration. Q2 addresses existing capacity-control approaches. Q3 clarifies the table columns.

The metrics are:

MetricDefinition
Lifetime mean rewardJH=H−1∑t=0H−1rttrueJ_H=H^{-1}\sum_{t=0}^{H-1}r_t^{\mathrm{true}}. Task reward is clip(1 − squared error, −1, 1), with additional penalties for resource violations and invalid actions.
Mean deployed parametersH−1∑t=0H−1P(Mt)H^{-1}\sum_{t=0}^{H-1}P(M_t), using the model serving each actual interaction.
Consumed maintenance workPreparation and update work actually spent, including partial work and work on subsequently canceled candidates, measured in declared simulation units.

Reward is the performance objective. Parameter use and maintenance work describe the resources consumed to obtain it. Reporting them separately allows performance–work trade-offs to be studied.

W2. The maintenance algorithm lacks pseudocode.

We have made the procedure explicit below. The public-information deadline guard reserves time for cancellation, zero-update preparation and deployment before a threatening deadline. Let A be the resulting admissible action menu and g any guard-prescribed recovery action.

At each controller decision:
    Read public status, issued requests and maintenance prices.
    Construct admissible menu A and any guard action g.
    If g exists:                         return g
    If Deploy is in A:                   return Deploy
    If a candidate exists:
        If it is ready and Cancel in A:  return Cancel
        Otherwise:                       return Noop
    Obtain an idle request (w, n) from the chosen scheduling rule.
    If the rule requests no job:          return Noop
    If Start(w, n) is in A:               return Start(w, n)
    If Periodic-ASAP requests shrinking and Start(w, 0) is in A:
                                         return Start(w, 0)
    Return Noop.

ASAP targets the largest model fitting all currently announced future reductions. ALAP initially uses current capacity and switches to a smaller target when the deadline falls within the estimated job duration plus the decision interval and one deployment boundary. Periodic-ASAP uses size/training clocks and growth hysteresis, and can wait between check times. The guard takes priority for all rules. Task interactions and candidate work continue until the next decision.

Q1. How are the two families in Section 4 designed? Why are they suitable as benchmarks?

Thank you for the question. We explain the design in more detail below.

Overall goal. We aim to construct a task stream for studying continual adaptation: tasks active during a given period are learnable, tasks share reusable knowledge, and the limited model faces capacity pressure when representing all tasks together. We then add an externally changing deployment parameter budget to study how the model can respond to resource requests while maintaining prediction performance over time.

Shared structure. To meet this goal, both families group related tasks, switch the active group over time, and revisit earlier groups. Members of a group share features but use different coefficients to form their prediction rules. Shared structure within a group makes knowledge transfer useful; differences between groups call for adaptation; returning to earlier groups lets us observe retention and recovery of earlier knowledge.

Why use this simple design? Explicit generation rules make task relationships, switching frequency and dwell times easy to set and inspect, and allow fresh labeled samples to be generated throughout the stream. Fixed random seeds let methods face the same inputs, task stream and resource events. Small predictors also help limit the computational cost of long-horizon, multi-seed experiments. This design keeps continual adaptation, knowledge transfer and forgetting central while making experiments easier to repeat, results easier to inspect and algorithms easier to compare.

Interaction. Each member has a fixed prediction rule, and the task distribution changes as the active group changes. Groups recur in shuffled cycles with varying dwell times. Each interaction samples a member of the active group and a fresh input; the model predicts before receiving the label. Stable task IDs identify the requested task, while group membership, switches and teacher parameters remain hidden. The model learns and adapts through data and feedback.

Linear tasks: analytically specified capacity conditions. This family makes it possible to set a clear condition: each group can be represented, while one fixed model cannot exactly represent all tasks. It has 64 tasks with 32-dimensional inputs, local group ranks at most eight and overall rule-matrix rank 32. The largest deployed bilinear predictor has rank eight, so it can represent any individual group but cannot exactly represent the whole bank as one fixed model. Equation (12) gives the best fixed-model uniform-task MSE within that model family.

Nonlinear tasks: studying the same problem with a conventional neural network. This family uses shared nonlinear teacher features and an ordinary task-conditioned MLP. Its profile was selected through local learning, same-group transfer, revisitation and joint fitting, then checked on three held-out banks. Table 1 reports 71/72 local diagnostic passes, mean revisit rewards above .93, and substantially lower joint-fitting MSE at width 128 than width 16. This supplies empirical calibration for the specified model and training budgets. The families provide complementary analytical and nonlinear test settings.

Combining tasks with the resource protocol. Both families use the same protocol for capacity reclamation, returns and timed maintenance. The external platform determines the available deployment parameter budget; the controller determines model size, training effort and deployment timing. Appendix A specifies timing and work units. A maximum-size 16-update job requires at least 13 linear or 16 nonlinear interactions, compared with the default notice of 64 interactions. These settings put resource deadlines and training duration on the same interaction clock. They allow algorithms to be compared on prediction performance over time, resource compliance and maintenance work under the same task and resource streams.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.