acceptodds
Under review as a conference paper at ICLR 2027

Autonomy and Performance Trade-off in Adaptive LLM Cascades

Abstract

We present an adaptive system where two models, a large teacher and a small student, form a cascade in an online deployment scenario. As the queries stream in, a router decides whether the teacher model is needed for each query. We continually update the router and distill from the stronger, larger model to the smaller student, whenever the larger model is queried. This creates a non-stationary system, with both the router and the student constantly adapting. The process trades-off the overall performance of the system and the autonomy of the small model, which reflects the privacy of the computation. We treat routing as a contextual bandit problem, and experiment with several non-stationary data stream distributions. Experiments demonstrate the effectiveness of our method, and show that competing incentives can be balanced to significantly advance the Pareto efficiency of LLM serving, even without access to the teacher’s logits (i.e., with API models).

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Online Learning, Distillation, LLM Cascade

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ckJfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper considers the problem of training a two-model cascade to handle an online data stream. The authors are specifically interested in the case where the task is changing over time.

This paper proposes the following idea. We have a small student model, and a large teacher model. On a stream of input queries, you first apply the student. The student produces a vector describing its internal state, which is fed to a regression model that is used to estimate the reward of moving to the teacher (including an uncertainty estimate). I this exceeds zero, we go ahead and ask the teacher for the result. If the student and the teacher differ, the reward is then positive. If the student and teacher are the same, the reward is negative. There is zero reward if the decision is not to call the teacher. This reward is then used to update the regression model and the process continues. When the teacher is called, the (query, student answer, teacher answer) triple is stored, and once enough are stored the student is updated by self-distillation using the teacher’s answer as an in-context demonstration.

On the negative side, the main body of the paper includes little in the way of detail on how the student is trained. The central algorithm is not complete. There is no learning for the student in the actual algorithm and the reader must figure how this will work from the material in Section 3.2, Figure 2, and some statements all the way on page 30 of the paper! The paper gives no details on how re-used rollouts are used when computing the Kl objective, though it may be there somewhere. The paper needs to be self-contained and these details would ideally be in the central algorithm. Statements like those on line 1608 of the paper are not “experiment configurations”, they are algorithmic details.

One of the biggest contributions claimed by the paper is that routing happens in “a non-stationary system, with both the router and the student constantly adapting” so the router is trying to learn to manage a student that is always adapting to the changing data. This is also tested experimentally. However, it seems that there is actually nothing algorithmically that’s meant to address this, so it’s not really a contribution. The parameters of the router seem to be accumulated forever, and the only mechanism to adapt to the “non-stationary system” is averaging of these parameters.

In fact, using standard LinUCB in this way with a shifting student may be a particular problem. Once the system has throughly explored a region of the space because it has seen a query many times, it decides that it is done there and the router will never choose to re-explore. This can be a problem specifically in a non-stationary system because the student may perform poorly in that part of the space after previously performing well there. The reward design may make this worse. IF the system chooses NO-T-CALL it cannot change any of the system parameters, so once it has a NO-T-CALL in a region it will not go back there and has no obvious mechanism to discover that the student has stopped doing a good job there.

Another issue is that the contribution of the paper is somewhat minimal. One big component here is the router. But the router is based on the standard LinUCB algorithm with a somewhat novel exploration weighting. Then there is the self-distillation. But this is also standard.

Finally, I will point out that that the paper simply assumes that self-distillation makes sense. The simplest thing to do would be to use supervised fine-tuning (SFT). Or, if one was worried that this would yank the model around too much, use SFT plus a KL-divergence regularizer. One obvious problem with self-distillation is that if the student simply can’t handle the query, then it will never learn (maybe this is what is happening with the Chemistry result?). Our group has played with small models where the model is given data that are far from anything they’ve ever seen before, and the model just can’t handle the input. Even with the teacher’s output, the model is likely to produce nothing of value before it has been trained. In such a situation, it can never learn. The paper could easily ablate this decision, and it should!

Critical strengths

The strongest aspect of the paper is the testbed and the problem definition: the authors have done a lot of work carefully designing a set of testbeds that have various changes in the task composition, introducing new tasks mid-stream. Prior work has not sufficiently considered the problem of non-stationary tasks, and this is a nice contribution.

The experiments are quite good. The two main components of the system are carefully ablated: are you self-distilling the student model, and which router will you use (LinUCB vs. MLP vs. confidence threshold)??

I really like that a key design aspect is the paper is to build a system that does not need access to the internals of the teacher model. Especially if the teacher is a closed-weight commercial frontier model this is absolutely necessary.

Critical weaknesses

Though I've listed several weaknesses above, I think that only one of them may support rejection: the paper sets up an interesting problem (routing in “a non-stationary system, with both the router and the student constantly adapting”) but then it's not clear that there is anything algorithmic to support this. In fact, a thought experiment (as described) would seem to indicate that (as I wrote) if "the system chooses NO-T-CALL it cannot change any of the system parameters, so once it has a NO-T-CALL in a region it will not go back there and has no obvious mechanism to discover that the student has stopped doing a good job there."

Questions for authors

I would like to read what the author have to say about the lack of specific algorithmic innovations that are meant to support use in a non-stationary system.

3 comments
AuthorsOpenReview ↗

Author Response

Addressing the critical weakness:

W.1 "actually nothing algorithmically that's meant to address [the changing student and changing data]"

A contextual bandit router adapts naturally to a changing student and data.

It updates its routing rule continuously and uses the context embedding to distinguish query types, including new ones. As the student improves on a query type, teacher calls on that type more often receive negative rewards, shifting later queries of that type toward no teacher call (Lines 202-204).

Queries from a new task are unfamiliar to the router, so its uncertainty is high and it explores with teacher calls. If the student cannot yet do the task, these calls raise system accuracy and train the student; if it already can, the negative rewards quickly steer the router away from teacher calls on that task. This makes the contextual bandit handle changing data.

An MLP router would also update its routing rule continuously, but empirically we find it performs worse, with teacher call rate fluctuates more sharply across a stream (Line 459-470). We think it could be that the gradient updates to a neural classifier tend to overfit to recent local patterns.

Thought-experiment failure case: the student suddenly degrades on a task it already did well on, after the system has explored enough to be confident that it does well there

You are right that the router receives no feedback on NO-T-CALL rounds. However, we did not observe this degradation in our experiments: the student retained performance on learned tasks even after long absences (S2 in Sec 6.2 and Appendix C.2).

We agree this risk is conceivable with a suboptimal student learning algorithm (catastrophic forgetting may happen). However, the student learning process should be designed to prevent it in the first place, rather than relying on the router to detect and correct it. Many continual-learning techniques mitigate forgetting and should be used for student learning (we use self-distillation and standard replay).

In a hypothetical extreme case where forgetting may occur after the router has considered the task as well-learned, the forgetting may also affect the student’s hidden state (thus the context vector), which increases the router’s uncertainty and prompt renewed exploration. Additionally, this hypothetical case can be addressed by keeping a small fixed rate of forced teacher calls to maintain exploration and recalibrate. With an effective student learning method this is unnecessary, so we leave this scenario to future work.

Addressing other points raised by the reviewer

Comment on presentation: student-training details are spread across two subsections and parts of the appendices …

We will reorganize to add the student update step to Algorithm 1 and move other method details into Section 3, and make the main sections self-contained.

Novelty: the router lacks novelty (standard LinUCB) and self-distillation is standard

We do not claim a new bandit or distillation algorithm. Our contributions are the problem formulation, the overall framework (a student that keeps learning from a black-box teacher, with a router that adapts online), and the evaluation suite with distribution shifts. Showing that we can combine LinUCB and self-distillation in this new setting is a novel application of these techniques, and is part of our contribution.

Self-distillation is assumed. Ablate against SFT or SFT + KL regularization

We will add experiments with the SFT + KL regularization alternative in the camera-ready if accepted. Swapping the student learning method leaves the router and the evaluation suite unchanged, so this comparison does not affect our conclusions, which focus on router comparison and our evaluation suite.

AuthorsOpenReview ↗

Gentle reminder about the posted author response

As the discussion period approaches its end, we wanted to see if you had a chance to consider our response. In particular, we explained how our router adapts to the changing student and data, and addressed the failure case in your thought experiment. We hope our response addresses the concerns you raised, and would be grateful if you would consider updating your assessment.

Thanks to the authors for responding to my comments. I've read them carefully. My review of the paper was already positive and I'll take these comments into account during the discussion.

Reviewer_foCsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents Evolving Cascade, an online LLM cascade that jointly adapts a small local student model and a router to balance system performance and student autonomy. For each query, the student first generates an answer, after which the router decides whether to invoke a larger teacher model. Routing is formulated as a contextual bandit problem using a modified LinUCB algorithm, with feedback derived from student–teacher agreement when the teacher is called. Teacher responses also support continual student learning through self-distillation, without requiring access to the teacher’s parameters or logits. The reported results show improved performance–autonomy trade-offs, measured by system accuracy and teacher call rate, across the evaluated streams.

Critical strengths
  1. The paper addresses a relevant online learning problem under limited feedback. Evolving Cascade jointly adapts the router and the student using student–teacher agreement feedback observed only when the teacher is called. It does not require ground-truth labels for online learning or access to the teacher’s parameters or logits. Each teacher call both answers the current query and provides a demonstration for subsequent student learning. This integration is useful for deployments in which a local student learns from a black-box teacher while the router adjusts to the student’s changing capabilities.

  2. The evaluation provides empirical support across multiple stream settings and component comparisons. The evaluation covers five data streams across Balanced, Abrupt Task Swap, and Incremental Task Introduction. Supplementary experiments include a thinking model as the student, two additional Abrupt Task Swap streams using Banking77, and held-out evaluations of student–teacher agreement to examine learning and forgetting. Comparisons with w/o distill assess the benefits of student distillation while retaining the same adaptive router. The reported results show that the proposed system can achieve comparable or higher accuracy with fewer teacher calls, supporting the benefits of jointly adapting the student and the router.

Critical weaknesses
  1. The implications of historical routing statistics while the student evolves are not fully examined. Algorithm 1 accumulates contexts and rewards from teacher calls without an explicit forgetting mechanism, while continual distillation changes the student’s capabilities and potentially its hidden-state representations used for routing. It remains unclear how these changes affect the relevance of historical observations. If the student becomes less reliable on a familiar query type for which teacher calls have become infrequent, the router may receive limited feedback to detect the degradation. The Abrupt Task Swap experiments and the held-out analysis in Appendix C.2 provide useful evidence of adaptation and retention, but do not clearly establish recovery in this particular situation.

  2. Variability across independent runs is not reported. The paper reports consistent trends across several data streams, but does not specify the number of independent runs or provide standard deviations or confidence intervals. Stream order and reservoir sampling, as well as sampling-based decoding for Qwen3, may affect the coupled evolution of the student and router. Runs with different trade-off parameters do not establish variability at a fixed setting. This is particularly relevant to the smaller differences between the proposed router and the w/ Vanilla LinUCB variant in Appendix A.1 and Figure 9.

Questions for authors
  1. How to retain the full history of contexts and how does rewards affect routing as the student’s capabilities and hidden-state representations evolve? In particular, did the reported experiments consider cases in which a previously learned query type becomes harder for the student after teacher calls have become infrequent? If so, what evidence shows that the router detects the degradation and recovers? This would help distinguish the retention demonstrated in Appendix C.2 from recovery after degradation.

  2. How many independent runs, random seeds, and data-stream orderings were used for the reported results? The authors should provide the mean and variability of system accuracy and teacher call rate at representative trade-off settings, particularly for the comparison with the w/ Vanilla LinUCB variant in Figure 9?

2 comments
AuthorsOpenReview ↗

Author Response

W1 & Q1: Historical routing statistics as the student evolves

"The implications of historical routing statistics while the student evolves are not fully examined"; "without an explicit forgetting mechanism"; "how does rewards affect routing as the student's capabilities and hidden-state representations evolve?"

A contextual bandit router adapts naturally to a changing student and data: historical statistics for a query type lose influence as similar queries keep arriving. For example, as the student improves on Task A, new teacher calls on Task A increasingly find agreement and receive negative rewards. These newer observations come to outweigh the old ones, so the router relies more on the student. Figure 5 shows this: the teacher call rate decreases as student agreement rises.

The evolving hidden-state representations can make the context of the same task type less familiar to the router, which is actually desirable when the small model’s ability improves. It makes the routing rely less on the stale reward statistics that correspond to the older representations for the task.

Keeping historical statistics is intentional. When a query type reappears after a long absence, even after many queries of other types, its saved statistics remain influential, so the router decides correctly without exploring again. In the S2 Abrupt Task Swap streams, when the first task returns in Stage 3, the teacher call rate immediately drops to roughly its level at the end of Stage 1 (Section 6.2, Figures 13-14). A forgetting mechanism would give up this benefit and pay the exploration cost again whenever a familiar task returns after a long absence.

"did the reported experiments consider cases in which a previously learned query type becomes harder for the student after teacher calls have become infrequent?"

How infrequent a query type becomes does not matter as long as the student's performance on it holds (the retention study was with S2 in Sec 6.2 and Appendix C.2). In our experiments, we did not observe the student degrading on a task it had already learned well.

We agree this risk is conceivable with a suboptimal student learning algorithm (catastrophic forgetting may happen). However, the student learning process should be designed to prevent it in the first place, rather than relying on the router to detect and correct it. Many continual-learning techniques mitigate forgetting and should be used for student learning (we use self-distillation and standard replay).

In a hypothetical extreme case where forgetting may occur after the router has considered the task as well-learned, the forgetting may also affect the student’s hidden state (thus the context vector), which increases the router’s uncertainty and prompt renewed exploration. Additionally, this hypothetical case can be addressed by keeping a small fixed rate of forced teacher calls to maintain exploration and recalibrate. With an effective student learning method this is unnecessary, so we leave this scenario to future work.

W2 & Q2: More seeds / independent trials

We addressed this weakness with the "Overall Response" above (linked here). The results from 4 independent trials are consistent with the results in the paper, so our conclusions are not affected.

AuthorsOpenReview ↗

Gentle reminder about the posted author response

As the discussion period approaches its end, we wanted to see if you had a chance to consider our response. In particular, we have included results across 4 random seeds (see our Overall Response), which further support the conclusions in our paper. We also explained how the router adapts as the student evolves and the implications of historical routing statistics. We hope our response addresses the concerns you raised, and would be grateful if you would consider updating your assessment.

Reviewer_W7uLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Online LLM cascades where a router decides when to call an expensive teacher and each call is both a cost and a distillation example. Contributions: a LinUCB-variant router (zeroed NO-T-CALL bonus, calls when the call-reward UCB is positive), online self-distillation of the student, and a five-stream shift suite. Validated at matched agreement against MLP/Conf. routers, reporting fewer teacher calls and 5.7-14.6 pp distillation savings across 15 cells.

Critical strengths
  1. The problem is genuine and cleanly framed. Within an online cascade a teacher call doubles as a cost and a training example, and the router receives feedback only when it pays for a call (Sec. 3, Alg. 1). Evaluating methods at matched agreement (Table 1, Tables 2-4) is the correct protocol, and across every matched cell MLP and Conf. sit below Ours in agreement, so the matching is not tilted toward the proposed method. The stream suite (Fig. 3) is clean and reusable.

  2. The distillation finding holds up. Holding the same adaptive router in both arms, distilling the student cuts the teacher call rate by 5.7 to 14.6 pp at matched agreement across all 15 cells of Tables 2-4 (e.g. 41.8 vs 52.3 on S1 at 90%). The same pattern appears for the Qwen3-1.7B student (Fig. 11) and on the Banking77 streams (Fig. 23). This contrast is therefore independent of how MLP or Conf. were tuned, and it is the claim I trust most.

Critical weaknesses
  1. The assertion that the router beats existing cascades (end of Sec. 1, Sec. 6.1 l. 373-375, Sec. 6.3, "significantly" in the abstract) rests on single reported runs. Tables 1-4 list one value per cell with no error bars, and App. D.3 notes that when no initial run fell near a target agreement, further trade-off values were tried until one did, so each cell is a single run drawn from a sweep. Measured against the strongest other router, 9 of the 15 cells in Tables 2-4 differ by under 5 pp (S1 80%: 23.2 vs 25.4; S3 easy-first 95%: 60.5 vs 63.0). Sec. 6.3 cites 5.3-10.8 pp fewer calls than MLP on the shift streams, yet against Conf. the S3 easy-first gaps are 2.5-4.8 pp. Ours does win all 15 cells and the S2 Easy-Hard-Easy gaps reach 7.3-10.8 pp, but these cells share streams and selection. I would want mean and std over 3+ seeds for the S1 and S3 matched cells, with softer wording wherever the gaps fail to hold across seeds.

  2. Fig. 6 shows a lower false positive rate for the router than for MLP, but the cause is unclear. With NO-T-CALL pinned at zero reward and zero bonus, Alg. 1 calls the teacher exactly when the UCB of the call reward is positive (Line 10). That is close to selective sampling (label-efficient prediction with a query cost), which goes uncited. Beyond the reward design, the sole change to LinUCB is zeroing the NO-T-CALL bonus, which App. A.1 (Fig. 9) calls only "slightly better", while Sec. 7 still bills the method a novel combination. The baselines may also be handicapped: Conf. is calibrated on a 500-instance forced-call cold start with no later recalibration described as the student drifts (Sec. 5, App. D.3), and both MLP hyperparameters sit at their grid edges (lr 1e-5, decay 0.9). A greedy (α=0\alpha=0) router given Conf.'s 500-call cold start, a recalibrating Conf., an exploring MLP, and a "teacher for the first N queries, then stop" baseline would separate these explanations, and all of them can reuse the cached teacher outputs.

Questions for authors
  1. (Could change my score) Do the sub-5 pp router gaps (S1 at 80-90%, S2 Hard-Easy-Hard at 80%, both S3 streams at all three levels) survive as mean and std over 3 seeds under varying stream order and LoRA init?

  2. How many trade-off values per method and stream underlie Fig. 4 and Table 1, how many were added to achieve matching (App. D.3), and on which data was the MLP grid searched?

  3. Sec. 5 states that the cached extraction-based comparison "does not impact the outcome", yet in deployment a teacher-as-judge could label agreement differently and shift the reward. Could you report judge-vs-extraction agreement on the cached pairs of one stream, per task?

  4. Every learning run uses replay (App. D.3). Is there a no-replay run that isolates replay's contribution to the retention reported in Sec. 6.2?

3 comments
AuthorsOpenReview ↗

Author Response

W1 & Q1:

Request for more runs with different seeds (mean and std over 3+ seeds for the S1 and S3 matched cells)

See our Overall Response. Our improvement margins across the seeds closely align with the single-seed results in the paper, so our conclusions are not affected.

W2: Router design and baselines

"Fig. 6 shows a lower false positive rate for the router than for MLP, but the cause is unclear."

The direct cause is that our router predicts better than MLP which instances need a teacher call (Lines 445-449). Because Fig. 6 compares methods at the same system-level performance, the comparable MLP run must call the teacher more aggressively to make up for its weaker predictive power. Like lowering a classifier’s threshold to raise recall, this raises its false positive rate. Lines 460-470 give examples of MLP’s failure modes. We will make this cause clearer in the next revision.

“Selective Sampling”

Thank you for pointing us to this line of work. We will discuss these works ([1][2][3]) in the camera-ready version if accepted, with the extra page allowed. Selective Sampling studies a related but different problem: a query only reveals the label for learning and does not change the system's prediction. In our setting, a teacher call also serves the teacher's output to the user, so our reward reflects both the call’s benefit and its cost. Also, selective sampling queries a label when the predictor is uncertain. Our router calls the teacher when it is uncertain (exploration) or when it is confident the two models will disagree.

[1] Cohn et al., Improving generalization with active learning, 1994

[2] Cesa-Bianchi et al., Robust bounds for classification via selective sampling, 2009

[3] Dekel et al., Selective sampling and active learning from single and multiple teachers, 2012

"Beyond the reward design, the sole change to LinUCB is zeroing the NO-T-CALL bonus, …, while Sec. 7 still bills the method a novel combination"

By “novel combination” we mean the overall framework: an LLM cascade whose student evolves via self-distillation, paired with a continually updated LinUCB router. This is a new application of existing techniques. We do not claim a new contextual bandit algorithm. Our main contributions are the problem formulation, this framework, and new evaluation tools covering distribution shifts. We will make this clearer in the next revision.

“The baselines may also be handicapped.” "Conf." has no later recalibration described as the student drifts… (other suggested baselines)

Thank you for the suggestions. Our paper compares against published LLM-cascade baselines, and the published “Conf.” has no recalibration. A recalibrating Conf. or an exploring MLP would be new methods requiring substantial design choices (e.g., how to recalibrate, which hyperparameters to tune, how to estimate uncertainty for exploration). Developing them properly is a separate contribution beyond our scope, and promising future work. Since we propose a new problem, we put more effort into establishing the framework and evaluation suite, rather than studying more alternatives of our proposed method. We hope our results motivate stronger methods for this setting.

greedy router and “teacher for the first N queries”

We appreciate the suggestions. Both are weak baselines with poor capacity to adapt to distribution shifts in the data stream. They would not change our conclusion that our method outperforms the published baselines (MLP and Conf). We will include both as additional diagnostics in the camera-ready version.

"both MLP hyperparameters sit at their grid edges (lr 1e-5, decay 0.9)"

We expanded our search to include lr=1e-6 and 5e-6; decay = 0.8 and 0.85. None of the new combinations outperformed our chosen hyperparameters, so they did not affect our method’s improvement margin over MLP.

AuthorsOpenReview ↗

Author Response (continued)

Response to questions
Q1: addressed while responding to W.1

Q2:

How many trade-off values per method and stream underlie Fig. 4 and Table 1, how many were added to achieve matching (App. D.3)

We updated the paper pdf with Tables 8-9 (in Appendix E), which report the number of trade-off values underlying the Pareto frontier plots, and the runs added to achieve matching performance. We also added Figure 28, which shows that the additional runs align closely with each method's Pareto frontier. We added these results to the end of the paper's pdf for better visualization and cleaner presentation of tables. The rest of the paper stays intact, so that section and line numbers referred throughout the reviews are not affected.

on which data was the MLP grid searched?

The MLP's hyperparameters were searched on the same TREC validation set that was used for our method.

Q3:

"judge-vs-extraction agreement on the cached pairs of one stream, per task?"

The following table presents the judge-vs-extraction-grader agreement for all tasks. We randomly sampled 500 instances for each task from S1 balanced (the only data stream that contains all tasks). The LLM judge is the same as the teacher model (Gemini 3.1 pro), which receives the student's answer after it produces its own answer.

The LLM judge shows high agreement with the extraction-based automatic grader, as shown by the agreement rate and Cohen’s kappa. Therefore, using cached teacher responses and the grader from the underlying task dataset is a valid and faithful approximation of the real-life scenario with LLM judges. As frontier models continue to improve, we expect the LLM judges to become more and more reliable, which will further reduce the already-small gaps between the two approaches.

TaskLLM judge vs. autograder agreementCohen's kappa
banking77100.0%1
clinic15099.6%0.98
GSM99.2%0.98
minicheck99.0%0.97
chemistry97.2%0.93
tablebench96.4%0.93
all98.6%0.97

Q4: replay ablation
Replay is a standard approach for continual learning and we applied it to all the methods (baselines and ours). Therefore, it does not affect our conclusions and we did not conduct this ablation for our current submission. We will include a replay ablation in the camera-ready if accepted.

AuthorsOpenReview ↗

Gentle reminder about the posted author response

As the discussion period approaches its end, we wanted to see if you had a chance to consider our response. In particular, we have included results across 4 random seeds (see our Overall Response), which further support the conclusions in our paper. We also clarified our router design and baseline choices. We hope our response addresses the concerns you raised, and would be grateful if you would consider updating your assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Response (results from more random seeds)

We repeated our experiments and now report the results across 4 seeds. The mean improvement margins¹ of our method are consistent with the single-seed margins in the paper. The table below reports the mean margin of Ours over each baseline (or ablated variant), along with the standard deviation over 4 seeds. Every seed has its own LoRA initialization, data shuffling, student generation, and replay sampling.

¹ Improvement margin: teacher call (TC) savings of Ours over baseline at matched performance (baseline TC% - Ours TC%).

In 22 of the 30 comparisons against the baselines (MLP and Conf.), the mean margin is within 1 percentage point (pp) of the paper's value (Tables 1-4). Besides these, the largest increase is on S1 Balanced at 80% vs. MLP (2.1 → 3.3 pp), and the largest decrease is on S2 Hard-Easy-Hard at 90% vs. Conf. (18.3 → 15.5 pp).

The smaller margin or similar performance compared with "Ours (w/ vanilla LinUCB)" is expected and matches our observation in the paper (Line 676-680), because it is a variant of our method that only differs from our main method in exploration weighting.

Table: Mean Improvement Margin (with standard deviation)

The boxed entries mark the 9 comparisons where the single-seed margin over the strongest baseline was below 5 pp in the paper (the gaps Reviewer W7uL asked us to verify across seeds). In all 9, the improvement margin survived when averaged across 4 seeds. The smallest mean margin is 3.1 ± 1.3 pp (S3 Easy First, 95%, comparison with Conf.), whose corresponding single-seed margin in the paper was 2.5 pp.

StreamAgree rate (≈%)MLPConf.Ours (w/o distill)Ours (w/ vanilla LinUCB)
S1 Balanced958.1 ± 1.913.0 ± 0.98.0 ± 1.5-0.6 ± 1.4
905.3±0.8\boxed{5.3 \pm 0.8}12.8 ± 0.910.8 ± 0.41.1 ± 1.2
803.3±1.1\boxed{3.3 \pm 1.1}14.4 ± 0.614.2 ± 0.70.3 ± 0.7
S2 Easy-Hard-Easy9511.1 ± 2.419.1 ± 0.85.1 ± 1.94.4 ± 1.0
9011.1 ± 2.117.6 ± 1.19.8 ± 0.74.7 ± 1.7
805.3 ± 1.912.2 ± 1.112.5 ± 1.50.7 ± 1.8
S2 Hard-Easy-Hard957.5 ± 1.014.8 ± 1.54.6 ± 1.40.7 ± 3.1
908.4 ± 2.315.5 ± 2.08.8 ± 1.11.9 ± 1.6
803.5±0.7\boxed{3.5 \pm 0.7}11.0 ± 0.813.7 ± 0.91.9 ± 0.9
S3 Incremental Easy First957.2 ± 2.03.1±1.3\boxed{3.1 \pm 1.3}8.6 ± 1.34.0 ± 1.1
907.6 ± 0.73.4±0.3\boxed{3.4 \pm 0.3}11.6 ± 0.62.5 ± 1.3
805.2 ± 1.05.1±0.6\boxed{5.1 \pm 0.6}11.8 ± 1.41.1 ± 1.5
S3 Incremental Hard First954.6 ± 1.84.0±0.9\boxed{4.0 \pm 0.9}7.1 ± 1.30.5 ± 1.4
903.3±1.3\boxed{3.3 \pm 1.3}5.8 ± 1.311.0 ± 0.9-0.3 ± 0.8
804.4±0.9\boxed{4.4 \pm 0.9}7.1 ± 0.611.2 ± 0.20.1 ± 0.5

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.