acceptodds
Under review as a conference paper at ICLR 2027

A Two-Resource View of Adapter-Library Arbitration

Abstract

Growing adapter libraries shift the central challenge of parameter-efficient continual learning from forgetting to arbitration: given an input, which independently trained adapter should answer? Across frozen-language adapter libraries with identical adapters under every arbitration rule, we identify an empirical two-resource pattern. Fitted routers depend on sufficient cross-task meta-training data relative to routing difficulty: holding adapter training fixed, reducing router labels from 1,000 to 100 lowers accuracy from 86.8% to 53.7%; jointly starving adapter training lowers it to 35.4%, while the unfitted competition reaches 83.9% in that cell. Unfitted abstention-based competitions avoid this dependence but pay a different cost: as the library grows, max-over-K arbitration amplifies the upper tail of imperfectly comparable wrong-expert scores. This leads to Reference-Anchored Slack (RAS) training for frozen-language libraries: a small shared, domain-adjacent reference trained as slack inside each adapter's existing head, producing more comparable native confidence scores without K-way fitting or refitting. The reference alone adds +6.3 ± 3.9 points to the confidence competition at K=75 on CLINC150, and its value tracks reference proximity—exchanging an adjacent reference for a distant one costs up to 11.5 points. Within the reference-trained substrate the competition's K-cost stays shallow from 125M to 32B, whereas matched no-reference controls incur larger K-costs at every tested scale, so scale alone does not ensure cross-expert comparability. For generation, the same training principle supplies the shared out-of-scope coordinate that first-token margin arbitration requires. Our results give a regime map for choosing adapter-library arbitration.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
adapter libraries, continual learning, task-free arbitration, LoRA, abstention, cross-expert comparability, reference-anchored slack

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QrhsOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper focus on continual learning with adapter libraries where a frozen model is accompanied with several adapters, and during inference one adapter has to be chosen for an input question. In one setting, they keep the number of adapters fixed, and the adapters are trained with full replay buffer. When the training data for the router decreases, the router accuracy drops. Next, when the replay buffer for adapter training decreases, the router accuracy drops even further. In these cases, a model that selects an adapter with maximum confidence has stable accuracy. In the second setting, they keep the training data fixed, and increase the number of adapters. The accuracy of model that predicts by the most confident adapter, drops. And the accuracy of the router based approach remains stable. They suggest a fix called as Reference Anchored Slack (RAS). In this case, they add 250 reference examples to each adapter training which teaches the model to abstain on the same examples for the different adapters. They show that adding such examples, improve the accuracy of the most confident adapter model.

Critical strengths
  • Paper shows that adding RAS training improve the accuracy of the most confident adapter model compared to an approach that doesn't include these examples. The advantage holds when the number of adapters is increased from 15 to 75
Critical weaknesses
  • The paper is very hard to read. In the introduction itself, for example lines 50-76 "A fitted router", "cross-task meta-training data", "router-label starvation", "surface-separable intents", "max-over-K arbitration amplifies the upper tail of imperfectly comparable wrong-expert score", "substrate", "thinner wrong-expert score tails" Many new terms are coined but not explained.

  • The paper assumes that the reference data is fixed. If a new task arrives, and its data is similar to the reference data, then the RAS training will abstain on such examples. And if the new task data is very dissimilar to the reference data, then will RAS training help?

  • Table 3 results have a confounding factor of the addition of 250 examples for training the RAS model. The plain model doesn't contain these training examples.

  • There are insufficient details of the datasets and no examples

Questions for authors
Reviewer_mYQpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of expert selection in adapter libraries for continual learning and analyzes the limitations of different arbitration methods from the perspective of two resource constraints. Fitted routers require sufficient cross-task training data to achieve good performance, while confidence-based competition without additional fitting suffers from poor comparability of confidence scores across independently trained adapters.

Based on these observations, the paper proposes Reference-Anchored Slack (RAS), which introduces a shared, domain-related reference set as slack supervision during the training of each adapter. This aims to improve the comparability of confidence scores across experts without training a global router or retraining existing adapters.

The paper uses a series of controlled experiments and ablation studies to support its analysis and evaluate the proposed method. It also explores the application of RAS to generation tasks. The results show that RAS can improve arbitration performance across adapter libraries with different frozen language models. At the same time, the experiments reveal that its effectiveness depends on how closely the reference data matches the task domain, and that its benefits do not clearly extend to vision tasks.

Critical strengths

1. Clear problem analysis and sufficient controlled experiments.

This paper analyzes the main challenges of adapter-library arbitration from two perspectives: limited cross-task training data and poor comparability of confidence scores across experts. By conducting a series of controlled experiments on the same set of adapters, the authors systematically show the failure modes of different arbitration methods. These analyses help explain the motivation of the paper and provide useful experimental insights for future research.

2. A simple method with relatively extensive experiments.

The proposed RAS method is simple and does not require training an additional global router or retraining existing adapters when new tasks are added, which gives it practical value. The authors conduct experiments and ablation studies with different data budgets, numbers of experts, model sizes, and datasets, showing the effectiveness of the method in various settings. They also study how the similarity between reference data and task domains affects performance, and report negative results on vision tasks. These experiments help clarify the effective scope and limitations of the proposed method.

Critical weaknesses

1. The additional benefits of RAS have not been fully verified.

When comparing RAS with the plain baseline, the paper does not fully control for the extra training data used by RAS. Although Appendix E.1 provides a comparison with the same number of samples, the results show that simply increasing the replay data can already bring a large part of the performance improvement. More importantly, the authors do not provide a similar comparison in the K=75 setting, where RAS shows larger improvements. Therefore, it remains unclear how much additional benefit RAS provides compared with simply adding more negative training samples.

2. The dependence on reference data limits the robustness and practical use of RAS.

The reference swapping experiments show that RAS cannot consistently outperform the baseline without reference data when the reference comes from a distant domain, while references from similar domains bring clear improvements. However, these similar references mainly come from held-out classes or relations in the same datasets, which can also be seen as additional negative samples related to the target tasks. Therefore, the improvements may come from more effective negative training samples, rather than the sharing of reference data across experts. The results do not fully prove that improved cross-expert score comparability is the main reason for the performance gains.

In addition, in an adapter library containing tasks from different domains, a single fixed reference may not be suitable for all experts. When new tasks from different domains are added, the original reference may become less effective. However, updating the reference may require retraining existing adapters to keep the same shared reference across all experts, which would weaken the add-without-refit advantage claimed by the paper. The paper does not evaluate this situation or provide a clear strategy for selecting and updating reference data. This limits the robustness and practical use of the proposed method.

Questions for authors

1. Difference between RAS and simply adding more training data:

The results in Appendix E.1 show that simply increasing the replay data for the plain baseline can already bring a large part of the performance improvement, while the additional gain of RAS is relatively small and uncertain. Therefore, what are the unique advantages of RAS compared with simply adding more negative samples to the replay buffer? Why do the authors not provide a similar comparison with matched resources in the K=75 setting, where RAS shows larger improvements?

2. Actual role of the shared reference:

In the reference swapping experiments, references from similar domains can also be considered as additional negative samples related to the target tasks. Could the authors further clarify whether the performance improvement mainly comes from more effective negative supervision or from better cross-expert score comparability introduced by the shared reference?

3. Domain sensitivity of the reference and library growth:

The reference swapping experiments show that the effectiveness of RAS strongly depends on how close the reference data is to the task domain. However, in a growing adapter library, there is no guarantee that future tasks will always be close to the fixed reference selected at the beginning. Therefore, the robustness of the reference to domain changes is important. When new tasks come from different domains, how can the authors ensure that the original reference remains effective? Is it necessary to select or update the reference? If updating the reference requires retraining existing adapters to maintain consistency across experts, would this weaken the add-without-refit advantage emphasized in the paper?

I would be happy to raise the score if the author could provide more evidence to answer my question.

Reviewer_gge5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies arbitration in adapter libraries, where a frozen backbone carries one independently trained LoRA adapter per task and the system must decide, without task identity, which adapter should answer a given input. Each adapter has a head over its own classes plus a slack (abstention) class trained on foreign replay samples. Comparing several arbitration rules on identical adapters, the authors identify a two-resource pattern: fitted routers and gates depend on the amount of cross-task meta-training data relative to routing difficulty and collapse under replay-budget starvation, whereas the unfitted max-confidence competition is robust to starvation but degrades as the library grows, because the maximum over K imperfectly comparable scores amplifies the upper tail of wrong-expert scores. Since the oracle stays near 99%, the loss is attributed to arbitration rather than to the experts. Building on this, the paper proposes Reference-Anchored Slack (RAS) training, in which a small, fixed, domain-adjacent reference set is trained as slack inside every adapter’s head, giving independently trained experts a shared score coordinate without any cross-adapter fitting or refitting. On CLINC150, RAS improves the confidence competition by 6.3 points at K=75 and 2.3 points in the budget-starved cell, and within-benchmark swaps show that the gain depends on reference proximity. A backbone sweep from RoBERTa-base to Qwen2.5-32B shows that the library-size cost stays small with RAS but not with matched no-reference controls. The same principle is applied to generation through a first-token margin, while vision experiments on CIFAR-100 do not show transfer, so the claims are restricted to frozen language backbones. The paper concludes with a regime map for choosing between fitted and unfitted arbitration.

Critical strengths

The first strength is the clear framing of the problem together with a carefully controlled experimental design. Treating arbitration, rather than forgetting, as the central difficulty of adapter libraries is timely, and the oracle results make convincingly clear that the experts themselves are capable. All arbitration rules are compared on identical adapters, and the fixed-adapter control that starves only the router labels, the pure candidate-set control, the resource-symmetric control and the within-benchmark reference swaps isolate the relevant factors well. This makes the two-resource pattern a useful and actionable way to think about when fitted or unfitted arbitration should be used. The second strength is that the proposed method is simple, practical and evaluated broadly. RAS requires no cross-adapter fitting and no refitting as the library grows, it consistently improves the confidence competition on CLINC150, and the backbone sweep from 125M to 32B with matched no-reference controls is a substantial piece of evidence that comparability depends on the training protocol rather than on scale alone. I also appreciate that the authors report negative results in vision and restrict their claims accordingly.

Critical weaknesses

My main concern is that the fitted baselines may be too weak, which risks overstating the claim that fitted routing collapses under data scarcity. The router is a one-hidden-layer scikit-learn MLP on the concatenated logits of all adapters, so in the starved cell it must learn a high-dimensional mapping from only about six or seven samples per route. Notably, in the generation experiment the router uses frozen-backbone representations instead, and at B=100 it achieves higher routing accuracy than the RAS competition (88.7% vs. 85.3%). This suggests that part of the collapse in classification may come from the choice of router features rather than from data scarcity alone. Prototype routing (15–37%) and AdapterFusion (11.5% in the starved cell) are also surprisingly low. Since the overall narrative rests on the collapse of fitted arbitration, I would like to see stronger routing baselines, such as linear or kNN routing on backbone embeddings, as well as the task-free methods discussed in the related work (EASE, SEED, HESTIA). A second concern is that the evidence for RAS is somewhat thin and its practical requirements are not fully specified. All results use three seeds, and the headline gain of 6.3 ± 3.9 points has high variance; some readout comparisons are driven by a single seed, and significance tests are mostly missing on the language side. The method depends entirely on a domain-adjacent reference set, but the selection rule is deferred to the released scripts, and there is no sensitivity analysis of λ_R or the reference size |R| on the language benchmarks. CLINC150 happens to provide its own out-of-scope data, so it remains unclear how such a reference would be obtained in practice. In vision, only a distant reference (SVHN) was tested, so the negative result does not show whether RAS would work with an adjacent reference. In addition, the library-size axis is obtained by repartitioning a fixed set of 150 intents (two intents per task at K=75), with genuine growth tested only from K=50 to K=75, and the recipe differs across benchmarks (for example, hard-negative mining and a far CLINC reference in the banking77 main stream). Several further issues are less critical but should be addressed. The generation results are presented favourably, although the fitted router routes better and both reach the same Rouge-L, and the failure without a reference partly follows from the lifecycle design in which the first expert never sees slack samples. The O(K) inference cost, estimated at about one second per input for a 32B backbone at K=75, is only discussed in the appendix. The three propositions mainly serve as intuition: the first relies on an independence assumption the authors acknowledge does not hold, the second is close to trivial, and the third is a textbook bound without quantitative connection to the experiments. The writing is extremely dense, with numbers in almost every sentence and key definitions scattered across the appendices, and there are small inconsistencies such as “six natural repairs” in the contributions versus “five post-hoc failures” in Appendix C, and different standard-deviation conventions across tables. Finally, the related work should discuss the line of work that decomposes class-incremental learning into within-task prediction and OOD-based task identification (e.g., Kim et al., NeurIPS 2022), and clarify how RAS differs from outlier exposure applied per expert.

Questions for authors

My main clarification requests concern the baselines and the reference set. How does a stronger router, for instance a linear or kNN router on frozen-backbone embeddings as used in the generation experiment, perform in the starved classification cell, and does the two-resource pattern still hold? How is the reference set selected in practice, how sensitive are the results to λ_R and |R| on the language benchmarks, and what would a practitioner do when no out-of-scope data like CLINC’s is available? Could the authors also clarify why the banking77 main stream uses the far CLINC reference and hard-negative mining rather than the default protocol? As suggestions that could further strengthen the paper, I would encourage the authors to run a vision experiment with an adjacent reference, for example held-out CIFAR classes, to test whether the vision failure is due to reference distance; to compare against the task-free methods discussed in the related work; and to report more seeds or confidence intervals for the main language results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.