- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies the problem of sequentially assigning online prediction tasks to an agent (human or AI) subject to specific capacity constraints. An ideal routing policy is defined which demonstrates the best strategy assigns tasks based on how much better one agent is than another on a given task, adjusted by a penalty that prevents any single agent from being overworked. An online algorithm is proposed for this task. They demonstrate good results against selected baselines on different tasks.
Critical strengths
The "critical" strength of the paper is the rigorous mathematics. The authors are exhaustive in their proofs, backing up pretty much every claim that they make, and working around edge-cases (like when one agent dominates vs. when they disagree). Whilst it's quite a theoretical paper, they try make it as "realistic" as possible.
Critical weaknesses
My primary "critical" weaknesses is that the paper misses quite a few related works, of which may narrow the novelty claims.
Firstly, the paper writes: "Our work differs from prior work in two primary aspects. First, we jointly learn assignment policies and agent expertise online (useful, for example, under distribution shift), unlike prior work on learning to defer or LLM routing, which learns agent expertise offline (Mozannar & Sontag, 2020; Claure et al., 2020; Chen et al., 2023)." I believe the existing "population-adaptive" L2D works of L2D-Pop [1] and IFD [2] (whilst not marketed as online-learning L2D) can do this. Whilst these papers meta-train their architectures (classifier and rejector) offline, at inference they utilize a set of data (similar to an online stream?) to learn and update agent expertise online under bandit feedback. I think it's especially relevant to IFD's Bayesian expert competence updating, as they update the expert's competence posterior as new data comes in. It's quite similar to the paper's Bayesian logistic/tree reward model in Sec. 3. Additionally, there seems to be many other works not cited/discussed within the paper which seem quite relevant. I have not read these papers in detail, though. See [3] and [4].
Next, the authors claim their work differs from prior work because they study "capacity constraints on the agents, unlike fairness-constrained bandits, which enforce a shared minimum-rate constraint across arms (Chen et al., 2020), and cost-constrained LLM routing, which enforces a global budget rather than agent-specific capacity limits." There are a few issues with this. On page 1, when introducing the author to related works and noting how theirs is different, the authors do not discuss capacity constraints L2D work. But, in Sec. 5, they cite five L2D works which explicitly enforce agent-specific workload/capacity constraints. This is important because I believe the theoretical characterization in Sec 2.3, Thm 2.1 is already established in L2D -- see [5].
Unrelated, the authors do not release the code. In my opinion, in the modern era where ICLR receives 66K submissions and there is a reproducibility crisis, I think this severely hampers the ability to accept a paper if the reviewers cannot audit the results.
[1] Tailor, D., et al. (2024). Learning to Defer to a Population: A Meta-Learning Approach. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (AISTATS 2024), PMLR 238:3475–3483.
[2] Strong, J., et al. (2026). Identity-Free Deferral For Unseen Experts. Proceedings of The Fourteenth International Conference on Learning Representations (ICLR 2026).
[3] To Defer or Not to Defer? Online Learning with LLMs and Congested Experts. https://arxiv.org/html/2610.15128v1
[4] Online Decision Deferral under Budget Constraints. https://arxiv.org/abs/2409.20489
[5] Post-hoc Estimators for Learning to Defer to an Expert, NeurIPS 2022
Questions for authors
The only question which would materially effect my assessment are those pertaining the related works (see critical weaknesses). Can the authors please address these?
4 comments
Summary
With these works now cited, our contribution is stated more precisely. Each prior line covers part of the problem: population-adaptive L2D [1,2] estimates expertise from labeled data supplied before routing; workload-constrained L2D (Alves et al., 2024; Nguyen et al., 2025; Zhang et al., 2026, in our paper) enforces per-agent capacities but learns expertise offline; and online deferral methods [3, 4] learn from their own decisions, but with a single expert or expert pool under a congestion or cost budget. Combining these changes the problem: under bandit feedback, agent-specific capacity constraints themselves determine which agents' outcomes are observed. We address this directly: our capacity guarantee holds for any reward model (App. G), and our new ablation shows that the constraints supply the exploration a greedy learner otherwise lacks (App. H.9).
Code
Thank you; we agree that reviewers should be able to audit the results. Our code is now available at https://anonymous.4open.science/r/learning_to_assign/ We will release it publicly with the final version.
[6] Peyré, G., & Cuturi, M. (2019). Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning, 11(5–6), 355–607.
Thanks for the rebuttal.
My remaining concern is the limited novelty of the revised contribution. Theorem 2.1 is now acknowledged as known, Appendix H.12 establishes the equivalence between the queue update and Euclidean dual mirror descent under rescaling, and Propositions 2.2 & 2.3 appear to follow fairly directly from the established threshold characterization.
The additional capacity analysis and empirical comparisons do indeed strengthen the paper, but I remain unconvinced that they establish a sufficiently substantial advance beyond existing constrained contextual-bandit and primal-dual approaches. I therefore maintain my weak rejection recommendation, with my remaining reservation focused on novelty and significance.
We thank Reviewer UYem for the reply and for acknowledging that the additional capacity analysis and empirical comparisons strengthen the paper.
Queue update and DMD. We respectfully disagree that App. H.12 establishes an equivalence between our method and DMD. It shows only that the two share a price update, which we do not claim as a contribution. As detailed in our response to Reviewer MRow (W1 & Q1), DMD solves a different problem: it has upper budgets and a void action, whereas our capacities are exact and every task must be assigned. To run DMD in our setting, we had to modify it in two ways: learn rewards from bandit feedback, and, since there is no void action, send all remaining tasks to the other agent once an agent’s budget is spent. Our soft queue keeps correcting throughout the stream, so each agent’s share stays close to its target at every point, for any reward model (Theorem G.1(a)). DMD’s budget stop instead meets the capacities only as totals at the end of the horizon, and its window deviation is up to 10 times ours on Bank, with no gain in accuracy. We have revised App. H.12 to state this difference explicitly, replacing the earlier wording that could be read as equivalence.
Propositions 2.2–2.3. We agree that these follow directly from the threshold characterization, and we present them as new direct corollaries as opposed to new technical results. Their role is to express the gain from contextual assignment in measurable quantities: the size of the disagreement regions and the average reward gap within them. Our experiments match these predictions. On Camelyon17, the gain is largest at , the capacity closest to full exploitation of disagreement (Prop. 2.2), and shrinks as the capacity moves away from it. On the tabular datasets, where one agent dominates, the gain comes from choosing which tasks the weaker agent receives (Prop. 2.3).
Significance. Our contribution is the problem and what we learn about it, not a new update rule. We study learning multiple agents’ context-dependent expertise from bandit feedback while enforcing exact per-agent capacities. In this setting, the capacities determine which agents’ outcomes are observed, and we show that they supply the exploration a greedy learner otherwise lacks (App. H.9). In constrained contextual bandits such as CBwK (Agrawal & Devanur, 2016) and DMD (Balseiro et al., 2020), budgets only cap consumption, a null action is available, and exploration has to come from the learner. We validate this across eight datasets with ML, LLM, and human agents.
The initial review noted that related work was the main issue affecting the assessment, and we have addressed each of the five suggested papers. To our knowledge, no existing constrained-bandit or primal–dual method addresses this realistic setting, where every task must be assigned under exact per-agent capacities and agents learn expertise online. We would be glad to compare against it if the reviewer could point us to one.
We thank Reviewer UYem for the careful reading and for these references, and we are glad the reviewer found our theoretical analysis rigorous, including the treatment of edge cases such as agent dominance and disagreement. We have read each suggested paper and revised the introduction and related work accordingly. They help us position our work more precisely, and we address each below.
Population-adaptive L2D [1, 2].
Thank you for pointing us to L2D-Pop [1] and IFD [2], which we now cite in the introduction and related work. Both meta-train offline so that, at test time, the rejector can adapt to an unseen expert from a small context set of that expert’s past predictions with true labels. IFD’s Beta–Binomial competence posterior is indeed close in spirit to our Bayesian reward models. The key difference is the source of the data. In both papers, the context set is a labeled sample collected before deployment and independent of the system’s own decisions, for example “a few ‘warm-up’ diagnoses (on historical data)” in L2D-Pop (see their Section 4). IFD studies larger context sets by re-evaluating a fixed, pre-trained model with larger random draws from a labeled pool (see their App. G.3–G.4). In this sense, their expertise estimation happens at test time but is offline with respect to the decision stream, and it is not bandit feedback, since the expert’s prediction and the true label are available for every context example. In our setting, tasks arrive sequentially and we observe only the outcome of the selected agent, so what we learn depends on our own assignments. In addition, IFD models competence per class rather than as a function of the input (see their Section 4.1), whereas we model context-dependent expertise μa(x). Neither method enforces agent-specific capacities. IFD’s deferral budget is a quantile threshold over the evaluation set (see their Section 5), while we enforce each agent’s capacity online.
Online deferral [3, 4].
Reid et al. (2024) [4] was already cited. We have revised our description to state that it studies online deferral between a model and a single human as a contextual bandit with a budget on deferral costs, under both bandit and full-information feedback. Our setting differs in that tasks are assigned among multiple agents, each with its own capacity target, rather than limiting a budget on a single costly expert with the model as a free fallback.
Saketos et al. [3] appeared on arXiv on 10 October 2026, after the ICLR submission deadline, and we now cite it as concurrent work. It is close in spirit, with online learning, selective feedback, and limited expert capacity. However, it learns the reliability of a single LLM from a discrete confidence signal and defers to a shared queue of experts who are assumed to be always correct, with capacity entering as a congestion cost. In contrast, we learn the context-dependent expertise of every agent and enforce agent-specific capacity targets.
Capacity-constrained L2D and Theorem 2.1 [5].
Thank you for pointing out that our introduction did not discuss learning-to-defer methods with workload constraints, although our Related Work (Section 5) cites them. We have revised the introduction to discuss workload-constrained learning to defer directly. It now states that workload constraints have been studied in learning to defer (Alves et al., 2024; Nguyen et al., 2025; Zhang et al., 2026, in our paper), where agent expertise is learned offline. Our setting differs in combining these constraints with online learning of agent expertise from bandit feedback, where capacities must be enforced as tasks arrive.
We also agree on Theorem 2.1. Its general form is a known result from semi-discrete optimal transport [6], and the two-agent threshold rule (Proposition 2.1) matches the constrained deferral rules in learning to defer [5]. We now cite both after the theorem and present it as a foundation of our analysis rather than as contributions. Our contribution in Section 2 is to characterize when and by how much context-dependent assignment improves on non-contextual assignment under capacity constraints, in terms of the size of the disagreement regions and the contextual reward gap (Propositions 2.2–2.3), and when it outperforms both individual agents (App. D.3).
[6] Peyré, G., & Cuturi, M. (2019). Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning, 11(5–6), 355–607.