acceptodds
Under review as a conference paper at ICLR 2027

Cost-Aware Threshold Adaptive-K Learning to Defer under Budget Constraints

Abstract

Learning to defer routes each input to a prediction source, trading accuracy against consultation cost. In practice several predictors and human experts are available, their costs differ widely, and existing multi-expert and top- methods regulate costs via a hyperparameter, yet they fail to guarantee strict adherence to budget constraints. We introduce CATABC, a budget-constrained adaptive top-k architecture that selects how many and which experts to consult per input, and weighs the consulted ones by masked attention over the image and expert features, under an average-cost constraint enforced by a dual price. We prove that the threshold selection optimally solves the Lagrangian of the average-budget proxy, that the price is the slope of the risk–budget frontier, and that the excess risk decomposes into pool, selection and fusion gaps. On CIFAR-10, CIFAR-100 and SVHN, CATABC stays within the prescribed budget and surpasses the baselines in most settings.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
learning to defer, budget-constrained inference, model routing, selective prediction, ensemble fusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TnKfOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies learning to defer under budget constraints. The authors provide a new framework for this problem, show that the budget is respected, develop a lagrangian of the problem that they solve, and provide experiments that show that their algorithm outperforms the baseline models.

Critical strengths

I could not identify strengths.

Critical weaknesses

I had trouble reading this paper. This made understanding the results somewhat impossible for me. I will provide some examples in hopes that the authors may improve the writing such that the paper becomes more readable. As it stands right now I can not recommend accepting the paper.

The first improvement lies in the first paragraph, the second sentence. Here, a pool is mentioned. But it is completely unclear what kind of pool this is. It can be guessed from the text, but it could be a pool of classification systems, a pool of inference times, a pool of human experts, a pool of pre-trained models, or any combination of these. This type of unclarity is present throughout the text. For example, on the first page alone there are several highly unclear phrases: routing each input, decide which members each input needs, instance-dependent routing, per-entity penalties whose coefficients are tuned before training, aggregation (line 44, what are we aggregating?), fusion weights, induce the committee size per input, price-adjusted utility, dual price, and realised average cost. New concepts are continuously used without introduction. I strongly recommend the authors to carefully read the paper and see where new concepts are used without introduction, and then write the paper in such a way that every new concept that is needed is carefully introduced.

The problem is also highly unclear. The learning objective as stated is to minimize the expected loss of some distribution q as a function of θ\theta and ϕ\phi such that the expected cost of training adheres to some budget. There are several problems. Nowhere in the problem setting the authors specify how qq, θ\theta, and ϕ\phi are related. The loss function ell has an unspecified parameter ϵℓ\epsilon_\ell and I have no idea what role this parameter serves, CreportC_{report} is introduced, but it does not seem to appear in the objective. But most egregious of all: I never learned what we can do to minimize the expected loss. Do we have access to a data set of feature and output pairs? Do we have access to features, but not outcomes and do we have to pay for these outcomes? Do we have access to outcomes, but not features and do we pay for the features? This makes it impossible for me to read the paper, as I really do not understand what the precise goal is.

In section 3.2, several parameters are introduced, but never defined nor are their roles explained. This is true for UiU_i, σ\sigma, sgsg, NN, ϵc\epsilon_c, uθu_\theta, and sis_i. On line 151 si(x)s_i(x) and mi(x)m_i(x) are introduced and on line 155 the authors have decided to use sis_i and mim_i. Are these the same objects? Are they different? It is completely unclear to me.

Questions for authors

.

3 comments
AuthorsOpenReview ↗

Response to Reviewer TnKf (1/3)

Thank you for pointing out where the presentation was unclear. We have revised the introduction and Sections 3.1–3.4 to clarify the learning task, available training data, learnable parameters, and notation. Below, we address each concern and indicate the corresponding definitions and revisions.

W1. Predictor pool and routing/fusion terminology

The predictor pool is a fixed collection of prediction sources with different capabilities and consultation costs. These sources can include pre-trained AI classifiers, human experts, and simulated experts. We also refer to them collectively as experts. Classifiers provide class-probability vectors, and categorical expert responses are represented as one-hot vectors, with one at the returned class and zero elsewhere. Our experiments use trained classifiers and simulated experts.

Routing assigns an input image xx a binary selection vector m(x)m(x), where mi(x)=1m_i(x)=1 indicates that the raw rule selects expert ii, and zero indicates that it does not. The shared cost price is denoted by λ≥0\lambda\ge0, and the raw selected set by Sλ(x)=i:mi(x)=1S_\lambda(x)=\\{i:m_i(x)=1\\}. Selection uses the price-adjusted score si(x)s_i(x), obtained by subtracting the shared price times the training-unit cost from learned utility.

The set actually consulted is denoted by Seff(x)S_{\mathrm{eff}}(x): it retains every nonempty raw selection and activates the highest-scoring expert, indexed by i⋆(x)i^\star(x), when the raw selection is empty; ties are resolved first by lower training-unit cost and then by lower expert index. The effective selection vector meff(x)m^{\mathrm{eff}}(x) records this consultation set. Committee size is the number of experts actually consulted. Fusion combines their forecasts using nonnegative weights that sum to one, with strictly zero forward weights for unconsulted experts. CATABC jointly learns selection and fusion from the final classification loss and allocates consultation resources under an average-cost constraint.

The introduction opens with pre-trained models and human experts, defines the predictor pool at its first occurrence, and introduces utility, price adjustment, selection, fusion, and cost feedback in order. Realised average cost is computed using the effective selection vector, and budget deviations update the shared price. These concepts precede the subsequent discussions that depend on them.

AuthorsOpenReview ↗

Response to Reviewer TnKf (2/3)

W2. Available data, parameter dependence and optimization

We jointly learn the image encoder, predictor selector, and fusion-weight network from labeled data over a fixed predictor pool, using the final classification loss and an average consultation-cost constraint.

Training uses a labeled dataset D=(xj,yj)\mathcal D=\\{(x_j,y_j)\\}, where xjx_j is an image and yjy_j its ground-truth class. Images, labels, and full-pool responses are available during training. CIFAR-100's simulated experts follow a prescribed label-conditioned response mechanism. The selector and fusion-weight network take image features and known source information as inputs.

The parameters θ\theta comprise the image encoder and selector parameters, and ψ\psi parameterizes the fusion network. We have placed these definitions before the learning objective and written their effect on prediction explicitly at the fixed current price:

qθ,ψ(c∣x)=∑iαi(x;θ,ψ)pi(c∣x).q_{\theta,\psi}(c\mid x)=\sum_i\alpha_i(x;\theta,\psi)p_i(c\mid x).

Here, cc is a candidate class, pi(c∣x)p_i(c\mid x) is expert ii's fixed forecast for that class, αi(x;θ,ψ)\alpha_i(x;\theta,\psi) is its strict forward fusion weight, and qθ,ψ(c∣x)q_{\theta,\psi}(c\mid x) is the final fused probability. Updating θ\theta and ψ\psi changes the selection and weights, hence the probability assigned to the observed class. Each minibatch supplies labels for the fused classification loss. The forward pass uses strict fusion, while backpropagation retains a smooth fusion surrogate. The selector receives approximate gradients through straight-through estimation (ST), which uses binary selections in the forward pass and the derivatives of continuous gates in the backward pass. The image encoder, selector, and fusion network all receive classification gradients through this surrogate path. The complete computation appears in Appendix A.3. Source predictors and simulated-response mechanisms remain fixed.

Equation (1) defines the classification loss as

ℓϵ(q,y)=−log⁡(qy+ϵℓ).\ell_\epsilon(q,y)=-\log(q_y+\epsilon_\ell).

Here, qq abbreviates the fused distribution above, qy=qθ,ψ(y∣x)q_y=q_{\theta,\psi}(y\mid x) is the strict forward fused probability of the observed class yy, and ϵℓ>0\epsilon_\ell > 0 is a fixed small offset that keeps the logarithm finite at zero. Section 3.3 separately explains the forward fused probability, the backward surrogate, and the roles of their numerical constants.

Equation (2) distinguishes the cost units used for training from those used for reporting. Expert ii's costs in these units are denoted by c~i\tilde c_i and cic_i, respectively, giving

Ctrain(x)=∑imieff(x)c~i,Creport(x)=∑imieff(x)ci.C_{\mathrm{train}}(x)=\sum_i m_i^{\mathrm{eff}}(x)\tilde c_i, \qquad C_{\mathrm{report}}(x)=\sum_i m_i^{\mathrm{eff}}(x)c_i.

Both charge the same set actually consulted, including the expert activated when the raw selection is empty. Accounting and cost-unit transfer are given in Appendix A.5. This budget accounts for consultation cost at prediction time. Training obtains full-pool responses to compute approximate gradients, and their acquisition cost is recorded separately. Dataset construction and total training computation are training expenses.

The budget in training units is denoted by BtrainB_{\mathrm{train}}. The price λ\lambda is fixed during each minibatch network update and enters both the selection score and the cost penalty in Eq. (7). After each complete pass over the training data, Eq. (8) updates the price using the difference between the sample-weighted mean effective consultation cost and BtrainB_{\mathrm{train}}. Sections 3.1 and 3.4 now state the cost accounting and update schedule explicitly.

AuthorsOpenReview ↗

Response to Reviewer TnKf (3/3)

W3. Symbol definitions and changes in notation

Equation (3) specifies the feature, utility, and score computations. We have expanded the accompanying definitions: z=fθ(x)z=f_\theta(x) is the image feature, fθf_\theta the image encoder, vv a learned projection, eie_i a learned embedding identifying expert ii, uθu_\theta the scalar utility network, and cˉi\bar c_i the normalized training-unit cost. Brackets denote vector concatenation, and ϵc>0\epsilon_c > 0 is the fixed offset in the cost-normalization denominator. Adjusting the utility Ui(x)U_i(x) by price gives the score

si(x)=Ui(x)−λc~i.s_i(x)=U_i(x)-\lambda\tilde c_i.

For a fixed input, sis_i abbreviates si(x)s_i(x). The same convention applies to the soft selection gate πi(x)\pi_i(x) and raw binary selection mi(x)m_i(x). We now state this before Eq. (4). In that equation, σ\sigma is the sigmoid function, Ts>0T_s > 0 the selector temperature, and τ\tau the threshold, with default τ=0\tau=0. The soft gate is πi=σ((si−τ)/Ts)\pi_i=\sigma((s_i-\tau)/T_s), and the binary selection is mi=1[si>τ]m_i=\mathbf1[s_i > \tau], where 1\mathbf1 is the indicator function, equal to one when its condition holds and zero otherwise. The ST variable is miST=mi+πi−sg⁡(πi)m_i^{\mathrm{ST}}=m_i+\pi_i-\operatorname{sg}(\pi_i). The stop-gradient operator sg⁡\operatorname{sg} preserves its forward value and has zero derivative in backpropagation.

In Section 3.3, we further distinguish the source forecast pip_i, selection soft gate πi\pi_i, and strict forward fusion weight αi\alpha_i. The symbol ρi\rho_i specifically denotes a smooth surrogate weight used in classification backpropagation. ϵm>0\epsilon_m > 0 smooths the raw ST selection values in this surrogate, and ϵw>0\epsilon_w > 0 provides its additional weight normalization. Raw selection mim_i and effective consultation selection mieffm_i^{\mathrm{eff}} also use distinct notation, making selection, fusion, and cost accounting easier to compare.

Figure 1 summarizes the modules and their computational relationships. The notation guide at the end of the appendix collects these symbols in computation order for reference alongside Sections 3.1–3.4.

Please let us know if any part of the problem setup or notation is still unclear.

Reviewer_kgm8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework that adaptively selects and fuses multiple predictors under an average consultation-cost budget. The problem is practical, and the combination of cost-aware selection, adaptive committee size, attention-based fusion, and budget feedback is coherent. Experiments on CIFAR-10, CIFAR-100, and SVHN show promising accuracy–cost trade-offs in several settings. However, the theoretical analysis does not sufficiently support some of the paper’s central claims, particularly regarding risk–budget optimality and the budget feasibility of the proposed training procedure. These issues weaken the central budget claims.

Critical strengths
  • Clear background and literature review. The author provides a clear overview of learning to defer, model routing, and top-kk prediction. It situates the proposed method within these lines of work and explains the gap it aims to address.

  • Important and practical problem setting. The paper studies settings in which multiple predictors or experts have different levels of accuracy and consultation cost, and the resources needed may vary across inputs. Selecting and combining multiple predictors for each input can exploit complementary strengths that single-predictor routing may miss.

  • Coherent method design. CATABC combines cost-aware selection, an input-dependent committee size, attention-based fusion, and average-budget feedback in a unified framework. Jointly training the selector and fusion module is a reasonable design choice, as it allows the system to learn both which predictors to consult and how to combine their outputs.

Critical weaknesses
  1. Theorem 1 does not establish risk–budget optimality of the proposed training procedure. Theorem 1 characterizes a threshold policy for a fixed-utility proxy objective, whereas Algorithm 1 trains the selector through the fused NLL objective. The paper does not establish a formal link between optimality for the proxy objective and optimality for predictive risk. Therefore, Theorem 1 should not be interpreted as guaranteeing that the proposed training procedure finds the optimal risk–budget trade-off.

  2. Theorem 2 is a local gradient analysis, not a learning guarantee. Theorem 2 derives the local straight-through gradient received by the selection scores. This analysis helps explain how the fused prediction loss and consultation cost affect the selector’s updates, but it does not guarantee convergence or that the learned score ordering is optimal at the population level.

  3. The dual update does not prove convergence or budget feasibility for the actual training procedure. The algorithm updates λ\lambda using the observed average cost at each epoch, while training a non-convex encoder and fusion network and using straight-through gradients for selection. The paper does not prove that this joint procedure converges to a budget-feasible solution for finite samples and finite training time. Nor does it characterize when the price may oscillate, reach a clipping boundary, or yield a test-time cost above the budget. The reported experiments show budget compliance in the evaluated settings, but this does not establish a general strict budget guarantee.

  4. The empty-selection case appears to use the entire predictor pool at zero cost. In Equation (6), each mask is augmented by a positive ϵm\epsilon_m. When the selected set is empty, this factor cancels across predictors, and the fusion weights reduce to a softmax over all fusion logits. However, the consultation cost remains zero under the mask-based cost definition. Thus, when no predictor is selected, the model can still fuse outputs from the entire pool without incurring consultation cost. Since the theoretical analysis allows the selected set to be empty, this appears to be an implementation-level issue rather than a purely notational one. The paper should clarify how this case is handled and whether it occurs in practice.

  5. The relationship between training cost and reporting cost is unclear. The paper distinguishes training costs c~i\tilde c_i from reporting costs cic_i, and states that they need not be proportional. The dual update constrains training cost, while the reported consultation cost is based on the reporting costs. If these cost vectors are not proportional, satisfying the training-cost budget does not necessarily imply satisfying the reporting-cost budget. The paper should specify their relationship in the experiments and explain how the target reporting budget is enforced when the two cost vectors differ.

Questions for authors
  1. How is the empty-selection case handled in the implementation, how often does it occur, and are unselected predictors’ outputs treated as available at no cost? If obtaining those outputs incurs a cost, how does the implementation prevent this case from violating the budget semantics?

  2. What relationship between c~i\tilde c_i and cic_i ensures that enforcing the training-cost constraint also controls the reported consultation cost? If they are not proportional, how is the reporting-cost budget guaranteed?

1 comment

Thank you for your careful review.

W1. Fixed-score optimality and NLL risk

Theorem 1 optimizes additive net utility at fixed scores and price; highest-score fallback extends it to nonempty sets (A.2). For complete policy ϑ\vartheta (parameters, price, test settings), let R(ϑ)R(\vartheta) be population stabilized negative log-likelihood (NLL) risk and C(ϑ)\mathcal C(\vartheta) expected training-unit consultation cost. Let Θ\Theta be the policy class and RB∗R_B^* its optimal risk under budget B=BtrainB=B_{\mathrm{train}}. For β≥0\beta\ge0, set Jβ(ϑ)=R(ϑ)+β(C(ϑ)−B)J_\beta(\vartheta)=R(\vartheta)+\beta(\mathcal C(\vartheta)-B). If a budget-feasible ϑ^\widehat\vartheta is within εopt≥0\varepsilon_{\mathrm{opt}}\ge0 of its infimum over Θ\Theta, then

\le\varepsilon_{\mathrm{opt}}+\beta(B-\mathcal C(\widehat\vartheta)).$$ This conditional NLL–budget bound evaluates complete policies, including their selection price. **W2. Local training signal** Theorem 2 gives local straight-through (ST) gradients with binary attention and fallback fixed; A.3 details module paths and 234 differentiation checks. Strict forward fusion uses a smooth $\log(m_i^{\mathrm{ST}}+\epsilon_m)$ backward surrogate, with $\epsilon_m>0$ the smoothing offset. ST gates equal raw binary selections forward and use sigmoid derivatives backward. Full-pool comparisons train fusion on singletons, whose strict fusion-score derivatives are zero. All modules use this declared backward rule. **W3. Budget control** Training trajectory: price uses step 0.01 and clipping to [0,50]. For $H$ epochs, let $\widehat C_{\mathrm{train},t}$ be sample-weighted mean effective cost in training units, including fallback, and \$u\_t=

\lambda_t+0.01(\widehat C_{\mathrm{train},t}-B_{\mathrm{train}})-50

\_+\$, where $[z]\_+=\max(z,0)$. Then $$\frac1H\sum_{t=0}^{H-1}\widehat C_{\mathrm{train},t} \le B_{\mathrm{train}}+\frac{\lambda_H-\lambda_0+\sum_tu_t}{0.01H}.$$ This covers changing parameters. In 68 runs, 3,881 updates had 0 upper/1,492 lower-bound hits. Reversals/run ranged over 0–3/0–38/0–52 on CIFAR-10/CIFAR-100/SVHN (A.6). Final model: select highest validation accuracy at mean training-unit cost at most $1.01B_{\mathrm{train}}$; until one qualifies, retain the smallest cost gap. Test weights, price, and settings are fixed. The 1% is a validation tolerance. Population cost: for a locked policy on $n$ independent, identically distributed inputs independent of training, tuning, and selection, with probability at least $1-\delta$, $$\mu_C\le\widehat C+R_c\sqrt{\frac{\log(1/\delta)}{2n}}.$$ Here $\widehat C$ is mean reporting cost, $\mu_C$ its population mean, $R_c$ the cost-range width, and $\delta\in(0,1)$ the failure probability. An upper bound within budget gives population budget control (A.4). **W4/Q1. Empty selections and charges** The old epsilon normalization mixed the pool at zero raw-mask cost; we corrected it. For a fixed input, given utility $U_i$, training-unit cost $\tilde c_i$, and price $\lambda$, select experts with $s_i=U_i-\lambda\tilde c_i>0$. Empty selections use the highest score; ties use lower cost, then index. Attention, queries, fusion, and charges share this effective set; other experts have zero weight. NaNs (not-a-number markers) in unselected responses preserve checked predictions (A.5–A.6). Original-model empties occurred only at these three settings. We retrained 17 budgets × four seeds. Rates precede fallback; accuracy/cost follow the main table. | Dataset/budget | Original empty (%) | Retrained empty (%) | Accuracy (%) | Cost | |---|---|---|---|---| | CIFAR-10/2 | 9.050±5.350 | 0.1200±0.1203 | 90.41±0.10 | 1.94±0.01 | | SVHN/0.03 | 0.6271±1.0846 | 0.8259±0.5540 | 74.02±2.80 | 0.0235±0.0060 | | SVHN/0.06 | 0.0029±0.0058 | 0.0010±0.0019 | 76.67±0.61 | 0.0529±0.0064 | CIFAR-10/2 empties fell 98.7%. Empty rates use four-seed means ± sample standard deviations (A.6). Full-pool training-response acquisition (8/input on CIFAR-10; 0.2 on CIFAR-100/SVHN) is recorded separately from prediction-time consultation. **W5/Q2. Training/reporting cost transfer** Both costs use the same set. CIFAR-10 costs/budgets coincide. CIFAR-100/SVHN use training prices (0.1,4.5,5,4,3.5,3), reporting prices (0,0.045,0.05,0.04,0.035,0.03), and $B_{\mathrm{train}}=100B_{\mathrm{report}}$, for reporting budget $B_{\mathrm{report}}$. Thus $$C_{\mathrm{train}}=100C_{\mathrm{report}}+0.1m_0^{\mathrm{eff}} \ge100C_{\mathrm{report}},$$ where $m_0^{\mathrm{eff}}$ indicates consulting expert 0. Training-budget feasibility then controls reporting cost on the same samples/distribution. Generally, $\tilde c_i\ge\kappa c_i$, $B_{\mathrm{train}}\le\kappa B_{\mathrm{report}}$, and $\kappa>0$ suffice; $c_i$ is reporting-unit cost. The 1% validation tolerance transfers too. All 17 mappings and proofs are in A.5.
Reviewer_CUKcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CATABC, a framework that chooses the experts to be consulted within a budget and proposes a way to combine them. For each input, an expert utility score is calculated and a cost penalty is subtracted. Only those experts whose adjusted score crosses a threshhold are selected. This allows the input size to vary across inputs. The paper analysis theoretically the threshold optimality, selection behaviour, prediction risk etc. Experiments were conducted on 3 datasets and the approach shows improvements over baselines.

Critical strengths
  1. The Problem is relevant and the solution is novel - The problem is significant and is very relevant in the current scenario. The approach proposed is novel and explained with clarity.

  2. Experimental Rigor - The approach wass compared against many baselines across multiple datasets. Sufficient ablation study was also conducted.

  3. Theoretical Backing - The paper analysis theoretically the threshold optimality, selection behaviour, prediction risk etc

Critical weaknesses
  1. The guarantee that the paper provides in Theorem 1 is limited to the fixed utility case not the main adaptive setting.

  2. Missing Baselines (https://arxiv.org/pdf/2510.26706, https://openreview.net/forum?id=TAvGZm2Rqb)

Questions for authors
  1. Can the main experiments be also run by giving zero weight for unselected experts?

  2. Did Top-k/Top-k(x) use the same backbone and feature extractor as CATABC on all datasets? Did Top-k(x) baseline retain its original CLIP-based encoder.

  3. Can you provide sensitivity analyses for \eta, Ts/Tf, constants (\epsilon m, \epsilon l, \epsilon w)?

3 comments

Thank you for raising the fixed-utility question and suggesting these baselines.

W1. What does the fixed-utility result tell us about training?

For fixed predictor utilities and a fixed price, Theorem 1 shows that thresholding the price-adjusted utilities solves the proxy selection problem. During joint training, these utilities change as the selector learns from the fused prediction loss. Theorem 2 gives the local surrogate gradient from that loss to the selector, with the attention mask and fallback choice held fixed. Whether these updates converge to a globally optimal policy remains open.

W2. Comparisons with BMD and DeCCaF

We added BMD (DeSalvo et al., 2025) and DeCCaF (Alves et al., 2024) on all three datasets. BMD budgets expert queries during training. We evaluate its learned router at prediction time. For DeCCaF, we use its constraint-programming assignment with a batch consultation budget. Both baselines assign each input to the machine or one expert. Each cell gives test accuracy (mean ± standard deviation, %) and mean consultation cost over four seeds. Implementation details are in B.3.5-B.3.6.

Dataset / budgetCATABCBMDDeCCaF
CIFAR-10/391.66±0.09/2.7788.56±0.13/2.2089.94±0.23/2.45
SVHN / 0.0374.02±2.80/0.023569.87±0.17/0.023571.11±0.04/0.0247
CIFAR-100/0.0560.80±0.54/0.044961.81±2.20/0.033666.11±0.52/0.0348

On SVHN, CATABC and BMD had the same reported mean cost. CATABC’s mean accuracy was 4.15 percentage points higher, with greater variation across seeds. On CIFAR-100 at budget 0.05, DeCCaF was 5.31 points more accurate and used less of the budget. Tables 8-10 report the full results.

Q1. Do unselected predictors contribute at inference?

We reran the main experiments and ablations with strict forward fusion. In the original implementation, smoothing gave unselected experts positive weights. An empty selection could therefore use the whole pool at zero reported cost. The revised rule selects experts with positive price-adjusted scores. If none qualify, it calls the highest-scoring expert and charges its cost. Ties are resolved by lower training-unit cost and then by expert index.

For each consulted expert, we normalize exp⁡(ai(x)/Tf)\exp(a_i(x)/T_f) over Seff(x)S_{\rm eff}(x). Every expert outside Seff(x)S_{\rm eff}(x) receives exactly zero weight.

Here Seff(x)S_{\rm eff}(x) is the set actually consulted, ai(x)a_i(x) is the fusion score, and TfT_f is the fusion temperature. A fallback singleton receives weight one. Response access, attention, fusion, and charging all use this set. Shuffling unselected responses or replacing them with NaNs left the checked predictions unchanged. Training uses the strict rule on the forward pass and a smooth surrogate for backpropagation (A.3).

We also isolated the effect before retraining. At budget 2 on CIFAR-10, the original raw empty rate was 9.050±5.350%. On the same four checkpoints, changing only the fallback changed overall accuracy by -0.055±0.076 percentage points. Adding strict fusion and effective-set attention changed it by a further 0.000±0.000 points at the reported precision. After retraining across 17 dataset–budget settings with four seeds each, the CIFAR-10 result at budget 2 was 90.41±0.10% accuracy at cost 1.94±0.01 (A.6).

Q2. Which backbones do the methods use?

CATABC and both Top-k baselines use ResNet-18 on CIFAR-10 and ResNet-4 on CIFAR-100 and SVHN for routing. Yes, Top-k(x) retains the frozen CLIP ViT-B/32 image encoder used by the reference implementation. Its cardinality head uses CLIP features to choose how many predictors to consult. The parameter table lists this encoder’s 87.85 million frozen parameters separately from the trainable router and cardinality head. Reported cost covers consultations with the predictor pool. It excludes routing computation.

Q3. How sensitive are the results to the hyperparameters?

On CIFAR-10, we tested 18 variants at budgets 3, 4, and 5 with four seeds each. The last column gives the highest mean accuracy among the tested alternatives for each budget, in that order. The offsets smooth the backward weights, their normalization, and the loss logarithm.

AxisTested valuesDefaultBest variant accuracy (%)
Price step0.001, 0.005, 0.02, 0.050.0191.34/91.47/91.42
Selector temperaturefixed 0.2, 0.5, 1, 2geometric annealing 2.0→0.291.50/91.49/91.53
Fusion temperature0.5, 1, 2, 31.591.53/91.50/91.61
Offsets10⁻¹⁰, 10⁻⁶10⁻⁸91.50/91.47/91.46

Default accuracy was 91.66/91.72/91.61%. Some settings had similar accuracy but different costs. At budget 5, a fusion temperature of 3 matched the default accuracy and raised mean cost from 2.96 to 3.66. A price step of 0.05 raised mean cost to 4.61. Changing the offsets shifted accuracy by at most 0.5 percentage points. B.4.2 reports accuracy and cost for every configuration.

The authors have performed the baseline comparisons I requested and also resolved (within a limited scope -- eg: 1 dataset sensitivity analysis) other concerns that were pointed out. I will therefore raise my rating.

We sincerely thank you for carefully reading our paper and for your constructive and positive feedback. Your comments and suggestions were very helpful to improve our work, and we truly appreciate your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.