acceptodds
Under review as a conference paper at ICLR 2027

Conformal Routing of Partially Transferable Experts

Abstract

Transferability need not be a property of an entire source model; useful computations can coexist with source-specific shortcuts, and their value can vary across target inputs. We introduce ROUTECP (Conformal Routing of Partially Transferable Experts), which aligns reusable source experts to a target representation and learns a sparse input-dependent route over them. Training directly targets conformal efficiency under a compute budget, while a finite-partition safety certificate suppresses expert contributions that cannot be certified as nonharmful. An untouched target calibration split forms a calibration firewall, preserving finite-sample marginal target coverage under calibration-test exchangeability despite adaptive source training, routing, selection, and certification. We derive an oracle inequality for sparse local routing, prove a separation from whole-source transfer under partial sharing, and establish simultaneous cell-average safety over a finite target partition. Simulations and real-data experiments show smaller prediction sets, improved low-resource transfer, reduced negative transfer, and favorable compute-efficiency tradeoffs at nominal coverage across heterogeneous target domains.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Conformal prediction, Mixture of experts, Negative transfer, Sparse routing, Transfer learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VcWGOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This work proposes RouteCP a method that aims to dynamically reuse parts of source models (denoted as “experts”) for a given target task at the instance level. The authors construct that routing mechanism without needing to evaluate all experts at test time by introducing expert embeddings that are used together with the representations of the target datapoints from the shared backbone to predict what experts to use for that specific datapoint. They then enforce sparsity on the expert selection by manually keeping only the top-qq experts, renormalize the routing logits over that subset and construct datapoint representations by incorporating a weighted sum of these experts to the base backbone representation. The authors then create a specific non-conformity score involving the classification probability of the label under the model that incorporates the shared experts along with a variance term that corresponds to the spread of the representations given by the selected experts. They then introduce the overall loss to train the system that involves 5 terms, a standard CE loss, a conformal coverage loss that aims to reduce the set size, an alignment term between the class-conditional representations of the experts and two terms, which are relatively standard in mixture-of-experts literature, that encourage reduction in entropy of the expert selection mechanism along with load balancing across the experts. Finally, the authors discuss some theoretical properties of the algorithm and evaluate it on both toy tasks and domain adaptation tasks showing improved accuracy and smaller conformal prediction set sizes.

Critical strengths

I unfortunately couldn’t pinpoint something.

Critical weaknesses

I believe that this work has several critical weaknesses which make me recommend rejection. More specifically

  • The manuscript is quite hard to parse and follow, primarily due to a quite aggressive AI use in writing. There are several areas that do not immediately follow from what was discussed or are generally not clear. For example
    • At section 1
      • What is a “cell” and its “cell-average”? Discussed at line 74
      • What “safety certification” do you talk about at line 71?
    • At section 2
      • Line 99 talks about a “transfer unit” being “auditable”. What does this mean? What is the “audit” that it goes through?
      • At line 135, there is a paragraph about an “efficiency analysis” but, as far as I am concerned, it doesn’t connect with the text above.
      • There are frequent mentions about “certification” but the concept was not introduced so it is unclear what it refers to
      • The authors talk about a “safety stage” and a “target only counterfactual”. Again none of these were introduced and it is unclear what they even mean.
    • At section 3
      • What are the “cells”? What are you actually certifying?
      • How is equation 3.4 obtained? It seems to be dropped out of the blue.
  • Limited novelty: overall, it seems that the authors apply classical methods from the Mixture of Experts literature (expert embeddings for routing, load balancing and sparse expert selection) while making them aware to the conformal prediction procedure that happens at test time. This has also been introduced in, e.g., Stutz et al. (which the authors do not seem to cite)

Stutz et al., Learning Optimal Conformal Classifiers, ICLR 2022

Questions for authors

Some questions for the authors are the following

  • The amount of ueu_e is fixed and ueu_e is learned per expert and enters re(x)r_e(x) only via an inner product with AuA_u. Why do you even need AuA_u? You can just learn an expert specific bias to shift A0h0(x)A_0h_0(x).
  • Why is ∥π(x)∥0≤q\|\pi(x)\|_0 \leq q and not ∥π(x)∥0=q\|\pi(x)\|_0 = q when you mention in the text that you “select the qq largest logits Iq(x)I_q(x)"? Furthermore, in the text that follows the authors argue about a “no-transfer” action which means that no expert was selected. How does this happen if you “select the qq largest logits”? Is there some threshold also involved that allows for no expert to be selected? If yes, it should be mentioned in the text.
  • Do the conditions for the results in section 4 (the ones discussed in lines 244-246) hold in practice?
  • How big does the target dataset need to be in order to learn a reliable routing for RouteCP?
Reviewer_brkPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper focuses on a model design for model transfer. Assume a set of experts exist for a task on different domains, and we want to transfer them into a new target domain. The authors build a set where every expert block ( a residual endpoint, or a collection of LoRA, or a set of attention blocks) can all return internal embeddings. These embeddings are transformed (with an affine transformation), and the results are combined using a trained router that is trained in a sparse way through an importance function. Also, the training of the model is done in a conformal endpoint regime. The design overall is interesting and seems like many considerations regarding the application are applied to it. But in general, I think the paper lacks a clear description and motivation behind each block.

Critical strengths

I can only come up with one strength, and that is that the paper's method is well engineered, and there are interesting components in it considering specific endpoint use cases. For example, the conformal error loss, or the scaling based on the diversity of the accepted expert are interesting blocks that were not quite trivial for me if I was asked to design a pipeline for this application. I think also that these clear engineering choices are reflected in the empirical results, which I think is roughly sufficient.

Critical weaknesses

W1. The paper does not onboard the reader; motivation and problem setup are largely missing before page 4. The paper sits at the intersection of two areas (modular/component-level transfer learning and conformal prediction), yet the introduction does not state the problem setup in a way that a reader from only one of these communities can follow. Concretely:

  • The introduction (Section 1) moves from transfer learning to conformal prediction within a few lines without explaining what the problem instance is (what is given, what is learned, what is guaranteed). A reader who is not already an expert in modular transfer cannot reconstruct the setting from the introduction alone. I think the paper needs roughly half a page to a page more of introduction, and the current page budget spent on it is not adequate for an intersection paper.

  • Section 2 switches from the design of the transfer architecture to conformal scores at line 124 with no transition. LAC, APS and RAPS are used there as if they were defined, but there is no background on conformal prediction or on these score functions anywhere in the main paper. The reader has to stop and work out why conformal prediction appears at this point and what role it plays.

  • Lines 170 to 172: the loss terms in (3.1) are listed but not motivated at the point where they are introduced. For example, it is not explained why class-conditional means and marginal covariances of the transported source features need to be matched to the target backbone (L_align); a one-line justification before each term would remove the need for the reader to reverse-engineer it. The one-sentence summary of the five roles at lines 180 to 183 comes after the fact and is too compressed.

  • Line 133: the sentence "We do not interpret d(x) as a calibrated epistemic uncertainty estimate" appears without anything in the surrounding argument having raised epistemic uncertainty. Either a citation or a short explanation of why this disclaimer is needed is missing.

  • Sections are not broken into paragraphs with headings or subsections, so the shifts between transfer architecture, scores, training losses, selection, and certification are abrupt.

Overall, up to page 4 the paper reads as a technical report of a system built for a specific task rather than as a paper that motivates each building block and leads the reader to the idea. This is consequential to my assessment because the main contribution is a multi-stage pipeline, and if the stages are not motivated the reader cannot judge which parts are essential and which are engineering choices.

  • "Safety", "certificate", "certified", "nonharmful" and "harmful" are used from the abstract onward, but the paper never says up front what threat or failure model these words refer to. In the security and robustness literature these words mean certified robustness to perturbations (randomized smoothing, verifiers, norm balls), and in the LLM literature "safety" and "harmful" refer to alignment and jailbreaks. Neither seems to be what is meant here; as far as I can reconstruct from Section 3, "harmful" means that deleting an expert decreases a smoothed set-size loss on average within a cell, and the "certificate" is a one-sided empirical-Bernstein bound on that cell-average deletion effect (Eq. 3.3 to 3.4, Proposition 4.3). This should be stated explicitly and early, because the current wording oversells what is guaranteed (it is a statistical bound on an average set-size effect, not a robustness certificate).

  • Lines 165 to 166: the differentiable surrogate for conformal set size, built from a smoothed quantile and a sigmoid relaxation of set membership, is the conformal training construction of Stutz et al. (2022, "Learning Optimal Conformal Classifiers", ICLR) and closely related to Einbinder et al. (2022). Neither is cited at this point (Stutz et al. is not cited anywhere in the paper), and the text presents the surrogate as if it were part of the contribution. Beyond this specific instance, the method sections (Sections 2 and 3) carry very few citations relative to the number of borrowed constructions, which makes it hard to delineate the paper's actual novelty. This is the first thing I could spot easily; I would ask the authors to audit the method sections for other uncredited components.

Questions for authors

Q1 (line 81, shared label space). The setup assumes all sources share the label set [C]. Since only intermediate expert maps h_sk are reused, and they are re-aligned by an affine transport into the target space, nothing in the architecture seems to require shared labels. Is the shared-label assumption needed only for L_align (class-conditional means), or also elsewhere? Could sources trained on different classification tasks be used, and if so, what breaks?

Q2 (choice of the expert bank). The partition of each source into experts is "prespecified" and "fixed before target model selection" (Section 2). How is it chosen in practice? Is there any automated procedure, or do the authors hand-pick which residual stages, adapters, attention blocks and LoRA updates enter the bank? This matters for assessing whether the gains could come from cherry-picking a favorable decomposition. Please state how the banks were constructed for each benchmark and whether results are sensitive to that choice (Figure 5, middle, seems related, but the construction itself is not described in the main text). From the text, I can assume that if we throw all the sources to the model, the model should be robust, not get misled by some of them, and be able to train easily.

Q3 (is the model designed for conformal prediction, and what is left without it?). Is the model trained specifically to be deployed with conformal prediction at the output, as in conformal training (Stutz et al., 2022; Einbinder et al., 2022), or is L_conf used only as a calibration-like regularizer? Related: what does the method offer if a user wants a single point prediction rather than a set? Is the final conformal guarantee simply split conformal prediction applied on top of a frozen predictor (in which case it follows from existing results and is not specific to this system), or is conformal prediction used inside the system in a way that is essential to the method? Please make this distinction explicit.

Q5 (Algorithm 1, decoupling training from inference). Which steps of Algorithm 1 are offline and which happen at inference time? My reading is that steps 1 to 7 are all offline (steps 6 and 7 consume the certification and calibration splits once), and that inference consists only of computing the route, with all objects frozen. Please confirm and mark this in the algorithm.

Reviewer_5cc9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces RouteCP, a method that aims to improve domain adaptation by combining expert routing with conformal prediction and certification. It learns input-dependent routing to individual expert components (rather than entire expert models) in a conformal way and uses certification for eliminating expert components that are potentially harmful. The method is validated in simulation and on established domain adaption benchmarks.

Critical strengths

The paper combines a lot of interesting ideas, such as conformal prediction, expert routing at component granularity, and certification into a unified method. Perhaps most interesting is the use of certification for eliminating potentially harmful expert contributions, which is a contribution that seems like it could be useful for the MoEs more broadly. I furthermore appreciate how well the evaluation disentangles the individual contributions of the different methodological components.

Critical weaknesses

===Novelty=== I think it would be helpful to more clearly describe the contributions of this work in the context of prior work. For example, one of the main contributions is routing at component-level granularity, and while I think the way this is implemented and evaluated here is novel, there are prior works in this area that are currently not cited (e.g., [1]). There is furthermore some recent work in conformal prediction using MoEs [2].

===Component routing=== In the empirical applications, it is currently unclear how expert models are decomposed into components, which makes it difficult to assess the component-focused expert routing as one of the main contributions. The paper says "Residual stages, adapters, grouped attention blocks, and LoRA updates form the expert bank", but it is unclear to me how exactly these are selected and how we end up with 12-32 candidate experts. It would furthermore be interesting how the set of selected expert components is distributed across source models (e.g., do most selected expert components come from one model?).

[1] https://dl.acm.org/doi/pdf/10.1145/3639592.3639615 [2] https://proceedings.iclr.cc/paper_files/paper/2026/hash/97c8a8eb0e5231d107d0da51b79e09cb-Abstract-Conference.html

Questions for authors
  • Can you clarify exactly how the expert bank are constructed in the empirical section?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.