acceptodds
Under review as a conference paper at ICLR 2027

When Does Input Adaptation Need a Selector? The Role of Routing-Capable Baselines

Abstract

Input adaptation gives a frozen predictor several possible corrections. A learned selector can choose one for each example. Its value, however, depends on what a single candidate can already do. We study this dependence by adding routing candidates built from the same fitted maps. This enlargement preserves the pointwise oracle for fixed row-wise maps. It strengthens the best complete candidate and cannot increase selector headroom. An exact value decomposition locates the remaining losses in a learned tree. The controlled study makes the effect concrete. With 200 development examples, the original selector gains 9.6 accuracy points over the frozen predictor. A routed candidate gains 12.4 points using the same maps. Across 36 datasets, cross-fitted selectors add little to enlarged libraries with three frozen networks. Split learning and TabICLv2 retain positive increments, showing that the outcome depends on the learning setup. Natural-shift sensitivity checks separate adaptation gains from inactive input pathways. Finally, we distinguish testing an overall improvement from testing a selector upgrade. The resulting evaluation asks what selection adds after simple routing is already available to the baseline.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
data finetuning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_6WbLOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary
Critical strengths
Critical weaknesses
Questions for authors
Reviewer_MweLOpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper claims the selector can receive credit for routing that was already available from the fitted partition and corrections. The authors strengthen the comparison library before evaluating selection. Cross-fitted selectors add little to routing-capable libraries with three networks. Split learning gives positive increments, and TabICLv2 adds 0.80 pp.

Critical strengths

The distinction between a single correction map and a complete routing procedure is insightful. It exposes how weak baselines can overstate the benefit of a learned selector.

The controlled experiments are well designed: candidate libraries share the same fitted maps, making it possible to attribute differences to routing capacity rather than better corrections.

Critical weaknesses

The writing is not easy to follow, with abrupt transitions between definitions, methodological choices, and conclusions. The relationships among maps, routed candidates, defaults, and selectors need clearer explanation, especially when the experimental setup changes.

The theoretical contribution is modest: the headroom result follows directly from enlarging the candidate library without changing the pointwise choices. It explains the comparison but provides limited guidance for constructing effective routing candidates.

Selector gains depend strongly on the learning protocol, making the practical conclusion less straightforward. Cross-fitting changes maps between selector training and deployment, while split learning can weaken the default by reducing its fitting data.

The evidence mainly concerns shallow tree selectors and a restricted family of input corrections. The positive TabICLv2 result further limits generalization, since the predictor and candidate family change together.

Questions for authors

How should routing partitions and candidate combinations be chosen when the appropriate grouping is unknown? Please clarify whether the conclusions hold beyond the two-cluster construction.

Can the effect of cross-fit-to-refit changes be isolated while keeping map quality and training-data access comparable? This would help explain the difference between cross-fitted and split selectors.

Do more expressive or better-regularized selectors improve over the same routing-capable defaults? A controlled comparison that changes only the selector would clarify whether the small gains reflect limited headroom or selector limitations.

Reviewer_HY7gOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

To make the best of a foundation model, we usually need a good selector to target on the key part of the inputs for the foundation model to make its best prediction. The proposed method emphasized the routing candidates can substitute the use of selector.

Critical strengths

The paper addressed an important research issue when people need to know how to achieve the best of a foundation model by input manipulation techniques. The research topic also belongs to a hot topic.

Critical weaknesses

• The technical depth of the proposed method is limited.

• The writing style is a bit awkward. Some part of the writing is like in a memo style. There are very few sentences to make the two consecutive paragraphs connected. Moreover, a lot of contextual background should be provided in the text to make the article readable to human readers.

Questions for authors

• The author(s) should provide definitions or some explanation for the keywords used in this work. For instance, “selector”, “candidate”, “routed candidate”, “map” as they are used quite often in scientific work. One may not notice that the usage of those in this text belongs to the restricted reserved meanings or their common meanings. For instance, the sentence: “Yet that benefit may reflect a weak comparison library rather than a need to learn another decision rule.” could be hard to understand when showing in the very beginning of an article. TabICLv2 to be mentioned in the first paragraph without saying the TabICLv2 model while it was just published in 2026. The author(s) did not even add a citation. The author(s) also did not mention that what the foundation models are focused on this work. Overall, I may expect that readers will have hard time to understand this work after a few years when this line of research is no longer a hot research topic.

• The presentation like Section 2 & 3 or even Section 4 contain almost no references. The contributions of this work are unclear, unless the author(s) claim that all the content of these sections are novel.

• The role of selector and routing candidates look possible to combine, as long as both of them are training-free. I do not understand why we have to separate them into two parts of the work for foundation model usage or input manipulation.

• What is the scope and limitation of the proposed work. It is unclear from the writing. For instance, what kind of data or what kind of foundation models can be benefited from the proposed method was not discussed in the work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.