acceptodds
Under review as a conference paper at ICLR 2027

Adaptive View Harness: Learning to Select Target-Preserving Views for Large Language Models

Abstract

Different input views have been shown to affect the practical performance of large language models (LLMs). We aim to leverage historical multi-view behavior to identify a more suitable input form for a new query before inference. To this end, we propose Adaptive View Harness (AVH), which selects a target-preserving view for a new query using only pre-call information while preserving single-call inference. AVH first uses Identity (ID) as the reference and defines the expected switching gain of each candidate view in terms of conditional accuracy change, estimating its relative utility from historical multi-view outcomes using calibrated heterogeneous predictors. It then performs instance-level view selection based on the estimated relative utility, while development-level Conservative Activation disables adaptive selection when the available development evidence is insufficient and retains ID instead. We evaluate AVH on BBH and MMLU-Pro with Qwen2.5-7B-Instruct, Hunyuan-A13B-Instruct, Ling-mini-2.0, and GLM-4.5-Air. Across the eight model–benchmark settings, AVH yields non-negative point estimates relative to ID and achieves a maximum accuracy gain of pp.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Adaptive View Selection, Target-Preserving Views, Prompt Sensitivity, Inference-Time Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_JvM1OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper tackles multi-view selection for LLM tasks. It proposes Adaptive View Harness (AVH) which leverages historical multi-view behavior to propose adaptive views for new queries without additional target LLM calls using predictive relative utility. AVH is evaluated across eight model--benchmark settings yielding gains of up to +3.42 pp compared to using the original input.

Critical strengths
  • The paper is clear and easy to follow.
  • Additional analysis separates improvement from global dataset-level view preference and instance-level preference, as well as giving the maximum possible improvement with an oracle, which helps put the performance improvement in perspective.
Critical weaknesses
  • The improvement in performance seems marginal compared to just using the original prompt and yet it requires access to historical multi-view interactions, which may limit its use in most practical settings. It is not not compared to a baseline that uses any other type of view selection or prompt transformation. Moreover, the method doesn't show any improvement on three of the four models on the MMLU-Pro dataset while the fallback avoids negative performance.
  • The three views proposed by the authors (OV, LV, and OLV) seem very simple, and it's unclear why they would enhance performance in a predictable and consistent way that can be leveraged by a trained predictor.
Questions for authors
  • Why were these three view construction methods in particular chosen? Have you experimented with more sophisticated ways to construct prompts, and do you believe this could help the method achieve stronger results
  • Have you tried assessing the number of training samples needed to obtain reliable predictors for view selection?
  • Have you tried comparing AVH to a prompt optimization baseline that has access to the same historical data?
  • The GLM model seems to benefit the most from AVH despite not having more Oracle headroom than the other models, which would suggest that its sensitivity to view choice is not larger. Have you investigated why?
2 comments

Thank you for reading our paper and for your positive comments on the presentation and Oracle analysis. We appreciate your suggestions and have revised the experimental comparisons and discussion of the results.

  1. About your concern that the improvements are modest compared with the cost of collecting historical data

We agree that collecting multi-view responses requires an initial investment. For each target model, we use 5,260 historical responses on BBH and 12,000 on MMLU-Pro during development. Once the selector is trained, each new question requires one target-model call. AVH achieves positive gains in five of eight settings, while CA retains Identity in the remaining three. We see this approach as more suitable for fixed models serving many questions from a stable task distribution. We've clarified the data requirements and deployment procedure in Section 4.1 and Appendix B.

  1. About your suggestion to compare AVH with other prompt selection methods

We added a fixed-view adaptation of Prompt-OIRL using the same four candidate views and historical responses. We trained a separate query–view reward model and included the results in Section 4.3, with its configuration described in Appendix D. We also added task-level and correctness-based routing comparisons in Appendix D. These experiments help us examine AVH alongside other selection strategies under matched target-model responses.

  1. About your question on why we chose these three simple view transformations

We chose option ordering and layout as two presentation factors. The four combinations give us ID, OV, LV, and OLV, while keeping the question, options, answer labels, and instruction unchanged. Even these simple changes produce different outcomes. For example, on Hunyuan/BBH, LV improves accuracy from 90.03% to 91.93%, while OV reduces it to 86.30%. We use these variations to study whether historical responses can help select a suitable presentation for a new question. We've expanded the explanation in Section 4.1 and Appendix A.

  1. About your question on why GLM benefits more despite having less Oracle headroom

This is an interesting observation. Oracle headroom measures the potential improvement available across views, while AVH must predict which view to use before seeing the model's response. For GLM, output parseability appears to play an important role. On MMLU-Pro, the parse success rate increases from 66.4% with Identity to 80.8% with AVH, alongside a 3.20 pp accuracy gain. On the all-four-parseable subset, the accuracy difference is −2.04 pp. We've added these results and their interpretation in Section 4.4 and Appendix F.

Thank you for the response and the additional experiments.

My main concerns about the source of the reported gains remain. The added comparisons suggest that simpler task-level selection or routing achieves similar results, so the benefit of the proposed instance-level selection is still unclear. The parseability analysis also indicates that some of the largest improvements may come from changes in output formatting rather than better answers.

These points leave my concerns about the soundness of the method and its evaluation unresolved. I maintain my score.

Reviewer_KRodOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AVH(Adaptive View Harness), which aims to reuse historical multi-view outcomes to select a target-preserving input representation before querying an LLM. AVH estimates correctness gains relative to the identity view and applies thresholding and development-set activation criteria to decide whether to switch. Experiments on BBH and MMLU-Pro evaluate four fixed views across four models under a single-call inference setting.

Critical strengths
  • Clear deployment objective. AVH maintains a strict single target-model call per query while learned estimators dynamically select views using pre-call information. Switching thresholds and Conservative Activation provide fallback to the identity view when predicted gains or development evidence are insufficient.
  • Clear reporting of switching outcomes. The paper reports Rescue, Harm, switching rates alongside aggregate accuracy. These statistics isolate corrected errors from newly introduced mistakes, making the observed outcomes of view switching easier to inspect.
Critical weaknesses
  • Limited comparisons with related methods. The paper cites Prompt-OIRL and other related works, but evaluates AVH against fixed-view strategies and its own ablations, leaving its advantage over existing offline prompt selection methods unclear. AVH's advantage over DS-FV is also inconsistent, as its accuracy is higher in three of eight settings, lower in two, and unchanged in three (Table 1).

  • Underexplored Estimator Capacity and View-Space Design. AVH falls back to ID for three of four models on MMLU-Pro despite substantial Oracle headroom (Table 1), but does not investigate whether richer representations or more expressive predictors could improve selection. Moreover, the candidate space is limited to four combinations of one fixed permutation (C,A,D,B) and one layout change, with no evaluation of alternative candidate sets. The experiments therefore leave the estimator's limitations and sensitivity to the chosen view space insufficiently characterized.

Questions for authors
  1. Baseline comparison. How does AVH compare with an adapted Prompt-OIRL under matched candidate views, historical data, and target-model call budgets? A paired comparison with DS-FV, including confidence intervals, would also clarify the benefit of adaptive selection.

  2. Estimator limitations. What limits useful adaptation for the three MMLU-Pro models that revert to ID, and how does increasing predictor capacity affect gains and inference cost?

  3. View-space design. What motivates the specific C,A,D,B permutation and bracket-based layout? Do the gains persist with alternative permutations, layouts, or answer-preserving instruction variants?

  4. Input encoding and shortcut learning. How are questions and options encoded, and what evidence shows that selection uses semantics beyond answer-label or position biases?

2 comments

Thank you for your thoughtful comments. We appreciate your recognition of our single-call setting and the Rescue/Harm analysis. We've revised the paper to explain the baseline comparisons and several design choices more clearly.

  1. About your suggestion to compare AVH with Prompt-OIRL and DS-FV

We added a fixed-view adaptation of Prompt-OIRL using the same four candidate views and historical target-model responses. The complete eight-setting results and the reward-model configuration are provided in Section 4.3 and Appendix D. We also agree that the comparison with DS-FV varies across settings. On Qwen2.5/BBH, DS-FV retains Identity at 41.32%, while AVH reaches 43.30%, with a paired 95% bootstrap interval of [0.84, 3.12] pp for the difference. DS-FV performs better on Hunyuan/BBH and GLM/BBH. These results show that the relative advantage of adaptive selection varies across target models and benchmarks.

  1. About your question on why three MMLU-Pro settings fall back to Identity

The Oracle results suggest substantial potential gains, but selecting a beneficial view before inference is considerably harder. Oracle uses the observed outcomes of all four views, while our selector estimates their correctness from question features. For Qwen2.5, Hunyuan, and Ling on MMLU-Pro, none of the thresholds achieves positive Net on both Train OOF and Validation, so CA retains Identity. We've clarified this behavior and its relationship to Oracle headroom in Section 4.4.

  1. About your question on the C-A-D-B permutation and bracketed layout

We chose four views around two presentation factors, option ordering and layout. C-A-D-B gives us a fixed ordering change, while bracketed labels provide a separate formatting change. Combining these factors produces ID, OV, LV, and OLV. We keep the question, option contents, label mapping, and answer instruction unchanged across views. This allows us to study changes in input presentation under a consistent task setting. We've expanded the explanation in Section 4.1 and Appendix A.

  1. About your question on input encoding and what the selector learns

We use a 384-dimensional MiniLM embedding of the question and labeled options, 28 numerical features, and task/category indicators. We've added the feature construction and predictor details in Section 3.2 and Appendix B. We also included a task-level baseline that selects one fixed view per category. On Qwen2.5/BBH, AVH improves over this baseline by 1.52 pp, with a paired 95% interval of [0.15, 2.89] pp. The complete comparison is available in Appendix D.

Thank you for your response. While new baseline comparisons have been added and the encoding method clarified, the issues regarding estimator capacity and the limitations of the candidate view space have not been sufficiently addressed. I therefore retain my original rating.

Reviewer_pPfuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes Adaptive View Harness, a system that chooses one of several presentations of a multiple-choice question before the target language model is called. The key insight is that a fixed model can answer the same question differently when only the way the options are displayed changes, therefore a presentation chosen per question can recover some of these answers. The paper develops a selector that learns from past questions answered under every presentation which presentation is likely to help a new question. Experiments on two multiple-choice reasoning benchmarks with four models demonstrate that the selector improves over the original presentation in some settings and keeps the original presentation in the others.

Critical strengths
  • The writing is easy to follow.
  • The paper frames the switching decision as an expected gain over the original presentation instead of a best-presentation label, which keeps the estimate tied to the accuracy change that switching causes.
Critical weaknesses
  • The method relies on offline data that limit its practical use. It needs gold-labeled questions from the deployment task, each answered by the same target model under all four presentations, and this data must be rebuilt for every new model and every new task.

  • The claim that presentation selection improves answer accuracy is not supported by the paper’s own results. For the model with the two largest gains, the gain disappears on the questions whose outputs can be parsed under every presentation, so the gain comes entirely from questions where one presentation failed to parse.

  • The evaluation protocol is unreasonable. It forces every model to answer with a bare option label and no reasoning, which drives the models far below their reported accuracy and inflates the presentation sensitivity that the method exploits.

  • The claim of question-level selection lacks experimental support. The selector receives the task label as an input feature, but the paper does not compare against one fixed presentation chosen per task.

Questions for authors
  • How does the method perform when the model is allowed to reason before answering within the same single call?
1 comment

Thank you for reading our paper carefully. We appreciate your positive comments on the writing and our use of expected gains relative to Identity. We've considered your concerns and revised the relevant sections.

  1. About your concern regarding the cost of collecting historical multi-view data

We agree that collecting historical responses is an important cost of AVH. For each target model, we use 5,260 development responses on BBH and 12,000 on MMLU-Pro. A new model or task distribution would require corresponding historical observations. We see AVH as particularly useful when a fixed model serves many questions from a relatively stable task distribution. The historical data support training, while each new query requires one target-model call. We've clarified these costs and the evaluation procedure in Section 4.1 and Appendix B.

  1. About your concern that the GLM improvements mainly come from output parsing

You raise a fair point. Our evaluation measures parsed-answer accuracy, so output formatting contributes to the measured results. On GLM, AVH improves full-set accuracy by 3.42 pp on BBH and 3.20 pp on MMLU-Pro, while the differences on the all-four-parseable subsets are −0.20 and −2.04 pp. We also examined Qwen2.5/BBH, where all four views have 100% parse success and AVH still improves accuracy by 1.98 pp. We've added these results and their interpretation in Section 4.4 and Appendix F.

  1. About your question on whether task-level selection is sufficient

Since task/category information is part of our input representation, we agree that a task-level comparison is useful. We added a baseline that selects one fixed view per task using training outcomes. On Qwen2.5/BBH, AVH reaches 43.30% compared with 41.78% for task-level selection, a gain of 1.52 pp. Task-level selection performs better in other settings, including GLM/BBH. The complete eight-setting comparison and paired confidence intervals are included in Appendix D.

  1. About your concern with the direct-answer protocol and your question on allowing reasoning

We use the same direct-answer instruction across all views to study changes in option ordering and layout under a fixed evaluation protocol. We agree that allowing a reasoning trace before the final answer, within the same target-model call, is another meaningful setting. Our current measurements concern direct-answer responses. Evaluating reasoning-allowed prompts would require collecting the corresponding four-view responses.

Reviewer_DctGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Adaptive View Harness (AVH), which selects how to present a multiple-choice question before making a single call to a fixed LLM. Using historical correctness outcomes, AVH trains four predictor families and averages their calibrated predictions to estimate each view’s advantage over the original. A threshold is then determined via validation, where the highest-advantage view is only selected if it crosses said threshold, and no transformation is applied otherwise. Results are mixed across benchmarks.

Critical strengths
  • Clear practical objective and development from motivation to the method design.
  • Achieved some demonstrated improvements with some relevant ablation studies.
Critical weaknesses
  • Very limited novelty: this method is basically correctness-based routing with a threshold. This is further obfuscated by the experiments having no other competing approaches as baselines for comparison, where there (at least) are multiple Prompt-OIRL-style works.
  • Insufficient justification on method specification, in terms of design, hyperparameter configurations, results, and ablation thoroughness (see questions)
  • The strongest improvement (with GLM) is questionable: accuracy gains reverse (goes negative) on the all-four-parseable subset.
Questions for authors
  • What are existing methods that aim to solve the same proposed problem, and how does AVH fare against them?
  • What exactly are the inputs to these correctness classifiers? And relevantly, how are the classifiers’ architectures designed, and how exactly are the OOF/folds specified?
  • How does AVH fare with different subsets of classifiers, and/or different weightings? What are their (dis)agreements across predictions?
  • Why was CADB selected? And what are the results with different choice-set permutations?
  • What is the training cost, and latency incurred by this screener?
  • When no threshold satisfies the eligibility criterion, which threshold is used for “AVH without CA,” and how is it selected?
2 comments

Thank you for taking the time to read our paper and for your detailed comments. We've gone through your questions carefully and revised several parts of the paper to explain our design choices and results more precisely.

  1. About your concern that AVH resembles correctness-based routing and your question about Prompt-OIRL

We agree that AVH is closely related to correctness-based routing and adaptive prompt selection. We focus on the decision to switch away from the original input. By estimating the expected accuracy change relative to Identity, we can use an instance-level threshold and Conservative Activation (CA) to decide when an alternative view is worth using. We expanded the discussion of related methods in Section 2 and added comparisons with an adapted Prompt-OIRL and direct correctness-based routing. The Prompt-OIRL comparison uses the same four views and historical responses, with its configuration specified in Section 4.3 and Appendix D. Since the comparison was conducted retrospectively under a particular reward-model configuration, we treat it as a supplementary reference. The correctness-based routing results also help explain how thresholding changes the balance between Rescue and Harm.

  1. About your questions on classifier inputs, OOF training, and predictor combinations

We have added the implementation details in Section 3.2 and Appendix B. We use a 384-dimensional MiniLM embedding of the question and labeled options, 28 numerical features, and task/category indicators. The feature dimensions are 423 for BBH and 426 for MMLU-Pro. Our predictors include Linear, MLP, Extra Trees, and HGB. We use five category-stratified folds to generate base OOF predictions, then fit separate Platt calibrators for each predictor family and view. The four families receive equal weights of 0.25. Figure 6 presents the recorded results for the individual predictors and their mean ensemble, with the available protocol details discussed in Appendix E. We agree that testing different predictor subsets, weights, and prediction agreement would help explain the ensemble in more detail.

  1. About your question on why we chose C-A-D-B and whether other permutations would work

C-A-D-B gives us a fixed ordering transformation that we can apply consistently across models, while the bracketed layout provides a separate formatting change. We find this setup useful for studying presentation sensitivity without adding examples or new reasoning instructions. The current results use this fixed candidate set, and testing more permutations would be useful for understanding how the choice of views affects performance.

  1. About your question on training cost and selector latency

You're right that offline cost matters for practical use. For each target model, our development stage uses 5,260 historical responses on BBH and 12,000 on MMLU-Pro. We have added these figures and explained the difference between offline response collection and online selection in Section 4.1 and Appendix B. After training, the selector uses MiniLM encoding and local predictor inference to choose a view before making one target-model call. A full latency measurement would need to include all of these local operations, and we agree that such a measurement would help assess the deployment tradeoff more precisely.

  1. About your concern that the GLM improvements reverse on the all-four-parseable subset

We appreciate you pointing this out. Our metric is end-to-end parsed-answer accuracy, so output parseability is an important part of interpreting the GLM results. On the complete Test sets, AVH improves GLM accuracy by 3.42 pp on BBH and 3.20 pp on MMLU-Pro. On the all-four-parseable subsets, the differences are −0.20 and −2.04 pp. The selected outputs also have higher parse success rates, increasing from 76.9% to 95.7% on BBH and from 66.4% to 80.8% on MMLU-Pro. We have added these figures and a fuller discussion in Section 4.4 and Appendix F. Qwen2.5/BBH provides another useful case. All four views have 100% parse success, while AVH still improves accuracy over Identity by 1.98 pp. We find the contrast between these models interesting, as it suggests that the contribution of output formatting varies across targets.

Thank you for the response. While the reply answered some of the questions I posed, it did not address the underlying concerns/help make the case for this work's novelty, significance, nor thoroughness. For that, I'm keeping my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.