acceptodds
Under review as a conference paper at ICLR 2027

Decision-Focused Acquisition of Preference Data for Learning Inference-Time Verifiers

Abstract

Modern Large Language models increasingly use their compute budget for inference-time computation. In applications such as code generation and tool-use, this often involves generating multiple candidate responses and learning a verifier that applies a decision rule such as best-of-K selection, reranking or shortlisting, or escalating to an expert to decide the best response. In such cases, the final utility of the response depends both on the verifier’s score and the decision operator applied. In this paper, we consider the problem of active learning to train such verifiers. Existing active learning methods for preference data select the comparisons that most reduce uncertainty in the verifier’s parameters, and are indifferent to which directions of parameter space the deployed decision actually consults. Instead, we propose the Decision Focused Acquisition (DFA) criterion, a c-optimal experimental design that targets the decision. We assume access to a small Decision Set where utilities are known and the DFA criterion selects the comparisons that most reduce the posterior variance of a linearised decision loss. This formulation applies to any decision operator that has a differentiable surrogate and we instantiate it for best-of-K selection, top-m shortlisting, and accept-or-escalate cascades. Across four datasets spanning preference reranking, code generation, and tool-use agents, we show that our method outperforms existing active preference learning methods, yielding verifiers with higher decision utility under fixed acquisition budget. Lastly, we show that the benefit of DFA is largest at small and medium budgets, where acquisition is most critical, and that it comes from acquiring comparisons at the decision boundary of the deployed operator.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Active Learning, Inference-Time Compute, LLM data selection, Learning Verifiers

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_YyfpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work approaches the problem of active acquisition of preference data for learning verifiers. It raises the concern that most methods in the literature focus on reducing epistemic uncertainty in the verifiers without awareness of the decision operator itself. It then proposes DFA, a method that targets the posterior variance of a linearised decision loss, rather than the posterior over the weights. The paper provides experimental evidence in several datasets (preference learning, coding, tool use), showing that the proposed method outperforms other acquisition criteria and one active learning recent method (ActiveDPO) in single batch acquisition.

Critical strengths
  • The paper is well written and well presented. Clear to understand. It also approaches a relevant problem (active learning for verifiers).

  • The paper presents a good experimental design: it asks relevant questions, provide evidence in a good number of benchmarks. The results/plots are clear and communicate the claims.

  • The employed methodology is novel, and the derivation is solid and mathematically well justified. Furthermore, the algorithm and practical implementation is mindful about scalability and discusses important aspects of the algorithm complexity costs.

Critical weaknesses
  • The work build on top of the motivation that many previous methods are based on parameter (epistemic) uncertainty as the criteria to select samples, but misses standard baselines in the literature like BALD [1] and variants [2], which were also used in the past in the same problem setting [3].

    • In this same line, there are some recent work that is very relevant to this work but missing in the related work section [3-7]
  • The work adopts a particular posterior approximation based on linearization and develops a decision criteria on top. However, other forms of posteriors are widely used in literature, from GPs to Deep Ensembles (which can be also easily approximated on top of frozen features). The work does not discuss the particular design choice for this posterior approximation, nor compare with other possibilities. It is also unclear how much the linearity assumptions affect the posterior approximation and the downstream decision.

  • While the experimental setup is very clear and convincing, it looks simplistic: a single batch acquisition, with a small batch of 32 samples. Usually preference learning deals with very large batches, iteratively. This is a crucial design because batch active learning is a challenge on its own (as opposed to single-point acquisition), and the work does not discuss this.

  • No discussion on limitations of the method, which I believe is important for the reader.

References

[1] Lindley. On a measure of the information provided by an experiment. The Annals of Mathematical Statistics, 1956.

[2] Kirsch et. al. Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning, 2021.

[3] Melo et. al. Deep Bayesian Active Learning for Preference Modeling in LLMs, 2023.

[4] Gleave and Irving. Uncertainty estimation for language reward models, 2022.

[5] Mehta et al. Sample efficient reinforcement learning from human feedback via active exploration, 2024.

[6] Ji et al. Reinforcement learning from human feedback with active queries, 2024.

[7] Dwaracherla et al. Efficient exploration for LLMs, 2024.

Questions for authors
  • Is the experimental setup only about a single-batch acquisition? Are there any results on sequential acquisition?
2 comments

Response to Reviewer 3

We thank the reviewer for the positive assessment of the paper and for recognizing the strength of the experimental design and methodology. We respond to the questions below.

1. Sequential acquisition

Is the experimental setup only about a single-batch acquisition? Are there any results on sequential acquisition?

In all experiments, we perform sequential acquisition in batches of eight labels. After each batch, we refit the verifier and recompute its posterior using all labels acquired so far. Within a batch, each selection is also reflected in the criterion through a rank-one covariance update before the next pick, helping avoid redundant comparisons. These alternating steps are detailed in Algorithm 1 and Section 3.3.

2. Epistemic-uncertainty baselines

The work build on top of the motivation that many previous methods are based on parameter (epistemic) uncertainty as the criteria to select samples, but misses standard baselines in the literature like BALD [1] and variants [2], which were also used in the past in the same problem setting [3]

We thank the reviewer for pointing us to this relevant literature. We will add these references and discuss how their epistemic-uncertainty criteria differ from targeting the deployed decision.

3. Choice of posterior approximation

The work adopts a particular posterior approximation based on linearization and develops a decision criteria on top. However, other forms of posteriors are widely used in literature, from GPs to Deep Ensembles (which can be also easily approximated on top of frozen features). The work does not discuss the particular design choice for this posterior approximation, nor compare with other possibilities. It is also unclear how much the linearity assumptions affect the posterior approximation and the downstream decision.

We agree that other posterior approximations could be used in the decision criterion. We chose Laplace because the main experiments use a regularized convex Bradley–Terry verifier, for which a local Gaussian approximation is natural and computationally practical (Bishop (2006)). Deep ensembles would require fitting multiple models and add computation cost.

To assess the Laplace approximation, we used Monte Carlo samples from the Bradley–Terry posterior as a reference on all four datasets at B=32B=32. We compared the two posteriors along the decision direction. The Laplace variance differs from the sample-based estimate by 0–9% on average, and the resulting acquisition rankings from the true posterior agree with Laplace’s rankings at Spearman correlation 0.95–0.99. Finally, the top candidate selected using the two posterior estimates agrees on 92–98% of evaluation prompts. These results support using Laplace in our linear-verifier setting, and we will include the sampling procedure and comparisons in the revision.

No discussion on limitations of the method, which I believe is important for the reader.

We will also add limitations of the current work in the final draft.

Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.

Dear authors,

Thank you for answering and commenting on the raised points and questions. It helped clarifying my questions, although no new evidence was provided in order to change my perspective in the paper. Therefore, I will maintain my score.

Reviewer_STUHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DFA, an active preference learning method that emphasizes final decision-making performance when training a verifier to select LLM responses. By utilizing a small decision set with known true utilities, it selects preference comparisons to reduce the posterior variance of the linearized decision loss based on c-optimal design. Across four datasets and three types of decision rules, it achieves high decision-making performance with fewer labels than existing methods.

Critical strengths

While conventional active preference learning emphasizes model prediction accuracy and uncertainty, this study prioritizes the collection of information critical to actual decision-making. The derivation of the acquisition function based on c-optimal design is also well motivated.

The method demonstrates improvements across four datasets and three types of decision rules. In addition to analyses regarding label budgets and decision set size, the results show that the acquired comparisons are concentrated near the decision boundary, which helps explain the performance improvements.

Critical weaknesses

Reliance on a decision set with known true utilities. The proposed method requires obtaining true utilities for multiple candidate responses in a decision set. However, the cost of acquiring this information is not included in the evaluation of label efficiency. Furthermore, the comparative methods do not utilize this additional information, and it remains unclear whether DFA would retain its advantage if the same information were available to the baselines.

The generalizability of the experimental setup is limited. In the experiments, only a low-dimensional linear verifier using features from a frozen LLM was trained. It has not been verified whether the method remains effective with large-scale nonlinear reward models or in settings where candidate distributions change due to updates to generative models.

Theoretical guarantees are limited to local approximations. Theorem 1 demonstrates a reduction in the posterior variance of the linearized decision loss under Gaussian approximation, but it does not directly guarantee improvements in actual decision loss or regret. Therefore, the theoretical contribution is limited.

Questions for authors

Q1. Does the proposed method's superior label efficiency hold even when accounting for the cost of obtaining the true utilities in the decision set? Furthermore, is it possible to compare against baselines that have access to the same information, such as training the verifier directly using the same decision set?

Q2. Can similar performance improvements be obtained using nonlinear reward models or LoRA-based training, rather than a linear verifier?

Q3. How does the reduction in posterior variance of the linearized decision loss relate to the actual improvement in decision-making performance? Please explain the conditions under which the approximation is effective.

1 comment

We thank the reviewer for the positive assessment of the paper and for recognizing the strength of aligning the acquisition criterion with downstream decisions. We respond to the questions below.

1. Reliance on a decision set with known utilities

The proposed method requires obtaining true utilities for multiple candidate responses in a decision set. ... it remains unclear whether DFA would retain its advantage if the same information were available to the baselines.

As the reviewer suggested, we ran additional experiments in which all methods start from a verifier trained on the same decision set. We report regret on EvalPlus and BigCodeBench with |D|=20 and 50 seeds, keeping the evaluation set from the main paper.

EvalPlus (|D|=20, lower regret is better)

MethodB=128B=256
D + Random0.168 ± 0.0060.170 ± 0.006
D + Min-margin0.179 ± 0.0060.174 ± 0.006
D + Max-margin0.179 ± 0.0060.160 ± 0.006
D + D-optimal (ActiveDPO)0.186 ± 0.0060.164 ± 0.006
D + DFA (ours)0.151 ± 0.0060.153 ± 0.005

BigCodeBench (|D|=20, lower regret is better)

MethodB=128B=256
D + Random0.243 ± 0.0060.264 ± 0.008
D + Min-margin0.240 ± 0.0050.253 ± 0.007
D + Max-margin0.250 ± 0.0050.249 ± 0.005
D + D-optimal (ActiveDPO)0.244 ± 0.0050.255 ± 0.004
D + DFA (ours)0.225 ± 0.0050.228 ± 0.005

While all baselines benefit from the decision-set supervision, DFA still outperforms them by using D to select comparisons that improve downstream decisions.

2. LoRA-based verifier training

Q2. Can similar performance improvements be obtained using nonlinear reward models or LoRA-based training, rather than a linear verifier?

We performed an additional experiment to show how DFA can be used with LoRA-based training. On EvalPlus, we train a Qwen3-4B verifier with LoRA updates. We run the other acquisition baselines with the same training setup at budgets B=32,128,512.

EvalPlus, LoRA verifier (lower regret is better)

MethodB=32B=128B=512
Random0.234 ± 0.0250.260 ± 0.0160.210 ± 0.022
Min-margin0.267 ± 0.0170.237 ± 0.0200.221 ± 0.017
D-optimal (ActiveDPO)0.285 ± 0.0190.278 ± 0.0250.224 ± 0.026
DFA (ours)0.214 ± 0.0200.221 ± 0.0190.191 ± 0.016
Random candidate0.2770.2770.277
All pairs, true labels0.1390.1390.139

These results show that DFA-based acquisition can also be used when training a LoRA verifier, and it achieves the lowest mean regret at each reported budget. We will include the LoRA experiment in the revision.

3. Posterior variance and decision loss

Q3. How does the reduction in posterior variance of the linearized decision loss relate to the actual improvement in decision-making performance? Please explain the conditions under which the approximation is effective.

Under the local Gaussian model, Theorem 1’s variance reduction also equals the expected squared change in our estimate of the linearized decision loss after observing q’s label. This follows from the law of total variance where the updated covariance does not depend on the label, so the variance reduction measures how much the posterior mean can move. DFA therefore selects comparisons whose labels can meaningfully inform the deployed decision. Figure 2 shows that these comparisons lie near decision boundaries, and Table 1 and Figure 1 show improved decision performance.

The criterion uses two approximations:

  1. Gaussian posterior. The Laplace approximation is most appropriate for a concentrated, approximately unimodal posterior, as with a regularized convex linear Bradley–Terry model (Bishop (2006)). In additional experiments on all four datasets at B=32, Laplace variance in the decision direction is within 0–9% of the Monte Carlo estimate of true Bradley-Terry posterior on average and the top selected candidate agrees on 92–98% of evaluation prompts.

  2. Locally linear decision loss. We noted that the linearization works well when the uncertainty in the verifier is small relative to how sharply the decision changes. This happens when the surrogate temperature τ is not too small, and the top candidates are not near ties. In our additional checks, the linear prediction tracks the actual change in decision loss with correlation 0.95 in one reported setting and 0.74–0.90 at the largest budgets. Across all four datasets, the linearized decision loss correlates with the actual (smoothed) decision loss at 0.86–0.93 at B = 32.

We will clarify these conditions and include the additional approximation checks in the revision.

Reviewer_UPw1OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

AI was used to rephrase/polish wording. The reviewer wrote the original assessment from scratch and reviewed the final wording.

Original reviewer-written assessment supplied as input:

Flag For Ethics Review

No.

Critical Weaknesses

  1. The label-efficiency claim leaves out a substantial source of utility supervision that only DFA's acquisition rule can use. In Table 3, decision sets contain 90–200 prompts with utilities for 8–16 candidates per prompt, yet the headline comparison counts only the 32 acquired preference labels. Although the experiments with smaller decision sets are valuable, even 25–50 fully scored prompts may amount to substantial supervision. The present comparisons therefore show a benefit that is conditional on access to this utility set; they do not show an advantage in overall annotation cost. The all-pairs oracle reference does not settle the matter, since it changes both the amount and the source of supervision. To clarify how much of the benefit comes from the acquisition criterion itself, the paper would need a learner that uses only the decision set and methods allowed to train or tune on the same utility information.

  2. Realized coverage in the accept-or-escalate evaluation needs to be accounted for more clearly. Appendix D.5 argues that the escalation term is constant because coverage is fixed at 0.7; however, the threshold is estimated on D and then carried over unchanged to E. This fixes empirical coverage on D, but not necessarily on E. If methods accept different fractions of the evaluation prompts, leaving out the escalation cost could change how they compare. While the selective-regret result remains informative, it does not by itself establish higher cascade utility at matched realized coverage or at a fixed escalation price. Because this concerns one of the three headline decision operators, it should be resolved by reporting acceptance rates and an explicitly defined evaluation denominator.

Questions For Authors

  1. How does DFA compare with training the same verifier directly on the known utilities in D, or with combining that supervision with the same B weak preference labels? Please also include a cost comparison that accounts for constructing D.
  2. In the accept-or-escalate setting, what fraction of E does each method actually accept? Do the reported values represent conditional regret over accepted prompts, or regret summed across all evaluation prompts? Does DFA keep its advantage at matched realized coverage or under an explicit escalation cost?
  3. According to Appendix F, lambda is chosen based on the B=32 mean. What data and metric determine this choice, and how are hyperparameters chosen in the main experiments? Please make clear that evaluation outcomes play no role in model selection.

Rating

3: Weak acceptance.

I lean toward acceptance. The acquisition formulation is coherent, the decision-dependent mechanism is well motivated, and the experiments extend beyond a single favorable benchmark. My positive view is contingent on the stated availability of a decision set labelled with utilities; I do not consider the reported savings in preference labels to demonstrate a lower total supervision cost. The paper would be considerably strengthened by clearer comparisons under equal information and by clearer cascade accounting. Should the cascade gains be explained by differences in realized coverage, that part of the conclusion would have to be narrowed.

Soundness

2: I have minor or fixable concerns about the paper’s soundness.

Paper Summary

The paper asks which preference comparisons should be acquired when a learned verifier will be deployed through a particular decision operator. DFA relies on a separate prompt set with known candidate utilities to compute the gradient of a differentiable decision loss. Using a local Gaussian approximation, it scores each candidate comparison by how much it reduces the posterior variance of that linearized loss. The resulting criterion amounts to a greedy c-optimal design step whose direction depends on the decision. Best-of-K selection, top-m shortlisting, and accept-or-escalate decisions are instantiated, and rank-one updates are derived to make batch acquisition efficient. Across four datasets with frozen low-dimensional verifier features, DFA attains lower mean regret in every one of the twelve dataset–operator settings at a budget of 32 comparisons. Further experiments study decision-set size, operator mismatch, annotator degradation, alternative feature models, and distribution shift.

Presentation

3: Clear and easy to follow.

Critical Strengths

  1. Distinguishing accurate preference learning from acquiring information that serves a deployed decision has real consequences. The c-optimal formulation turns this distinction into something concrete: a comparison is worth acquiring when it lowers uncertainty along the local gradient of the decision loss, not merely when it resolves an uncertain ranking. Theorem 1 spells out the approximations under which this interpretation holds, and the covariance updates make the criterion practical, without asserting a general guarantee on final decision regret.

  2. The evaluation tests several implications of the proposed mechanism. At B=32, best-of-K regret drops relative to random acquisition from 0.2008 to 0.1470 on EvalPlus and from 0.1737 to 0.1049 on tau-bench. Decision specificity is supported by the operator-mismatch experiment and the analysis of acquired pairs. Importantly, the appendices also study a different feature model, degraded annotators, and a shift from MBPP+ to HumanEval+. The zero-correlation annotator control is particularly valuable: when acquired labels carry no information, DFA's advantage disappears, which shows that the decision set does not by itself solve the preference-learning task.

Informativeness

3: I gained a substantially deeper understanding of the problem or question.

Critical strengths

The paper asks which preference comparisons should be acquired when a learned verifier will be deployed through a particular decision operator. DFA relies on a separate prompt set with known candidate utilities to compute the gradient of a differentiable decision loss. Using a local Gaussian approximation, it scores each candidate comparison by how much it reduces the posterior variance of that linearized loss. The resulting criterion amounts to a greedy c-optimal design step whose direction depends on the decision. Best-of-K selection, top-m shortlisting, and accept-or-escalate decisions are instantiated, and rank-one updates are derived to make batch acquisition efficient. Across four datasets with frozen low-dimensional verifier features, DFA attains lower mean regret in every one of the twelve dataset–operator settings at a budget of 32 comparisons. Further experiments study decision-set size, operator mismatch, annotator degradation, alternative feature models, and distribution shift.

Critical weaknesses
  1. Distinguishing accurate preference learning from acquiring information that serves a deployed decision has real consequences. The c-optimal formulation turns this distinction into something concrete: a comparison is worth acquiring when it lowers uncertainty along the local gradient of the decision loss, not merely when it resolves an uncertain ranking. Theorem 1 spells out the approximations under which this interpretation holds, and the covariance updates make the criterion practical, without asserting a general guarantee on final decision regret.

  2. The evaluation tests several implications of the proposed mechanism. At B=32, best-of-K regret drops relative to random acquisition from 0.2008 to 0.1470 on EvalPlus and from 0.1737 to 0.1049 on tau-bench. Decision specificity is supported by the operator-mismatch experiment and the analysis of acquired pairs. Importantly, the appendices also study a different feature model, degraded annotators, and a shift from MBPP+ to HumanEval+. The zero-correlation annotator control is particularly valuable: when acquired labels carry no information, DFA's advantage disappears, which shows that the decision set does not by itself solve the preference-learning task.

Questions for authors
  1. The label-efficiency claim leaves out a substantial source of utility supervision that only DFA's acquisition rule can use. In Table 3, decision sets contain 90–200 prompts with utilities for 8–16 candidates per prompt, yet the headline comparison counts only the 32 acquired preference labels. Although the experiments with smaller decision sets are valuable, even 25–50 fully scored prompts may amount to substantial supervision. The present comparisons therefore show a benefit that is conditional on access to this utility set; they do not show an advantage in overall annotation cost. The all-pairs oracle reference does not settle the matter, since it changes both the amount and the source of supervision. To clarify how much of the benefit comes from the acquisition criterion itself, the paper would need a learner that uses only the decision set and methods allowed to train or tune on the same utility information.

  2. Realized coverage in the accept-or-escalate evaluation needs to be accounted for more clearly. Appendix D.5 argues that the escalation term is constant because coverage is fixed at 0.7; however, the threshold is estimated on D and then carried over unchanged to E. This fixes empirical coverage on D, but not necessarily on E. If methods accept different fractions of the evaluation prompts, leaving out the escalation cost could change how they compare. While the selective-regret result remains informative, it does not by itself establish higher cascade utility at matched realized coverage or at a fixed escalation price. Because this concerns one of the three headline decision operators, it should be resolved by reporting acceptance rates and an explicitly defined evaluation denominator.

1 comment

We thank the reviewer for the positive assessment of the paper and for recognizing the strength of aligning the acquisition criterion with downstream decisions. We respond to the questions below.

1. Decision-set information

The label-efficiency claim leaves out a substantial source of utility supervision that only DFA's acquisition rule can use. To clarify how much of the benefit comes from the acquisition criterion itself, the paper would need a learner that uses only the decision set and methods allowed to train or tune on the same utility information.

In this work, we focused on learning a verifier from the acquired data rather than directly from the decision set. However, as the reviewer suggested, we ran additional experiments to compare methods when the verifier is first trained on the decision set and then used as a starting point for each acquisition method. Below, we report regret on EvalPlus and BigCodeBench with a small decision set of |D|=20 and different acquisition budgets, keeping the same evaluation set as in the main paper.

EvalPlus (|D|=20, 50 seeds, lower regret is better)

MethodB=128B=256
D + Random0.168 ± 0.0060.170 ± 0.006
D + Min-margin0.179 ± 0.0060.174 ± 0.006
D + Max-margin0.179 ± 0.0060.160 ± 0.006
D + D-optimal (ActiveDPO)0.186 ± 0.0060.164 ± 0.006
D + DFA (ours)0.151 ± 0.0060.153 ± 0.005

BigCodeBench (|D|=20, 50 seeds, lower regret is better)

MethodB=128B=256
D + Random0.243 ± 0.0060.264 ± 0.008
D + Min-margin0.240 ± 0.0050.253 ± 0.007
D + Max-margin0.250 ± 0.0050.249 ± 0.005
D + D-optimal (ActiveDPO)0.244 ± 0.0050.255 ± 0.004
D + DFA (ours)0.225 ± 0.0050.228 ± 0.005

While the baselines benefit from a verifier trained on the decision set, DFA still outperforms them by using that set to guide which comparisons will most improve downstream decisions. The decision-set-size experiments in Figure 1 in the main paper and Appendix G (Figure 3 and Table 5), and the matched versus mismatched acquisition experiment in Table 2 provide further evidence that the decision set steers acquisition toward the deployed operator.

2. Accept-or-escalate coverage

In the accept-or-escalate setting, what fraction of E does each method actually accept? Do the reported values represent conditional regret over accepted prompts, or regret summed across all evaluation prompts? Does DFA keep its advantage at matched realized coverage or under an explicit escalation cost?

The values in Table 1 for accept-or-escalate are conditional regret on the accepted prompts, rather than regret summed over all evaluation prompts. The following table gives the fraction of E accepted by each method.

Fraction accepted on E (B=32)

MethodUltraFeedbackEvalPlusBigCodeBenchτ-bench
Random0.691 ± 0.0110.699 ± 0.0140.706 ± 0.0090.698 ± 0.016
Min-margin0.680 ± 0.0130.711 ± 0.0110.698 ± 0.0090.722 ± 0.015
Max-margin0.671 ± 0.0150.681 ± 0.0150.697 ± 0.0080.692 ± 0.016
D-optimal0.689 ± 0.0130.687 ± 0.0120.707 ± 0.0090.702 ± 0.017
DFA0.662 ± 0.0140.716 ± 0.0140.713 ± 0.0110.710 ± 0.014

All methods accept between 66% and 73% of evaluation prompts. As the reviewer suggested, we also evaluated regret at exactly 70% realized coverage on E, with acceptance ranked by the learned verifier's confidence.

Selective regret at 70% realized coverage on E (B=32, lower is better)

MethodUltraFeedbackEvalPlusBigCodeBenchτ-bench
Random1.065 ± 0.0490.172 ± 0.0110.222 ± 0.0070.126 ± 0.008
Min-margin1.177 ± 0.0600.197 ± 0.0120.224 ± 0.0090.132 ± 0.011
Max-margin1.134 ± 0.0460.182 ± 0.0100.236 ± 0.0080.125 ± 0.008
D-optimal1.076 ± 0.0540.178 ± 0.0070.228 ± 0.0070.128 ± 0.008
DFA0.686 ± 0.0330.134 ± 0.0070.202 ± 0.0060.085 ± 0.006

DFA retains its advantage at matched eval-set coverage. We will clarify the evaluation protocol and report these results in the revision.

3. Choosing λ\lambda

According to Appendix F, lambda is chosen based on the B=32 mean. What data and metric determine this choice, and how are hyperparameters chosen in the main experiments? Please make clear that evaluation outcomes play no role in model selection.

We run experiments with multiple values of λ\lambda and, for each dataset, choose the one that gives the lowest decision regret on D, which acts as a validation set. Since DFA already uses D for acquisition and has no additional validation set, we use the value lamda selected for the random baseline as default value. No outcomes from E are used in this selection. We will clarify these implementation details in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.