acceptodds
Under review as a conference paper at ICLR 2027

How Much Evaluation Is Enough? Risk-Aware Adaptive Evaluation (RAAE) of LLMs in High-Stakes Financial Applications

Abstract

Large language models are increasingly used in financial applications where evaluation errors can have highly asymmetric consequences: overlooking a minor formatting issue is fundamentally different from missing an unsafe financial recommendation, a privacy violation, a fraud-enabling response, or sensitive-data disclosure. Yet current LLM evaluation pipelines commonly rely on fixed-size test sets and aggregate metrics that treat evaluation examples uniformly. We propose Risk-Aware Adaptive Evaluation (RAAE), a sequential evaluation strategy that allocates LLM-as-a-Judge effort by both statistical uncertainty and domain-specific risk, stopping only once an anytime-valid confidence sequence – constructed correctly for sampling without replacement from each finite risk stratum – is tight enough and every critical risk stratum has received sufficient coverage. On a 267-item, 8-stratum financial evaluation suite with 3 candidate LLMs and 2 independent judges, adaptive sampling shows a real, statistically significant advantage over naive sampling in the low-to-mid evaluation-budget range; that advantage narrows as budget approaches the adaptive methods’ own (necessarily late, since the underlying guarantee is strict) natural stopping point, where all methods converge toward near-exhaustive recall – reported plainly rather than only at the budget most flattering to our own method. RAAE’s specific contribution over plain uncertainty-driven sampling is not a general recall improvement – at a moderate coverage requirement it is statistically indistinguishable, and can even slightly underperform – but a structural capability: it can enforce a pre-declared minimum coverage requirement for critical-risk strata that plain uncertainty-driven sampling has no mechanism to offer at any setting. Achieving complete coverage costs additional evaluation budget, and matched-budget comparisons show no consistent recall advantage for RAAE, confirming that its benefit is the enforceable coverage guarantee itself, not superior sampling efficiency. We further find that judge choice alone changes the detected critical-failure count by more than 2× on this dataset, underscoring that evaluation reliability depends on judge selection as much as on sampling strategy. We provide a formal anytime-valid stopping guarantee grounded in confidencesequence theory, with explicitly stated assumptions and limitations.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM evaluation, LLM-as-a-Judge, adaptive evaluation, risk-aware evaluation, confidence sequences, financial AI, AI safety, sequential evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_32t7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how much evaluation is sufficient for LLM systems in high-risk financial applications. It formulates LLM evaluation as a sequential stopping problem over a finite, stratified evaluation pool and proposes Risk-Aware Adaptive Evaluation (RAAE), a method that combines uncertainty-driven allocation with pre-declared minimum coverage requirements for critical risk strata. The method uses without-replacement confidence sequences and a union-bound construction to provide an anytime-valid guarantee for the population-weighted mean score estimate at stopping time. Empirically, the authors construct a 267-item, 8-stratum financial evaluation suite, evaluate three OpenAI mini-class candidate models with two OpenAI mini-class judges, and compare RAAE against exhaustive, uniform, stratified, and uncertainty-only baselines through resampling from one exhaustive judged pass. The empirical results are nuanced: model ranking fidelity is already saturated near the adaptive methods' natural stopping point; RAAE stops only after evaluating about 90% of the pool, yielding roughly 10% savings; its early-budget critical-failure recall advantage over naive sampling is small and sensitive to the judge-defined labels; and its most robust contribution is enforcing a pre-declared critical-stratum coverage floor rather than consistently improving recall over uncertainty-only sampling.

Critical strengths
  1. The paper addresses a practical limitation of aggregate LLM evaluation: a precise estimate of average performance does not ensure adequate coverage of high-risk categories. Incorporating pre-declared critical-stratum coverage requirements into the stopping rule provides a useful formulation for settings with asymmetric failure costs.

  2. The statistical construction accounts for adaptive allocation across finite strata sampled without replacement. The paper clearly distinguishes its anytime-valid guarantee on population-weighted mean scores from the deterministic coverage requirement and the empirically measured critical-failure recall.

  3. The empirical analysis includes matched-budget comparisons, coverage-threshold ablations, and judge-specific sensitivity analyses. These experiments help separate the benefit of enforcing coverage from improvements in sampling efficiency. The authors report unfavorable results and support the judge-disagreement analysis with a blinded human check, while acknowledging its limited scope.

Critical weaknesses
  1. The practical advantage of RAAE over simpler baselines is limited. At its natural stopping point, RAAE evaluates about 90% of the pool, so the realized cost saving is modest. At the matched budget of 240/267, RAAE does not improve critical-failure recall over uniform, stratified, or uncertainty-only sampling. Its budget-90 advantage over uniform under the primary OR-combined labels is statistically significant but small, corresponding to roughly one additional critical item out of 52 on average, and uncertainty-only sampling performs comparably. Under the gpt-5-mini-only sensitivity analysis, this early-budget advantage largely disappears. Thus the evidence supports RAAE more as an enforceable coverage mechanism than as a generally superior adaptive sampling strategy.

  2. The empirical conclusions depend heavily on judge-defined labels and a limited evaluation setting. All candidate and judge models are OpenAI mini-class models; two judges also appear as candidates; self-judgments are included in the OR-combined critical-failure ground truth; and gpt-4o-mini alone flags the same 52 critical-failure cases as the OR-combined pool while gpt-5-mini alone flags only 23, even though the single-rater human disagreement check mostly sides with gpt-5-mini. The experiment is also an offline resampling study from one exhaustive judged pass on a 267-item author-validated dataset. These design choices do not invalidate the paper, but they make the recall and judge-reliability conclusions difficult to generalize without independent judges, stronger human adjudication, and larger or externally sourced evaluation pools.

Questions for authors
  1. How do the main recall, stopping-budget, and ranking results change if self-judgments are excluded from the OR-combined critical-failure ground truth?

  2. How sensitive are the conclusions to the label aggregation rule? For example, what happens under consensus, majority, or human-adjudicated labels rather than OR-combination?

  3. Can the authors add a non-OpenAI or otherwise more independent judge and report whether RAAE's early-budget advantage survives under that judge or judge ensemble?

  4. Can the judge-reliability analysis be strengthened with multiple independent human raters and a sample that is not concentrated on judge-disagreement cases?

  5. In practical deployment, how should users choose the risk taxonomy and coverage thresholds τ_s, and how should RAAE handle important risk categories that are unknown or emerge after the taxonomy is fixed?

Reviewer_bn1hOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors propose a way to decide which AI answers to check and when to stop checking. The method gives special attention to important risk categories. However, the experiments show little practical improvement over choosing answers at random, and the claimed benefit of minimum coverage is not tested against a simpler method following the same requirement.

Critical strengths
  • The problem matters.
  • The comparison between automatic judges, together with checks by a human reviewer, provides useful evidence about how unreliable the failure labels can be (§§7.2–7.3).
  • The authors also report results that do not favor their method.
Critical weaknesses
  • Figure 1 shows the proposed method following random sampling very closely across the available budgets. After checking 90 items, the reported improvement amounts to approximately one additional detected failure out of 52 (§7.2). Near the stopping point, Table 1 reports that the method finds 89.8% of failures, compared with 89.9% for random sampling. These results do not show a meaningful practical advantage. Small differences being statistically significant across repeated sampling orders does not make those differences useful.

  • The remaining benefit is a rule requiring a minimum number of checks in each important risk category. However, the paper does not compare its method with a simpler sampling procedure following the same rule (§6.3). This leaves the most important comparison missing. The experiments also use the same 267-item set and one complete evaluation pass, repeatedly changing which items are selected and their order (§6.2).

  • The explanation of the method is incomplete. Sections 4.2–4.5 explain why a particular statistical approach is needed, but I could not work out how the uncertainty estimates are actually calculated and updated. These estimates determine both what gets checked next and when checking stops, so this is a central missing explanation. Figure 2 als

Questions for authors
  1. Section 4.1 states that “RAAE has no separate seeding phase,” but Figure 2 begins with “Stratified seed sample (all strata represented).” Which starting procedure was actually used, and how were the uncertainty estimates updated after each check?
Reviewer_48vLOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Fixed-budget, aggregate LLM evaluation sits awkwardly in high-stakes settings, where evidence requirements vary from item to item and a missed critical failure can cost far more than an ordinary error. The paper's response is Risk-Aware Adaptive Evaluation, which allocates LLM-as-a-Judge calls sequentially according to uncertainty and risk and halts only once an anytime-valid without-replacement confidence sequence is narrow enough and every critical stratum clears a pre-declared coverage floor. On a financial evaluation suite the method holds its ranking quality and shows a modest early edge in catching critical failures under the primary judge, but the edge fades under an alternative judge, which leaves the coverage-floor guarantee as the contribution that holds up most consistently.

Critical strengths
  1. The paper is candid about negative, judge-dependent, and corrected findings, which makes its stated contribution easier to assess.
  2. The paper combines a clear methodological proposal with a formal guarantee, making the intended safety property explicit rather than relying only on empirical performance.
Critical weaknesses
  1. The paper's practical efficiency advantage is modest and is not robust to the choice of evaluation judge.
  2. The empirical evidence is limited to a small, domain-specific financial suite and relies on simulated sequential evaluation rather than live deployment.
  3. Section 6.2 evaluates all methods by resampling one exhaustive set of 1,602 judge-scored responses, so the paper never measures online stopping, adaptive-call overhead, or realized stopping-time variability on its own. The authors acknowledge this design in Appendix G, but a live sequential evaluation would be needed to back the practical cost claim.
  4. The validation uses one rater on 60 pairs, with 30 selected because the judges disagreed, and initially omits the rubric/expected-behavior field. This is informative diagnostic evidence, but it does not establish full-pool precision/recall or inter-rater reliability. The generalization of the judge-quality conclusion therefore remains uncertain.
Questions for authors

see weakness section

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.