acceptodds
Under review as a conference paper at ICLR 2027

Scalable Delphi: Large Language Models for Structured Risk Estimation

Abstract

Quantitative risk assessment relies on structured expert elicitation to estimate unobservable properties. The Delphi method produces calibrated, auditable estimates but requires months of coordination and specialist time, placing rigorous risk assessment out of reach for most applications. We propose Scalable Delphi, adapting the classical protocol for LLMs with diverse expert personas, iterative refinement, and rationale sharing. Beyond lowering cost, this makes the assessment analyzable and dynamic. Rationales and revision histories record what each estimate rests on, information can be ablated to test which evidence matters, and the elicitation can be rerun with new evidence, changed assumptions, or adverse scenarios. Because target quantities are unobservable by construction, we design an evaluation framework based on necessary conditions any reliable estimator must satisfy: accuracy and calibration on verifiable proxies, and sensitivity to evidence. Agreement with expert panels and reasoning quality serve as corroboration. Across two domains (AI-augmented cybersecurity risk, ice-sheet contribution to sea-level rise), three benchmarks, and three reproduced expert studies, the estimates pass these tests: they improve systematically as evidence is added, agree with expert panels on most quantities, and correlate strongly with ground truth (Pearson r=0.91-0.98).

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, expert elicitation, risk estimation, Delphi, AI risk

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_a4LFOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an approach to scaling the Delphi method with the use of LLMs. The Delphi method is the established methodology for high-stakes decisions when they cannot be answered via direct measurement. This entails experts providing probability estimates with written rationales, which are iteratively revised until convergence. This process is both expensive and time-consuming, The proposed framework swaps out experts for LLMs because they can provide probably estimates and be queried repeatedly for modified conditions and/or assumptions.

The provided results suggest that Scalable Delphi is a valid framework, with LLM probability estimates exhibiting similar trends to the experts on the included benchmarks. There are obviously differences but the LLMs appear to agree within the bounds of the differences across experts.

Critical strengths

Firstly, the selected problem seems to be an ideal setting for the use of LLMs. The primary challenges with Delphi are the cost, both in resources and time, of engaging with experts over multiple rounds until convergence. This presents a fertile ground for the use of LLMs due to their relative cost-effectiveness and speed. Additionally, the ability to repeatedly query them as conditions shift offer an additional advantage over the use of experts in Delphi. The presented results substantiate the case for Scalable Delphi being a viable alternative to Delphi; further testing in real settings is probably necessary but the included benchmarks make a fairly compelling case.

I also greatly appreciate the authors' writing. The manuscript is clearly structured, with notation clearly explained and used to clarify meaning. Even as a reader relatively new to the domain of expert elicitation, i was able to follow the reasoning and explanations. The experiments and benchmarks have also been structured to paint a clear picture behind the design choices made and validity of the proposed method.

Including multiple benchmarks was a nice touch too as it exemplifies that Scalable Delphi was not designed with a specific domain in mind. Delinating the limitations of the alternate benchmark was useful too. This further strengthened the case for Scalable Delphi with the appropriate caveats for readers and practitioners when viewing the results,

Critical weaknesses

The manuscript would benefit from a more detailed discussion of the related works. At present, this has been included in Section 6 and reads almost like an afterthought. However, this discussion is important to position this work relative to existing research.

I have also included some pointers and questions in the next section.

Questions for authors
  1. On line 76, the authors state that "the relevant quantities can be elicited again". While I understand how this is useful, especially relative to experts, does this not introduce the risk that repeated queries may lead to a "shift" in the LLM outputs? This could be problematic for how well-calibrated the model probability estimates are.
  2. The frontier models used are relatively old (GPT 5.1 AND Claude Opus 4.1). Why were these chosen? And are there any reasons to expect Scalable Delphi to not work with newer models?
  3. It would be useful to define ECE in the main text, instead of the appendix because it is one of the key metrics. Additionally, contextualizing what good values for MAE and ECE in Table 1 would be useful. Currently, it is unclear whether a MAE of 4.5 is any good.
2 comments

Thank you for the careful review and for the follow-up comment on positioning. We have reworked the related work accordingly (W1), report run-to-run variation for re-elicitation (Q1), and moved the ECE definition into the main text with context for the reported values (Q3).

W1. Related work

We agree that the distinction from prior LLM-based Delphi work deserves more prominence than a closing related-work section gives it. In the revision, related work follows the introduction, is expanded, and states the distinction explicitly (see the general response).

Q1. Does re-eliciting risk shifting the outputs, and with it their calibration?

Re-elicitation is a fresh run, so the only source of change is a deliberately changed input. Each rerun starts from an empty context, without memory of earlier answers (S3.2; App. E). Keeping all other parameters fixed (model, settings, questions) therefore allows running a fresh study without contamination from prior runs. The protocol is meant to move estimates when the evidence changes and leave them in place otherwise: withholding agent and benchmark identities degrades correlation to noise, and supplying them restores it (S5.3). Variation between runs with the same evidence is small: rerunning an identical Cybench configuration gives r = 0.967 and 0.983 (MAE 4.2 and 3.8 pp) for GLM-5, and r = 0.738 and 0.737 (MAE 10.8 and 11.0 pp) for Llama-4 Maverick. Unlike a reconvened human panel, a fresh run also does not anchor on earlier answers (S5.4, S6).

Q2. Why GPT-5.1 and Opus 4.1, and should newer models work?

They were chosen for their knowledge cutoffs (September 2024 and January 2025; S5.1), which predate both human studies (Murray et al., March 2025; Barrett et al., December 2025) and two of the three benchmarks (BountyBench, CyberGym). Validating against these requires models that cannot have seen them. Newer models are likely to have both the studies and the leaderboards in their training data. Nothing in the method ties it to these two models, and we have no reason to expect newer models to perform any worse. GLM-5 and Kimi K2.5, released in 2026, perform on par with them (Table 1), and the only model that falls behind on Cybench is the one without extended reasoning (Llama-4 Maverick, S5.2). For a new model, the proxy tasks can be rerun on benchmarks released after its cutoff before it is used.

The contamination risk for newer models is real. In an additional check, we tested two state-of-the-art models, GPT-6.1-Sol and Claude Opus 5.5. They reach r = 0.96–0.99 in the full-information setting, but Opus 5.5’s reasoning cites recalled leaderboard figures. The paper’s models show no such recall (App. G).

Q3. ECE in the main text; what are good values for MAE and ECE?

We have moved the ECE definition into S5.1 and added the following context to S5.2. Thank you for the suggestion!

  • Baselines and human disagreement. Predicting the mean of the other benchmark agents gives 12.3, 20.0 and 16.7 pp (S5.2); the two human panels in Murray et al. differ by 16.6 pp (S5.4).
  • ECE is within sampling noise on BountyBench and Cybench. A simulated, perfectly calibrated panel produces ECE up to 3.7–4.8 pp on these benchmarks, and every model falls inside that range (App. B).
Reviewer_a4LFOpenReview ↗

Thanks for onboarding reviewer comments

I thank the reviewers for onboarding the concerns raised by the reviewers. The clearer positioning should certainly help distinguish this work relative to existing research in this field.

I maintain that this manuscript is worthy of acceptance and specifically, that the revisions have gone a long way in addressing the concerns raised by Reviewer WAXg.

Reviewer_WAXgOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes an extension of the standard Delphi method for risk assessment by replacing human-experts by LLMs agent with diverse expert personas, iterative refinement and rationale sharing, These agents operate under diverse domain personas, share anonymous rationales via a neutral mediator, and engage in multi-round iterative refinement. The authors also suggested appropriate proxy to estimate ground truth. The newly Scalable Delphi method has been tested across three benchmarks (BountyBench, Cybench, CyberGym) and three reproduced expert studies, its estimates are found to be accurate, calibrated, and aligned with expert judgement (r = 0.91 to 0.98), proving an appealing and dynamic estimation of risk assessment that can be traceable back to LLM reasoning as well, demonstrating increased transparency.

Critical strengths
  • Although not fully novel the idea of using LLMs as expert in Delphi is interesting and technically very sound, together with new estimation of ground truth based on calibration, evidence sensitivity, and alignment with human expert panels.
  • Strong empirical evaluation across three benchmarks demonstrating strong domain transfer across structurally distinct fields: AI-augmented cybersecurity risk and cryospheric sea-level rise projections, while combining proxy task calibration, systematic evidence ablation, and human-to-LLM panel comparisons.
Critical weaknesses
  • The novelty is very missing, and the state-of-the-art is quite incomplete, see comments in the question for authors
  • There are other known vulnerabilities of LLMs in long horizon when dealing with open-ended count quantities, error amplification in multi.-interactions, positional bias, among others, which are not discussed in the discussion part, nor limitation part. . The mediator feedback loop can create artificial consensus when ignoring minority opinions and narrowing the variance.
Questions for authors
  • As there is already active research on LLM-based Delphi method, see, e.g., https://arxiv.org/pdf/2502.21092, https://ieeexplore.ieee.org/document/11357620, https://www.sciencedirect.com/science/article/abs/pii/S0016328726001722, among others, where the key ideas brought by the papers, such as use of LLMs for diverse expert personas, have to a large extend already been implemented. The state art should have been devised to distinguish authors' contributions with respect to this existing research. In addition there is also a large body of literature on LLM multiagent architectures, multi-agent debates, LLM-as-a-judge, which support the idea of replacing experts by LLMs. The authors should at least show a clear knowledge of this relevant body of literature.
  • It will be interesting to address the variance reduction caused by the mediator feedback, possibly by designing a more elaborated frameworks that exploit the results of individual rounds.
  • How to handle long-term projections, which are prone to errors by the LLM agents
4 comments
Reviewer_a4LFOpenReview ↗

Need to position paper relative to existing research

I urge the authors to delinate how this work is placed relative to existing research. Existing works that may have significant overlap include:

Also, I do not believe having a version of the manuscript uploaded to a preprint server is a violation of the ethics policy. in fact, I would put to Reviewer WAXg that this is in fact common practice with many submissions to ML conferences.

AuthorsOpenReview ↗

Author response (1/2)

Thank you for your time in reviewing our work, for updating the review, and for pointing out the additional references. We have included them in our revised related work section.

W1, Q1. What does the paper contribute beyond existing LLM-based Delphi work, and is the related literature covered?

The paper does not claim persona-based LLM Delphi as new. The Delphi protocol was developed in the 1950s. Our main contribution is the validation of LLM elicitation for latent quantities, which lack ground truth: a framework of necessary conditions (S4) and the evidence that LLM panels meet it (S5). The second is what this changes for risk assessment: a replication ran in 5–24 minutes (App. I), and an elicitation can be ablated (S5.3), audited through its reasoning traces (App. F.2), and rerun when evidence changes (App. E). Section 6 of the original submission (now Section 2) already reviews structured expert elicitation, LLM-based Delphi (Nóbrega et al., 2023; Mueller et al., 2024; Papakonstantinou et al., 2025; Barrett et al., 2025), multi-agent debate and negotiation (Du et al., 2024; Liang et al., 2024; Abdelnabi et al., 2024), and LLM forecasting and calibration (Halawi et al., 2024; Schoenegger et al., 2024; Kadavath et al., 2022; Tian et al., 2023), and Section 5 (now Section 6) relates our findings to LLM simulation of survey respondents (Bisbee et al., 2024).

We agree that the distinction from this line of work deserves more prominence than the current related-work section gives it. In the revision, we moved related work to Section 2, after the introduction, expanded it, and state the distinctions explicitly.

Thank you for the additional references, which we have added:

  • Şahin (Futures, 2026) has LLMs rate 15 foresight statements for agreement, and names the absence of a human panel, benchmark comparisons and backtesting as its primary limitation. It was published online on 16 September 2026, the ICLR deadline, and is contemporaneous under ICLR policy.
  • Bertolotti & Mari (arXiv, 2025) run a Delphi with one LLM (gpt-4o-mini) for qualitative foresight. Its only check is repeating the run three times, with no external validation.
  • Nagumo et al. (IEEM, 2025) lets consumer personas rate three snack ideas. The ratings are not validated against any external reference: the analysis covers only how the personas' criteria and scores change across rounds.

All three produce foresight consensus or idea ratings. None validates estimates, tests sensitivity to evidence, or compares with human expert elicitations. These are what our paper contributes: 73 held-out cells across three benchmarks with known answers, five base models, evidence ablation under four information conditions, 79 quantities from three published expert elicitations, and a comparison of 1,862 human and LLM rationales on 11 reasoning patterns (S5.5). Beyond this validation, the paper shows its impact on risk modeling: assessments that can be ablated, audited through their reasoning traces, and rerun when evidence changes, as set out above. We have also added an LLM-as-a-judge reference where S5.5 uses an LLM judge to label rationales.

AuthorsOpenReview ↗

Author response (2/2)

W2, Q3. Are the weaknesses of LLMs on open-ended counts and long horizons discussed, and how should such estimates be handled?

We disagree that these settings are not discussed: the paper addresses them explicitly in the introduction (S1, l.94–95), in the results for counts (S5.4, l.426–429) and long horizons (l.465–467), in a dedicated analysis (App. F.3), and in S6 (l.509–511), where they are named as the estimates that deserve the most scrutiny.

These are the settings where uncertainty is highest for any estimator, human or LLM: estimates far from known reference values carry the uncertainty of the underlying problem. The desirable property is therefore that an estimator's uncertainty grows there, and this is what we observe: dispersion increases with horizon and warming in all three Bamber panels, human and LLM alike (S5.4, l.465–467), and intra-panel variance grows with extrapolation across all three studies (S5.4, l.473–477). At 2300 H the human panel's own 5–95% interval spans −520 to 13,000 mm, and both LLM medians (1,250 and 3,320 mm) lie inside it (Fig. 5). Divergence there is disagreement between panels, not a measured error: there is no ground truth, and we treat human estimates as a reference (S4; S5.4, Trends).

To handle such estimates (Q3), the protocol offers three tools:

  1. Detect them. Three signals flag these estimates within a run. Stated confidence falls with horizon: at 2300 H every GPT-5.1 and Opus-4.1 panelist rates its confidence “low”, against 0–7% at 2050 H. Intra-panel spread grows with extrapolation: on the count tasks, the first-round coefficient of variation rises from 0.06 to 0.20 (GPT-5.1) and from 0.14 to 0.45 (Opus-4.1) between tiers 1 and 5. And panels disagree: 6.5× on the number of actors at tier 5 (App. F.3) and 2.6× at 2300 H (S5.4).
  2. Condition them. These are conditional quantities, not forecasts (S1, l.66–72), elicited under explicit assumptions such as LLM capabilities or the warming trajectory. The assumptions can be varied and the quantity re-elicited (S3.2).
  3. Update them. The elicitation can be rerun with new evidence: adding post-2018 observations shifts the medians by 11–28% (S5.4; App. E).

W2, Q2. Does the mediator's feedback create artificial consensus, and are error amplification and positional bias addressed?

Minority opinions are surfaced by design, not ignored. As in classical Delphi, the mediator reports key arguments for higher and lower values and areas of disagreement (S3.2). In the ransom case (App. F.2) it flagged the outlier explicitly ("one expert estimates notably higher"). Convergence across rounds is the intended effect of Delphi feedback and occurs in human panels too (S5.5).

On exploiting individual rounds (Q2): the protocol already does, because the rounds do different jobs. Round 1 collects independent judgments, so its spread measures how much panelists disagree (S6). Mediation then lets each panelist respond to arguments it had not considered, which is where Delphi's gains in accuracy come from (Rowe & Wright, 1999). Mediation also surfaces cruxes: in the ransom case, the second round is where a panelist named the two competing readings of the question (App. F.2). All three human studies we replicate use two rounds, so the comparison requires them (S5.4).

Error amplification across rounds. Round-1 estimates are produced independently, so no panelist can influence another before the mediator’s summary, and whether interaction history carries across related quantities is an explicit design choice (S3.2). Influence between rounds is the intended mechanism of Delphi: panelists revise in light of each other’s arguments, and this exchange is also what exposes errors, as in the ransom case (App. F.2). Influence runs only through the mediator’s anonymized summary, which attributes no view to a panelist and recommends no value (S3.2; App. H).

Positional bias. In the benchmark prompts, agents are listed in a fixed random order rather than sorted by score, so an agent’s position carries no information about its success rate (now stated in S5.2).

AuthorsOpenReview ↗

Reply to reviewer a4LF follow-up

Thank you for the follow-up. Our general response discusses each of these references individually and the revised Section 2 covers them in the paper (changes in blue).

In short: what they share with our work is the Delphi protocol, which we do not claim as novel. DeLLMphi targets event forecasting, Bertolotti & Mari qualitative foresight, and Nagumo et al. idea screening. None of them tests whether LLM-elicited estimates of latent quantities can be trusted. That validation, and what it enables for risk modeling, are our main contributions.

Please let us know if any questions remain, we are happy to clarify.

Reviewer_HmWuOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks how to estimate latent quantities (e.g. probability that an AI agent could meaningfully uplift a cyberattack, ice-sheet contribution to sea-level rise by 2100, etc) which affect decisions today but have no ground truth to check against.

Its proposed instrument is a Delphi-style panel run entirely by LLMs. Roughly five agents, each given a domain persona, independently estimate the quantity in question. An LLM mediator then synthesizes their views anonymously, the agents revise their estimates over several rounds, and the results are aggregated into a final answer. This can be seen as a way to make expert elicitation cheap, fast, and repeatable. It is also auditable, because every estimate carries a reasoning trace, and the whole panel can be re-run whenever the evidence changes.

The more interesting contribution is how the authors propose to check an answer that can never be observed. They argue you cannot verify such a quantity directly, so they test it in three indirect ways. First, they check accuracy on problems where the answer is known, by predicting how well AI systems score on the cybersecurity benchmarks BountyBench, Cybench, and CyberGym. Second, they check whether the estimates actually move when evidence is added or taken away. Third, they check whether the panel lands close to published human expert panels.

The results are split. The panel's central estimates are good, but its sense of how sure it should be is not. On the problems with known answers, the estimates track the truth closely. GPT-5.1 reaches a correlation of 0.97 on Cybench, with small average error. The estimates also move the right way with evidence, falling to near-zero correlation when the evidence is taken away and recovering when it is put back. And on the three replicated human studies (Murray et al. 2025, Barrett et al. 2025, Bamber et al. 2019), the panel lands inside the range that the human panels themselves disagree over. The weak spot is uncertainty. The panel's plausibility ranges are too narrow. They contain the true answer only 39–80% of the time, when you would want about 90%, and the mediation step narrows them further.

Critical strengths
  1. The paper offers a careful way to test answers that can never be checked. The real contribution is the method for validating an estimate that has no ground truth. The authors put together four checks: predicting known answers on the cybersecurity benchmarks, seeing whether the estimates move when evidence is added or removed, comparing the panel against published human panels, and re-running the panel when the assumptions change. None of these checks is new on its own, but assembling them for quantities with no ground truth, and spelling out why each one is needed, is a real service to this area and is the paper's strongest claim.

  2. It is tested against real human studies, and the gap is measured. The authors replicate three well-known human studies across two fields and report how far off the panel is in actual numbers. This includes moving to a new field (ice-sheet estimates) and showing that adding newer satellite data shifts the panel's medians down by 11–28%. Checking against published human panels and reporting the difference is stronger support than earlier LLM-Delphi work has offered.

  3. The method is shown to be auditable, and the authors are honest about where it fails. When the panel badly overcounted ransomware threat actors, the authors traced the error to an unclear way the question had been framed, fixed it, and re-ran. This is exactly the "you can inspect the reasoning" claim the method promises, actually paying off. They are also candid about the failure modes (poor calibration, trouble with counts, and the base model mattering more than the persona), which is itself useful guidance for the field.

Critical weaknesses
  1. The "diverse expert panel" idea does not hold up under the paper's own tests. Every member of a panel is the same base model wearing a different persona, and the authors never mix different models in one panel. Their own numbers show the base model moves estimates about 4.8 points while the persona moves them only about 2.4 (and just 0.5 after mediation). So this is closer to one expert in five costumes than to five independent experts, something the authors half-admit when they suggest using disagreement between different models as the better measure of uncertainty. Other recent work reportedly already runs panels that mix different model families, so this paper skips the very step that would justify the "panel" label.

  2. The point estimates are accurate, but the uncertainty is not. The impressive headline numbers are really about accuracy. The separate check of how sure the panel should be fails. Its ranges cover the true answer far less often than they should, and the mediation step makes them even narrower. For a method sold for risk estimation, the range often matters more than where its center is. This is a first-order problem that should be stated up front, not mentioned late.

  3. Doing well on known answers does not prove the method works on the unknowable ones, and the method itself is thin. Being accurate on benchmarks with known answers does not guarantee the method works on the unobservable quantities it is actually meant for. The test task, which is to rank a new system among known systems given a description of it, is clearly easier than estimating a latent quantity cold, and the correlations rest on small samples (20–23 systems per benchmark). On top of this, the method itself is largely a repackaging of existing multi-round, persona-plus-mediator LLM-Delphi work. The closest prior art, DeLLMphi (Williams et al.), is not cited, despite using essentially the same architecture: a panel of persona-conditioned LLM agents, an LLM mediator, and iterative rounds emulating the Delphi protocol. Omitting it lets the paper overstate how new its machinery is. The contribution should be framed clearly as an evaluation, not a new method.

Questions for authors
  1. How does Scalable Delphi differ from existing multi-round, persona-plus-mediator LLM-Delphi work, in particular DeLLMphi? Once that lineage is on the table, what in the apparatus is genuinely new rather than a reimplementation at larger scale?

  2. Your own ablation shows the base model moves estimates about twice as much as the persona, and the persona effect nearly vanishes after mediation. Given that, and given you never run a mixed-model panel, in what sense is this a panel of diverse experts rather than one model in five costumes? Would heterogeneous, mixed-model panels not be the natural test, as some recent work already does?

  3. If the method is largely inherited, is the intended contribution the evaluation framework itself? If so, does that framework alone clear the bar you want the paper judged against, and can you say so plainly in the paper?

  4. Plausibility intervals cover the truth only 39 to 80 percent of the time where you want about 90, and mediation narrows the spread further. Should the final consensus interval be reported as the method's uncertainty at all, given you yourself recommend using first-round and between-model spread instead?

  5. On the benchmarks the held-out unit is a whole system's aggregate score, with only 20 to 23 points per benchmark, and the panel sees all the other systems' scores. That is closer to ranking a new model among known ones than to estimating a quantity cold. What evidence is there that calibration on this proxy transfers to the genuinely unobservable quantities the method is sold for?

  6. For the cybersecurity benchmarks, how do you rule out that the models partly recall published results rather than estimating them? Do the benchmarks postdate the training cutoffs, or are they otherwise held out?

3 comments
AuthorsOpenReview ↗

Author response (1/2)

Thank you for the specific and constructive review. We ran the mixed-model panels you suggest (Q2), measured panelists' stated confidence against their hit rates (Q4), and have clarified the contribution in the introduction (Q3).

Q1. How does Scalable Delphi differ from DeLLMphi, and what in the apparatus is new?

The round structure is Delphi's by design, and the paper does not claim it (see the general response). We agree that this positioning deserves more prominence and have expanded the related work and moved it to follow the introduction. Thank you for the pointer to DeLLMphi (MTI-LLM @ NeurIPS 2025), which we have added. It is a forecasting method, evaluated by Brier score on 35 resolved ForecastBench questions with one base model per panel and compared only against aggregate human forecaster medians.

Our apparatus has a different target: latent conditional quantities in risk models, which never resolve (S3.1). What is new is that elicitation becomes a controllable instrument whose evidence, assumptions, question wording, base model and context are explicit, separable inputs (S3.2). This is what makes the information ablations (S5.3), the trace audit of the ransom study (App. F.2) and the re-elicitation with new evidence (App. E) possible; DeLLMphi lists evidence-importance analysis as future work.

Q2. In what sense is this a panel of diverse experts? Would mixed-model panels not be the natural test?

We ran the mixed-model test you suggest. Results are on par with single-model panels. We construct mixed panels with one persona on each of the five base models (GPT-5.1, Opus-4.1, GLM-5, Kimi K2.5, Llama-4 Maverick) for all three proxy benchmarks. Averaged over all 120 persona-to-model assignments, they reach r = 0.96, 0.98 and 0.93 (MAE 5.9, 4.8 and 5.1 pp) and ECE 2.5, 3.8 and 2.5 pp on BountyBench, Cybench and CyberGym, with SD of r ≤ 0.01 across assignments. We added these results to Table 1.

The persona ablation in App. A.1 shows the same: replacing all five personas with an identical "superforecaster" persona shifts the estimates by only 1.9–3.0 pp. The estimates are robust to changing both model and personas, which is what a risk-estimation method needs. In Delphi, the panel's role is to produce reliable aggregate estimates and to surface the arguments behind them. The personas do the second: in the ransom task, one persona per panel read the quantity as conditional on payment while the other four did not (\560kvs.560k vs. \\180k at tier 1), and this disagreement revealed the ambiguity in the question (App. F.2).

Q3. Is the contribution the evaluation framework, and does it clear the bar?

Yes, as set out in the general response, it is the primary contribution, and we have clarified this in the introduction. It clears the bar because, for risk estimation, the method cannot be used without it. Running the Delphi protocol with LLM panelists is straightforward, as the prior work shows. Establishing whether its estimates can be trusted is not. The target quantities never resolve, so validity has to be built from indirect tests, each designed and assembled for this paper: proxy tasks with verifiable answers and contamination controls (S5.2, App. G), information ablations (S5.3), and protocol-matched replications of published elicitations, using per-expert estimates and rationales provided by the original study authors (S5.4). Without this evidence, LLM-elicited risk estimates cannot be used. Our second contribution is showing how validated LLM elicitation changes what is possible in risk assessment. Each replication ran in 5–24 minutes for under \$80 of API cost (App. I), compared to months for a human panel. Because all inputs are explicit, an assessment can be analyzed by ablating evidence (S5.3), audited through its reasoning traces, which located a framing error in the ransom study (App. F.2), and rerun when new evidence becomes available (App. E).

AuthorsOpenReview ↗

Author response (2/2)

Q4. Should the final consensus interval be reported as the method's uncertainty at all?

For comparison with human panels, yes, because that is what the human studies report; as the method's own uncertainty estimate, S6 recommends first-round spread and disagreement between base models instead. For the 39–80% figure, 90% is not the right reference: following the four-point format of the IDEA protocol (Hanea et al., 2017), used in the Barrett et al. (2025) study we replicate, panelists state their confidence that their range contains the true value, and a range is calibrated if its hit rate matches that confidence. Pooled over the benchmarks, stated confidence averages 73% (GPT-5.1) and 75% (Opus-4.1) against hit rates of 65% and 69%, so panelists are overconfident. Human experts using the same format are overconfident as well (a 68% hit rate for 80% intervals in Speirs-Bridge et al., 2010). Structured expert judgment therefore does not take stated ranges at face value but scores experts on calibration questions with known answers (Cooke, 1991), the role our benchmark tasks play.

What the paper claims as calibrated is the estimates themselves: ECE is 1.6–4.1 pp, inside the sampling-noise null on BountyBench and Cybench, and Brier reliability is 0.006 against resolution 0.040 (App. B).

Q5. Does calibration on the proxy transfer to genuinely unobservable quantities?

Transfer is tested directly on latent quantities (S5.4). The proxy is a necessary condition (S4), the role calibration questions play in structured expert judgment. The proxy is not a ranking task: in the "none" condition the panel sees the same scores of the other systems, yet r = −0.22 (Cybench) and −0.40 (CyberGym). With benchmark and agent information given, correlation rises to 0.97 and 0.92 (S5.3). Elicitation is typically not cold: Delphi experts work from briefing materials, and the studies we replicate supply reference values such as pre-LLM baselines, which the proxy mirrors. On the latent quantities themselves, panels agree with human experts within the range humans disagree among themselves in three studies. On sample size: with 30, 20 and 23 held-out cells, the correlations are still well determined. Their 95% cell-bootstrap intervals have lower bounds of at least 0.91 (BountyBench), 0.92 (Cybench) and 0.85 (CyberGym) for both frontier models, and the same cells give r = −0.40 to 0.30 when agent identities are withheld (S5.3), so the sample size alone does not produce high correlations.

Q6. How do you rule out that the models recall published results?

Three checks (S5.1, S5.3, App. G). (1) BountyBench and CyberGym were released after both frontier models' knowledge cutoffs. (2) For Cybench, whose early entries predate them, the models cannot reproduce the published scores from memory (App. G), and restricting to agents published after each cutoff leaves the error within its confidence intervals (App. G). (3) Most of the gain in correlation comes from showing agent names without naming the benchmark (S5.3), and the rationales in that condition reason about relative capability (App. C), not about remembered leaderboard entries.

Reviewer_HmWuOpenReview ↗

Acknowledged

Thank you for your response and the additional experiments. I maintain my positive score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response (1/2): contribution and positioning

We thank the reviewers for their careful and constructive reviews. They agree on the strength of the evaluation: HmWu calls the validation of estimates without ground truth "a real service to this area" and the comparison with published human panels "stronger support than earlier LLM-Delphi work has offered"; WAXg finds the approach "technically very sound"; a4LF finds that the benchmarks "make a fairly compelling case". The shared request is sharper positioning against prior LLM-based Delphi work (HmWu Q1 and Q3, WAXg W1, a4LF), which we address here. Individual questions are answered in each reviewer's thread.

Contribution and positioning. Delphi is the established method for eliciting expert estimates of quantities that cannot be measured, and risk models in high-stakes domains depend on it (S1). Its cost and slow turnaround are well known, and replacing human panelists with LLMs is a natural response that several groups have explored: for foresight and idea screening (Nóbrega et al., 2023; Bertolotti & Mari, 2025; Nagumo et al., 2025; Şahin, 2026), scenario ratings (Mueller et al., 2024), risk identification (Papakonstantinou et al., 2025), cyber risk models (Barrett et al., 2025), and event forecasting (Williams et al., 2025). The protocol is therefore not our contribution; Scalable Delphi names the agent instantiation we validate. The submission's related work (original S6) already discusses four of these prior studies (Nóbrega et al., Mueller et al., Papakonstantinou et al., Barrett et al.) and positions our contribution as validation (original l.477–478). The revision adds the four that the reviewers raise and moves the section to follow the introduction.

What these prior studies have not established is whether the estimates can be trusted. For risk estimation this is the deciding question: the target quantities never resolve, so an estimate that cannot be validated cannot responsibly inform a decision. For such quantities, prior studies check their outputs, if at all, against human judgments: Mueller et al. (2024) find moderate correlation and positive bias on scenario ratings, and Barrett et al. (2025) close agreement on probabilities but lower estimates on quantities. This comparison cannot establish trust on its own. The human estimates are not ground truth either, and where the two disagree, it cannot tell which is right.

Our main contribution is a validation across multiple axes that goes beyond human agreement alone (S4). Its necessary conditions are tests any reliable estimator must pass: accuracy on verifiable proxies and sensitivity to evidence. Its corroborating evidence is agreement with human experts across multiple studies, and the quality of the reasoning:

  • Accuracy on verifiable proxies. On 73 held-out cells across three benchmarks, both frontier models reach r = 0.91–0.98 with MAE 4.5–6.1 pp, against 12–20 pp for baselines, and ECE 1.6–4.1 pp (S5.2). Two of three open-weight models match them, and the results survive contamination checks (App. G).
  • Sensitivity to evidence. With agent identities withheld, correlation falls to r = −0.40 to 0.30; with benchmark and agent information, it reaches 0.71–0.97 (S5.3).
  • Agreement with human experts. On 79 quantities from three published elicitations in two domains, with matched protocols, estimates fall within the range over which the human panels disagree (e.g. MAD 5.0 pp to a human panel, against 16.6 pp between the two human panels; S5.4). Their uncertainty grows where the human panels' does.
  • Reasoning quality. An LLM judge labels 1,862 human and LLM rationales from the Barrett study on 11 reasoning patterns. The LLM panels invoke the same constraints as the human panel at broadly comparable rates (technical and non-technical ceilings, baseline anchors, attacker-side framing), within Opus 8.8 pp to the human rates on average (S5.5; App. F). Where panels disagree, the rationales locate the source, as for the ransom quantity (App. F.2).

Second, the paper shows what validated LLM elicitation changes for risk modeling. Each replication ran in 5–24 minutes for under $80 of API cost (App. I), against months for a human panel. Because every input is explicit, an assessment becomes analyzable: evidence can be ablated (S5.3), and reasoning traces can be audited, which located a framing error in the ransom study (App. F.2). It also becomes dynamic: it can be rerun when evidence changes, as with post-2018 observations in the sea-level study (App. E).

The works pointed out by the reviewers. Each of the four applies the protocol to a different purpose than ours, and none tests whether LLM-elicited estimates of latent quantities can be trusted. The revised Section 2 discusses each of them.

  • DeLLMphi (Williams et al., MTI-LLM @ NeurIPS 2025) targets event forecasting. Its questions resolve, so it can be scored against outcomes. Risk quantities offer no such route, and DeLLMphi does not address them. Its evaluation is the Brier score on 35 ForecastBench questions with one base model per panel, where it stays behind the superforecaster median (0.157 vs. 0.136). It does not vary the evidence given to the panel, and names evidence-importance analysis as future work.
  • Bertolotti & Mari (arXiv, 2025) is a qualitative foresight study: personas of a single model produce themes on how generative AI will evolve. The study has no comparison with human experts or ground truth, states that the validity of the consensus was not considered, and describes itself as a preliminary experiment.
  • Nagumo et al. (IEEM 2025) is an idea-screening study: 15 consumer personas rate three snack product ideas on a 1–5 scale. These are preference scores without a correct value, and the study has no comparison with human raters. Its scores rise with every round (from 3.7–3.8 to 4.2–4.5), which the authors attribute to the personas influencing one another.
  • Şahin (Futures, 2026) is a technology foresight study: ten LLMs rate 15 statements about telecommunications to 2050 for agreement, and 14 reach consensus in the first rating round. The author names the absence of a human panel, benchmark comparisons and backtesting as the study's primary limitation, and notes that the consensus may reflect shared training data.

A note on timing, for completeness: Under the ICLR policy, Şahin (2026) is contemporaneous work: it was published online on 16 September 2026, right at the ICLR deadline. DeLLMphi and Bertolotti & Mari have no version in archival peer-reviewed proceedings (a NeurIPS workshop and arXiv). This did not affect how we treat them, and we discuss and cite all four in the revision.

Changes to the paper, marked in blue in the revised PDF. Section and line numbers our replies refer to the revised PDF.

  • Related work expanded and moved to follow the introduction, including the works the reviewers point to, and a reference added for the LLM judge in S5.5 (all reviewers; WAXg for the LLM judge).
  • Contributions sharpened in the introduction (HmWu Q3).
  • Mixed-model panels added to Table 1 (HmWu Q2).
  • Panelists' stated confidence and hit rates reported in S5.2 (HmWu Q4).
  • ECE defined in S5.1, with context for the MAE and ECE values in S5.2 (a4LF Q3).
  • Protocol controls stated in S3.2, and the fixed shuffled order of agents in the benchmark prompts in S5.2 (WAXg W2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.