acceptodds
Under review as a conference paper at ICLR 2027

FIELD: A Prospective Cross-Sport Benchmark for Language-Model Forecasting and Information Use

12 views

Abstract

Forecasting unresolved events requires models to use evolving evidence, express uncertainty, and update beliefs, yet how observable information use and belief updating relate to forecast quality remains unclear. Sports provide a natural testbed for studying these relationships, with scheduled events, verifiable outcomes, and meaningful intermediate states. We introduce FIELD, a dynamic, prospective cross-sport benchmark for early and near-event forecasting of how sporting events unfold and end. FIELD freezes forecasts and observable workflow records before each event, enabling assessment of both predictive quality and information use. We evaluate seven frontier language-model systems on 232 football, basketball, and baseball events. Across sports, forecast performance is strongest for final Outcome, while richer score distributions and trajectories remain harder. We further find that better trajectory forecasts are associated with better outcome forecasts; observable workflow does not reliably track forecast quality; and later forecasts mostly refine probabilities rather than change the leading prediction. A separate STEP study prospectively tests whether these findings can be translated into a structured forecasting procedure, with effects varying substantially across events.

Primary area:
datasets and benchmarks
Keywords:
large language models, probabilistic forecasting, datasets and benchmarks, agentic systems, uncertainty quantification, information retrieval, belief revision, prospective evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8ckfOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a new benchmark for forecasting that uses data from sports events.

Critical strengths

I appreciate that the evaluation goes beyond outcome accuracy and include the trajectory of the forecast. The focus is matches played between sides in which points are scored by each.

Critical weaknesses
  • While I appreciate the effort to evaluate the uncertainty of the forecast, I do believe that the using the difference between the points scored by each team to evaluate the uncertainty is not an appropriate method. The reason is that one can be very confident that a match will be tight, however this cannot be captured here. From an ML perspective it is the reason that a linear regressor is not considered a probabilistic method, but a Gaussian Process regressor is; the latter gives a probability distribution over the prediction. Previous work (https://arxiv.org/abs/2502.19676) has proposed to use crowdsourced uncertainty estimates over outcomes, which genenarlises to tasks not involving two sides scoring points, e.g. timeframe forecasts.
  • How was avoiding test data leakage avoided, i.e. how was it ensured that the sport events used in the dataset hadn't appeared in the training data of the LLMs? My concern is that some of the LLMs are very recent
  • As far as I can see, the events included in the benchmark are likely to be included in the training data of future LLM. Thus my concern is that it will not be very useful to future research, as it will not be assessing genuine forecasting abilities
  • A lot of important information is not in the main text but in the appendix. E.g. the human validation mentioned in the workflow view is mentioned without any numbers. These are in the appendix but they should be in the main paper
Questions for authors

None

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8ckf

We thank Reviewer 8ckf for taking the time to review our submission. We apologize for replying close to the discussion deadline—we spent the rebuttal period collecting an additional 62-event prospective cohort and conducting new human audits. We have also uploaded a revised manuscript, with all additions and updates highlighted in blue for ease of review. Below, we clarify three key design aspects of FIELD and summarize our revisions.

1. Capturing Uncertainty and Tight Matches via Predictive Quantiles (Weakness 1)

  • Clarification: We agree completely that a single point estimate of score difference cannot express high confidence that a match will be tight. In FIELD, however, Margin (Sa−SbS_a - S_b) and Total (Sa+SbS_a + S_b) are not point predictions. As defined in Section 3.1 and Appendix C.1, systems submit five predictive quantiles (q.10,q.25,q.50,q.75,q.90q_{.10}, q_{.25}, q_{.50}, q_{.75}, q_{.90}) for each match, scored with the strictly proper Weighted Interval Score (WIS) (Eq. 6). A narrow quantile interval centered around zero Margin (e.g., q.10=−2,q.50=0,q.90=+2q_{.10}=-2, q_{.50}=0, q_{.90}=+2) directly expresses high confidence in a close game, whereas a wide interval expresses high uncertainty.
  • Paper Revision: Added an explicit explanation of how quantile intervals capture high-confidence tight matches in Section 3.1.

2. Pre-Event Prospective Freezing and Long-Term Benchmark Utility (Weaknesses 2 & 3)

  • Clarification:

  • Strict pre-event temporal boundary: All 232232 primary events took place in September 2026 and were registered before kickoff (Appendices A.1 and C.2). Forecasts were collected and archived within fixed pre-event windows at H1=T−24hH_1 = T - 24\text{h} and H2=T−2hH_2 = T - 2\text{h}, with no post-event regeneration or backfilling, strictly preserving the temporal boundary between forecasting and outcome resolution.

  • Long-term utility via rolling prospective evaluation: Rather than a static dataset, FIELD is a continuously runnable prospective protocol. To demonstrate this directly, we collected and evaluated a new 62-event prospective cohort (2222 football, 2020 basketball, 2020 baseball) after the initial benchmark (Section 5.2, Appendix F), replicating our core trajectory–outcome finding on newly scheduled events. Any future LLM release can be evaluated on newly registered upcoming matches under the exact same pipeline.

  • Paper Revisions: Highlighted FIELD’s rolling prospective registration in Sections 1, 3.1, and 5.3, and incorporated the new 62-event cohort in Sections 4.3, 5.2, and Appendix F.

3. Human Validation Statistics in the Main Text (Weakness 4)

  • Clarification & Revision: In the original paper, Section 4.1 reported the human validation agreement for W3/W4 semantic labeling (89.5–90.689.5\text{--}90.6\\% automated-to-human agreement; inter-annotator κ=.883\kappa=.883). In the revision, we also brought our new 80-record, two-annotator W4 extraction-sensitivity audit into Section 4.3 (94.594.5\\% human–human agreement across 563563 fixed spans, and only a +0.89+0.89 percentage-point shift in weighted mean W4 when confirmed omissions are incorporated across 7272 comparable records), with full details in Appendices D.4–D.5.
Reviewer_8ckfOpenReview ↗

I appreciate the revisions

  • The quantile-based approach is interesting but it raises the question how these are obtained from LLMs, as they are not part of the standard output; is eliciting the quantiles part of the benchmark or something the systems participating have to produce somehow?

  • The limitation of relying on score differences remains, as events not characterised in this way can't be handled.

  • The rolling perspective is useful, but it needs to be acknowledged that this means that comparisons across different points in time become harder

I maintain my score

AuthorsOpenReview ↗

Additional Clarifications

Clarification on Task Interface, Cross-Sport Scope, and Rolling Evaluation

We thank Reviewer 8ckf for the follow-up comment and for acknowledging the interest of our quantile-based uncertainty formulation and the usefulness of our rolling prospective design. To ensure complete clarity for the reviewers and Area Chair, we briefly address the three points:

  1. Structured quantile elicitation as a standard benchmark interface: FIELD explicitly defines five numerical predictive quantiles (q.10,q.25,q.50,q.75,q.90q_{.10}, q_{.25}, q_{.50}, q_{.75}, q_{.90}) for Margin and Total in its task prompt and output schema. Defining task-specific output structures is standard practice in widely adopted LLM benchmarks: SWE-bench [1] requires systems to generate code patches, while MMLU [2] evaluates predictions within a predefined multiple-choice space, without relying on these artifacts being native output modalities of the underlying LLM. Similarly, FIELD requires participating systems to directly submit five numerical predictive quantiles, without requiring native continuous-distribution logits. These submitted quantiles are evaluated directly via the strictly proper Weighted Interval Score (WIS) without post-hoc numerical repair or quantile reordering (Section 3.1, Appendices A.5 and C.1).
  2. Intentional cross-sport scope: FIELD is designed as a cross-sport forecasting benchmark for two-sided competitive matches with verifiable scoring dynamics, rather than a universal benchmark for arbitrary event types. While currently instantiated on football, basketball, and baseball, the framework is designed to accommodate additional sports with compatible scoring structures and well-defined sport-specific target mappings (such as American football or ice hockey; Section 3.1). We do not claim coverage of non-score forecasting tasks in the present study, consistent with the scope and transferability limitations discussed in Section 5.3.
  3. Cross-time comparability in rolling benchmarks: We agree that differences in event difficulty and model versions require careful treatment in cross-time comparisons. FIELD currently reports its original 232-event evaluation and subsequent 62-event prospective cohort separately, using shared valid support for within-cohort comparisons (Section 5.2, Appendix F). As additional prospective batches accumulate, we plan to introduce cumulative leaderboards alongside cohort-specific analyses. This follows established precedents in dynamic prospective evaluation: FutureX [3] maintains both weekly and overall leaderboards, while ForecastBench [4] continuously updates its evaluation and employs difficulty-adjusted Brier scoring [5] to compare forecasters evaluated on different question sets.

We appreciate the discussion and hope these clarifications are helpful.

References

  • [1] Jimenez, C. E., et al. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ICLR, 2024.
  • [2] Hendrycks, D., et al. Measuring Massive Multitask Language Understanding. ICLR, 2021.
  • [3] Zeng, Z., et al. FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction. ICLR, 2026.
  • [4] Karger, E., et al. ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities. ICLR, 2025.
  • [5] Kucinskas, S., Bastani, H., & Karger, E. ForecastBench: Updated Ranking Methodology. Technical Report, Forecasting Research Institute, 2025.
Reviewer_nvEsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FIELD introduces a prospective benchmark for studying both the quality of language model sport forecasts and how models use information to produce them. It evaluates seven systems on 232 football, basketball, and baseball events, collecting forecasts 24 hours and two hours before each event under provider-native and shared-harness settings. The paper’s main contribution is an integrated evaluation framework and descriptive evidence about LLM forecasting behavior.

Critical strengths

The paper’s main strength is its coherent evaluation of forecasting beyond final-outcome accuracy; instead, it combines uncertainty, intermediate game states, belief updating, and observable information use across three sports. Its prospective design, proper scoring rules, and explicit separation of predictive performance from workflow diagnostics provide a useful foundation for studying forecasting systems.

Critical weaknesses

The broader meaning of the benchmark’s performance differences remains unclear. Scores measure improvement over specific historical references, and observed system differences combine model ability with deployment conditions and availability. For instance, the basketball reference and evaluation cover different competitions, while only 18 basketball events receive the same aggregate weight as each larger sport sample.

The evidence in this paper supports descriptive findings more strongly than actionable lessons about forecasting. Better trajectory predictions correlate with better outcome predictions, but that does not establish a benefit from explicitly modeling trajectories.

Having said that, I'm willing to increase point if these weakness and questions can be properly addressed during rebuttal.

Questions for authors
  1. How sensitive are your main findings to the historical reference forecasts? In particular, does the conclusion that models perform better on final outcomes (than on score distributions or intermediate states) still hold when references are drawn from the same competitions as the evaluated events?

  2. Could incomplete workflow measurement explain the lack of a clear relationship between information use and the forecast quality?

4 comments
AuthorsOpenReview ↗

Response to Reviewer nvEs (1/2)

We sincerely thank Reviewer nvEs for the thoughtful, encouraging review and for recognizing FIELD’s prospective design, proper scoring rules, and separation of predictive performance from workflow diagnostics. We apologize for replying close to the discussion deadline—we spent the rebuttal period collecting additional prospective cohorts and conducting new human audits so that we could answer each of your questions with concrete empirical evidence. We have also uploaded a revised manuscript, with all additions and updates highlighted in blue for ease of review.

1. Sensitivity to Historical References, Deployment Conditions, and the Basketball Sample (Weakness 1 & Question 1)

  • Acknowledgment & Clarification: You raise an important point: normalizing the 18 international/European basketball matches against WNBA historical data can affect the magnitude of reference-relative basketball scores compared to football and baseball. As clarified in Section 3.4, our frozen historical reference serves as a fixed, sport- and target-specific normalization scale rather than a matchup-aware forecaster. In addition, FIELD explicitly separates provider-native deployment (Native) from a shared bounded retrieval harness (Controlled; Table 6) and evaluates models on shared valid support (Section 4.1, Appendix C.4) to disentangle deployment and availability differences from model comparisons.

  • New Empirical Evidence: Holding all model forecasts and resolved outcomes fixed, we substituted competition-matched historical references wherever historical records allowed (Appendix C.12, Table 18):

  • Original European football (48 matches across 5 major leagues, 2025/26 league-matched references; T1T1 unavailable in source): Outcome still outperforms Distribution (13.3813.38 vs. 8.888.88; paired difference +4.50+4.50, 95% CI [−3.58,+13.66][-3.58, +13.66]).

  • Original basketball (18 matches, competition-matched references): Pooled diagnostic scores are 42.8542.85 (Outcome), 10.2210.22 (Distribution), and 17.8917.89 (Match Trajectory), preserving the Outcome-over-Distribution ordering.

  • Independent prospective Live NBA cohort (25 matches normalized against 1,230 historical NBA games): Outcome exceeds Distribution by +19.35+19.35 points at H1H_1 (95% CI [+8.39,+28.45][+8.39, +28.45]) and +12.59+12.59 points at H2H_2 ([+0.80,+23.74][+0.80, +23.74]).

  • Sport weighting check (Appendix C.11): Reweighting the three sports proportionally to their Balanced event union counts (126126 football, 1818 basketball, 8282 baseball) preserves the complete seven-system rank ordering.

  • Paper Revisions: Updated Sections 3.4, 4.1, and 4.2, and added Appendix C.11–C.12 (Table 18).

2. Impact of Incomplete W4 Extraction on Workflow Conclusions (Question 2)

  • Acknowledgment & Clarification: Thank you for examining the W4 completeness numbers in Appendix D.4. First, we clarify that the 28.928.9\\% YES rate in Appendix D.4 reflects the proportion of whole owner records judged to have zero omissions, rather than unit-level fact recall.
  • New Human Audit: To test whether omitted revision reasons distort our aggregate W4 estimates, we conducted an independent two-annotator audit on 8080 H2H_2 owner records spanning 6565 events (Appendix D.5). Across 563563 fixed extracted spans, the two annotators achieved 94.594.5\\% agreement (532/563532/563; automated-to-human agreement is 89.989.9\\% and 93.593.5\\% on the 169169 spans with valid archived automated labels). Both annotators confirmed 99 eligible omitted reasons across the exact same 66 records. Holding the original spans and human labels fixed and incorporating all confirmed omissions across the 7272 denominator-comparable records shifts weighted mean W4 on the audited sample by only +0.89+0.89 percentage points (0.4343→0.44320.4343 \rightarrow 0.4432 for reference A; 0.4396→0.44850.4396 \rightarrow 0.4485 for reference B), indicating that extraction omissions do not materially alter aggregate revision-support patterns.
  • Paper Revisions: Added these findings to Section 4.3 and detailed the audit in Appendix D.5.
AuthorsOpenReview ↗

Response to Reviewer nvEs (2/2)

3. From Trajectory Correlations to Actionable Forecasting Procedures (Weakness 2)

  • Acknowledgment & Clarification: We completely agree that an observational link between trajectory quality and Outcome quality does not by itself prove that prompting models to generate trajectories will improve final forecasts.

  • New Prospective Studies: To test this directly across new prospective cohorts (Section 5.2, Table 3):

  • New 62-event prospective cohort (22 football, 20 basketball, 20 baseball; Appendix F): Untreated H1H_1 BASE records again replicate the positive trajectory–Outcome association across all three eligible audit-defined execution subsets (6060 events total; β=0.118,0.267,0.482\beta = 0.118, 0.267, 0.482, each 95% CI strictly above zero; Table 25). Comparing BASE against trajectory-only prompting (TRAJ, component S only) and full STEP across 7,2327,232 strict common-support target positions yields favorable point estimates (TRAJ−BASE=+0.17\text{TRAJ}-\text{BASE}=+0.17, 95% CI [−0.91,+1.31][-0.91, +1.31]; STEP−BASE=+0.56\text{STEP}-\text{BASE}=+0.56, [−0.52,+1.71][-0.52, +1.71]; Table 24).

  • Separately designed 33-event matched two-stage exploration (17 NBA, 16 baseball; Controlled H1H_1; Appendix G): Requiring Stage 1 to submit and lock T1T1/T2T2 probabilities and trajectories before Stage 2 submits the final forecast (PROCESS) yields a favorable point-estimate reduction of +0.00316+0.00316 in Outcome half-Brier loss over an unconstrained memo (GENERAL; 95% CI [−0.00446,+0.01126][-0.00446, +0.01126]; Table 26).

  • Paper Revisions: Clarified the observational boundary in Sections 4.3 and 5.1, added Table 3 in Section 5.2, and presented full results in Appendices F and G.

Thanks for the careful rebuttal. I think most of my concerns are addressed and I've increased my score accordingly.

AuthorsOpenReview ↗

Response to Reviewer nvEs

Thank you very much for your positive feedback and for taking the time to reconsider our work. We sincerely appreciate your constructive comments, which have helped us strengthen the paper.

Reviewer_YUjVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a benchmark of LLM forecasting accuracy using several sports (soccer, basketball, and baseball). For each of several types of pre-registered events, AI systems forecast outcomes at different time horizons, as well as the final outcome of the game and the final score (a continuous outcome). The authors run each model on both their native system and in a controlled setting.

On around 200 events, comparing seven frontier systems, all systems beat a baseline (reference) forecast. But there is no clear relationship between observable types of work the AI system was accomplishing as part of its forecasting process and forecasting accuracy after correction for multiple hypothesis testing.

Critical strengths

I appreciate a few specific features of this paper: (1) it provides a rigorous and transparent way of defining time horizons and questions, avoiding data leakage in the benchmark; (2) the benchmark extends beyond standard binary yes/no questions to questions with continuous outcomes, and to a careful analysis of how forecasting ability correlates over time within a game/sport; (3) comparisons of binary and continuous forecasts on the same types of questions let the authors do a careful comparison of coherence between forecasts by the same model of outcomes tracked in different ways (e.g. the question of whether a team will win vs. point margin).

Critical weaknesses

I would focus on three key weaknesses of the paper:

  1. Because each sport is weighted equally, the results are extremely sensitive to the question distribution. For example, basketball games seem to be only (less than) 10% of forecasting questions, but they receive significantly higher levels, which leads to quite different results from other sports. If you drop basketball, the leading model flips, and some key findings seem to fall apart. This raises concerns about the generalizability of these results. Readers will struggle to assess generalizability (especially to non-sport domains) if the results don't generalize across different types of sports.

  2. I have a key statistical concern about power. One of the main points of this paper seems to be to compare model performance and workflows to baseline metrics, and to examine whether 'observable workflow' correlates with final accuracy, using multiple hypothesis testing to argue for a null effect. But these corrections seem likely to reduce power in this domain. What minimum effect size are you trying to reject (under the null?). Do you have enough data to confidently make the statistical claims you're making (from a statistical power perspective).

Questions for authors
  1. Can you add a bound or achievable reference level for each target? A good one might be the actual betting lines for these markets.
  2. For claims of null effects or statistical significance, please report statistical power and minimum detectable effect sizes.
1 comment
AuthorsOpenReview ↗

Response to Reviewer YUjV

We thank Reviewer YUjV for the insightful comments on sport weighting, statistical power under multiplicity correction, and competitive baselines. We apologize for replying close to the discussion deadline—we used this period to run targeted weighting checks, power simulations, and additional prospective cohorts to address your questions thoroughly. We have also uploaded a revised manuscript, with all additions and updates highlighted in blue for ease of review.

1. Sensitivity to Sport Weighting and the Small Basketball Sample (Weakness 1)

  • Acknowledgment & Clarification: You rightly point out that equal sport weighting gives the 1818 original basketball games a 1/31/3 macro weight, and that omitting basketball shifts the top point estimate from Gemini to DeepSeek (by 0.0710.071 points, with a paired 95% CI spanning zero; Appendix C.11).

  • New Analyses:

  • Event-count-proportional weighting (Appendix C.11): When we reweight the three sports proportionally to their Balanced event union counts (126126 football, 1818 basketball, 8282 baseball) without discarding any sport, the complete seven-system rank ordering remains identical (Gemini>Kimi>DeepSeek>GPT>Grok>Claude>Qwen\text{Gemini} > \text{Kimi} > \text{DeepSeek} > \text{GPT} > \text{Grok} > \text{Claude} > \text{Qwen}). Meanwhile, because pointwise 95% intervals for all six adjacent model pairs include zero (Table 8), we explicitly frame the leaderboard as a cohort-specific summary where top ranks remain close.

  • Generalizability of Section 4 patterns (Appendix C.11 & Appendix F): Excluding basketball preserves the positive trajectory–Outcome association (H1H_1: β=0.158\beta=0.158, 95% CI [0.100,0.217][0.100, 0.217]; H2H_2: β=0.166\beta=0.166, [0.118,0.215][0.118, 0.215]), and this association recurs in a balanced new 62-event prospective cohort (2222 football, 2020 basketball, 2020 baseball; Table 25). We therefore center Section 4.3 on this robust cross-cohort relationship while retaining the full diagnostic atlas and sensitivity checks in Appendix C.7.

  • Paper Revisions: Updated Sections 4.1 and 4.3, and expanded Appendix C.11.

2. Statistical Power and Minimum Detectable Effect Size (Weakness 2 & Question 2)

  • Acknowledgment & Clarification: We agree that non-significance after Benjamini–Hochberg (BH) correction across 1616 tests does not prove zero correlation, and that readers should know what effect sizes our sample can detect.
  • New Power Simulation (Appendix C.13, Table 19): We conducted an event-clustered wild-sign simulation conditioned on our exact observed support, fixed-effect structure, and joint 16-test BH procedure. Under a single-signal alternative on the R1–W4 pairing (672672 records across 167167 events; observed β=−0.00521\beta=-0.00521, 95% CI [−0.01403,+0.00361][-0.01403, +0.00361]), an injected effect of 0.0180.018 R1 quality units per SD of W4 achieves 85.285.2\\% detection power (84.884.8\\% under Bonferroni). Across multi-signal R1–W1..W4 scenarios, injected thresholds of 0.00650.0065–0.01580.0158 achieve 80.480.4\\%–86.986.9\\% power (Table 19).
  • Paper Revisions: Added this power summary to Section 4.3 and reported the simulation grid in Appendix C.13 (Table 19).

3. Achievable / Competitive Baselines (Question 1)

  • Acknowledgment & Future Integration: We agree that competitive baselines—especially time-aligned betting-market odds and sport-specific statistical forecasting models—are important benchmarks for interpreting real-world forecasting performance. FIELD currently uses a frozen historical reference constructed from 120120 pre-cutoff observations per sport–target cell to provide a consistent normalization scale across all five targets (including intermediate states T1T1 and T2T2). As an initial complementary comparison, we also evaluated a historical team-strength baseline on the independent 25-match Live NBA cohort (Appendix C.12). We plan to incorporate time-aligned market odds and statistical forecasting baselines into future FIELD cohorts wherever comparable pre-event data can be reliably archived, as highlighted in Section 5.3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Author Responses and Manuscript Revisions

We thank all three reviewers for their constructive feedback. We have provided detailed responses to each review and uploaded a revised manuscript, with additions and modifications highlighted in blue. For ease of reference, the table below summarizes how we addressed the main concerns and where the corresponding evidence can be found.

Main ConcernReviewer(s)Our Response and New EvidenceRevised Manuscript
Forecast uncertainty and close matches8ckfClarified that Margin and Total are elicited as five predictive quantiles, evaluated using WIS, allowing forecasts to express confidence in close matches.Sec. 3.1; App. C.1
Prospective validity and future benchmark utility8ckfClarified the pre-event registration and forecast-freezing protocol. Demonstrated continued operation through a new 62-event prospective cohort.Secs. 1, 3.1, 5.3; Apps. A.1, F
Historical reference sensitivitynvEsRe-evaluated original football and basketball subsets using competition-matched historical references and added evidence from a separate 25-match prospective NBA cohort.Secs. 3.4, 4.2; App. C.12, Table 18
Deployment conditions and forecast availabilitynvEsClarified the Native and Controlled evaluation settings, shared valid-support comparisons, and the interpretation of deployed-system performance.Secs. 3.2, 4.1; App. C.4; Table 6
Sport weighting and ranking robustnessYUjV, nvEsEvent-count-proportional sport weighting preserves the complete seven-system ranking. Clarified leave-one-sport sensitivity and explicit ranking uncertainty.Sec. 4.1; Apps. C.5, C.11; Table 8
Human validation and incomplete W4 extractionnvEs, 8ckfHighlighted the original human-validation statistics and conducted a new two-annotator audit of 80 records. Human agreement reached 94.5%; incorporating confirmed omissions shifted the audited mean W4 by +0.89 percentage points.Secs. 4.1, 4.3; Apps. D.4–D.5
Statistical power of Forecast–Workflow analysesYUjVAdded conditional detection-sensitivity simulations under the observed support and joint 16-test BH correction, reporting detectable effect scales and simulation detection rates.Sec. 4.3; App. C.13, Table 19
Trajectory associations and actionable interventionsnvEs, YUjVReplicated the positive trajectory–Outcome association in a new 62-event cohort. Added prospective TRAJ/STEP comparisons and a separately designed 33-event matched two-stage experiment, with effect estimates and uncertainty reported.Secs. 4.3, 5.2; Table 3; Apps. F–G, Tables 24–26
Competitive forecasting baselinesYUjVAdded an initial historical team-strength comparison on the 25-match Live NBA cohort and outlined the integration of time-aligned market odds and statistical baselines in future FIELD cohorts.Sec. 5.3; App. C.12

Together, these additions strengthen FIELD's empirical foundation as a continuously usable cross-sport forecasting benchmark. The new prospective cohorts also demonstrate how FIELD can support the replication of diagnostic findings and the evaluation of forecasting interventions.

We appreciate the reviewers' time and welcome any further feedback during the remaining discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.