acceptodds
Under review as a conference paper at ICLR 2027

Unlocking the Forecasting Economy: A Suite of Datasets For The Full-Lifecycle of Prediction Markets

Abstract

Prediction markets are markets for trading claims on universal future events (e.g., presidential elections). Fueled by a meteoric surge with over $50 billion trading volume, they have emerged as a promising forecasting mechanism, where their prices provide continuously updated signals of collective beliefs. In decentralized platforms (e.g., Polymarket), the prediction market lifecycle include six stages: market creation, token registration, trading, oracle interaction, dispute, and final settlement. However, comprehensively tracking this complete pipeline remains a major challenge, as the underlying data are severely fragmented across heterogeneous on-chain smart contracts and off-chain sources. To fill this critical gap, we present the first continuously synchronized dataset suite for the full-lifecycle of decentralized prediction markets. To achieve large-scale cross-source integration, incomplete linkage, and continuous synchronization, we build a unified relational data system that integrates three canonical layers: i) market metadata, ii) fill-level trading records, iii) oracle-resolution events, through identifier resolution, on-chain recovery, and incremental updates. The resulting dataset spans from October 2020 to update-to-date and comprise more than 770 thousand market records, over 943 million order execution records, and nearly 2 million oracle events. We describe the data model, collection pipeline, and consistency mechanisms that make the datasets reproducible and extensible. We further demonstrate its utility for multiple communities through NBA outcome calibration for sport traders, CPI expectation reconstruction for economists, and oracle-risk analysis for blockchain researchers. A public website with dataset access, interactive visualizations, and lightweight LLM-assisted exploration tools are publicly available at bluehttps://www.polymonitor.club/.

Primary area:
datasets and benchmarks
Keywords:
prediction markets, Polymarket, blockchain, lifecycle dataset, market microstructure

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_Ya4qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a continuously synchronized dataset suite for studying the full lifecycle of decentralized prediction markets, using Polymarket as the primary setting. The main contribution is a unified data infrastructure that links three heterogeneous layers—market metadata, fill-level trading records, and oracle/resolution events—through canonical identifiers, on-chain recovery, and incremental synchronization. The September 2026 snapshot contains approximately 3.29M market/protocol identities, 1.90B deduplicated OrderFilled events, and 21.03M oracle and resolution events, including more than 2.4M markets with both observed trading activity and verified final outcomes. 24811_Unlocking_the_Forecastin 24811_Unlocking_the_Forecastin The paper validates the utility of the dataset through several downstream studies. In NBA markets, pre-game market probabilities are shown to be broadly well calibrated, while additional trading features do not yield persistent out-of-sample alpha. 24811_Unlocking_the_Forecastin For CPI markets, the authors reconstruct continuous market-implied inflation expectations from bucket contracts and show that they provide a high-frequency complement to conventional nowcasts, although they are not uniformly more accurate. 24811_Unlocking_the_Forecastin The paper also introduces a supervised benchmark for predicting final market resolution from partial trading histories, comparing tabular, regular time-series, and irregular event-sequence models; notably, the latest market price remains the strongest probabilistic baseline in terms of Log Loss and Brier score. 24811_Unlocking_the_Forecastin Overall, the work is primarily a dataset and research-infrastructure contribution aimed at enabling reproducible studies of forecasting, market microstructure, information aggregation, and oracle dynamics.

Critical strengths
  1. A genuinely valuable full-lifecycle prediction-market dataset at substantial scale.
    The paper goes beyond collecting market prices or metadata by linking market definitions, fill-level trading activity, and oracle/resolution events under a common identity structure. The resulting snapshot contains roughly 3.29M market/protocol identities, 1.90B deduplicated fills, and 21.03M oracle-related events, with more than 2.4M identities having both trading activity and verified resolution evidence. This is consequential because it enables analyses that require connecting information formation, trading behavior, and realized outcomes, which are difficult to conduct reliably using existing fragmented data sources.

  2. The validation demonstrates practical research utility rather than merely dataset scale.
    The downstream studies cover materially different use cases, including leakage-aware NBA probability calibration and trading evaluation, reconstruction of market-implied CPI expectations, and resolution prediction from partial trading histories. In particular, the evaluation uses point-in-time information and held-out temporal splits, and the paper reports negative results when more complex models or trading signals fail to outperform the market baseline. This strengthens my confidence that the dataset can support rigorous empirical research rather than serving only as a large descriptive archive.

Critical weaknesses
  1. The cross-source linkage and data-quality validation are not yet sufficiently rigorous for a dataset whose main contribution depends on correctness of entity resolution.
    The paper carefully describes how market metadata, outcome tokens, executions, and oracle events are linked using canonical identifiers and fallback matching rules, but it provides limited quantitative evaluation of linkage accuracy itself. In particular, there is no systematic estimate of precision/recall, large-scale manual audit, or breakdown of matching reliability across different protocol generations and fallback cases. This is consequential because even a small rate of incorrect market–trade–resolution associations could materially affect downstream calibration, forecasting, and behavioral analyses. Since the primary contribution is the dataset rather than a new modeling method, stronger evidence that the reconstructed lifecycle links are correct is important for acceptance.

  2. The downstream validation is relatively limited compared with the scale and breadth of the dataset, which weakens the evidence for its claimed general research utility.
    Despite containing millions of market identities and billions of executions, the supervised learning benchmark uses only 120 resolved markets, with only 18 markets in the test split, while the CPI study covers only 13 target months. In addition, only about 73% of catalog identities contain both observed trading activity and verified final-resolution evidence. These experiments demonstrate that the dataset can support several interesting analyses, but they do not yet establish that the proposed infrastructure yields robust conclusions across market categories, time periods, and protocol variants. A substantially larger and more diverse benchmark evaluation would make the empirical case for the dataset considerably stronger.

Questions for authors
  1. How was the accuracy of the cross-source entity resolution quantitatively validated?
    Since the main contribution relies on correctly linking market metadata, outcome tokens, fill-level executions, and oracle/resolution events, could the authors report the proportion of links obtained through direct identifiers versus fallback matching, together with an estimate of linkage precision/recall or a manual audit on a representative sample? It would also be useful to know whether matching accuracy varies across protocol generations or market types. The answer would materially affect my confidence in the correctness of the reconstructed lifecycle dataset.

  2. Why is the supervised-learning benchmark restricted to only 120 resolved markets, despite the dataset containing millions of linked market lifecycles?
    Could the authors clarify the filtering criteria that reduce the available data to this subset, and whether these 120 markets are representative of the broader dataset in terms of category, liquidity, duration, and time period? In particular, the test set contains only 18 markets, which leads to fairly wide confidence intervals. Understanding whether this small benchmark is due to intrinsic task constraints or implementation choices would materially affect my assessment of the empirical validation.

  3. How should the approximately 73% market-level coverage of both trading activity and verified final resolution be interpreted with respect to selection bias?
    Could the authors provide a breakdown of the missing lifecycle components by time period, market category, protocol generation, and liquidity level? If markets with incomplete lifecycles are systematically older, less liquid, or structurally different from fully linked markets, this could affect the validity of downstream analyses and the claimed generality of the dataset.

Reviewer_AgTXOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a continuously maintained dataset suite for decentralized prediction markets, linking market metadata, on-chain fills, and oracle and resolution events through unified identifiers. Its main contribution is an integrated research resource and collection pipeline, accompanied by the Polymonitor platform. Validation consists of coverage and linkage statistics, NBA probability calibration and trading-signal evaluation, CPI expectation reconstruction, a supervised resolution-prediction benchmark, and an audited multi-agent analysis example.

Critical strengths
  1. A useful integration of contract definitions, fills, and final-resolution evidence. Fragmentation across APIs and smart contracts is a practical obstacle to prediction-market research. By linking these records at scale while retaining source identifiers and intermediate resolution events, the resource can reduce repeated data-reconstruction work and support analyses that require both trading histories and outcome labels.
  2. Meaningful baselines and candid reporting of negative results. The experiments compare learned models with the latest market price and acknowledge that better classification or ranking does not necessarily improve probabilistic prediction. The NBA analysis also fails to establish persistent excess returns under the tested execution assumptions. These findings help researchers choose appropriate baselines and avoid interpreting historical fit as a reliable trading advantage.
Critical weaknesses
  1. The CPI application is described using two different methods. Section 3.2 (p. 7, lines 336-353) uses a rolling 24-hour window, weights by trade value, removes buckets with estimated probability at most 0.10, and fits Gaussian interval probabilities to obtain a mean. Appendix D.2 (pp. 24-25, lines 1295-1334) instead aggregates traded share quantities by hour, carries forward the latest estimate when there is no trade, normalizes bucket probabilities, and averages bucket representative values. The appendix explicitly identifies the latter as the method behind Figure 5. These differences affect both the input probabilities and the final estimate; they are not interchangeable descriptions of one calculation. The paper therefore leaves unclear which method produced Figure 5 and Tables 14-15. This weakens a prominent demonstration of the dataset's usefulness. The authors should identify the method actually run, provide the corresponding script and parameters, and reproduce the reported values. This issue does not by itself invalidate the underlying dataset.
  2. The central claim of full-lifecycle reconstruction is not sufficiently validated. The paper reports associations and high coverage, but having a trade and a final outcome under one market identifier does not show that all applicable intermediate events are correctly linked or that historical metadata states can be recovered. Section 2.1 allows title-and-deadline matching when direct identifiers are absent, whereas Appendix C.2 says unresolved oracle mappings are deferred rather than assigned heuristically; their respective scope is unclear. Appendix C.2 also describes in-place metadata updates, and the Table 3 note explicitly distinguishes the retrieved state from the state observable at the historical execution time. These are material limitations for the paper's main data contribution. The submission needs an operational definition of lifecycle completeness, validation of the mapping rules and event sequences, and results that separate record-association coverage from reconstruction accuracy.
Questions for authors
  1. Which CPI reconstruction procedure generated Figure 5 and Tables 14–15? Please identify the actual aggregation window, weighting variable, filtering rule, and estimator, and explain the discrepancy between Section 3.2 and Appendix D.2.
  2. The paper demonstrates associations across lifecycle stages, but has not sufficiently validated whether these associations recover complete market histories, particularly historical metadata states and intermediate resolution events. Please define the criterion for full-lifecycle coverage and report validation results under that criterion.
  3. Appendix D.3 defines the 20% and 50% stages using final settlement time, while Section 3.3 lists remaining time among the features. Was the final settlement time used in the inputs, normalization, or sample selection? Which time information was available at prediction time, and is the task intended as retrospective analysis or online forecasting?
  4. Which data and code are accessible during review? The conclusion describes the dataset as publicly available, whereas Appendix A promises full release after acceptance. Please distinguish the currently available materials from the planned release and identify the data version and scripts corresponding to the reported results.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final Author Rebuttal Summary: Key Revisions, Linkage Reliability, Dataset Scale, and Practical Impact

Dear reviewer,

We thank the Area Chair and reviewers for their feedback. We summarize the concerns raised and our revisions below.

1. Dataset scale (Reviewer 45Aa: insufficient evidence of scale and utility).

Our continuously updated, full-lifecycle dataset contained 3.29M markets, 1.9B+ executions, and 21M+ oracle events in September 2026, and now exceeds 7M markets and 2.1B executions. This surpasses Marketlens (349M trades; https://marketlens.trade/data) and predmktdata (865M trades; https://predmktdata.com) in reported scale, although coverage differs. Crucially, we integrate market metadata, fill-level executions, and oracle-resolution histories across protocol generations since 2020. Public downloads: https://prediction-market-data-engine.vercel.app/

2. Linkage correctness (Ya4q Q1 and 45Aa Q1: accuracy and audit evidence; AgTX Q2: lifecycle reconstruction).

Our cross-source joins use contract-defined identifiers (conditionId, clobTokenIds, questionId), following Polymarket's official APIs and smart contracts (https://gamma-api.polymarket.com/docs), rather than primarily heuristic matching. 1,901,194,698 of 1,901,194,769 OrderFilled records map to catalog identities; the remaining 71 concern 21 combinatorial conditions outside the snapshot catalog. Only 10,395 of 21,028,317 lifecycle events (0.0494%) require fallback matching. A manual audit of 1,000 fallback-linked markets found no incorrect matches in the sample. Section 2.1 and Appendix C document identifier examples, transaction-level reconciliation, and intermediate events. Similar identifier-based approaches appear in predmktdata (https://predmktdata.com/docs) and poly_data (https://github.com/warproxxx/poly_data).

3. Benchmark scale and temporal validity (Ya4q Q2 and 45Aa Q4: limited sample; AgTX Q3: look-ahead leakage).

We expanded the prediction benchmark 1,000-fold, from 120 to 120,000 markets, adding metrics and 95% confidence intervals. MLP retains the highest AUROC at both scales (0.827 originally; 0.791, 95% CI [0.785, 0.796] when expanded). We explain differences in Log Loss/Brier between cohorts. The task is retrospective: final settlement time defines post-hoc stages and labels, not model inputs; features use only information available at simulated prediction times.

4. Lifecycle completeness and selection bias (Ya4q Q3 and 45Aa Q2: coverage and missingness; AgTX Q2: lifecycle definition).

2,402,991 markets (73.02%) have observed trading and verified final resolution, well above the below-50% coverage reported for related datasets (Qin & Yang, 2026; Akey et al., 2026), although definitions differ. We do not claim every market has a complete historical record. We added missingness statistics by year (2020–2023: 28.31%; 2024: 95.99%) and category (Crypto: 89.97%; Sports: 61.79%), discussed legacy AMM/FPMM mechanisms and snapshot cutoffs, and proposed stratified sensitivity analyses.

5. CPI methodology (AgTX Q1 and 45Aa Q3: inconsistent methods for Figure 5 and Tables 14–15).

We clarify two distinct estimators. Figure 5 (Gaussian fitting) uses 24-hour rolling windows, trade-notional (price × size) weights, complementary NO prices, excludes buckets with probability ≤0.10, and takes the fitted mean as CPI. Tables 14–15 (share-weighted buckets) use hourly share-quantity weights, carry-forward for no-trade periods, normalized bucket probabilities, and probability-weighted representative values; Table 14 covers 13 months. These are complementary procedures, not competing descriptions of one estimator.

6. Downstream adoption and public access (45Aa: practical utility; Ya4q Q1: linkage correctness; AgTX Q4: data/code access).

Our linked records already support public tools, notably Polymonitor (https://polymonitor.club/) for market intelligence and Backtest Lab (https://backtest.polymonitor.club/) for historical replay and strategy research. These applications provide practical corroboration of cross-source linkage usability (Ya4q) and demonstrate real downstream value at scale (45Aa). Further industry-associated recognition comes from Qin and Yang (2026), affiliated with quantitative trading firm Time Seventeen Capital, who cite our synchronized, cross-source data contribution (https://arxiv.org/abs/2606.04217).

We also clarify that the MAS is a proof-of-concept panel in the polymonitor (https://www.polymonitor.club/), not a new algorithmic claim (45Aa Q5).

Experiments use the September 1, 2026 snapshot; data are accessible through the Data Engine above, and anonymous code is available at https://anonymous.4open.science/r/polymonitor-DBC2. The frozen dataset is planned for release in Kaggle after acceptance.

We respectfully ask the Area Chair to consider these substantive improvements and the dataset's scientific and practical significance.

Best regards,
Authors of Submission 24811

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.