acceptodds
Withdrawn from ICLR 2027

Semantic Expert Following in Prediction Markets

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We develop an expert-following strategy for binary prediction markets using public wallet histories. The central idea is that expertise is local to the question being traded. A fixed text encoder represents market questions, and a thresholded similarity kernel gives greater weight to a wallet's performance in closely related, settled markets. The strategy first waits for a market to meet observable volume, price, and scheduled-horizon conditions. It then aggregates qualified wallet flow, requires support from several distinct addresses, and controls both signal concentration and capital exposure. No predictive model is trained on trading outcomes. We evaluate four allocation configurations on a historical Polymarket archive with 41,891 candidate markets and 183.74 million valid public trade records. In a delayed, quantity-unconstrained price replay, the primary configuration returns 257.25% with a 13.64% daily maximum drawdown. A higher-exposure configuration returns 318.32% with a 19.02% maximum drawdown. The comparison shows how local expert selection and capital allocation interact within a common consensus-based strategy.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
prediction markets, semantic representations, expert aggregation, risk control

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Nda8OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a semantic expert-following strategy for prediction markets. Wallet expertise is modeled as topic-dependent: past settled trades are weighted by the semantic similarity between historical and current market questions. The strategy combines this local skill estimate with market activation rules, multi-wallet consensus, and risk-aware capital allocation. On a historical Polymarket archive, the primary configuration reports a 257.25% return with 13.64% maximum drawdown.

Critical strengths
  1. Interesting formulation of local rather than global trader expertise. Using semantic similarity to condition wallet skill on the current question is intuitive and better motivated than assuming that successful traders are uniformly skilled across unrelated market domains.
  2. Careful portfolio-level analysis. The paper considers drawdown, capital allocation, wallet concentration, execution delay, fees, and several sensitivity analyses rather than reporting only raw prediction accuracy or trade-level returns.
Critical weaknesses
  1. The evaluation is not out-of-sample. The authors explicitly state that parameters and configurations were selected after inspecting the same historical archive used for evaluation. Given the strong sensitivity to thresholds—for example, changing the semantic threshold from 0.50 to 0.45 or 0.55 reduces returns substantially—it is difficult to assess how much of the reported performance would generalize to unseen markets.
  2. The execution model does not establish realizable trading performance. The replay ignores order-book depth and allows a single observed print to fill the requested quantity. In the primary configuration, 94 of 104 simulated purchases exceed the quantity of the print used for pricing, with a median purchase-to-print ratio of 40.23. This makes the headline return difficult to interpret as practically executable performance.
Questions for authors
  1. How would the strategy perform under a genuinely out-of-sample temporal split with all hyperparameters fixed beforehand?
  2. Can the authors estimate how sensitive the reported returns are to realistic order-book depth and fill constraints?
  3. How much of the gain specifically comes from semantic expert selection compared with a global wallet-skill baseline?
Reviewer_NrZFOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a rule-based expert-following strategy for binary prediction markets. Historical wallet performance is weighted according to the semantic similarity between previously resolved market questions and the current question, using a frozen text encoder and a thresholded similarity kernel. After a market satisfies volume, activity, price, and horizon conditions, the strategy aggregates trades from qualified wallets, requires directional consensus and historical support, and allocates capital subject to exposure limits. The method is evaluated on an archive containing 41,891 markets and 183.74 million public taker-trade records. In a delayed but quantity-unconstrained price replay, the primary configuration returns 257.25% with a 13.64% maximum drawdown. However, all parameters were selected on the same archive, all main closed positions have positive settlement PnL, and the replay does not model available order-book quantity.

Critical strengths
  1. Interesting and intuitive expert-selection problem. Measuring wallet expertise relative to the semantic topic of the target market is more plausible than assigning each wallet one global skill score. The paper provides a clear and transparent implementation of this idea using chronologically available wallet histories.
  2. Large-scale and carefully documented replay. The analysis processes more than 183 million trade records and carefully specifies market activation, label availability, wallet scoring, portfolio state, settlement, and capital constraints. The authors also report several parameter and execution stresses and are unusually explicit about the retrospective and quantity-unconstrained nature of the analysis.
Critical weaknesses
  1. The evaluation does not establish that semantic expert following provides a generalizable advantage. All thresholds and configurations were selected after inspecting the same archive used for evaluation. The result is highly sensitive to the semantic threshold: changing it from 0.50 to 0.45 or 0.55 reduces return from 257.25% to approximately 64–72% and increases drawdown to approximately 30%. There is no chronological held-out period or rolling walk-forward evaluation.
  2. Essential baselines needed to identify the source of the return are missing. The paper does not compare against global wallet skill, shuffled semantic representations, unweighted wallet consensus, or a simple price-only strategy. Market activation already requires a quantity-weighted price of at least 0.90, and all main closed positions are profitable. Therefore, the results may reflect buying mature high-price favorites rather than semantic expert selection.
  3. The execution model cannot support conclusions about achievable trading returns. A single observed trade can fill the entire requested position regardless of its quantity. In the primary configuration, 94 of 104 purchases exceed the quantity of the print supplying the execution price, with a median ratio of approximately 40. Delay, fees, and fixed slippage do not address unavailable depth, spread, queue priority, partial fills, or market impact.
  4. Historical skill and information timing may be contaminated. The market inventory and metadata were assembled retrospectively, the public records contain only taker activity, and some historical trades may occur after the real-world outcome is already known but before the recorded resolution time. These issues make it difficult to interpret the wallet score as forecasting expertise.
Questions for authors

The main concerns are already described in the Critical Weaknesses. I have two questions that could materially affect my assessment:

  1. Can the authors evaluate the fixed strategy on an untouched chronological test period and include matched baselines based on buying the market favorite, global wallet skill without semantic conditioning, and shuffled question embeddings?

  2. Can the authors report a quantity-constrained replay that limits simulated fills to observed trade quantity or available order-book depth, together with an audit of whether the relevant outcomes were already publicly known at the times used for wallet scoring and strategy entry?

Reviewer_tZiHOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper asks whether public Polymarket trade records can be used to find wallets with experience on the current question and to follow their trades for profit. A wallet’s skill is estimated from its settlement returns on similar past questions. Similarity is based on sentence embeddings of the question text. When a market is close to closing and the price leans clearly to one side, the strategy trades with the consensus of qualified wallets. In a replay on the same historical data, the main setting returns about 257 percent with a maximum drawdown of about 14 percent.

Critical strengths
  1. The data handling is careful, and the reporting is candid. The replay strictly avoids future information, and the limitations are stated openly. This makes the results easy to interpret.
  2. The comparison of capital deployment is clean. With the same signals, changing only the speed of deployment changes the return a lot, while the drawdown stays almost the same. This shows that how capital is allocated has a large effect on the results.
Critical weaknesses
  1. How similar questions are chosen is the key of the method, but the design is not well justified or tested. The paper uses sentence embedding similarity of the question text, with a hand set threshold, to decide which past questions count as similar. It does not explain why this design is chosen or why it is better than other options, such as grouping by event type. It also does not compare with how related work handles this. In addition, the results are very sensitive to this threshold. I suggest the paper explain the design and show that skill estimated with this similarity predicts later wallet performance better than other choices.
  2. The evidence for profit is not yet rigorous. The parameters are chosen on the same data, there is only one historical path, and there is no significance test. The paper itself notes that two baselines are missing: global skill scoring and simply buying the favorite. I suggest fixing the parameters and testing on a later period of new data, with these baselines and confidence intervals.
Questions for authors
  1. Why use sentence embedding similarity of the question text to decide which past questions are relevant? How does it compare with other choices, such as grouping by event type?
  2. Could you fix the parameters, test on a later period of data, and compare with global skill scoring and simply buying the favorite?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.