acceptodds
Withdrawn from ICLR 2027

FutureMap: Omnimodal Future Prediction with Temporal Hypergraphs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Future prediction asks what will happen, and when, from only the evidence visible at a vantage point, and large language model (LLM) agents now make it feasible over archives whose precursors span five modalities. Forecasters have progressed from agentic search to temporal knowledge graphs and hypergraphs, and recently to omnimodal inputs. However, they leave omnimodal evidence unstructured, guess the regularities linking precursors to outcomes, and date an event on whatever deadline they are shown. We formalise Omnimodal Future Prediction (OmniFP) and release OmniFP-Bench, 1,530 questions over eight domains and 7,422 frozen resources in five modalities. We propose FutureMap, a training-free forecaster that builds an omnimodal temporal hypergraph, backtests hyper-rules with time and magnitude kernels, and reads one deadline-blind deduction out in code. Against twelve baselines, FutureMap leads on all three backbones by +4.1 to +5.3 points.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
future prediction, omnimodal reasoning, temporal hypergraph

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2ZR5OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces FutureMap, a training-free framework for omnimodal future prediction based on temporal hypergraphs. The paper first formulates the Omnimodal Future Prediction (OmniFP) task and releases OmniFP-Bench, consisting of 1,530 questions, 712 storylines, 7,422 frozen resources, five modalities, and eight domains. FutureMap addresses three challenges: integrating evidence across modalities, estimating the temporal and outcome regularities between precursors and future events, and avoiding deadline-anchored timing bias. Its three-stage pipeline constructs an omnimodal temporal hypergraph, backtests typed hyper-rules with temporal and magnitude kernels, and finally performs a deadline-blind deduction using a backbone LLM to generate dated branches followed by a deterministic read-out. Experiments with three frozen backbones and twelve training-free baselines show that FutureMap achieves the best overall accuracy on all three backbones, with gains of 4.1–5.3 points over the strongest baselines.

Critical strengths
  1. The paper proposes a coherent and technically well-motivated framework for omnimodal forecasting. The separation into three stages is conceptually clear: the temporal hypergraph addresses cross-modal evidence aggregation, the known-zone deduction explicitly estimates rule reliability and temporal/magnitude distributions, and the unknown-zone deduction separates the forecasting process from the final deadline read-out.

  2. The paper proposes a novel algorithm with strong empirical results. The paper proposes a novel training-free forecasting framework, FutureMap, and demonstrates its effectiveness through comprehensive experiments across three backbones and twelve baselines, consistently achieving the best performance with a 4.1–5.3 percentage-point improvement over existing methods.

Critical weaknesses
  1. FutureMap relies on many manually specified thresholds, horizons, and weighting choices. The author doesn't provide sufficient evidence to valid their choice, and the reported ablations do not sufficiently establish their robustness.

For example, the hypergraph construction uses a 3-day temporal matching window and cosine threshold of 0.86; the rule mining uses a 42-day maximum lag and a 14-day backtesting horizon; the multimodal validity function introduces manually chosen parameters such as η=2 and β=0.25; and the final read-out uses fixed weights such as α=1/4 and ω=0.75. These choices are central to the behavior of the system, but the ablation study mainly removes components rather than testing sensitivity to these numerical hyperparameters.

  1. The evaluation does not fully isolate the contribution of native multimodal understanding from the contribution of structured preprocessing.

Some modalities are passed through fixed parsers when a backbone cannot directly consume them, while the hypergraph itself is built using Gemini 2.5. Thus, the final comparison involves not only different forecasting strategies but also a substantial upstream multimodal processing pipeline. Although the paper reports that all methods use the same frozen archive and that parsers are fixed, it would be useful to disentangle errors caused by modality understanding from errors caused by forecasting/reasoning.

Questions for authors
  1. How much of FutureMap's performance gain comes from the benchmark's repeated precursor–outcome structure rather than from genuinely transferable forecasting ability?

  2. Can the authors report calibration metrics for the final probabilities, such as Brier score or expected calibration error, rather than evaluating probabilities mainly through binary accuracy?

  3. I find the paper writing very crowded and difficult to read. The author defined a lot of variables, formulas, new definitions. The illustration figures are also filled with content, making readers hard to get the main flow of the author's method.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 2ZR5 (Part 1/3)

Thank you very much for your time and thorough review, and for recognizing that FutureMap is a coherent and technically well-motivated framework whose three stages are conceptually clear, and a novel algorithm with strong empirical results. Your concerns are the many manually specified thresholds, horizons and weighting choices (W1(a)), the sensitivity to these numerical hyperparameters (W1(b)), isolating native multimodal understanding from preprocessing (W2), how much of the gain comes from the benchmark's repeated precursor–outcome structure (Q1), calibration metrics for the final probabilities (Q2), the crowded writing (Q3(a)) and the illustration figures filled with content (Q3(b)).

We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite you to open the revised PDF to view them.

In the revised PDF, our revisions for you are in blue, together with those in red, which marks changes that address points raised by several reviewers, i.e., the overlapping concerns. They are in the new §5.8 (RQ8, p. 9) with Figures 10 and 11, and in Appendix F.8 with Table 14 (pp. 29–30).

The new runs (App. F.8) use the released code and data. They score FutureMap at 30.0/32.8/29.0 on Gemini/Qwen/Nemo (Table 1: 31.2/33.6/29.7) and grade 89 to 94% of the items alike. Rows marked with a paper table are the paper's own runs, and intervals are paired 95% CIs between runs of the same code, resampling the 712 storylines.


W1(a): "FutureMap relies on many manually specified thresholds, horizons, and weighting choices ..."

Thank you very much. All constants form one frozen setting: App. C lists every constant, and §5.8 ("Constants Fixed on One Slice") now states how the setting was fixed.

  • One setting for all backbones and domains: δ=3\delta=3, γ=0.86\gamma=0.86, η=2\eta=2, β=0.25\beta=0.25, L=42L=42, H=14H=14, α=1/4\alpha=1/4, ω=0.75\omega=0.75. Following your list, δ\delta and γ\gamma decide which reports are linked into one hyperedge, η\eta and β\beta set its validity, LL and HH bound the lags and the backtest of the rules, and α\alpha and ω\omega weigh the archive's vote and the last observed value in the read-out.
  • Fixed once: all rows' hyperparameters, FutureMap's read-out weights α\alpha and ω\omega included, were set on one fixed 480-item development slice and then frozen, and no ported prompt was tuned.
  • Outside the slice: on the 1,050 items left, FutureMap still leads the best baseline by +3.0/+5.5/+3.3, with every storyline-cluster interval above +0.6 (Figure 11a).

Thank you for raising this point. Therefore, the manually specified thresholds, horizons and weights are one frozen setting, and the lead holds on items that had no part in fixing it.


W1(b): "... testing sensitivity to these numerical hyperparameters."

Thank you very much. We added sensitivity sweeps for each constant you name (settings in App. F.8, results in Figure 10c), and every setting stays above the strongest baseline on every backbone.

  • Read-out constants act after the deduction, so we sweep them on the stored deductions of all 1,530 items:
Read-out sweep, Overall (default in bold)GeminiQwenNemo
best baseline (Table 1)27.228.325.5
α\alpha ∈ {0, .1, .25, .4, .5}30.4–31.232.5–33.629.0–29.7
uniform-prior weight ∈ {0, .25, .5, .75}30.7–31.232.5–33.629.3–29.7
Yes threshold ∈ {.40, .45, .50, .55, .60}30.1–31.232.5–33.628.9–29.7
ω\omega ∈ {.25, .5, .75, 1, 1.5}28.8–31.229.9–33.627.7–29.8
  • Structural constants shape the graph or the rules, so we change them one at a time on the 500-item analysis subset. Each change re-builds the graph or re-mines the rules, then re-runs stage 3 (Gemini and Nemo, Table 14; η\eta on Nemo only). Each row gives its two non-default values in order (e.g., δ 1/7 means δ=1\delta=1 and δ=7\delta=7):
Structural, 500 itemsGeminiNemo
best base.28.425.6
δ 1/732.2/32.830.8/29.4
γ .80/.9231.0/32.429.4/29.8
β 0/.532.4/31.430.4/29.6
η 1/4–30.8/32.0
L 28/5633.8/32.029.6/30.2
H 7/2134.0/31.829.8/30.0
default31.630.6
  • The widest read-out range comes from ω\omega, and even its lowest setting stays above the best baseline. In addition, several non-default structural settings are slightly better than the default, so the reported default is not the best point of the sweep.

Thank you for this suggestion; it added a sensitivity analysis to the paper. Hence, the sensitivity to each numerical hyperparameter is now tested, and the margin does not hinge on finely tuned constants.


AuthorsOpenReview ↗

Response to Reviewer 2ZR5 (Part 2/3)


W2: "The evaluation does not fully isolate the contribution of native multimodal understanding from the contribution of structured preprocessing."

Thank you very much. We now separate the two in three steps (§5.8), and the gain comes from the forecasting stages.

  • ① The upstream pipeline is shared: every row of Table 1 reads the same frozen archive, and Gemini 2.5 Flash-Lite builds every store once for all backbones (App. E, Table 8), so a weaker backbone does not get a weaker store. Table 1 also states per backbone which media are read natively and shades every cell served through a fixed parser, so on each backbone every row sees the same media through the same parsers.
  • ② Understanding fixed, forecasting changed ("Stage 3 on Identical Evidence" and "Without Non-Text Media or Fusion"): plain RAG gets exactly FutureMap's excerpts and media, through the same parsers, with the same answer contract and backbone (App. F.8), and articles only removes FutureMap's non-text inputs altogether:
Overall (1,530)GeminiQwenNemo
best baseline (Table 1)27.228.325.5
plain RAG on FutureMap's own evidence24.428.221.6
articles only (no non-text report, no media)29.432.729.5
  • On identical evidence, FutureMap leads plain RAG by +5.6/+4.6/+7.3 (every paired interval above +2.3), and with articles only it still leads the best baseline on every backbone. If the advantage came from a richer parse of the media, plain RAG on FutureMap's own evidence would inherit it, yet it falls below the best baseline. If it came from the non-text media, articles only would lose it, yet it keeps the lead.
  • ③ Errors split by evidence coverage (§5.8, "Calibrated and Grounded"): items are grouped by how many of their gold precursors reach FutureMap's prompt ("some" means at least one but not all; the remaining 119 items have none):
Gold precursors in FutureMap's promptitemsNo source (G/Q/N)FutureMap (G/Q/N)
some29617.9 / 20.6 / 17.226.7 / 28.4 / 26.0
all1,11523.4 / 25.0 / 22.535.0 / 37.8 / 33.1
  • When all gold precursors of an item reach the prompt, missing gold evidence cannot explain an error, and here FutureMap gains 11 to 13 points over the same backbone without sources. Accuracy is in single digits only on the 119 items whose gold precursors are not retrieved (7.6/6.7/7.6), a coverage problem that stage 1 already reduces, with 84.9% gold-precursor coverage, 12.1 points above flat retrieval of equal length (Figure 6b).

Thank you for prompting these controls; they make the source of the gain explicit. Hence, with multimodal understanding held fixed, the contribution that remains is that of the forecasting stages.


Q1: "How much of FutureMap's performance gain comes from the benchmark's repeated precursor–outcome structure ...?"

Thank you very much. Three checks in the paper show that the gain holds where that structure cannot help.

  • ① The mining uses only the archive: stage 2 reads only the known-zone archive, never the questions, answers or storyline labels, and backtests at pseudo-cutoffs inside K\mathcal{K} (§4.2). Each rule is scored on forecasts made at weekly dates before the real cutoff and checked against the days that followed in the archive. The outcomes lie after the cutoff, so stage 2 cannot learn which outcomes the benchmark asks about.
  • ② The rules hold out of sample in time: re-mined up to b−14b-14, the 838 rules forecast the 14 days the miner never saw with ECE 0.014 against 0.093 for the base rate, and the 9,192 realised lags have near-uniform PIT (KS 0.074 against 0.203). At a b−7b-7 cutoff, ρϕ\rho_\phi again has the lowest ECE (0.017) and the kernels the lowest KS (0.109) (§5.5, Figure 7, Table 11). ECE is the gap between a rule's predicted and observed hit rate, and a near-uniform PIT (low KS) means the realised lags fall where the kernels put them.
  • ③ The margin holds across storylines and memory: resampling whole storylines, all 39 paired CIs of Table 13 still exclude zero (lowest bound +1.9; §5.8, "Clustered and Audited"). The margin over the best baseline is positive in all 18 stratum cells (deadline, domain-chain and memory strata on each backbone), including the 1,249 items not answerable from memory (Table 13), and FutureMap built from a third of K\mathcal{K} already beats the best baseline using the full archive (Figure 6c).

Thank you for this question. Therefore, the gain rests on rules mined without the questions and calibrated on unseen days, and it survives storyline resampling and items the backbone cannot recall.


AuthorsOpenReview ↗

Response to Reviewer 2ZR5 (Part 3/3)


Q2: "Can the authors report calibration metrics for the final probabilities, such as Brier score or expected calibration error?"

Thank you very much. Yes, following your suggestion, §5.8 ("Calibrated and Grounded") now reports Brier score, ECE and AUC on the 333 yes/no items, and FutureMap has the lowest Brier score of all rows on every backbone, by 0.05 to 0.15.

  • The three metrics are defined for yes/no forecasts, so we compute them on the yes/no items. For Brier and AUC, the baselines' probabilities are the stated ones of mini-prophet and LEAP, and every other answer counts as a hard Yes or No forecast (App. F.8). The first row gives, per metric and backbone, the best value reached by any baseline:
Yes/no items (G/Q/N)Brier ↓ECE ↓AUC ↑
best baseline value.384 / .384 / .286.347 / .367 / .181at most .55
FutureMap.233 / .233 / .235.093 / .087 / .112.64 / .64 / .66
  • FutureMap's ECE is below the best baseline value on every backbone, and it ranks Yes above No items better than any baseline (AUC .64 to .66 against at most .55).
  • Its probabilities are also informative. On Gemini, the observed Yes rate rises across the five probability bins of width 0.2, at 20%, 46%, 55%, 63% and 76%, so higher stated probabilities go with more frequent Yes outcomes.

Thank you for requesting these metrics; they are now part of RQ8. As a result, the final probabilities come with Brier score, ECE and AUC, and FutureMap is the best calibrated row on every backbone.


Q3(a): "I find the paper writing very crowded and difficult to read ..."

Thank you very much. We reworked the writing so that the main flow is easier to follow; these edits run through the whole paper, so they are not colour-marked.

  • §4 (pp. 4–5): there is one displayed equation fewer, since the backbone's branch output is now one sentence. Symbols are glossed in words where they appear (e.g., V∘\mathcal{V}^{\circ}, Kϕ(h)K_\phi(h), yqlasty^{\mathrm{last}}_q), and each Proposition now follows its plain-word claim with its formal condition in one sentence.
  • §5: every result paragraph opens with a bold one-line finding. All new analyses sit in one section, §5.8, with the setting of each run in App. F.8, so a reader who skims gets each result from its first line.
  • Appendix: App. A gives the full prompts of stages 1 and 3 (stage 2 runs in code), App. B the proofs and App. C the algorithm with every constant.
  • Main flow, as §4 opens and Figure 4 shows: stage 1 builds a hypergraph once from the archive, stage 2 mines and backtests rules in code, and stage 3 makes one deadline-blind call per item and reads out the answer in code.

Thank you for this feedback; it made the paper easier to read. Therefore, the writing is now lighter, and each step of the method can be followed in words before its formula.


Q3(b): "... The illustration figures are also filled with content."

Thank you very much. We redrew Figures 1–4 in one consistent style, and each panel of Figure 4 now names the challenge it answers and when it runs (C1 built once, C2 mined in code, C3 one call per item).

  • The caption explains each stage in one clause, so the figure reads from left to right as the three stages.
  • The result figures share one palette and print their values (e.g., Figure 5).

Thank you for pointing this out. As a result, the illustration figures are now organised by stage and share one consistent style.


Summary of Revisions (all in the revised PDF)

WhereChange
§5.8 (new RQ8), Figures 10c, 11aOne 480-item slice for all rows; constant sweeps; lead outside the slice
§5.8, Figure 10bArticles-only and no-fusion variants; plain RAG on identical evidence
§5.8Brier, ECE and AUC on 333 yes/no items; accuracy when all gold precursors reach the prompt
§4, Figure 4, §5Readability: fewer displays, glossed symbols, panel tags, one-line findings
App. F.8, Table 14Setting, data and metric of every new run

At last, we sincerely appreciate your valuable feedback. We have carefully considered all your suggestions and substantially improved and updated the paper accordingly. If our responses and revisions address your concerns, we would be deeply grateful if you could kindly reconsider raising the score to 3 or above. Thank you very much!

Reviewer_wgUsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FutureMap forecasts events from a frozen multimodal archive using temporal hypergraphs, backtested rules, and deadline-blind prediction followed by deterministic readout. The paper introduces OmniFP-Bench with 1,530 questions, multiple answer types, and paired deadlines designed to test when an event occurs. Separating branch generation from deadline-specific readout is intended to reduce direct anchoring on the queried horizon.

Critical strengths
  1. The frozen information cutoff and negative deadline twins provide a useful evaluation design for separating event plausibility from event timing. The construction and scoring procedures are described in detail.

  2. FutureMap consistently improves overall accuracy over the evaluated baselines on three backbones. Component ablations and paired comparisons provide additional support for FutureMap's effectiveness. The shared frozen archive provides a clear information boundary for comparing methods under common evidence availability.

Critical weaknesses
  1. The specific contribution of multimodal fusion remains unclear: Removing the hypergraph also removes rules and anchors, so that ablation changes several components together. A matched text-only or unfused-event variant is needed to distinguish the value of additional modalities and fusion from downstream processing.

  2. The benchmark uses automated generation and verification,but it lacks a reported human quality check. In addition, the 1,530 items share 712 storylines and deadline twins, while confidence intervals resample individual items. Label quality and within-story dependence therefore remain uncertainties despite the positive reported intervals.

Questions for authors

How much of the gain remains with text-only inputs or without cross-modal fusion, while keeping the rules, readout, and inference budget comparable?

2 comments
AuthorsOpenReview ↗

Response to Reviewer wgUs (Part 1/2)

Thank you very much for your time and thorough review, and for recognizing that the frozen information cutoff and negative deadline twins provide a useful evaluation design, that the construction and scoring procedures are described in detail, and that FutureMap consistently improves overall accuracy over the evaluated baselines on three backbones. Your concerns are how much of the gain remains with text-only inputs or without cross-modal fusion (W1 & Q(a)), whether the rules, readout and inference budget stay comparable in this test (W1 & Q(b)), the lack of a human quality check for the automated generation and verification (W2(a)), and confidence intervals that resample individual items although the items share storylines and deadline twins (W2(b)).

We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite you to open the revised PDF to view them.

In the revised PDF, our revisions for you are in green, together with those in red, which marks changes that address points raised by several reviewers, i.e., the overlapping concerns. They are in the new §5.8 (RQ8, p. 9), Figure 10 and Appendix F.8.

The new runs (App. F.8) use the released code and data on the same frozen archive. This rerun scores FutureMap at 30.0/32.8/29.0 on Gemini/Qwen/Nemo (Table 1: 31.2/33.6/29.7) and grades 89 to 94% of items alike. Rows marked with a paper table are the paper's own runs. All intervals are paired 95% CIs resampling the 712 storylines.


W1 & Q(a): "How much of the gain remains with text-only inputs or without cross-modal fusion ...?"

Thank you very much. We added the two matched variants you ask for (§5.8, Figure 10b): the lead over the strongest baseline remains in both on every backbone, and the extra gain from non-text media is small.

  • Text only (called articles only in the paper) drops document, podcast, photo and clip reports before fusion and attaches no media. No cross-modal fusion merges reports only within one medium, and the cross-modal gain β\beta, the per-medium bonus in the validity of a hyperedge, is 0:
Overall (1,530)GeminiQwenNemo
best baseline (Table 1)27.228.325.5
text only29.432.729.5
no cross-modal fusion31.233.329.1
FutureMap (Table 1)31.233.629.7
  • Non-text media add +1.8/+0.9/+0.2 points over text only and under one point over no fusion (Table 1 against new runs). These gaps are not statistically robust, so we claim no reliable overall gain from non-text media. Text only keeps most of FutureMap's lead on Qwen and Nemo and less of it on Gemini, while no fusion keeps most of it on all three backbones.
  • Where the media matter: on the 135 items with a non-text precursor (a small subset), the rerun scores 34.1/38.5/32.6 against 28.9/37.0/35.6 for text only, so the media gain sits there on Gemini and Qwen, not on Nemo.
  • Fusion shapes the rules where corroboration exists. Fused hyperedges (events reported in two or more media) are 1.1% of the store but 11.4% of rule options, the precursor and rule pairs offered to the deduction (Figure 6a).

Thank you for suggesting these variants; they made the role of the media much clearer. Therefore, the lead over the best baseline remains with text-only inputs and without cross-modal fusion on every backbone.


W1 & Q(b): "... while keeping the rules, readout, and inference budget comparable?"

Thank you very much. Both variants keep all three comparable: they re-mine the rules, keep the read-out, and make the single deduction call (RQ8, "Without Non-Text Media or Fusion").

  • Rules: the same code mines the rules again on each variant graph, so they adapt to the variant graph and are not copied from the full one.
  • Read-out: the read-out constants are unchanged.
  • Inference budget: each variant makes one call per item, over 8.8k and 9.35k characters of excerpts and media, so the two variants use a similar amount of evidence:
Variantrulesread-outexcerpts + media (chars)calls / item
text onlyre-minedunchanged8.8k1
no cross-modal fusionre-minedunchanged9.35k1
  • As you note, "−hypergraph" removes the graph together with the rules and anchors built on it, so the new variants are the ones that show what fusion itself adds.

Thank you for spelling out these three conditions; the comparison is now controlled on each of them. As a result, the two variants differ from FutureMap only in the media and the fusion, with rules, read-out and inference budget kept comparable.


AuthorsOpenReview ↗

Response to Reviewer wgUs (Part 2/2)


W2(a): "The benchmark uses automated generation and verification, but it lacks a reported human quality check."

Thank you very much. Based on your comment, we now report a manual check of the video items after construction (§5.8, "Clustered and Audited"). It flagged 27 items whose clip does not match the event of the question, and the margin stays intact without them:

FutureMap − best baselineGeminiQwenNemo
all 1,530 items+4.1+5.3+4.2
without the 27 items (1,503)+4.2+5.4+4.3
  • Each margin re-chooses the best baseline on the remaining 1,503 items and is slightly larger on every backbone. If faulty labels favoured FutureMap, removing them would shrink its margin, so these labels do not drive the result.
  • Scope: this human check covers the video items; for the other media, label quality rests on the automated gates (App. D.2, Table 6). A pinned construction model of no backbone's family writes the items, and Table 6 counts every rejection by cause, read from the build log.
  • VERIFY checks six criteria (faithful, verifiable, temporal, no leak, matcher-gradable, well-posed) against an opposed ATTACK judge, so an item passes only if the opposing judge cannot refute its gold, and a claim counts only if its quote is found in the resource. Matcher-gradable means that the typed, rule-based matcher can score the gold without an LLM judge.
  • Leak search: S5c then re-searches all of K\mathcal{K} in each item's own wording and drops an item, with its twin, when the archive already states the answer.
  • Release check: a verifier re-checks every shipped file, for example that every quote occurs in the resource it cites, that every item has its precursors in the known zone and none in the unknown zone, and that every twin has its parent in the release. It reports no failure on the released files.

Thank you for asking for a human check; it made the label evidence more complete. Hence, the benchmark now has a reported human quality check on the video items, and the margin does not depend on the flagged labels.


W2(b): "The 1,530 items share 712 storylines and deadline twins, while confidence intervals resample individual items."

Thank you very much. We recomputed all 39 paired CIs of Table 13 with a cluster bootstrap over the 712 storylines (4,000 resamples; §5.8, App. F.8), and all 39 paired comparisons still exclude zero (lowest bound +1.9, against +1.8 with item resampling in Table 13).

  • Resampling unit: the 1,530 items come from 712 storylines, so each resample draws whole storylines, with replacement, together with all their items. A twin and its parent share a storyline, so they are always resampled together.
  • Pairing: within each resample, both methods are scored on the same drawn items, so the interval keeps the pairing as well as the clustering.
  • Against the strongest baseline on each backbone:
FutureMap − best baselineGemini (LEAP)Qwen (MiroFlow)Nemo (LEAP)
Δ [95% CI, storyline clusters]+4.1 [1.9, 6.1]+5.3 [2.7, 7.8]+4.2 [2.0, 6.5]
  • The lowest of all 39 bounds, +1.9, is the Gemini comparison with LEAP, and the other two stay above it.

Thank you for this point; the intervals now respect the storyline structure. As a result, the confidence intervals resample storylines and twins as units, and the conclusion is unchanged.


Summary of Revisions (all in the revised PDF)

WhereChange
§5.8 (new RQ8), Figure 10bText-only and no-fusion variants with the same rules pipeline, read-out and one call
§5.8Margin without the 27 video items flagged by the manual check
§5.8, App. F.8Storyline-cluster bootstrap, twins kept with their parents: all 39 comparisons exclude zero

At last, we sincerely appreciate your valuable feedback. We have carefully considered all your suggestions and substantially improved and updated the paper accordingly. If our responses and revisions address your concerns, we would be deeply grateful if you could kindly reconsider raising the score further. Thank you very much!

Reviewer_1dzrOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a benchmark for forecasting events up to 3 weeks after an archive cutoff from multi- modal media. They also introduce a training-free method that tries to solve their benchmark: an event graph with statistical counting rules and a final question the LLM answers. Their method achieves a modest increase in scores over 12 baselines on their benchmark (+4.1 to +5.3 points on three backbones).

Critical strengths

The benchmark is crafted carefully with checks on data leakage, and the evaluation based on fixed rules instead of an LLM judge removes ambiguity. The measured overall gains are statistically significant (Tab. 13).

Critical weaknesses

The papers claim is that other methods disregard multimodal data while the multimodal effects of the cross media fusion methods are not isolated in the experiments. The only related ablation ("-hypergraph", Tab. 2) removes the entire event graph structure with the statistical counting leaving a plain RAG like structure. The results suggest that much of the margin over the baselines does not come from the proposed components as FutureMap without all the hypergraph, hyper rules and anchors already beats the best baseline.

Furthermore, as reported only 1.1% of the events in the graph have more than one medium behind them (Fig. 6a), so I am sceptical about the cross-media corroboration being as integral as presented since omnimodality is the main framing of this work. Fig. 4 and Sec. 4.1 show audio being processed natively, while App. A.1 and D.2 report podcasts being transcribed and read as text. I would like to see a more transparent breakdown of what share of the data actually relies on non-text media, e.g. the media of the precursors per item. The choice of backbone models is generally okay but I think the models are too weak to claim that current models just copy the data from the question; the test of this claim is also only run inside FutureMap’s own stage 3 (Tab. 12), and on Qwen it barely changes the share of branches dated on the deadline (26.8% to 24.2%).

The paper does not justify the choice of backbones beyond modality coverage, and the benchmark is small enough (roughly $3 per run at the reported cost) to also evaluate stronger models released before the cutoff of 21 Jul 2026, e.g. Gemini 2.5 Pro. Without this, it is unclear whether the claimed problem and the claimed gain hold for stronger models.

The designed method is built to fit the hidden 1/2/3 weeks grid closely: the horizon curve is read out exactly at 7, 14 and 21 days and Eq. 10 contains a term built on 42 = 2 x 21 days, while the grid is never disclosed to the systems (App. D.3). Either ablate this, e.g. with deadlines off the grid, to show the generality of the method or prove the method on another benchmark

Questions for authors
  1. How does FutureMap perform with cross-media fusion disabled, or on text only? How many items have non-text precursors, and how much of the margin comes from them?
  2. The "-hypergraph" variant (Tab. 2) already beats every baseline. What makes this plain retrieve-then-deduce variant stronger than the baselines, and how much of the reported margin is due to the three proposed stages?
  3. How does the method perform with deadlines off the 7/14/21 grid, and how were the hyperparameters (App. C) chosen?
  4. Does the deadline-copying effect (Tab. 12) also appear for a plain LLM and for a stronger backbone released before the cutoff?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 1dzr (Part 1/3)

Thank you very much for your time and thorough review, and for recognizing that the benchmark is crafted carefully with checks on data leakage, that the evaluation based on fixed rules removes the ambiguity of an LLM judge, and that the overall gains are statistically significant (Tab. 13). Your concerns are how FutureMap performs with cross-media fusion disabled or on text only (W1 & Q1(a)), how much of the margin comes from non-text media (W1 & Q1(b)), the share of the data that relies on non-text media (W5), the few events with more than one medium behind them (W3), audio shown as native while podcasts are transcribed (W4), what makes the "−hypergraph" variant stronger (W2 & Q2(a)), how much of the margin is due to the three proposed stages (W2 & Q2(b)), the fit to the hidden weekly grid through the read days and the term in the read-out equation (W8 & Q3(a)), how the hyperparameters were chosen (Q3(b)), deadline copying for a plain LLM (W6 & Q4(a)), stronger backbones (W7 & Q4(b)), and the modest increase in scores (Summary).

We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite you to open the revised PDF to view them.

In the revised PDF, our revisions for you are in orange, together with those in red, which marks changes that address points raised by several reviewers, i.e., the overlapping concerns. You will find them on p. 4 (Figure 4 caption, §4.1), in the new §5.8 (RQ8, p. 9) with Figures 10 and 11, in Appendices E.3, F.8 (with Table 14) and H, and in the Conclusion (§6).

The new runs (App. F.8) use the released code and data. This rerun scores FutureMap at 30.0/32.8/29.0 on Gemini/Qwen/Nemo (Table 1: 31.2/33.6/29.7) and grades 89 to 94% of items alike. Rows marked with a paper table are the paper's own runs, and all intervals are paired 95% CIs resampling the 712 storylines.


W1 & Q1(a): "How does FutureMap perform with cross-media fusion disabled, or on text only?"

Thank you very much. We added both settings as two matched variants (§5.8, Figure 10b), and the lead over the best baseline holds in both on every backbone.

  • Articles only (text only) drops every document, podcast, photo and clip report before fusion and attaches no media. No fusion keeps the media and merges reports only within one medium (β=0\beta=0 in Eq. (4)).
  • Both re-mine the rules and keep the read-out and one call over about 9k characters of evidence:
Overall (1,530)GeminiQwenNemo
best baseline (Table 1)27.228.325.5
articles only29.432.729.5
no cross-media fusion31.233.329.1
FutureMap (Table 1)31.233.629.7

Thank you for proposing these two controls. Therefore, with cross-media fusion disabled or on text only, FutureMap still scores above the best baseline on every backbone.


W1 & Q1(b): "... how much of the margin comes from them?"

Thank you very much. The marginal gain of non-text media is small, +1.8/+0.9/+0.2 points over articles only and under one point over no fusion (Table 1 against new runs; RQ8, "Without Non-Text Media or Fusion"), so the lead over the baselines does not rest on them.

  • These gaps are not statistically robust, so we claim no reliable overall gain from non-text media.
  • On the 135 items with a non-text precursor (a small subset), the rerun scores 34.1/38.5/32.6 against 28.9/37.0/35.6 for articles only, so the media gain sits there on Gemini and Qwen, not on Nemo.

Thank you for asking us to quantify this share. As a result, a small part of the margin comes from non-text media, and the rest holds on articles alone.


W5: "a more transparent breakdown of what share of the data actually relies on non-text media"

Thank you very much. 135 of 1,530 items (8.8%) have at least one non-text precursor, and 315 items settle in a non-text medium.

  • Precursors: documents 103, photos 22, podcasts 17, clips 3. W1 & Q1(b) above reports how these items score with and without their media.
  • Settlement (Table 7): image 187, document 64, audio 37, video 27.

Thank you for asking for this breakdown. Hence, the share of the data that relies on non-text media is small, and it is listed above per medium.


W3: "only 1.1% of the events in the graph have more than one medium behind them (Fig. 6a)"

Thank you very much. Corroboration across media is rare, as Figure 6a shows, and the lead holds without it.

  • The new no-fusion variant keeps the lead on every backbone (W1 & Q1(a)).
  • Where corroboration exists, fusion shapes the rules: fused hyperedges are 1.1% of the store but 11.4% of rule options (Figure 6a).

Thank you for this observation. Therefore, the few events with more than one medium behind them still shape the rules, and the lead does not rely on them.


AuthorsOpenReview ↗

Response to Reviewer 1dzr (Part 2/3)


W4: "Fig. 4 and Sec. 4.1 show audio being processed natively, while App. A.1 and D.2 report podcasts being transcribed."

Thank you very much. We aligned the figure and the text: Figure 4 now reads "Native or as Text", and its caption ("natively or as transcripts") and §4.1 ("natively (a podcast as its transcript)") say so in orange, as do App. E.3 and App. H.

  • Podcasts are transcribed locally and read as text (App. D.2), while TV clips are read natively with their audio. The builder prompt names each medium as it is read (App. A.1, Figure 12: "a podcast transcript", "a TV news clip with its audio").
  • Table 1 states per backbone which media are native and which pass a fixed parser (OCR, text layer, ASR), so every row has the same media access (App. E).

Thank you for this careful check. As a result, the figure, the text and the appendices now agree on how audio is processed.


W2 & Q2(a): "The '−hypergraph' variant already beats every baseline. What makes it stronger ...?"

Thank you very much. The "−hypergraph" variant is stronger because it still runs stage 3, and we added a true plain RAG row that shows this (§5.8, "Stage 3 on Identical Evidence").

  • The variant drops only the hypergraph context and keeps stage 3 on the source excerpts, with its dated branches, horizon curve and typed read-out (App. E).
  • Plain RAG makes one call with exactly the excerpts and media of stage 3, the question as posed, the same answer contract and backbone, and no structured deduction:
Overall (1,530)GeminiQwenNemo
best baseline (Table 1)27.228.325.5
plain RAG, same evidence, one call24.428.221.6
−hypergraph, keeps stage 3 (Table 2)27.630.829.6
FutureMap (Table 2)31.233.629.7

Thank you for this question. Therefore, the variant beats the baselines through its structured deduction: without it, the same evidence gives 24.4/28.2/21.6, below the best baseline.


W2 & Q2(b): "... how much of the margin is due to the three proposed stages?"

Thank you very much. On identical evidence, stage 3 alone adds +3.2/+2.7/+8.0 points over plain RAG, and the full method leads plain RAG by +5.6 [3.5, 7.8], +4.6 [2.4, 6.9] and +7.3 [4.9, 9.4] (table above), with every paired interval above +2.3 (paired 95% CIs over the 712 storylines).

  • Plain RAG against −hypergraph isolates stage 3.
  • −hypergraph against the full method isolates stages 1 and 2, which add more on top of stage 3 (Table 2), most clearly on Gemini and Qwen.

Thank you for this follow-up. As a result, the margin over plain RAG is due to the proposed stages: stage 3 gives the first part, and stages 1 and 2 add to it.


W8 & Q3(a): "The method is built to fit the hidden 1/2/3-week grid: the curve is read at 7, 14 and 21 days and [Eq. 9] contains a term built on 42 = 2 × 21."

Thank you very much. We ran the ablation you proposed (§5.8, Figure 10a; App. F.8): with the deadlines moved off the grid, FutureMap's lead over plain RAG grows from +12.0/+12.0/+19.7 to +32.6/+36.6/+38.8 points, with every storyline-cluster interval above +26.

  • The 42 comes from the public task window. The term with 42 is a uniform-timing prior in Eq. (9), Δq/42=12⋅Δq/(c−b)\Delta_q/42 = \tfrac12\cdot\Delta_q/(c-b), where c−b=21c-b = 21 days is the public protocol window τ=(a,b,c)\tau=(a,b,c) of the task (§3). Eq. (9) mixes the curve at the deadline with this prior, which spreads the event evenly over the window. The term is linear in Δq\Delta_q, so no grid day is favoured, and its weight is swept in Q3(b) below.
  • Setup: of the 333 yes/no items, the 330 binaries and perturbed binaries each move to one day d′∈{1,…,20}∖{7,14}d' \in \{1,\dots,20\}\setminus\{7,14\} after bb. The gold stays correct: a Yes stays Yes for a later deadline, a No stays No for an earlier one, and a Yes with no later day free moves to a day on or after its event. The 5 with no such day are left out, so 325 remain (191 Yes, 134 No).
  • Read points: FutureMap's deduction omits the deadline and its read-out applies d′d'. One variant also asks the curve at days 4/10/17/21 in place of 7, 14 and 21:
325 yes/no items (G / Q / N)on gridoff grid
No source45.5 / 43.1 / 46.545.2 / 45.9 / 46.5
plain RAG, same evidence49.2 / 48.9 / 41.551.4 / 48.3 / 44.6
curve asked at 4/10/17/2163.7 / 62.5 / 60.380.6 / 84.0 / 81.2
FutureMap61.2 / 60.9 / 61.284.0 / 84.9 / 83.4
  • The 4/10/17/21 variant keeps the lead in both columns, and the stored deductions of Table 1, read out at the same days, score 84.3/85.5/83.1. Plain RAG and No source read the re-dated question.

Thank you for proposing this test. Therefore, the 42 reflects the public window, and on the yes/no items FutureMap's lead does not depend on deadlines lying on the 7/14/21-day grid.


AuthorsOpenReview ↗

Response to Reviewer 1dzr (Part 3/3)


Q3(b): "How were the hyperparameters (App. C) chosen?"

Thank you very much. We fixed all rows' hyperparameters, FutureMap's two read-out weights α\alpha and ω\omega included, once on one 480-item development slice and then froze them (§5.8; App. F.8). No ported prompt was tuned, and App. C gives the single setting for all backbones and domains.

  • On the 1,050 items where no row was tuned, FutureMap still leads the best baseline by +3.0/+5.5/+3.3, with every storyline-cluster interval above +0.6 (Figure 11a).
  • Sweeps (Figure 10c): read-out constants act after the deduction, so we sweep them on all 1,530 items. Structural constants change one at a time on the 500-item analysis subset, and each change re-builds the graph or re-mines the rules (on Gemini and Nemo, Table 14; hence the dashes for Qwen):
Overall (G / Q / N)range over all settingsbest baseline
α\alpha, uniform-prior weight, Yes threshold, ω\omega (1,530 items)28.8–31.2 / 29.9–33.6 / 27.7–29.827.2 / 28.3 / 25.5
δ,γ,β,L,H\delta,\gamma,\beta,L,H and η\eta (Nemo), two values each (500 items)31.0–34.0 / – / 29.4–32.028.4 / – / 25.6
  • Every setting stays above the strongest baseline on every backbone. Even the low end of the read-out range, 28.8/29.9/27.7 (from ω\omega), stays above 27.2/28.3/25.5.

Thank you for asking this. Therefore, the hyperparameters were chosen once on a fixed slice, and the margin does not hinge on their exact values.


W6 & Q4(a): "Does the deadline-copying effect (Tab. 12) also appear for a plain LLM?"

Thank you very much. Yes, the effect also appears for a plain LLM, and we added this test in §5.8 ("Deadlines Mislead Plain LLMs", Figure 11b).

  • The earlier test ran only inside FutureMap's own stage 3. We now ask each backbone with no source for the event date on the 117 twin pairs, with the deadline shown or withheld:
234 twin itemsdated answers on the deadline, shown / withheldsame Yes/No on both twinsboth right: plain LLM / FutureMap
Gemini 2.5 Flash-Lite51.0 / 18.8%85.5%7.7 / 25.6
Qwen3.5 Flash20.8 / 15.4%86.3%9.4 / 34.2
Mistral Nemo33.9 / 21.9%84.6%9.4 / 25.6
  • ① The plain LLM mostly ignores the deadline, giving the same answer on both twins in 84.6 to 86.3% of pairs, although each pair's twins have opposite answers. ② When it does date the event, showing the deadline pulls the date onto it (on Gemini, a gap of 32.3 points).
  • Stage 3, whose deduction never sees the deadline, removes both: withholding the deadline in FutureMap raises twin accuracy from 30.8/29.1/13.7 to 38.5/41.0/30.8 (Table 12), and both twins are right far more often (last column).

Thank you for suggesting this test. As a result, the deadline-copying effect is now shown for a plain LLM as well.


W7 & Q4(b): Stronger backbones.

Thank you very much. FutureMap is training-free, and its hypergraph, hyper-rules and dated read-out sit outside the backbone, so it can be paired with any stronger omnimodal model.

  • All 13 systems run on the same frozen backbones, released by the benchmark's information cutoff, and all five modalities reach each backbone natively or through the same fixed parsers, so every system sees the same evidence under the same leakage control.
  • FutureMap leads the strongest baseline on all three backbones (+4.1 to +5.3 points; Table 1).

Thank you for raising this point. Hence, a stronger backbone can be plugged in directly.


Summary: "a modest increase in scores."

Thank you very much. The increase is consistent: all 39 paired comparisons of Table 13, each setting FutureMap against another row of Table 1 on the same items and backbone, exclude zero, also when whole storylines are resampled (lowest bound +1.9, §5.8).

Thank you for this assessment. Therefore, the increase in scores holds in every comparison, and also outside the development slice and off the deadline grid.


Summary of Revisions (all in the revised PDF)

WhereChange
Figure 4, §4.1, Apps. E.3, HPodcasts read as transcripts
§5.8 (new RQ8), Figure 10aDeadlines off the 7/14/21-day grid
§5.8, Figure 10bArticles-only and no-fusion variants; plain RAG row
§5.8, Figures 10c, 11aOne 480-item slice for all rows; constant sweeps
§5.8, Figure 11bPlain-LLM deadline test on the 117 twin pairs
App. F.8, Table 14Setting, data and metric of every new run
§6The lead also holds off the grid

At last, we sincerely appreciate your valuable feedback. We have carefully considered all your suggestions and substantially improved and updated the paper accordingly. If our responses and revisions address your concerns, we would be deeply grateful if you could kindly reconsider raising the score to 3 or above. Thank you very much!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers,

Thank you very much for your time and thorough reviews. We are glad that you found FutureMap "a coherent and technically well-motivated framework" and "a novel algorithm with strong empirical results" (2ZR5), that "the frozen information cutoff and negative deadline twins provide a useful evaluation design" and FutureMap "consistently improves overall accuracy over the evaluated baselines on three backbones" (wgUs), and that "the benchmark is crafted carefully with checks on data leakage" and "the measured overall gains are statistically significant" (1dzr). Your shared questions concern the value of multimodal fusion, the source of the margin over the baselines, the choice of constants and the shared storylines.

We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite all reviewers to open the revised PDF to view them. Each reviewer's revisions have their own colour, orange for Reviewer 1dzr, blue for Reviewer 2ZR5 and green for Reviewer wgUs, and red marks changes that address points raised by several reviewers (the overlapping concerns).

The revisions for every reviewer are in §5.8 and App. F.8. The table lists each change, where it is in the revised paper, and who asked for it:

Concern / changeWhere in the revised paperReviewer(s)
New research question RQ8 for the new experiments, with the setting, data and metric of every new run in the appendix. The new runs use the released code and data on the same frozen archive, and intervals are paired 95% CIs resampling the 712 storylines§5.8 (p. 9), Figures 10 and 11; App. F.8 with Table 14 (pp. 29–30) (red)all
Non-text media: articles only drops every document, podcast, photo and clip report before fusion and attaches no media. The lead over the strongest baseline holds, so it does not rest on non-text media§5.8, Figure 10b (red)1dzr, 2ZR5, wgUs
Cross-media fusion: no fusion merges reports only within one medium (β=0\beta=0 in Eq. (4)). Both variants re-mine the rules and keep the read-out and one call over about 9k characters of evidence, and the lead holds in both on every backbone. Where several media report one event, fusion shapes the rules: fused hyperedges are 1.1% of the store but 11.4% of rule options (Figure 6a)§5.8, Figure 10b (red)all three
Plain RAG on identical evidence: a new row shows that stage 3 alone adds +3.2/+2.7/+8.0 points, and the full method leads it by +5.6/+4.6/+7.3, every paired interval above +2.3§5.8 (orange); Figure 10b (red)1dzr, 2ZR5
Constants: sweeps of the read-out and structural constants, where every setting stays above the strongest baseline; with all rows' constants fixed on one 480-item slice, the lead is +3.0/+5.5/+3.3 on the 1,050 items outside it§5.8, Figures 10c, 11a; App. F.8 (red)2ZR5, 1dzr
Shared storylines: a storyline-cluster bootstrap, twins kept with their parents; all 39 paired comparisons of Table 13 still exclude zero (lowest bound +1.9)§5.8 (green); App. F.8 (red)wgUs, 2ZR5
Off-grid deadlines: 325 yes/no items moved off the 7/14/21-day grid; the lead over plain RAG grows from +12.0/+12.0/+19.7 to +32.6/+36.6/+38.8 points§5.8, Conclusion, §6 (orange); Figure 10a (red caption)1dzr
Deadline copying: a plain-LLM test on the 117 twin pairs; showing the deadline pulls the date onto it, and both twins get the same Yes/No in 84.6 to 86.3% of pairs§5.8 (orange); Figure 11b (red caption)1dzr
Calibration: Brier score, ECE and AUC on the 333 yes/no items; FutureMap has the lowest Brier score of all rows on every backbone§5.8 (blue)2ZR5
Label quality: a manual check of the video items, which flagged 27 items; without them the margin over the best baseline is +4.2/+5.4/+4.3 (all items: +4.1/+5.3/+4.2)§5.8 (green)wgUs
Media in Figure 4 (p. 4): Figure 4 now reads "Native or as Text". Its caption ("natively or as transcripts"), §4.1, App. E.3 and App. H now state, in line with App. A.1 and D.2, that podcasts are transcribed locally and read as text, while TV clips are read natively with their audiop. 4 (caption and text), Apps. E.3 and H (orange)1dzr
Presentation: one displayed equation fewer and symbols glossed in words; figures redrawn in one consistent style; every result paragraph opens with a bold one-line finding. These edits run through the whole paper§4 (pp. 4–5); Figures 1–4; §5 (not colour-marked)all

Each reviewer's thread below gives the full evidence and ends with a table of where every change appears. We hope these revisions resolve your concerns, and we would be glad to answer any further question during the discussion period. Thank you again for your valuable time!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.