acceptodds
Under review as a conference paper at ICLR 2027

Benchmarking Conflict Event Forecasting

Abstract

Conflict-event forecasting is a high-stakes task with direct implications for humanitarian response, early warning, and policy planning. Despite the growing availability of fine-grained conflict-event data, the task remains underexplored in the machine learning community. We introduce ConflictCast, the first machine learning benchmark for conflict event forecasting. Given an observed history of geo-referenced conflict-events, models predict an ensamble of future event trajectories, with their times, locations, and types; sampled trajectories can then be evaluated directly or aggregated to the space–time grids used by existing operational forecasting systems. ConflictCast builds on four conflict-event datasets and includes neural spatio-temporal point processes, conflict-tailored neural models, tree-ensemble binned regressors, and simple baselines. We evaluate models under different conditioning regimes and a variety of metrics. Across datasets, we identify where current architectures succeed and fail, highlight directions for future work, and make standardized evaluation results publicly available through our website and code repository (anonymized links in the paper).

Primary area:
datasets and benchmarks
Keywords:
conflict-event forecasting

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_FqveOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ConflictCast, a benchmark for conflict-event forecasting that extends conventional event-count prediction to forecasting complete spatio-temporal event trajectories. It integrates four conflict-event datasets covering different reporting mechanisms and conflict settings.

The benchmark evaluates models under three updating regimes: fully conditioned, periodic update, and no update, using metrics for trajectory fidelity, event counts, probabilistic calibration, and next-event likelihood. It compares neural spatio-temporal point processes, Hawkes processes, tree-based regressors, and simple baselines, alongside two proposed conflict-tailored models.

The experiments show that Hawkes-based models perform well overall, but strong next-event prediction does not necessarily translate into reliable long-horizon forecasting. The results also reveal persistent calibration limitations across existing approaches.

Critical strengths
  1. Move Beyond Aggregate Event Counts Toward Fine-Grained Spatio-Temporal Conflict Forecasting. The paper addresses an important forecasting problem with applications in humanitarian response and early-warning systems. By moving beyond aggregated event counts to event-level spatio-temporal trajectories, it enables a richer assessment of conflict dynamics and forecasting capabilities.
  2. Provide unified evaluation across different updating regimes, forecasting models, and metrics. The benchmark provides a unified framework for comparing diverse forecasting approaches. The three updating regimes distinguish next-event prediction from longer-horizon simulation, while multiple evaluation metrics capture complementary aspects of forecasting quality. The standardized data processing and evaluation pipelines further enhance its practical utility.
  3. Reveals important limitations in long-horizon forecasting through extensive empirical evaluation. The paper evaluates a diverse set of models and includes additional analyses of forecast horizons, reporting delays, covariates, and model design choices. A particularly valuable finding is that strong next-event prediction does not necessarily lead to reliable long-horizon simulation, highlighting an important limitation of existing approaches.
Critical weaknesses
  1. Insufficient comparison with existing forecasting benchmarks. Although the paper discusses related benchmarks such as ViEWS, WeatherBench, and ForecastBench, it lacks a systematic comparison establishing the novelty and necessity of ConflictCast. In particular, differences in forecasting targets, task granularity, updating regimes, and evaluation capabilities are not clearly summarized.

  2. Unexamined ranking sensitivity and potential selection bias in test-Fold construction. (1)Sensitivity of the overall model ranking. The overall ranking equally weights five metric families, despite substantial differences in model performance across forecasting regimes. For instance, CAM-HP ranks first overall but performs poorly under the no-update regime. It remains unclear whether the overall rankings would remain consistent under alternative metric weights or greater emphasis on long-horizon forecasting. (2)Potential selection bias in test-fold construction. Forecast origins are selected using event-time quantiles, and only folds with at least 40 test events are retained. While this ensures sufficient observations for evaluation, it may systematically exclude low-activity periods and bias the evaluation toward more active conflict settings. The impact of these selection criteria on model rankings and benchmark representativeness is not examined.

  3. Limited Dataset Diversity and Generalizability. Although the benchmark includes four datasets, they cover only two conflict settings, with three datasets focusing on Central Africa. Moreover, ACLED, UCDP-GED, and SAGE have overlapping geographic and temporal coverage and may record some of the same underlying conflict events. While their reporting mechanisms differ, the potential event overlap is not quantified or discussed, raising questions about the independence of the evaluation settings. These limitations may affect the generalizability of the findings, and further analysis of dataset overlap or evaluation across more diverse conflict settings would strengthen the benchmark.

  4. Presentation Could Be Improved. The manuscript contains several presentation issues, including incorrect cross-references (e.g., Appendix L.1 for the Energy Score definition and Section E.6 in the kernel ablation discussion), an unresolved "??" placeholder in the Compute Statement, and occasional typographical and grammatical errors.

Questions for authors
  1. What distinguishes ConflictCast from existing forecasting benchmarks, particularly ViEWS, in terms of task formulation and evaluation capabilities?
  2. How sensitive are the overall model rankings to alternative metric weights or greater emphasis on long-horizon forecasting?
  3. How does the minimum requirement of 40 test events affect evaluation representativeness? Would alternative splitting strategies change the results?
  4. To what extent do ACLED, UCDP-GED, and SAGE contain overlapping events, and how might this affect the independence and generalizability of the evaluation?
  5. What does the evaluation-history fix mentioned in Table 8 involve, and does it affect any reported results or conclusions?
4 comments

We thank the reviewer for their time. We concisely reply to the stated weaknesses:

  1. Comparison to other benchmarks: Respectfully, we would like to note that WeatherBench and ForecastBench are benchmarks on other domains. Regarding ViEWs, it is limited to binned models on event counts, with grids that are not more granular than ~50km per side at the equator. Our benchmark evaluates the full predictive distribution over future event sequences (of which ‘expected counts’ are just a functional of the distribution), testing neural point processes previously never used. We expand on this already in our related work.

  2. Unexamined ranking sensitivity: We don't think this is true. Ranks of table 1 summarize all metrics and regimes, hence we weight each column the same. However, we also provide other ranks: for example, figure 2 ranks models on the spatio-temporal energy score, showing simple baselines winning over longer horizons and ranking them first. Indeed, our result section highlights that different models have different strengths. On this point, we want to importantly remark that the objective of the paper is not to show the superiority of a class of models, but to provide 1) an understanding of which model class fails when, and 2) a benchmark (software, evaluation framework, website …) that can help bringing progress forward in neural point process models and their applicability.

  3. Generalizability and evaluation folds / Selection Bias. We think this is the fairest criticism. Since the submission we did two things: i) Added additional 34 countries for ACLED and UCDP, with a total of 149 evaluation folds across countries, periods and data sources. We retrained on each fold-country-source. We note that the overall conclusions have stayed consistent. ii) We changed the fold selection criterion: we agree that filtering on test-set could be seen as a kind of 'leakage', hence we filtered on training instead, excluding training periods that had less than 50 events, to ensure that deep learning models would have enough data to train.

  4. On overlapping events: the three sources are different because they come from different reporting strategies (media sources, newswire and hand collection), and these indeed give different data distributions. To get a sense of this: SAGE records 11.6 events per day over the region against 7.5 for ACLED and 1.7 for UCDP; only 4% of SAGE events have an ACLED event within 1 km on the same day and 1% a UCDP event.

  5. This caption remained from a previous bug in the code. Although the bug had been fixed, the caption was inadvertently left unchanged. Thank you for catching this; we have now removed it.

We thank the reviewer again for their time and engagement with the work. If the reviewer was at least partially convinced by our responses, we would appreciate their reassessment of the work.

Dear Reviewer,

Thank you again for your review! As the discussion period is coming to an end in a few days, we wanted to check whether our response addresses your concerns. Thank you.

After considering the authors’ rebuttal, I am inclined to revise my recommendation from Weak Reject to Weak Accept. The authors have addressed several of my main concerns, most importantly by expanding the evaluation to 38 countries and 149 folds and by removing the test-set event-count threshold, which alleviates the concerns about limited geographic generalizability and selection bias. The clarification of the distinction from ViEWS also makes the benchmark’s task-level contribution more convincing.

A few questions remain:

  1. Sensitivity of overall rankings: The sensitivity of the overall ranking to alternative metric-family weights is still not directly examined.

  2. Cross-dataset event overlap: The reported overlap is quantified only under a strict same-day/1-km matching criterion, which may not capture events recorded with temporal or spatial discrepancies.

  3. Validation-fold stability: Some newly included validation folds contain very few events, which may affect the reliability of hyperparameter selection.

We thank the reviewer for the rebuttal and for the continued engagement.

We will take the additional points into account in the camera ready version of the paper or we will add them as limitations.

Reviewer_At6AOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ConflictCast, a benchmark for conflict-event forecasting that evaluates simulated future event trajectories across four datasets. It compares neural spatio-temporal point processes, conflict-tailored models, binned regressors, and simple baselines using likelihood, energy scores, grid-count metrics, and posterior predictive checks. The experiments show that strong next-event prediction does not necessarily translate into reliable longer-horizon forecasts and that calibration remains challenging.

Critical strengths
  • The benchmark provides a common evaluation pipeline for event-level forecasts across multiple conflict datasets, model families. This is a useful contribution to conflict-event forecasting research.
  • The experiments reveal important differences between next-event prediction and longer-horizon forecasting, as well as limitations in the calibration of simulated trajectories.
Critical weaknesses
  • The year-minus-one covariate alignment uses the nearest available raster year when historical data are unavailable. This may introduce future information for earlier events, potentially affecting comparisons involving covariate-aware models.
  • Daily timestamps in ACLED, UCDP-GED, and SAGE are randomly jittered within the first hour. The resulting artificial within-day ordering may affect next-event likelihood and fully conditioned evaluations, but its influence is not examined.
Questions for authors
  • Which folds are affected by the use of raster covariates from later years, and does this influence the reported results?
  • Are the next-event likelihood and fully conditioned results robust to the temporal jitter applied to daily-resolution datasets?
3 comments

We thank the reviewer for the positive review. Regarding the weaknesses, we clarify as follows:

  1. Our original wording was imprecise: we just meant that when covariates are not updated annually, we use the most recent available historical year. Indeed, we only selected evaluation folds for which historical covariates were available, which means 2017-2024 for ACLED, UCDP and SAGE, and 2004-2009 for SIGACT, for which we use a subset of available covariates at that time. Importantly, since the submission we have also expanded our evaluation to include 30+ additional countries from ACLED and UCDP (amounting to 38 conflict zones in total). We have revised the manuscript to clarify the alignment procedure.

  2. This is something that was flagged also by the reviewer xcGg, and we agree on this point. We have added a sensitivity analysis (Appendix E.7) evaluating all models across ACLED and UCDP folds (SAGE and SIGACT have continuous timestamps). We recomputed the scores using different timestamp conventions, including day-level aggregation and randomizing event times within their recorded day. The results show that ranking among model remain stable (median Spearman correlation of 0.97–0.99). We want also to note that jittering is a typical data processing step when modelling point processes, as it generally doesn't hamper between-model comparability.

Dear Reviewer,

Thank you again for your review! If our response has addressed your concerns, we would be grateful if you would consider increasing your score.

Dear authors,

Thank you for your additional experiments and comments. I think the revised description about the yearly alignment of covariates addresses my concerns about the use of future information. The sensitivity analysis of timestamp preprocessing is also useful for showing that model ranking don't change much when timestamps are handled differently.

Accordingly, I will keep my positive rating.

Reviewer_xcGgOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ConflictCast, a benchmark for predicting conflict-event times, locations, and types across four datasets from Central Africa and Iraq. It compares point-process models, binned regressors, and simple baselines under fully conditioned, weekly-update, and no-update settings. The paper reports that strong next-event performance does not consistently translate into strong multi-week forecasts, and that simulated trajectories often fail its calibration checks.

Critical strengths

The common event-stream interface connects point-process and binned forecasting methods under explicit update regimes. This is a useful contribution because it exposes differences that next-event likelihood or aggregate count accuracy alone would miss.

The experimental coverage is good. The paper includes competitive simple baselines and examines longer horizons, reporting delays, covariates, kernel choices, spatial resolution, and simulation sample size.

Critical weaknesses

The calibration conclusion is not sufficiently validated. Several predictive-check statistics are discrete, including event count and maximum daily count, but the paper does not specify how empirical p-values handle ties.

The claimed 90% and 40% pass rates are not automatic for discrete statistics. Appendix E.6 checks stability with different numbers of simulated paths, but does not establish the expected behavior under a correctly specified simulator. This matters because calibration is a headline finding and contributes to the overall ranking.

The fully conditioned comparison may not apply the same procedure to every model. The stated regime generates a next-event prediction from each realized history and fixes the number of predictions to the observed count. However, DKMPP’s no-update results are reused for this regime because its intensity is history-free. History independence does not establish equivalence between these two sampling procedures. This needs justification or correction before the cross-model rankings can be treated as comparable.

Questions for authors

What exact empirical p-value and tie-handling rule is used for the predictive checks? Do correctly specified discrete and continuous synthetic models recover the stated 90% and 40% pass rates?

Why is reusing DKMPP’s no-update output valid under the stated fully conditioned procedure? Can a common Poisson-process test verify consistent conditioning and event-count handling across model implementations?

How sensitive are the temporal rankings to timestamp preprocessing? Placing all daily-resolution events within the first hour introduces artificial ordering and bursts; a jitter-scale or day-resolution sensitivity check would help establish robustness.

Does the nearest-year raster fallback ever use an asset unavailable at the actual forecast cutoff? Please provide the relevant fold and asset dates to substantiate the no-lookahead claim.

4 comments

We thank the reviewer for their assessment and for recognising important strengths in the paper. Before addressing the weaknesses, we would like to respectfully offer a different perspective on the paper’s broader significance, which we believe also bears on the low “informativeness” score.

The main motivation behind this work was to understand how off-the-shelf neural point processes (NSTPPs) perform on a real-world forecasting task. This is a fundamental question whenever one is interested in having methods used in practice. It turns out that current NSTPPs performance is not particularly competitive. If we try to understand how these results fit into the broader research culture of our community, we see two things:

  • There is a consensus on a lack of application-driven benchmarks. Mukherjee et al. [2] observe that neural STPPs remain “rarely utilized to inform policy decisions or design interventions.” Shchur et al. [1] note a “lack of standardized experimental setups and benchmark datasets.” Our paper speaks directly to this gap.
  • Such benchmark contributions are often undervalued, either because they are deemed uninformative (especially when they uncover negative results like in our case), or because they are not seen as sufficiently generalizable.

We believe that the tension between points (1) and (2) is hindering progress in our community, as the lack of realistic benchmarks contributes to a misalignment between methods and applications. We would therefore sincerely ask the reviewer to consider whether their assessment of the paper's limited informativeness might, at least in part, reflect the tendency described above.

Regarding the weaknesses, we clarify as follows:

  1. Our implementation uses randomized tie-breaking when computing empirical PPC p-values, meaning that observations tied with simulated values are assigned random ranks. This ensures uniformly distributed ranks also for discrete statistics (in our case 3/5 quantities). It is true that this implementation detail was not explicited in the manuscript, and we have now added it.

  2. DKMPP is an inhomogeneous Poisson process (Daley et. al [3]), which is history-independent by construction. Consequently, reconditioning on realised events does not change its predictive distribution, making its fully conditioned and no-update simulations equivalent.

  3. We find this a very useful suggestion. We have added a sensitivity analysis (Appendix E.7) evaluating all models across ACLED and UCDP folds (SAGE and SIGACT have continuous timestamps). We recomputed the scores using different timestamp conventions, including day-level aggregation and randomizing event times within their recorded day. Ranking remain highly stable (median Spearman correlation of 0.97–0.99). We want also to note that jittering is a typical data processing step when modelling point processes.

  4. Our original wording was imprecise: we just meant that when covariates are not updated annually, we use the most recent available historical year. Indeed, we only selected evaluation folds for which historical covariates were available, which means 2017-2024 for ACLED, UCDP and SAGE, and 2004-2009 for SIGACT, for which we use a subset of covariates available for that time. Since the submission, we have also expanded our evaluation to include 30+ additional countries from ACLED and UCDP (amounting to 38 conflict zones in total). We have revised the manuscript to clarify the alignment procedure.

If our points were convincing, we would be very grateful if the reviewer would reassess their score.

References

[1] Shchur, O., Türkmen, A. C., Januschowski, T., & Günnemann, S. (2021). Neural Temporal Point Processes: A Review. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI), 4585–4593.

[2] Mukherjee, S., Elhamdi, M., Mohler, G., Selby, D. A., Xie, Y., Vollmer, S., & Großmann, G. (2025). Neural Spatiotemporal Point Processes: Trends and Challenges. Transactions on Machine Learning Research (TMLR).

[3] Daley, D. J., & Vere-Jones, D. (2003). An Introduction to the Theory of Point Processes, Volume I: Elementary Theory and Methods (2nd ed.). Springer.

The rebuttal clarifies tie handling and addresses timestamp sensitivity, but synthetic null-model tests are still needed to validate the claimed calibration pass rates. The DKMPP comparison requires further verification of equivalent conditioning, event-count handling, and sampling procedures across forecasting regimes. For these reasons I maintain my score.

We appreciate the reviewer's continued engagement with our rebuttal and are glad that some of our clarifications have addressed their concerns. We also understand and appreciate the reviewer's diligence in pressing us on the two remaining points.

1. Synthetic null-model validation. We have performed the requested experiment, and the results confirm that the procedure is correctly calibrated: under correctly specified Poisson and Hawkes generators, the observed PPC-in-90 and PPC-in-40 pass rates are 90.0% and 40.0%, respectively, including for discrete statistics.

Importantly, this result is not surprising, since uniformity under the null follows theoretically from randomized tie-breaking. This was precisely why we had initially not considered an additional empirical validation necessary. Nevertheless, we understand the reviewer's request for direct experimental evidence and are happy to have provided it.

2. DKMPP under full conditioning. We are particularly grateful to the reviewer for pressing us on this point. We took the last few days to rerun the simulations for DKMPP using the fully conditioned next-event (FC) simulator and observed results that differ from the periodic-update (PU) regime. The reviewer was therefore correct that the two simulation procedures are not equivalent: although DKMPP does not incorporate new event history, the FC simulator generates one prediction per observed event, whereas PU does not fix the number of predictions to the observed count.

In practice, this correction does not change our overall conclusions or the model rankings, and it concerns only one of the thirteen evaluated models. The updated results are now reported in the manuscript. We thank the reviewer again for catching this.

On the overall assessment. We hope these additional experiments and corrections address the two remaining technical concerns. Given that the synthetic validation confirms the theoretical expectation and the DKMPP implementation has now been corrected without changing the overall conclusions, we would be grateful to understand whether there are further concerns motivating the score of 1.

If the reviewer remains unconvinced despite these efforts, we would also welcome a discussion of the broader point raised in our initial response: the scientific value of application-driven benchmarks, particularly when their main findings expose limitations of existing methods rather than improvements over them. We believe this is an important question for how contributions of this kind are assessed.

Dear Reviewer,

Thank you again for your review! As the discussion period is coming to an end, we wanted to check whether our response addresses your concerns. Thank you.

Comments on the paper

1 comment

As the discussion period is ending, we would like to briefly summarize our contributions in light of the strengths and weaknesses highlighted in the rebuttals:

Contributions

We provide the first machine learning benchmark on conflict-event forecasting , evaluating 4 different datasets and 38 countries, testing ML models that were never used with this data (i.e. neural point processes) and traditional tabular ones. The usefulness of this contribution was recognized by all reviewers. Reviewer Fqve, in particular, initially raised concerns about generalizability, which we agreed with and addressed, having expanded the evaluation from 4 to 38 countries. The reviewer explicitly acknowledged this improvement and revised their recommendation to weak accept.

Our benchmark is an important resource for aligning research with applications: our findings identify limitations in the predictive performance of recent neural models under practical forecasting regimes. The importance of these findings was explicitly recognized by reviewers At6A and Fqve. Reviewer xcGg, instead, states, "I learned little or nothing new from this paper." We challenged this judgment and invited the reviewer to discuss the broader role of benchmarks and evaluations in ML research. We think that this is an important discussion especially in the neural point process subcommunity, as recent surveys have stressed the need for benchmarks that align research directions with practical applications, given that neural point processes are rarely used in practice. We would also like to emphasize that, especially today, it is important to carve out space for ML research that addresses significant interdisciplinary and societal problems.

Reviewers' concerns and additional analyses

We want to note that most of the technical concerns were resolved through clarifications of our analysis (e.g., Reviewer xcGg requested empirical validation of an already theoretically justified calibration procedure) or additional sensitivity analyses (e.g., Reviewers xcGg and At6A requested an analysis to ensure that timestamp jitter did not affect the results).

The only implementation error was identified by Reviewer xcGg and concerned one of the three simulation regimes for one of the thirteen models tested: we had assumed that, for inhomogeneous Poisson processes, fully conditioned simulation would be equivalent to the other regimes. However, the fully conditioned procedure fixes the number of predictions to the observed event count, making the simulation procedures slightly different. We therefore reran the simulations for that model using the already implemented fully conditioned simulator and updated the corresponding results. We want to emphasize that this affected only one of the thirteen models, and neither the overall conclusions nor the model rankings changed in any of the evaluation metrics.

We are still awaiting a response from Reviewer xcGg on whether these corrections address their concerns. Overall, we thank all reviewers for reading our paper and engaging with the rebuttals.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.