- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a graph-based neural network for hotspot detection. Once trained, the model can be used as a surrogate for conventional hotspot detection methods such as the local K-function. The major advantages are its computational efficiency and zero-shot generalisation capability, e.g., a model trained in one city can be applied to other unseen cities as well.
Critical strengths
- The authors incorporate road network information into spatial event datasets for hotspot detection and propose a new graph-based approach to model the relationships between events and road networks.
- Manually designed node features are extracted to facilitate hotspot detection.
- Once trained, the model is much more efficient than conventional hotspot detection methods, but with a trade-off in detection accuracy.
Critical weaknesses
-
While spatial hotspot detection is a well-established research problem, this paper uses an existing spatial hotspot detection method, i.e., the local K-function, to derive event-level hotspot labels and associated radii, and uses the derived information as ground truth to train a graph neural network. While this evaluation is appropriate for measuring surrogate approximation fidelity, it does not establish the actual hotspot detection accuracy or statistical reliability of the proposed method. Thus, the main advantage of this method is computational efficiency. However, do practical applications require real-time processing at the cost of reduced detection accuracy? Can the authors give a few examples where efficiency is a critical requirement in real-world scenarios?
-
I believe the original spatial event datasets do not include road network information. It seems that the authors enrich the datasets with road network information. However, it is unclear to me what the data source of the road network is. Is it from OpenStreetMap or other sources? What is the quality of the map data? It is possible that the road network has evolved over time. How do the authors ensure that the road network is correctly associated with historical events, considering missing roads or newly constructed roads? Is the method robust to errors in the map data? More analysis is required on these aspects.
-
The evaluation is not convincing. The authors only compare their method with a simple MLP baseline. This only shows that a graph-based neural network is able to learn additional information from graph structures, which is not surprising at all. The authors should compare their method with more advanced models, such as alternative graph neural networks or other deep learning based approaches.
-
There is also a lack of sufficient ablation studies for the proposed architecture. For example, how do accuracy and efficiency change with the number of GNN layers? There is insufficient verification of the contributions of each individual component, such as graph connectivity and handcrafted node features. Are there any other node features that could be incorporated to further enhance performance?
-
The generalization experiments demonstrate the model's ability to transfer across different cities. However, the event categories and their distributions in each dataset are not clearly specified. It is therefore unclear whether the cross-city experiments evaluate geographical generalization alone or also involve shifts in event types. The authors should clarify the event categories used in each dataset and conduct additional experiments to evaluate cross-event-type generalization, which is different from cross-city generalisation capabilities.
Questions for authors
Please refer to the questions listed in the weaknesses.
3 comments
WLsB: W3, W4
WLsB-W3
We have added GCN, GAT and R-GCN as GNN baselines, trained on the same graph with the same features and protocol; please refer to Table 3 (Section 5.2) and Appendix A.7. R-GCN, which also distinguishes relation types, reaches nearly the same detection quality as GraphSAGE-MT, showing that our graph construction and features also work with other GNN architectures; GraphSAGE-MT uses less GPU memory, and GCN and GAT are weaker.
We have also added XGBoost as a stronger non-graph baseline, given the same information through precomputed neighborhood aggregates, to measure the contribution of message passing. It reaches nearly the same classification quality as GraphSAGE-MT but predicts radii less accurately. This demonstrates the effectiveness of our graph construction and features, which benefit even a non-graph model.
These baselines complement our main comparison with AH-IBT, the statistical detector our method aims to accelerate.
Data
Table 3 (paper): four-city mean, same inputs.
| Model | F1 | Spec. | AUROC | AP | MAPE |
|---|---|---|---|---|---|
| GraphSAGE-MT | 0.895 | 0.858 | 0.953 | 0.970 | 1.8% |
| R-GCN | 0.894 | 0.851 | 0.952 | 0.969 | 1.9% |
| GCN | 0.662 | 0.733 | 0.931 | 0.953 | 1.9% |
| GAT | 0.859 | 0.770 | 0.924 | 0.950 | 2.0% |
| XGBoost | 0.887 | 0.888 | 0.953 | 0.971 | 3.2% |
Table 13 (paper): LA 8M, one 24 GB GPU, seed 42.
| Model | Forward (s) | Peak memory (GiB) |
|---|---|---|
| GraphSAGE-MT | 0.729 | 10.23 |
| R-GCN | 1.084 | 14.73 |
| GCN | OOM | OOM |
| GAT | OOM | OOM |
Four-city mean. XGBoost-prop32 is the XGBoost of Table 3; XGBoost-8 drops the multi-hop aggregates.
| Model | F1 | Prec. | Recall | Spec. | AUROC | AP | MAPE |
|---|---|---|---|---|---|---|---|
| GraphSAGE, 4 layers | 0.895 | 0.904 | 0.891 | 0.859 | 0.954 | 0.970 | 1.9% |
| GraphSAGE, 1 layer | 0.870 | — | — | — | 0.928 | — | 1.8% |
| XGBoost-8 | 0.882 | 0.860 | 0.915 | 0.740 | 0.949 | 0.969 | 2.8% |
| XGBoost-prop32 | 0.887 | 0.920 | 0.862 | 0.888 | 0.953 | 0.971 | 3.2% |
Without propagation, XGBoost specificity drops from 0.888 to 0.740.
WLsB-W4
Regarding GNN depth, accuracy improves up to three to four layers and then levels off, as each layer extends the receptive field by one hop along the graph, and forward time grows roughly linearly with depth.
Event features describe the road segment each event is on, which mainly determines the radius. Intersection features describe how many roads and events are around each intersection. The graph connections pass this information between nearby segments, so the model can tell whether events are concentrated over a wider area rather than on a single segment, which helps detect hotspot membership.
We have added a non-graph baseline, XGBoost, given the same input information through neighborhood aggregates precomputed on our graph (Table 3 and Appendix A.7); it reaches nearly the same classification quality as GraphSAGE-MT but predicts radii less accurately. We have also evaluated GraphSAGE without intersection–intersection edges: F1 is unchanged, but specificity drops, i.e., more false positives, showing the importance of connectivity between intersections.
We also zeroed each input feature in turn and retrained; the event-density rank of each event and the number of events around each intersection matter most, while each other feature has little effect. We have tried different features, including total distance, number of events, left and right distances for E-nodes, as well as average O–O and O–E distances and the numbers of E and O nodes for O-nodes. We also experimented with density relative to each graph’s mean density and raw event counts; neither improved on the current features overall.
Data
Depth, four-city mean; forward on LA 8M, seed 42.
| Layers | Params | F1 | AUROC | MAPE | F1 LA | Chicago | Detroit | Seattle | Forward A10 (s) | Forward 3090 (s) |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 10,434 | 0.870 | 0.928 | 1.8% | 0.841 | 0.868 | 0.881 | 0.889 | 0.175 | 0.126 |
| 3 | 59,970 | 0.895 | 0.953 | 1.8% | 0.876 | 0.895 | 0.888 | 0.920 | 0.543 | 0.458 |
| 4 | 84,738 | 0.894 | 0.954 | 1.9% | 0.877 | 0.894 | 0.885 | 0.921 | 0.729 | 0.573 |
| 6 | 134,274 | 0.891 | 0.952 | 1.9% | 0.877 | 0.893 | 0.872 | 0.920 | 1.092 | 0.707 |
Layers 1, 3, 4, 6 shown.
Finer ablation, four layers, four-city mean (zeroed in training and testing).
| Variant | F1 | Prec. | Recall | Spec. | AUROC | MAPE |
|---|---|---|---|---|---|---|
| Full model | 0.894 | 0.904 | 0.891 | 0.859 | 0.954 | 1.9% |
| No O–O edges | 0.895 | 0.889 | 0.907 | 0.825 | 0.951 | 1.9% |
| Zero E1 (event-density rank) | 0.880 | 0.830 | 0.948 | 0.658 | 0.947 | 1.7% |
| Zero O3 (#E neighbours rank) | 0.841 | 0.939 | 0.772 | 0.933 | 0.945 | 1.8% |
| Zero all O features | 0.844 | 0.924 | 0.786 | 0.912 | 0.941 | 1.9% |
Each other single feature changes F1 by ≤ 0.005.
WLsB: W5
WLsB-W5
The datasets contain crime reports (Chicago), police 911 calls (Detroit), traffic collisions (Seattle) and mixed city records (LA, mainly inspections and permits). Please refer to Appendix A.11 for the categories and event distribution per city.
The cities record different types of events, so our cross-city tests involve a shift in event types as well as in geography. The model still transfers well to the unseen cities, which suggests that it generalizes across event types, not only across locations. Please refer to Table 4.
For cross-event-type generalization, please refer to Appendix A.8: the LA-trained model performs well on each of eight Chicago event types, and changing either the training data or the test type affects performance only slightly, showing that the model generalizes across event types.
Data
Table 19 (paper).
| City | Source | Event type | Time span | Events |
|---|---|---|---|---|
| LA | City of Los Angeles open data | Mixed city records | 1996–2025 | 17,641,139 |
| Chicago | Chicago Data Portal, Crimes | Crime reports | 2001–2023 | 7,679,723 |
| Detroit | Detroit Open Data, 911 Calls for Service | Police 911 calls | 2016–2022 | 5,171,281 |
| Seattle | Seattle SDOT Collisions | Traffic collisions | 2003–2020 | 213,645 |
Table 20 (paper): LA record types.
| Type | Time span | Percentage |
|---|---|---|
| Inspections | 2013–2025 | 54% |
| Permits | 1996–2025 | 28% |
| Arrests | 2010–2025 | 9% |
| Crime reports | 2020–2024 | 5% |
| Traffic collisions | 2010–2025 | 3% |
| Service requests, COVID-19 | 2016–2023 | < 1% |
Table 4 (paper): F1 on the full test graph, LA-trained model.
| City | F1 |
|---|---|
| LA (8M) | 0.899 |
| Chicago | 0.891 |
| Detroit | 0.886 |
| Seattle | 0.939 |
Table 14 (paper): LA-trained model on single-type Chicago datasets (250K events each).
| Type | Hotspot (%) | F1 | Prec. | Recall | Spec. | AUROC | MAPE |
|---|---|---|---|---|---|---|---|
| All types (control) | 55.9 | 0.912 | 0.931 | 0.894 | 0.917 | 0.973 | 1.2% |
| Theft | 56.5 | 0.934 | 0.946 | 0.922 | 0.931 | 0.982 | 1.0% |
| Criminal damage | 46.9 | 0.869 | 0.861 | 0.878 | 0.874 | 0.955 | 1.5% |
Hotspot (%) = percentage of hotspot events in each dataset.
Highest (theft) and lowest (criminal damage) of eight types shown; F1 0.869–0.934. Full table: Table 14 (paper, Appendix A.8).
Table 15 (paper): F1, rows = training data, columns = Chicago test type.
| Trained on | All | Theft | Battery | Crim. dam. | Narcotics | Assault | Burglary | MV theft | Robbery |
|---|---|---|---|---|---|---|---|---|---|
| Chicago theft | 0.920 | 0.943 | 0.926 | 0.873 | 0.941 | 0.922 | 0.880 | 0.881 | 0.936 |
| Chicago battery | 0.923 | 0.924 | 0.946 | 0.875 | 0.967 | 0.938 | 0.884 | 0.882 | 0.945 |
| Chicago all types | 0.930 | 0.938 | 0.939 | 0.885 | 0.961 | 0.934 | 0.892 | 0.892 | 0.946 |
| LA | 0.912 | 0.934 | 0.912 | 0.869 | 0.926 | 0.909 | 0.872 | 0.875 | 0.924 |
Reading down each column, F1 changes by at most 0.041 when the training data change; reading across each row, it changes by 0.065–0.092 between test types. Both effects are small, showing that the model generalizes across event types.
WLsB: W1, W2
WLsB-W1
We thank the reviewer for this observation. We agree that our method is a surrogate for the local K-function-based AH-IBT detector, and that our evaluation measures fidelity to its outputs rather than independently establishing statistical significance guarantees.
Real-time applications of hotspot detection can trade accuracy in some specific circumstances in which detecting all hotspots offline is infeasible. For example, frequently updated applications, such as traffic congestion monitoring, service calls during natural disasters in populated regions, and thermal hotspots in energy systems, where repeatedly executing expensive statistical detection methods over millions of events delays updates, might prefer best effort detection rather than slow responses. Surges after storms or outbreaks also need quick updates. Each such query is a full statistical run over millions of events. On 8 million events, AH-IBT takes about 24.5 minutes end to end, whereas our model takes 8.4 seconds end to end.
When accuracy matters more, it can be recovered with extra computation: the optional calibration (Section 4.5) verifies uncertain predictions using AH-IBT, and its thresholds set how much is verified. On LA 8M, verifying part of the events brings recall and precision close to those of the statistical labels at a fraction of the cost of a full AH-IBT run.
Data
Table 17 (paper), largest graph per city, seed 42, thresholds (0.10, 0.95), two CPU cores.
For comparison on LA 8M: GraphSAGE-MT alone has recall 85.1%, precision 95.4%, F1 90.0%; with calibration, verification takes 128.9 s, about 11× less than a full AH-IBT run (1,464.6 s, excluding preprocessing).
WLsB-W2
The event data come from the cities' open-data portals cited in the paper, e.g., the Chicago Data Portal and the City of Los Angeles Open Data Catalog. Please refer to Appendix A.11 for the source, event type and time span of each dataset.
The road networks were obtained from OpenStreetMap (OSM) using OSMnx (Appendix A.11). We preprocess the networks by projecting coordinates, consolidating nearby intersections, removing parallel edges, and converting them to undirected graphs. Events are mapped to their nearest road segments using OSMnx's nearest_edges() function. Events on roads missing from the OSM snapshot (e.g., removed since 2010) are therefore not dropped but assigned to the nearest existing segment. Their positions along the edges are then calculated to support network-distance-based hotspot detection.
Please refer to Appendix A.9 for a robustness test with four types of map errors, each at three levels: missing roads, added roads, segment-length errors and event-position errors. Compared with the K-function detector rerun on the perturbed maps, F1 drops only slightly and only when roads are added, and GraphSAGE-MT's decisions change about as much as or less than the detector's own.
Data
Table 16 (paper), mean over LA 1M, Chicago 1M, Detroit 1M, Seattle 214K. (a) Against the K-function detector re-run on the perturbed map.
(b) Against the labels of the clean map.
Flips = share of events whose hotspot decision differs from that on the clean map. GraphSAGE-MT flips fewer decisions than the detector under added roads and length errors, about as many under missing roads, and slightly more under position errors.
Highest level of each error shown. Full table (three levels each): Table 16 (paper, Appendix A.9).
Table 19 (paper).