- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces CHAIN, a probabilistic event prediction framework based on a causal-temporal hypergraph. The method begins by constructing a causal-temporal hypergraph from retrieved evidence, aggregates this structured evidence through causal-chain scoring and a direction-aware deduplication mechanism, and finally adaptively fuses predictions derived from this causal evidence with those generated by an LLM. The authors conducted experiments across multiple event prediction datasets using various LLM backbones, comparing their approach against several baselines in terms of both prediction accuracy and probabilistic calibration. Overall, CHAIN achieves strong results across various settings, and the experimental scope is quite comprehensive.
Critical strengths
- Proposes a relatively complete framework for structured probabilistic prediction. CHAIN integrates temporal information, causal relationships, evidence aggregation, and probabilistic prediction into a cohesive framework. Moving beyond simply prompting an LLM for probabilities, the paper explores the use of explicit, structured evidence to improve both prediction and calibration—a meaningful direction.
- Comprehensive experimental evaluation. The authors conducted experiments across multiple datasets and LLM backbones, comparing various prediction, retrieval, and calibration methods. They also performed extensive analyses, including ablation studies, out-of-distribution (OOD) testing, evaluations of different retrieval configurations, and multi-sample analysis. Consequently, the primary issue with the paper is not a lack of experimental scope.
- Core modules are grounded in sound intuition. Mechanisms such as deduplicating redundant causal chains and adaptively weighting causal branches based on the extent of evidence coverage are intuitive design choices. My primary question is not whether these designs are reasonable, but rather whether the current experiments sufficiently validate the specific motivations behind them. ## Key Issues
Critical weaknesses
1. Does "Causal Coverage" truly reflect the reliability of the causal branch?
The paper employs "Causal Coverage" to determine the adaptive fusion weight . The core intuition is that when a question is supported by more, stronger, and more sufficient causal evidence, the causal branch should be more reliable—and thus, should be assigned a higher weight in the final prediction.
While the paper demonstrates the overall effectiveness of "Adaptive Fusion" by comparing it against a fixed-weight approach—a compelling result—the current experiments do not directly verify whether Causal Coverage itself serves as a valid measure of the causal branch's relative reliability.
For instance, Table 18 illustrates the distribution of values corresponding to different levels of . However, since is derived directly from the definition of , this merely confirms that the formula behaves as intended, rather than validating the relationship between and predictive reliability. It would be highly beneficial if the authors could group samples based on their values and compare the Brier scores or prediction errors of versus for each group. Ideally, one would observe that as increases, the predictive reliability of the causal branch—relative to the LLM branch—also improves. Such an analysis would provide more direct support for the design rationale behind Adaptive Fusion.
2. Is Adaptive Fusion truly effective when the LLM branch and the causal branch yield conflicting predictions?
Since CHAIN fuses and , a critical scenario arises when the two branches produce significantly different—or even diametrically opposed—predictions.
For example, if the LLM branch assigns a high probability to an event occurring, while the causal branch assigns a high probability to it not occurring, does the final Adaptive Fusion mechanism successfully correct an error made by the LLM, or does it potentially undermine an otherwise correct prediction? Gaining clarity on this point is essential for understanding why Adaptive Fusion is effective. While the paper presents the fusion formula and overall performance, it lacks specific reporting on the following:
- The proportion of samples for which the two branches produce conflicting directions (i.e., opposing predictions);
- The individual performance of the LLM branch, the causal branch, and the final fused result on these conflicting samples;
- Whether "Adaptive Fusion" effectively selects between the two branches—based on the degree of coverage provided by the causal evidence—in these cases of conflict.
If the predictions from both branches have been preserved in your existing experimental logs, a simple post-hoc analysis of these samples should suffice; there is no need to rerun the experiments.
Minor Points
The paper’s description of existing probability calibration methods is somewhat overly broad. While classic techniques like Temperature Scaling, Platt Scaling, and Isotonic Regression are indeed forms of "post-hoc calibration," recent research has explored calibration from perspectives such as reasoning, evidence quality, and internal model mechanisms. I suggest refining the terminology: it would be more accurate to characterize CHAIN as a method for calibration and fusion during the prediction phase, leveraging causal-temporal structures and evidence quality, rather than framing existing calibration literature as solely focused on adjusting final output probabilities.
Furthermore, given that the current set of baselines is sufficiently diverse and comprehensive, I do not recommend adding more methods simply to increase the number of baselines. Instead, a more direct analysis of the two mechanisms mentioned above would be far more valuable.
Questions for authors
Please refer to the "Weaknesses" section.
Overall, I believe the paper’s framework holds merit, and the current experimental setup is quite robust. My primary concern is not whether CHAIN is effective in terms of overall performance, but rather that the mechanistic explanation for why "Adaptive Fusion" works is not sufficiently direct. Providing the analyses suggested above would significantly strengthen the paper’s argumentation, and I would be willing to increase my score from 2 to 3.
2 comments
Dear Reviewer oRz5,
As the discussion period is drawing to a close, we would like to follow up on our response to the two mechanism analyses you suggested. We grouped samples by Causal Coverage and compared the Brier scores of the LLM, causal, and fused predictions (Sec. U.3). We also analyzed the 291 conflicting predictions, where adaptive fusion yields a net gain of 19 correct predictions and assigns higher causal weights when the causal branch is correct (Sec. U.4). In addition, we revised the calibration terminology as you suggested. We fully understand that your time is limited, and we would be grateful if our response could be taken into consideration in your final assessment. If you have any further questions, we would be glad to discuss them.
Thank you very much for your time and effort in reviewing our submission.
Best regards, Authors
Thank you for your careful review and valuable feedback. We respond point by point below.
W1 / Q1. Causal Coverage and causal-branch reliability (see Sec. U.3)
Thank you for the comment. We group samples by Causal Coverage Ω(q) and compare the Brier scores of the LLM, causal, and fused predictions:
Causal-branch reliability is not strictly monotonic in Coverage: its Brier score is lower than the LLM branch only in [0.25,0.5). Coverage is therefore neither a per-sample reliability guarantee nor a monotonic reliability measure. At Ω=0, α=0 by design and CHAIN returns the LLM prediction. Fusion lowers Brier relative to the LLM in every positive-Coverage group. These results support Coverage as an input to Adaptive Fusion, without implying that higher Coverage makes the causal branch more reliable for every sample. We will clarify this scope in the revision.
W2 / Q2. Adaptive Fusion under conflicting predictions (see Sec. U.4)
Thank you for the comment. We report conflict frequency, prediction changes, and the relationship between fusion weights and branch correctness.
Conflict definition and overall performance. A conflict means opposing LLM and causal decisions at the 0.5 threshold. There are 291 conflicts, or 28.42% of all samples. Fusion has lower Brier and higher Accuracy than either branch:
Net correction gains. Prediction correctness before and after fusion is:
Fusion increases correct predictions from 196 to 215, a net gain of 19; Accuracy rises from 67.35% to 73.88%.
Coverage-stratified outcomes. We report Brier and Accuracy in LLM/Causal/Fused order, and strict per-sample Brier wins of fusion against LLM/Causal:
Fused Accuracy is at least as high as the better branch in every Coverage group. At Ω=0, the fused and LLM predictions coincide.
Fusion weights and branch correctness. α is the causal fusion weight. We compare mean Coverage and α according to which branch predicts the correct event direction:
Conflicts correctly predicted by the causal branch have higher mean Coverage and receive higher mean causal weights than those correctly predicted by the LLM. This also holds at positive Coverage. Adaptive Fusion thus yields net correction gains, with an empirical association between weight allocation and branch correctness.
Minor point. Calibration terminology and baseline scope (see Sec. 2)
Thank you for the comment. We agree that the original description was too broad. Temperature Scaling, Platt Scaling, and Isotonic Regression are standard post-hoc methods; recent work also studies calibration through reasoning, evidence quality, and model-internal mechanisms.
We describe CHAIN as prediction-stage calibration and fusion based on causal-temporal structured evidence and evidence quality. Evidence weighting, causal aggregation, and branch fusion form the final probability, rather than applying a uniform post-prediction transformation. We have revised Related Work accordingly. The current baselines provide broad coverage; we focus on the two mechanism analyses above rather than adding methods solely to increase their number.
Finally, thank you again for your valuable feedback. We have carefully considered all your suggestions and revised the paper accordingly. If our responses and revisions address your concerns, we would be very grateful and respectfully ask you to consider raising your score. Thank you very much!