acceptodds
Under review as a conference paper at ICLR 2027

CHAIN: Calibrating Probabilistic LLM Forecasts via Causal-Temporal Hypergraph Inference

Abstract

Large language models have achieved significant progress in event forecasting, yet their probability outputs exhibit systematic calibration bias that varies heterogeneously across different domains and question types, undermining the trustworthiness of probabilistic outputs for decision-making under uncertainty. However, existing calibration methods typically correct probability outputs after prediction is complete, without modeling the structural sources of bias within the prediction process itself. To address this challenge, we decompose probabilistic prediction over causal-temporal hypergraphs into three stages, evidence weighting, evidence aggregation, and source fusion, and propose CHAIN, which designs stage-specific mechanisms to mitigate bias at each stage: (i) modulating the temporal decay function by causal topological distance, (ii) aggregating approximately independent causal chains via Noisy-OR after direction-aware deduplication, and (iii) driving adaptive fusion by causal coverage and directional balance. Experimental results on cross-domain forecasting benchmarks show CHAIN outperforms existing methods in expected calibration error, Brier score, and accuracy. Our project is available.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Large Language Models, Hypergraph, Probabilistic Event Forecasting, Probability Calibration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oRz5OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CHAIN, a probabilistic event prediction framework based on a causal-temporal hypergraph. The method begins by constructing a causal-temporal hypergraph from retrieved evidence, aggregates this structured evidence through causal-chain scoring and a direction-aware deduplication mechanism, and finally adaptively fuses predictions derived from this causal evidence with those generated by an LLM. The authors conducted experiments across multiple event prediction datasets using various LLM backbones, comparing their approach against several baselines in terms of both prediction accuracy and probabilistic calibration. Overall, CHAIN achieves strong results across various settings, and the experimental scope is quite comprehensive.

Critical strengths
  1. Proposes a relatively complete framework for structured probabilistic prediction. CHAIN integrates temporal information, causal relationships, evidence aggregation, and probabilistic prediction into a cohesive framework. Moving beyond simply prompting an LLM for probabilities, the paper explores the use of explicit, structured evidence to improve both prediction and calibration—a meaningful direction.
  2. Comprehensive experimental evaluation. The authors conducted experiments across multiple datasets and LLM backbones, comparing various prediction, retrieval, and calibration methods. They also performed extensive analyses, including ablation studies, out-of-distribution (OOD) testing, evaluations of different retrieval configurations, and multi-sample analysis. Consequently, the primary issue with the paper is not a lack of experimental scope.
  3. Core modules are grounded in sound intuition. Mechanisms such as deduplicating redundant causal chains and adaptively weighting causal branches based on the extent of evidence coverage are intuitive design choices. My primary question is not whether these designs are reasonable, but rather whether the current experiments sufficiently validate the specific motivations behind them. ## Key Issues
Critical weaknesses

1. Does "Causal Coverage" truly reflect the reliability of the causal branch?

The paper employs "Causal Coverage" Ω(q)\Omega(q) to determine the adaptive fusion weight α(q)\alpha(q). The core intuition is that when a question is supported by more, stronger, and more sufficient causal evidence, the causal branch should be more reliable—and thus, should be assigned a higher weight in the final prediction.

While the paper demonstrates the overall effectiveness of "Adaptive Fusion" by comparing it against a fixed-weight approach—a compelling result—the current experiments do not directly verify whether Causal Coverage itself serves as a valid measure of the causal branch's relative reliability.

For instance, Table 18 illustrates the distribution of α\alpha values corresponding to different levels of Ω\Omega. However, since α\alpha is derived directly from the definition of Ω\Omega, this merely confirms that the formula behaves as intended, rather than validating the relationship between Ω\Omega and predictive reliability. It would be highly beneficial if the authors could group samples based on their Ω\Omega values and compare the Brier scores or prediction errors of PcausalP_{\mathrm{causal}} versus PllmP_{\mathrm{llm}} for each group. Ideally, one would observe that as Ω\Omega increases, the predictive reliability of the causal branch—relative to the LLM branch—also improves. Such an analysis would provide more direct support for the design rationale behind Adaptive Fusion.

2. Is Adaptive Fusion truly effective when the LLM branch and the causal branch yield conflicting predictions?

Since CHAIN fuses PllmP_{\mathrm{llm}} and PcausalP_{\mathrm{causal}}, a critical scenario arises when the two branches produce significantly different—or even diametrically opposed—predictions.

For example, if the LLM branch assigns a high probability to an event occurring, while the causal branch assigns a high probability to it not occurring, does the final Adaptive Fusion mechanism successfully correct an error made by the LLM, or does it potentially undermine an otherwise correct prediction? Gaining clarity on this point is essential for understanding why Adaptive Fusion is effective. While the paper presents the fusion formula and overall performance, it lacks specific reporting on the following:

  • The proportion of samples for which the two branches produce conflicting directions (i.e., opposing predictions);
  • The individual performance of the LLM branch, the causal branch, and the final fused result on these conflicting samples;
  • Whether "Adaptive Fusion" effectively selects between the two branches—based on the degree of coverage provided by the causal evidence—in these cases of conflict.

If the predictions from both branches have been preserved in your existing experimental logs, a simple post-hoc analysis of these samples should suffice; there is no need to rerun the experiments.

Minor Points

The paper’s description of existing probability calibration methods is somewhat overly broad. While classic techniques like Temperature Scaling, Platt Scaling, and Isotonic Regression are indeed forms of "post-hoc calibration," recent research has explored calibration from perspectives such as reasoning, evidence quality, and internal model mechanisms. I suggest refining the terminology: it would be more accurate to characterize CHAIN as a method for calibration and fusion during the prediction phase, leveraging causal-temporal structures and evidence quality, rather than framing existing calibration literature as solely focused on adjusting final output probabilities.

Furthermore, given that the current set of baselines is sufficiently diverse and comprehensive, I do not recommend adding more methods simply to increase the number of baselines. Instead, a more direct analysis of the two mechanisms mentioned above would be far more valuable.

Questions for authors

Please refer to the "Weaknesses" section.

Overall, I believe the paper’s framework holds merit, and the current experimental setup is quite robust. My primary concern is not whether CHAIN is effective in terms of overall performance, but rather that the mechanistic explanation for why "Adaptive Fusion" works is not sufficiently direct. Providing the analyses suggested above would significantly strengthen the paper’s argumentation, and I would be willing to increase my score from 2 to 3.

2 comments

Thank you for your careful review and valuable feedback. We respond point by point below.

W1 / Q1. Causal Coverage and causal-branch reliability (see Sec. U.3)

Thank you for the comment. We group samples by Causal Coverage Ω(q) and compare the Brier scores of the LLM, causal, and fused predictions:

CoveragenLLM BrierCausal BrierFused Brier
Ω=05040.20700.25000.2070
(0,0.1)2300.22190.31320.2097
[0.1,0.25)760.20910.27280.1937
[0.25,0.5)330.23320.14960.1477
[0.5,1]1810.19700.20240.1740

Causal-branch reliability is not strictly monotonic in Coverage: its Brier score is lower than the LLM branch only in [0.25,0.5). Coverage is therefore neither a per-sample reliability guarantee nor a monotonic reliability measure. At Ω=0, α=0 by design and CHAIN returns the LLM prediction. Fusion lowers Brier relative to the LLM in every positive-Coverage group. These results support Coverage as an input to Adaptive Fusion, without implying that higher Coverage makes the causal branch more reliable for every sample. We will clarify this scope in the revision.

W2 / Q2. Adaptive Fusion under conflicting predictions (see Sec. U.4)

Thank you for the comment. We report conflict frequency, prediction changes, and the relationship between fusion weights and branch correctness.

Conflict definition and overall performance. A conflict means opposing LLM and causal decisions at the 0.5 threshold. There are 291 conflicts, or 28.42% of all samples. Fusion has lower Brier and higher Accuracy than either branch:

PredictionBrierAccuracy
LLM0.206467.35%
Causal0.319832.65%
Fused0.192173.88%

Net correction gains. Prediction correctness before and after fusion is:

LLM predictionFused correctFused incorrect
Correct16135
Incorrect5441

Fusion increases correct predictions from 196 to 215, a net gain of 19; Accuracy rises from 67.35% to 73.88%.

Coverage-stratified outcomes. We report Brier and Accuracy in LLM/Causal/Fused order, and strict per-sample Brier wins of fusion against LLM/Causal:

CoveragenBrier L/C/FAccuracy L/C/F (%)Fused wins L/C
Ω=01480.1578/0.2500/0.157885.14/14.86/85.140/126
(0,0.1)650.2459/0.5146/0.241460.00/40.00/61.5426/39
[0.1,0.25)250.2412/0.4261/0.241452.00/48.00/52.0012/13
[0.25,0.5)90.3356/0.0466/0.11990.00/100.00/100.009/0
[0.5,1]440.2656/0.2624/0.221340.91/59.09/61.3626/18

Fused Accuracy is at least as high as the better branch in every Coverage group. At Ω=0, the fused and LLM predictions coincide.

Fusion weights and branch correctness. α is the causal fusion weight. We compare mean Coverage and α according to which branch predicts the correct event direction:

ScopeΩ: Causal correctΩ: LLM correctα: Causal correctα: LLM correct
All conflicts0.32620.10060.35220.1427
Ω>0 conflicts0.42450.28180.45830.3996

Conflicts correctly predicted by the causal branch have higher mean Coverage and receive higher mean causal weights than those correctly predicted by the LLM. This also holds at positive Coverage. Adaptive Fusion thus yields net correction gains, with an empirical association between weight allocation and branch correctness.

Minor point. Calibration terminology and baseline scope (see Sec. 2)

Thank you for the comment. We agree that the original description was too broad. Temperature Scaling, Platt Scaling, and Isotonic Regression are standard post-hoc methods; recent work also studies calibration through reasoning, evidence quality, and model-internal mechanisms.

We describe CHAIN as prediction-stage calibration and fusion based on causal-temporal structured evidence and evidence quality. Evidence weighting, causal aggregation, and branch fusion form the final probability, rather than applying a uniform post-prediction transformation. We have revised Related Work accordingly. The current baselines provide broad coverage; we focus on the two mechanism analyses above rather than adding methods solely to increase their number.

Finally, thank you again for your valuable feedback. We have carefully considered all your suggestions and revised the paper accordingly. If our responses and revisions address your concerns, we would be very grateful and respectfully ask you to consider raising your score. Thank you very much!

Dear Reviewer oRz5,

As the discussion period is drawing to a close, we would like to follow up on our response to the two mechanism analyses you suggested. We grouped samples by Causal Coverage and compared the Brier scores of the LLM, causal, and fused predictions (Sec. U.3). We also analyzed the 291 conflicting predictions, where adaptive fusion yields a net gain of 19 correct predictions and assigns higher causal weights when the causal branch is correct (Sec. U.4). In addition, we revised the calibration terminology as you suggested. We fully understand that your time is limited, and we would be grateful if our response could be taken into consideration in your final assessment. If you have any further questions, we would be glad to discuss them.

Thank you very much for your time and effort in reviewing our submission.

Best regards, Authors

Reviewer_NNamOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a framework of calibrating LLM forecasting using topological distance and temporal distance of events. It further proposes to deduplicate the dependent causal chains by using overlapped edges between chains. The proposed framework has shown better performance than existing calibration methods in 4 in-distribution datasets and 4 out-of-distribution datastes.

Critical strengths
  1. the idea of using topological distance and temporal distance to adjust prediction score is sound. Also, the idea of de-duplicating dependent chains in prediction is also sound (philosophically similar to variable selection in prediction in supervised learning).

  2. the experiment results are strong. The proposed CHAIN methods outperform other calibration methods in both in-distribution and out-distribution datasets.

Critical weaknesses
  1. The writing of this paper is somewhat hard to follow. Should this paper be accepted, I’d suggest author(s) of this paper make sure that every term is defined when first introduced. For instance, “score-qualified candidate chain” in line 205 is introduced without enough context, it is explained only in the later sentences such that some chains will be pruned. “Let PπP_{\pi} count its prevents edges” is also wrong in grammar and confusing at the same time: why counting prevents edges only, and why not edges with “causes”/”enables”, and what about the edges mixed with prevents and enables?

  2. Although using Jaccard overlap makes sense to rule out the dependent/duplicate chains, using Jaccard overlap implicitly assumes this mental model: each edge is equally important and each chain is uniquely defined by the set of edges. However, in real world events, would it be possible that some of the nodes will be “vital” in predicting the final outcome, so that edges associated with those nodes should receive much higher weight? For instance, when we want to predict if earth’s average temperature next year shall increase or decrease, and two of the chains both mention a news report of a significant increase in predicted gasoline car consumption in a large market. Suppose the later events in the two chains are mostly different, due to the shared edge of “increased gasoline car causing increased carbon release”, both chains cause temperature to increase. In this scenario, this forementioned edge is vital such that the two chains should not be considered independent, because once the news report is revealed to be unreliable, both chains’ prediction will fail, and the Noisy-OR logic in equation (16) fails. I’d suggest authors explain more about ”why defining independence based on proportion of edges shared”.

  3. In line 140, why is the set of edge types set as [causes, enables, prevents]? This set seems to arbitrary because “enables” is semantically similar to “causes”. What is the difference between “causes” and “enables”? Are they playing the same role in p+(q)p^{+}(q) of formula (16)?

  4. In formula (7), exponential decay makes sense, and it is common to use exponential function to capture temporal decay. However, I was wondering whether authors try other embedding method of the temporal distance (say, linear) to see if exponential decay is the best way to model temporal distance.

Questions for authors

See point 2 to 4 in weakness and please answer them correspondingly.

Another follow-up question is: In line 428, in the finding of ” Noisy-OR safeguards OOD accuracy”, intuitively it makes sense for Noisy-OR to improve accuracy as it prevents predicted probability inflation caused by redundant chains. But why would the effect of redundant chains especially severe in OOD datasets? Is there any correlations core or anecdotes to explain this in those OOD datasets?

2 comments

Thank you for your careful review and valuable feedback. We respond point by point below.

W1 / Q1 — Definitions and mixed-chain polarity (see Secs. 4.1, 4.2).

At first use, a “score-qualified candidate chain” is now defined as a candidate path with ungated score at least τ_Φ that terminates at an oriented target. Top-B_π truncation forms the candidate pool Π_q. We now write “P(π) denotes the number of prevents edges in path π.” All causes, enables, and prevents edges contribute through edge strength and CTVF. Each prevents edge flips propagation polarity; the parity of their count determines the direction of a mixed chain.

W2 / Q2 — Jaccard and approximate independence (see Secs. 4.2, V.3).

Edge-set Jaccard, shared edges divided by their union, is a deterministic structural-redundancy proxy within each polarity. It is reproducible from directed edge sets without external labels. Edge strengths and CTVF enter chain confidence Φ; Jaccard measures structural overlap without additional node-importance weighting. Paths can share critical nodes, reports, or latent events; shared critical edges with different later edges are a boundary case. Noisy-OR aggregates retained chains under within-polarity approximate independence. Node-, source-, and semantic-aware deduplication are future extensions.

Table 40 reports structural and lexical overlap for all within-case scored-chain pairs before deduplication, restricting cases to at least two scored chains. Mean edge J averages pairs within each case, then cases. Both rates use all within-case pairs in each subset. Lex. denotes token Jaccard; Disjoint + Lex. requires disjoint edges and token Jaccard ≥0.4.

SubsetPair casesMean chainsMean edge JDisjointDisjoint + Lex.
ID23252.800.171830.69%7.95%
OOD17662.770.158430.67%5.13%

W3 / Q3 — Relation semantics and aggregation (see Sec. 4.2).

The three types distinguish direct, facilitating, and inhibitory effects while providing a common polarity encoding. causes is a direct directional causal effect; enables increases downstream feasibility without guaranteeing occurrence; prevents is a directional negative effect. Relative to the queried event, aligned causes and enables provide positive evidence, and prevents negative evidence; the parity of prevents determines polarity flips. All enter Eq. (16), with no type-specific multiplier; causes and enables remain semantically distinct.

W4 / Q4 — Temporal-decay sensitivity (see Sec. Q.1).

We compare exponential with continuous truncated linear decay. For x=Δt≥1 and ρ>0, h=2^(2/ρ) is the shared half-life. Linear recency is max{0, 1−(x−1)/[2(h−1)]}, reaching 0.5 at x=h and zero at x=2h−1; it is one at ρ=0. Candidate chains, relation directions and strengths, cutoffs, deduplication, LLM outputs, and fusion parameters stay fixed; chain scores, aggregation, and adaptive fusion are recomputed. Macro averages follow; ECE and Accuracy are in %, Brier on [0,1].

Recency functionECE↓Brier↓Accuracy↑
Exponential8.14%0.198970.02%
Continuous truncated linear8.84%0.207169.53%

Lower ECE and Brier and higher Accuracy support CHAIN's exponential temporal form.

Follow-up — OOD redundancy (see Sec. Q.3).

We report Noisy-OR ablation and chain-removal diagnostics. Entries are ECE/Brier/Accuracy in %.

OOD datasetFull CHAINWithout Noisy-OR
Clinical-Trial3.61 / 22.73 / 62.505.02 / 23.47 / 60.16
FOReCAst6.98 / 17.90 / 74.228.43 / 18.87 / 72.66
Golf-Forecasting5.15 / 23.91 / 58.597.76 / 24.02 / 57.03
Kalshi7.25 / 22.95 / 62.509.13 / 23.20 / 60.16
OOD macro average5.75 / 21.87 / 64.457.58 / 22.39 / 62.50

Removing Noisy-OR raises ECE and Brier and lowers Accuracy on all four OOD datasets. Accuracy falls from 64.45% to 62.50% in OOD, versus 75.59% to 74.41% in ID, showing a larger OOD accuracy benefit. ID/OOD chain-removal fractions are 48.81%/50.36%; removal occurs in 23.63%/13.87% of cases. Among cases with removal, mean removed-chain counts are 49.67/78.69: OOD redundancy is concentrated in fewer, more redundant cases. OOD removed-chain count correlates positively with paired Brier improvement (Spearman ρ=0.188, p=1.85×10⁻⁵). This concentration is consistent with the larger OOD accuracy benefit from direction-aware aggregation.

Finally, thank you again for your valuable feedback. We have carefully considered all your suggestions and revised the paper accordingly. If our responses and revisions address your concerns, we would be very grateful and respectfully ask you to consider raising your score. Thank you very much!

Dear Reviewer NNam,

As the discussion period is drawing to a close, we would like to follow up on our response to your questions. We revised the definitions of score-qualified candidate chains and P(π) at first use and clarified the polarity of mixed chains (Secs. 4.1–4.2). We also explained the scope of Jaccard-based deduplication with pairwise overlap diagnostics (Sec. V.3), clarified the roles of causes, enables, and prevents (Sec. 4.2), compared exponential decay with a continuous truncated linear alternative (Sec. Q.1), and reported chain-removal and correlation diagnostics for the OOD redundancy question (Sec. Q.3). We fully understand that your time is limited, and we would be grateful if our response could be taken into consideration in your final assessment. If you have any further questions, we would be glad to discuss them.

Thank you very much for your time and effort in reviewing our submission.

Best regards, Authors

Reviewer_fkKrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CHAIN, a causal-temporal hypergraph framework for improving probability calibration in LLM-based event forecasting. It addresses three sources of bias: inappropriate evidence weighting, redundant causal-chain aggregation, and inflexible fusion of evidence-based and LLM-based estimates.

CHAIN introduces: (1) a Causal-Temporal Validity Function (CTVF) that combines causal proximity and temporal recency; (2) direction-aware causal-chain deduplication followed by Noisy-OR aggregation; and (3) adaptive fusion based on causal coverage and directional balance. The final prediction is [ \hat{p}=\alpha(q)P_{\mathrm{causal}}(q)+(1-\alpha(q))P_{\mathrm{LLM}}(q). ]

Experiments across eight forecasting datasets, three LLM backbones, and thirteen main baselines demonstrate consistent improvements in ECE, Brier score, and accuracy. The paper further provides ablations, OOD evaluations, theoretical analyses, and robustness studies.

Critical strengths
  1. Well-motivated framework. The decomposition into evidence weighting, aggregation, and fusion provides a coherent approach to diagnosing structural sources of forecasting miscalibration. Each component addresses a distinct modeling concern.

  2. Strong empirical performance. Tables 1 and 2, together with Appendix H, demonstrate consistent improvements across all 24 dataset--backbone configurations. The gains extend beyond ECE to Brier score and accuracy, suggesting improvements in overall predictive quality.

  3. Comprehensive experimental validation. The paper includes component ablations, comparisons with modern post-hoc calibrators, alternative calibration estimators, graph-scale sensitivity, extraction-model sensitivity, and causal-edge perturbations. The supplementary statistical analyses further strengthen the empirical evidence.

  4. Reproducibility and temporal controls. The appendices document prompts, algorithms, hyperparameters, retrieval settings, and computational complexity. Test-record exclusions, temporal admissibility rules, and leakage-safe calibration protocols address important evaluation concerns.

  5. Practical interpretability. Explicit causal chains, evidence polarity, and question-dependent fusion weights offer a more inspectable forecasting process than direct confidence elicitation or opaque post-hoc transformations.

Critical weaknesses
  1. Insufficient isolation of the structural calibration mechanism. The LLM-side estimator already receives hypergraph-derived context, introducing an additional improvement mechanism independent of CTVF, Noisy-OR, and causal fusion. Appendix O reports that 49.22% of evaluation cases have no retained causal chains, giving α(q)=0\alpha(q)=0 and p^=PLLM\hat{p}=P_{\mathrm{LLM}}. Although Figure 6 examines fixed fusion weights, a separate analysis of cases with k=0k=0 and k>0k>0, including a graph-context-matched LLM-only comparison, is needed to establish how much improvement comes specifically from causal aggregation.

  2. Limited theoretical justification for calibration improvements. Proposition 1 requires favorable error-ordering and within-group weighting conditions, while Proposition 3 establishes the benefit of adaptive fusion only under restrictive conditional-moment assumptions. These results do not establish that the proposed CTVF or sigmoid fusion rule generally reduces calibration error. Similarly, Proposition 2 proves reduced Noisy-OR accumulation after removing overlapping paths, rather than improved calibration. The theoretical claims should be distinguished more carefully from empirical findings.

  3. Approximate independence remains insufficiently validated. Edge-overlap-based deduplication does not guarantee statistical independence: distinct paths may still originate from correlated reports or represent the same underlying event. Moreover, causal relation types, directions, and strengths are extracted by LLMs without a direct evaluation against independently annotated causal relations. The direction-reversal experiment is informative but does not establish causal extraction correctness.

  4. Limited evaluation scale and qualified OOD generalization. Each dataset contributes only 128 test questions, which limits the precision of finite-sample calibration estimates. Appendix D explicitly characterizes the main OOD setting as cross-source rather than strictly domain-disjoint transfer. The domain-specific corpus control in Appendix R uses only 9--12 documents per dataset, and its accuracy advantage over the strongest retrieval baseline is approximately 0.39 percentage points. Larger, temporally audited evaluations would better establish external validity.

  5. Computational efficiency and fusion sensitivity. Appendix L reports approximately 13,371 tokens per CHAIN forecast, versus 2,751 for five-sample Direct-MO, despite matching the number of LLM calls. Thus, the comparison does not control for token budget. Furthermore, Equation (21), with the default parameters, assigns approximately α(q)=0.289\alpha(q)=0.289 as causal coverage approaches zero from above, but α(q)=0\alpha(q)=0 when no chain exists. The resulting discontinuity and its robustness to weak or spurious evidence deserve further analysis.

Questions for authors
  1. Contribution attribution: Could the authors report ECE, Brier score, and accuracy separately for k=0k=0 and k>0k>0? How does CHAIN compare with a graph-context-matched PLLMP_{\mathrm{LLM}} baseline using the same prompts and inference budget?

  2. Causal reliability: How accurately does the LLM extractor recover causal direction, relation type, and strength? Can the authors evaluate these properties on an independently annotated subset and test deduplication under semantically duplicated but edge-disjoint evidence?

  3. Theoretical assumptions: Can the authors empirically verify the error-ordering and conditional-moment assumptions used in Propositions 1 and 3? How sensitive is calibration to the discontinuity at k=0k=0 and to the fusion parameters α0\alpha_0, Ω0\Omega_0, and ζ\zeta?

  4. Evaluation and efficiency: Can the authors provide a token-budget-matched comparison with strong retrieval and multi-sample forecasting baselines, together with larger-scale temporally audited OOD results? In particular, does the improvement remain substantial on domain-specific corpora?

2 comments

Thank you for your careful review and valuable feedback. We respond point by point below.

W1/Q1. Attribution (Sec. U.1)

The graph-context-matched LLM shares CHAIN's question, cutoff, graph context, prompt and P_LLM output, matching prediction-stage calls and input/output tokens. It returns P_LLM; CHAIN adds causal estimation and adaptive fusion. k counts chains retained after direction-aware deduplication. Entries: LLM/causal/CHAIN; ECE and Accuracy in %.

knECEBrierAccuracy
05046.875/4.167/6.875.2070/.2500/.207066.47/45.83/66.47
>052014.490/23.775/5.054.2121/.2583/.191069.81/70.38/73.46

Causal weight is zero at k=0; at k>0, fusion improves all metrics: paired Brier difference −.0211 and Accuracy gain 3.654 points.

W3/Q2. Extraction and dependence (Secs. V.2–V.3)

Eight annotators audited a graph-construction subset; two annotated each structure independently, and disagreements were adjudicated. Cohen's κ=.79.

CriterionSupport (%)
Source text90.89
Causal validity77.65
Direction78.34
Chain polarity77.92
Relation type77.60
Strength75.31

On fixed trajectories, edge-disjoint semantic aliases yield ECE (%)/Brier/Accuracy (%) of 8.14/.1989/70.02 (original), 9.289/.2031/68.75 (2×), and 10.647/.2103/68.07 (4×). Jaccard measures edge overlap; correlated reports can link disjoint chains. The alias test measures semantic repetition.

W2/Q3. Theory, boundary and sensitivity (Sec. U.2)

Proposition 1 requires error ordering and within-group weighting; Proposition 2 concerns redundant Noisy-OR accumulation; Proposition 3 assumes conditional moments. For r_C=P_causal−y and r_L=P_LLM−y, (D_C,g,D_L,g,D_CL,g) are stratum means of (r_C²,r_L²,r_Cr_L): low/high coverage gives (.2664,.2122,.1964)/(.2002,.1981,.1430). Population counterparts equal latent-error moments plus a common outcome-variance term; these diagnostics do not verify all latent assumptions.

Eq. (21) gives α→.289 as Ω→0+, but α=0 at k=0. Gate and transition metrics are overall results. Eq. (85): ατ=min(Ω/τ,1)min{2αbσ(ζ[Ω−Ω₀]),α₀}; τ is the coverage transition width.

Gate settingECE (%)BrierAccuracy (%)
Original gate8.14.198970.02
Removed gate8.754.202670.02
τ=.01/.05/.108.135/8.522/9.150.1989/.1991/.199870.02/70.31/70.41

Each scan varies only one parameter:

ParameterValuesECE (%)BrierAccuracy (%)
α₀.3/.5/.8/1.08.808/7.873/8.612/8.865.2000/.1987/.2005/.201469.82/70.21/69.92/69.92
Ω₀.1/.5/.79.269/8.643/9.570.2009/.1993/.200668.95/70.12/70.02
ζ1/5/109.361/8.432/9.696.2005/.1987/.200068.85/70.12/70.12

W4/Q4. OOD scale and temporal controls (Secs. P, Q.2)

We expand four OOD datasets to 256 cases each (1,024 total) and prediction cutoffs to 2023-01-01–2026-03-10. Both estimates use cutoff-conditioned graph G_q, excluding t≥t_c evidence. Missing/unparseable times require a verified availability upper bound before t_c and use CTVF fallback v_fb. We checked every cutoff and explicit source publication date; all identified dates preceded their cutoffs.

MethodECE (%)BrierAccuracy (%)
CHAIN15.67.259958.30
Direct LLM18.06.276257.52
LightRAG20.90.283657.32
HyperGraphRAG22.30.293555.37

Against Direct LLM, ECE/Brier decrease by 13.23%/5.90% relatively and Accuracy rises from 57.52% to 58.30%. All four domain-corpus audits passed: retained documents met cutoffs and excluded evaluation questions/answers/labels/outcomes. Methods share documents, cutoffs and exclusions. Against HyperGraphRAG, CHAIN improves ECE/Brier/Accuracy from 12.19%/.2295/62.70% to 8.22%/.2257/63.09%. These ECE (≈32.6% relative reduction), Brier and Accuracy gains persist with domain corpora and shared temporal boundaries.

W5/Q4. Costs and fusion boundary (Secs. L.1, U.2–3)

MethodCalls/caseTokens/caseECE (%)Brier (%)Accuracy (%)
Direct-MO1.0055015.2123.4760.25
Five-sample Direct-MO5.002,75114.9123.0760.84
LightRAG1.001,50816.5824.6559.86
HyperGraphRAG1.001,56518.1124.1959.96
CHAIN5.0013,3718.1419.8970.02

Tokens support retrieval context, auxiliary judgments and forecasting. A common total budget reallocates context/generation lengths or sample counts, changing evidence volume and inference steps. Default costs and performance show this trade-off. Gate and τ scans quantify the boundary; zero coverage returns P_LLM.

For 0<Ω<.1 and .1≤Ω<.25, LLM/causal/fused Brier (%) is 22.19/31.32/20.97 and 20.91/27.28/19.37, respectively. Fusion achieves the lowest Brier in both low-coverage groups, supporting coverage-adaptive weighting when the causal branch performs less well.

Thank you again. We have revised the paper based on your suggestions and respectfully ask you to consider raising your score if our responses address your concerns.

Dear Reviewer fkKr,

As the discussion period is drawing to a close, we would like to follow up on our response to your questions. In particular, we reported (i) a k=0/k>0 attribution against a graph-context-matched LLM baseline (Sec. U.1); (ii) a human audit of extracted causal direction, relation type, and strength, together with an edge-disjoint semantic-alias stress test (Secs. V.2–V.3); (iii) gate, continuous-transition, and fusion-parameter sensitivity, with the theoretical assumptions stated explicitly (Sec. U.2); and (iv) an expanded, temporally audited OOD evaluation on 1,024 cases, together with the domain-specific corpus control (Secs. P and Q.2). We fully understand that your time is limited, and we would be grateful if our response could be taken into consideration in your final assessment. If you have any further questions, we would be glad to discuss them.

Thank you very much for your time and effort in reviewing our submission.

Best regards, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.