acceptodds
Withdrawn from ICLR 2027

: A Unified Framework for Causal Discovery in Event Sequences

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Complex systems such as vehicles, patients, genomes emit discrete event sequences whose operative question is causal, not predictive: which events cause which other events, and which cause higher-level outcomes such as failures or diseases? This question decomposes along two axes: dependency type (event event vs. event outcome) and causal scope (single sequence vs. population), yielding four structurally distinct regimes with different identifiability conditions. No existing method addresses more than one, because all assume multi-stream structure with low vocabulary, and none scales beyond a few hundred event types. We present , a unified framework that resolves all four regimes through a single shared primitive: a pretrained autoregressive model repurposed as an amortized conditional independence testing engine requiring no task-specific retraining. We establish a prediction–causality duality: the model's excess cross-entropy simultaneously bounds causal identification error across all four regimes, so that every improvement in next-token prediction tightens causal guarantees for free. On nonlinear SCMs (vocabularies up to 8,000 types) and real-world vehicle diagnostic logs (29K event types, 474 failure outcomes), is the first method to populate all four regimes at scale with a single frozen backbone. Existing methods are either inapplicable, inaccurate, or computationally intractable in this setting.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
causal discovery, event sequences, autoregressive density estimation, high-dimensionality

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_fYxMOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SEQ2CAUSE, a framework for causal discovery in discrete event sequences with large vocabularies. It organizes the problem into four regimes: event-to-event versus event-to-outcome discovery, each at the level of an individual sequence or a population of sequences. A pretrained autoregressive event model, together with an outcome head, supplies conditional probabilities for information-gain and conditional-independence calculations. The framework uses randomized intermediate events for lagged event effects and an error-corrected minimum-description-length criterion to aggregate evidence into population graphs. The theoretical contribution is a claimed connection between autoregressive prediction error, conditional-mutual-information estimation, and causal graph recovery. Experiments include nonlinear synthetic sequence generators with vocabularies up to 8,000 and vehicle diagnostic data with 29,100 event types and 474 outcomes. The reported results are promising, including 91.3% F1 for the synthetic sample-level task and 56.8% F1 for population-level outcome boundaries. My main concern is whether the probability comparisons and available error estimates support the specific causal objects and guarantees claimed.

Critical strengths

S1 A useful computational approach to a difficult event-sequence setting. The paper addresses a practical setting in which the event vocabulary is large, sequences are relatively short, and many classical conditional-independence procedures have poorly populated contingency tables. Reusing a learned sequence distribution across many tests is a sensible way to amortize density estimation. The four-regime organization also helps distinguish questions that are often conflated, such as a time-indexed dependency in a particular trace and a type-level dependency across traces. The parallel construction in Appendix G provides a concrete implementation idea rather than only a conceptual analogy between prediction and causality. If the causal interpretation can be established, this could make structure exploration considerably more accessible for systems with rich event logs. S2 The experiments provide encouraging evidence of practical signal and investigate meaningful failure modes. Table 2 compares the proposed sample-level event method with alternatives using the same LLaMA backbone, which helps separate the proposed scoring procedure from representation quality. The vehicle outcome experiments are also relevant to the intended application and go beyond a small synthetic demonstration. The difference between corrected and uncorrected population aggregation is substantial: the reported synthetic population precision increases from 4.2% to 81.5%. The vocabulary, sequence-length, mediator-sampling, and context-truncation analyses help characterize computational behavior. These findings make the method worth investigating even if the present theoretical claims need revision. I view the empirical evidence as support for a useful dependency-scoring procedure; whether it validates direct causal structure requires the clarifications below.

Critical weaknesses

W1 The proposed tests do not yet establish recovery of the direct causal objects defined in the paper. The most concrete problem is in the event-to-outcome regime. Section 4.3 states that intermediate-event randomization is unnecessary because outcomes have no children or spouses. However, an outcome being a sink does not eliminate mediation by later events. Consider an unconfounded, temporally ordered chain in which the first event causes the second event, and the second event causes the outcome, with no direct edge from the first event to the outcome. With an exact probability model, observing the first event still changes the outcome distribution relative to the preceding prefix. Consequently, the sequential information-gain test can be positive even though that event is not in the outcome's parent set or Markov boundary. Conditioning on the second event would remove this association, but conditioning only on the history before the candidate event does not. This counterexample satisfies the autonomous event chain and terminal-outcome structure assumed in Equation 2, so it cannot be dismissed solely by invoking the two-chain factorization. The proof in Appendix D.4 does not resolve this issue. It writes the conditioning history as the entire prefix before the target time while also testing an earlier event contained in that history; taken literally, the candidate indicator is then already determined by the conditioning set, and the conditional mutual information is zero for parents as well as nonparents. If the intended conditioning set instead stops before the candidate, the mediated-outcome example above applies. The event-to-event implementation in Appendix G, which changes prefixes and randomizes intermediate tokens, also needs an explicit connection to the conditional-independence statement used in the theorem. These are consequential distinctions between a direct parent, an ancestor, and an informative observation. They affect both the claimed sample Markov boundaries and the root-cause traversal in Corollary 1. I would need a precise conditioning/intervention definition and a corrected soundness argument before accepting the central identification claim. W2 The prediction-error guarantees are not adequately connected to the available diagnostics or to the implemented decision rules. Assumption 1 bounds the event model's next-event conditional KL error. It does not explicitly bound the separately trained outcome conditional probabilities, although Theorems 1 and 2 are presented as covering both event and outcome targets. A perfect event predictor does not by itself make the outcome head accurate. In addition, Equation 8 and Proposition 1 average over histories generated by the model, whereas the proof of Theorem 1 in Appendix D.3 explicitly treats histories as fixed and says they are not sampled from the model's joint distribution. The relevant distributions, and any accumulation of error across a generated history, therefore need to be specified consistently. The practical calibration problem is equally important. Appendix H.2 estimates the unknown entropy floor by the best converged validation loss. If that checkpoint retains approximation error, subtracting its loss removes the residual error rather than certifying an upper bound on it. A dataset-average validation loss also does not certify the local error for each target and conditioning context, particularly under the randomized histories queried by the algorithm. Proposition 2 establishes that a binary projection cannot increase KL divergence; it does not establish the approximately frequency-proportional error allocation used in the rare-event example on page 18. Finally, the sample experiments report small fixed thresholds without the corresponding certified local errors, while the theorem uses an error-derived threshold, and Equations 12 and 14 appear to use inconsistent conventions for whether the MDL coding penalty has already been subtracted. These gaps matter because soundness and calibration without additional tuning are central contributions. The empirical success of a threshold would still be interesting, but it should be distinguished from a verified error-controlled causal-discovery rule.

Questions for authors

Q1 Direct outcome parents — clarification and a minimal validation. For the simple mediated chain described in W1, what exact conditional distributions does the implemented event-to-outcome test compare, and which events enter the recovered boundary? Please show the result with an exact oracle, so that model approximation cannot explain any discrepancy. If the method is intended to identify ancestors or sequential information contributions rather than direct parents, how should Definitions 1–2, Theorem 2, and Corollary 1 be revised? Q2 Theoretical and implemented estimands — clarification. Please specify the conditioning set, distribution over history particles, intervention distribution, and fixed realized values for each regime. In particular, how are the cause-containing conditioning set in Appendix D.4, the sampled histories in Equation 8, the fixed-history argument in D.3, and the staircase comparisons in Figure 6 reconciled? This would materially affect my assessment of whether the proof applies to the reported algorithm. Q3 Error estimates and actual thresholds — clarification and reporting from existing experiments. What assumption controls the outcome-head error, and how are per-target errors estimated or upper-bounded without knowing the true distribution? Please distinguish raw KL error from the normalized oracle score in Equation 42 and report the thresholds actually used for Table 2 and Figure 2. Is the approximately frequency-proportional binary-error scaling an additional assumption? If the theoretical bound was not used directly, please state which claims remain empirical. Q4 Population objective — clarification. Does the quantity on the left of Equation 14 denote the raw accumulated log-likelihood gain or the penalized gain defined in Equation 12? Please state the inclusion criterion in one convention. Also, Appendix D.5 drops under-resolved probability terms: is the resulting guarantee for the full-data objective or only for the retained positions, and why does the stated MDL gain sum over matched positive occurrences rather than all affected observations? Q5 What is validated by the reported graphs — clarification of existing evaluation. Appendix H.1 says that evaluation is always on a summary graph, whereas the main claims include recovery of sample-time graphs. Could you identify which reported numbers validate each object and provide the time-edge result if it already exists? For the vehicle boundaries, please explain the expert annotation protocol, which parts are incomplete, and how unlabeled candidate causes are treated in precision and recall. This determines whether the results assess direct causal boundaries, broader diagnostic relevance, or both.

Reviewer_5T34OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Seq2Cause, a unified causal‑discovery framework for high‑dimensional discrete event sequences. It uses a pretrained autoregressive model to estimate conditional mutual information (CMI), combined with the minimum description length (MDL) criterion for sample‑ and population‑level causal‑structure learning. The studied problem is practically meaningful, and experiments demonstrate decent effectiveness and scalability. Still, its core causal‑identification method and theoretical guarantees require further improvement.

Critical strengths
  1. This work investigates the challenging causaldiscovery problem for high dimensional discrete event sequences, which carries practical application value.
  2. It handles four causal discovery tasks in a unified manner and reuses probability estimates via pretrained autoregressive models, demonstrating moderate innovation.
Critical weaknesses
  1. In Section 4.3, CMI tests for X→Y may mistake indirect effects for direct causal relations. Synthetic causal chains such as A‑to‑B‑to‑Y are suggested to verify the correctness of Markov Boundary recovery.
  2. In Equations (29)–(30) of Appendix D.4, candidate events appear to be already contained within the conditioning set , which would make CMI identically zero. Please clarify the definition of conditioning sets and their correspondence with the practical algorithm.
  3. Assumption 1 imposes constraints on event prediction distributions, yet X→Y also relies on the outcome prediction module. Please elaborate how the event‑prediction error bounds the estimation error of outcome conditional probabilities.
  4. Existing experiments evaluate different tasks with synthetic SCMs and real‑world vehicle data respectively. Additional synthetic experiments with ground‑truth direct causal edges are recommended to uniformly assess structure‑recovery performance across all four tasks.
Questions for authors

See Weakness

Reviewer_Yp4LOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a framework for causal discovery from sequences of categorical events using an arbitrary pretrained autoregressive (AR) model. The authors consider four regimes of causal discovery: event-to-event and event-to-outcome discovery, each at the individual-sequence (sample) and dataset-wide (population) levels. At the sample level, they use conditional mutual information (CMI) tests, including simulated interventions on intermediate events to distinguish direct from indirect effects. At the population level, they aggregate predictive evidence across sequences using a Minimum Description Length (MDL) criterion, which penalizes graph complexity by assigning a cost to adding edges. The paper also presents theoretical results claiming that the accuracy of the AR model determines how reliably causal relationships can be identified, both within individual sequences and across the population.

Critical strengths
  1. Scalability: The framework reuses pretrained AR models for causal discovery without retraining. The experiments demonstrate its applicability to synthetic data with up to 8,000 event types and real-world vehicle diagnostic data with 29,100 event types and 474 outcomes. This is a potentially valuable practical contribution, particularly in settings where conventional causal discovery methods struggle computationally.

  2. Promising empirical results: The proposed error-corrected MDL criterion (i.e. using a sparsity penalty term) substantially improves the reported precision of population-level causal discovery.

Critical weaknesses
  1. Serious theoretical concerns: The main theoretical guarantees are not adequately justified. Theorem 1 has inconsistencies between the estimator and its proof, while Theorem 2 does not establish the identification of direct causal relationships. The population-level error correction also relies on questionable assumptions about model approximation errors. These issues undermine a central contribution of the paper.

  2. Limited novelty and insufficient comparisons: The main ideas of using predictive models for causal discovery and connecting event-level and population-level causality have been explored in previous work. The paper does not sufficiently distinguish its contributions from existing methods. Relevant work such as ISAHP (AISTATS 2024) is omitted, and comparisons with methods such as CASCADE are insufficient to establish the claimed advantages.

Questions for authors
  1. Equation (8) samples histories from the learned model, whereas the proof assumes fixed observed histories. Could the authors clarify this inconsistency and explain how the claimed error bound applies to the actual estimator, including the outcome predictor?

  2. The proof uses I(T;Et−ℓ∣X<t)I(T;E_{t-\ell}\mid X_{<t}), but the conditioning set already contains Et−ℓE_{t-\ell}, making this CMI zero. How does the method distinguish direct from indirect causes in event-to-outcome discovery? For example, in A→B→YA\to B\to Y, what prevents AA from being incorrectly identified as a direct parent of YY?

  3. The theoretical guarantees rely on knowing or bounding ϵ\epsilon. However, estimating the unknown entropy H(P)H(P) using the minimum validation loss does not necessarily provide a valid error bound. How do the authors justify their claim that the thresholds require no additional calibration?

  4. Equation (12) already subtracts the MDL complexity penalty (via ctypec_{\text{type}}), but Equation (14) appears to include it again. Could the authors clarify this apparent double counting and explain how Lemma 1 remains valid when poorly resolved observations are excluded?

  5. Could the authors clarify the methodological novelty relative to existing event-level and population-level causal discovery methods, particularly ISAHP (AISTATS 2024)? Also, can they provide meaningful accuracy comparisons with CASCADE on smaller problems where both methods can complete?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.