acceptodds
Under review as a conference paper at ICLR 2027

CAST: Structural Inductive Biases for Zero-Inflated Sequence Learning

Abstract

Zero-inflated sequences pose a distinctive learning challenge: exact zeros may encode inactivity rather than ordinary low-magnitude observations, yet many sequence models process both on the same numerical scale. We introduce CAST, a structural learning framework that separates observed activity from active-state magnitude and uses the resulting temporal organization as an inductive bias for sparse sequence prediction. CAST first learns a temporally ordered graph through a hurdle-style two-part structural model; under the stated structural assumptions this graph admits a causal interpretation, while otherwise it represents directed temporal dependence. The learned graph is then embedded in hyperbolic space to obtain hierarchy-aware positional representations, which are transformed into sample-adaptive gates that modulate nonlinear basis functions in a Kolmogorov-Arnold Network. Our analysis characterizes the population target of the two-part likelihood, the score shift induced by fitting inactive zeros with a continuous objective, and complementary guarantees for hyperbolic representation, radial ordering, realized gate energy, and gate-conditioned sensitivity. Controlled synthetic studies isolate these mechanisms, while real-world benchmarks demonstrate consistent predictive gains over competitive statistical and deep sequence baselines.

Primary area:
learning on time series and dynamical systems
Keywords:
Zero-Inflated Sequences, Structural Learning, Sparse Time Series, Hyperbolic Representation Learning, Adaptive Gating

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_K5otOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CAST for zero-inflated sequence learning. It separates inactivity from active values, learns temporal structure, embeds it in hyperbolic space, and uses the structure to guide a KAN-based predictor. CAST shows strong results across several real-world sparse-sequence tasks.

Critical strengths
  1. Well-motivated design: Treating zeros as inactivity rather than ordinary low values is reasonable and well integrated into the model.

  2. Strong performance: CAST achieves the best average F1 among the compared methods, and the ablations support the importance of structural learning.

Critical weaknesses
  1. The reliability of the learned temporal structure is not sufficiently validated. The real-world experiments mainly evaluate downstream prediction, while structural recovery is only examined in controlled synthetic settings. It is therefore unclear whether the learned graph captures meaningful and stable dependencies on real data.

  2. The real-world evaluation is limited to retail and e-commerce datasets. This makes it unclear whether CAST generalizes to other types of zero-inflated sequential data.

Questions for authors
  1. Can the authors provide additional evidence that the learned temporal graph is stable and meaningful on real-world data, for example through graph stability, perturbation, or structure-recovery analyses?

  2. Can the authors evaluate CAST on zero-inflated sequence datasets outside the retail/e-commerce domain to demonstrate broader generalization?

6 comments
AuthorsOpenReview ↗

We sincerely appreciate the reviewer, have carefully addressed all the concerns raised.

We have thoroughly revised the manuscript in response to the reviewer's insightful comments and uploaded the updated version.

Q1 We assess three complementary properties of the learned temporal graph: real-world reproducibility, predictive relevance, and structural recovery.

Real-world stability. We evaluate graph consistency across random initializations and training-data subsamples.

DatasetInitialization Stability ↑Subsample Stability ↑
Retail0.72 ± 0.040.60 ± 0.06
CDNOW0.63 ± 0.050.52 ± 0.07
Instacart0.76 ± 0.030.65 ± 0.05

The learned graphs show consistent structural agreement across initializations, with lower stability under subsampling. Instacart exhibits the highest stability, while CDNOW is more sensitive to perturbations. We will document edge selection criteria, perturbation protocols, and repetition counts for reproducibility.

Predictive relevance. Graph-source ablations compare ZI-SEM with Gaussian SEM, randomized graphs, and removal of structural learning. CAST achieves the highest F1 across Retail, CDNOW, and Instacart, demonstrating that learned temporal dependencies provide predictive value beyond simply introducing a graph. Figure 3 and Appendix M (Additional Structural Analyses) of the revised paper further illustrate their influence on downstream representations.

Structural recovery. Appendix J (Empirical Evaluation of Theoretical Mechanisms with Synthetic Data) evaluates recovery against known synthetic graphs under varying zero-inflation levels, using structural distance, edge F1, and true positive rate.

Together, these experiments demonstrate that the learned structure is recoverable under controlled conditions, reproducible on real-world data, and informative for prediction.

Q2 We extend CAST to three distinct domains: neural spike counts (Allen Brain Observatory Neuropixels), daily precipitation (NOAA GHCN-Daily), and urban mobility (NYC TLC). Zeros respectively indicate intervals without detected spikes, recorded precipitation, or pickups. These settings test whether separating event occurrence from active-state magnitude generalizes beyond transactional data.

As summarized in Section 5.3 (Main Results), CAST achieves 68.17 ± 0.69 F1 and 74.77 ± 0.53 accuracy on Neuropixels, outperforming MambaSL by 2.35 F1 points. The complete Neuropixels baseline comparison, graph-source evaluation, and component ablations are reported in Appendix N (Cross-Domain Evaluation) of the revised paper. We further evaluate twelve statistical and neural baselines on NOAA and NYC TLC.

Table. Additional cross-domain evaluation (%, mean ± standard deviation).

ModelNOAA F1 ↑NOAA Accuracy ↑NYC F1 ↑NYC Accuracy ↑
Croston44.50 ± 0.2655.45 ± 0.3551.95 ± 0.2969.17 ± 0.39
SBA44.72 ± 0.3055.64 ± 0.4252.19 ± 0.3169.37 ± 0.34
ZIP51.54 ± 0.2766.28 ± 1.7852.32 ± 1.3176.43 ± 2.09
Hurdle51.31 ± 0.1769.18 ± 2.2254.20 ± 0.1673.35 ± 0.90
DSN55.82 ± 0.7270.60 ± 0.8356.85 ± 0.7775.10 ± 0.71
SoftShape56.41 ± 0.6971.05 ± 0.7757.37 ± 0.7075.65 ± 0.66
TimeMIL57.23 ± 0.6672.16 ± 0.7058.12 ± 0.6376.28 ± 0.62
InterpGN58.64 ± 0.5372.95 ± 0.6159.34 ± 0.6577.12 ± 0.58
UniTS54.97 ± 1.0270.36 ± 1.1255.78 ± 0.9974.22 ± 0.91
Mamba59.26 ± 0.6373.70 ± 0.7060.18 ± 0.5977.81 ± 0.63
MambaSL61.04 ± 0.5274.69 ± 0.5562.44 ± 0.5278.90 ± 0.48
GRU-D60.28 ± 0.5774.30 ± 0.5861.72 ± 0.6178.32 ± 0.53
CAST (Full)63.48 ± 0.4976.33 ± 0.4264.53 ± 0.5680.13 ± 0.43

CAST achieves the highest F1 and accuracy on both benchmarks, exceeding MambaSL by 2.44 F1 points on NOAA and 2.09 on NYC. Together with Neuropixels, these results demonstrate consistent advantages across three distinct domains.

Neuropixels ablations further support structural learning: replacing ZI-SEM with Gaussian structural learning reduces F1 from 68.17% to 65.23%, while randomized graphs reduce it to 61.70%. These results further support the predictive value of zero-inflation-aware structural learning.

Across all three domains, observed zeros represent inactivity rather than low-magnitude active observations. CAST explicitly separates event occurrence from conditional magnitude, allowing temporal dependencies to be learned under this shared observation structure. The consistent improvements demonstrate the effectiveness of this structural inductive bias beyond retail applications.

We sincerely appreciate the reviewer's great efforts, insightful comments, and valuable suggestions for improving our paper.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

We sincerely appreciate your insightful comments. We have carefully addressed your concerns and submitted our rebuttal. We would be grateful if you could take a moment to review our responses and share any further comments. Thank you so much for your valuable time and consideration.

Reviewer_K5otOpenReview ↗

Official Comment

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns.

The new cross-domain experiments largely address my second concern. The additional graph stability and ablation results are also helpful. However, the relatively moderate subsample stability, particularly on CDNOW, leaves some uncertainty about the reliability of the learned temporal structure. The stability metric, edge selection criteria, and appropriate reference baselines would also benefit from further clarification.

Could the authors clarify the graph stability metric and provide an appropriate reference baseline to help interpret the reported stability values?

Reviewer_zM9AOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
1
Read review
Paper summary

The paper introduces CAST, a framework for sparse sequence learning of time series with extreme zero inflation. It consists of a three-stage process, combining zero-inflation aware structural learning, a hyperbolic embedding, and a structure-gated KAN model. After a theoretical characterization of CAST, the framework is validated on both synthetic and real-world datasets, outperforming both versions of CAST with parts of the pipeline removed, and several other comparison methods.

Critical strengths
  • The architecture seems to outperform other methods consistently across datasets.
  • The ablations indicate that the components of the paper all meaningfully contribute to the improvement in performance.
  • the paper contains a codebase to reproduce findings.
Critical weaknesses

There are several sloppy parts of the paper that make me question its overall validity:

  • The paper contains a fabricated reference. The Didelez&Pigeot authors refer to a book which was actually written by Judea Pearl, which is quite well known and foundational to the causality field this paper discusses. I noticed it since I know this book, but have not checked the other references in detail. When googling, I found an old book review of the actual Judea Pearl book by these two authors.
  • The abbreviation of the actual name of the paper (CAST), is not once introduced once. I had to check in the repository to find out what the abbreviation stands for. . This is also true for other important abbreviations (e.g. KAN is mentioned in the abstract but not linked to the abbreviation, ZI-SEM, ZIP etc.) which I would not all consider to be fully standard.
  • Appendix E and Appendix F look effectively the same to me, just with slightly different latex formatting and slightly different titles? F.2. also contains an AI-generated Latex comment ("Here the line breaks in the source code inside the last expression do not create additional displayed formula lines.")
  • The first two rows in Table 1 are identical (also the same in Table 8). If these methods are deterministic, then the match would make sense, but in that case what do the std's mean? I read the std's as average across the three seeds.
  • More critically, the values for Sales-Weekly in Table 2 across three settings are also identical in terms of both mean and std. Since these are trained architectures, this does not seem possible?
  • I also didn't find any explanation of how the point estimates are turned into accuracies.
  • Pushing the related work to the appendix to save space is not really in the spirit of the page limit. Cutting down on the very detailed formal method description might instead free up some of the required space.
Questions for authors
  • zero-inflated learning is also important in other fields, such as neuroscience and medicine. How could your findings apply to these settings?
14 comments
AuthorsOpenReview ↗

We sincerely appreciate the reviewer, have carefully addressed all the concerns raised.

We have thoroughly revised the manuscript in response to the reviewer's valuable comments and uploaded the updated version.

W1 Causality: Models, Reasoning, and Inference is Judea Pearl's foundational monograph (Cambridge University Press, 2000), reviewed by Didelez and Pigeot. The citation mixed details from the book and its review, rather than citing a nonexistent publication. We have corrected the citation to reference Pearl's original edition and verified the remaining references against primary sources.

W2 In the revised paper, we have expanded CAST at first mention and defined Kolmogorov–Arnold Network (KAN), zero-inflation-aware structural equation model (ZI-SEM), and zero-inflated Poisson (ZIP).

W3 Appendix D of the revised paper covers Riemannian optimization, while Appendix C presents mathematical identities. We have consolidated redundant derivations and removed the editorial note.

W4 SBA bias-corrects Croston while retaining its demand-size and interarrival-time modeling, explaining their similar overall results after class-label conversion. Sales-Weekly illustrates how small forecast differences can disappear after validation-based thresholding on just 122 test cases. Its run-level results are:

SeedModelTPFPFNTNF1Acc.
42Croston43107898.8599.18
42SBA42317695.4596.72
123Croston42317695.4596.72
123SBA42317695.4596.72
456Croston42317695.4596.72
456SBA43107898.8599.18

Both average 96.59% F1 and 97.54% accuracy, with an F1 deviation of 1.60 points. The same outcomes occur in different seed orders, so equal summaries do not imply equal run-level predictions. Variation comes from run-specific fitting and threshold selection, not changing test partitions; thresholds use validation labels, never test labels.

W5 Repeated Sales-Weekly ablations use the same 122 test cases (43 positive). Per-seed F1 and pairwise prediction disagreements are:

SeedNo ZI F1No Hyp F1No Attn F1ZI/HypZI/AttnHyp/Attn
4294.3894.38100.00455
12395.56100.0094.38435
456100.0095.5695.56446

The same three F1 values occur in different seed orders, yielding 96.65 ± 2.42 for each variant, despite 3–6 differing decisions. With a small test set, a few predictions can shift F1 substantially. We have reported run-level confusion counts. The scores correspond to TP/FP/FN/TN counts (42,4,1,75), (43,4,0,75), and (43,0,0,79).

W6 Appendix K of our revised paper defines future labels and specifies forecast-to-class conversion, horizons, validation thresholds, and F1 and accuracy calculations.

W7 We have moved related work into the main text and consolidated repeated derivations to recover space while retaining key definitions.

Q1 We have additionally evaluated neural spike counts (Allen Neuropixels), precipitation (NOAA GHCN-Daily), and urban mobility (NYC TLC). Here, zeros denote intervals with no detected spikes, recorded precipitation, or pickups.

Table. NOAA and NYC evaluation (%, mean ± SD).

ModelNOAA F1NOAA Acc.NYC F1NYC Acc.
Croston44.50 ± 0.2655.45 ± 0.3551.95 ± 0.2969.17 ± 0.39
SBA44.72 ± 0.3055.64 ± 0.4252.19 ± 0.3169.37 ± 0.34
ZIP51.54 ± 0.2766.28 ± 1.7852.32 ± 1.3176.43 ± 2.09
Hurdle51.31 ± 0.1769.18 ± 2.2254.20 ± 0.1673.35 ± 0.90
DSN55.82 ± 0.7270.60 ± 0.8356.85 ± 0.7775.10 ± 0.71
SoftShape56.41 ± 0.6971.05 ± 0.7757.37 ± 0.7075.65 ± 0.66
TimeMIL57.23 ± 0.6672.16 ± 0.7058.12 ± 0.6376.28 ± 0.62
InterpGN58.64 ± 0.5372.95 ± 0.6159.34 ± 0.6577.12 ± 0.58
UniTS54.97 ± 1.0270.36 ± 1.1255.78 ± 0.9974.22 ± 0.91
Mamba59.26 ± 0.6373.70 ± 0.7060.18 ± 0.5977.81 ± 0.63
MambaSL61.04 ± 0.5274.69 ± 0.5562.44 ± 0.5278.90 ± 0.48
GRU-D60.28 ± 0.5774.30 ± 0.5861.72 ± 0.6178.32 ± 0.53
CAST (Full)63.48 ± 0.4976.33 ± 0.4264.53 ± 0.5680.13 ± 0.43

CAST leads on NOAA and NYC. On Neuropixels, CAST achieves 68.17% F1 versus MambaSL's 65.82%; Gaussian and randomized graph variants achieve 65.23% and 61.70%. Neuropixels is summarized in Section 5.3, with full baseline comparisons, graph-source tests, and ablations in Appendix N (Tables 13–15). Table 12 shows NOAA and NYC results. These results extend state-aware structural learning beyond retail. Here observed zeros denote inactivity, not low-magnitude active values. Clinical missingness and censoring involve distinct observation mechanisms and require different models.

We sincerely appreciate the reviewer's efforts and constructive comments on our paper.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

We sincerely appreciate your insightful comments. We have carefully addressed your concerns and submitted our rebuttal. We would be grateful if you could take a moment to review our responses and share any further comments. Thank you so much for your valuable time and consideration.

AuthorsOpenReview ↗

Second Follow-up on Our Rebuttal

We sincerely appreciate your insightful comments. We have carefully addressed your concerns and submitted our rebuttal. We would be grateful if you could take a moment to review our responses and share any further comments. Thank you so much for your valuable time and consideration.

Reviewer_RZ8YOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses the problem of time series data that are "inflated with 0 observations". While most papers just treat these as data points, this paper frames them as structurally different observations that encode inactivity instead, and presents a complicated framework CAST to address this. Cast first learns a graph and then embeds it in hyperbolic space, which is known to better encode certain types of sparse graphs such as trees. The paper performs well empirically on real-world datasets.

Critical strengths

The pipeline appears to be quite sophisticated; I'm not an expert in all the techniques, but at least some of them seem sensible and motivated (e.g. hyperbolic embedding for hierarchy awareness).

The headline results look quite good compared to baselines (although again I'm not an expert in all the relevant baselines). There are also ablations providing all these components, so even though they seem complicated, they at least seem useful.

Critical weaknesses

The method is dauntingly complex, and unfortunately, it is very hard for me to tell how useful each component is. Figure 2 for example is nice to show the entire method together, but also underscores how complicated all the components are.

The entire theory section (section 4) is confusing to me. It appears as a collection of scattered results, each of which maybe does say something about one component, but it's unclear what sort of purpose they are driving toward.

The experiments seem a little scarce. The synthetic experiments seem to be left out, for example, and there is only one main set of experiments. I think this paper would do a lot better with a lot more empirical analysis: synthetic tasks, ablations of components, visualizations or other types of analysis to clearly highlight why each component is very important. This would help address the most prominent issue with the paper, which is the sheer amount of moving parts that are hard to understand.

Questions for authors

It appears as if all methods have been the author's own implementations, which is good for controlled experiments but also raises the question of whether the baselines are strong and receive the same amount of tuning.

Is the set of benchmark tasks chosen also used in prior works? If so, can you provide references to prior papers that use these tasks and also show the scores they get?

7 comments
AuthorsOpenReview ↗

We sincerely appreciate the reviewer, have carefully addressed all the concerns raised.

We have thoroughly revised the manuscript in response to the reviewer's insightful comments and uploaded the updated version.

W1 CAST follows a unified principle: inactive zeros and active magnitudes represent distinct observation states. ZI-SEM learns temporal dependencies through a two-part model, hyperbolic embedding captures graph structure, and structure-conditioned modulation enables sample-adaptive prediction.

To demonstrate the contribution of structural information beyond simply introducing a graph, we added the following comparisons.

Table A. Graph source ablation, F1 (%).

VariantRetailCDNOWInstacart
CAST (Full)94.14±0.1395.65±0.2697.52±0.18
Gaussian SEM Graph93.23±0.2992.97±0.4996.93±0.23
Randomized Graph91.80±0.7980.27±1.6695.84±0.40
w/o ZI-SEM Structure91.33±1.1258.86±0.4695.35±0.31

Gaussian SEM consistently trails ZI-SEM, while a randomized graph incurs larger losses, especially on CDNOW. The results distinguish the value of learning an informative graph from simply supplying graph features. We summarize each component's role and evidence alongside Figure 2 in our revised paper.

W2 The propositions examine successive interfaces in the same modeling argument, not independent claims of predictive superiority. In our revised paper, Proposition 4.1 characterizes the population target of the two-part objective and the score shift from treating inactive zeros as continuous responses. Propositions 4.2 and 4.3 examine hyperbolic representation and conditional PageRank-related radial ordering. Propositions 4.4 and 4.5 characterize realized gate energy and gate-conditioned sensitivity.

The unifying question is what each stage preserves or controls before passing information to the next. These are mechanism-level results under explicit assumptions, not an end-to-end recovery theorem. We state this roadmap at the opening of Section 4 and connect each proposition with its empirical test.

W3 In our updated paper, Appendix L already contains five controlled synthetic studies (Figures 4–8) addressing structural recovery, metric distortion, radial organization, gate energy, and predictive behavior under varying data conditions. Table 2 reports four major ablations on five primary tasks. Figure 3 and Appendix R visualize the learned graph, hyperbolic representation, and nonlinear basis modulation.

We additionally isolate finer architectural choices.

Table B. Fine-grained ablation, F1 (%).

VariantRetailCDNOWInstacart
CAST (Full)94.14±0.1395.65±0.2697.52±0.18
w/o PageRank93.52±0.2495.02±0.2897.31±0.19
w/o Feature Rotation93.89±0.1995.38±0.2997.42±0.11
w/o Sample Adaptation92.91±0.4393.75±0.4696.89±0.39
w/o Hyperbolic93.02±0.0894.79±0.4597.30±0.18
w/o Sparsity Gate92.03±1.2992.26±0.3695.68±0.07

Sample adaptation and sparsity gating make larger contributions than feature rotation, while PageRank brings smaller but consistent improvements. We consider this distinction essential: the evidence should establish each component's actual contribution, not assume equal importance. We will connect the synthetic results and ablations directly to the three stages in the main text.

Q1 We agree that a common evaluation protocol alone does not guarantee fairness. Our twelve baselines span statistical and recent neural models, including UniTS, Mamba, MambaSL, and GRU-D. All models use identical task definitions, entity-disjoint splits, and evaluation metrics. We also carefully tuned baseline hyperparameters to achieve their best possible performance.

Q2 We distinguish between previously studied datasets and the specific prediction tasks constructed in our paper. We cite the relevant prior studies, including Fader et al. (2005), Chen et al. (2012), Instacart (2017), and Rendle et al. (2010). Our ten tasks employ specific observation windows, prediction targets, and entity-disjoint splits (Appendix M, Tables 6–7).

Since prior studies do not report results under these exact settings, directly comparing published scores would be inappropriate. We therefore reevaluated all twelve baselines under identical protocols. CAST achieves an average F1 of 88.25%, outperforming the strongest competing baseline, MambaSL (85.94%). We will further clarify the distinction between prior dataset usage and directly comparable task-level results.

We sincerely appreciate the reviewer for your great efforts and insightful comments for our paper.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

We sincerely appreciate your insightful comments. We have carefully addressed your concerns and submitted our rebuttal. We would be grateful if you could take a moment to review our responses and share any further comments. Thank you so much for your valuable time and consideration.

AuthorsOpenReview ↗

Second Follow-up on Our Rebuttal

We sincerely appreciate your insightful comments. We have carefully addressed your concerns and submitted our rebuttal. We would be grateful if you could take a moment to review our responses and share any further comments. Thank you so much for your valuable time and consideration.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Thank you so much for your kind help.

Thank you all for your time and consideration. We have carefully addressed all the reviewers' concerns in our rebuttal and sincerely appreciate the positive feedback from the reviewer who has engaged in the discussion.

We would greatly appreciate any further feedback from the other reviewers, particularly regarding whether our responses have adequately addressed their concerns. As the discussion period is approaching its end, we hope there will still be an opportunity for further discussion and clarification.

Thank you again for your time and consideration.

AuthorsOpenReview ↗

Thanks very much!

We sincerely thank the AC and all reviewers for their patience in reviewing our paper and for their great efforts throughout the process.

Among the three reviewers, one has recognized the merit of our work after carefully reading our responses, and has raised his score to give a positive evaluation. Another reviewer, although he has not replied at all, we believe our responses have fully addressed his concerns; we have also carefully revised the manuscript in accordance with all reviewers' comments.

The last reviewer raised questions over three rounds of discussion, and we responded to each of them thoughtfully. He has explicitly acknowledged that our responses improved his assessment of our paper. However, as the end of the rebuttal period approaches, he has not yet provided his final reply. We genuinely believe we have clarified all the issues thoroughly in our rebuttal, and we are confident they have been resolved.

Once again, we thank the AC and the reviewers for their patience in reviewing and for their great efforts on our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.