acceptodds
Under review as a conference paper at ICLR 2027

OGC-MoE: Online Granger-Causal Mixture-of-Experts for High-Dimensional Nonstationary Time Series

Abstract

Causal discovery in high-dimensional multivariate time series is challenging because the number of candidate relations grows rapidly, while nonstationarity may cause the underlying causal mechanisms to evolve over time. Existing methods often assume a single stable graph or estimate local graphs independently, limiting the reuse of historical knowledge and producing unstable estimates from short windows. We propose Online Granger-Causal Mixture-of-Experts (OGC-MoE), an online graph-level mixture-of-experts framework for causal discovery in high-dimensional nonstationary time series. OGC-MoE maintains a library of causal experts, each pairing a graph-conditioned temporal predictor with a sparse Granger-causal graph. For each incoming window, the model evaluates historical experts using prediction and causal-residual information, refines their routing weights, and combines the selected graphs to represent either a single historical mechanism or a recombination of multiple mechanisms. When the mixture is insufficient, sparse local adaptation is applied to moderate deviations, whereas a window identified as a genuinely unseen state is used directly to construct a new expert. Experiments on synthetic benchmarks and real-world financial data demonstrate the strong and robust causal discovery performance of OGC-MoE.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-series causal discovery, Granger causality, causal nonstationarity, mixture of experts, high-dimensional time series

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_R3SHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OGC-MoE for online Granger causal discovery in nonstationary time series. The method learns predictor–graph experts from historical regimes, then uses prediction errors and residual profiles to route incoming windows and combine expert graphs. Local graph adaptation and expert expansion accommodate changing mechanisms. Experiments cover linear and nonlinear synthetic systems and a real-world financial dataset.

Critical strengths
  1. For online causal discovery under nonstationarity, reusing historical mechanisms while adapting to current observations is a feasible and useful approach. The framework provides a practical way to balance historical knowledge reuse with adaptation to emerging regimes.
  2. The framework integrates expert routing, graph combination, local adaptation, and expert expansion into a coherent pipeline. Synthetic experiments report strong graph-recovery performance, while ablations provide supporting evidence for the main components.
Critical weaknesses
  1. The motivation is insufficiently developed. The paper does not clearly explain why an MoE formulation is needed or establish its advantages over alternative online adaptation strategies.
  2. The selection criteria and sensitivity analyses for historical segmentation, expert training lengths, and online window configurations are insufficient.
  3. Several experimental aspects require further strengthening, including baseline fairness, the long-term benefits of expert expansion, and the reliability of graph combination. Specific concerns are detailed in Questions For Authors.
Questions for authors
  1. What supports the emphasis on high-dimensional causal discovery? Each expert still learns an N*N graph, and constructing new experts retains this quadratic parameter scale. What specific contribution addresses high-dimensional estimation? I suggest removing “high-dimensional” from the title, reducing its emphasis in the introduction, and presenting higher-dimensional settings as supplementary experiments.

  2. How are historical regimes and temporal configurations determined? The synthetic experiments use four predefined historical domains, while the financial data are segmented into four intervals. What justifies these configurations, and how robust is the method to different regime counts or inaccurate boundaries? How are each expert’s training length, the online window length, and the stride selected? Please provide reproducible selection procedures and sensitivity analyses.

  3. Could the baseline training protocol affect comparison fairness? According to Appendix B.8, static-graph baselines are refitted using accumulated historical observations, potentially exposing them to outdated mechanisms. Does OGC-MoE retain its advantage when applicable baselines use recent-window refitting or temporal weighting? Please also clarify the historical segmentation information and hyperparameter tuning budgets available to each method.

  4. Does expert expansion provide sustained benefits on subsequent observations? Table 5 primarily evaluates graph recovery after training a new expert on an observed segment. Does that expert improve performance on subsequent unseen observations, particularly when the new mechanism recurs? Please report long-term expert reuse, unnecessary expansion, and the evolution of expert count, runtime, and memory consumption.

  5. How does graph combination control redundant or erroneous edges? The nonnegative weighted combination in Equation (9) does not automatically eliminate shared false positives and may retain historical edges that are no longer valid. How are such errors controlled, particularly when the mixed graph is accepted without local adaptation?

2 comments

Responses to Weakness 1: Our work initially aimed to analyze dynamic dependencies in financial time series. Financial data are continuously generated and nonstationary, while some relationships may persist or recur. We therefore introduced MoE to preserve structures from different historical states, combine historical information when a single expert cannot adequately explain current observations, and accommodate changes through local adaptation and expert expansion.We have clarified the design motivation and scope of our conclusions in the revision.

Responses to Weakness 2 and Question 2: We added sensitivity analyses of historical partitioning and test window length in Sections 4.2.5 and 4.2.6, respectively.

Responses to Question 1: Our initial focus was on the high‑dimensional and nonstationary nature of financial data, making high dimensionality an important context for this work. Offline, we adopt a graph‑conditioned message‑passing recurrent architecture, which reduces parameter duplication compared with constructing an independent prediction network. Online, historical experts’ predictor parameters and structural matrices remain frozen, and routine optimization primarily updates routing weights. Local structural adaptation or new‑expert training is performed only when needed.

We conducted experiments with 64, 128, and 256 variables. As shown below, the model retains good structural recovery performance. Computational costs are reported in Appendix B.11.

DatasetVariablesAUROCAUPRC
VAR640.971 ± 0.0120.858 ± 0.021
VAR1280.944 ± 0.0150.802 ± 0.018
VAR2560.929 ± 0.0120.730 ± 0.023
Lorenz‑96640.983 ± 0.0150.834 ± 0.017
Lorenz‑961280.978 ± 0.0160.815 ± 0.014
Lorenz‑962560.951 ± 0.0160.715 ± 0.025

Responses to Question 3: We conducted additional experiments retraining the baselines using the current test‑window data. Each baseline uses only observations within the current sliding window, without earlier historical data or historical domain‑partition information.The results are reported in Appendix B.10.

MethodVAR AUROCVAR AUPRCLorenz‑96 AUROCLorenz‑96 AUPRC
eSRU0.5140.0950.6190.092
NGM0.5120.1270.6970.326
PCMCIΩ0.6480.1820.6760.232
CUTS+0.6460.2490.7210.392
TS‑CausalNN0.6620.1950.7190.492
DCD0.7120.3650.8190.582
OGC‑MoE (Ours)0.9440.8020.9780.815

Responses to Question 4: We compared the full method with a setting that prohibits the creation of new experts. As shown in the table below, disabling expansion reduces structural recovery performance on both datasets. Since new experts participate in inference only from subsequent windows onward, these results support the overall benefits of expert expansion for subsequent online structure estimation, complementing the post‑training recovery results on observed segments in Table 5.

DatasetSettingAUROCAUPRC
VARWith expert expansion0.944 ± 0.0150.802 ± 0.018
VARWithout expert expansion0.856 ± 0.0130.748 ± 0.011
Lorenz‑96With expert expansion0.978 ± 0.0160.815 ± 0.014
Lorenz‑96Without expert expansion0.829 ± 0.0180.727 ± 0.012

The number of variables is fixed at 128 in the runtime experiment below (Appendix B.12).

DatasetExpertsOnline runtime per window (min)
VAR43.80
VAR85.64
VAR169.72
Lorenz-9643.87
Lorenz-9685.81
Lorenz-96169.68

Responses to Question 5: Weighted expert combination alone cannot guarantee the removal of erroneous edges shared by multiple experts and may retain historical relationships that are no longer valid. However, offline sparsity regularization discourages unnecessary edges, while expert matching and routing reduce the contributions of incompatible experts. When the mixture cannot adequately explain current observations, local adaptation can reduce the relevant edge weights using current data.

Thank you for the author's response. I believe that the author has still not addressed my concerns regarding the motivation statement and high-dimensional scenarios. In my view, a variable space with thousands of dimensions is what truly qualifies as a high-dimensional scenario. Therefore, I will maintain my score unchanged.

Reviewer_JtGmOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a Granger causality algorithm that can handle non-stationarity. OGC-MoE uses a mixture of experts to handle non-stationarity and presents an online adaptation for new windows that could not be expressed by the different experts learned during offline training, where they use domain labeled data.

Critical strengths

The paper presents an interesting idea for online causal structure learning adaptation. Even though the paper claims different contributions (for me the only novelty is the online adaptation, but the authors claim other novelties that I do not agree with, see weaknesses and questions), I have many concerns regarding the other contributions, the motivation, the soundness and the writing of this paper.

Critical weaknesses

In the abstract, you claimed that other methods estimate local graphs independently which makes them vulnerable and not robust, but OGC-MoE does the same during offline training, where you first have labeled data (I’ll detail this point more below), and you train different fθkf_{\theta_k}, so your argument of not using historical knowledge does not hold. You also don’t use historical knowledge to learn your first domain-specific graphs.

One of your arguments (in the rebuttal) might be that you have a testing set where you use a historical set, but many state-of-the-art models that your paper does not cite or discuss learn domain indices from data, so they do not need train and test validation because they learn them.

The paper lacks very important papers in the field: R-PCMCI [1], Castor [2], SpaceTime [3], LIN [4], Fantom [5], On the Identifiability of Switching Dynamical Systems [6], FlowMSM [7], Causal discovery with endogenous context variables [8] and MCD [9]... Please note that state-of-the-art citations and related work are very important scientifically; they help researchers get a clean overview of the different papers. The authors did not compare to these papers in their settings with instantaneous links, non-Gaussian noise, history-dependent noise and heteroscedastic noise. Also, they did not discuss these papers in their manuscript, explaining the differences, where these models outperform their paper, where their paper performs better, or what it offers that is different from these papers. All these papers treat domain indices as latent variables and learn both the regime indices and the graphs. A couple of them assume really challenging settings, including instantaneous links, which are dropped in this paper.

The paper assumes Granger causality, which only takes care of time-lagged relationships. It also returns only a summary graph for every regime, hence we cannot learn which lags change from regime to regime or domain to domain. Also, assuming access to domain indices is a very big assumption.

The idea is poorly motivated. In causality, the objective is to show that this variable X causes that variable Y while estimating the mechanism ff of the structural equation. Assuming that the relationship X->Y can be explained by a weighted sum of experts is not well grounded, to me. What is the role of the different experts? The authors used the top 50 or 200 experts in their experiments. Learning a good estimation with different experts, where the information is spread across them, does not help practitioners. Also, I am not aware of any real-world application where you can split the causal relationship into a weighted sum across experts. What would each expert be? Why would I want this? It complicates the interpretability and controllability that causality aims for. Using top-1 routing could be motivated, but top-K with K>1K>1 is, to me, a trick to get great estimation results while trading away the essential spirit of causality: controllability, interventions...

In line 57, you mentioned that state-of-the-art methods “provide limiting support for distinguishing a recombination of known mechanisms from genuinely unseen causal state.” I want to discuss this point: why is this necessary? I cannot find any application for this, i.e. a new domain where, for a variable xix_i, the structural equation model is a weighted sum of the structural equation models of other regimes. Could you provide real-world examples, with datasets or research papers, confirming that?

I have concerns about the idea of top-K experts for causal discovery. I find it a way of improving estimation, but not very adequate for causal discovery. Happy to discuss!

In line 103, you tried to distinguish your model from causalMoE, but you only describe the difference; you do not motivate why you made this design choice or in which sense it helps your method.

In Methodology, notation and mathematical symbols are not clearly introduced and are used in a confusing way: the authors use non-bold uppercase letters for scalars, vectors, and matrices alike. Using a regular lowercase font for scalars (xx), bold lowercase for vectors (x\boldsymbol{x}), and bold uppercase for matrices (X\boldsymbol{X}) would make the paper easier to read. I understood your method, but I found your methodology section very disorganized and not smooth to read. You do not define AkA_k as a summary graph for every regime; the reader needs to be able to understand it on its own. You do not clearly state that NN is the number of features, nodes..., and you do not explain that you assume access to domain labels during offline training. You mention αk\alpha_k very early at line 137 and only truly explain it at line 244, almost two pages later. You do not define your space ΔK−1\Delta^{K-1} mathematically. Sometimes you put regime indices as superscripts and sometimes as subscripts, which is not rigorous and is misleading. Please unify the notation: A(k),Z(k),e(k)A^{(k)}, Z^{(k)}, e^{(k)}...

I also have concerns about how you decide whether the state is novel or not. Mathematically, any state could be expressed as a weighted sum of existing causal states if these states are a basis for the different graphs (maybe mutually linearly independent, see MCD, Fantom, and On the Identifiability of Switching Dynamical Systems), and somehow your model chooses to create a new state because your states were not expressive enough. The opposite could also happen: your model could use the different causal states to estimate a new state that could never be expressed accurately with the current states. Ablation studies on that and mathematical faithfulness are really needed to trust and use your method.

You introduce a hyperparameter HH without discussing how you use it and why it is fixed for all the domains.

ZkZ_k were never introduced and explained properly in the paper. I never understood how you compute them or how you optimize them. fθkf_{\theta_k} was also never properly explained: what NN do you use to estimate ff, and how are ZZ and AA used inside these models...

You introduce Ik\mathcal{I}_k in line 191 without explaining what a valid prediction is. Since you have access to domain labels, what makes a trajectory valid or not?

As you train different fkf_k during offline training, a comparison with MCD, Rhino, CASTOR, FANTOM, Dynotears, PCMCI+... is required to see how you perform against them in their settings.

I acknowledge that the online adaptation is novel; however, I feel that Section 3.3 is somewhat LLM-generated. The algorithm and the equations there are not motivated, and the choices are weird, for example using clip(., ., 20): the choice of 20 is difficult to see as a human choice and as a justified one. The choices of MA and RStd are commonly agent engineering choices. The statistics used in hh are weird and not motivated at all. I am not against using LLMs to brainstorm or to run experiments, but I feel the agent was pushed blindly to find a solution for this optimization, so it figured out a way that turns out to be complicated and not motivated. The ct,kc_{t,k} equation is non-intuitive for me. Also, agents have the habit of introducing many controllable parameters, which is the case in your paper, like λR,ρz,ρd\lambda_R, \rho_z, \rho_d...

In line 250, you made the choices of 2 experts why this choice, it is never motivated how the model performs with top1 or top 20...?

Why you need calibrated parameters, you introduce new things without motivation and also without goal oriented is just because somehow it worked better.

[1] Saggioro, Elena, et al. "Reconstructing regime-dependent causal relationships from observational time series." Chaos: An Interdisciplinary Journal of Nonlinear Science 30.11 (2020).

[2] Rahmani, Abdellah, and Pascal Frossard. "Causal temporal regime structure learning." AIStats 2025

[3] Mameche, Sarah, et al. "Spacetime: Causal discovery from non-stationary time series." Proceedings of the AAAI conference on artificial intelligence. Vol. 39. No. 18. 2025.

[4] Liu, Chenxi, and Kun Kuang. "Causal structure learning for latent intervened non-stationary data." International Conference on Machine Learning. PMLR, 2023.

[5] Rahmani, Abdellah, and Pascal Frossard. "Flow based approach for Dynamic Temporal Causal models with non-Gaussian or Heteroscedastic Noises." Advances in Neural Information Processing Systems 38 (2026): 8300-8349.

[6] Balsells-Rodas, Carles, Yixin Wang, and Yingzhen Li. "On the identifiability of switching dynamical systems." arXiv preprint arXiv:2305.15925 (2023).

[7] Hulsman, Roel, Carles Balsells-Rodas, and Sara Magliacane. "Identifiable Markov Switching Models with Instantaneous Effects and Exponential Families." arXiv preprint arXiv:2606.02231 (2026).

[8] Günther, Wiebke, et al. "Causal discovery with endogenous context variables." Advances in Neural Information Processing Systems 37 (2024): 36243-36284.

[9] Varambally, Sumanth, Yi-An Ma, and Rose Yu. "Discovering mixtures of structural causal models from time series data." arXiv preprint arXiv:2310.06312 (2023).

Questions for authors

See weaknesses, we can discuss more during the discussion period, but I truly feel the paper needs to be rewritten and also needs a lot of extensive work. But the online adaptation is a really interesting idea, I would encourage continuing on it, it could lead to great new algorithm.

2 comments

We thank the reviewer for the valuable comments.

First, we would like to clarify the motivation behind this work. Our initial goal was to discover Granger‑causal relationships among variables in high‑dimensional financial time series. Financial data are inherently nonstationary and continuously generated, making high‑dimensional modeling, evolving dynamics, and online inference our main challenges. We initially followed a conventional approach by estimating separate Granger‑causal graphs for different temporal segments. Motivated by financial dynamics, we subsequently considered that dependencies may persist and evolve gradually: structures identified in one period may remain partly relevant in subsequent periods. This led us to explore a mixture‑of‑experts mechanism that combines historical structures through weighted aggregation. Our experiments showed that the resulting mixed structures performed well under the evaluated settings.

However, we acknowledge that our initial work did not adequately address a fundamental question: whether the structure obtained through weighted aggregation retains a rigorous Granger‑causal interpretation. We thank the reviewer for highlighting this issue. After careful consideration, we agree that the MoE‑derived structure cannot be regarded as a Granger‑causal graph without further justification. In the revision, we therefore describe it as a predictive dependency structure guided by historical causal information, rather than an established true causal graph. Following the reviewer’s comments, we have also substantially revised the manuscript and added relevant comparative experiments.

1)Regarding historical knowledge reuse and dependence on domain labels We acknowledge that the original manuscript did not sufficiently distinguish offline expert construction from online knowledge reuse. Initial experts are trained separately on historical segments, while knowledge reuse primarily occurs online through expert matching, routing, and structure combination. Pre‑segmented training initializes the expert library and mitigates the cold‑start problem, but remains an assumption of our method. We recognize that methods such as CASTOR jointly learn regime segmentation and causal structures. Therefore, “independent estimation and lack of knowledge reuse” should not be presented as limitations shared by all existing methods.

2) Regarding comparisons with additional baselines We have added CASTOR, LIN, MCD, and PCMCI+. We could not obtain the FANTOM code from the link provided in the paper or locate Rhino in the version of Causica we examined. The results are shown in the table below and have also been included in Appendix B.13 of the revised PDF. These methods are also briefly discussed in Section 2.

MethodVAR AUROCVAR AUPRCLorenz‑96 AUROCLorenz‑96 AUPRC
CASTOR0.6550.2400.7360.492
LIN0.5120.1660.5050.139
MCD0.5220.1680.5390.152
PCMCI+0.7740.3830.8040.453

3)Regarding lag orders We estimate variable‑level summary structures because influence delays may change while dependencies between variables persist. This facilitates structural reuse across states.

4)Regarding the notation We acknowledge that some definitions are incomplete and symbols are used inconsistently. We have supplemented the definitions, standardized the notation, and reorganized the explanations in the revised manuscript.

5)Regarding the graph parameters and predictor implementation ZkZ_k is a learnable logit matrix. AkA_k is obtained by applying an element‑wise sigmoid to ZkZ_k and masking the diagonal to zero. The predictor combines graph‑weighted message passing with a GRU. Separate Adam optimizers alternate predictor and graph updates, using Gumbel sampling and sparsity regularization for graph learning. IkI_k contains indices with complete within‑domain inputs and targets: {ak+H,...,bk}\{a_k+H,...,b_k\} for one‑step prediction, where aka_k and bkb_k are the start and end indices of domain kk. These definitions and implementation details are provided in Section 3.2 and Appendix B.8.

6)Regarding Section 3.3 Moving averages, normalization, and clipping stabilize residual scores. Prediction error and residual features provide complementary matching signals, balanced by λR\lambda_R; ρz\rho_z and ρd\rho_d are historically calibrated scales. The clipping bound of 20 is empirical. The corresponding calibration procedures and fixed settings are described in Appendices B.4~B.6, while sensitivity analyses of the residual weight and local‑adaptation anchor strength are reported in Appendix B.7.

Reviewer_JtGmOpenReview ↗

Reviewer JtGm comment

I want to thank the authors for their answers and rebuttal. Unfortunately, the response to our review came just 2 days before the end of the discussion period; hence, it was difficult to have a constructive discussion.

I believe all the baselines deserve to be cited and discussed in your paper; I encourage the authors to do that in their paper. My concerns are not yet solved. I believe the paper needs another round of writing and review and is not ready for publication. I feel the storytelling and all what I mentioned in my first review need to be taken into consideration. In order to evaluate that properly, another round of review is important to check the soundness and rigor of the math, notation, and the motivation of this paper.

I also believe that the experiments conducted are not convincing and the generated synthetic data does not reflect reality and challenging scenarios. For these reasons, I conducted new tests using the code provided by the authors, and I found out that the method fails in different scenarios (the first one was obvious because of instantaneous links, while the second one is surprising where I had only time lagged).

I could find FANTOM code in NeurIPS supplementary material in openreview. I compared OGC_MoE and FANTOM on nonstationary time series generated using FANTOM’s nonlinear generator, with 10 nodes, two regimes containing 2,500 and 2,000 observations, and three random seeds. Table 1 includes instantaneous and lagged effects; Table 2 includes only lagged effects with FANTOM’s allow_instantaneous=False. Both methods receive identical data within each setting, without ground-truth graph or regime-label supervision. The tables report mean and standard deviation. FANTOM achieves stronger graph recovery in both settings, while OGC_MoE has slightly higher mean regime accuracy. (code: https://anonymous.4open.science/r/multiregime-causal-comparison-820D/README.md)

Table 1. Instantaneous and lagged effects

Metric ↑OGC_MoEFANTOM
Lag-1 AUROC0.593 ± 0.0760.949 ± 0.015
Average precision0.291 ± 0.0320.810 ± 0.087
F10.073 ± 0.0460.856 ± 0.037
Regime accuracy (%)96.1 ± 2.792.0 ± 5.6

Table 2. Lagged effects only

Metric ↑OGC_MoEFANTOM
Lag-1 AUROC0.656 ± 0.0770.968 ± 0.014
Average precision0.322 ± 0.0330.904 ± 0.075
F10.053 ± 0.0140.886 ± 0.118
Regime accuracy (%)96.1 ± 2.795.4 ± 1.8

I encourage the authors to conduct more experiments and highlight more the fact that they present online adaptation, which I truly find interesting and promising. However, I truly believe the paper is not ready for publication in its current form. I'll keep my clear reject score.

Reviewer_Sih8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces OGC-MoE, an online mixture-of-experts method for Granger-causal discovery in high-dimensional nonstationary time series. The model maintains a library of historical experts, each pairing a graph-conditioned temporal predictor with a sparse soft Granger graph. On each incoming window, it evaluates historical experts via prediction loss and residual profiles, updates routing weights to mix the corresponding graphs, and decides whether to keep the mixture, apply sparse local adaptation, or spawn a new expert for novel regimes. Experiments on synthetic nonstationary VAR and Lorenz-96 benchmarks (up to 256 variables) and the real-world THS financial index dataset show strong performance in AUROC/AUPRC and forward-lift metrics compared to existing baselines.

Critical strengths
  1. Practical formulation for nonstationary regimes: Combining historical mechanism reuse, graph mixing, local structural adaptation, and continual expert expansion makes intuitive sense for nonstationary time series where causal regimes both recur and evolve.

  2. Comprehensive synthetic benchmarks and diagnostics: The empirical study covers both linear (VAR) and nonlinear (Lorenz-96) settings up to 256 variables, backed by helpful ablations and routing/novelty diagnostics.

Critical weaknesses
  1. Potential same-window transductive leakage in online evaluation: For incoming windows, routing weights and local graph adaptations are optimized using the observed targets of the current window, and causal graph recovery is then evaluated on that exact same window. Without a within-window holdout split or strictly causal (next-window) evaluation, it is unclear how much of the reported gain comes from fitting current-window observations versus true out-of-sample causal discovery. Similarly, when a novel regime is detected, the reported expansion AUROC reflects the expert after training on that segment, whereas the actual online provisional graph on that window is much weaker.

  2. Limited real-world validation and missing scalability metrics: The THS financial dataset lacks ground-truth causal graphs and relies solely on forward-lift proxies. Furthermore, for an online method that continually expands its expert pool in high dimensions (up to 256 variables), the paper does not report runtime, memory footprint, or latency as the expert library grows.

Questions for authors
  1. Could you clarify the exact timeline for synthetic data generation under the 400-step window / 200-step stride setting? How is the ground-truth graph defined if regimes switch between overlapping windows? Please also provide results under a non-overlapping protocol (stride = 400).

  2. What is the computational overhead (wall-clock runtime and memory) as the number of experts and variables (D=128,256D=128, 256) increases?

  3. How does OGC-MoE perform if routing refinement and local graph adaptation are fit only on the first half of a window and evaluated on the second half (or evaluated strictly on t+1t+1 before observing targets at t+1t+1)?

1 comment

Responses to Weakness 1: Thank you for the reviewer's comment. We first clarify how the windows are used: each 400‑step context window contains the preceding 200 steps and 200 newly arrived steps. Routine expert matching, routing optimization, and local adaptation use only the newly arrived 200 steps. The full 400‑step window is used to train a new expert only when expert expansion is triggered. The new expert participates in inference from subsequent windows onward, while the current window retains the provisional mixed structure. The current results primarily measure structural estimation on observed segments. The expansion experiment indeed compares the new expert’s structure after training on the new‑state data with the provisional mixed structure before training. Our purpose is to examine whether adding an expert at the expansion point improves recovery of relationships that the existing mixture cannot adequately recover, thereby providing empirical support for the expansion decision.

Responses to Weakness 2: Our model was initially motivated by the nonstationary and continuously generated nature of financial data. We aim to accommodate evolving dependencies through historical structure reuse and online adaptation. However, we acknowledge that the THS dataset lacks ground‑truth causal structures. Forward‑lift metrics provide evidence of temporal associations but cannot replace ground‑truth structural accuracy evaluation.

To quantify computational costs, we conducted additional experiments measuring online runtime and peak GPU memory usage with 64, 128, and 256 variables, as reported in Appendix B.11.

DatasetVariablesOnline runtime (min)Peak GPU memory (MB)
VAR6418.88304.8
VAR12834.20786.7
VAR25667.282252.8
Lorenz‑966418.89326.3
Lorenz‑9612834.86811.2
Lorenz‑9625676.382512.6

Under these settings, doubling the number of variables increased runtime by approximately 1.8–2.2 times, while GPU memory usage grew more rapidly. We also fixed the number of variables at 128 and measured per‑window online runtime for different expert counts, as reported in Appendix B.12.

DatasetExpertsOnline runtime per window (min)
VAR43.80
VAR85.64
VAR169.72
Lorenz‑9643.87
Lorenz‑9685.81
Lorenz‑96169.68

Increasing the expert count from 4 to 16 increased per‑window runtime by approximately 2.5–2.6 times. These results quantify the computational overhead associated with variable dimensionality and expert‑library size. The hardware configuration is provided in Appendix B.11.

Responses to Question 1: The 400‑step context window and 200‑step target segment are explained in Responses to Weakness 1. The ground‑truth structure is defined by the data‑generating mechanism, which remains fixed within each target segment. We additionally kept the window length at 400 and increased the stride to 400, so that adjacent input contexts no longer overlap. The results are provided below as additional evidence for this response.

DatasetStrideAUROCAUPRC
VAR2000.944 ± 0.0150.802 ± 0.018
VAR4000.931 ± 0.0140.769 ± 0.013
Lorenz‑962000.978 ± 0.0160.815 ± 0.014
Lorenz‑964000.973 ± 0.0210.787 ± 0.011

Although performance declined after removing context overlap, the model retained good structural recovery performance.

Responses to Question 2: The runtime and GPU memory results are provided in Responses to Weakness 2.

Responses to Question 3: Our goal is to estimate the dependency structure after observing the current segment and compare it with that segment’s ground‑truth generating graph. The segment’s observations are used for estimation, while the ground‑truth graph is used only for evaluation. Within a segment with a fixed mechanism, both halves share the same ground‑truth graph. Therefore, computing AUROC/AUPRC for a frozen estimated structure still compares the same pair of graphs and yields unchanged scores. We have clarified that the reported results measure structural recovery on the currently observed segment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.