acceptodds
Under review as a conference paper at ICLR 2027

Learning Time-Varying Directed Graphs via Convex TV-Regularized Optimization and DAG Projection

Abstract

Continuous optimization methods for learning directed acyclic graphs (DAGs) have made progress in structure learning. However, they face two key limitations: (1) they assume a single, stationary graph structure across all observations, and (2) their nested nonconvex optimization with matrix exponential evaluations scales inefficiently as the number of variables grows—a bottleneck in high-dimensional applications. To address these limitations, we propose DACORD: DAG via Convex Optimization with Regime Detection, a two-stage method for learning time-varying DAGs. We utilize the insight that for models with a fixed feature map, acyclicity is the core source of nonconvexity in score-based DAG learning. Stage 1 jointly estimates window-specific coefficients by solving a convex regression problem with sparsity and total variation regularization. A single-loop primal–dual algorithm solves this problem, and changes in the estimated coefficients provide candidate regime boundaries. Stage 2 constructs a DAG for each estimated regime using greedy ordering and back-edge removal. Extensive experiments show that, at with regimes, DACORD completes in under 34 seconds with less than 1GB of GPU memory, whereas several existing methods either time out or fail numerically, and the completed baselines achieve lower precision and F1. Across completed comparable settings including the DREAM4 dataset, DACORD achieves the highest precision and F1 at high dimensional situation.

Primary area:
learning on time series and dynamical systems
Keywords:
Non-stationarity, Optimization for DAG

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_smsNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents DACORD, a method to learn causal graphs (DAGs) from time series data, in the case where the true graph doesn't stay the same over time. The authors call these different periods regimes, and the points where the graph changes are change points. Existing methods are slow and memory consuming when number variables increases and it gets more expensive when we have to model multiple time periods (regimes). DACORD targets this and split it into two stages - 1. drop the acyclicity constraint to make the problem convex and solve it with a primal-dual method, 2. given the results, average the coefficients within each detected regime and make it acyclic afterward using a greedy ordering. Experimental results show impressive gain in terms of speed and memory consumption.

Critical strengths

The idea of DACORD is simple. If a feature map is fixed, the loss function and sparsity regularization are already convex. Acyclicity is what makes the optimization process non-convex. The paper takes out acyclicity part and solve the convex problem first and restore acyclicity afterwards. Authors showed that step sizes can be set to 15\frac{1}{\sqrt{5}} irrespective of how many time windows, variables or basis functions we have.

Critical weaknesses

Section 4.2 demonstrated regime accuracy decrease as a theoretical result of signal-to-noise ratio tradeoff (O(d)(\mathcal{O}(\sqrt{d}) signal increase vs. O(d2)\mathcal{O}(d^2) noise in parameters). However, Appendix E shows that the reduction is caused by the fact of window starvation: using default rule nw=max⁡(2d,125)n_w = \max(2d, 125) allows only one window per regime at d≥100d \ge 100. In this case, the temporal sequence is missing and makes DACORD applicability questionable.

Questions for authors
  • Appendix E shows the accuracy drop stems from window starvation rather than the section 4.2 signal-vs-noise trade-off. Can you clarify this?

  • Since setting the window size to nw=125n_w = 125 restores a regime accuracy of 1.001.00 up to d=200d=200 (Table 15), does the signal-vs-noise trade-off discussed in the main text have any actual empirical weight, or is performance entirely governed by the number of available windows per regime?

  • At d = 200, is there anything to compare against? Tables 1 and 2 show fantom returning nan, rpcmci too slow, and spacetime too slow at d = 200, so DACORD is the only method that completes.

  • Table 1 has no standard deviations for DACORD when every baseline row does?

2 comments

Thank you for your careful review and thoughtful questions. In the revised manuscript, we have color-coded the changes by reviewer. Blue marks revisions addressing Reviewer Zmp7’s comments, green marks those addressing Reviewer ZrRA ’s, yellow marks those addressing Reviewer smsN’s, and red marks revisions addressing comments from multiple reviewers.

** W1. Window availability and the SNR interpretation.**

Section 4.2 now describes the dimension trend as sensitive to window construction. When one window covers a whole regime, no within-regime coefficient difference can be measured, although the sequence of windows across successive regimes remains available. The SNR explanation remains a hypothesis.

Q1. How much of the decrease is attributable to window construction?

In the linear-basis rows of Table 15, the default rule gives regime accuracy 0.467 at d=100d=100 and 0.600 at d=200d=200. Using two 125-sample windows per 250-sample regime gives 1.000 at both dimensions under the same knee-selected threshold procedure. This directly supports a window-construction effect in that experiment.

Q2. Do two windows per regime fully explain the outcome?

Every true boundary in the 250-sample design falls on the 125-sample grid, so the experiment tests detection on aligned boundaries. With the corrected proximal update and a half-maximum threshold, the same detection grid produced extra boundaries on nonlinear, medium-density graphs. Mean regime accuracy ranged from 0.667 to 0.950 across settings. Window count alone cannot explain the results across these protocols.

Q3. Which baselines completed at high dimension? At d=200d=200, CD-NOD (Fisher-zz) completed 32 of 36 original datasets, with two cancellations and two out-of-memory failures. RPCMCI completed 12 of 36, all at R=3R=3. The records contain no jobs for its other 24 cases or for FANTOM at this dimension. The new DAGMA-linear comparisons also completed at d=200d=200 with both true and estimated boundaries. Section 4.2 reports these results.

Q4. Why do the DACORD rows lack standard deviations? The graph table (submitted Table 1, revised Table 14) omits DACORD’s SD. Determinism for a fixed dataset does not justify that omission. The seed analysis (submitted Table 13, revised Table 12) reports variation across generated datasets, e.g., F1 0.403±0.060.403\pm0.06 at d=200d=200, R=3R=3 (revised Table 14 gives 0.469). In the separate threshold diagnostic in §4.2, F1 is 0.265±0.0030.265\pm0.003 after averaging over RR within each of three seeds and taking the sample SD across seeds. These SD estimates describe different summaries from the mean of 0.469 in revised Table 14.

Thank you again for highlighting the role of window size. We clarified that using two windows per regime restored accuracy to 1.00 in the linear-basis experiment and that the proposed SNR explanation remains a hypothesis. We hope you will consider this clarification in your final assessment.

Reviewer_ZrRAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes DACORD, a two-stage method for learning time-varying DAGs from non-stationary data. Stage 1 drops the acyclicity constraint entirely and solves, per time window, a basis-expanded regression of every variable on all others with an L1 / group-lasso sparsity penalty and an elementwise total-variation (fused-lasso) penalty across adjacent windows. The convex problem is solved with Chambolle–Pock. Regime boundaries come from thresholding the norm of the adjacency matrix differences. Stage 2 averages coefficients within each detected regime, orders the vertices greedily by "net flow", and deletes back-edges (a greedy minimum-feedback-arc-set heuristic) to output one DAG per regime. Theory consists of standard Chambolle–Pock convergence, a restatement of known identifiability results, a triangle-inequality argument for boundary recovery, and a sufficient condition for the greedy ordering. Experiments on synthetic ER graphs (d up to 200), a concatenated DREAM4 benchmark and an S&P case study report higher precision/F1 and large speedups over FANTOM, RPCMCI, SpaceTime and oracle-segmented stationary methods.

Critical strengths
  • The problem matters: regime-switching causal structure in high dimension is relevant, and existing methods do scale poorly.
  • Stage 1 is a clean, convex, deterministic estimator with a simple solver and predictable cost. The operator-norm bound on \tilde{K} gives data-independent step sizes.
  • The authors are candid in several places in the appendix (Remark B.3, the "scope of the guarantee" paragraph after Lemma B.3, Tables 10, 11 and 17). These tables are what make the central concern below checkable.
  • The empirical study is broad: many baselines, runtime/memory accounting, a threshold-free AUPRC comparison (Table 9), and ablations over \lambda1, \lambda2, basis, window size and temporal penalty.
Critical weaknesses

W1. The core formulation: dropping acyclicity and "projecting" afterwards is not a sound DAG estimator (main concern)

The central claim is that acyclicity is "the core source of nonconvexity" and can be deferred to a post-processing step. Removing the acyclicity constraint changes the statistical target, though, and the paper does not supply a sound way back.

Without the acyclicity constraint (and with self-loops excluded), problem (5) decouples into d independent regressions, i.e. neighborhood selection (Meinshausen & Bühlmann, 2006) with a fused-lasso penalty over time. In the linear-Gaussian case the population coefficient of X_i in the regression for X_j is \beta_ij = −\theta_ij / \theta_jj, where \theta is the precision matrix. Consequences:

  • The population support of Stage 1 is the moral graph (parents, children and spouses), not the DAG, and it is symmetric: \beta_ij \neq 0 iif \beta_ji \neq 0. The support therefore carries no directional information. Tables 11 and 17 are consistent with this: the Stage-1 output is 0% acyclic, and roughly two thirds of its arcs belong to two-cycles even after regularization and thresholding.
  • All directional information in the edge scores comes from ratios of conditional variances. The "net flow" ordering is then a function of these scale-dependent quantities, not of any identifiability principle. This is closely related to the varsortability / R²-sortability artifacts documented by Reisach et al. (2021; 2023).

Critically, the projection fails even in the population for a simple identifiable model. Take the chain X1 -> X2 -> X3 or the a v-structure X1 -> X3 <- X2 with weight b \neq 0 on both edges and unit noise variances. These are linear Gaussian with equal error variances, so they are identifiable by the paper's own Prop. 3.2(b). However, it seems like the paper's setup will fail in these simple examples.

Since the orientation is driven by conditional variances, the output is not invariant to rescaling the variables. As stated earlier, this is very similar to varsortability I would ask the authors to (a) report all results on standardized data and (b) compare against sortnregress and R²-sortnregress (Reisach et al., 2021; 2023). The synthetic protocol (equal unit noise variance, weights with |w| ≥ 0.5, ER graphs) is known to be highly varsortable.

The paper's own evidence points the same way:

  • Table 10: the projection raises precision, the paper's stated priority. However, it reduces F1 relative to the unprojected Stage-1 output in all nine settings. App. E also reports that skeleton F1 changes by small amounts, so the projection mainly decides orientations, and Table 17 shows it often decides them wrongly.
  • Table 17: after projection, 37–45% of retained edges are false positives, and 21–32% of removed edges are true edges whose reverse was kept. App. E concludes that the gap is "an orientation gap rather than a support gap". That is exactly the issue: Stage 1 recovers (roughly) an undirected moral graph, and Stage 2 orients it by a heuristic without causal justification.
  • App. B.5 concedes that the population minimizer can "favor a directed supergraph over the true DAG" and that the Stage-1 objective cannot serve as a projection criterion (see also Remark B.3).
  • Table 6 (\lambda1 = 0 rows): without the sparsity penalty, precision is about 0.08 regardless of \lambda2. Edge selection by \lambda1 and the threshold, not the structure of the estimator, appears to drive the reported precision.

Unconstrained regression followed by thresholding or ordering has been studied and critiqued before. Loh & Bühlmann (2014) explicitly estimate the moral graph via the inverse covariance and then search over DAGs consistent with it, using a proper score rather than a net-flow heuristic. Reisach et al. (2021) show that apparent DAG-learning performance on such benchmarks can come from scale artifacts, and Kaiser & Sipos (2021) show that NOTEARS-type outputs are not scale-invariant. Stage 1 is a basis-expanded variant of TV-penalized neighborhood selection for time-varying networks (Ahmed & Xing, 2009; Kolar, Song, Ahmed & Xing, 2010). The closest Gaussian analogue with abrupt changes is Kolar & Xing (2012). Related work on change points in graphical models includes the time-varying graphical lasso (Hallac et al., 2017) and the group-fused graphical lasso (Gibberd & Nelson, 2017). None of this literature is discussed, and it would substantially reframe the contribution: Stage 1 is a close relative of known estimators of time-varying undirected dependence structure, and the novelty would rest largely on Stage 2, which is the unsound part.

Unless the authors can give conditions, verifiable in terms of the SEM, under which net-flow ordering of neighborhood-regression coefficients recovers a topological order, and show that these conditions are not just varsortability, I do not think the method can be presented as a DAG / causal-discovery method. At most it learns time-varying Markov-blanket structure with a heuristic orientation.

W2. Theory is either standard or does not connect to the claims

  • Thm 3.1 is the textbook Chambolle–Pock result applied to a convex problem. It guarantees convergence to the minimizer of a problem whose minimizer is not the target DAG (see above).
  • Prop 3.2 restates LiNGAM, equal-variance and ANM identifiability (the proof says each part "restates an existing result"). It concerns the population distribution, not the estimator. Including it gives the impression that DACORD exploits these identifiability results, but none of them is used: the estimator never uses non-Gaussianity, equal variances, or residual independence. The three-node chain above is in setting (b) and the method fails there.
  • Prop 3.3 is a triangle inequality given a uniform estimation error e, with no bound on e (no rate, no conditions on n_w, d, \lambda, design). The "reference sequence" W^0 is never tied to the SEM coefficients. As Remarks B.3 and App. B.5 indicate, the population regression target can include non-parent edges and is not the structural coefficient matrix, so even a consistent Stage 1 converges to something other than the structural W.

W3. Experimental design and fairness

  • Missing baselines. (i) Pooled or per-window lasso + the same FAS step with no TV. The paper's own grid (Table 6, RBF10) is mixed here: at the default \lambda1 = 0.1, TV adds little, while at \lambda1 = 0.05 it helps substantially. A systematic comparison with the no-TV variant across settings is needed. (ii) sortnregress / R²-sortnregress. (iii) A simple two-step pipeline: an off-the-shelf multivariate change-point detector (e.g. on covariances) followed by a fast modern DAG learner per segment, e.g. DAGMA (Bello et al., which the paper cites but does not compare to), GOLEM, or PC with Fisher-z. These should be feasible at d = 200.
  • Reports such as DYNOTEARS "too slow" at d = 100 within 10 h, and NTS-NOTEARS at 77 h for d = 200, need justification (implementation, tolerance 1e-8, CPU/GPU, number of threads). The 20–40x speedup compares a solver for an unconstrained convex problem to solvers for a constrained nonconvex problem, so it is not an apples-to-apples comparison of methods solving the same task.
  • Unit-variance Gaussian noise, ER graphs and |w| \in [0.5, 2] is a highly varsortable regime (see Point 1). Results on standardized data, unequal noise variances, and scale-free graphs are needed.

References mentioned in this review

  • Ahmed, A. & Xing, E. P. (2009). Recovering time-varying networks of dependencies in social and biological studies. PNAS.
  • Gibberd, A. J. & Nelson, J. D. B. (2017). Regularized estimation of piecewise constant Gaussian graphical models: the group-fused graphical lasso. Journal of Computational and Graphical Statistics.
  • Hallac, D., Park, Y., Boyd, S. & Leskovec, J. (2017). Network inference via the time-varying graphical lasso. KDD.
  • Kaiser, M. & Sipos, M. (2021). Unsuitability of NOTEARS for causal graph discovery. Neural Processing Letters (arXiv:2104.05441).
  • Kolar, M., Song, L., Ahmed, A. & Xing, E. P. (2010). Estimating time-varying networks. Annals of Applied Statistics.
  • Kolar, M. & Xing, E. P. (2012). Estimating networks with jumps. Electronic Journal of Statistics.
  • Loh, P.-L. & Bühlmann, P. (2014). High-dimensional learning of linear causal networks via inverse covariance estimation. JMLR.
  • Meinshausen, N. & Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Annals of Statistics.
  • Reisach, A. G., Seiler, C. & Weichwald, S. (2021). Beware of the simulated DAG! Causal discovery benchmarks may be easy to game. NeurIPS.
  • Reisach, A. G., Tami, M., Seiler, C., Chambaz, A. & Weichwald, S. (2023). A scale-invariant sorting criterion to find a causal order in additive noise models. NeurIPS.
Questions for authors
  1. For the chain X1 -> X2 -> X3 with equal unit noise variances, the population Stage-1 scores give net flow F(X2) > F(X1) = 0, so Stage 2 deletes X1 → X2. Do you agree? If so, under what SEM-level conditions does the net-flow margin of Lemma B.3 hold, and how do they differ from varsortability?
  2. How do results change if every variable is standardized (zero mean, unit variance) before running DACORD? Please report Tables 1, 3 and 10 under standardization.
  3. How does DACORD compare with sortnregress and R²-sortnregress, and with lasso/neighborhood selection + net-flow FAS without TV, on the same data?
  4. How does Stage 1 relate to TV-penalized neighborhood selection (Ahmed & Xing, 2009; Kolar et al., 2010; Kolar & Xing, 2012) and to Loh & Bühlmann (2014)? What is new beyond the basis expansion and the choice of solver?
  5. Why is F1 lower with the DAG projection than without it (Table 10)? Given that skeleton F1 barely changes, what is the evidence that the orientations chosen by Stage 2 are better than chance for reasons other than variance artifacts?
  6. Which window rule was used in Tables 1, 2 and 16, and why do Tables 1 and 12 report identical numbers under different sample sizes?
  7. Can you explain FANTOM's identical P/R/F1 across R = 3, 5, 10 and K-SVARM predicting 88–100% of all arcs? Were thresholds tuned per method? Which edge scores does the AUPRC in Table 9 use?
  8. Can you report DAGMA (or GOLEM) with oracle segmentation and with a simple off-the-shelf change-point detector at d = 100 and 200?
3 comments

Thank you for your careful review and thoughtful questions. In the revised manuscript, we have color-coded the changes by reviewer. Blue marks revisions addressing Reviewer Zmp7’s comments, green marks those addressing Reviewer ZrRA ’s, yellow marks those addressing Reviewer smsN’s, and red marks revisions addressing comments from multiple reviewers.

W1. What Stage 1 estimates and what Stage 2 contributes.

Stage 1 regresses each variable on the others. In an unpenalized linear-Gaussian population regression, βij=−Θij/Θjj\beta_{ij}=-\Theta_{ij}/\Theta_{jj} for precision matrix Θ\Theta. Its support is symmetric, and its coefficient magnitudes can differ by direction. DACORD uses these predictive dependencies to track temporal change and applies an existing greedy ordering to each regime summary. Q1 shows exactly where the resulting causal orientation can fail.

W2. Scope of the theoretical guarantees.

Section 3.3 assigns a distinct target to each statement. Theorem 3.1 gives the standard convergence guarantee for the convex Stage-1 objective. Proposition 3.2 identifies population DAGs under specified SEM assumptions. Proposition 3.3 separates changes in a reference coefficient sequence given a uniform error bound. In the original implementation, zeroing self-blocks after an unrestricted update did not compute the constrained proximal step. The paired experiments in §4.2 use the corrected step. Its exactness applies to that update.

W3. Comparison protocol, graph family and runtime. Section 4.2 reports matched no-TV, sorting and DAGMA-linear comparisons on Erdős–Rényi graphs with equal Gaussian noise variances. Unequal-noise and scale-free graphs fall outside this evaluation. The 32.4 s and 824 MB figures at d=200d=200, R=10R=10 describe 12 original-solver GPU runs with true boundaries. The corrected DACORD and DAGMA timings use different CPU thread allocations. The job records do not substantiate a common ten-hour cutoff or the quoted 77.1-hour run.

Q1. Does the identifiable chain defeat the ordering rule? Yes. For the unit-weight chain 1→2→31\to2\to3 with independent standard Gaussian errors, unpenalized population regression on the original scale gives net flows (0,1/2,−1/2)(0,1/2,-1/2). Node 2 is selected first, so the rule retains 2→12\to1 and 2→32\to3. On population-standardized variables, node 2 is still selected first. The equal-error-variance SEM is identifiable on its generating scale, yet the required source margin fails. Lemma B.3 assumes a margin in regression scores and supplies no separate SEM-level criterion independent of scale effects.

Q2. Standardization in the graph, DREAM4 and projection tables.

The synthetic DACORD fits examined here already z-score each variable within each window. Feature columns are then divided by their standard deviations without additional centering, as specified in §4.1. The graph and projection tables (submitted Tables 1 and 10, revised Tables 14 and 9) report synthetic results, but the available records do not establish a paired raw-versus-standardized change for their individual cells. The DREAM4 benchmark (submitted Table 3, revised Table 2) has its own protocol. There is no corresponding standardized DREAM4 comparison from which to give a change in F1.

Q3. How do no-TV and sorting baselines compare? With true boundaries and the same ordering algorithm, refit and edge threshold, changing λ2\lambda_2 from 0 to 0.01 increased F1 on all 108 datasets at each sparsity level. The mean gains were 0.034 at λ1=0.1\lambda_1=0.1 and 0.042 at λ1=0.05\lambda_1=0.05. SHD increased in 7 of 72 configuration cells. Raw-input sortnregress had higher F1 than DACORD on 108/108 datasets and R2R^2-sortnregress on 85/108. After standardizing the inputs to sortnregress, DACORD had higher F1 on 89/108. Section 4.2 reports these full-pipeline comparisons.

Q4. How does DACORD relate to temporal neighborhood selection and DAG score search? Ahmed and Xing (2009) and Kolar et al. (2010) estimate changing dependency networks through temporally coupled neighborhood regressions. Kolar and Xing (2012) also treat networks with abrupt changes. DACORD uses a linear splitting of sparsity and temporal differences to fit window coefficients in one primal–dual loop. Coefficient jumps locate candidate boundaries, and each segment receives a DAG through greedy ordering. Loh and Bühlmann (2014) instead use inverse-covariance information to restrict a DAG score search. DACORD’s contribution is to move acyclicity enforcement outside temporal fitting and apply it at the regime level.

Q5. Why can projection lower F1, and how reliable are the directions? Projection removes the edge opposing the estimated order, even when that edge is the true direction. The projection comparison (submitted Table 10, revised Table 9) illustrates the trade-off at d=200d=200, R=3R=3. Precision rises from 0.518 to 0.661, recall falls from 0.483 to 0.364, and F1 falls from 0.499 to 0.469. In the separate ordering diagnostic in §5, 51–61% of true edges point forward on linear data and 43–53% on nonlinear data. A uniform random order has expectation 50%. These ranges give limited support for causal orientation.

Q6. Which windows were used, and why are table entries repeated?

For the recorded oracle runs with 250 samples per regime, regime-aligned splitting gives two windows of 125 at d=50d=50 and one of 250 at d=100,200d=100,200. The GPU timing in §4.2 (submitted Table 2, revised Table 1) used the original solver with true boundaries. The corrected detection comparison uses a first pass on chronological 125-sample windows. The records do not link this detector to the original Table 16 cells. The graph and regime-length tables (submitted Tables 1 and 12, revised Tables 14 and 11) repeat the same DACORD values at d=200d=200, whereas the recorded 250- and 500-sample experiments give different results. Those rows therefore cannot substantiate a comparison of sample sizes.

Q7. Why are predictions dense, and what scores enter AP?

At the fixed threshold 0.1, FANTOM selected all off-diagonal pairs in all 216 recorded regimes at d=50,100d=50,100. Its recall is then one and its precision equals the true edge density, explaining the repeated values when density is unchanged. The K-SVARM comparison (Table 18 in both versions) uses its documented relative cutoff of 0.05. It retained 88.7–100% of possible arcs across the reported settings. Both cutoffs were fixed in the reported comparisons. For the AUPRC comparison (submitted Table 9, revised Table 8), the manuscript specifies stepwise AP on unthresholded scores. In the stored runs with 500 samples per regime, the scores are window-averaged block norms after projection and least-squares refitting, with removed pairs scored zero. Those runs establish the score construction for that dataset. Their AP values differ from the entries in revised Table 8, so this reconstruction identifies the scores for those runs only.

Q8. How does DAGMA compare with true and estimated boundaries?

Section 4.2 reports DAGMA-linear at d=100,200d=100,200. On raw inputs it achieved higher F1 than DACORD on all 72 datasets with true boundaries and on all 72 when KernelCPD supplied DAGMA’s boundaries and DACORD detected its own. The KernelCPD penalty was selected on separate validation data. In a follow-up analysis with segment-wise standardization and unchanged DAGMA defaults, DACORD had higher F1 on 67/72 datasets in each comparison. Standardized DAGMA had lower SHD on all 72. DACORD standardizes by window throughout.

Thank you again for the three-node chain example. We showed why our ordering rule fails on that chain and added the requested comparisons with no-TV, sorting, and DAGMA baselines. We hope you will consider these results and the clarified scope of our claims in your final assessment.

Reviewer_Zmp7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses two limitations of continuous (score-based) DAG learning: (i) stationarity — a single fixed graph is assumed across all observations — and (ii) scalability — the augmented-Lagrangian × L-BFGS × line-search scheme re-evaluates the matrix exponential repeatedly and fails numerically or times out at scale. The authors propose DACORD, a two-stage method. Stage 1 jointly estimates window-specific coefficient matrices {W_t} by solving a convex regression with ℓ1/group-lasso sparsity plus a total-variation (TV) penalty through a linear time-difference operator K, solved by a single-loop Chambolle–Pock primal–dual algorithm; regime boundaries are then detected by thresholding primal differences ‖ĤW_{t+1} − ĤW_t‖. Stage 2 averages coefficients within each detected regime and builds a DAG via greedy vertex ordering and back-edge removal (a greedy feedback-arc-set projection). Theory covers O(1/N) ergodic saddle-gap convergence for Stage 1 (Thm 3.1), standard identifiability results for the population DAG target (Prop. 3.2), a TV-based boundary-recovery condition (Prop. 3.3), and a sufficient net-flow-margin condition for the greedy ordering to preserve true edges (Lemma B.3). Empirically, DACORD is fully deterministic and completes d=200, R=10 in ~33 s / <1 GB GPU memory where several baselines fail (FANTOM → NaN; SpaceTime/RPCMCI → timeout), reporting the highest precision and F1 plus strong regime accuracy.

Critical strengths
  1. Genuinely elegant decomposition. Deferring acyclicity to a post-processing step makes the entire estimation stage convex, replacing the nested nonconvex loop with a single fixed-step primal–dual loop. This directly attacks a real bottleneck and is the paper's core contribution.

  2. TV penalty yields automatic change-point detection without knowing the number of regimes — a practical advantage over RPCMCI, which requires R as input.

Critical weaknesses
  1. Identifiability assumptions do not match the experiments; no CPDAG metric (most substantive gap). Prop. B.2 establishes a unique DAG target only under (a) linear non-Gaussian, (b) linear Gaussian with equal error variances, or (c) restricted nonlinear additive under causal minimality. Yet the generator uses Gaussian noise (σ = 1.0) with tanh mechanisms. In the linear-Gaussian case with unequal variances only the CPDAG is identifiable, and causal minimality for the generated tanh mechanisms is never verified — nevertheless all metrics, including the appendix SHD, are scored against the ground-truth DAG. The paper should either verify identifiability for its generator or report CPDAG-relative metrics (e.g., SHD against the true CPDAG, or orientation-error counts).
  2. The main results omit SHD. Precision/Recall/F1 are edge-level and do not penalize direction errors or cycle structure; SHD/NHD appear only in Appendix Table 7. For a causal-discovery paper, SHD belongs in the main text.
  3. Overloaded notation and an implicit convexity statement. g_{ij} (Eq. 2, the edge function — nonlinear in x through φ) and g(w) (Eq. 5, the data-fidelity term) are unrelated objects sharing a symbol. The paper also never states plainly that convexity holds "in w for a fixed feature map φ," only implying it. The convexity claim is sound, but the presentation invites exactly the confusion it deserves to avoid.
  4. Stage-2 projection is a heuristic with a narrow, unverified guarantee. Greedy FAS has no optimality guarantee; Lemma B.3 requires a net-flow margin γ and estimation error < γ/2 at every step, which is never measured empirically. Also, "20–40× speedup at the same O(d³)" is a constant-factor win, not asymptotic — worth stating precisely.
Questions for authors
  1. Given Gaussian noise in your generator, which identifiability regime of Prop. B.2 actually holds? Will you report CPDAG-based SHD (or orientation-error SHD) so that non-identifiable orientations are not penalized?
  2. Can you move SHD/NHD (and AUPRC) to the main text? What is the precision–recall tradeoff — at what threshold does DACORD recover a meaningful fraction of true edges?
  3. For Lemma B.3's margin γ: how often, empirically, do the greedy steps satisfy it, and how do these conditions relate to your λ1λ_1,$ λ_2$ choices?
  4. Will you state the convexity of Eq. 5 explicitly as "convex in w for fixed φ," and resolve the gijg_{ij} vs g(w)g(w) collision? When p is chosen adaptively in d, do convexity/identifiability still hold? In Theorem 3.1, (5) becomes a convex composite optimization problem, while before (5) you stated it is a Unified convex formulation, and a convex composite optimization problem may not be convex. Can you resolve this conflict?
  5. How sensitive are the results to the fixed edge threshold 0.1 and the boundary threshold ε_cp (Prop. 3.3 requires 2e < ε_cp < Δ_reg − 2e — is this satisfied empirically)?
2 comments

Thank you for your careful review and thoughtful questions. In the revised manuscript, we have color-coded the changes by reviewer. Blue marks revisions addressing Reviewer Zmp7’s comments, green marks those addressing Reviewer ZrRA ’s, yellow marks those addressing Reviewer smsN’s, and red marks revisions addressing comments from multiple reviewers.

W1. Identifiability of the evaluation target.

Section 4.1 specifies the contemporaneous synthetic generators. The linear SEM has independent Gaussian errors of equal variance on the generating scale. The nonlinear SEM uses a two-layer LeakyReLU network over each node’s parents and can include interactions outside our additive univariate feature model. Gaussian noise alone does not verify the restricted nonlinear identifiability assumptions. Q1 gives an orientation diagnostic for the linear data.

W2. Directed structural metrics.

Section 4.2 reports SHD, NHD and average precision (AP) for a threshold diagnostic. Directed precision, recall and F1 already count an edge reversal as one false positive and one false negative. Our directed-adjacency SHD therefore counts a reversal twice. Acyclicity is assessed separately.

W3. Structural functions and regression loss.

In our notation, fij(t)f_{ij}^{(t)} denotes a structural edge function. Under the additive representation, βt,ijSEM\beta_{t,ij}^{\mathrm{SEM}} denotes its structural coefficient block, and βt,ij\beta_{t,ij} denotes the fitted regression block in WtW_t. The optimization loss g(w)g(w) is quadratic in the fitted coefficients for fixed feature matrices Φt\Phi_t.

W4. Ordering guarantee and computational cost. Lemma B.3 gives a sufficient condition for preserving true input edges during ordering. Spurious forward edges can survive. Section 5 now gives the dense per-iteration cost as O(Td3p2)O(Td^3p^2) for pp features per variable. At fixed pp, the dependence on dd remains cubic. The computational saving comes from moving acyclicity enforcement outside the fitting loop.

Q1. Which identifiability assumption holds, and which reversals matter?

The linear generator satisfies the equal-error-variance case of Proposition 3.2 before standardization. This identifies a population target and gives no recovery guarantee for the net-flow rule. In a separate linear benchmark diagnostic at d≥100d\geq100, 90–96% of DACORD’s reversed edges had a compelled direction in the true CPDAG. Markov equivalence can account for only the remaining reversals in this diagnostic. For the nonlinear generator, we make no general identifiability claim.

Q2. Where are SHD, NHD and AP reported, and how does the threshold affect recall? Section 4.2 gives a concrete operating point. On nine nonlinear, medium-density datasets at d=200d=200, the corrected solver with window-level thresholding and majority voting gave mean SHD 1,723, NHD 0.043 and AP 0.094. At threshold 0.1, precision and recall were 0.200 and 0.396. Lowering it to 0.01 raised recall to 0.435 and reduced precision to 0.140. AP uses the continuous scores before the final threshold.

Q3. How often does the ordering margin hold? In the recorded ordering diagnostic at d≥50d\geq50, the first selected node was a true source in at most 11% of runs across the reported settings, and complete topological-order recovery was 0%. The lemma requires both a population margin and an estimation error below half that margin at every step. Those diagnostics measure the resulting orders. The frequency of the joint margin-and-error condition at individual steps, and its dependence on tuning parameters, cannot be inferred from them.

Q4. Why is Eq. (5) convex when the basis dimension varies? At each dd, we choose p(d)p(d) and construct Φt\Phi_t before fitting. Equation (5) then adds a convex quadratic loss to convex sparsity and TV penalties composed with a linear operator. The zero self-block constraints are linear. Thus the objective is convex in ww for each chosen p(d)p(d). Population identifiability depends separately on the SEM assumptions.

Q5. How sensitive are edge and boundary thresholds? Q2 gives the measured edge-threshold trade-off. For boundaries, the appendix sweep reports a range of thresholds yielding the same segmentation, with extra splits below the range and merged regimes above it. The detector evaluated in §4.2 uses half the largest adjacent coefficient jump. Any positive maximum forces at least one split. Proposition 3.3 requires 2e<ϵcp<Δreg−2e2e<\epsilon_{\rm cp}<\Delta_{\rm reg}-2e for a specified reference sequence. The measured threshold sweep does not estimate ee or establish how often this inequality holds.

Thank you again for raising the identifiability question. We clarified which assumptions hold for our generators and reported how many reversed edges had a compelled direction in the true CPDAG on the linear benchmark. We hope you will consider these findings in your final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.