acceptodds
Under review as a conference paper at ICLR 2027

MirageBench: When Does Causal Discovery Survive Latent Confounding?

Abstract

Latent confounding occurs when an unobserved common cause influences multiple observed variables, inducing dependence that can be mistaken for a direct causal relation. Yet the reliability of causal discovery under increasing confounding remains difficult to assess. In this work, we study this problem in observational multivariate time series, where direct lagged relations provide an explicit structural target. We introduce MirageBench, which preserves the same lagged-edge ground truth while varying confounding strength, the number of observed variables, the record length, and latent complexity. To compare heterogeneous method outputs, we introduce POLE, an output-aware protocol that separately evaluates Pair ranking, Operating points, Lag localization, and Error control. Controlled experiments reveal three findings. First, methods and evaluation layers respond differently as confounding strengthens. Second, near-perfect pair ranking can coexist with substantial false discoveries in the selected set. Third, error-control recovery depends on observational resources, latent complexity, and selector choice: observing more variables can restore control, but the number required for recovery increases with latent complexity and record length. Together, these results show that reliability under latent confounding depends on what is evaluated, which selector is used, and what observational resources are available.

Primary area:
datasets and benchmarks
Keywords:
causal discovery, latent confounding, multivariate time series

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QGREOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MIRAGEBENCH, which varies latent confounding while preserving the underlying direct lagged causal structure. Its POLE protocol separately evaluates pair ranking, operating points, lag localization, and error control. Experiments with 22 methods show that strong ranking performance can coexist with substantial false discoveries. Increasing the number of observed variables improves error control for some methods, whereas longer time series do not provide comparable benefits.

Critical strengths
  1. Important problem and clear evaluation. Separating ranking quality, structural recovery, and error control avoids characterizing reliability through a single metric.
  2. Informative empirical findings. Spectral deconfounding achieves an AUPR of 0.996 while exhibiting a mean FDP of 0.200, illustrating that excellent ranking does not guarantee reliable edge selection. Reporting power alongside FDP and explicitly stating the scope of the findings are additional strengths.
Critical weaknesses
  1. The contribution of shared confounding remains insufficiently isolated. In Table 13, spectral deconfounding has an FDP of 0.272 under independent latent forcing, compared with 0.204 under shared forcing. False discoveries may therefore also reflect temporal dependence or model misspecification. Their contributions to the observed resource-dependent trends remain unclear.
  2. Resource variation changes other problem characteristics. Different values of N correspond to independently generated systems with changing graph properties. Increasing H also increases total latent influence without energy normalization. The effects of observation count, network complexity, and latent influence are therefore not fully separated.
  3. Method and scenario coverage is limited. Experiments focus on linear-Gaussian systems. Methods specifically designed to accommodate latent confounding, such as LPCMCI, are not scored because compatible ground truth is unavailable, limiting the benchmark’s coverage.
  4. Recovery-threshold robustness requires further validation. Classification depends on 30 repetitions, a t-test, and multiplicity correction. Identical data can yield different thresholds under different correction families. Some settings with mean FDP displayed as 0.00 in Figure 9 are nevertheless classified as undecided and require further explanation. (Appendix A.1; Table 15)
Questions for authors
  1. Do the main resource-dependent trends persist when the shared-versus-independent latent-forcing comparison is extended across different (N,T) settings?
  2. Do the recovery boundaries remain stable when the core causal structure is fixed and latent input energy is normalized across different values of H?
  3. Could an evaluation track with compatible ground truth be developed for methods such as LPCMCI to improve coverage of approaches that accommodate latent confounding?
  4. Why are settings with near-zero mean FDP classified as undecided? What are their selection sizes and power, and do the thresholds remain stable with more repetitions or alternative inference procedures?
4 comments
AuthorsOpenReview ↗

Response to Reviewer QGRE (1/2)

We thank the reviewer for the constructive comments and for recognizing the importance of the problem, the separation of ranking quality, structural recovery and error control, and the informativeness of the empirical findings. We address each of your concerns point by point below.

Shared versus independent forcing across (N,T)(N,T) (W1, Q1). To address the reviewer's question, we have run additional experiments that extend the shared-versus-independent comparison along both NN and TT, and as a further control we have added variance-matched, temporally independent private forcing. For spectral deconfounding and Knockoffs, the FDP trends along both resource axes persist under independent (private) AR(1) forcing, although their endpoint changes are smaller than under shared forcing. The table gives the mean FDP at the endpoints of each axis:

MethodForcingNN: 20 →\to 120 (T=600T=600)TT: 300 →\to 4000 (N=20N=20)
SpectralClean0.034 →\to 0.0140.032 →\to 0.054
Shared0.533 →\to 0.0230.303 →\to 0.851
Private AR(1)0.426 →\to 0.0840.336 →\to 0.630
Private, temporally independent0.048 →\to 0.0120.037 →\to 0.048
KnockoffsClean0.082 →\to 0.0890.087 →\to 0.103
Shared0.686 →\to 0.1090.501 →\to 0.875
Private AR(1)0.466 →\to 0.1590.386 →\to 0.651
Private, temporally independent0.088 →\to 0.0930.085 →\to 0.080

Across the two methods and both axes, the endpoint changes under private AR(1) forcing are 53–71% of those under shared forcing, whereas temporally independent private forcing stays close to the clean reference. The Table 13 ordering is a crossing of the NN-response curves: for spectral deconfounding, shared forcing starts higher at N=20N=20 but declines more steeply, leaving private forcing higher at N=40N=40. Sharing has a distinct effect on marginal ranking: SIS pair AUPR collapses under shared forcing at every evaluated (N,T)(N,T), but remains high under both private controls. We report the full comparison, including CPSS, in Table 14 of Appendix D.1 and summarize this distinction in Section 4.2.

Fixed core and energy-normalized driver count (W2, Q2). We thank the reviewer for raising this concern. We examined each of the two sources of variation with its own control: (1) a fixed causal core across NN, and (2) energy-normalized loadings across HH. The main spectral-deconfounding pattern remains in both: FDP on the fixed core decreases with NN, and the recovery boundary still moves outward with HH.

(1) Fixed core across NN. We added a nested control in which a fixed 20-variable core is observed together with additional variables, so that the core causal structure and the truth of its edges are the same at every NN. Mean FDP / power on the core edges at H=1H=1, c=2c=2, and T=600T=600 are:

MethodN=20N=20N=40N=40N=80N=80N=120N=120
Spectral deconfounding0.533 / 1.000.239 / 0.990.120 / 0.990.094 / 0.98
Knockoffs0.686 / 0.980.464 / 0.970.479 / 0.940.501 / 0.90
CPSS0.616 / 0.570.580 / 0.520.558 / 0.500.542 / 0.50
SIS0.679 / 0.180.825 / 0.190.876 / 0.260.897 / 0.30

Spectral deconfounding reduces FDP from 0.533 to 0.094 while power stays above 0.97. The decline is smaller for Knockoffs and CPSS, and SIS moves in the opposite direction. Thus, observing more variables of the same system lowers FDP on fixed targets for spectral deconfounding, while the size and direction of the effect remain selector-specific. We have added this comparison to Table 19 of Appendix D.2 and summarized it in Section 4.2.

(2) Energy-normalized driver count across HH. The outward shift of the spectral recovery boundary also remains after controlling for total latent input energy. As described in Appendix C.3, the direct structure and the earlier drivers are held fixed across HH. We additionally rescaled the loadings so that each variable has the same latent forcing variance as at H=1H=1, and classified the (N,H)(N,H) grid at T=600T=600 as its own 30-cell family, of the same width as the family behind the benchmark-loading thresholds:

Loading schemeH=1H=1H=2H=2H=3H=3H=4H=4H=5H=5
Benchmark loadings608080120120
Energy-normalized6080120120120

Under energy normalization, the spectral threshold still moves from 60 to 80 and then 120. Thus, within this generator, the effect of driver count is not explained by total latent energy alone. We report the full comparison in Table 21 of Appendix D.3 and now state this result in Section 4.3.

AuthorsOpenReview ↗

Response to Reviewer QGRE (2/2)

Latent-aware methods (W3, Q3). We thank the reviewer for this suggestion. A compatible ground-truth track for methods such as LPCMCI is feasible in principle and would broaden the benchmark's coverage, because the generator retains the observed and latent structural metadata needed to derive a latent-projected reference. However, LPCMCI estimates a partial ancestral graph (PAG) whose adjacencies can include relations induced by latent common causes and whose endpoint marks encode ancestral constraints rather than one-step direct effects. Scoring this output against our direct lagged-edge truth could therefore count some correct PAG adjacencies as false direct-edge discoveries. A valid track requires a lag-window-matched latent-projected reference together with a mark-aware scoring rule. We nonetheless recognize this as a valuable extension and have added it as a concrete direction in Appendix E.6.

Recovery-threshold robustness (W4, Q4). To address the reviewer's question, we answer its two parts in turn. (1) The undecided cells that print a mean FDP of 0.00 in Figure 9 are the five Knockoffs cells at N=120N=120, T=300T=300, one for each number of latent drivers H=1,…,5H=1,\dots,5. As stated in Section 3.3 and Appendix A.1, a mean FDP below qq does not by itself establish recovery, and a cell in which every run has zero FDP has an undefined tt statistic and is left undecided. In these five cells the screen selects on average only 0.8 to 7.3 lagged edges per run, at mean power below 0.01, so the printed 0.00 accompanies an almost empty selection, and recovery remains undecided under the zero-variance rule. We now state this link between Figure 9 and the classification rule in Appendix D.2.

(2) We evaluated separately the three sources of sensitivity the reviewer raises: the test statistic, the number of repetitions, and the multiplicity family. The table counts the changes relative to the reported classification:

VariationRuns per cellCell verdicts changedSlice thresholds changed
Wilcoxon signed-rank test300 of 4500 of 75
Studentized bootstrap300 of 4500 of 75
Holm within each record length301 of 4501 of 75
Benjamini–Hochberg309 of 4506 of 75
Original runs, threshold cells only30–0 of 13
Doubled runs, threshold cells only60–1 of 13

The choice of test statistic changes no verdict or threshold. Narrowing the Holm family to each record length moves only the spectral-deconfounding threshold at H=1H=1, T=600T=600, by one grid step from 80 to 60, the cell that Section 4.3 already names as sensitive to the correction family. Doubling the runs at the 13 numeric spectral-deconfounding thresholds, classified in a Holm family of those 26 cells, moves the same threshold to 60 and leaves the other 12 unchanged. The original 30 runs, classified in that same family, leave all 13 unchanged, so this shift reflects the added runs rather than the narrower family. Benjamini–Hochberg moves the same threshold and additionally resolves five slices that the Holm rule leaves unresolved, one at N⋆=120N^\star=120 and four as right-censored beyond the grid. The numeric thresholds therefore hold within one grid step, while whether a few boundaries can be resolved depends on the multiplicity family. We have added this sensitivity analysis to Table 20 of Appendix D.2.

Thank you again for your positive assessment and for supporting our work. Please feel free to let us know if you have any further questions. Thank you!

Thank you for the author's response; my concerns have been resolved, and I will keep my score unchanged.

Thank you for your valuable suggestions and comments, which have helped us strengthen the paper.

Reviewer_8C77OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates confounding issues for Causal Discovery (specifically time-series methods and lagged effects) and introduces a benchmark as well as a specific evaluation protocol that allows for a more fine grained analysis of failure cases. It compares various causal discovery methods for time-series and finds that responses may vary. They also conclude that more time steps do not have a positive effect while including a wider cross section (more visible variables) can potentially improve performance.

Critical strengths

The paper introduces an evaluation protocol that seems very reasonable. It is true that the current literature falls short of metrics that allow finegrained comparison of behavior as most use a single quality metric such as AUROC, F1 or SHD. Therefore, the protocol would be a meaningful contribution

The paper is clearly written and easy to follow. The conclusions are also quite interesting.

Critical weaknesses

While this paper targets an important issue and the methodology is sound, the title of the paper suggests a much broader investigation of what is actually happening in the paper:

  • The paper only targets lagged links and no instantaneous effects
  • The paper relies on an AR(1) model for confounding
  • The paper relies on a linear SCM
  • The confounding type in the main section is always dense.
  • The paper relies on oddly specific effect size ranges for actual links and confounding.
  • Innovation noise is always Gaussian
  • 4 specific confounding levels.
  • Stationary

With this, the mathematical model and the presented results are quite narrow. I am unsure if this is sufficient for a main track paper.

Further points see below.

Questions for authors

As TCD-Arena does investigate a very similar setup, I am missing a direct comparison of results. Are the conclusions similar?

In a linear SCM with Gaussian noise, the false links that are introduced by confounding and their inseparability from real links should depend exactly on the strength of the confounding links and when it crosses the strength of the actual links. I am missing an analysis here, especially as the confounding and the link ranges are not equal.

Finally, I am concerned with the fact that the paper does not investigate the instantaneous predictions of methods under investigation. I believe introducing AR(1) confounding is introducing many instantaneous predictions if the method is capable of them. Analyzing this behavior and whether methods suffer more if they do not estimate these effects would be very interesting.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 8C77 (1/2)

We thank the reviewer for the constructive comments and for recognizing the fine-grained evaluation protocol and the clarity of the paper. We address each of your concerns point by point below.

Scope and effect-size ranges (W). We thank the reviewer for the detailed comments. MirageBench is designed to identify when selected-set error control survives latent confounding. To this end, it combines graded latent confounding, evaluated NN and TT variation, multi-resolution output-aware evaluation and a prespecified error target; among the benchmarks compared in Table 1, only MirageBench offers this combination, which allows error-control recovery boundaries to be located. The title states this scientific question, and the abstract and Section 3 now specify the stationary linear-Gaussian regime in which we answer it. Within this regime, the direct lagged-edge target stays fixed across confounding strengths, and Tables 3 and 16 also evaluate full, half and quarter loading breadth. In addition, at the RQ1 setting, Student-tt innovations and alternative latent autocorrelation change pair AUPR and FDP by less than 0.10 for each of the four tested methods. Finally, we address instantaneous outputs in our answer below. Nonlinear and non-stationary dynamics are natural extensions of the present design.

Regarding the effect sizes, the ranges for links and confounding, together with the four confounding levels, define a fixed reference calibration rather than domain-specific estimates. Across this calibration, the 95th percentile of pairwise coefficients on non-edges crosses the 5th percentile of true-edge coefficients at c=1c=1, as Appendix D.1 shows. We thank the reviewer for raising this point and now state this interpretation in Appendix B.1.

Comparison with TCD-Arena. We thank the reviewer for this helpful question. Comparing the results of the two benchmarks, we find that their conclusions are similar for ranking under lagged hidden confounding and complementary for error control. The six methods evaluated in both benchmarks (PCMCI, F-PCMCI, VARLiNGAM, GVAR, DYNOTEARS and NTS-NOTEARS) all rank well under lagged hidden confounding (AUROC 0.83–0.96 in TCD-Arena [1]; pair AUPR 0.88–1.00 at c=2c=2 in ours). However, the two benchmarks serve different purposes: TCD-Arena measures how graph recovery degrades across a broad range of assumption violations, whereas MirageBench asks whether selected-set error control survives latent confounding. Our central findings therefore concern quantities outside TCD-Arena's design, namely selected-set FDP against a prespecified error target and its recovery over NN and TT: ranking can remain strong while error control fails, and NN and TT affect error-control recovery differently in the evaluated regime. We now state this result-level comparison in Section 2.

AuthorsOpenReview ↗

Response to Reviewer 8C77 (2/2)

Confounding versus link strength. We thank the reviewer for this insightful point. Because the latent-loading and direct-effect ranges do not translate one-to-one into observed associations, we have added a population-level analysis of the 30 RQ1 instances, using their exact stationary autocovariances to obtain the lagged regression coefficients without sampling noise. The table compares the absolute coefficients on non-edges with those on true edges:

Pairwise regressionFull-conditional regression
ccNon-edge 95th pct.True-edge 5th pct.Non-edge maxTrue-edge 5th pct.
00.0680.251$<$0.0010.258
0.50.0870.2350.0540.257
10.1820.1790.0950.255
20.3950.0820.1210.254

In pairwise regressions, the crossing first occurs at c=1c=1, and at c=2c=2, 55.6% of non-edges exceed the 5th percentile of true-edge coefficients. In contrast, even the largest full-conditional non-edge coefficient stays below the corresponding true-edge percentile throughout the evaluated grid. This contrast between pairwise and full-conditional coefficients is consistent with the ranking split in Section 4.1, where marginal scores deteriorate sharply while conditional scores remain near the ceiling. We have added this analysis to Table 15 of Appendix D.1.

Instantaneous predictions. To address the reviewer's question, we answer its two parts in turn.

(1) At N=40N=40, H=1H=1, and T=600T=600, strong AR(1) confounding produces many more lag-0 predictions in both methods that estimate them:

MethodLag-0 pairs, c=0c=0Lag-0 pairs, c=2c=2Same latent-driver lag, c=2c=2
DYNOTEARS3.6215.593.8%
VARLiNGAM6.7135.892.6%

At c=2c=2, more than 92% of these pairs join variables that receive the latent driver at the same lag. Because the benchmark truth contains only direct positive-lag edges, we analyze these outputs separately rather than include them in the primary scoring.

(2) For DYNOTEARS, forbidding lag-0 edges raises mean lagged FDP under strong confounding. We reran DYNOTEARS on the same records with all lag-0 edges forbidden, while keeping its penalties, solver, lagged selection rule and scoring fixed, using 30 paired runs at each confounding strength:

Lagged FDPSelected edges
ccLag-0 estimatedLag-0 forbiddenLag-0 estimatedLag-0 forbidden
00.0700.071 ($\uparrow$0.002)116.4116.6
20.5690.735 ($\uparrow$0.166)238.0383.4

At c=0c=0, the two variants nearly coincide. At c=2c=2, forbidding lag-0 edges increases the selected set by about 145 edges while power changes only from 0.942 to 0.935, indicating that the additional selections are false discoveries rather than additional true-edge recovery. Thus, within DYNOTEARS at this setting, estimating lag-0 edges mitigates but does not eliminate the confounding-related increase in lagged FDP.

We report both analyses in Table 25 of Appendix D.5 and clarify the primary scoring rule in Appendix B.3.

Thank you again for the detailed comments, which helped us improve the paper. We hope the additional analyses and revisions address your concerns, and we would be grateful if you could take them into account in your final assessment. Please feel free to let us know if you have any further questions; we are happy to discuss them.

References

[1] TCD-Arena: Assessing Robustness of Time Series Causal Discovery Methods Against Assumption Violations. ICLR, 2026.

Thank you for addressing my concerns and adding additional experiments. While I remain with my assessment that the scope of the experiments are a) Very narrow for the venue and b) Mismatched with the title of the paper, I believe the additions to the Supplementary material and the adjustment of the abstract are sufficient to slightly raise my score.

Thank you for the follow-up and for raising the score. We appreciate your acknowledgment of the added experiments and the revised abstract. Regarding the title, we will make the lagged time-series setting explicit in the final version, so that the scope is clear from the title alone. Thank you!

Reviewer_ZXCfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces MirageBench, a synthetic benchmark for evaluating time-series causal discovery with unobserved common causes. Its linear-Gaussian generator varies the strength of latent forcing while keeping the direct lagged-edge ground truth fixed. The proposed POLE protocol assesses pair ranking, operating points, lag localization, and error control separately. Experiments on 22 methods show that near-perfect pair ranking can coexist with substantial false discoveries. The authors also study how the number of observed variables, record length, and latent drivers affect error control, and report empirical recovery thresholds for selected methods.

Critical strengths
  1. The fixed direct-edge ground truth across confounding levels is a useful design choice. It allows the paper to compare methods under stronger latent forcing without also changing which direct effects should be recovered (Section 3.1).

  2. The separation of ranking quality from selected-set error is important. Figure 4 gives a concrete example: Spectral deconfounding has pair AUPR of 0.996 but mean FDP of 0.200. Reporting power alongside FDP also helps prevent low-error but nearly empty selections from looking artificially good.

Critical weaknesses
  1. I am not convinced that the reported failures can always be attributed specifically to shared latent confounding. In Appendix Table 13, Spectral deconfounding has a higher FDP with private forcing (0.272) than with shared forcing (0.204). Both depart from the clean condition, but only one contains a common latent driver across observed variables. This raises a question about how much of the deterioration comes from shared confounding as opposed to temporal forcing or model misspecification.

  2. The interpretation of the resource experiments is not fully supported by the comparisons used. In Section 4.2, systems at different N are generated independently, so the reported reduction in FDP does not isolate the effect of adding observations to the same underlying system. Similarly, as T grows, the selected set and power also increase. The resulting increase in FDP could partly be an operating-point effect rather than evidence that longer records make the causal structure harder to identify.

Questions for authors
  1. Appendix Table 13: why does private forcing produce a higher FDP than shared forcing for Spectral deconfounding (0.272 versus 0.204)? How should this result be interpreted in relation to the paper’s focus on shared latent confounding? A comparison with variance-matched temporally independent noise would help distinguish temporal disturbance effects from shared confounding.

  2. Figures 5–6: does the reduction in FDP with larger N persist if a core causal graph and its target edges are held fixed, while additional observed variables are introduced? Evaluating the original edges in such a nested setting would help establish whether the improvement comes from observing more variables rather than changes in the generated graph.

  3. Section 4.2: does FDP still increase with T when methods are compared at the same number of selected lagged edges, or at matched power? This would clarify whether the reported trend reflects weaker identification or simply more liberal selection with longer records.

5 comments
AuthorsOpenReview ↗

Response to Reviewer ZXCf (1/2)

We thank the reviewer for the constructive comments and for recognizing the separation of ranking from selected-set error and the joint reporting of power and FDP. We have completed the three analyses the reviewer proposed: variance-matched temporally independent forcing, a nested core with fixed target edges, and comparisons at matched selection size and matched power along TT. We address each of your concerns point by point below.

Shared versus private forcing (W1, Q1). Regarding attribution, the reviewer is right that the error-control response of spectral deconfounding and Knockoffs is not specific to a common latent driver, while the collapse of SIS marginal scores in Table 13 remains specific to sharing. Following the reviewer's suggestion, we have added variance-matched temporally independent noise and extended the shared/private comparison along both resource axes, which gives the following mean FDP for spectral deconfounding at T=600T=600:

ForcingN=20N=20N=40N=40N=120N=120
Clean0.0340.0330.014
Shared0.5330.2220.023
Private AR(1)0.4260.2710.084
Private, temporally independent0.0480.0310.012

The N=40N=40 reversal in Table 13 is a crossing of the NN-response curves: shared forcing starts with a higher FDP at N=20N=20 but declines more steeply, leaving private forcing higher at N=40N=40. Across the full comparison, the temporally independent noise remains close to the clean reference, while private AR(1) forcing retains 53–71% of the resource-axis change under shared forcing for both selectors, along both NN and TT. Sharing therefore amplifies the error-control response, and within the unshared condition the excess FDP is associated with temporal dependence rather than with the matched added variance. We report the full comparison in Table 14 of Appendix D.1 and now state this distinction in Section 4.2.

Nested cross-section (W2, Q2). To address the reviewer's question, we have added an experiment in the nested setting: with the core causal graph and its target edges held fixed, the FDP reduction along NN persists for spectral deconfounding and is partial for Knockoffs and CPSS. A fixed 20-variable core, whose series are identical at every NN, is observed together with additional variables that receive edges from the core and share the latent driver but send no edge into it, so the core edges keep the same truth; on these edges at c=2c=2 and T=600T=600, the mean FDP and power are:

MethodN=20N=20N=40N=40N=80N=80N=120N=120
Spectral deconf.0.533 / 1.000.239 / 0.990.120 / 0.990.094 / 0.98
Knockoffs0.686 / 0.980.464 / 0.970.479 / 0.940.501 / 0.90
CPSS0.616 / 0.570.580 / 0.520.558 / 0.500.542 / 0.50
SIS0.679 / 0.180.825 / 0.190.876 / 0.260.897 / 0.30

For spectral deconfounding, observing more variables of the same system lowers the error on fixed targets almost as much as the comparison of independently generated systems suggests; for Knockoffs and CPSS most of that decline does not carry over, and SIS moves in the opposite direction, so the size of the cross-section effect is selector-specific. We report these results in Table 19 of Appendix D.2 and summarize them in Section 4.2.

AuthorsOpenReview ↗

Response to Reviewer ZXCf (2/2)

Operating point along TT (W2, Q3). To address the reviewer's concern that the rise in FDP with TT may be an operating-point effect, we have reanalyzed the existing record-length runs at N=20N=20 and c=2c=2, comparing methods at the same number of selected lagged edges and at matched power: for spectral deconfounding and Knockoffs, FDP falls with TT under both, while their reported FDP rises. We rank each method's lagged edges by its own evidence and score either the top ∣E⋆∣|E^\star| edges or the smallest set reaching power 0.9, which gives the following mean FDP from T=300T=300 to T=4000T=4000:

MethodSame number of edgesEdge power 0.9Reported set
Spectral deconf.0.170 →\to 0.0170.311 →\to 0.0010.311 →\to 0.858
Knockoffs0.242 →\to 0.0190.525 →\to 0.0050.482 →\to 0.883

The rise therefore reflects more liberal selection with longer records rather than weaker identification, which is what the error-control layer measures: the realized FDP at each selector's own operating point. CPSS and PCMCI show little change at the same number of selected edges. On the crossed grid, pair AUPR is higher at T=4000T=4000 than at T=300T=300 at every evaluated (N,H)(N,H) for spectral deconfounding and Knockoffs. We report this reanalysis in Table 17 of Appendix D.2 and now state this reading in Section 4.2.

Thank you again for the detailed comments, which have helped us improve the paper. We hope the additional analyses and revisions address your concerns, and we would be grateful if you could take them into account in your final assessment. Please feel free to let us know if you have any further questions; we are happy to discuss them.

Thank you for your response. Some of my concerns have been addressed. However, the title of the paper is inappropriate, and it may mislead readers into thinking it applies to the cases of static causal discovery and instantaneous causal discovery in time series.  The application scenarios proposed in the benchmark are too limited to be a significant contribution. It does not even include nonlinear and instantaneous causal discovery in time series. Therefore, I want to keep the score unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.