- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Mask2Cause, an end-to-end Transformer framework for causal discovery through conditional mean and variance prediction. Its main idea is to capture relationships that affect a variable’s variability but leave its mean unchanged. The experiments cover several benchmarks and downstream tasks, including forecasting and root cause localization.
Critical strengths
-
The motivation is clear. Looking beyond mean prediction addresses a useful limitation of MSE-based methods.
-
The experiments are fairly broad, with useful comparisons, ablations, and additional analyses in the appendix.
Critical weaknesses
-
In the multi-layer encoder, blocking a direct connection does not prevent information from reaching the target through another token. This makes the interpretation of the learned adjacency as direct dependencies less clear.
-
RHINO also achieves perfect recovery in the pure-variance setting in Tab. 4. A clearer comparison of recovery accuracy and computational cost would help show M2C’s practical advantage.
-
Several gains in Tab. 3 are small, and may require statistical evidence.
Questions for authors
-
How does information passing through intermediate tokens affect the interpretation of the learned adjacency in the multi-layer encoder?
-
Can you summarize the accuracy and computational cost trade-off between M2C and RHINO under the same experimental settings?
-
Are the small improvements in Tab. 3 consistent across repeated experiments?
2 comments
Thank you for the detailed rebuttal and additional experiments. While I appreciate the authors' efforts, the rebuttal does not change my overall assessment of the paper. Therefore, I will maintain my original score.
Q1
This possibility is explicitly discussed in Appendix G. Lines 1109–1110 state: “In a multi-layer setting, a theoretical discrepancy arises between the ideal estimator assumption and the structural mask.” We can therefore interpret the adjacency as a ranking of candidate parents. We discuss what supports that interpretation in our response to Reviewer yjvT, W1.
Q2
Table 4 shows that M2C-NLL matches or exceeds RHINO’s mean off-diagonal AUROC in six of seven settings. On RHINO’s history-dependent Erdős–Rényi benchmark, M2C-NLL leads at N=5,10,40, with the largest gain at N=5 (0.885 versus 0.798). The two methods are nearly tied at N=20 (0.888 versus 0.890). On Mixed Physics, M2C-NLL leads by 0.030 and 0.023 in the 50:50 and 75:25 regimes, respectively, and both achieve 1.000 in the pure-variance regime.
We measured the wall-clock time for a full training run using the selected settings, including the training duration selected during calibration for M2C-NLL (Appendix~O.3). The timings exclude hyperparameter search.
On Mixed Physics (50:50, 75:25, and 100:0), M2C-NLL took 2.96, 1.42, and 0.83 minutes, respectively, compared with 24.22, 24.70, and 24.60 minutes for RHINO. On RHINO ER (N=5,10,20,40), M2C-NLL took 1.02, 3.88, 0.86, and 12.64 minutes, respectively, compared with 97.75, 42.04, 39.36, and 44.63 minutes for RHINO. Across the seven datasets, M2C-NLL achieved a 3.5–95.8× speedup over RHINO, with a mean of 30.2x.
Table 20 shows the scaling of compute with N and L for a single forward pass. RHINO has a smaller footprint at N=10,L=5, with 0.0323M parameters and 1.80M FLOPs versus 0.1006M and 2.05M for M2C-NLL. However at scale, M2C-NLL is significantly cheaper. At N=2000,L=5, M2C-NLL uses 4.10M parameters and 2.51B FLOPs versus 70.66M and 8.28B for RHINO. At N=10,L=2000, M2C-NLL uses 0.2283M parameters and 4.60M FLOPs versus 1.0298M and 555.20M for RHINO.
Q3
We repeated the DREAM3 experiments with five training seeds per model and network. The data and hyperparameters were fixed. Only weight initialization, mini-batch order, and dropout varied. AUROC was stable across runs, with standard deviations of 0.001–0.005. The clearest gain was on E. coli-2, where M2C-NLL averaged 0.703±0.003 versus 0.666 for the strongest baseline. Its leads on E. coli-1 and Yeast-2 were smaller (0.004 and 0.003), and we do not claim significant gains there. The intended role of DREAM3 is stated in Section 5.2.1, lines 325–326: “DREAM3, which features extremely short trajectories T=21, is purely used to evaluate how the model fares against baselines in highly data-scarce domains.” The new five-seed results in Appendix W support the stability of M2C’s scores in this setting.