acceptodds
Under review as a conference paper at ICLR 2027

fDAG-ACDC: Uncertainty-Adaptive Factor-Count Selection for High-Dimensional Causal Discovery

Abstract

Causal representation learning (CRL) methods aim to discover low-dimensional representations whose components correspond to the mechanisms that generate high-dimensional observations. Changing the number of latent components affects the interpretation of the inferred mechanisms, yet the correct number of mechanisms is rarely known. Hence there is a need for model-selection criteria to determine the number of components. However, existing criteria are generally inapplicable to CRL models because adding latent structure can improve global fit by absorbing misspecification rather than capturing distinct mechanisms. In this work, we address this gap, focusing on factor-directed acyclic graphs (f-DAGs) as a test case. We introduce fDAG-ACDC, an uncertainty-adaptive extension of the accumulated cutoff discrepancy criterion (ACDC), which compares observed and posterior-predictive distributions of each node and its estimated parents within each perturbation environment. Theoretically, we prove model selection consistency of fDAG-ACDC under intuitive conditions. In simulations, fDAG-ACDC's mean absolute selection error is just 0.5, while validation likelihood and oracle graph-recovery baselines over-select by more than 3 on average. On Perturb-seq and organoid data, fDAG-ACDC selects fewer factors than validation likelihood while matching or nearly matching larger models in predicting biological summaries, and its Perturb-seq factors are enriched for known gene programs.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
causal representation learning, causal discovery, model selection, mechanistic interpretability, model misspecification, posterior predictive checks

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 4 reviews, above 99% of the others.
Reviewer_qTZ1OpenReview ↗
Rating
3(+2)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose fDAG-ACDC, a methodology focusing on estimating the number of latent components in a causal representation learning model. The proposed methodology doesn’t learn the causal DAG itself, but rather employs any compatible fDAG learning method (in particular ABCDEFG) for a given number of factors m and then uses the modified criterion ACDC to assess the quality of the estimation. The method is evaluated in two synthetic experimental settings with a fixed fDAG learning method and comparing 4 distinct criteria for assesing the choice of m. Finally it is evaluated on a real world dataset where it shows improvement over Validation NLL.

Critical strengths

The authors study a narrow but important and motivated problem in the CRL literature. Their estimation of the number of the latent factors was previously “ignored” in the literature.

The method shows improvement in two experimental settings and in two real datasets. The improvement is in the choice of factors m comparing to other criteria that can be used to choose m.

Critical weaknesses

In theory, I am afraid that when the number m of factors is large, then the complexity of searching it in a brute force way rises significantly. The theory provides a guarantee for identifying the minimum m0, and it is mentioned that under an identifiability guarantee the unique generating factor can be identified (lines 110-112). The latter is never analyzed.

The experimental analysis is quite narrow. Only two experimental setups are considered with a small number of factors m. The baselines chosen as criteria are only three and other criteria are excluded with no supportive justification. In particular AIC/BIC were avoided as they are not well defined for neural models (as claimed in the text), but the experiments use a linear SEM (Eq. 34). Moreover, such important experimental details regarding the data generation (Eq. 34), should be in the main text rather in the appendix.

Questions for authors

What is the complexity of looking for all m’s? Wouldn’t that be a bottleneck in problems with significantly large m?

Can you please explain the identifiability guarantee for estimating the unique generating factor number? Please see weknesses above.

It is a bit confusing to me why F1 and SHD are called oracles whereas they don’t achieve optimal performance.

Can you please explain why you didn’t use AIC/BIC on a linear SEM setup? Also, can you please confirm that the data you generate come from a linear SEM?

4 comments
AuthorsOpenReview ↗

Response to Reviewer qTZ1 (Q1–Q2)

We sincerely thank you for your time and effort in reviewing our paper.

Q1: What is the complexity of looking for all m’s? Wouldn’t that be a bottleneck in problems with significantly large m?

Candidate enumeration is linear in grid size at fixed per-candidate cost. With K=∣M∣K = |\mathcal{M}| candidates and SS paired fitting replicates, this requires SKSK candidate fits. We evaluate a finite grid of candidate counts, fitting each candidate with ABCDEFG. This is a sweep over factor counts rather than exhaustive enumeration of graph structures. A validation-NLL comparison using the same candidate grid and replicates shares the underlying fitting cost.

Post-fitting complexity. Candidate enumeration scales linearly with KK at fixed per-candidate costs; dependence on mm enters through model size and local neighborhoods. For the additional post-fitting computation, let dd be the number of observed nodes and BB the number of bootstrap resamples. With at most RR scorable environments per local unit, nn observations per environment, and kk variables per neighborhood, dense covariance estimation and Gaussian KL evaluation cost O(nk2+k3)O(nk^2 + k^3) per local environment. Writing TgraphT_{\mathrm{graph}} and TppT_{\mathrm{pp}} for per-fit bounds on graph/neighborhood construction and predictive generation, the post-fitting cost is bounded by

Tpost=O(SK[Tgraph+Tpp+dR(nk2+k3)]+BKSlog⁡S).T_{\mathrm{post}} = O\left(SK\left[T_{\mathrm{graph}} + T_{\mathrm{pp}} + dR(nk^2 + k^3)\right] + BKS\log S\right).

The bootstrap term comes from sorting SS values per candidate and resampling cached criterion curves.

Large models or wide grids can become computational bottlenecks. Yes, substantially larger candidate models or a wider candidate grid can make candidate-model fitting and evaluation a computational bottleneck. A wider grid increases the number of fits, while larger mm can increase fitting, predictive-generation, and local-discrepancy costs. Thus, the dependence on KK is linear at fixed per-candidate costs, while overall scalability also depends on the size of each candidate model. Candidate fits can run independently in parallel.

We have added a scalability summary to Section 4.1 and training-time measurements and complexity analysis to Appendix B.2 (Tables 8–9, Equation (36)).

Q2: Can you please explain the identifiability guarantee for estimating the unique generating factor number? Please see weaknesses above.

Sufficient conditions: local adequacy and separation from smaller candidates. Let m⋆∈Mselm^\star \in \mathcal{M}_{\mathrm{sel}} be the minimal, nonredundant generating count. First, the population fitted model at m⋆m^\star must reproduce every scored local distribution, giving C(m⋆)=0C(m^\star) = 0. Second, each smaller candidate must mismatch at least one scored local distribution with positive environment weight. For the Gaussian implementation, a difference in a local mean or covariance is sufficient, assuming positive-definite covariances. For such a local unit jj and environment rr, nonnegativity of KL gives

C(m)≥wjmrd kj,mKL[Pjmr∥Qjmr]>0=C(m⋆),m<m⋆.C(m) \geq \frac{w_{jmr}}{d\,k_{j,m}} \mathrm{KL}\left[P_{jmr}\parallel Q_{jmr}\right] > 0 = C(m^\star), \qquad m < m^\star.

Thus m0=m⋆m_0 = m^\star, with a positive underfitting gap on the finite candidate grid.

Linear–Gaussian illustration. Consider candidates with a shared linear decoder X=LmZ+bm+ϵX = L_m Z + b_m + \epsilon, where Lm∈Rd×mL_m \in \mathbb{R}^{d \times m}, bmb_m and the independent, mean-zero Gaussian observation noise are environment-invariant, and interventions shift Gaussian factor means. At the true count, suppose L⋆L_\star has full column rank and the factor-mean contrasts span Rm⋆\mathbb{R}^{m^\star}. Write μr=E[X∣r]\mu_r = \mathbb{E}[X \mid r] and νr=E[Z∣r]\nu_r = \mathbb{E}[Z \mid r]. Across environments r=0,…,Rr = 0, \ldots, R, the observed mean-contrast matrix satisfies

= L_\star[\nu_1 - \nu_0, \ldots, \nu_R - \nu_0], \qquad \operatorname{rank}(D) = m^\star.$$ Every $m$-factor candidate in this shared-decoder family has a predictive mean-contrast matrix of rank at most $m$. Hence a candidate with $m < m^\star$ cannot reproduce all environment-specific observed means. If all these environments are scored with positive weight, some observed node has a mean mismatch in its own local unit, ensuring positive local Gaussian KL. Together with an accurate population predictive fit at $m^\star$, this verifies the two conditions above. The rank requirement expresses an intuitive intervention condition: the shifts must reveal every factor direction through the observations. **Recovery follows under the consistency assumptions.** Under these sufficient conditions and the respective assumptions of Theorems 1 and 2, the pointwise and contiguous selectors recover $m^\star$ with probability tending to one as $S$ increases. We have added the sufficient conditions linking the generating count to the consistency target in Appendix D.9 (Equation (100)).
AuthorsOpenReview ↗

Response to Reviewer qTZ1 (Q3)

Q3: It is a bit confusing to me why F1 and SHD are called oracles whereas they don’t achieve optimal performance.

F1 and SHD require access to the true graph. Both metrics compare an estimated graph with the true graph, so they cannot be computed in real-world applications where the true graph is unknown. The term “oracle” refers to this access to ground truth. It does not guarantee optimal factor-count selection: maximizing F1 or minimizing SHD targets graph recovery, which need not yield the generating factor count.

Checkpoint selection aims to improve candidate-model quality. In the original simulation experiments, we retained the checkpoint with the highest graph-recovery F1 for each candidate factor count and fitting replicate. This was intended to obtain candidate models with good graph-recovery quality and reduce the influence of imperfect fitting when assessing factor-count selection using ACDC. This use of the true graph occurs during checkpoint selection; the ACDC score and factor-count selection rule do not directly use the true graph.

We have clarified checkpoint selection in Appendix B.2 and made the F1/SHD factor-count selection rules explicit in Appendix B.4 (Equation (41)).

AuthorsOpenReview ↗

Response to Reviewer qTZ1 (Q4)

Q4: Can you please explain why you didn’t use AIC/BIC on a linear SEM setup? Also, can you please confirm that the data you generate come from a linear SEM?

The simulation data come from a linear SEM. Section 4.1 (p. 8, Equation (23)) now states the linear SEM and additive noise. Table 5 (p. 17) details the interventions; Table 6 (p. 18) specifies the linear generative mechanism used for fitting.

AIC/BIC are not plug-in criteria for the fitted pipeline. Linearity of the simulator does not by itself make textbook AIC/BIC applicable to the fitted latent-variable pipeline. Section 1 (p. 2) already discusses the effective-parameter-count requirement and its interpretation in overparameterized CRL models. The classical formulas are

AIC(m)=−2ℓtrain(θ^mML)+2pm,\mathrm{AIC}(m) = -2\ell_{\mathrm{train}}(\hat{\theta}_m^{\mathrm{ML}}) + 2p_m,

BIC(m)=−2ℓtrain(θ^mML)+pmlog⁡N,\mathrm{BIC}(m) = -2\ell_{\mathrm{train}}(\hat{\theta}_m^{\mathrm{ML}}) + p_m \log N,

where ℓtrain\ell_{\mathrm{train}} is the observed-data training log-likelihood, θ^mML\hat{\theta}_m^{\mathrm{ML}} is its maximum-likelihood estimate, NN is the training sample size, and pmp_m is the dimension of the statistical model under the relevant regularity assumptions—not simply the latent count mm or the number of entries in the training code (Akaike, 1974; Schwarz, 1978).

There are two concrete obstacles to applying these formulas directly to our retained fits.

The reported NLL is not the required maximized training likelihood. First, ABCDEFG is fitted using a masked variational autoencoder with learned graph masks, variational KL terms, and sparsity penalties (Appendix B.2, Figure 4 and Table 6, p. 18), rather than by maximizing an unpenalized observed-data likelihood for each fixed classical SEM. The validation NLL reported in Appendix B.4, Equations (37)–(38) (p. 21), is explicitly the held-out decoder reconstruction term. It is not the maximized training log-likelihood obtained by integrating out the latent variables. Adding 2pm2p_m or pmlog⁡Np_m \log N to that reconstruction score would therefore not produce classical AIC or BIC.

A suitable statistical model dimension is needed. Second, a linear input–output mechanism does not make an overparameterized latent-factor representation a regular, uniquely parameterized statistical model. Counting all trainable weights would include variational inference parameters and redundant parameterizations, whereas counting only selected graph edges would omit other distributional parameters. Redundant factors, learned masks, and factorized linear maps can also create nonidentifiability or singularities. The distinction between regular and singular models—and the resulting limitations of ordinary parameter-count corrections—is established in singular learning theory (Watanabe, 2010, 2013). Thus, the concern is not that software cannot count parameters, but that the resulting count and likelihood must have the statistical meaning required by the criterion.

The selection targets differ. There is also a difference in the selection target. AIC estimates global predictive performance, while classical BIC approximates Bayesian model evidence under its regular-model assumptions. Our target is instead the smallest count attaining optimal local structural adequacy under the chosen criterion (Section 3.3, Equation (15)), including settings where added latent structure may absorb misspecification. Neither classical construction automatically guarantees recovery of this target. For these reasons, AIC/BIC are not directly applicable as plug-in baselines for the present fitting-and-scoring protocol.

References

Akaike, H. (1974). A New Look at the Statistical Model Identification. IEEE Transactions on Automatic Control, 19(6), 716–723.

Schwarz, G. (1978). Estimating the Dimension of a Model. The Annals of Statistics, 6(2), 461–464.

Watanabe, S. (2010). Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory. Journal of Machine Learning Research, 11, 3571–3594.

Watanabe, S. (2013). A Widely Applicable Bayesian Information Criterion. Journal of Machine Learning Research, 14, 867–897.

We have moved the linear-SEM data-generation details to Section 4.1 (Equation (23)) and made the validation-NLL definition and selector explicit in Appendix B.4 (Equations (37)–(38)).

Thank you again for your thoughtful questions and constructive suggestions. We hope that our responses have addressed your concerns and clarified the points you raised.

Reviewer_qTZ1OpenReview ↗

Reply to rebuttal

I thank the authors for their time and effort in carefully addressing my questions and concerns. In particular, I appreciate the clarification of the identifiability guarantees, the computational complexity of the proposed approach, and the rationale for not directly applying AIC/BIC to the considered fitting pipeline. I am also glad to see that these clarifications and the additional theoretical conditions have been incorporated into the revised manuscript. Overall, the response has addressed my main concerns, and I therefore increase my score to 3 (Weak Accept).

Reviewer_UjdAOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The authors present a method for identifying the number of latent factors in Causal Representation Learning (CRL) models. In particular, they adapt a previous scoring algorithm, namely the accumulated cutoff discrepancy criterion (ACDC), to factor DAGs. The authors further provide theoretical guarantees for the proposed score, as well as quantitative results on both simulated and real data

Critical strengths

This work fills a relevant gap in the current CRL literature. It provides a principled method model selection, especially in terms of the number of latent factors that should be included in the model. The experimental evaluation is particularly robust, spanning both simulated data, as well as perturb-seq and organoid experiments.

Critical weaknesses

One possible criticality might be information leakage during the simulated analysis. The author state at line 971: "The known graph is used for this purpose only in the simulation study; all factor-count selectors are subsequently applied to the same retained candidate checkpoints". Unfortunately, I was not able to understand from this sentence alone whether information from the known graph could at any point inflate the performance of fDAG-ACDC in the simulated experiments.

Questions for authors

I would first require the author to elaborate on whether any information leakage happened during the simulation analyses (see critical weaknesses).

Furthermore, I would suggest the following additions, which in my opinion would improve the quality of the work:

  1. the authors should elaborate more on runtime and scalability. Reporting computational times would be extremely useful. A complexity analysis, along with a study on scalability (for instance, with respect to the number of latent factors), would also be very informative.

  2. The authors focus on fDAGs, however they state that "Our approach can be extended to models of class (i) by building the local units from the measurement graph" It would be interesting if the authors could actually further elaborate on this topic. While adapting the ACDC method to class (i) of CRL algorithms is clearly outside the scope of this work, the authors could still provide indications on how this could be obtained.

  3. The author should double check the text for typos and other minor issues. For instance, at line 80 the word "odes" appears in place of "does". In the previous paragraph, the structural hamming distance (SHD) is introduced, but no explanation is provide, as well as no proper reference.

2 comments
AuthorsOpenReview ↗

Response to Reviewer UjdA

We sincerely thank you for your time and effort in reviewing our paper.

Regarding information leakage during the simulation analyses:

Ground truth is used in checkpoint selection, not directly in ACDC scoring or selection. In the simulation experiments, we selected the checkpoint with the highest graph-recovery F1 for each candidate factor count to obtain ABCDEFG models with good graph-recovery quality and reduce the influence of optimization and checkpoint selection on the comparison. Our focus is on determining which criterion more accurately selects the factor count from a collection of fitted candidate models, rather than comparing training strategies for the underlying learner.

ACDC scoring and selection do not directly use the true graph. Ground-truth-assisted checkpoint selection was used to control the underlying fitting quality and focus the simulation experiment on the performance of the selection criteria. Thus, the use of the true graph occurs upstream in checkpoint selection, rather than in the ACDC score or factor-count selection rule.

Additional comparison without ground-truth-assisted checkpoint selection. We also evaluated checkpoints selected solely by the lowest validation loss (val_loss). Computational and time constraints limited this comparison to two high-dimensional settings: untargeted hard and untargeted soft interventions, both with d=100d = 100 and true factor count m0=10m_0 = 10. We chose these two settings because their combination of high dimensionality and unknown intervention targets is expected to make model identification more challenging. For each candidate factor count and random seed, all four criteria use the same validation-loss-selected checkpoint. fDAG-ACDC achieves the smallest absolute factor-count selection error among the four criteria in both settings, as shown below.

Factor-count selection using minimum-validation-loss checkpoints.

SettingTrue m0m_0fDAG-ACDCF1SHDNLL
Untargeted hard109141413
Untargeted soft1011131312
MAE across these two settings—1.003.503.502.50

Entries in the first two rows are selected factor counts. F1 and SHD require the true graph to score candidate models and select the factor count; checkpoint selection in this comparison uses only validation loss.

We have clarified checkpoint selection and added the comparison using minimum-validation-loss checkpoints in Appendix B.2 (Table 7).

Q1: The authors should elaborate more on runtime and scalability. Reporting computational times would be extremely useful. A complexity analysis, along with a study on scalability (for instance, with respect to the number of latent factors), would also be very informative.

We have added a runtime and scalability summary to Section 4.1 and detailed training and aggregation timings, training times across factor counts, and complexity analysis to Appendix B.2 (Tables 8–9, Equation (36)).

Q2: The authors focus on fDAGs, however they state that "Our approach can be extended to models of class (i) by building the local units from the measurement graph" It would be interesting if the authors could actually further elaborate on this topic. While adapting the ACDC method to class (i) of CRL algorithms is clearly outside the scope of this work, the authors could still provide indications on how this could be obtained.

We have added a possible extension to class (i) models in the final paragraph of Section 5: a sparse decoder supplies the measurement graph for constructing local comparison units. This remains a preliminary proposal requiring experimental validation.

Q3: The author should double check the text for typos and other minor issues. For instance, at line 80 the word "odes" appears in place of "does". In the previous paragraph, the structural hamming distance (SHD) is introduced, but no explanation is provided, as well as no proper reference.

Thank you for pointing this out. We have checked the manuscript for spelling errors and corrected “odes” to “does” in Section 1 (p. 2, line 80).

Appendix B.4 now explicitly defines SHD, with each insertion, deletion, or reversal counted once (Equations (39)–(40)), the F1/SHD factor-count selectors (Equation (41)), and subset-selection frequencies (Equation (42)).

We have also added factor-alignment and consensus-ranking definitions in Appendix C.1 (Equations (44)–(47)), GSEA quantities in Appendix C.2 (Equations (48)–(53)), expression standardization and UPR scores in Appendix C.3 (Equations (54)–(55)), and cell-composition targets in Appendix C.4 (Equation (57)).

Thank you again for your thoughtful questions and constructive suggestions. We hope that our responses have addressed your concerns and clarified the points you raised.

I would like to thank the authors for their efforts in answering my questions. Yes, my concerns have been adequately addressed.

Reviewer_1zwSOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a new method for choosing the number of latent factors, mm, in CRL for f-DAGs by extending the accumulated cutoff discrepancy criterion (ACDC). The main difference from existing ACDC methods is replacing a manually tuned, problem-specific tolerance threshold with a standard-error-based selector, where the uncertainty is estimated from the variability of the criterion across repeated model fits. The authors prove that their method selects the minimum-sufficient mm (i.e., m0m_0, the smallest mm that minimizes the population criterion) with probability tending to one as the number of refits SS goes to infinity, where each refit produces one criterion curve over the candidate values of mm. They also report simulated and real-data experiments. In simulations, the proposed method selects mm more accurately than validation NLL, which, as the authors argue, tends to select larger mm because additional factors keep improving fit. On real data, it selects fewer factors than validation NLL while predicting external biological summaries comparably. Overall, the paper raises a key question in CRL and proposes a solution to it.

Critical strengths

(1) The paper is well motivated by a key open question in CRL. It offers a method, together with theoretical guarantees that it chooses the most parsimonious mm under the stated assumptions. (2) The paper provides an uncertainty-based criterion for choosing mm, as a replacment for the existing manually calibrated, problem-specific tolerance threshold. The experiments provide context and good support for the applicability of the method.

Critical weaknesses

(1) The ablation (Table 2) shows that, without the uncertainty-based rule, taking the minimum of the proposed criterion gives an MAE of 3.00, comparable to validation NLL (3.17). The greatest improvement therefore seems to come from the "smallest candidate within κS\kappa_S standard errors of the best" rule. The same rule applied to validation NLL is not reported, which would be a key baseline. Moreover, κS\kappa_S, which replaces the tolerance threshold of the existing method, is itself tunable: the paper proposes any κS\kappa_S that grows sufficiently slowly with SS, and log⁡S\sqrt{\log S} as one option. A sensitivity analysis over κS\kappa_S is not reported. (2) Validation NLL selects the largest available mm, i.e., m=15m = 15, in both real-data experiments. It would be informative if the candidate range were extended (e.g., to m=25m = 25), to show whether validation NLL eventually stops improving and how far its selection is from that of the proposed method. In current form, any selector that does not pick the boundary would select fewer factors than validation NLL.

Questions for authors

(1) How sensitive are the selections to the choice of κS\kappa_S (e.g., κ∈{1,1.5,2,3}\kappa \in \{1, 1.5, 2, 3\})? (2) Can the candidate range on the real datasets be extended beyond m=15m = 15, so that the validation NLL comparison is informative. (3) What MAE does the same uncertainty-based selection rule achieve when applied to validation NLL?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1zwS (Q1–Q2)

We sincerely thank you for your time and effort in reviewing our paper.

Q1: How sensitive are the selections to the choice of κS\kappa_S (e.g., κ∈{1,1.5,2,3}\kappa \in \{1, 1.5, 2, 3\})?

The default and multipliers 1.5 and 2 give MAE 0.500. Our prespecified default, κS=log⁡S\kappa_S = \sqrt{\log S}, gives approximately 1.5174 for S=10S = 10 and achieves MAE 0.500, matching the lowest error among the tested specifications. We evaluated fixed multipliers κ∈{1,1.5,2,3}\kappa \in \{1, 1.5, 2, 3\} across all six simulation settings, using the same fitted models and eligible candidate grids as in the paper. The corresponding mean absolute selection errors were 1.833, 0.500, 0.500, and 1.667, respectively.

More extreme multipliers change the selections. These results show that the favorable performance does not depend on a narrowly tuned multiplier: both tested values near the default, 1.5 and 2, retain the same low aggregate error. More extreme choices affect the balance between adequacy and parsimony, as expected. A smaller multiplier tightens the competitive set, whereas a larger multiplier admits more candidates and can favor insufficiently complex models.

Sensitivity to the uncertainty multiplier.

Multiplier specificationMultiplier valueMean absolute selection error (MAE)
Fixed1.00001.833
Fixed1.50000.500
Current default: κS=log⁡S\kappa_S = \sqrt{\log S}1.51740.500
Fixed2.00000.500
Fixed3.00001.667

All results use fDAG-ACDC with the original eligible candidate grids. MAE is averaged over the six simulation settings; S=10S = 10 paired fitting replicates are used in each setting.

We have added the per-setting multiplier sensitivity results to Appendix B.3 (Table 12), with a brief summary in Section 4.1.

Q2: Can the candidate range on the real datasets be extended beyond m=15m = 15, so that the validation NLL comparison is informative.

We evaluated the grid from m=1m = 1 to 2020. Validation NLL selects m=20m = 20 for both datasets, whereas fDAG-ACDC selects m=10m = 10 for Adamson and m=16m = 16 for Amin. We evaluated the expanded candidate grid m=1,…,20m = 1, \ldots, 20, retaining the original fitting and selection protocols.

New candidates shift Amin's reference optimum. For Amin, the selection changes from m=12m = 12 to m=16m = 16 because the additional candidates achieve lower local discrepancy, shifting the empirical reference optimum from m=14m = 14 to m=18m = 18. The contiguous competitive block containing the new optimum begins at m=16m = 16. Previously evaluated candidate scores remain unchanged.

Consistency applies to a fixed candidate set. Our consistency results concern selection within a fixed eligible candidate set: under the stated assumptions, the adaptive rule recovers the smallest population-optimal factor count as the number of replicates increases. Expanding the candidate set can change that target. Thus, the change in Amin's selection does not itself contradict the theoretical guarantee, but it does indicate sensitivity to the candidate range.

Real-data selections on the 1–15 and 1–20 grids.

DatasetCandidate gridValidation NLLfDAG-ACDC
Adamson Perturb-seqm=1,…,15m = 1, \ldots, 151510
Adamson Perturb-seqm=1,…,20m = 1, \ldots, 202010
Amin organoid screenm=1,…,15m = 1, \ldots, 151512
Amin organoid screenm=1,…,20m = 1, \ldots, 202016

We have added the expanded-grid results and their range-sensitivity interpretation to Appendix C.5 (Table 19), with a summary in Section 4.2.

AuthorsOpenReview ↗

Response to Reviewer 1zwS (Q3)

Q3: What MAE does the same uncertainty-based selection rule achieve when applied to validation NLL?

Adaptive NLL achieves MAE 1.667. The adaptive rule reduces validation NLL's MAE from 3.167 to 1.667, whereas fDAG-ACDC achieves 0.500 under the same default specification—a 70% lower MAE than adaptive NLL. Across individual settings, fDAG-ACDC has lower absolute selection error in four settings, equal error in one, and higher error in one. Thus, although uncertainty adaptation improves NLL-based selection, fDAG-ACDC provides more accurate factor-count selection overall in the evaluated settings. This matched comparison supports the value of combining the adaptive rule with our local structural discrepancy criterion.

Matched selection protocol. We applied the same uncertainty-based selection procedure to validation NLL, using the Harrell–Davis central criterion, paired-bootstrap uncertainty, the contiguous selection rule, and the same default κS=log⁡S\kappa_S = \sqrt{\log S}. Both methods were evaluated on the same eligible candidate grids across the six simulation settings.

Simulation factor-count selection: NLL and fDAG-ACDC.

Simulation settingTrue m0m_0Validation NLL: minimumValidation NLL: adaptive rulefDAG-ACDC: adaptive rule
High-dimensional targeted hard10131310
High-dimensional targeted soft10141310
High-dimensional untargeted hard1013129
High-dimensional untargeted soft10121112
Low-dimensional targeted hard5965
Low-dimensional targeted soft5855
MAE—3.1671.6670.500

Both adaptive methods use S=10S = 10 and κS=log⁡10≈1.5174\kappa_S = \sqrt{\log 10} \approx 1.5174. All comparisons use the original eligible candidate grids.

We have added the adaptive-NLL comparison to Appendix B.4 (Table 13) and summarized its main result in Section 4.1.

Thank you again for your thoughtful questions and constructive suggestions. We hope that our responses have addressed your concerns and clarified the points you raised.

I thank the authors for addressing my questions. The responses were accurate and adequate to address my review.

Reviewer_m2tBOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the important challenge of selecting the appropriate number of latent causal components in Causal Representation Learning (CRL), where conventional model-selection criteria may favor overly complex models due to improved global fit rather than the identification of meaningful causal mechanisms. The authors propose fDAG-ACDC, an uncertainty-adaptive extension of the Accumulated Cutoff Discrepancy Criterion (ACDC), specifically investigated for factor-directed acyclic graph (f-DAG) models. The proposed approach evaluates discrepancies between observed and posterior-predictive distributions of individual nodes and their estimated parents across perturbation environments to determine an appropriate number of latent factors. Overall, the paper presents a theoretically supported and empirically evaluated model-selection framework that aims to improve the interpretability and parsimony of causal representations without substantially compromising predictive performance.

Critical strengths
  1. The paper addresses an important and relatively underexplored challenge in Causal Representation Learning (CRL): determining the appropriate number of latent causal components without introducing unnecessary model complexity. The proposed fDAG-ACDC framework extends the existing Accumulated Cutoff Discrepancy Criterion (ACDC) through uncertainty-adaptive model selection and perturbation-specific posterior-predictive assessments. This is a meaningful methodological contribution because conventional likelihood-based criteria may favor overparameterized representations that improve statistical fit without identifying distinct causal mechanisms. The theoretical establishment of model-selection consistency further strengthens the methodological foundation of the proposed approach. This contribution supports acceptance by addressing a fundamental problem in CRL with a principled and theoretically justified solution.

  2. The paper provides encouraging empirical evidence through synthetic experiments and evaluations on real-world Perturb-seq and organoid datasets. The reported mean absolute selection error of 0.5 in simulations, compared with over-selection exceeding three factors on average for the considered baselines, demonstrates the potential effectiveness of the proposed criterion. Furthermore, the selection of more compact representations while maintaining comparable performance in predicting biological summaries suggests that the method can reduce unnecessary latent complexity without substantially sacrificing predictive utility. The enrichment of selected factors for known gene programs provides additional evidence of biological relevance. These results support acceptance by demonstrating the potential practical value of the framework beyond theoretical analysis, particularly for interpretable causal modeling in high-dimensional biological applications.

Critical weaknesses
  1. Although the paper addresses an important model-selection problem in Causal Representation Learning (CRL), the proposed fDAG-ACDC framework is primarily developed and evaluated within the specific setting of factor-directed acyclic graphs (f-DAGs). Consequently, its applicability to broader CRL architectures, nonlinear causal mechanisms, heterogeneous latent structures, and more complex real-world causal systems remains unclear. Furthermore, the theoretical model-selection consistency guarantees depend on specific assumptions whose practical validity and sensitivity require further investigation. The absence of comprehensive robustness analyses under model misspecification, distribution shifts, varying perturbation strengths, and violations of theoretical assumptions potentially limits the generalizability of the findings. This weakness is consequential because the paper's broader methodological contribution depends on demonstrating that the proposed criterion remains reliable beyond favorable modeling conditions.

  2. Although the reported experimental results are promising, the validation appears insufficient to conclusively establish the superiority and causal interpretability of the proposed framework. In particular, comparisons against validation likelihood and oracle graph-recovery baselines may not adequately represent the range of existing model-selection approaches, including information-theoretic, complexity-penalized, and alternative posterior-predictive criteria. Additional ablation studies are needed to isolate the contribution of uncertainty adaptation and assess sensitivity to sample size, latent dimensionality, and perturbation characteristics. Moreover, the biological experiments primarily demonstrate predictive performance and enrichment for known gene programs, which do not necessarily establish that the selected latent factors correspond to genuine causal mechanisms. This limitation is significant because the central claims concern causal representation selection rather than merely obtaining compact and predictive latent representations.

Overall Assessment: These weaknesses raise concerns regarding the generalizability, comparative effectiveness, and causal validity of the proposed framework. More comprehensive baseline comparisons, robustness experiments, and direct evidence supporting causal mechanism recovery would be necessary to substantiate the broader claims and strengthen the case for acceptance.

Questions for authors
  1. Could the authors clarify the specific methodological novelty of fDAG-ACDC compared with the original ACDC criterion and existing model-selection approaches? In particular, what theoretical or empirical advantages arise specifically from the uncertainty-adaptive mechanism rather than the underlying discrepancy-based selection procedure?

  2. The paper establishes model-selection consistency under certain assumptions. Could the authors explain how restrictive these assumptions are in practical CRL applications? What happens to the consistency guarantees when the assumed causal structure is misspecified, perturbations are insufficiently informative, or the posterior-predictive distributions are inaccurately estimated?

  3. Could the authors justify the selection of validation likelihood and oracle graph-recovery as the principal comparison baselines? Have alternative model-selection criteria, such as BIC, AIC, complexity-regularized approaches, or other posterior-predictive discrepancy methods, been evaluated? If not, what evidence supports the claimed advantages over existing model-selection techniques?

  4. Could the authors provide quantitative evidence isolating the contribution of uncertainty adaptation to the overall performance of fDAG-ACDC? Specifically, how does the method perform without uncertainty adaptation, and how sensitive are its results to sample size, perturbation strength, observation noise, latent dimensionality, and model misspecification?

  5. The experimental results demonstrate enrichment of selected latent factors for known gene programs. However, biological enrichment and predictive performance do not necessarily establish causal validity. Could the authors clarify what direct experimental evidence supports the claim that the selected factors correspond to distinct causal mechanisms rather than statistically correlated or predictive latent representations?

  6. The simulations report a mean absolute selection error of 0.5, compared with over-selection exceeding three factors for the baselines. Could the authors provide the variability across independent experimental runs, confidence intervals, and statistical significance of these differences? Additionally, how consistently does fDAG-ACDC select the same number of factors across random initializations and different perturbation environments?

  7. Could the authors clarify the computational complexity and practical scalability of fDAG-ACDC relative to conventional validation-likelihood-based selection? In particular, what additional computational overhead is introduced by posterior-predictive discrepancy estimation across multiple nodes, candidate factor counts, and perturbation environments?

  8. The biological experiments indicate that fDAG-ACDC selects fewer factors while maintaining comparable predictive performance and enrichment for known gene programs. However, these findings do not necessarily establish that the selected factors represent distinct causal mechanisms. Could the authors provide stronger quantitative evidence of causal validity, such as factor-level recovery on synthetic data, stability across independent runs and perturbation environments, or agreement with independently established intervention-response relationships? How do the authors distinguish successful causal mechanism identification from the selection of compact predictive representations?

5 comments
AuthorsOpenReview ↗

Response to Reviewer m2tB (Q1–Q2)

We sincerely thank you for your time and effort in reviewing our paper.

Q1: Could the authors clarify the specific methodological novelty of fDAG-ACDC compared with the original ACDC criterion and existing model-selection approaches? In particular, what theoretical or empirical advantages arise specifically from the uncertainty-adaptive mechanism rather than the underlying discrepancy-based selection procedure?

Two changes: local structural discrepancies and an uncertainty-adaptive cutoff. We retain ACDC’s minimum-sufficient selection principle, but modify how adequacy is assessed and the selection cutoff is calibrated.

Local units replace component-wise attribution. Original ACDC compares each component’s fitted distribution with its attributed data. Such attribution is generally unavailable in f-DAGs, where factors jointly determine observed variables. We use observed child–parent neighborhoods as scoring units, compare observed and posterior-predictive distributions within each perturbation environment, and normalize by neighborhood size. This accommodates interacting causal factors, heterogeneous environments, and candidate-dependent neighborhoods (Section 3.1).

Uncertainty calibration replaces the fixed tolerance. Original ACDC uses a fixed, problem-specific tolerance ρ\rho. Our selector compares each candidate’s criterion gap with its estimated uncertainty:

Δ^S(m)≤κSs^S(m,b^S).\widehat{\Delta}_S(m) \leq \kappa_S \widehat{s}_S(m,\widehat{b}_S).

Conditional consistency for the smallest population-optimal count. Under the stated assumptions, Theorems 1 and 2 establish consistency even when larger candidates share the population minimum. Paired resampling preserves cross-candidate covariance, while the contiguous rule selects the smallest candidate in the competitive block containing the empirical optimum. Numerical closeness is thus assessed relative to fitting variability instead of a fixed tolerance.

Without uncertainty adaptation, MAE rises from 0.50 to 3.00. Across six simulation settings, the no-uncertainty variant achieves MAE 3.00 versus 0.50 for fDAG-ACDC (Table 2). Holding fitted models, validation observations, and discrepancy curves fixed, the “No paired uncertainty” variant selects the empirical minimum of the same central criterion curve (Appendix B.3, Table 11). Since discrepancy calculation is unchanged, this demonstrates the contribution of uncertainty-adaptive selection.

Q2: The paper establishes model-selection consistency under certain assumptions. Could the authors explain how restrictive these assumptions are in practical CRL applications? What happens to the consistency guarantees when the assumed causal structure is misspecified, perturbations are insufficiently informative, or the posterior-predictive distributions are inaccurately estimated?

The guarantee requires stable scores and valid paired uncertainty. Assumptions 1–2 require convergence of fitted scores and valid paired uncertainty estimates. Assumption 3 requires a positive gap below the smallest population minimizer and a shrinking cutoff; κS=log⁡S\kappa_S = \sqrt{\log S} meets its rate conditions under root-SS estimation. Assumption 4, only for the contiguous selector, requires population-optimal candidates to form an interval, without requiring overall monotonicity.

Consistency can survive misspecification if the assumptions hold. A misspecified family can yield a stable population criterion C(m)C(m), even with a positive minimum. Under the assumptions, the selector recovers m0=min⁡arg⁡min⁡mC(m)m_0 = \min \arg\min_m C(m), the smallest count achieving optimal local adequacy. Interpreting m0m_0 as the generating factor count additionally requires identifiability. If misspecification prevents convergence or valid uncertainty estimation, the guarantee no longer applies.

Uninformative perturbations can prevent identification of the generating count. Weak perturbations may produce small positive criterion gaps. Asymptotic consistency holds under the assumptions, although resolving these gaps requires more precise estimation. If environments cannot distinguish smaller representations from the generating model, the criterion-defined m0m_0 may be smaller than the generating count.

Persistent predictive bias can change the selection target. Finite-sample errors are allowed when convergence and uncertainty conditions hold. Persistent approximation bias may shift the limiting criterion and its minimizer. If the resulting limit satisfies the assumptions, consistency applies to its minimum-sufficient factor count. Errors preventing a stable limit or valid uncertainty estimation fall outside the guarantee.

We have added these practical implications to Appendix D.10.

AuthorsOpenReview ↗

Response to Reviewer m2tB (Q3–Q4)

Q3: Could the authors justify the selection of validation likelihood and oracle graph-recovery as the principal comparison baselines? Have alternative model-selection criteria, such as BIC, AIC, complexity-regularized approaches, or other posterior-predictive discrepancy methods, been evaluated? If not, what evidence supports the claimed advantages over existing model-selection techniques?

NLL measures predictive fit; F1 and SHD measure graph recovery. The principal baselines were chosen to address two relevant but different selection objectives. Validation NLL is a deployable predictive-fit criterion that does not require the true graph. Graph F1 and SHD are ground-truth-assisted simulation diagnostics: they test whether optimizing observed-graph recovery also selects an appropriate factor count. They are called “oracles” because they use unavailable-in-practice graph information, not because they guarantee optimal factor-count selection. Together, these comparisons test whether the proposed adequacy criterion adds value beyond predictive fit and graph accuracy.

The simulation comparison supports lower factor-count selection error. On the original eligible simulation grids, Table 1 reports MAE 0.50 for fDAG-ACDC versus 3.17 for validation NLL, 3.50 for graph F1, and 3.33 for SHD, supporting more accurate factor-count selection in the evaluated settings. As discussed in the Introduction, selecting the minimum-sufficient number of factors in interventional causal representations remains underexplored. Many existing criteria were developed for predictive model selection or conventional factor models, so obtaining a directly matched baseline for fitted f-DAGs requires additional choices about the likelihood, model complexity, or discrepancy construction. Our baseline selection therefore reflects this methodological gap: validation NLL and graph-recovery scores provide complementary comparisons against predictive fit and structural recovery.

AIC/BIC were not evaluated as formal baselines. For brevity, we refer to our response to Reviewer qTZ1, Question 4 for the detailed rationale and references.

Alternative posterior-predictive discrepancies were not evaluated. This is a limitation and a direction for future work.

Section 5 now explicitly identifies this missing comparison as a limitation and proposes comparing alternative discrepancies on the same fitted models as future work.

Q4: Could the authors provide quantitative evidence isolating the contribution of uncertainty adaptation to the overall performance of fDAG-ACDC? Specifically, how does the method perform without uncertainty adaptation, and how sensitive are its results to sample size, perturbation strength, observation noise, latent dimensionality, and model misspecification?

Removing uncertainty adaptation increases MAE from 0.50 to 3.00. Table 2 in Section 4.1 directly isolates the contribution of uncertainty-adaptive selection: removing this component increases the mean absolute factor-count selection error (MAE) from 0.50 to 3.00 across the six simulation settings. Specifically, the “No paired uncertainty” variant selects the empirical minimum of the same central discrepancy curve instead of using the uncertainty-adaptive selection rule. The fitted models, validation observations, discrepancy criterion, and eligible candidate grids are held fixed. This provides direct quantitative evidence for the contribution of uncertainty-adaptive selection in the evaluated settings.

Systematic sensitivity analyses remain future work. A systematic assessment of sensitivity to sample size, perturbation strength, observation noise, latent dimensionality, and model misspecification would require additional controlled simulation studies, with repeated model fitting across candidate factor counts and random seeds. This entails substantial training time and computational resources and is an important direction for future work.

We have specified these untested sensitivity dimensions and controlled simulations with repeated fitting as future work in Section 5.

AuthorsOpenReview ↗

Response to Reviewer m2tB (Q5–Q6)

Q5: The experimental results demonstrate enrichment of selected latent factors for known gene programs. However, biological enrichment and predictive performance do not necessarily establish causal validity. Could the authors clarify what direct experimental evidence supports the claim that the selected factors correspond to distinct causal mechanisms rather than statistically correlated or predictive latent representations?

The real-data evidence is indirect. Our experiments support biological relevance and predictive utility, but do not directly establish that each factor represents a distinct causal mechanism. fDAG-ACDC selects a parsimonious factor count by comparing observed and posterior-predictive local distributions across perturbation environments. Its causal interpretation remains conditional on the underlying modeling and identification assumptions.

This scope is stated in the manuscript. Section 4.2 (pp. 9–10) evaluates independently defined biological summaries. Appendix C (p. 24) describes these analyses as post-selection diagnostics. Section 5 (p. 10) explicitly acknowledges that “the real-data evidence is indirect.”

Direct mechanism validation requires additional identifying evidence and remains an important direction for future research.

We have expanded Section 5 to state the lack of direct factor-level causal validation and outline future validation using synthetic truth and independent intervention evidence.

Q6: The simulations report a mean absolute selection error of 0.5, compared with over-selection exceeding three factors for the baselines. Could the authors provide the variability across independent experimental runs, confidence intervals, and statistical significance of these differences? Additionally, how consistently does fDAG-ACDC select the same number of factors across random initializations and different perturbation environments?

Additional independent pipeline repetitions remain future work. Each setting uses ten paired data-and-fitting runs. Repeating the full selection pipeline on additional independent batches would require refitting every candidate model, with substantial computational and time costs. We therefore leave this assessment to future work.

Existing fits provide conditional uncertainty estimates. The first table reports resampling variability and 95% intervals. All three baseline-minus-ACDC MAE intervals are positive. Exact two-sided paired sign-flip tests across the six settings give p=0.03125p = 0.03125 against F1 and SHD and p=0.06250p = 0.06250 against NLL.

Selection stability depends on the setting. Deleting one fitting run preserves selection in 2/10–9/10 subsets, including 9/10 in both targeted-hard settings. Deleting one environment from scoring preserves selection in 54.5%–91.7% of deletions across simulation settings and 100% for both real datasets. These diagnostics assess stability conditional on existing fits.

Selection-error variability and baseline comparisons.

MethodOriginal MAEResampling SD of MAEConditional 95% MAE intervalConditional 95% MAE difference interval (baseline – ACDC)p-value
fDAG-ACDC0.5000.525[0.500, 2.421]——
Validation NLL3.1670.348[2.246, 3.500][0.333, 2.667]0.06250
F13.5000.548[1.833, 3.833][0.079, 3.000]0.03125
SHD3.3330.531[1.833, 3.833][0.167, 3.000]0.03125

MAE averages six setting-level selections. Percentile intervals use 500 paired resamples of the ten existing seed-index blocks, with 3,000 inner ACDC bootstrap draws and the original eligible grids. Pairing is preserved across candidates, methods, and settings; tests use six setting-level absolute-error differences.

Selection stability under leave-one-environment scoring.

Dataset / settingOriginal selectionEnvironment deletionsSelection range after deletionAgreement with original selection
High-dimensional targeted hard1011[9,10]81.8%
High-dimensional targeted soft1011[10,14]54.5%
High-dimensional untargeted hard912[9,10]91.7%
High-dimensional untargeted soft1212[11,12]91.7%
Low-dimensional targeted hard511[4,5]90.9%
Low-dimensional targeted soft511[5,8]81.8%
Adamson1086[10,10]100%
Amin1246[12,12]100%

Each environment is removed from scoring with fitted models fixed; remaining weights are renormalized. Selection uses 3,000 bootstrap draws and the original eligible grids. Agreement is relative to the full-environment selection under the same settings; brackets show the selection range.

We have added stability summaries to Sections 4.1–4.2, conditional error intervals and simulation diagnostics to Appendix B.4 (Tables 14–15), and real-data environment diagnostics to Appendix C.6 (Table 20).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.