- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies causal inference when treatment is global and continuous. In this setting, all units are exposed to a common continuous treatment path, with treatment variation occurring over time rather than across units. The authors propose structural models for outcomes and macroeconomic dynamics and develop partial identification strategies to construct intervals for their causal estimands. Financial stress testing provides the motivating application.
Critical strengths
-
The paper considers a challenging and important setting. The authors highlight the distinction between prediction and causality, emphasizing that good historical predictive performance does not necessarily imply reliable causal conclusions.
-
The paper separates historical prediction, future prediction, and intervention outcomes, which is conceptually meaningful. This distinction helps clarify the different quantities of interest and the assumptions needed to connect them.
-
Applying partial identification and sensitivity analysis to this setting is a useful contribution. Rather than treating predictive forecasts as causal estimates, the proposed approach explicitly acknowledges uncertainty arising from unobserved confounding.
Critical weaknesses
- A major conceptual weakness is that uncertainty about confounding strength is not uncertainty about the confounding model. The paper still relies on strong assumptions about how unobserved confounders affect treatment and outcomes, while its sensitivity analysis does not examine violations of those assumptions. In particular, the latent confounder is assumed to enter the outcome and macro equations additively, with constant coefficients.
A concrete example of potential misspecification is that credit tightening may be more damaging during a downturn than under normal economic conditions. An illustrative alternative is Y=m(I,A)+(a+bA)U+xi, where A represents unemployment, U represents tighter credit conditions, and b>0. In this model, the effect of the latent financial factor becomes stronger as unemployment rises. Varying the magnitude of a constant confounding coefficient does not address this departure from the assumed model. The authors should clarify the scope of the proposed robustness analysis and explain why the maintained restrictions on the confounding mechanism are credible in the motivating application.
-
The paper’s calibration does not sufficiently establish plausible bounds on unobserved confounding. Assumption 3 allows the analyst to choose or calibrate upper bounds on \rho_A and \rho_Y. The substantive usefulness of the resulting interval depends on whether these bounds plausibly contain the relevant confounding strengths. More justification is needed for how the bounds are selected and why they should be regarded as credible limits on unobserved confounding, rather than illustrative choices. Without such justification, it is difficult to assess how informative the sensitivity analysis is in practice or which conclusions should be considered robust.
-
The causal estimand is not sufficiently justified. The paper usefully distinguishes prediction from causality, but two different questions are relevant to the motivating application: What losses should we expect in an adverse economic episode in which unemployment follows the specified path? What losses would occur if unemployment were externally set to this path while the latent financial-condition process retained its specified intervention law? The paper targets the second question. I personally find the first more closely aligned with the idea of stress testing, where deteriorating financial conditions may be part of the adverse scenario of interest rather than a source of confounding to remove. The authors should explain more clearly why isolating the effect of unemployment while maintaining the specified law of the latent financial process is the appropriate objective for their application.
-
There are substantive proof mistakes that need correction, particularly in Proposition 2 and Theorem 3.
- Proposition 2 changes the inferential target between calibration and testing. The calibration scores use errors against realized cross-sectional average outcomes, whereas Appendix D.1 defines the test score using error against the population predictive mean. The proof then assumes exchangeability of these differently constructed scores without establishing it.
- Theorem 3 incorrectly states that the relevant bias vanishes for affine outcome models. The proof compares a deterministic rollout of the structural function m with the predictive mean and treats their difference as a nonlinear mean-state bias. Under confounding, that difference need not vanish even when m is affine.
Questions for authors
-
The paper describes its framework as “model-agnostic,” but it imposes structural assumptions on the outcome, macroeconomic, and latent confounding processes. How should “model-agnostic” be interpreted here? Does it refer only to flexibility in the choice of predictive learner, rather than flexibility in the confounding model or causal assumptions?
-
Why is it appropriate to assume that the specified treatment path can be imposed while the latent financial-condition process retains its maintained law? In the motivating stress-testing application, why should financial conditions not deteriorate jointly with unemployment? What economic intervention or decision makes the proposed causal estimand the relevant target?