- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes methods for improving partial identification intervals using known symmetries of causal response functions. It analyzes both the direct addition of symmetry constraints and the use of data augmentation to change the observational distribution, and provides conditions governing the causal interpretation of augmentation, validity, interval size, and changes in error. For approximate symmetry, it uses interval padding and intersection with the original interval, and also discusses an extension that uses the transformation itself as an instrumental variable. Experiments cover synthetic data, an optical apparatus, cigarette demand, and do-MNIST.
Critical strengths
- The paper systematizes additional domain knowledge for partial identification. Symmetry is used as information that restricts the set of compatible functions, rather than only as a regularizer for point estimation. Distinguishing and comparing direct constraints with data augmentation is also useful (Sections 3–4).
- The paper decomposes interval narrowing into several factors. It distinguishes confounding dilution, residual risk, mean shifts, and geometric changes induced by augmentation, and gives conditions for improvements in interval size and error (Propositions 1–2, Theorem 2; pp. 7–8). This goes beyond the observation that adding constraints makes a set smaller.
- Approximate symmetry and potential failures are considered. The paper does not assume only exact symmetry; it addresses tolerance-dependent padding and budget recalibration. The experiments also examine symmetry misspecification and changes in budgets and sample sizes (Theorems 3.A–B, Appendix F.2).
Critical weaknesses
-
There is a condition-dependent difference between the general IV guarantee for approximate symmetry and the implementation budget. The general recalibration in Appendix C is whereas the implementation in Appendix E uses , without the cross term (pp. 18 and 20). The smaller budget corresponds to an additional condition that the relevant residual components of symmetry violation and confounding are orthogonal; the paper separately explains this condition (Appendix G.3, Remark 2; p. 29). The two expressions therefore cannot be interchanged unconditionally. To assess the robustness claim, the paper needs to show why orthogonality holds or is reasonable in the experiments, and whether interval improvements persist under the conservative general budget.
-
The intersection of valid population sets must be distinguished from the combination of finite-sample confidence intervals. The dominance statement in Remark 1 concerns valid population intervals, whereas the implementation is described as using bootstrapped 95% Imbens–Manski confidence intervals (pp. 8 and 20). Simply intersecting two intervals that each have 95% coverage does not guarantee 95% coverage for their intersection. This does not contradict the population theorem; it identifies the need for an additional adjustment when applying the result to the intervals actually reported. The explanation of the bootstrap resampling unit, which must account for dependence among augmented rows, and the handling of joint coverage for the intersection is insufficient to assess the relationship between width reduction and statistical coverage guarantees.
-
Interval validity in the experiments depends substantially on access to ground truth and the choice of assumptions. Synthetic symmetries are constructed from the null space of the true coefficients, and the optical experiment approximates ground truth through regression using recorded confounders (p. 21). For do-MNIST, the confounding budget is selected as the minimum value that covers the true function on 99.5% of a separate split (p. 22). These settings are useful as controlled demonstrations of the principle, and I am not claiming that they constitute data leakage. However, they do not establish reliability when symmetry error and budgets must be chosen without access to causal ground truth. The cigarette-demand example is also illustrative rather than a validation of validity, as the authors acknowledge. In particular, the infeasibility of direct symmetry constraints on do-MNIST is intertwined with the function class that adjusts only the final layer, making it difficult to interpret this result as evidence of the general superiority of augmentation.
Questions for authors
- What justifies omitting the cross term in each experiment? If the main existing experiments are recomputed using the conservative IV budget from Appendix C, how much of the validity and width improvement is retained?
- Does the bootstrap for the reported confidence intervals resample original observations or augmented rows? How is joint coverage adjusted when intersecting the baseline and augmented intervals?
- For do-MNIST, could you report both width and coverage across a range of budgets rather than only at a single budget chosen using ground truth? This would help assess how much the improvement depends on that particular calibration using ground truth.
We thank the reviewer for their feedback, especially regarding the experimental corrections. We have provided updated results in the manuscripts and described the changes below. We would be grateful if the reviewer would consider whether these address their concerns and the overall rating. For reference; red highlights mean new additions to the manuscript, blue highlights mean re-written/edited content.
W1 + Q1. IV budget recalibration
The reviewer is correct. The submission introduced the no-cross-term budget in App. C (lines 938–940), calling orthogonality between symmetry violation and confounding "plausible" (Rem. 2, lines 1557–1565), and used it in the implementation (App. E, line 1078). This is an additional assumption that need not hold in general, however plausible, and we cannot justify it experiment by experiment. We have now removed it entirely in the revision. Rem. 2 is deleted, and every experiment is recomputed with the conservative budget γz(ε)=(ε/σ+γz/ρ)2 of Thm. 3.B (App. B). All results and trends are retained.
W2 + Q2. Bootstrap and intersection coverage
Intersection coverage. We thank the reviewer for pointing this out; the reported 95% indeed did not apply to the intersected bounds. In the revision every individual set is built at the 97.5% level, so that the intersection of two sets, as in Rem. 1, retains ≥95% asymptotic coverage by a union bound (App. D of the revision).
Bootstrap resampling. The revision no longer bootstraps intervals. We replaced the Imbens–Manski CIs with the finite-sample budget adjustment of Prop. 4 (App. D of the revision) to solve certain infeasibility artefacts at small n in the previous version (Fig. 14, and Fig. 6 pi+iv and pi+inv).
W3 + Q3. do-MNIST budget sweep
We have provided a gamma sweep and aggregated results for do-MNIST in Table 4 of Ablations, with accompanying analysis.
The reviewer is right that validity in our experiments is assessed with access to ground truth. This is a limitation of methods research in observational causal inference generally: validity can only be demonstrated where the truth is known or constructed, hence the standard practice of semi-synthetic experiments, which ours follow. Budget calibration without ground truth is likewise an open problem for sensitivity analysis as a whole. Cigarette demand shows the workflow we have in mind (γ benchmarked against an observed covariate, γz swept up to γ; App. E.1 of the revision), and our response to Reviewer hAEC (W3) covers ε.
Lastly, the last-layer infeasibility in do-MNIST was not meant to claim any general superiority of DA. But rather the implementation/operational simplicity of DA. Of course this comes at the cost of DA being least sharp of the symmetry-informed PI methods (Prop. 3). We have re-written the do-MNIST passage in Experiments to avoid such a confusion, and also gave more commentary on this in Ejkh W4 .