- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies intervention prediction from images with known, numerically supervised causal variables. It freezes a concept-supervised visual model, learns an invertible per-dimension calibration from verified intervention values, and uses calibrated readouts for observational structure estimation and intervention-based orientation. Linear local mechanisms support population interventional means and factual-conditioned counterfactuals, with inverse calibration before decoding. Evaluation spans Domino, PnP, Balance, Gears, CAUSAL3D, and CausalWorld; supplementary controls separate calibration, acquisition, and decoder effects. The central question is whether aligning visual readouts with intervention units improves downstream causal queries.
Critical strengths
- The paper addresses a practical problem in visual causal modeling. Even when a latent dimension corresponds to the right variable, its numerical value may not match the setting used to intervene on the environment. This mismatch is worth addressing when using visual representations to predict intervention outcomes.
- The paper includes several informative controls. Comparing against affine calibration and changing only the decoder help show where improvements come from. The negative results on Gears are also useful: they show that more accurate state readouts do not necessarily produce better intervention predictions.
Critical weaknesses
-
The added value of intervention-based calibration is not convincingly established. The observational training images already have numerical state labels, but the paper does not evaluate the same calibration family fitted only on those labels. The need for additional calibration interventions therefore remains unresolved. Downstream gains are also mixed: nonlinear calibration lowers mean effect error relative to no calibration on Domino but does not consistently outperform affine calibration (Table A13), while calibration increases mean prediction errors in the two-seed Gears control (Table A14). The demonstrated gains are too limited to support calibration as the paper's main contribution.
-
The large gains over CausalVAE and ILCM do not establish the contribution of calibration or active target selection. Active targeting shows no advantage over round-robin in the reported budget controls (Table A19). In a separate control using the same encodings, GIES has similar factor-level counterfactual error point estimates on three of four tasks (Table A8). Corresponding component ablations under the configurations of Tables 3–4 are missing. The main comparisons also score only executed queries without establishing common retained-query sets; prediction seed and query counts are unspecified for most main results (Table A2). These omissions leave both the comparability of the headline results and the contribution of the proposed stages unresolved.
Questions for authors
- Could the authors report prediction seed counts, executed/planned query counts, and errors on common query sets for the main comparisons in Tables 3–4? Please specify how results were aggregated across seeds and report uncertainty, keeping references with different information access separate.
- How do the Balance and Gears SHD values in Table 1 (6 and 7) relate to Appendix C.1's reports of 7/8 correct Balance recoveries and no wrong arrows in eight Gears runs? Please identify the corresponding ground-truth graphs, seed sets, and orientation backends, and provide per-seed SHD and its aggregation for Table 1.
- Under the fixed-graph protocol of Table 2, how does calibration using verified intervention values compare with the same invertible mapping family fitted only on existing observational state labels? Please keep the encoder, graph, mechanism-fitting records, decoder, and test queries shared, refit the mechanisms in each coordinate system, and report paired per-seed downstream results with uncertainty for the mean differences.
- Using the configurations of Tables 3–4 on a representative subset, such as Domino SCM and one other MEC task, how do errors and execution coverage change when calibration is removed, and separately when active targeting is replaced by round-robin? Please compare on common queries, matching fitting records for the calibration ablation and total intervention budgets, including calibration calls, for the acquisition comparison.