acceptodds
Withdrawn from ICLR 2027

Intervene, Calibrate, Discover: Learning Causal Structure from Pixels

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learning high-level causal variables from low-level observations is a central problem for artificial intelligence (Schölkopf et al., 2021). With known node semantics but an unknown graph, readout errors can misalign model settings with environment intervention values. With encoder and decoder frozen, verified values from fixed round-robin interventions train dimension-wise invertible calibration; after freezing it, active experiments orient edges, local mechanisms predict consequences, and inverse calibration supports decoding. Across Domino, Pick-and-Place, Balance, Gears, CAUSAL3D, and CausalWorld, we test graph discovery, intervention consequences, and counterfactual images. Controlled domino experiments show that interventions resolve orientation ambiguity left by equal additional observations. In a matched-data domino control, calibration reduces the mean error in predicting how changing an intervention alters other variables within the same factual context, relative to an uncalibrated model.

Primary area:
causal reasoning
Keywords:
causal discovery, causal representation, intervention

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_7ibeOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies intervention prediction from images with known, numerically supervised causal variables. It freezes a concept-supervised visual model, learns an invertible per-dimension calibration from verified intervention values, and uses calibrated readouts for observational structure estimation and intervention-based orientation. Linear local mechanisms support population interventional means and factual-conditioned counterfactuals, with inverse calibration before decoding. Evaluation spans Domino, PnP, Balance, Gears, CAUSAL3D, and CausalWorld; supplementary controls separate calibration, acquisition, and decoder effects. The central question is whether aligning visual readouts with intervention units improves downstream causal queries.

Critical strengths
  1. The paper addresses a practical problem in visual causal modeling. Even when a latent dimension corresponds to the right variable, its numerical value may not match the setting used to intervene on the environment. This mismatch is worth addressing when using visual representations to predict intervention outcomes.
  2. The paper includes several informative controls. Comparing against affine calibration and changing only the decoder help show where improvements come from. The negative results on Gears are also useful: they show that more accurate state readouts do not necessarily produce better intervention predictions.
Critical weaknesses
  1. The added value of intervention-based calibration is not convincingly established. The observational training images already have numerical state labels, but the paper does not evaluate the same calibration family fitted only on those labels. The need for additional calibration interventions therefore remains unresolved. Downstream gains are also mixed: nonlinear calibration lowers mean effect error relative to no calibration on Domino but does not consistently outperform affine calibration (Table A13), while calibration increases mean prediction errors in the two-seed Gears control (Table A14). The demonstrated gains are too limited to support calibration as the paper's main contribution.

  2. The large gains over CausalVAE and ILCM do not establish the contribution of calibration or active target selection. Active targeting shows no advantage over round-robin in the reported budget controls (Table A19). In a separate control using the same encodings, GIES has similar factor-level counterfactual error point estimates on three of four tasks (Table A8). Corresponding component ablations under the configurations of Tables 3–4 are missing. The main comparisons also score only executed queries without establishing common retained-query sets; prediction seed and query counts are unspecified for most main results (Table A2). These omissions leave both the comparability of the headline results and the contribution of the proposed stages unresolved.

Questions for authors
  1. Could the authors report prediction seed counts, executed/planned query counts, and errors on common query sets for the main comparisons in Tables 3–4? Please specify how results were aggregated across seeds and report uncertainty, keeping references with different information access separate.
  2. How do the Balance and Gears SHD values in Table 1 (6 and 7) relate to Appendix C.1's reports of 7/8 correct Balance recoveries and no wrong arrows in eight Gears runs? Please identify the corresponding ground-truth graphs, seed sets, and orientation backends, and provide per-seed SHD and its aggregation for Table 1.
  3. Under the fixed-graph protocol of Table 2, how does calibration using verified intervention values compare with the same invertible mapping family fitted only on existing observational state labels? Please keep the encoder, graph, mechanism-fitting records, decoder, and test queries shared, refit the mechanisms in each coordinate system, and report paired per-seed downstream results with uncertainty for the mean differences.
  4. Using the configurations of Tables 3–4 on a representative subset, such as Domino SCM and one other MEC task, how do errors and execution coverage change when calibration is removed, and separately when active targeting is replaced by round-robin? Please compare on common queries, matching fitting records for the calibration ablation and total intervention budgets, including calibration calls, for the acquisition comparison.
Reviewer_p9LkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies causal predictions from images. It builds a system that reads in an image via frozen encoder, and proposes a way to calibrate the encoder's numerical estimates so that the models' estimates agree with interventions. This system is then evaluated on intervention-based graph discovery, outcome prediction, and generated images in simulated environments. Evaluations span several physical settings: Domino SCM, Physical Domino, Gears, CAUSAL3D Spring and Seesaw, CausalWorld, etc. The system achieves lowest SHD in 6/8 settings and highest AUPRC in 7/8 settings against baselines. Furthermore, nonlinear calibration reduces mean counterfactual effect error from affine and uncalibrated baselines across certain metrics. Finally, their method achieves the lowest pixel MSE on 7/8 tasks compared to baselines.

Critical strengths
  • The system shows promising graph recovery results across sufficiently many environments. It achieves the lowest SHD distance in 6/8 settings, with 0 error on Physical Domino and PnP. The system can recover causal structure across several varied tasks.
Critical weaknesses
  • The results for the proposed nonlinear calibration in Table 2 are weak. Simple affine calibration performs better on 3/4 downstream metrics, and nonlinear calibration only improves shifted amplitude effect error slightly. These provide limited evidence that the proposed method has a meaningful advantage over simpler baselines.
Questions for authors
  • Can you display some benefit of nonlinear calibration over affine calibration? Are these results consistent across seeds, or is there evidence of a meaningful advantage in other environments?
  • How much of the improvement in Tables 1, 3, and 4 comes from calibration? Does the same pipeline without calibration and with affine calibration achieve similar results across these environments?
Reviewer_bnsTOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors tackle the problem of predicting what is the effect of a high level intervention (in some latent conceptual space; e.g. a robotic arm action) on an image, via causal representation learning methods.

Critical strengths

The problem is potentially relevant in robotics applications.

Critical weaknesses

The paper was un-readable. I mention concrete examples in the questions for authors. But it was very hard to understand what is the exact mathematical problem or real world problem the authors are trying to solve, what various symbols mean and why the problem is important or sufficiently motivated. The authors need to rewrite the paper to make the contribution understandable.

Questions for authors
  1. What do you mean by "require a target" in your opening paragraph. What is a target here? Why don't you explain it in more understandable terms?
  2. What does the jargon sentence: "query pixel-derived variables in environment intervention units." mean? Isn't there an easier way to define what you are trying to do?
  3. What is a "labeled-aligned readout"?
  4. What is T in the introduction. Don't you think it is important for readability of the paper to define that symbol before using it in your contributions. How would one understand what your contribution is otherwise?
  5. Why do you want to achieve the task you are proposing? There is not clear motivation of why this task is useful in your introduction.
  6. In problem formulation, you start in your opening: "Node semantics are known; the direct graph and structural equations are unknown"; which nodes? of the latent causal graph? Why is it reasonable that you know the nodes and their semantics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.