- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether residual independence, the signature of a causal mechanism under an additive noise model, can be used to select a better symbolic equation among candidates that predict roughly equally well. The method, DiSCO-SR, fits a finite library of symbolic candidates and restricts attention to candidates whose selection-set risk lies within a small band of the minimum-risk candidate, and among those picks the one whose residuals are least dependent on the input, as measured by HSIC. They provide conditional guarantees on population risk, and evaluate across three pre-registered synthetic campaigns, the Tübingen pairs and a physical testbed (Causal Chamber).
Critical strengths
The experiments are quite thorough, and the authors matched-gate regression baselines separate the effect of equation selection from that of the direction rule. Proposition 2 (the wrong-direction intervention floor) and the direction/equation attribution identity are intuitive and nicely presented. The authors are also commendably honest that their method does not outperform the minimum-risk reference.
Critical weaknesses
I’m struggling with what the central claim of the paper is. The motivation is explicitly causal: equations that predict equally well can imply different intervention responses, so finding the equations with some sense of causal semantics will greatly aid generalisation. Yet the paper repeatedly states that its procedure provides no causal certificate and no evidence of a correct mechanism. If the method’s value is not causal, it’s hard to see why one would care about directionality at all, since predictively Y|X and X|Y are equally valid models of the joint. Can you help me understand who the "auditable interface" is for, and what decision it supports, if not causal mechanism recovery? My suggestion would be to state the causal claim once, conditionally on an identifiable two-node ANM without confounding, and evaluate against that.
My reading is that the null result is to be expected, and I would like to understand whether the authors agree. Under an ANM, a candidate’s residual is N + (f* − f)(X). Both its excess risk and its residual dependence on X are driven by the same term (f* − f)(X). Inside a tight risk band this term is constrained to be small in L^2, leaving HSIC little room to discriminate between candidates, particularly with only 50–125 selection rows. If this reading is correct, a discussion of it in the paper would help provide a clear explanation of the finding.
Relatedly, the finding I find most interesting seems to me to be one the authors underplay. Joint+gate, which applies residual dependence without the risk band, significantly improves OOD error in both V4 and V5. This suggests that residual guidance may help precisely where the risk constraint is removed, which is where extrapolation behavior differs. Could you comment on this, and on whether a design centered on OOD performance would be the more natural test of the hypothesis?
In the expriments, the gate never abstained on 160 nonidentifiable or 180 assumption-violating cases. Given that the causal interpretation rests on these assumptions, this deserves more than a single sentence.
If these points could be addressed adequately I would raise my score.
Questions for authors
The paper is quite hard to read. Almost every paragraph ends with a qualifier ("this is not a causal certificate"), and the density of these makes it difficult to find what is actually being claimed. The abstract is largely a list of numbers. The interleaving of three versioned campaigns (V4, V5, V6) with different selectors and locked parameters makes the paper tough to read at points, and I found myself constantly referring back to Table 4. A single clearly stated hypothesis, one primary campaign in the main text, and the remainder in the appendix would, in my view, improve readability considerably.
Notation is also overloaded in places: b denotes both the risk-reference index and the number of blocks, B both the loss bound and the block count, and L the log term, the loss and the number of bootstrap draws.