- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper is about auditing methods for figuring out when a model is using shortcuts or spurious correlations to learn relationships between variables; some methods may predict that a model barely uses shortcuts, but if there is not much useful data available, then the method's prediction can be meaningless. Hence it's useful to audit these methods to detect shortcuts, and this paper provides a framework for it. Specifically, they formalize when the amount of available data is sufficient to detect that a model is relying on shortcuts. They propose an audit procedure and then conduct some experiments to illustrate their method.
Critical strengths
The problem motivation is an interesting one, and being able to detect when a shortcut auditing method is reliable or not is an important problem.
Critical weaknesses
The text is extremely difficult to follow. For example, starting the abstract with "A shortcut can be real yet observationally undiagnosable" is unclear, and one could instead say something like "Shortcuts learned by machine learning models to make predictions can be difficult to diagnose". The next phrase, "a fixed predictor may rely on identity-associated information even when the data lack the same-state cross-identity comparisons needed to reveal that reliance," reads as a really roundabout way of saying "models might rely on spurious features for making predictions, but sometimes there might not be enough data available for a diagnostic method to detect that those features are indeed spurious." Here, "identity-associated information" and "same-state cross-identity" makes no sense without context. The same applies for the rest of the abstract--there was no sentence in it that was easily comprehensible.
In the introduction, when the problem is set up, "full-target predictor-specific reliance" reads similarly: "target" has not been defined, nor has "full-target," it's not clear what the "predictor" is, etc. In Figure 1, the variables are not defined (until later) and the captions are inadequate for comprehension. After staring at it for a while, my understanding is that panel (a) means that you take some physical state s, measure it using two different methods m and m', and then pass them each through the model f, and then figure out how different f's predictions where on the two different measurements. This could be way easier to understand if one included text in the figure instead of variables that haven't been defined yet!
In Section 3, a lot of the notation could be made a lot more clear if one had a running example (e.g., "Let denote the task-relevant physical state (for example, ___)").
Generally, the above comments apply for the later sections (evaluation design, results, discussion, conclusion) as well. They are very difficult to parse--the results might be interesting, but it would be much easier to tell if the paper used a more clear writing style. Hence, it is difficult to make an informed conclusion about the soundness or novelty of the methods in the paper.
Questions for authors
- Can you define what counts as a shortcut? I think the big idea in the paper is to measure how model predictions differ across different kinds of measurements or data sources, but that doesn't establish the presence of a shortcut. I think you also acknowledge this when you say that your method doesn't establish causal effects, but it would helpful to clarify what exactly you mean then.
- How do you select ? I think this is equivalent to asking how you select the possible relevant comparisons.
- Section 4 starts with "We evaluate four links in the identification chain from Section 3," what are all the links and what is the identification chain? Additionally, Stanford EIS and CWRU are not introduced with any context. I googled "Stanford EIS" and it took me some searching before I could figure out that that refers to a dataset. The rest of the terms in "Stanford EIS examines deployment degradation when attribution remains unidentified" are also not easy to understand clearly.
- In Section 3, is undefined.
- Section 3.2 starts with the assumption that , and is only briefly defined in 3.1 but it's not super clear how they differ
- In Figure 3, the gray dots are labeled "Full target state space (Q)" and the blue "Auditor-visible support (N=300)". Typically the things in parenthesis should reference the same kind of information. What is 'state dimension 1'? Further, it's not clear what the plot is supposed to be showing. There are some numbers referenced in the caption but they don't seem to relate to the plot. In Figure 4(a), the caption refers to the 20th, 50th, and 80th SOH percentiles but in the figure legend it says 'SOH = 1, 0.98, 0.96" and it's not clear what that means.