acceptodds
Withdrawn from ICLR 2027

A Shortcut Can Be Real Yet Undiagnosable: Identifiability Limits of Observational Shortcut Auditing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A shortcut can be real yet observationally undiagnosable. Shortcut auditing is therefore an identification problem before it is an estimation problem: a fixed predictor may rely on identity-associated information even when the data lack the same-state cross-identity comparisons needed to reveal that reliance. We define predictor-specific reliance under an explicit target comparison distribution and distinguish full-target reliance, the support-restricted population estimand, and the finite audit estimate. If the target assigns positive mass to unsupported comparisons, full-target reliance need not be point-identified. Under bounded prediction discrepancy, partial support yields an identified interval whose width contracts with target-weighted support coverage. This result motivates a support-aware audit that reports prediction contrast, admissible pair count, and target coverage separately, and returns Abstain when no admissible requested comparison is available. In a controlled intervention that varies only auditor-visible support while holding the predictor and finite full-target oracle reference fixed, recovery fidelity improves in model–seed endpoints, while all zero-support views abstain. In CWRU vibration sensing, support thinning degrades recovery of the full-support observational reference in all five repetitions. At a fixed pair count, concentrated support incurs the recovery error of broadly distributed support at the median seed. Stanford battery EIS provides a complementary boundary case: substantial deployment degradation does not identify its source. Shortcut audits must therefore establish which predictor comparisons are observationally supported before interpreting estimated reliance.

Primary area:
interpretability and explainable AI
Keywords:
Shortcut Learning, Observational Identifiability, Model Auditing, Distribution Shift, Physical Sensing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PoGBOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is about auditing methods for figuring out when a model is using shortcuts or spurious correlations to learn relationships between variables; some methods may predict that a model barely uses shortcuts, but if there is not much useful data available, then the method's prediction can be meaningless. Hence it's useful to audit these methods to detect shortcuts, and this paper provides a framework for it. Specifically, they formalize when the amount of available data is sufficient to detect that a model is relying on shortcuts. They propose an audit procedure and then conduct some experiments to illustrate their method.

Critical strengths

The problem motivation is an interesting one, and being able to detect when a shortcut auditing method is reliable or not is an important problem.

Critical weaknesses

The text is extremely difficult to follow. For example, starting the abstract with "A shortcut can be real yet observationally undiagnosable" is unclear, and one could instead say something like "Shortcuts learned by machine learning models to make predictions can be difficult to diagnose". The next phrase, "a fixed predictor may rely on identity-associated information even when the data lack the same-state cross-identity comparisons needed to reveal that reliance," reads as a really roundabout way of saying "models might rely on spurious features for making predictions, but sometimes there might not be enough data available for a diagnostic method to detect that those features are indeed spurious." Here, "identity-associated information" and "same-state cross-identity" makes no sense without context. The same applies for the rest of the abstract--there was no sentence in it that was easily comprehensible.

In the introduction, when the problem is set up, "full-target predictor-specific reliance" reads similarly: "target" has not been defined, nor has "full-target," it's not clear what the "predictor" is, etc. In Figure 1, the variables are not defined (until later) and the captions are inadequate for comprehension. After staring at it for a while, my understanding is that panel (a) means that you take some physical state s, measure it using two different methods m and m', and then pass them each through the model f, and then figure out how different f's predictions where on the two different measurements. This could be way easier to understand if one included text in the figure instead of variables that haven't been defined yet!

In Section 3, a lot of the notation could be made a lot more clear if one had a running example (e.g., "Let SS denote the task-relevant physical state (for example, ___)").

Generally, the above comments apply for the later sections (evaluation design, results, discussion, conclusion) as well. They are very difficult to parse--the results might be interesting, but it would be much easier to tell if the paper used a more clear writing style. Hence, it is difficult to make an informed conclusion about the soundness or novelty of the methods in the paper.

Questions for authors
  1. Can you define what counts as a shortcut? I think the big idea in the paper is to measure how model predictions differ across different kinds of measurements or data sources, but that doesn't establish the presence of a shortcut. I think you also acknowledge this when you say that your method doesn't establish causal effects, but it would helpful to clarify what exactly you mean then.
  2. How do you select QQ? I think this is equivalent to asking how you select the possible relevant comparisons.
  3. Section 4 starts with "We evaluate four links in the identification chain from Section 3," what are all the links and what is the identification chain? Additionally, Stanford EIS and CWRU are not introduced with any context. I googled "Stanford EIS" and it took me some searching before I could figure out that that refers to a dataset. The rest of the terms in "Stanford EIS examines deployment degradation when attribution remains unidentified" are also not easy to understand clearly.
  4. In Section 3, dYd_{\mathcal{Y}} is undefined.
  5. Section 3.2 starts with the assumption that U=SU=S, and UU is only briefly defined in 3.1 but it's not super clear how they differ
  6. In Figure 3, the gray dots are labeled "Full target state space (Q)" and the blue "Auditor-visible support (N=300)". Typically the things in parenthesis should reference the same kind of information. What is 'state dimension 1'? Further, it's not clear what the plot is supposed to be showing. There are some numbers referenced in the caption but they don't seem to relate to the plot. In Figure 4(a), the caption refers to the 20th, 50th, and 80th SOH percentiles but in the figure legend it says 'SOH = 1, 0.98, 0.96" and it's not clear what that means.
Reviewer_Bnz8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is concerned with understanding shortcut biases in models. Specifically, it is concerned with predictor-specific reliance (if a model actually relies on a specific shortcut) and whether it is identifiable in the first place. To this end, the paper distinguishes full-target reliance (fully theoretical), support-restricted reliance (the in theory recoverable reliance from infinite observational data) and the finite-sample estimate (what can actually be computed). It then introduces 4 experiments that isolate specific aspects of the identification chain, to check whether the frameworks predictions hold.

Critical strengths

The proposes framework is well thought through, seems mathematically sound and provides a real nonidentiiability result.

Separating shortcut-learning into predictor-specific reliance, identity decodability and deployment error seems quite intuitive for me and focusing on one of them specifically is a good grounding the framework.

The provided experiments provide meaningful insights into the framework behavior under various conditions.

I particularly like the feature of the framework to abstain from judgement if there is no support available which is different than having zero reliance.

The separation of pair count and target-weighted coverage (5.3) and the triple reporting norm seems like a meaningful insight for practitioners.

Critical weaknesses

The intro front-loads "predictor-specific reliance," "identity-associated variation," "same-state cross-identity comparisons," "support-restricted estimand," "target-weighted support coverage," etc. before any are defined. For readers that are not familiar with the literature around this work, the introduction is quite dense.

My most fundamental concern is about the estimation of K^_Q. As far as I understand, given some observational data, we can always confirm presence of support but never absence of support. With this, K^_Q features a one-sided detectability limit. Further K^_Q seems to also depend on the auditor`s chosen U. How does this affect the practical application of this framework?

Concerning the empirical section of this work, I am missing two concrete components:

  1. A comparison to another way of estimating predictor-specific reliance. How do other approaches do on the synthetic experiments you provide? Do they differ in their conclusions? If yes how?
  2. A real-world example of an f with a known shortcut bias ( I am thinking some simple image dataset with camera information in the bottom corner). How does your proposed framework behave in such scenarios?

I might be wrong here but the partial-identification interval (eq. 8 , while being a valid theoretical contribution, is never actually analyzed in the results. Why is that?

Questions for authors

See my critical points above.

Reviewer_anWfOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper seems to be about methods for determining when shortcut learning occurs in deep learning. From what I can tell, the authors appeal to causal identifiability as the right approach to this problem.

Critical strengths

None

Critical weaknesses

As we all know, the conference has a high volume of sloppy AI assisted papers. This is a particularly bad offender and I didn't spend much time considering its merits, as it is clearly well below publication standard.

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.