When Same Sign Is Not Enough: Auditing Modality-Dominance Claims
Abstract
A modality score is not yet a modality-dominance claim. Prior work established that removal semantics can change explanations and that importance estimates can be uncertain, but these issues are usually treated separately. We ask whether a directional modality claim remains statistically resolved across a declared family of plausible interventions. We audit a fixed text–numeric predictor under three removal operators, separate the observed reliance gap from an operator-specific empirical no-signal reference, and use paired inference that propagates dependent-sample and finite-null-bank uncertainty. In a leakage-audited financial forecasting stress test, all 21 operator–measurement null-centred point estimates favour numeric reliance. Yet only 5/7 measurements are resolved across the full operator family; 2/7 remain unresolved under same-date regime-matched removal. A prespecified marginal-versus-regime-matched magnitude contrast excludes zero on all seven measurements. Thus complete point-sign agreement can coexist with incomplete family-level inferential resolution. The contribution is a claim-level audit that determines when intervention semantics and uncertainty support a modality-dominance statement; the underlying attribution score is unchanged.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.