acceptodds
Under review as a conference paper at ICLR 2027

Decision Equivalence Does Not Identify the Explanation: Surrogate Multiplicity in Cluster Explanations

Abstract

A common way to explain a clustering is to train a classifier on its labels and explain that surrogate with SHAP. How much does the resulting explanation depend on the classifier? Across six datasets and six model families, estimated attributions were less similar across families than across bootstrap refits of the same family. Under a shared probability-space KernelSHAP protocol, the median paired similarity gap was ; its sign persisted across tested estimator settings, while its magnitude changed. We show that this ambiguity can remain even when decisions agree at every input. An exact construction rescales all class scores by the same positive, input-dependent factor, preserving every decision while making a decision-irrelevant feature uniquely most important. The result depends on what is explained: centering and normalizing scores gives exact attribution invariance within the common positive affine family. Decision fidelity alone therefore does not identify a score-based explanation. We recommend reporting the surrogate set and fidelity criterion, explained target, value function and background, and explanation variation across the selected models.

Primary area:
interpretability and explainable AI
Keywords:
feature attribution, SHAP, clustering, surrogate models, explanation multiplicity, non-identifiability, Rashomon effect, interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_A9zROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the explanation multiplicity that arises from prediction-preserving perturbations of the class-wise scores of multi-class classifiers. The main focus is on classifiers used as surrogates for clustering models, although the core contributions are not specific to clustering. The authors argue that, to make explanations less sensitive to these prediction-preserving perturbations, one should explain standardised scores instead of the raw model output.

Critical strengths
  • The paper investigates a topic that I think is understudied in the explainability literature: what quantity should be explained for multi-class classifiers? Existing work either explain the logits, the constrast, or the softmax whil rarely considering the implications of these choices Proposition 3 demonstrates that explaining the scores s, the constrast, or the softmax probalities all attribute importance to the dummy x3x_3 variable, putting these common practices into question.
  • The proof are correct and well-written. As a result it is easy to verify their validity.
Critical weaknesses
  • Some important citations are missing. Notably the prior studies of [1] and [2] are missing from the related work. [1] previously demonstrated that additive model interpretations are underspecified because of the translation orbits (they did not consider scaling the scores by a(x)). Since interventional shapley values will coincide with the centered shape functions of additive models, the current paper is actually a generalization of [1] for model with interactions. [2] previously demonstated that gradient-based explanations of deep neural networks are arbitrary when explaining the logits because of the translation orbits .

  • While the theoretical section of the paper is solid, the experiments are hard to follow. Notably, based on Figure 2 from the appendix, I was expecting the bulk of the results to demonstrate that explaining H instead of the score, contrast, or softmax leads to reduces multiplicity across models types and boostrap refits. From my understanding, only Table 8 from the appendix shows this results ( on two datasets. ). Why were these results put in appendix if they are the most aligned with the paper premise? The results in the main paper do not show the merits of explaining H so the message of the paper gets diluted.

[1]Zhang, X., Tan, S., Koch, P., Lou, Y., Chajewska, U., & Caruana, R. (2019, July). Axiomatic interpretability for multiclass additive models. In Proceedings of the 25th ACM SIGKDD international conference on Knowledge Discovery & Data Mining (pp. 226-234).

[2]Srinivas, S., & Fleuret, F. (2020). Rethinking the role of gradient-based attribution methods for model interpretability. arXiv preprint arXiv:2006.09128.

Questions for authors

The problem the paper studies is the ambiguity of explaining multiclass classifiers. So why does the paper focus so much on clustering surrogates? I think to proposed approach should work out of the box the any multiclass classifier (including deep learning)?

Can you explain in more details how you compute the least square coefficients in section 7.1? Is it a linear model of x used to predict the difference between the centered f and g at any sample x_i?

4 comments

CW1: We thank the reviewer for recognizing the strength of the theoretical section and for pointing us to two directly relevant references. These connections help position the contribution more precisely: the additive ambiguity is established in prior work, and our Proposition 2 should be presented explicitly in that context. The main reversal result concerns the additional ambiguity introduced by an input-dependent positive multiplier, which preserves decisions while changing contrasts. We also appreciate your observation that the experimental presentation should make the role of HH more visible; the responses below address both the relationship to prior theory and the alignment between the theoretical claims and the empirical evaluation.

Both are now discussed in §3. Both concern the additive symmetry of the softmax: adding g(x)g(x) to every logit leaves the probabilities unchanged. Zhang et al. show that multiclass GAM shape functions are therefore underspecified and resolve this by post-processing. Srinivas & Fleuret show that logit gradients are arbitrary for the same reason. Our Prop. 2 is the Shapley-value form of this symmetry, and we now say so; since interventional Shapley values of a GAM are its centered shape functions, Prop. 2 reduces to [1] in that case.

We would distinguish the main result from a generalization of [1] to interactions. The reversal uses the multiplicative part a(x)a(x), which:

  • preserves the argmax but not the softmax (it acts as an input-dependent temperature);
  • changes contrasts (Remark 1);
  • is therefore not removed by the remedies of [1, 2].

The new Prop. 4 (§4.3) characterizes what removes it: HH is the maximal invariant of the positive affine orbit, so every pointwise target invariant to it (one computed from xx and s(x)s(x)) is a function of HH. HH must be computed in the space where the transformation acts. Applied to logits, a(x)a(x) is an input-dependent temperature, removed by HH of the logits; it changes the probabilities nonlinearly, so HH of the probabilities generally does not remove it. Our experiments explain HH of probability outputs, which is invariant to positive affine changes of the probabilities, not to temperature scaling of logits (§4.3).

CW2: We agree with this presentation concern and appreciate the concrete connection you draw to Figure 2. Readers should be able to assess the proposed target alongside the theoretical motivation, using comparable experiments and target-specific refit baselines. The main text now reports the probability/HH/hard-label comparison under the full protocol on all six datasets, with refit baselines for each target (Table 4, §6.2), showing both where HH improves agreement and where its benefit is limited. It supersedes the two-dataset comparison you mention (former Table 8, now Table 14).

  • Across families. HH raises the median cross-family similarity from 0.8500.850 to 0.8690.869.
  • Across refits. The refit baseline is essentially unchanged (0.9840.984 vs. 0.9820.982).
  • What limits the gain. Most of the remaining multiplicity disappears only when the decisions at SHAP's hybrid points are forced to agree, and any target that reflects the decision inherits that disagreement: the hard label still gives 0.0940.094.
  • Where HH works. When the two surrogates' decisions are forced to agree at every input, HH reduces the median gap from 0.0270.027 to 0.0060.006 (for the binary dataset, HH reduces to the decision, so there the reduction holds by construction).

To make room, the multiplier diagnostics (former §7) were moved to Appendix F, and legacy or mixed-protocol analyses are either labelled as such or removed.

Q1: The theory holds for any multiclass score map, including deep networks, and the reversal also holds for gradient-based attributions (Integrated Gradients, gradients, gradient×input; reply to ZFdK). Clustering is the motivating case because the partition has no scores of its own: the surrogate's score representation is added by the analyst, whereas supervised probabilities target P(Y∣X)P(Y\mid X). The workflow is also used in practice, including at ICLR (Manduchi et al., 2022, App. I; further examples in our reply to Ewbc, CW1).

Q2: No model of x is fit. At each input x, a(x) is a scalar regression through the origin across the K class coordinates,

a(x)=⟨gc(x),fc(x)⟩∥fc(x)∥2,fc=f−fˉ1,a(x) = \frac{\langle g_c(x), f_c(x) \rangle}{\| f_c(x) \|^2}, \qquad f_c = f - \bar{f}\mathbf{1},

the pointwise least-squares fit of the centered vector gc(x) on fc(x). It is evaluated by calling both models at x, including at hybrid points. The equation now appears in the definition in the main text.

Reviewer_A9zROpenReview ↗

Response

Thank you for your detailed answers. I am satisfied with the updated manuscript and will increase my score to a 3. The paper now positions itself correctly relative to prior work that investigated translation orbits for additive models and neural networks. Also, Table 4 shows extended results regarding the H function across all datasets. This figure demonstrates that a large portion of disagreements are due to model extrapolation (as expected from prior work). What is really novel is that disagreements remain even after aligning the predictions over all inputs. These remaining disagreements are further reduced by explaining H instead of the output probabilities. These empirical results strengthen the message of the paper.

My only issue with the manuscript is that Table 4 is referenced in section 4.3 (the theoretical section) and not in the experimental section, which is odd. Table 4 should be referenced in section 6.2.

I now understand why the focus on clustering is relevant : classification has a natural target P(Y∣X)P(Y|X), wheras numerical scores at 100% arbitrary in clustering. This exacerbates prediction-preserving perturbations. Still, I think the discussion section could clarify that the insights from this work may extend to classification. Indeed, although it is natural to explain the conditional probabilities for classification, the probabilities returned by the model are not necessarily calibrated. One could calibrate uncertainties with temperature scaling but doing so change model interactions in unpredictable ways, especially when extrapolating the data.

Reviewer_EwbcOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates whether decision-equivalent surrogate models produce consistent feature attributions for cluster explanations. The authors provide an exact construction showing that models with identical decisions everywhere can have reversed SHAP feature rankings. They also propose an attribution target invariant under positive common affine transformations. Experiments across six datasets and six surrogate families demonstrate substantial cross-family explanation disagreement despite high predictive fidelity.

Critical strengths
  1. Interesting and well-motivated problem. The paper raises an important question about whether surrogate fidelity is sufficient to support reliable cluster explanations.

  2. Elegant theoretical construction. The analytic counterexample clearly demonstrates explanation non-identifiability under identical decisions, independent of fitting or attribution estimation errors.

  3. Careful empirical evaluation. The experiments include multiple model families, bootstrap baselines, and additional controls for attribution estimation variability.

Critical weaknesses
  1. Conceptual significance could be better justified. SHAP explains numerical model outputs rather than discrete decisions. Therefore, different attributions for decision-equivalent models are not necessarily surprising. The authors should clarify what additional insights this result provides beyond the distinction between score-based and decision-based explanations.

  2. Gap between theory and experiments. The theoretical construction guarantees global decision equivalence, whereas the empirical experiments compare highly accurate but not necessarily decision-equivalent surrogates. It remains unclear how much of the observed disagreement is attributable to the proposed mechanism.

  3. Limited practical effectiveness of the proposed invariant target. The invariance guarantee applies only to a restricted family of transformations, and empirical improvements are small or inconsistent. A clearer demonstration of its practical benefits over alternative explanation targets would strengthen the contribution.

  4. Limited experimental scale and diversity. The main experiments cover only six tabular datasets, with at most 150 explained instances per dataset, and primarily focus on K-means clustering. Evaluation on larger, higher-dimensional datasets and more diverse clustering methods would strengthen the generalizability of the findings.

Questions for authors

See weakness.

5 comments

CW1: We thank the reviewer for the detailed assessment and for identifying the central interpretive question: what global decision equivalence tells us beyond the familiar fact that SHAP explains numerical outputs. We agree that this distinction must be connected explicitly to the purpose of surrogate-based cluster explanation. Your questions also help clarify the relationship between the theoretical construction, the empirical disagreement, and the scope of the invariant target. Below, we explain why perfect decision fidelity still leaves the explanation target underdetermined, and present controlled comparisons that bring the empirical evaluation closer to the theorem's setting.

The distinction between scores and decisions is established; our contribution is to show how consequential it can be even under ideal fidelity conditions. In particular, a feature that is irrelevant to the decision can become uniquely most important in the attribution while every decision remains unchanged. In surrogate-based cluster explanation, the object of interest is the partition. The clustering assigns no scores (KK-means is a hard assignment), so the surrogate's probabilities are a representation added by the analyst, and the standard validation, fidelity, certifies decisions only. In supervised classification, observed outcomes provide an external quantity against which predicted probabilities can, at least in principle, be calibrated. A clustering supplies only the partition, with no analogous outcome-defined target for the surrogate scores. The reversal holds with valid probabilities, independent inputs and no off-support behaviour, which excludes the usual explanations of explanation disagreement.

The workflow is used directly in prior work. Manduchi et al. (ICLR 2022, App. I) characterize the patient subgroups found by VaDeSC by training XGBoost to predict the cluster labels and reading TreeExplainer SHAP values, motivating the classifier as "inherently seeking those dimensions of maximal separability". TreeExplainer supports path-dependent as well as interventional values; our experiments use interventional values. Whichever value function is used, perfect agreement with the cluster labels does not determine the scores being explained. Their setting is binary; for two-class probability outputs, any two decision-equivalent score vectors lie pointwise in the same positive affine orbit away from ties, so even perfect agreement with their binary cluster labels leaves the score freedom that the theorem exploits. Morichetta et al. (2019) explain Ward clusters through an SVM and LIME, and already note that the supervised modeling step "inherently introduces a bias due to the application of a specific model". Ofek & Somech (PVLDB 2025) use the same XGBoost-on-cluster-labels plus SHAP construction as a baseline across 98 clustering results. Our result sharpens the concern: higher fidelity does not remove the ambiguity, since even globally decision-equivalent score maps can give reversed attributions.

CW2: We appreciate this distinction between the theorem's assumptions and the original empirical setting. High observed fidelity alone does not establish global decision equivalence, so the original comparisons cannot isolate the proposed mechanism. We therefore introduce a controlled post-processing experiment that enforces decision equivalence at every input and evaluates the attribution gap that remains.

  • Construction. For every pair (f,g)(f,g), g♯g^\sharp equals gg wherever the two decisions agree and swaps gg's scores for the two winning classes elsewhere. It is a probability vector that makes ff's decision at every input, including every point SHAP queries, so ff and g♯g^\sharp satisfy Definition 1 exactly. This constructs globally decision-equivalent score maps from fitted surrogate pairs; g♯g^\sharp is a controlled post-processing of gg, not a separately fitted model. It depends on the reference ff and can be discontinuous where the swapped classes change, and the control does not estimate how often naturally fitted models are globally decision-equivalent. Refit baselines are forced the same way (§6.2, Table 4; details in Appendix C.1).
  • Most of the observed gap disappears when hybrid-point decision disagreement is removed. Surrogates that agree on 98–100% of the observed rows (per-dataset medians, in-sample) disagree on a median 12.9% of SHAP's hybrid queries. Forcing the decisions to agree reduces the median gap from 0.1300.130 to 0.0270.027. Since the forcing swaps two score coordinates, it also changes the scores SHAP sees, so we read this as showing that a large part of the observed gap disappears under the decision-forcing intervention, not as an exact share.
  • Most of the remaining gap disappears under the affine-invariant target. Under exact decision equivalence, the gap stays positive in 6/6 datasets and every fit seed (0.0140.014–0.0760.076; the reverse direction gives the same median, 0.0270.027), and explaining HH reduces it to 0.0060.006. This supports the target's usefulness under exact decision equivalence. A second estimator seed moves these forced gaps by at most 0.00150.0015.
  • Breast Cancer isolates the multiplier. With K=2K=2, the forced pairs lie in the positive affine orbit at every row without an exact tie (equivalently, their centered outputs differ by a positive rescaling). Thus the 0.0390.039 forced gap provides a fitted-model realization of the theorem's positive-affine mechanism. (HH reduces to the decision when K=2K=2, so its forced gap is near zero by construction.)

This extends the existing control with identical predicted partitions on every observed point (Table 11 of the submission, now Table 17) to every point the explainer queries.

CW3: The invariance scope is deliberate and now characterized: HH is the maximal invariant of the affine orbit (new Prop. 4, §4.3). Broader invariance requires a coarser target, ultimately the label. Empirically, under exact decision equivalence HH removes most of the remaining gap (0.027→0.0060.027\to0.006). This demonstrates an empirical benefit of the target in this setting. Without forcing, what remains tracks disagreement about decisions at hybrid points: any target that reflects the decision changes where the decisions do, and the hard label, the coarsest such target, still gives 0.0940.094 (probability 0.1300.130, HH 0.1010.101). The new main-text Table 4 (§6.2) reports all three targets against their own refit baselines, so readers can see what each level of coarsening buys.

Reviewer_ZFdKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to explore if using different classifiers on the same clustering results can bring different feature attributions. It asks: if the classifiers always give each input to the same cluster, will the explanations of the output scores be the same? Theoretically, the authors construct a positive, input-dependent rescaling that preserves decisions at every input while changing exact Shapley attributions, even making a decision-irrelevant feature the most important. Next they make some progress before explanation, including center and normalize the scores. Uasually if all the class scores are processed by the same factor and have same shift, the feature attributions will keep same. However, this situation will not keep unchanged for every kind of transformation. Experiments across six datasets and six surrogate model families use a shared probability-space KernelSHAP protocol to compare cross-family attribution variation against within-family bootstrap-refit variation. Cross-family explanations are less similar on all six datasets, with a median paired similarity gap of 0.130; sensitivity analyses and additional controls examine the robustness of this finding. Finally they draw the conclusion : even though the classifiers can closely match the clustering labels, the explanations of the output scores can be different.

Critical strengths

A clear and convincing counterexample. The paper gives a strong example that even though two models output the same prediction, the internal features they used to make the decision can be different. The decision can keep identical even they use mixed features. The authors use Shapley values to evaluate the contribution of different features. This makes the result convincing: the authors show us that the difference is not caused by errors in estimating Shapley values or mistakes in fitting the models. I take this as an important strength because it clearly shows why matching clustering labels alone is not enough to guarantee the same explanation of output scores.

Critical weaknesses
  1. The contribution is limited compared with the known work. Earlier studies have already shown that different models can give different feature attributions. They also show the explanations’ difference while predictions stay the same. The main idea is that for the same output , the explanation of the features which influence the output can be different. This is useful, but the advance is relatively small. But scores after centering and normalization by authors is also similar to existing standardization methods, so this part adds limited novelty.
  2. This paper demonstrate the problem clearly, but it does not show how helpful the proposed method could be in practice sufficiently. The quantity of the samples that the authors use in the environment is too small. On Dry Bean, it reduces the disagreement gap only slightly, from 0.292 to 0.287. Differences in model probabilities on mixed inputs are also more strongly related to attribution differences than the proposed multiplier measure is. Overall, the experiments clearly show that explanations can differ. But they provide less evidence that the new target is useful in practice or that the proposed mechanism explains these differences well.
Questions for authors
  1. What does this paper add to earlier work? The example shows that feature rankings can reverse even when two models make the same decision for every input, not just the tested samples. Apart from this stronger condition, what else does the example teach us that earlier studies have not shown? Also, how is centering and normalizing the scores different from existing score-standardization methods, apart from using it for feature attribution?
  2. What is the practical value of the target ? Is the target mainly used to explain a theoretical example ? Or is it meant to make cluster explanations more stable in practice? It seems that there is no obvious improvement on Dry Bean. Besides, given hybrid inputs, TV is more strongly related to attribution disagreement than the multiplier diagnostic is. So based one these results, the proposed target is still useful in practice? Could you also clarify what kind of new understanding the affine-orbit analysis can bring?
3 comments

CW1, Q1: We thank the reviewer for the careful assessment and for recognizing that the paper demonstrates the explanation ambiguity clearly. We also appreciate the distinction you draw between agreement on observed samples and agreement at every input: this distinction is central to isolating ambiguity that persists even when surrogate fidelity is perfect. Your questions identify two points that deserve a clearer explanation, what the construction establishes beyond existing non-identifiability results, and what practical benefit the invariant target provides. We address these separately below, distinguishing the theoretical guarantee from the empirical improvements and their limitations.

What the example adds. Earlier empirical multiplicity studies commonly establish agreement on observed predictions, leaving open disagreement elsewhere. Decker et al. (2025), whom we cite, change explanations while preserving every prediction, but their temperature depends on the explainer's perturbation level, so the transformation itself depends on the coalition being evaluated. We compare two fixed functions of the input, s(x)s(x) and t(x)t(x), explained with the same value function, as two competing surrogates would be: the decisions agree at every input, including every hybrid; values are exact; inputs are independent; and off-support disagreement is not required. Beyond the stronger condition, the example shows three things:

  1. The disagreement lies in the estimand. No improvement in fitting, fidelity or SHAP estimation removes it.
  2. The standard remedy is not enough. For the known additive non-identifiability of multiclass scores, the remedy is to explain contrasts or centered scores (Zhang et al. 2019; Srinivas & Fleuret 2021; our Prop. 2). An input-dependent scale changes contrasts too, so this remedy does not cover it.
  3. It is not specific to Shapley. At x∗x^\ast, Banzhaf values give (2,1,0)→(8,4,9)(2,1,0)\to(8,4,9), Integrated Gradients (2,1,0)→(6,3,12)(2,1,0)\to(6,3,12), saliency (plain gradients) (1,1,0)→(7,7,36)(1,1,0)\to(7,7,36) and gradient×input (2,1,0)→(14,7,36)(2,1,0)\to(14,7,36); x3x_3 ranks first under each of these methods (new Table 8, Appendix A.2). More generally, any method satisfying Dummy and Sensitivity(a) (Sundararajan et al., 2017) gives x3x_3 zero credit everywhere under ss and nonzero credit under tt for some input and baseline (e.g. x=(1,0,1)x=(1,0,1) against b=(1,0,0)b=(1,0,0), which differ only in x3x_3). Whether x3x_3 ranks first depends on how the method splits interaction credit: leave-one-in occlusion at the zero baseline does not rank it first, and the table reports this case.

This also sharpens a concern raised in the work we build on. Morichetta et al. (2019), who explain Ward clusters through an SVM fit to the cluster labels, note that "the cluster modeling step through supervised learning inherently introduces a bias due to the application of a specific model". Our result shows that this bias does not vanish with a better surrogate: even perfect fidelity, at every input, does not determine the attribution (applied examples in our reply to Ewbc, CW1).

Centering and normalizing vs. standardization. The operations are standard; centering is the usual identifiability constraint for multiclass scores, and we do not claim it as new. The characterization is not: HH is the maximal invariant of the orbit, i.e. it parameterizes its equivalence classes (new Prop. 4 in §4.3: H(s)=H(t)H(s)=H(t) if and only if t=a s+b1t=a\,s+b\mathbf 1 with a>0a>0). Every pointwise orbit-invariant target (one computed from xx and s(x)s(x)) is a function of HH, and HH distinguishes exactly the positive-affine equivalence classes.

CW2, Q2: We appreciate the request for stronger practical evidence. You are right that the original small-scale Dry Bean comparison, by itself, does not establish a substantial practical benefit for HH. The expanded evaluation directly addresses this concern by comparing probability, HH, and hard-label explanations under the same protocol, and by examining when decision disagreement limits the benefit of score invariance.

Sample size. The Dry Bean figure (0.292→0.2870.292\to0.287) came from a 12-point study with a single fit. The revision explains probability, HH and hard label under the full protocol: all six datasets, 150 points, three fit seeds and their bootstrap refits (Table 4 in §6.2, reproduced in the general comment). We also repeated it with 500 explained points, where every gap stays within 0.00510.0051 of its 150-point value (Table 13), and with a second clustering algorithm (see our reply to Ewbc, CW4).

Practical value of HH.

  • Without forcing. HH lowers the median gap from 0.1300.130 to 0.1010.101 (lower in five of six datasets; on Wine it rises, 0.064→0.1000.064\to0.100). On Dry Bean the change is small (0.242→0.2310.242\to0.231), whereas forcing the two surrogates' decisions to agree at every input reduces the gap to 0.0140.014; Dry Bean also has the highest hybrid-point decision disagreement (18.718.7\\%). These are descriptive comparisons: forcing changes the scores as well as the decisions.
  • Its role. Under exact decision equivalence, HH removes most of the remaining gap (median 0.027→0.0060.027\to0.006), supporting its usefulness in this controlled setting. It cannot remove disagreement about decisions.

TV vs. the multiplier attribution Φ[a]\Phi[a] (Definition 3, Appendix F.1). The stronger association of hybrid-point TV is expected. The Shapley weights over coalitions that exclude jj sum to one, and ∣fk−gk∣≤TV|f_k-g_k|\le\mathrm{TV}, so

∣φj[fk](x)−φj[gk](x)∣≤2max⁡S,EB,TV(f(xS,B−S),,g(xS,B−S))\lvert \varphi_j\lbrack f_k\rbrack(x)-\varphi_j\lbrack g_k\rbrack(x)\rvert \le 2\max_{S}\\,\mathbb{E}_{B}\\,\mathrm{TV}\bigl(f(x_S,B_{-S}),\\,g(x_S,B_{-S})\bigr)

TV measures how much the outputs differ where SHAP queries them; Φ[a]\Phi[a] describes the affine component of that difference. The practical consequence, now stated in the paper (§6.4, §9), is to report agreement between surrogates where the explainer queries them, not only fidelity on observed data. These are different quantities: fidelity compares a surrogate with the clustering's labels, which need not be defined at hybrid points (Ward has no out-of-sample rule). Our surrogates agree on 98–100% of observed rows (per-dataset medians, in-sample), but disagree with each other on the decision at a median 12.9% of hybrid queries; observed-point TV has ρ=+0.005\rho=+0.005 with similarity.

What the affine-orbit analysis adds. It splits the gap into three descriptive levels (not additive shares):

  • the part that disappears when the decisions at hybrid points are forced to agree: 0.130→0.0270.130\to0.027;
  • the reduction in the remaining gap when explaining HH: 0.027→0.0060.027\to0.006;
  • the residual gap after applying HH, 0.0060.006: small and stable across estimator seeds (a second seed moves it by at most 0.00060.0006), although this check does not establish a noise floor.
AuthorsOpenReview ↗

Addendum to CW2 (practical value of )

The new MNIST and Fashion-MNIST experiments provide further evidence of the usefulness of HH in settings with relatively high decision agreement.

The surrogates agree on the decision at approximately 92%–93% of SHAP's hybrid queries. Without forcing, HH reduces the explanation gap by approximately 42% on MNIST (0.0430.043 → 0.0250.025) and 44% on Fashion-MNIST (0.0520.052 → 0.0290.029), compared with approximately 22% for the six-tabular-dataset median (0.1300.130 → 0.1010.101). Under enforced global decision equivalence, HH further reduces the probability gaps from 0.0320.032 to 0.0100.010 and from 0.0350.035 to 0.0080.008.

This contrasts with Dry Bean, where hybrid-point decision disagreement is higher (18.7%) and HH provides only a small reduction without forcing (0.2420.242 → 0.2310.231). The comparisons are descriptive, but consistent with a larger benefit from HH when surrogate decisions already agree more often.

Hard-label explanations give similar unforced gaps on the image datasets, but discard relative-score information that HH preserves.

Details are provided in our additional general comment and in §6.3 and Appendix O (Table 40) of the revised PDF.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Common Comments (first half)

We thank all three reviewers for their careful reading and constructive feedback. We appreciate Reviewer ZFdK's recognition that the paper demonstrates the problem clearly, Reviewer A9zR's assessment that the theoretical section is solid, and Reviewer Ewbc's assessment that the problem is well motivated and the construction elegant. No reviewer questioned the correctness of the theory or proofs; the concerns are about positioning, practical value, and the empirical link. Reviewer Ewbc's questions further sharpen the need to explain why the distinction between scores and decisions matters operationally in surrogate-based cluster explanation.

Taken together, the reviews identify three priorities: positioning the multiplicative mechanism relative to known additive ambiguities, evaluating the practical value and limitations of HH, and connecting the empirical comparisons more closely to the theorem's global decision-equivalence assumption.

The revised PDF addresses these priorities through a sharper theoretical characterization and expanded controlled experiments. Further changes (the TV bound, other attribution methods, Ward clustering, 500 explained points, the multiplier definition, and pruned legacy analyses) are described in the individual replies.

Where to find the changes. Abstract: updated to report the decision-forcing and target-ladder results, §3 (additive ambiguity); §4.3 and Proposition 4 (maximal invariant); §6.2 and Table 4 (target ladder and decision forcing; details in Appendix C.1); §6.3 with Tables 12–13 in Appendices C.2–C.3 (Ward, 500 points); §6.4 (TV bound, multiplier formula); Table 8 in Appendix A.2 (other attribution methods).

1. Theory: the additive symmetry vs. the positive affine orbit.

Zhang et al. (KDD 2019) and Srinivas & Fleuret (ICLR 2021), now discussed in §3, study the additive symmetry of the softmax; our Prop. 2 is its Shapley-value form. The reversal instead uses an input-dependent positive multiplier, which preserves every decision but changes contrasts. New Proposition 4 (§4.3) characterizes the target that removes it.

Proposition 4 (maximal invariant). For s,t∈RKs,t\in\mathbb R^K, H(s)=H(t)H(s)=H(t) if and only if t=a s+b1t=a\,s+b\mathbf 1 for some a>0a>0, b∈Rb\in\mathbb R. Hence any target T(x,s(x))T(x,s(x)) invariant under the orbit at every xx can be written T=ψ(x,H(s(x)))T=\psi\bigl(x,H(s(x))\bigr).

Proof. If sc,tc≠0s_c,t_c\neq 0 and H(s)=H(t)H(s)=H(t), then tc=λsct_c=\lambda s_c with λ=∥tc∥/∥sc∥>0\lambda=\lVert t_c\rVert/\lVert s_c\rVert>0, so t=λs+(tˉ−λsˉ)1t=\lambda s+(\bar t-\lambda\bar s)\mathbf 1. If sc=tc=0s_c=t_c=0, both vectors are constant and differ by a shift. The converse is the invariance in §4.3. ■\blacksquare

agreement between two score mapswhat it identifies (maximal invariant)
common shift s+b1s+b\mathbf 1centered scores scs_c, i.e. all contrasts (Zhang et al.; Srinivas & Fleuret; our Prop. 2)
positive common affine as+b1a s+b\mathbf 1H(s)H(s)
common strictly increasing mapthe ordering of the classes
same decision everywherethe label only (for K=2K=2 this coincides with the row above)

2. Decision-forced surrogates (new).

For each pair (f,g)(f,g), g♯g^\sharp equals gg wherever the two decisions agree and swaps gg's scores for the two winning classes elsewhere. It is therefore a probability vector that makes ff's decision at every input, including every point SHAP queries. This constructs globally decision-equivalent score maps from fitted surrogate pairs, placing their outputs in the decision-equivalent setting of the theorem. The construction is pair-specific (g♯g^\sharp depends on ff), can be discontinuous where the swapped classes change, and does not measure how often naturally fitted models are globally decision-equivalent. Refit pairs are forced the same way (§6.2; Appendix C.1).

AuthorsOpenReview ↗

Common Comments (second half)

3. A target ladder under the full protocol (new main-text Table 4, §6.2).

Probability, HH, and hard-label targets are compared on identical coalitions across all six datasets, three fit seeds, and their bootstrap refits. The decision-forced arm reports probability and HH; hard-label outputs coincide by construction. Paired similarity gap (refit baseline minus cross-family) on the assigned-cluster output, median over the three fit seeds; 150 points, 2048 coalitions:

IrisWineBr. CancerDigitsPendigitsDry Beanmedian
probability0.1110.0640.1850.1490.0780.2420.130
HH0.0990.1000.1720.1030.0660.2310.101
hard label0.0910.0920.1720.0970.0650.2330.094
probability, decisions forced0.0300.0230.0390.0760.0210.0140.027
HH, decisions forced0.0150.0040.00020.0140.0060.0050.006

Every gap is positive in 6/6 datasets and in every fit seed. A second estimator seed moves no entry by more than 0.0070.007, and no decision-forced entry by more than 0.00150.0015.

  • Surrogates disagree where SHAP queries them. Cross-family pairs agree on 98.0–100% of the observed rows (per-dataset medians, in-sample), but disagree on the decision at a median 12.9% of SHAP's hybrid queries (6.5–18.7% across datasets; refits: 5.7%).
  • What the table shows. Most of the observed gap disappears when that hybrid-point decision disagreement is removed by the forcing construction (0.130→0.0270.130\to0.027), which suggests it accounts for a large part of the gap. Any target that reflects the decision changes where the decisions do: the hard label, the coarsest such target, still gives 0.0940.094. Under exact decision equivalence, a positive gap remains in all six datasets (median 0.0270.027), and HH removes most of it (0.0060.006). This supports the usefulness of the affine-invariant target, but does not by itself establish that most of the score differences were affine.
  • Breast Cancer isolates the multiplier. With K=2K=2, decision equivalence coincides pointwise with the positive affine orbit (away from exact ties), so the forced pairs differ only by an input-dependent positive affine transformation (equivalently, a positive rescaling of the centered outputs). Thus the 0.0390.039 forced gap provides a fitted-model realization of the theorem's positive-affine mechanism. For K=2K=2, HH reduces to the decision, so its forced gap (0.00020.0002) is near zero by construction, not as evidence.

4. Practical relevance.

The use of supervised surrogate models to explain clustering results is well established in published work. Manduchi et al. (ICLR 2022, App. I) explain an XGBoost classifier trained on VaDeSC cluster labels with TreeExplainer SHAP; Morichetta et al. (2019) already note that this supervised modeling step "inherently introduces a bias due to the application of a specific model"; and Ofek & Somech (PVLDB 2025) use the same construction as a baseline across 98 clustering results (details in our reply to Ewbc, CW1). Fidelity validates the decisions, not the score representation that is explained.

The practical lesson is that fidelity alone does not identify a surrogate explanation. The expanded comparisons distinguish disagreement about decisions from sensitivity to score representation, and show both the benefit and the limits of choosing an affine-invariant explanation target.

AuthorsOpenReview ↗

Additional experiment: high-dimensional inputs (§6.3; Appendix O, Table 40)

Following Reviewer Ewbc's request for larger, higher-dimensional datasets, we repeated the full target-ladder protocol (probability, HH, and hard-label targets; decision forcing; three fit seeds with bootstrap refits; 150 explained points; 2048 coalitions; two estimator seeds) on MNIST and Fashion-MNIST (70,000 images, 784 pixels each) and CuMiDa Leukemia GSE28497 (281 samples, 22,283 genes).

Each dataset is reduced to 64 PCA components, the dimension at which our estimator settings were validated. These components retain 60%, 83%, and 72% of the standardized variance, respectively. Attributions therefore explain principal components, not individual pixels or genes.

DatasetProbabilityHHHard labelForced probabilityForced HH
MNIST0.0430.0250.0260.0320.010
Fashion-MNIST0.0520.0290.0280.0350.008
CuMiDa Leukemia0.1160.0760.0980.0470.019
  • Robustness. Every gap is positive in every fit seed at both estimator seeds; changing the estimator seed moves no gap by more than 0.00470.0047. Same-model repeatability is at least 0.980.98 for all three targets.
  • Image datasets. The surrogates achieve high fidelity (five-fold accuracy 0.9350.935–0.9950.995) and disagree on the decision at only 6.9%–7.9% of SHAP's hybrid queries. Most of the probability gap persists after enforcing decision equivalence (0.0430.043 → 0.0320.032; 0.0520.052 → 0.0350.035), and explaining HH reduces the remaining gaps to 0.0100.010 and 0.0080.008. Without forcing, HH lowers the gaps by approximately 42% and 44%, respectively. Hard-label gaps are comparable to those of HH, but discard the normalized relative-score information that HH retains.
  • CuMiDa Leukemia.: five-fold surrogate accuracy is only 0.650.65–0.870.87, and differences in surrogate fidelity may contribute to the observed gap. We therefore do not interpret this result as evidence of explanation multiplicity among highly faithful surrogates.

These results extend the evaluation described in our earlier CW4 reply. They provide additional empirical evidence that explanation multiplicity can persist under enforced global decision equivalence, without establishing that all remaining disagreement belongs to the positive-affine orbit.

The revised manuscript reports these results in §6.3, with the full table and preprocessing details in Appendix O (Table 40). Appendix O is placed last, preserving the earlier appendix and table references. The original six-dataset results and their reported medians remain unchanged; the new experiments are reported separately.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.