acceptodds
Under review as a conference paper at ICLR 2027

Evaluation Collapse: When Metrics Cannot Distinguish Model Behavior

Abstract

Scalar evaluation metrics (accuracy, AUROC, F1, perplexity) rest on three structural restrictions: pointwise scoring, permutation invariance, scalar aggregation. We prove this combination cannot represent behavioral properties, including detection delay, subgroup performance, and confidence trajectories, that depend on discarded information. For any such property, behaviorally distinct models receive identical metric scores, even at the maximum score where practitioners are most confident. No metric can recover what its representation discards. We prove: (1) collapse occurs whenever behavioral information is absent; (2) collapse families persist at optimal scores, making metric tuning futile; (3) for ordered sequences, collapse size follows an exact combinatorial law (Theorem 3). We instantiate this in three settings: anomaly detectors at differing by 50 steps in latency, classifiers with identical accuracy differing by 20 points in subgroup error rates, and language models with identical perplexity spanning full confidence-trajectory ranges. The phenomenon appears in real systems: COMPAS, a deployed recidivism-risk tool, exhibits this signature, with near-identical accuracy across race concealing a 20-point divergence in false-positive/negative rates, independently reproduced from public data. We operationalize the result as a lightweight audit on already-computed scores (no retraining), with characterized failure modes: it reliably detects collapse but can underestimate by if the perturbation distribution is too narrow. The core question: before certifying behavior from a metric, does the evaluation retain enough information to see what you're certifying?

Primary area:
datasets and benchmarks
Keywords:
evaluation metrics, impossibility theorem, model evaluation, fairness auditing, anomaly detection, calibration, permutation invariance, algorithmic accountability, COMPAS, metric design

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_VKZ2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates when evaluation metrics discard information necessary for assessing model behaviour. It formalises non-identifiability using evaluation representations, derives permutation-based collapse counts, and demonstrates the issue via detection latency, subgroup disparity, and language-model confidence trajectories. It also introduces a perturbation audit and presents synthetic experiments and a COMPAS case study. Its central message is that a metric cannot certify properties that its representation does not retain.

Critical strengths
  1. The work has studied a practical and useful problem existing in model evaluation.

  2. The audit is inexpensive to apply to stored scores, and the paper explicitly acknowledges that small observed spread does not establish the absence of collapse. The perturbation-locality ablation usefully illustrates this limitation.

Critical weaknesses
  1. The exposition and structure of this work need further refinement; specifically, definitions of collapse and behavioral information should be introduced early in the paper.

  2. While the research question is important and practical, both the scope of the study and the analysis remain limited, and the resulting conclusions offer little significant new insight or guidance for model evaluation.

  3. Theorem 1 assumes linearly many permutations with distinct behavioural values; it does not derive that diversity from the structural restrictions. A property can be non-identifiable yet take only two values, such as whether the first score exceeds a threshold. This contradicts Remark 1's suggestion that a single collision necessarily propagates into linearly many distinct behaviours. Theorem 2 likewise assumes that the orbit already contains a maximiser. These conditional results are valid, but do not establish the broad guarantees suggested in the abstract and Section 4. Furthermore, reordering complete examples with their group identities leaves subgroup accuracy unchanged, so the subgroup error-reallocation construction needs a transformation argument beyond Theorem 1's dataset permutations.

  4. Table 3 audits constructions with analytically fixed metric values, using resampled alert positions, error allocations, or shuffled log-probabilities. This demonstrates variability within the supplied perturbation families, but does not show that alternatives are attainable by models on the same task and data. Shuffling stored token log-probabilities does not by itself construct another autoregressive model or valid sequence with those conditional probabilities. COMPAS is a useful illustration, but compares groups within one classifier and is explicitly not an exact collapse-family instance. The empirical claims should distinguish representation-level ambiguity from demonstrated differences among feasible model behaviours.

Questions for authors
  1. How would authors revise Proposition 4 to handle constant and highly imbalanced score sequences? Which added occupancy assumptions support exponential growth, and which downstream claims would change?

  2. Is Theorem 3 intended to count only the permutation orbit of a fixed multiset? If it concerns the full metric-level family, how does it exclude equal-scoring sequences from other orbits?

  3. Can authors distinguish the behavioural-diversity assumptions of Theorems 1 and 2 from their conclusions, and specify the transformation under which the subgroup construction is an instance of the framework?

  4. Can the remedies state the retained information explicitly and show that each claimed single-restriction relaxation leaves the other restrictions intact?

  5. What constraints make an audit perturbation a valid alternative for the intended task or model class? Can the existing experiments distinguish score rearrangements from feasible differences in model behaviour?

3 comments
AuthorsOpenReview ↗

Response to Reviewer VKZ2: Theorem Scope, Proposition 4 and Empirical Claims Corrected

We thank the reviewer for a precise review. Most technical points are correct; below we state what we concede and what we will change in the manuscript.

Theorems 1-2 and Remark 1 (W3, Q3). Correct. Theorem 1 is conditional: it assumes Θ(n)\Theta(n) distinct values of ϕ\phi over the orbit and does not derive them. A two-valued property (e.g. "does the first score exceed τ\tau") is non-identifiable yet has only two values, so Remark 1 is wrong as written: a collision propagates to the whole orbit, but the number of distinct ϕ\phi-values depends on ϕ\phi. We will (i) restate Theorem 1 for any E\mathcal{E}-preserving transformation family TT (dataset reorderings, group relabelings) as "the collapse family contains every value ϕ\phi takes over the TT-orbit", with Θ(n)\Theta(n) an explicit hypothesis, verified in the constructions and derived for ϕord\phi_{\mathrm{ord}} (n!/∏jnj!≥nn!/\prod_j n_j!\ge n for non-constant sequences); (ii) state Theorem 2 as conditional on the orbit containing a maximiser (true in the AUROC construction); (iii) delete Remark 1's tightness claim; (iv) change the abstract's "for any such property, Θ(n)\Theta(n)..." to "for properties that vary across the orbit".

Subgroup construction (W3, Q3). Correct: permuting whole examples together with their group labels leaves subgroup accuracy unchanged. The right TT is relabelling groups against fixed scores: the evaluation representation depends only on the fixed scores and is invariant to group relabeling, whereas subgroup accuracy depends on which group labels are assigned to those scores (Θ(m)\Theta(m) distinct disparities; App. B.2 is the calculation). We will describe the construction this way, as an instance of Lemma 1 and the generalized Theorem 1.

Proposition 4 (Q1). The reviewer is right: the monotonicity step fails when nmax⁡>n/2n_{\max}>n/2. The correct bound is ∣Orb∣≥(nc)|\mathrm{Orb}|\ge\binom{n}{c} with c=min⁡(nmax⁡, n−nmax⁡)c=\min(n_{\max},\,n-n_{\max}). This bound equals 11 for constant sequences and is Θ(nc)\Theta(n^c) for fixed cc, so exponential growth needs an occupancy assumption: if c≥αnc\ge\alpha n for a fixed α∈(0,1/2]\alpha\in(0,1/2], then ∣Orb∣=Ω(2nH(α)/n)|\mathrm{Orb}|=\Omega(2^{nH(\alpha)}/\sqrt n). We will restate it so and soften "generically exponential" in Remark 2 and Limitations.

Theorem 3 (Q2). Yes, it counts only the permutation orbit of a fixed score sequence; it cannot exclude equal-scoring sequences from other orbits, so it is a lower bound on ∣Cr∣|\mathcal{C}_r|. We will restate it.

Remedies (Q4). Fair. We will restate each proposition with its retained information and the restrictions it leaves intact. (1) Order only: the scalar first-crossing time min⁡{t:st>τ}∈R\min\{t:s_t>\tau\}\in\mathbb{R} uses pointwise scores and gives scalar output, and with the known onset t0t_0 it determines ϕlatency=min⁡{i≥t0:si>τ}−t0\phi_{\mathrm{latency}}=\min\{i\ge t_0:s_i>\tau\}-t_0 (in our construction normal scores lie below τ\tau, so the two minima coincide). (2) Group access only: a symmetric scalar over pairs (si,gi)(s_i,g_i), e.g. worst-group accuracy, keeps scalar output and permutation invariance but determines only that statistic; the full vector ϕsub\phi_{\mathrm{sub}} also needs vector output. (3) Vector output only: the sorted scores keep the score distribution but not the ordered trajectory, which needs permutation invariance and scalar aggregation relaxed together. We will remove "individually relaxed" and "formally independent".

Empirical scope (W4, Q5). Agreed. Table 3 shows representation-level ambiguity within the supplied perturbation families, not differences among natural models; shuffled log-probabilities are not a trained autoregressive model, and COMPAS is a within-classifier comparison, not an exact collapse instance. On a fixed evaluation set a model can be defined to emit any such score vector, but we do not show trained models in that class. We will (i) define a perturbation as valid only if it lies in the feasible set F\mathcal{F} of score vectors realizable by the intended model class, and make F\mathcal{F} an input to Algorithm 1; (ii) label these experiments representation-level; (iii) list comparison of independently trained models as an open limitation.

Presentation (W1). We will move the definitions of evaluation representation, behavioral dimension and collapse family into the introduction.

These changes narrow the claims to what the proofs and experiments support: a precise account of when a metric cannot see a behavior, and an audit for it. We respectfully ask the reviewer to reconsider the ratings in light of them.

Thanks for authors' response.

The rebuttal answers the factual parts of my questions. My main concern remains and the scope of the study and the analysis remain limited. I maintain my original rating.

AuthorsOpenReview ↗

Scope of the study

Thank you for the follow-up, and for confirming that our responses address the factual parts of your questions. We accept your remaining point on scope: the framework covers pointwise, permutation-invariant, scalar evaluation, and the analysis is limited to the constructions and conditional results we state. We will state this scope explicitly in the introduction and the limitations section, so that it is not left to the reader to infer, and we will keep our claims within it. Your review caught an error in Proposition 4 that we would otherwise have missed, and we are grateful for its precision.

Reviewer_JtQfOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates "evaluation collapse", where identical aggregate metric scores may conceal differences in model behavior. The authors formalize this phenomenon through three structural restrictions of evaluation metrics and derive impossibility results and quantitative collapse bounds. They provide constructions involving AUROC, accuracy, and perplexity, together with a COMPAS case study. A perturbation-based Collapse Audit is proposed to identify behavioral differences not reflected by aggregate scores.

Critical strengths
  1. A good research question.

The paper highlights the risks of drawing behavioral conclusions from aggregate metrics. The examples in Section 3.7 illustrate this issue across multiple domains.

  1. Potentially useful diagnostic perspective.

The Collapse Audit (Section 4) provides a simple approach to investigating metric-preserving behavioral variations, while Section 4.2 acknowledges important limitations arising from insufficient perturbation coverage.

Critical weaknesses
  1. Limited novelty and questionable problem framing.

The paper presents the inability of aggregate metrics to capture detailed model behavior as evaluation collapse. However, metrics such as accuracy, AUROC, and perplexity are designed to summarize specific performance dimensions, not to characterize all aspects of model behavior. Detailed behavioral analysis generally requires sample-level inspection or complementary metrics, which is beyond the intended scope of a single aggregate score. Moreover, closely related phenomena have been studied under predictive multiplicity and underspecification (e.g., Marx et al., 2020; Coston et al., 2021; D'Amour et al., 2022). The paper does not convincingly establish what fundamentally new problem, insight, or capability its framework introduces beyond these existing observations.

  1. Concerns about the theoretical correctness and generality.

Theorem 1 assumes the existence of (Θ(n)\Theta(n)) behaviorally distinct permutations, making its linear bound conditional on a strong premise. Theorem 3 further presents (n!/∏jnj!n!/\prod_j n_j!) as an exact collapse-family size, although this expression counts permutations of a fixed score multiset, not necessarily all configurations sharing the same metric value as defined in Section 3.1. These issues directly affect the claimed quantitative theoretical contributions. The authors should clarify the assumptions and scope of these results, and revise the claims where necessary.

  1. Significant mismatch between the proposed Collapse Audit and its implementation.

After examining the code provided through the anonymous repository link, I identified substantial discrepancies that require explanation. In particular, Algorithm 1 requires perturbations to preserve the evaluation representation (E\mathcal{E}), whereas code/audit_core.py (lines 29–32) only checks equality of the final metric (M\mathcal{M}). These conditions are not generally equivalent. Additionally, the anomaly audit (audit_anomaly, lines 55–82) directly retrieves the latency value supplied by its synthetic perturbation generator, rather than independently recovering it from actual model predictions. These differences raise concerns about whether the released implementation faithfully realizes the proposed algorithm and whether the experiments demonstrate its claimed diagnostic capabilities.

  1. Real-world experimental claims are not adequately supported by the released code.

Several experiments described as real-world validation rely substantially on artificial constructions. In Section 3.7.1, the paper describes a trained detector evaluated on NAB data. However, the notebook implements a handcrafted temperature-drop scoring rule without training a detector, and subsequently simulates different detection delays through score resampling rather than evaluating actual detector trajectories. Similarly, the GPT-2 experiment permutes observed token log-probabilities instead of comparing realizable trajectories from distinct models. The UCI Adult experiment enumerates hypothetical error reallocations rather than evaluating independently trained classifiers. These experiments demonstrate properties of constructed score sequences, but do not directly establish the claimed behavioral differences between actual models. The authors should explain these discrepancies and clarify which conclusions are supported by real-model evaluations.

  1. Numerical inconsistencies and incomplete reproducibility.

Table 3 reports a theoretical trajectory spread of 2.0, but the implemented Spearman-correlation construction contains two equally sized tied score groups, for which the maximum attainable spread is approximately 1.7336. Furthermore, the code provided through the anonymous repository does not appear to include the perturbation-radius and sampling-budget experiments supporting the prominently reported 50-fold underestimation in Section 4.2. Although the standalone script reproduces the reported synthetic audit values, these issues raise concerns about the accuracy of the theoretical comparisons and the reproducibility of the failure-mode analysis. The authors should provide the missing experimental evidence and reconcile the reported quantities with their actual definitions.

Questions for authors
  1. How does Theorem 3 establish an exact collapse-family size when its formula counts only permutations of a fixed score multiset? What independently guarantees the (Θ(n)\Theta(n)) distinct behavioral values assumed in Theorem 1?

  2. The anonymous repository implementation appears inconsistent with Algorithm 1 and the NAB experimental description. Why does the implementation check metric equality rather than representation equality? Why is the NAB detector described as trained when the notebook implements a handcrafted scoring rule? Please clarify which implementation and experimental protocols correspond to the reported results.

  3. Which experiments demonstrate behavioral differences between actual models rather than artificially rearranged scores? Could the authors also provide the implementation supporting the 50-fold underestimation result and clarify the theoretical trajectory spread reported in Table 3? These details are important for assessing the validity and practical relevance of the proposed audit.

3 comments
AuthorsOpenReview ↗

Response to Reviewer JtQf: Every Code and Table 3 Issue Verified and Fixed

We sincerely thank the reviewer for the careful review, including the line-by-line inspection of our theoretical claims, numerical results, and released code (https://anonymous.4open.science/r/eval_collapse-CB6B/). The review identified several genuine discrepancies, which we have corrected in the repository and address below. Repository (done): code/audit_core.py (representation check, latency from scores, Table 3 reference value, negative control), new code/audit_sweeps.py with results/audit_sweeps_table.csv, and a regenerated results/audit_demo_table.csv. Manuscript changes are listed per point and will appear in the revision.

W2/Q1 (Theorems 1 and 3). Correct. The Θ(n)\Theta(n) distinct-behavior condition in Theorem 1 is a hypothesis, instantiated by the constructions in Sec. 3.7 and App. B. For ordered sequences it also follows from counting: n!/∏jnj!≥nn!/\prod_j n_j!\ge n for any non-constant score sequence, and Prop. 4 gives exponential growth under finite precision. Theorem 3 counts the permutation orbit of a fixed score sequence, all of which share one metric value. Other score multisets may share the same M=r\mathcal{M}=r, so the result is exact for the orbit and a lower bound for ∣Cr∣|\mathcal{C}_r|. We will restate Sec. 3.1 and Theorem 3 accordingly.

W3/Q2 (Algorithm 1 vs. code). Correct on both counts, and fixed. Algorithm 1 tests E\mathcal{E}-equality, whereas the old code tested only M\mathcal{M}-equality (sufficient for a metric-specific audit, but weaker). collapse_audit now supports both modes (rep_fn/observed_rep). In audit_anomaly, ϕ\phi is now computed from the perturbed scores (min⁡{i≥t0:si>τ}−t0\min\{i\ge t_0: s_i>\tau\}-t_0), with an assertion that it matches the generator. We also acknowledge that in our synthetic constructions every perturbation is metric-invisible by design, so the demo validated the audit against theory but did not show it discriminating. We therefore added a negative control using a latency-sensitive metric, AUROC × (1−latency/50)\times\,(1-\text{latency}/50). Because this metric changes with latency, the audit retains only the 54/3000 perturbations whose latency matches the observed one and reports zero spread, as it should.

W4/Q2-Q3 (real-model evidence). Agreed. The NAB detector is a handcrafted temperature-drop score (AUROC 0.83), not a trained model, and latency is simulated by score resampling. GPT-2 permutes real log-probabilities, and UCI Adult enumerates error allocations. These are score-level illustrations on real data, not comparisons of independently evaluated models. COMPAS is the only analysis of a deployed model's predictions. We will relabel these in Sec. 3.7, the abstract and the README ("confirmed on real GPT-2" becomes "illustrated using real GPT-2 log-probabilities").

W5/Q3 (numerics, sweeps). The reviewer is right about Table 3. With two tie classes of size T/2T/2, the maximum Spearman spread is 2⋅T/4(T2−1)/12=1.73362\cdot\frac{T/4}{\sqrt{(T^2-1)/12}}=1.7336 for our T=24T=24 (→3\to\sqrt3 as T→∞T\to\infty), not 2.02.0; the observed 1.48071.4807 is 85%85\% of the maximum. The repository now computes this value, and Table 3 will be corrected. The missing Sec. 4.2 sweeps are now in audit_sweeps.py. Re-running them showed two things we will state plainly: the locality sweep is analytic (spread =r/rmax⁡=r/r_{\max} of the true value, so "50×50\times" is the case r=1r=1 of 50) and is an illustration of the failure mode, not an empirical finding; and our earlier "98% / 100%" budget figures came from a single seed. Over 200 seeds the audit recovers about 49%49\% of the true spread at B=3B=3, 93%93\% at B=25B=25 and 100%100\% by B≈500B\approx500. We will report these means with standard deviations.

W1 (novelty). We agree and will cite and discuss predictive multiplicity and underspecification (Marx et al., 2020; Coston et al., 2021; D'Amour et al., 2022). Those works show that equal-risk models can differ. Our focus is to characterize structural conditions on evaluation representations under which behavioral distinctions are unrecoverable by any scoring rule (Cor. 1), to identify which relaxation recovers which behavior (Props. 1-3), and to give a score-only audit. We do not claim metrics should capture all behavior (Sec. 5).

The remaining contribution is narrower than we first stated and, we believe, better supported: a precise account of when a scalar metric cannot see a behavior, and a checked audit for it. We respectfully ask the reviewer to reconsider the soundness and overall ratings in light of these verified fixes.

Thank you for the detailed response and for addressing several of the identified issues.

I acknowledge the theoretical clarifications, numerical corrections, and implementation updates. However, my main concerns remain, particularly regarding the empirical validity of the proposed Collapse Audit.

While the code changes address specific implementation inconsistencies, they do not fully establish the audit's practical diagnostic capability. As the authors acknowledge, the original positive examples were constructed to be metric-invisible, while the added negative control uses a metric explicitly dependent on latency. These experiments mainly confirm expected behavior under controlled constructions rather than demonstrate effective auditing of actual model behavior.

Furthermore, the NAB, GPT-2, and UCI Adult experiments are now acknowledged as score-level illustrations rather than comparisons between independently evaluated models. Although this clarification improves the accuracy of the presentation, it does not resolve the limited empirical support for the broader claims.

Overall, I appreciate the authors' efforts and the corrections made. Nevertheless, my concerns regarding soundness, novelty, and practical significance remain sufficiently substantial that I will maintain my original ratings.

AuthorsOpenReview ↗

Scope of the Collapse Audit's validation

Thank you for the careful follow-up and for acknowledging the corrections. We agree with your remaining point. The positive examples are metric-invisible by construction and the negative control is latency-dependent by design, so together they only confirm expected behavior against known ground truth. They do not show diagnostic value on the behavior of actual models, and the NAB, GPT-2 and UCI Adult experiments do not close that gap.

In the revision we will therefore:

  • state the audit's validated scope precisely: it checks representation-level ambiguity over a user-specified perturbation family and is verified against constructions with known answers;
  • remove or qualify any wording that suggests it has been shown to diagnose real model differences;
  • make a feasible set of perturbations (those realizable by the intended model class) an explicit input to the audit;
  • list evaluation on independently trained models as the main open empirical step.

We understand and respect your decision. The review improved the paper's accuracy and the repository, and we are grateful for the time you spent on both.

Reviewer_sVHqOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

A metric gives one number for a model. One number cannot hold everything about how a model behaves. So two models that behave differently can get the same number. The authors call this “collapse”: different behaviors collapse into one score. They say this is a structural fact about metrics like accuracy, AUROC, F1 and perplexity. The authors say that these metrics have three properties: 1) they score each example separately (pointwise scoring), 2) they do not depend on the order of the examples (permutation invariance), and 3) they give one number (scalar aggregation). The authors claim that, because of these properties, the metrics cannot represent behaviors that need the discarded information: detection delay, subgroup performance, and the confidence of a model along a sequence.

Critical strengths

The authors point out an important concern on the classical evaluation strategy which becomes a modern machine learning evaluation problem and tried to provide simple theorem and proof.

Critical weaknesses

The main theory is weak, and one result is wrong. Theorem 1 assumes there exist Θ(n)\Theta(n) permutations that give pairwise different behavioral dimensions (ϕ\phi). Its conclusion that Θ(n)\Theta(n) behaviorally distinct configurations share the same metric value follows directly from this assumption. The only new step is that an order-invariant metric does not change under reordering, which is already known. Theorem 2 only adds the assumption that one configuration has the maximum score. Theorem 3 uses the standard formula for counting orderings of a list with repeated values, and its “behavior” is the order of the scores itself. A metric that ignores order cannot see order, so this is true by definition. These theorems are about one model f with reordered data, but the abstract says "Θ(n) behaviorally distinct models".

Questions for authors
  • Theorem 1 assumes Θ(n)\Theta(n) permutations with pairwise different ϕ\phi, and concludes the same count. What does the theorem prove beyond its own assumption?
  • Theorems 1 to 3 are about one model with reordered data, but the abstract says “behaviorally distinct models”. What exactly is proved for different models?
4 comments
AuthorsOpenReview ↗

Response to Reviewer sVHq: Abstract Corrected, Contribution Clarified

We thank the reviewer for the careful reading. We agree with part of the critique and have clarified the paper accordingly.

1. What does Theorem 1 prove beyond its assumption?

We agree the proof is short. Theorem 1 isolates the propagation mechanism: once kn=Θ(n)k_n=\Theta(n) configurations with distinct behavioral values are exhibited, permutation invariance places them in one fiber of the representation E\mathcal{E}, so every scoring rule G∘EG\circ\mathcal{E} assigns them the same score. Theorem 2 and Corollary 1 extend this to the maximum score r∗r^*: tuning GG cannot recover what E\mathcal{E} discards, even at optimal evaluation.

The contribution is not permutation invariance in isolation, but identifying the structural restrictions and showing the hypothesis holds in concrete settings. It is verified, not assumed (Sec. 3.7, App. B):

  • Detection latency: Θ(T)\Theta(T) detectors at AUROC=1.000\mathrm{AUROC}=1.000, with latency spanning 00 to (T−t0)/2(T-t_0)/2.
  • Subgroup error allocation: Θ(n)\Theta(n) allocations at fixed accuracy, with disparity spanning 00 to k/mk/m.
  • Confidence trajectories: (Tm)=Θ(2T/T)\binom{T}{m}=\Theta(2^T/\sqrt{T}) trajectories at identical perplexity.
  • Audit: an O(n)O(n), score-only Collapse Audit (Alg. 1) with characterized failure modes, including up to 50×50\times underestimation under narrow perturbation distributions (Sec. 4.2).
  • COMPAS: a reproduced 2.3-pt accuracy gap alongside 20.3-pt FPR and 21.2-pt FNR gaps (Table 2).

2. Theorem 3 and Proposition 4

We agree Theorem 3 is an orbit–stabilizer count and will say so explicitly. Its role is to give the exact collapse size n!/∏jnj!n!/\prod_j n_j! for ϕord\phi_{\mathrm{ord}}, yielding exactly (Tm)\binom{T}{m} trajectories in the perplexity construction. Proposition 4 (App. A.8) goes further: any evaluation with LL representable score levels generically has tie orbits of size Ω(2nH(1/L)/n)\Omega(2^{nH(1/L)}/\sqrt{n}), so large collapse families are not rare edge cases. We will move Proposition 4 into Sec. 3.6.

3. One model with permuted data vs. "distinct models"

The reviewer is right: Theorems 1–3 concern a fixed ff under permutations of DD, and the abstract's "Θ(n)\Theta(n) behaviorally distinct models" was too broad. The model-level claim comes from Lemma 1 (which quantifies over distinct (f,D),(f′,D′)(f,D),(f',D')) together with the constructions, where different alert positions tt and error allocations jj define different detectors/classifiers on a fixed DD. We add:

Corollary (Model-Level Collapse on Fixed Data). In each construction of Sec. 3.7, there exist models f1,…,fkf_1,\ldots,f_k on a fixed DD, with k=Θ(n)k=\Theta(n) (or Θ(T)\Theta(T), respectively), such that E(fi,D)=E(fj,D)\mathcal{E}(f_i,D)=\mathcal{E}(f_j,D) for all i,ji,j, while ϕ(fi,D)\phi(f_i,D) are pairwise distinct.

The abstract now says "behaviorally distinct configurations, including distinct models in our constructions." We also clarify that AUROC is rank-based rather than strictly pointwise; in the latency construction its invariance is verified directly, since every normal score lies below every anomalous score.

4. "One result is wrong"

The review does not name the result, so we address the error we found: the mismatch between the configuration-level statements of Theorems 1–2 and the abstract's model-level wording. We thank the reviewer for identifying this overstatement and have corrected it through the revised abstract and the explicit model-level corollary above. If a different mathematical error was intended, we would appreciate a specific pointer so we can address it directly.

5. Contribution

We do not claim scalar metrics are invalid or that these counting identities are new. We characterize when an evaluation representation cannot distinguish behaviors, quantify the resulting collapse, including at optimal scores and under finite precision, and provide a practical audit alongside a real-world case study. These constructions, the audit, and the COMPAS reproduction establish the scope of the contribution beyond the permutation argument itself. In light of these clarifications and revisions, we respectfully ask the reviewer to reconsider the rating.

A correction to point 2 above. While answering the other reviewers we found that Proposition 4 as stated does not hold for constant or highly imbalanced score sequences. The correct bound is (nc)\binom{n}{c} with c=min⁡(nmax⁡,n−nmax⁡)c=\min(n_{\max},n-n_{\max}), and exponential growth requires an occupancy condition such as c≥αnc\ge\alpha n. Our statement that large collapse families are "not rare edge cases" should be read with that qualification, and we will correct App. A.8 and the related remarks.

I appreciate authors response. It answers a factual part of my concerns, but the main concerns remain unclear. Therefore, I keep my score unchanged.

AuthorsOpenReview ↗

Remaining concerns

Thank you for reading our response and for the review. Your questions led us to restate Theorems 1-3 with explicit conditions and to correct the abstract's claim about models, and we are grateful for that. If you can tell us which concerns you consider still unresolved, we will address them directly. Otherwise we respect your assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.