- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates when evaluation metrics discard information necessary for assessing model behaviour. It formalises non-identifiability using evaluation representations, derives permutation-based collapse counts, and demonstrates the issue via detection latency, subgroup disparity, and language-model confidence trajectories. It also introduces a perturbation audit and presents synthetic experiments and a COMPAS case study. Its central message is that a metric cannot certify properties that its representation does not retain.
Critical strengths
-
The work has studied a practical and useful problem existing in model evaluation.
-
The audit is inexpensive to apply to stored scores, and the paper explicitly acknowledges that small observed spread does not establish the absence of collapse. The perturbation-locality ablation usefully illustrates this limitation.
Critical weaknesses
-
The exposition and structure of this work need further refinement; specifically, definitions of collapse and behavioral information should be introduced early in the paper.
-
While the research question is important and practical, both the scope of the study and the analysis remain limited, and the resulting conclusions offer little significant new insight or guidance for model evaluation.
-
Theorem 1 assumes linearly many permutations with distinct behavioural values; it does not derive that diversity from the structural restrictions. A property can be non-identifiable yet take only two values, such as whether the first score exceeds a threshold. This contradicts Remark 1's suggestion that a single collision necessarily propagates into linearly many distinct behaviours. Theorem 2 likewise assumes that the orbit already contains a maximiser. These conditional results are valid, but do not establish the broad guarantees suggested in the abstract and Section 4. Furthermore, reordering complete examples with their group identities leaves subgroup accuracy unchanged, so the subgroup error-reallocation construction needs a transformation argument beyond Theorem 1's dataset permutations.
-
Table 3 audits constructions with analytically fixed metric values, using resampled alert positions, error allocations, or shuffled log-probabilities. This demonstrates variability within the supplied perturbation families, but does not show that alternatives are attainable by models on the same task and data. Shuffling stored token log-probabilities does not by itself construct another autoregressive model or valid sequence with those conditional probabilities. COMPAS is a useful illustration, but compares groups within one classifier and is explicitly not an exact collapse-family instance. The empirical claims should distinguish representation-level ambiguity from demonstrated differences among feasible model behaviours.
Questions for authors
-
How would authors revise Proposition 4 to handle constant and highly imbalanced score sequences? Which added occupancy assumptions support exponential growth, and which downstream claims would change?
-
Is Theorem 3 intended to count only the permutation orbit of a fixed multiset? If it concerns the full metric-level family, how does it exclude equal-scoring sequences from other orbits?
-
Can authors distinguish the behavioural-diversity assumptions of Theorems 1 and 2 from their conclusions, and specify the transformation under which the subgroup construction is an instance of the framework?
-
Can the remedies state the retained information explicitly and show that each claimed single-restriction relaxation leaves the other restrictions intact?
-
What constraints make an audit perturbation a valid alternative for the intended task or model class? Can the existing experiments distinguish score rearrangements from feasible differences in model behaviour?
3 comments
Thanks for authors' response.
The rebuttal answers the factual parts of my questions. My main concern remains and the scope of the study and the analysis remain limited. I maintain my original rating.
Scope of the study
Thank you for the follow-up, and for confirming that our responses address the factual parts of your questions. We accept your remaining point on scope: the framework covers pointwise, permutation-invariant, scalar evaluation, and the analysis is limited to the constructions and conditional results we state. We will state this scope explicitly in the introduction and the limitations section, so that it is not left to the reader to infer, and we will keep our claims within it. Your review caught an error in Proposition 4 that we would otherwise have missed, and we are grateful for its precision.
Response to Reviewer VKZ2: Theorem Scope, Proposition 4 and Empirical Claims Corrected
We thank the reviewer for a precise review. Most technical points are correct; below we state what we concede and what we will change in the manuscript.
Theorems 1-2 and Remark 1 (W3, Q3). Correct. Theorem 1 is conditional: it assumes Θ(n) distinct values of ϕ over the orbit and does not derive them. A two-valued property (e.g. "does the first score exceed τ") is non-identifiable yet has only two values, so Remark 1 is wrong as written: a collision propagates to the whole orbit, but the number of distinct ϕ-values depends on ϕ. We will (i) restate Theorem 1 for any E-preserving transformation family T (dataset reorderings, group relabelings) as "the collapse family contains every value ϕ takes over the T-orbit", with Θ(n) an explicit hypothesis, verified in the constructions and derived for ϕord (n!/∏jnj!≥n for non-constant sequences); (ii) state Theorem 2 as conditional on the orbit containing a maximiser (true in the AUROC construction); (iii) delete Remark 1's tightness claim; (iv) change the abstract's "for any such property, Θ(n)..." to "for properties that vary across the orbit".
Subgroup construction (W3, Q3). Correct: permuting whole examples together with their group labels leaves subgroup accuracy unchanged. The right T is relabelling groups against fixed scores: the evaluation representation depends only on the fixed scores and is invariant to group relabeling, whereas subgroup accuracy depends on which group labels are assigned to those scores (Θ(m) distinct disparities; App. B.2 is the calculation). We will describe the construction this way, as an instance of Lemma 1 and the generalized Theorem 1.
Proposition 4 (Q1). The reviewer is right: the monotonicity step fails when nmax>n/2. The correct bound is ∣Orb∣≥(cn) with c=min(nmax,n−nmax). This bound equals 1 for constant sequences and is Θ(nc) for fixed c, so exponential growth needs an occupancy assumption: if c≥αn for a fixed α∈(0,1/2], then ∣Orb∣=Ω(2nH(α)/n). We will restate it so and soften "generically exponential" in Remark 2 and Limitations.
Theorem 3 (Q2). Yes, it counts only the permutation orbit of a fixed score sequence; it cannot exclude equal-scoring sequences from other orbits, so it is a lower bound on ∣Cr∣. We will restate it.
Remedies (Q4). Fair. We will restate each proposition with its retained information and the restrictions it leaves intact. (1) Order only: the scalar first-crossing time min{t:st>τ}∈R uses pointwise scores and gives scalar output, and with the known onset t0 it determines ϕlatency=min{i≥t0:si>τ}−t0 (in our construction normal scores lie below τ, so the two minima coincide). (2) Group access only: a symmetric scalar over pairs (si,gi), e.g. worst-group accuracy, keeps scalar output and permutation invariance but determines only that statistic; the full vector ϕsub also needs vector output. (3) Vector output only: the sorted scores keep the score distribution but not the ordered trajectory, which needs permutation invariance and scalar aggregation relaxed together. We will remove "individually relaxed" and "formally independent".
Empirical scope (W4, Q5). Agreed. Table 3 shows representation-level ambiguity within the supplied perturbation families, not differences among natural models; shuffled log-probabilities are not a trained autoregressive model, and COMPAS is a within-classifier comparison, not an exact collapse instance. On a fixed evaluation set a model can be defined to emit any such score vector, but we do not show trained models in that class. We will (i) define a perturbation as valid only if it lies in the feasible set F of score vectors realizable by the intended model class, and make F an input to Algorithm 1; (ii) label these experiments representation-level; (iii) list comparison of independently trained models as an open limitation.
Presentation (W1). We will move the definitions of evaluation representation, behavioral dimension and collapse family into the introduction.
These changes narrow the claims to what the proofs and experiments support: a precise account of when a metric cannot see a behavior, and an audit for it. We respectfully ask the reviewer to reconsider the ratings in light of them.