When Is One Importance Ranking Enough? Auditing Ranking Reuse Across Decisions
Abstract
Reusing one importance ranking across lesion-based functional diagnosis, concept correction, vulnerability screening, and specialization audits compresses distinct decision utilities into a common ordering. We evaluate this decision compression by separating compatibility, consequence, auditability, and authorization. Conflict structure does not determine sharing cost, and reliable measurement alone does not authorize reuse. Controlled redundancy separates single-unit effects from group responsibility; a construction shows that identical complete strict preferences can yield different minimum sharing costs even with fixed per-decision utility ranges. On CUB, same-feature common-linear and decision-specific linear rankers incur held-out mean normalized regrets of 0.4346 and 0.0785. A penalty persists under development-only stronger search and exhaustive optimization in an eight-concept test-utility control. A five-dataset non-CBM replication supports positive raw cost but reveals a denominator-sensitive normalized reversal. A prespecified blind AwA2 audit and separately frozen exploratory CelebA replication attain family-pair ordering accuracies of 0.956 and 0.929. Using 40% of audit observations, median accuracy retains over 90% of full-audit accuracy, but only 60.0–67.7% of subsampling runs meet the declared 0.85 criterion. In a 1,000-family LAD study, frozen direct-audit and cheap-proxy policies each select 100 of 500 held-out families at a fixed 20% reuse quota. At tolerance , both record 0/100 observed violations and overall normalized regret near 0.039, versus 0.3625 under always sharing. This improvement combines reduced reuse and selection; direct-audit superiority over the cheap proxy is not established. Separate prospective studies yield both certification and abstention under risk and coverage requirements. Ranking reuse therefore requires consequence evaluation and policy-specific validation, not confidence in an importance score alone.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.