Disagreement Is Not Always Information: Task-Relevant Counterfactual Consensus Repair Under Multi-View Corruption
Abstract
Disagreement between multiple observations is often treated as evidence of corruption or complementary information, but disagreement need not be useful for task-relevant repair. We ask when a sample's own disagreement pattern adds value beyond the pair consensus and scalar disagreement severity. Given clean representation and corrupted representations , we define and repair target . Under squared loss, disagreement has incremental value only when conditioning on it changes the task-visible conditional mean of . We introduce Counterfactual Consensus Repair (CCR), which compares the target's own disagreement detail with a severity-matched shuffled control while holding fixed the target consensus, scalar severity, global parent, pair architecture, and optimization. Our primary contrast is , so negative values indicate value from the target's own disagreement detail. Across Speech Commands V2, ESC-50, and UrbanSound8K, this contrast is consistently favorable in asymmetric, heteroscedastic, and structured cross-family regimes; in the cross-family regime, and , respectively. Exact representation-space probes recover the predicted IID Gaussian null and heteroscedastic boundary, and the matched effect replicates on ESC-50 with a frozen pretrained BEATs representation. Zero-shot DHAuDS-Pair experiments, with no repair retraining, show that the learned disagreement-to-repair mapping transfers selectively: external corruption families can be beneficial, near null, or adverse. These results support a conditional rather than universal view of disagreement.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.