acceptodds
Under review as a conference paper at ICLR 2027

Disagreement Is Not Always Information: Task-Relevant Counterfactual Consensus Repair Under Multi-View Corruption

Abstract

Disagreement between multiple observations is often treated as evidence of corruption or complementary information, but disagreement need not be useful for task-relevant repair. We ask when a sample's own disagreement pattern adds value beyond the pair consensus and scalar disagreement severity. Given clean representation and corrupted representations , we define and repair target . Under squared loss, disagreement has incremental value only when conditioning on it changes the task-visible conditional mean of . We introduce Counterfactual Consensus Repair (CCR), which compares the target's own disagreement detail with a severity-matched shuffled control while holding fixed the target consensus, scalar severity, global parent, pair architecture, and optimization. Our primary contrast is , so negative values indicate value from the target's own disagreement detail. Across Speech Commands V2, ESC-50, and UrbanSound8K, this contrast is consistently favorable in asymmetric, heteroscedastic, and structured cross-family regimes; in the cross-family regime, and , respectively. Exact representation-space probes recover the predicted IID Gaussian null and heteroscedastic boundary, and the matched effect replicates on ESC-50 with a frozen pretrained BEATs representation. Zero-shot DHAuDS-Pair experiments, with no repair retraining, show that the learned disagreement-to-repair mapping transfers selectively: external corruption families can be beneficial, near null, or adverse. These results support a conditional rather than universal view of disagreement.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multi-view learning, robust representation learning, disagreement, counterfactual learning, corruption robustness, test-time robustness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_spKfOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether the detailed disagreement between two corrupted observations provides task-relevant information beyond their mean representation and a scalar measure of disagreement severity. Counterfactual Consensus Repair first learns a correction from the mean representation, then learns a residual correction using a disagreement descriptor. Its main test compares a sample’s own descriptor with a descriptor borrowed from a severity-matched sample. Across three audio datasets, the authors report favorable centered-logit reconstruction results for several controlled corruption regimes, examine an exact representation-space Gaussian null, and test transfer to externally designed corruptions. The external results are mixed or adverse.

Critical strengths
  1. The paper poses a specific, falsifiable question. Its severity-matched control retains the target’s mean representation and exact scalar severity while matching the global correction, pair-model architecture, and optimization. This is more informative than merely comparing a model with disagreement input against one without it.

  2. The authors investigate limits of the effect. The representation-space IID Gaussian probe recovers the predicted no-value case; several controlled regimes show favorable effects across three audio datasets; and the paper reports adverse external-transfer cases rather than claiming universal benefit.

Critical weaknesses
  1. The central control does not clearly establish the value of within-family, sample-specific disagreement. Appendix C.5 describes donor matching by scalar severity within minibatches but does not specify matching within the same corruption regime or structured-corruption family. Since training mixes regimes, the correct descriptor might convey the target’s corruption-family identity while a donor descriptor conveys a different family. Table 4 shows substantial variation across structured families, making this alternative explanation consequential. The reported contrast establishes information beyond scalar severity, but its stronger sample-specific interpretation remains unresolved.

  2. The amount of graphical and tabular evidence exceeds the number of distinct conclusions it supports. Figure 2 and Tables 2–3 repeatedly present the same centered-logit MSE contrast, while the corresponding classification gains are generally small. Figure 3’s upper panels illustrate a representation-space theoretical construction; they do not establish the mechanism of practical CCR, whose descriptor also uses nonlinear intermediate feature-map information. The common-mode zero in Figure 2 follows from the gate by construction. Finally, Figure 4 and Tables 1, 8, and 9 largely document that external transfer is unstable or adverse. Taken together, the results support a controlled diagnostic more strongly than a reliable, transferable repair method.

Questions for authors
  1. Were severity-matched donors restricted to the same corruption regime and, for structured corruption, the same family during both training and evaluation? Do the main effects persist under within-family severity matching, and how does a family-identity-only control perform?

  2. What are the absolute global-only, correct-descriptor, and matched-control risks for each principal regime? Are the reported accuracy and macro-F1 differences statistically reliable at the seed or fold level, particularly where centered-logit MSE improves but classification changes little?

  3. How much of the practical CCR effect comes from representation disagreement versus intermediate feature-map energy terms? An ablation using the same repair architecture and matched control would clarify how directly the representation-space theory explains the empirical gains.

Reviewer_Y4UGOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates when disagreement between two corrupted views provides task-relevant information beyond their consensus and scalar disagreement severity. The authors propose Counterfactual Consensus Repair (CCR), which combines a consensus-only correction with a gated, disagreement-conditioned residual. The central evaluation compares sample-specific disagreement descriptors against severity-matched shuffled descriptors while controlling for model architecture, optimization, and the global repair model.

The paper derives a conditional-risk identity and an IID Gaussian no-value result, and evaluates CCR on three audio classification datasets. Controlled experiments show consistent benefits under asymmetric, heteroscedastic, and structured corruptions. Representation-space probes validate the theoretical boundary cases, while frozen-BEATs experiments and zero-shot transfer tests reveal both beneficial and adverse effects.

Critical strengths
  1. The severity-matched disagreement control is a useful experimental design that goes beyond simply comparing models with and without disagreement features. The shared global repair model, matched model capacity, severity-only ablations, and consistent results across three datasets provide credible evidence that correctly associated disagreement patterns can improve task-relevant reconstruction under the studied corruptions.

  2. The exact Gaussian null, heteroscedastic probes, pretrained-BEATs replication, and adverse zero-shot transfer results provide complementary evidence about the conditions under which disagreement helps. Importantly, the authors acknowledge negative results rather than claiming universal improvements.

Critical weaknesses
  1. The main theoretical result is a standard conditional-expectation risk decomposition, and the Gaussian null follows from classical independence properties. The consensus-plus-residual architecture also builds on established residual correction and multi-view fusion ideas. More importantly, the central question of whether additional features provide predictive value beyond existing information closely relates to conditional variable importance and Conditional Predictive Impact (e.g., Strobl et al., 2008; Molnar et al., 2024; Watson & Wright, 2021). The severity-matched shuffle also resembles conditional permutation methods. These connections are insufficiently discussed, making it unclear what fundamentally new conceptual or methodological insight CCR contributes beyond adapting established conditional predictive-value analysis to task-relevant multi-view repair.

  2. The severity-matched shuffle preserves the target consensus and scalar severity, but donor descriptors are matched only by severity, without preserving their conditional dependence on the consensus. The intervention may therefore introduce distributional changes beyond removing sample-specific disagreement information. Moreover, the reported contrast compares separately trained finite-capacity models, whereas the theoretical identity concerns Bayes-optimal conditional predictors. The experiments convincingly establish an empirical advantage of correctly associated descriptors under the proposed control, but do not fully isolate their intrinsic conditional information value.

  3. Despite the general multi-view formulation, all experiments concern audio classification, and the practical disagreement descriptor relies on representation-specific channel statistics. The BEATs replication provides useful evidence across audio representations but does not establish applicability to other modalities or substantially different multi-view settings. This limits the demonstrated generality of the methodology relative to the breadth of its conceptual framing.

  4. CCR requires two semantically aligned but differently corrupted observations of the same instance at inference time, together with clean reference observations during repair training. The paper provides limited motivation for realistic applications where these conditions naturally hold. Moreover, the main positive results rely on controlled corruptions, while zero-shot transfer to external corruption families is mixed or adverse.

Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., & Zeileis, A. (2008). Conditional variable importance for random forests. BMC Bioinformatics, 9, Article 307. https://doi.org/10.1186/1471-2105-9-307

Molnar, C., König, G., Bischl, B., & Casalicchio, G. (2024). Model-agnostic feature importance and effects with dependent features: A conditional subgroup approach. Data Mining and Knowledge Discovery, 38(5), 2903–2941. https://doi.org/10.1007/s10618-022-00901-9

Watson, D. S., & Wright, M. N. (2021). Testing conditional independence in supervised learning algorithms. Machine Learning, 110(8), 2107–2129. https://doi.org/10.1007/s10994-021-06030-6

Questions for authors
  1. How does CCR differ substantively from conditional permutation importance and Conditional Predictive Impact? What specific methodological capability, theoretical insight, or research question does CCR address that cannot already be captured by these established frameworks?

  2. How do the authors distinguish the benefit of sample-specific disagreement from potential distributional mismatches introduced by matching donor descriptors only on severity? Is the primary Δdetail\Delta_{\mathrm{detail}} contrast intended to measure model-dependent predictive utility or intrinsic conditional information value? Could the authors clarify which interpretation is justified by the existing controls and results?

  3. Is CCR intended as a general multi-view representation repair framework or primarily as a method for studying disagreement in audio robustness? Given the representation-specific descriptor and audio-only validation, which aspects of the proposed framework are expected to generalize beyond the evaluated settings, and what existing evidence supports that expectation?

  4. What concrete real-world applications motivate the availability of two semantically aligned but differently corrupted observations at inference time and clean references during repair training? How representative are the constructed corruption pairs of those settings, particularly given the mixed and adverse results under external corruption distributions?

Reviewer_mMMJOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

N/A

Critical strengths

N/A

Critical weaknesses

N/A

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.