- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work studies 'confident source conflict': some modalities are replaced by clean observations of another example, so neither confidence nor input quality shows the corruption. To solve the problem, the authors proposed CEF, which keeps average-logit fusion as default, builds candidates (bounded residual or 31 source-subsets), and routes each example by a cross-fitted selector trained on 'candidate correct but default wrong', with a preservation-constrained threshold. On PolyMNIST and AV-MNIST, median wins when a majority remains, routing helps when no majority exists, and the selector transfers poorly across corruption counts.
Critical strengths
-
The authors are honest about CEF's limitation: it reports its own method losing to a zero-parameter median (97.3% vs 93.6%, Table 1B) and makes this the main finding.
-
The evaluation is well-designed: cross-fitting (Alg. C.1), a reserved AV-MNIST test used once, and a donor-assignment sensitivity check.
-
The rescue/harm decomposition (Fig. 2c, Table 3) clearly shows routing, not the correction, gives +6.1 points.
Critical weaknesses
-
There is no theory on when routing provably keeps clean accuracy, the authors provided only empirical evidence. I look forward to the author providing deeper theoretical analysis and guidance.
-
The AV-MNIST headline (+7.7pp, CI [5.1,10.2]) is on a 400-example test split. The CI is a t-interval over 5 seeds (Sec. 5.4) and ignores binomial noise, therefore I suspect the precision is overstated.
Questions for authors
-
Does the AV-MNIST CI consider the binomial variance on 400 examples, for example a paired per-example test?
-
Does CEF beat median-as-default routing at k=2 and k=3?
-
Please report the Eq. 7 baseline and the trainable fusion results in detail. Besides, in Eq. 7, how is the conflict c_m defined, and what value is kappa?
I have carefully read the authors' responses to the questions raised by me, and I appreciate the detailed information provided, especially the theoretical analysis part. I will keep my rating unchanged.