- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates diversification of ensembles and how it relates the effective dimension of the ensemble and the logit geometry. The diversity is achieved through coupled training of the ensemble, which regularizes the Gram matrix of the centered and normalized logits of the ensemble. The regularized training is evaluated empirically, comparing it to standard ensembles. The paper shows that orthogonality in logits does not necessarily imply prediction diversity.
Critical strengths
Proposition 2 shows that the ensemble members can change their raw logits to ensure orthogonality, while not changing the class probabilities. This gives a clear negative result regarding for penalties that rely on orthogonality of logits and is backed up by the experiments.
Critical weaknesses
The paper does not make its contribution and findings very clear. Proposition 1 and 2 are presented without any remarks about what they show, why they are interesting or how they are linked to the regularization and experiments. Section 3.1 and 3.2 present the regularization objective and ensembling, without stating which parts are novel contributions and why this normalization and regularization was chosen.
In the experiments the single seeds make the differences between the coupled and uncoupled ensemble very hard to interpret. The only significant difference between coupled and uncoupled ensembles shows that coupled training might not converge (Table 2, VGG16). It is not immediately clear what the experiments are designed to show and whether the very limited differences between coupled and uncoupled ensembles are due to the specific training setup or the shown limitations of orthogonal ensembles.
Formatting: The appendix is only in the supplementary material, making it harder to read the paper, as the references to the appendix point nowhere.
Questions for authors
- Could you clarify the contribution and goal of this paper and explain the impact of proposition 1 and 2?
- Does Figure 4 show the proposed regularization or just normal ensembles?
- Could you clarify what Section 5.7 shows?
- How would the results change if the penalty was applied to the post-softmax probabilities?