- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper analyzes the losses used during model training and inference (decoding in the paper), introduces bitwise and blockwise training losses, and proposes a blockwise probability training loss that can be applied to arbitrary binary codebooks while incorporating both general cross-entropy (CE) loss and binary cross-entropy (BCE) loss within its framework.
Critical strengths
The paper proposes a framework named blockwise probability loss that incorporates the commonly used cross-entropy (CE) loss and binary cross-entropy (BCE) loss, and it also accommodates the MSE loss and cosine similarity loss.
Critical weaknesses
- The issue discussed in this paper does not appear to be one that urgently needs to be addressed, has wide-ranging impact, or has attracted broad attention.
- The proposed BP loss is obviously more complex than the commonly used CE or BCE loss.
- The experimental results in Tables 1 and 2 do not provide strong evidence of the superiority of BP loss compared to CE or BCE loss. Furthermore, training ResNet34 with CE loss on CIFAR-100 typically achieves test accuracy exceeding 70%, whereas the accuracy reported in the paper is merely 57.04%, which is difficult to find convincing.
Questions for authors
What are logit-normalized versions of different loss functions?