- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Summary
This paper asks the question of whether the internal routing of an Attention-Residual (AR) transformer carries information about when its predictions are wrong, beyond what the model's confidence already shows. AR networks replace the fixed residual sum with input-dependent attention over earlier layers' outputs. The authors summarise each layer's routing weights as an entropy value and ask whether, among predictions of equal confidence, routing entropy helps separate correct from incorrect ones.
Critical strengths
Strengths
- I think the entire question the paper is looking to answer is an interesting one.
- Reproducibility is available, and the appendix includes more details to try and replicate the large code base.
Critical weaknesses
Weaknesses
- The abstract and introduction dive straight into technical jargon/details without any thorough motivation or introduction of terms used throughout the paper. I feel like this is typically necessary, and thus, without them, the paper will read strange/be difficult to read to someone not exactly familiar to the field.
- Following on from the above point, the paper's presentation is poor. There is undefined notation throughout, dense text with no explanations provided (as stated above), any explanations are hidden in dense appendix sections (which is fine in parts but if a reader has to read 30 pages of appendices to grasp any form of results/motivation, then it is poorly constructed). IMO, the paper needs a large overhaul on the presentation to be conference-ready.
- Whilst it is a stylistic choice, I do not like introducing some motivation via notation is the introduction. I feel like this is a reason for the authors to not include textual motivation (which I mentioned above) which hurts presentation, and does not fill me with confidence that they can explain the motivation of their own paper.
- The null result that the paper concludes is honest, but not very convincing for the venue. Whilst a null result is fine, analysis is weak and I feel like I did not gain additional knowledge from the paper.
- Only running experiments on small models and the CIFAR datasets is small-scale and I feel like is not thorough enough for the venue.
- If uncertainty information is a key theme of the paper, why have the authors not ran any tests in distributionally shifted data, such as out-of-distribution data/adversarially attacked data which is meant to induce uncertainty?
Questions for authors
Can you add a short, plain-language motivation to the abstract and introduction, defining AR, routing entropy, nats, and oracle gain before they are used?
Why did you not consider OOD/Adversarially attacked data in your experiments alongside the current evaluation?